Hi,
I have created a new site as an experiment using the asp padfile kit. I may
have set it up wrong!
http://www.shareware4all.com
I am having bandwidth problems any one help me understand the AWstats.
So far in July
Pages = 18981
Visits = 101
Unique Visitors = 71
71 visitors accessing 18981 pages seems very high (260 pages per visitor).
I am still with affordablehost at this stage!
66.249.66.15 = 9000 pages
84.248.66.182 = 2313 pages
203.146.247.19 = 2212 pages
I have disabled the submit page and changed all the pad-sysopp files.
And from Webalizer I have these...
9352 47.63% 9347 48.39% 47685 48.40% 1 1.10%
crawl-66-249-66-15.googlebot.com
2325 11.84% 2313 11.97% 11768 11.95% 1 1.10%
dsl-tregw3ne54f842b6.dial.inet.fi
2213 11.27% 2212 11.45% 11262 11.43% 2 2.20% 203.146.247.19
Thanks for any info.
Chris
Full Version: Bandwidth Issue
"Chris" <[Email Removed]> wrote in message
news:42ca5481$[Email Removed]...
news:42ca5481$[Email Removed]...
| QUOTE |
| Hi, I have created a new site as an experiment using the asp padfile kit. I may have set it up wrong! http://www.shareware4all.com I am having bandwidth problems any one help me understand the AWstats. So far in July Pages = 18981 Visits = 101 Unique Visitors = 71 71 visitors accessing 18981 pages seems very high (260 pages per visitor). I am still with affordablehost at this stage! 66.249.66.15 = 9000 pages 84.248.66.182 = 2313 pages 203.146.247.19 = 2212 pages I have disabled the submit page and changed all the pad-sysopp files. And from Webalizer I have these... 9352 47.63% 9347 48.39% 47685 48.40% 1 1.10% crawl-66-249-66-15.googlebot.com 2325 11.84% 2313 11.97% 11768 11.95% 1 1.10% dsl-tregw3ne54f842b6.dial.inet.fi 2213 11.27% 2212 11.45% 11262 11.43% 2 2.20% 203.146.247.19 Thanks for any info. Chris |
If I'm reading this right it is saying that google has crawled (or at least
visited) 9352 pages. The others may be similar search engine robots. Thus
you have a small number of visitors (bots) that are genuinely looking at a
large number of pages.
In some ways this is a good thing, presumably you want google to have
trawled most of the pages on your site. I can understand that it would eat
your bandwidth.
If you don't want these robots to trawl your site then you can use a
robots.txt file to tell them to stay away - but do you really want to?
Hope this helps,
Brian.
www.cryer.co.uk/brian
Using a pointed stick and pebbles, Brian Cryer scraped:
| QUOTE |
| In some ways this is a good thing, presumably you want google to have trawled most of the pages on your site. I can understand that it would eat your bandwidth. |
This is something that really annoys me about Googlebot. I find that it
seems to visit *every* page on each of my sites each and every day,
which with ~2000+ pages tends to eat up my bandwidth a hell of a lot.
I already include "last updated" headers on my pages, but that doesn't
seem to have any effect on the frequency that Google crawls my pages. Is
there any way to tell it to visit less often?
--
Dylan Parry
http://electricfreedom.org -- Where the Music Progressively Rocks
Once upon a time *Dylan Parry* wrote:
| QUOTE |
| Using a pointed stick and pebbles, Brian Cryer scraped: In some ways this is a good thing, presumably you want google to have trawled most of the pages on your site. I can understand that it would eat your bandwidth. This is something that really annoys me about Googlebot. I find that it seems to visit *every* page on each of my sites each and every day, which with ~2000+ pages tends to eat up my bandwidth a hell of a lot. I already include "last updated" headers on my pages, but that doesn't seem to have any effect on the frequency that Google crawls my pages. Is there any way to tell it to visit less often? |
I often see meta tags like this on pages:
<meta name="revisit-after" content="20 Days">
Is that doing any good, do the SE bots taking any notice?
--
/Arne
Top posters will be ignored. Quote the part you
are replying to, no more and no less! And don't
quote signatures, thank you.
Using a pointed stick and pebbles, Arne scraped:
| QUOTE |
| <meta name="revisit-after" content="20 Days" Is that doing any good, do the SE bots taking any notice? |
From what I've read, that was "invented" purely for use by "Vancouver
Webpages searchBC" bot, and is ignored by every other search engine bot,
including Googlebot. Apparently, searchBC no longer uses it either!
--
Dylan Parry
http://electricfreedom.org -- Where the Music Progressively Rocks
----- Original Message -----
From: "Chris" <>
Newsgroups: alt.www.webmaster
Sent: Tuesday, July 05, 2005 5:36 AM
Subject: Bandwidth Issue
From: "Chris" <>
Newsgroups: alt.www.webmaster
Sent: Tuesday, July 05, 2005 5:36 AM
Subject: Bandwidth Issue
| QUOTE |
| Hi, I have created a new site as an experiment using the asp padfile kit. I may have set it up wrong! http://www.shareware4all.com I am having bandwidth problems any one help me understand the AWstats. So far in July Pages = 18981 Visits = 101 Unique Visitors = 71 71 visitors accessing 18981 pages seems very high (260 pages per visitor). I am still with affordablehost at this stage! 66.249.66.15 = 9000 pages 84.248.66.182 = 2313 pages 203.146.247.19 = 2212 pages I have disabled the submit page and changed all the pad-sysopp files. And from Webalizer I have these... 9352 47.63% 9347 48.39% 47685 48.40% 1 1.10% crawl-66-249-66-15.googlebot.com 2325 11.84% 2313 11.97% 11768 11.95% 1 1.10% dsl-tregw3ne54f842b6.dial.inet.fi 2213 11.27% 2212 11.45% 11262 11.43% 2 2.20% 203.146.247.19 Thanks for any info. Chris |
Your offering free shareware downloads and your concerned with bandwidth?
LOL!
84.248.66.182
This is a Broadband access from Finland.
Is Finnish traffic beneficial to your website?
203.146.247.19
Is traffic from Thailand beneficial to your website?
Re: Google:
The extent of the crawls will lessen over time as will the frequency.
Some this for you to consider is if you desire crawling of each page and
section of your website?
If not? Modify robots.txt and google will honor those exceptions.
It's also a good idea to put images in directory not intended for crawls by
bots (unless you desire those images shared.) Depending upon the quanity of
images your site contains may increase your bandwidth dramitically.
Especially with crawling from a bot or bots visiting frequently.
An additional note!
The bots are likely crawling your shareware download files as well and they
too should be placed in a directory denied in robots.txt which will reduce
your bandwidth dramatically.
The bots are likely crawling your shareware download files as well and they
too should be placed in a directory denied in robots.txt which will reduce
your bandwidth dramatically.
"lostinspace" <[Email Removed]> wrote in message
news:P1wye.635$[Email Removed]...
news:P1wye.635$[Email Removed]...
| QUOTE |
| An additional note! The bots are likely crawling your shareware download files as well and they too should be placed in a directory denied in robots.txt which will reduce your bandwidth dramatically. |
I don't know the organisation of Chris's site, but most shareware sites
don't store the shareware download files locally they just store the link to
where the author has a download. So the actual number of files that Chris
will want to block from robots might be quite small.
Brian.
Dylan Parry wrote:
| QUOTE |
| Using a pointed stick and pebbles, Brian Cryer scraped: In some ways this is a good thing, presumably you want google to have trawled most of the pages on your site. I can understand that it would eat your bandwidth. This is something that really annoys me about Googlebot. I find that it seems to visit *every* page on each of my sites each and every day, which with ~2000+ pages tends to eat up my bandwidth a hell of a lot. I already include "last updated" headers on my pages, but that doesn't seem to have any effect on the frequency that Google crawls my pages. Is there any way to tell it to visit less often? |
Have you tried Google SiteMaps?
https://www.google.com/webmasters/sitemaps
Lets you tell the bot how frequently the content at the URL is likely to
change
Using a pointed stick and pebbles, Viper scraped:
| QUOTE |
| Have you tried Google SiteMaps? |
Nope, but I'm reading in to it now. Cheers :)
--
Dylan Parry
http://electricfreedom.org -- Where the Music Progressively Rocks!
Dylan Parry <[Email Removed]>
| QUOTE |
| Using a pointed stick and pebbles, Brian Cryer scraped: In some ways this is a good thing, presumably you want google to have trawled most of the pages on your site. I can understand that it would eat your bandwidth. This is something that really annoys me about Googlebot. I find that it seems to visit *every* page on each of my sites each and every day, which with ~2000+ pages tends to eat up my bandwidth a hell of a lot. I already include "last updated" headers on my pages, but that doesn't seem to have any effect on the frequency that Google crawls my pages. Is there any way to tell it to visit less often? |
You can create customized versions of robots.txt that allow/exclude
bots to areas of your site according to the policies you choose. If
you're using Unix or Linux, you can create a cron job that will
install the appropriate robots.txt at the appropriate time. I'm not
familiar enough with the Windows program for scheduling jobs for
certain times, but I imagine you can do similar things with it that
you can do with cron.
--gregbo
gds at best dot com
"Chris" <[Email Removed]> wrote in
news:42ca5481$[Email Removed]:
news:42ca5481$[Email Removed]:
| QUOTE |
| I have created a new site as an experiment using the asp padfile kit. I may have set it up wrong! http://www.shareware4all.com I am having bandwidth problems any one help me understand the AWstats. |
Use robot.txt to lock out parts of your site which you do not want indexed
into search engines. Also be aware that search engines have a purpose and
goal in mind. Google for example will visit you more often if you are a
"content" site. Pages they expect to change often such as forums and blogs
can vastly increase your visits from them.
If bandwidth CHARGES are a problem then consider splitting the site. A site
with low bandwidth charges might be low in other features (like stability)
but if you pick and choose what things to put where it can be to your
advantage.
Gandalf Parker
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.