Loading...
 
Features / Usability

Features / Usability


Fixed! Re: Googlebot ignoring robots.txt

posts: 9 United States

OK, I changed my robots.txt so that there are leading slashes in front of each page (previously, there were only leading slashes in front of vdir's).

See the attached "robotsneedroot.png" for a graph of my bandwith usage.

  • Late Oct - bots appear to have changed their behavior!
  • Nov - I had to pay for extra bandwidth this month eek
  • Early Dec - I made the changes to robots.txt. Shortly after,
  • PROBLEM SOLVED


More details at my bug report

There are no comments at this time.

Upcoming Events

1)  18 Apr 2024 14:00 GMT-0000
Tiki Roundtable Meeting
2)  16 May 2024 14:00 GMT-0000
Tiki Roundtable Meeting
3)  20 Jun 2024 14:00 GMT-0000
Tiki Roundtable Meeting
4)  18 Jul 2024 14:00 GMT-0000
Tiki Roundtable Meeting
5)  15 Aug 2024 14:00 GMT-0000
Tiki Roundtable Meeting
6)  19 Sep 2024 14:00 GMT-0000
Tiki Roundtable Meeting
7) 
Tiki birthday
8)  17 Oct 2024 14:00 GMT-0000
Tiki Roundtable Meeting
9)  21 Nov 2024 14:00 GMT-0000
Tiki Roundtable Meeting
10)  19 Dec 2024 14:00 GMT-0000
Tiki Roundtable Meeting