Archive for April, 2007
Wildcards in robots.txt
Posted on 20. Apr, 2007 by Daniel Aleksandersen.
A list ofsearch engine crawlers that support wildcards, and the Allow: extension in robots.txt configuration files.
[...]
Continue Reading
Blogs on doing better searches?
Posted on 16. Apr, 2007 by Daniel Aleksandersen.
There are thousands of blogs on how web developers can improve their search ranking. But there are (to my knowledge) non on how the search engine users can do better searches to get better results! Now is not that a bit odd?
[...]
Continue Reading
Sitemaps are everywhere!
Posted on 11. Apr, 2007 by Daniel Aleksandersen.
Even in sites’ robots.txt files sitemaps have start making there presence known! Ask! Search, Yahoo! Search, Windows Live, and Google have all said they will start looking for sitemaps in the robots.txt files. Sitemaps auto-discovery have been born!
[...]
