# /robots.txt file for http://webdex.com/ # mail webmaster@webdex.com for constructive criticism #User-agent: webcrawler nothing disallowed: it may go anywhere #Disallow: #User-agent: lycra all relative URLs starting with '/' disallowed #Disallow: / User-agent: * Disallow: /tmp Disallow: /logs Disallow: /12 Disallow: /bday Disallow: /boat Disallow: /c21 Disallow: /castles Disallow: /flash Disallow: /jakes Disallow: /mief Disallow: /sp Disallow: /stats Disallow: /laconia2003 Disallow: /includes Disallow: /accounts Disallow: /hurricane