# /robots.txt file for http://webcrawler.com/# mail webmaster@webcrawler.com for constructive criticismUser-agent: webcrawlerDisallow:User-agent: lycraDisallow: User-agent: *Disallow: