# Web Crawler Exclusion File, based on "http://www.robotstxt.org/wc/robots.html". # This file tries to tell WWW robots (web spiders, not real robots!) that they shouldnt # access files in some places of this website, since it may cause problems. # by Shervin Emami 14/12/07. User-agent: * Disallow: /resources/ # Don't crawl through anything in this folder Disallow: /current/ # Don't crawl through anything in this folder Disallow: /scripts/ # Don't crawl through anything in this folder