User-agent: *
Disallow: files/
Disallow: log
Crawl-Delay: 30