# # This file allows you to direct web "spiders" or "robots" to # stay out of certain files or directories by specifying them # in Disallow: clauses. By default, this file contains no # restrictions on robots. # # Some examples where you would want to restrict robots # would be: stock price, or other dynamic data that # is worthless to a long-term Web Index. Also, sensitive # or personal data directories should be restricted. # User-agent: * Disallow: