## robots.txt for myDataCation Pro ## GENERAL SETTINGS ## Enable robots.txt rules for all crawlers User-agent: * ## Crawl-delay parameter: number of seconds to wait between successive requests to the same server. ## Set a custom crawl rate if you're experiencing traffic problems with your server. # Crawl-delay: 30 ## Sitemap: uncomment and replace the URL to your sitemap file # Sitemap: http://www.example.com/sitemap/sitemap.xml ## DEVELOPMENT RELATED SETTINGS ## Do not crawl development files and folders: CVS, svn directories and dump files Disallow: /*.sql$ Disallow: /*.tgz$ ## GENERAL SETTINGS ## Do not crawl admin pages Disallow: /_ # _editor, _flvplayer, _uploader, etc. Disallow: /account/ Disallow: /admin/ Disallow: /data/ #Disallow: /includes/ #muss erlaubt sein, damit google auf js und css Dateien zugreifen kann ## Do not crawl common files Disallow: /edithandler.php Disallow: /fileServer.php Disallow: /fileUploadPopUp.php Disallow: /*.log Disallow: /statistics.php Disallow: /global.inc.php Disallow: /requestEnd.inc.php ## SEO IMPROVEMENTS ## Do not crawl special pages Disallow: /*?*handicappedCss* Disallow: /*?page=* Disallow: /*&page=* Disallow: /*?image=* Disallow: /*&image=* Disallow: /*?lang=* Disallow: /*&lang=* Disallow: /internet/imprint.php Disallow: /internet/dataProtection.php