benw2/wezm.net
1
0
Fork 0
forked from wezm/wezm.net
Code Pull requests Activity

Tidy up robots.txt

Browse source
This commit is contained in:
Wesley Moore 2010-06-16 20:21:55 +10:00
parent 8b9b96fcbb
commit 948193e6e8
1 changed files with 0 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
content/robots.txt
View file

@ -1,6 +1,3 @@
# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
#
# To ban all spiders from the entire site uncomment the next two lines:
User-Agent: * User-Agent: *
Disallow: / Disallow: /
Sitemap: <%= @site.config[:base_url] %>/sitemap.xml Sitemap: <%= @site.config[:base_url] %>/sitemap.xml