wezm/wezm.net
1
0
Fork 1
mirror of https://github.com/wezm/wezm.net.git synced 2024-12-19 02:39:54 +00:00
Code Issues Projects Releases Packages Wiki Activity

Tidy up robots.txt

Browse source
This commit is contained in:
Wesley Moore 2010-06-16 20:21:55 +10:00
parent 8b9b96fcbb
commit 948193e6e8
1 changed files with 0 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
content/robots.txt
View file

@ -1,6 +1,3 @@
# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
#
# To ban all spiders from the entire site uncomment the next two lines:
User-Agent: * User-Agent: *
Disallow: / Disallow: /
Sitemap: <%= @site.config[:base_url] %>/sitemap.xml Sitemap: <%= @site.config[:base_url] %>/sitemap.xml