X-Git-Url: https://git.mxchange.org/?p=mailer.git;a=blobdiff_plain;f=robots.txt;h=5b0391be8a17ddfbd617621c8a3dd1dfcc078a66;hp=deb99838938f81e30ac406dadeedb252e4596571;hb=0eea910f4bcaffeb4c6193f4a06f1d187d3a197d;hpb=167b7b3fded7327bb9a588854718623df42fbaf1 diff --git a/robots.txt b/robots.txt index deb9983893..5b0391be8a 100644 --- a/robots.txt +++ b/robots.txt @@ -430,7 +430,7 @@ Disallow: / User-agent: searchpreview Disallow: / -# This is for my webpages +# This is for my web pages using spider-trap User-agent: * Disallow: /guestbook/ Disallow: /blacklist.txt