X-Git-Url: https://git.mxchange.org/?p=mailer.git;a=blobdiff_plain;f=robots.txt;h=5b0391be8a17ddfbd617621c8a3dd1dfcc078a66;hp=deb99838938f81e30ac406dadeedb252e4596571;hb=0eea910f4bcaffeb4c6193f4a06f1d187d3a197d;hpb=d8987b6d6bd61ae46907220040c71fee40e47fd3 diff --git a/robots.txt b/robots.txt index deb9983893..5b0391be8a 100644 --- a/robots.txt +++ b/robots.txt @@ -430,7 +430,7 @@ Disallow: / User-agent: searchpreview Disallow: / -# This is for my webpages +# This is for my web pages using spider-trap User-agent: * Disallow: /guestbook/ Disallow: /blacklist.txt