projects
/
mailer.git
/ blobdiff
commit
grep
author
committer
pickaxe
?
search:
re
summary
|
shortlog
|
log
|
commit
|
commitdiff
|
tree
raw
| inline |
side by side
robots.txt forbids robots to enter inc and templates on weak servers
[mailer.git]
/
robots.txt
diff --git
a/robots.txt
b/robots.txt
index 6e16af6709b85b893ba0d8d8a67ee1aaf7847f34..7b26c7d62ea6f5cfa2179b1e6e2b7583d6c74880 100644
(file)
--- a/
robots.txt
+++ b/
robots.txt
@@
-400,3
+400,11
@@
Disallow: /whitelist.txt
User-agent: Googlebot
Disallow: /whitelist.txt
+
+User-agent: *
+Disallow: /inc/
+Disallow: /templates/
+
+User-agent: Googlebot
+Disallow: /inc/
+Disallow: /templates/