From a3c5ef59d6b79d91d8eceaeb2085407386182aed Mon Sep 17 00:00:00 2001 From: Mikael Nordfeldth Date: Fri, 26 Feb 2016 13:41:14 +0100 Subject: [PATCH] Fix merge #101 by replacing a non-working Yahoo! link with Wikipedia The link was meant to describe robots.txt crawl-delay info --- CONFIGURE | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONFIGURE b/CONFIGURE index 362448a3b6..b1773347d4 100644 --- a/CONFIGURE +++ b/CONFIGURE @@ -674,7 +674,7 @@ Web crawlers. See http://www.robotstxt.org/ for more information on the format of this file. crawldelay: if non-empty, this value is provided as the Crawl-Delay: - for the robots.txt file. see https://help.yahoo.com/kb/search + for the robots.txt file. see for more information. Default is zero, no explicit delay. disallow: Array of (virtual) directories to disallow. Default is 'main', 'search', 'message', 'settings', 'admin'. Ignored when site -- 2.39.5