robots-exclusion-issues - Revision history

ChristopheDucamp: Reverted edits by I MADE THOUSANDS OF ACCOUNTS IN MICROFORMATS WIKI GO TO CHECK OUT THE USER LIST ([[User talk:I MADE THOUSANDS OF ACCOUNTS IN MICROF

2013-09-22T11:25:23Z

Reverted edits by I MADE THOUSANDS OF ACCOUNTS IN MICROFORMATS WIKI GO TO CHECK OUT THE USER LIST ([[User talk:I MADE THOUSANDS OF ACCOUNTS IN MICROF

Show changes

I MADE THOUSANDS OF ACCOUNTS IN MICROFORMATS WIKI GO TO CHECK OUT THE USER LIST at 04:13, 22 September 2013

2013-09-22T04:13:19Z

Show changes

Martin.leese: Undo revision 33770 by PeterJ (Talk) As this is an open issue, I have moved it to the Issues section

2013-03-24T01:14:21Z

Undo revision 33770 by PeterJ (Talk) As this is an open issue, I have moved it to the Issues section

← Older revision		Revision as of 01:14, 24 March 2013
Line 19:		Line 19:
	* ''The "efficacy" issues from [[rel-nofollow#open_issues\|rel="nofollow"]] also apply.''		* ''The "efficacy" issues from [[rel-nofollow#open_issues\|rel="nofollow"]] also apply.''
	** ACCEPTED. The key here would be to get one or more search engines to implement it and get others to do so as well, just as the original [[meta robots]] specification.		** ACCEPTED. The key here would be to get one or more search engines to implement it and get others to do so as well, just as the original [[meta robots]] specification.
	*** Yahoo! has implemented a similar [http://help.yahoo.com/l/us/yahoo/search/webcrawler/slurp-14.html robots-nocontent] class.
	* ''The "collateral damage" issues from [[rel-nofollow#open_issues\|rel="nofollow"]] also apply.''		* ''The "collateral damage" issues from [[rel-nofollow#open_issues\|rel="nofollow"]] also apply.''
	** REJECTED EXAMPLE NEEDED.		** REJECTED EXAMPLE NEEDED.

Martin.leese: /* 2008 */ Typo

2013-03-24T01:13:05Z

2008: Typo

← Older revision		Revision as of 01:13, 24 March 2013
Line 50:		Line 50:

	=== 2008 ===		=== 2008 ===
	* {{OpenIssue}} 2008-10-09 raised by [[User: PeterJ\|PeterJ]]. Yahoo! has implemented a similar [http://www.ysearchblog.com/2007/05/02/introducing-robots-nocontent-for-page-sections/] class.		* {{OpenIssue}} 2008-10-09 raised by [[User: PeterJ\|PeterJ]]. Yahoo! has implemented a similar [http://www.ysearchblog.com/2007/05/02/introducing-robots-nocontent-for-page-sections/ robots-nocontent] class.

	== template ==		== template ==

Martin.leese: Moving PeterJ's 2008 open issue to the Issues section + updated the (dead) URL

2013-03-24T01:11:45Z

Moving PeterJ's 2008 open issue to the Issues section + updated the (dead) URL

← Older revision		Revision as of 01:11, 24 March 2013
Line 48:		Line 48:
	* {{OpenIssue}} Isn't the Robot Exclusion Profile designed for machines first and humans second instead of vice versa? Yes, just as much as [[rel-nofollow]], the deployed microformat that it's designed to replace.		* {{OpenIssue}} Isn't the Robot Exclusion Profile designed for machines first and humans second instead of vice versa? Yes, just as much as [[rel-nofollow]], the deployed microformat that it's designed to replace.
	* I'd like to echo this concern. We need to discuss whether or not this is a suitable microformat. --[[User:RyanKing\|RyanKing]] 13:34, 17 Jan 2006 (PST)		* I'd like to echo this concern. We need to discuss whether or not this is a suitable microformat. --[[User:RyanKing\|RyanKing]] 13:34, 17 Jan 2006 (PST)

			=== 2008 ===
			* {{OpenIssue}} 2008-10-09 raised by [[User: PeterJ\|PeterJ]]. Yahoo! has implemented a similar [http://www.ysearchblog.com/2007/05/02/introducing-robots-nocontent-for-page-sections/] class.

	== template ==		== template ==

PeterJ: Add reference to an implementation of a similar concept in response to a to-do item

2008-10-09T05:38:23Z

Add reference to an implementation of a similar concept in response to a to-do item

← Older revision		Revision as of 05:38, 9 October 2008
Line 19:		Line 19:
	* ''The "efficacy" issues from [[rel-nofollow#open_issues\|rel="nofollow"]] also apply.''		* ''The "efficacy" issues from [[rel-nofollow#open_issues\|rel="nofollow"]] also apply.''
	** ACCEPTED. The key here would be to get one or more search engines to implement it and get others to do so as well, just as the original [[meta robots]] specification.		** ACCEPTED. The key here would be to get one or more search engines to implement it and get others to do so as well, just as the original [[meta robots]] specification.
			*** Yahoo! has implemented a similar [http://help.yahoo.com/l/us/yahoo/search/webcrawler/slurp-14.html robots-nocontent] class.
	* ''The "collateral damage" issues from [[rel-nofollow#open_issues\|rel="nofollow"]] also apply.''		* ''The "collateral damage" issues from [[rel-nofollow#open_issues\|rel="nofollow"]] also apply.''
	** REJECTED EXAMPLE NEEDED.		** REJECTED EXAMPLE NEEDED.

Tantek: resolved issues from 2005, moved brainstorming to robots-exclusion-brainstorming

2007-11-13T19:35:50Z

resolved issues from 2005, moved brainstorming to robots-exclusion-brainstorming

@@ Line 16: / Line 16: @@
 == resolved issues ==
 Issues that are resolved but may have outstanding [[to-do]] items. As issues are resolved, they will be moved from the top of the [[robots-exclusion-issues#Issues|Issues list]] to the bottom of this section.
 === 2005 ===
-* ''The "efficacy" and "collateral damage" issues from [[rel-nofollow#open_issues|rel="nofollow"]] also apply.''
+* ''The "efficacy" issues from [[rel-nofollow#open_issues|rel="nofollow"]] also apply.''
 ==== Precedence ====
 * Should earlier values take precedence or later?  Does <code>class="robots-nofollow robots-follow"</code> means the same as <code>class="robots-nofollow"</code> or <code>class="robots-follow"</code>?
-* <code>meta</code> tag suggests not using conflicting or repeating directives and so does not specify precedence.  <code>&lt;p class="robots-noindex robot1-index"&gt;</code> is an apparent conflict but in this case the more specific should obviously override the general at its point of applicability, no matter what order the directives appear in.
+** ACCEPTED. Order {{must not}} matter in usage of the <code>class</code> attribute as it is defined to be a set by HTML4.  The interpretation of conflicting values could be defined conservatively, that is in the presence of both a directive and its "no" variant, the "no" variant supercedes.
-* Interaction with [[relnofollow]]: what does <code>class="robots-follow" rel="nofollow"</code> mean?  Currently [[relnofollow]] has no profile URI defined, so the Robot Exclusion Profile takes precedence.  In the future, per XMDP's [http://gmpg.org/xmdp/description#multiple Using Multiple Profiles], <q>the URIs in the 'profile' attribute are to be treated most significant (first) to least significant (last).</q>
+* <code>meta</code> tag suggests not using conflicting or repeating directives and so does not specify precedence.  <code>&lt;p class="robots-noindex robots-index"&gt;</code> is an apparent conflict but in this case the more specific should obviously override the general at its point of applicability, no matter what order the directives appear in.
 ==== Phrases ====
-Modern search engines normally support <i>phrase</i> queries.  A phrase query only maches documents that contain the words of the query, consecutively and in the same order.  That does beg the question of whether a matched phrase should be allowed to straddle a <code>class="robots-noindex"</code> region.
+* Modern search engines normally support <i>phrase</i> queries.  A phrase query only maches documents that contain the words of the query, consecutively and in the same order.  That does beg the question of whether a matched phrase should be allowed to straddle a <code>class="robots-noindex"</code> region. Intuitively this should not be allowed.  The phrase query <code>"word1 word2"</code> should not match a document that contains <code>word1 &lt;b class="robots-noindex&gt;ignore&lt;/b&gt; word2</code>.  This does allow for an interesting tool for webmasters can specify that juxtaposed words not be considered to be phrases -- just specify an empty unindexed region as in <code>word1 &lt;b class="robots-noindex&gt;&lt;/b&gt; word2</code>.
+** ACCEPTED FAQ. Yes, any content in a "noindex" element should be treated as if it is not present, and thus would not form part of a phrase.  The second example is less clear however, since many search engines search for phrases withoiut spaces as well, e.g. they provide results for "word1word2" even when the user searches for "word1 word2".
 ==== Specificity ====
 * Does not allow control of specific UAs à la [http://www.robotstxt.org/wc/norobots.html A Standard for Robot Exclusion]
+** REJECTED OUT OF SCOPE. The [[meta robots]] has no specific control of UAs either, and thus it is out of scope for [[robots-exclusion]].  Brainstorming on a possible extension for specific UAs has been moved to [[robots-exclusion-brainstorming]].
 ==== Keywords ====
 * The keywords <code>all</code> and <code>none</code> are defined by the Robots META Tag as convenience shortcuts to enable or disable the combination of <code>nofollow</code> and <code>noindex</code>, but predate Google's <code>noarchive</code> and should not be considered to include it.  As a result, for purposes of clarity and simplicity (the [http://gmpg.org/xmdp/description#principles XMDP Minimalism principle]), they are not included in this version of the Robot Exclusion Profile.
 === 2006 ===
 ==== Suitability as a microformat ====
-* Isn't the Robot Exclusion Profile designed for machines first and humans second instead of vice versa?  Yes, just as much as [[relnofollow]], the deployed microformat that it's designed to replace.
+* {{OpenIssue}} Isn't the Robot Exclusion Profile designed for machines first and humans second instead of vice versa?  Yes, just as much as [[rel-nofollow]], the deployed microformat that it's designed to replace.
 * I'd like to echo this concern. We need to discuss whether or not this is a suitable microformat. --[[User:RyanKing|RyanKing]] 13:34, 17 Jan 2006 (PST)
 == template ==
@@ Line 97: / Line 56: @@
 == related pages ==
 * [[robots-exclusion]]

Tantek: moved more issues here

2007-11-13T01:18:30Z

moved more issues here

@@ Line 20: / Line 20: @@
 === 2005 ===
 * ''The "efficacy" and "collateral damage" issues from [[rel-nofollow#open_issues|rel="nofollow"]] also apply.''
 == template ==

Tantek: drafted, moved issue(s) from spec to here

2007-11-13T00:28:02Z

drafted, moved issue(s) from spec to here

New page

<h1> robots exclusion issues </h1>
{{TOC-right}}
These are externally raised issues about [[robots exclusion]] with broadly varying degrees of merit. Thus some issues are REJECTED for a number of obvious reasons (but still documented here in case they are re-raised), and others contain longer discussions. Some issues may be ACCEPTED and perhaps cause changes or improved explanations in the spec.

'''IMPORTANT''': Please read the [[robots-exclusion-faq|robots exclusion FAQ]] ''before'' giving any feedback or raising any issues as your feedback/issues may already be resolved/answered.

Submitted issues may (and probably will) be edited and rewritten for better terseness, clarity, calmness, rationality, and as neutral a point of view as possible. Write your issues well. — [http://tantek.com/ Tantek]

For matters relating to the [[meta robots]] specification itself, see [[meta-robots-errata]] and [[meta-robots-suggestions]].

== closed issues ==
Resolved issues that have no further actions to take. These will likely be moved to a separate page like [[robots-exclusion-issues-closed]].

* ...

== resolved issues ==
Issues that are resolved but may have outstanding [[to-do]] items. As issues are resolved, they will be moved from the top of the [[robots-exclusion-issues#Issues|Issues list]] to the bottom of this section.

== issues ==
=== 2005 ===
* ''The "efficacy" and "collateral damage" issues from [[rel-nofollow#open_issues|rel="nofollow"]] also apply.''

== template ==
Please use this format (copy and paste this to the end of the list to add your issues):
* {{OpenIssue}} YYYY-MM-DD raised by [http://yourhomepage.example.com YOURNAME].
*# ''Issue 1: Here is the first issue I have.''
*# ''Issue 2: Here is the second issue I have.''

== related pages ==
* [[robots-exclusion]]