[uf-discuss] microformats for search engine spiders?

[Ryan King]

On Jan 16, 2007, at 1:56 PM, David Mead wrote:

> Myself and a colleague were discussing microformats today and we
> started wandering off on a tangent about the following.
>
> Would microformats be the way forward to mark defined areas of a web
> page for the sake of search engine spiders?  When anyone looks at a
> web page the instinctively break it into navigation, content,
> advertisements etc. What if we had standard names (microformats) to
> wrap these areas in to work in conjunction with the sitemaps.org
> standard in aiding the spiders to differentiate between the content
> area and, say, text-ads area.  You could then delve deeper into the
> XML file of your sitemap and tell it visit certain pages every day and
> then specify which section of that page they should concentrate on.

Hi David, welcome to the list.

I have 2 answers to your question.

First, in many ways it doesn't matter whether the consuming agent is  
a search engine or a different kind of User Agent, they all get the  
same value and use the same semantics.

Second, a long long time ago, on a website far away (the Technorati  
developer's wiki, which was precursor to microformats.org), there was  
research into a microformat for excluding parts of a page from robots  
[1]. The initiative died and frankly wasn't very close to user  
practice on the web. Anyway, it's probably worth taking a look at  
anyway.

-ryan

1. http://microformats.org/wiki/robots-exclusion