[microformats-discuss] Comments on robot exclusion

[Robert Bachmann]

http://microformats.org/wiki/robots-exclusion is quite interesting.

I wonder if an simple application (perhaps a PHP or Python script)
for visualization makes sense.

For example this application could be invoked with an URL of a HTML
document. The application phrases the HTML document and displays:
- links which would be followed in green.
- links which wouldn't be followed in red.
- text which would be indexed in black.
- text which wouldn't be indexed in grey.

Do you think such an application would make sense?
If yes, I could try to implement it.

Robert
-- 
Robert Bachmann <rbach at rbach.priv.at> (OpenPGP KeyID: 0x4A5CCF10)