RSS DoS problem (was Re: [uf-discuss] UID, URL, live microformats (was: Microformat auto-discovery WAS: Plazes & Microformats))

[Tantek Ç elik]

On 4/19/06 10:21 AM, "Scott Reynen" <scott at randomchaos.com> wrote:

> On Apr 19, 2006, at 11:39 AM, Tantek Çelik wrote:
> 
>> 2. Live Microformats.  This is the *really* interesting bit IMHO.
>> As the
>> Atom folks have explicitly designed with rel="self", when a chunk
>> of data
>> indicates where to get the latest version, you get self-contained
>> syndication.  Even "static" copies of the data contain the
>> necessary info to
>> *subscribe* to an update of the data.
> 
> This is very interesting, and I think it could be a major bridge
> between desktop and web applications.

Yes.


> But I'm a little worried about
> the overhead of loading an entire HTML document just to check if a
> single node within has changed since the last check.

Note that usage if URL UID in this fashion will probably lead to smaller
more specific pages for specific "nodes".  Even now Upcoming and Eventful
both have specific pages for each event and each venue.

Though you do point out that like all current popular forms of web
syndication, contacts and events have something similar to the "RSS Denial
Of Service" problem.

http://weblog.philringnalda.com/2002/10/19/joels-rss-problem


> If my address  
> book was doing daily update checks of every contact I have, that
> would be a huge waste of bandwidth.

For contacts, unlike typical news/blog syndication scenarios, it is doubtful
that we will need to have contacts be checked on a "daily" basis.  Perhaps
once a quarter would suffice.  Perhaps just before mailing someone a
package, or if you haven't emailed them in a while, you would do an update
check just to check to see if your info is up to date.

Thus contacts may actually be able to avoid the problem of RSS DoS that
blogs/news have, since the update semantics are different.

For events, again the updating semantics are different, in that you probably
want updates in advance of the event every day or so, but near the time of
the event, and during the event, you may want hourly updates.  After the
event, it is doubtful you will want many updates, perhaps check it every
week or so for a month for post-event updates and links etc.

This obviously makes the even more susceptible to heavy loads during the
most critical time (during the event), but is also a different problem than
the RSS DoS problem.


> And even more so if my calendar
> application starts subscribing to each event individually.  Is there
> some combination of HTTP headers that can be used to check if a
> particular section of a document has changed before loading the whole
> thing?

Services like Upcoming and Eventful allow you to subscribe to all your
events of interest in one feed, thus you don't have to subscribe to each
event individually, in the scenario you describe.  If however, you need to
force update a specific event, you could do so.

HTH,

Tantek