Decius wrote:
] Having said that, I'm recommending this article because these
] are fairly reasonable ideas. Yes your RSS aggregator should be
] signing up to receive a message anytime something new pops up,
] instead of going around to 1000 machines and asking every few
] minutes. As for using bit-torrent as an RSS feed distribution
] mechanism, yes, but no. Bit-torrent shouldn't know anything
] about RSS or vice-versa. My take is that bit-torrent should be
] pushed down below the applications, and it ought to cache any
] web page you hit if the owner marks it as cachable. Any url
] you try to pull should get pulled from bit-torrent first, if
] available.

I agree with Decius..

Better ways of tracking when content has changed are in order. Thats the key problem. Not the clients fetching the data, its the client's checking the data.. There are multiple systems out there already in a position to solve that problem, and they are likely going to be the ones to do it. Technorati, Newsisfree, Syndic8, etc.. A new pure distributed model isn't really necessary when the natural market place creates multiple sites that can play the role of content checker and knowledge cache. Its already distributed.. The solution to the problem lies in someone knowing what content you care about, and telling you when its changed.. That's could happen via some new great piece of distributed vaporware that's just gonna work and get widely adopted overnight, but its not likely. Its probably going to happen by subverting another key persistent connection the user has already had for many years that preforms a very, very, very, similar function: Instant Messaging.. [ think the presence mechanism.. not geting IMs.. -ed ]

Bittorrent like distribution models only makes sense for rich content. Something much larger then the average 10k - 30k size of most text only RSS documents. There is a certain overhead that would be created by such a network, and I don't see that as necessary unless the data being distributed warranted it. And wiring that into the syndication format is a bad idea.. The syndication format, as far as any given author/weblogger is concerned, is just their authoritative source what's available/new/changed.. It points to content associated with entries and the various methods of retrieving them, and the client can make a descision about how/if to get it based on that.. That is the point where bittorrent like models fit in, once there is content that is sizable enough to warrant the overhead of a p2p swarm/cloud kind of thing.

Even the current semantics in play enforce this way of looking at it. Its called a syndication format, not distribution format.

Don't read this as me totally dismissing the idea of distributed feeds.. Things like Konspire/kast highly interest me. I think they will be critical at some point when it comes to distributing content.. However, being informed content has changed or new content exists, and actually being given content are different things.. Its easy to lump it all together in the context of RSS when its just a few k of hypertext being pushed around, but that's going to change when more pictures and video start moving around. Those problems will not stay lumped together for long.

Right now we are still figuring out (and arguing) how the authoritative sources are going to work in what's already a pretty distributed environment.. The solutions that handle scaling and distributing the content will be built on top of that kind of thing, not eliminate or replace it.

Also worth noting, many people seem to be unaware that unless the RSS document has changed, the HTTP connection that checks is very small. If anyone is thinking that every one of those hits was the full size of the file, its not. Only if its changed. Hence, I worry this is leading people to do some very bad math in estimating their current overhead, and using it to justify that of a p2p network.. Granted, there are a number of sites that do not properly support cashing (we are still currently one of them), but that's their problem and their bandwidth. I think all the popular news readers get it right at least..

RE: Yahoo! News - BitTorrent and RSS Create Disruptive Revolution