I have updated this draft and released some code.
Regular readers of my blog know that I am obsessed with one thing: decentralisation. I hate having all my eggs in one basket. IMHO, if anyone has the ultimate power, then it’s not true Web 2.0. In my recent thinking this has extended to my view of social networks such as Facebook and MySpace.
One of the much-used features of these is personal messages. This can be public (ala Facebook wall) or private (ala Facebook message). I here intend to provide a TEP, which I will implement, suggesting a way to do this in a decentralised and RESTful manner.
Detecting the endpoint
I love XRDS! This format comes to us from the OpenID world but really provides a standard way to associate RESTful endpoints with a URL [example].
I think it makes most sense to have one type URL for this TEP and declare a namespace for extending XRDS to (optionally) tag endpoints with a ‘sub-type’. This could look like the following:
<?xml version="1.0" encoding="UTF-8"?><xrds:XRDS xmlns:xrds="xri://$xrds" xmlns="xri://$xrd*($v*2.0)" xmlns:dsn="http://singpolyma.net/dsn/xmlns"> <XRD> <Service priority="50"> <Type>http://singpolyma.net/dsn/pmt <URI>http://example.com/msg <dsn:msgtype>private</dsn:msgtype> </Service> </XRD> </xrds:XRDS>
(I will create these pages in a bit.) I would propose the following dsn:msgtype for now: private (only to you) and public (like Facebook wall). As this is a TEP, nothing is remotely final. I’m not sure I like msgtype as the tag name, for one thing.
Sending Data to the Endpoint
This part is easy. To support maximum compatibility with existing systems, either GET or POST data is to be accepted. All fields passed must be processed. The simplest system will simply take all data passed via both GET and POST and put the field name as a header and the value under/beside it in the message body. No data may be ignored (unless there is a particular value, such as a session ID, specific to your server which you want removed). The following is a list of fields which make sense to use in a special way. If you are going to do something special for one of these values you should use the name given here, but really any data can have anything done with it as long as it is included somewhere in the message.
subject, from (contact of sending user), cc, url (of sending user), name (of sending user), body
XHTML is allowed. Special data (such as calendar events) should be marked up with microformats where possible.
A way to contact the user back SHOULD be supplied. If either from or url is a URL, XRDS discovery can be performed on it in order to detect an endpoint for this TEP to use in replying.
I am going to write some code to implement this both for WordPress and independently soon. I may also create a simple service targeted at adding this capability to a blogger blog (or other locked-hosting website).
9 Responses
Ooh, note to self : XRDS service elements can include multiple type nodes! This is WAY useful! No extra namespace should be needed for this TEP, just specify one type node for the msg TEP and one for the type (if applicable). If none is specified the user should be notified that this may not be a secure message because no privacy level is specified. I should work this into the post and create the references pages soon.
This stuff goes a bit over my head now, this being a friday evening 😉 But, the link took me to your webos wiki. That seems to be a good resource to build upon! Recently, I have been reading about semantic web and got to know about the happenings in that space like sparql etc. You might want to use FOAF if it makes sense.
@Ramani – Everything makes more sense in code, I hope to release some services loosely based on this soon, which I will want your (and other Blogger Hackers’) feedback on at that time. For now I agree that this looks a bit nebulous unless you’ve been in my head.
FOAF is something I’ve been interested in, but I definitely support a microformatted approach (XFN+hCard) first, but that’s me.
I agree entirely with you about the importance of decentralization in the age of software as a service. I enjoyed your post about how this may be possible in a social networking context. But you your interests and comments be extended?
I would be interested in seeing a post about Yacy (but not Faroo at as it is not open source). What is the future of P2P search? Why doesn’t it work well? How can it be fixed? Does it just need more participants?
In addition, can you use decentralized computing to distribute computational according the time zone and by so doing use people’s computers in “dark” timezones to power the web for those in “light” timezones. This ameliorate the problem of energy storage.
It would be great to see such a movement begin in Waterloo and by someone from Africa in Canada. Simply put the culture of decentralization and cooperation in Canada could introduce a fascinating new Internet age.
Thanks for your insightful blog post!
@a — Yacy &c seems to be a different kind of ‘decentralisation’. They duplicate data across donated server space so that if one server goes down/goes evil, everything still works. Whereas with ‘web as a service’ decentralisation it is more so that you can do it how you want and so that if a website goes down/goes evil you still have YOUR part.
Decentralised computing based on timezone seems counterproductive. DC already uses only unused CPU clock cycles. This is the way to go. Then we can maximize cycles worldwide, no matter what time it is.
In order to make a decentralized ‘web as a service’ (decentralized in the sense that I could have my own, like I have my own linux distribution that I can change at will) you would need an enormous amount of storage, bandwidth and cycles. To accomplish search individually, to crawl & index the 9 billion or so webpages alone, just isn’t pragmatic. So, in order to have ‘my’ part of search or to have a search that doesn’t depend on the whims of a private corporation (company goes down/evil), I would have to admit that I would have to do it as a group.
The physical decentralization of a P2P search system accomplishes the decentralization that I thought you were talking about: no one really has ultimate power because search is owned by all of the peers in the network. What is more, because the search engine (crawler and indexing) is open source you can be confident that it is not evil (but Yacy has yet to address security issues that could lead to peers contaminating the distributed hash). What is also useful is that the power of information — what people search for — is also in no one person’s hands because the P2P framework of such engines as Yacy makes it almost impossible to store centrally all of an individual peer’s search queries.
So, I agree with you that the P2P model is decentralized in the physical sense and that it doesn’t accomplish the goal of giving everyone their own personal private web service (YOUR part, as you say), but it does accomplish the goal of taking the power of out of any one particular person or groups’ hands. No one and everyone would own search in such a model.
I think we are actually on the same page about distributed computing – that portion of the comment was intended only to show that an added benefit and natural byproduct of a decentralized search engine (in the decentralized as in ‘we all chip in’ sense) and distributed computing in general is that idle computers (which are mostly those in dark time zones) would be used.
I also like your xmpp idea…
@a – Ah, I see what you mean! Yes, in that sense, it’s very similar, in a ‘no one owns this’ kind of way. Of course, if all the major search engines went down/went evil starting a search engine project from scratch would not take too long to become decent (although starting now definitely gives a headstart!).
Hmm… one thing I’ve always thought is that while we can’t trust big companies, we should let them participate. Does the Yacy federation have the technology in place that could allow Google et al to federate if they wanted to?
I agree with you entirely about federation and that a private and non-governmental public mix would be excellent.
No, Yacy doesn’t appear to have opted into such a scheme, but it would be great if it did! The opensearch movement — which permits federation — is obviously the way to go.
An excellent combo, in my eyes, would be a cleversafe hash (perhaps), a p2p crawler and indexing system and p2p accepting pushed opensearch xml (opensearch rss). As well as a means for federation, as you say although you’d have to structure the license somehow to avoid the problem that wikipedia faced with answers.com.
Actually… yacy appears to accept opensearch format…