Singpolyma

Technical Blog

Archive for the "Tech" Category

Comments about Awards API

Posted on

@codenamebowser has written up a minimal Awards API spec similar in nature to what I proposed recently. @wolever has already made some comments. My comments are below.

OAuth: yes. Be aware that OAuth (and really any other system) cannot securely identify non-web apps. This is because auth keys stored in an app in the possesion of a user can be got by that user. Many people (including myself and the IETF OAuth spec editor, Eran Hammer-Lahav) feel that desktop/mobile apps should not use consumer tokens at all, but should use empty or “anonymous” tokens so the app can say to the user “an application claiming to be…”

User Accounts: whaaa? I guess you’re trying to let sites use the system if their users don’t, but you really probably want to be publishing to an actual usr account, not an invented thing. Make a way for new users to seamlessly get an account. You could start OAuth from the client and when they come if they have an account, log them in and associate. Otherwise get some ID from them (email address or web address) and use that to authenticate them (using email verification and/or openid and/or relmeauth).

I agree with @wolever that if you’re going to have limits, you should make them early on. Most things shouldn’t need limits, though.

I would make sure all properties are accessed as though they are just keys in the allswed kvp. That way the whole API is simpler and feels the same as itself. Some keys are just predefined with meaning and some aren’t.

Threshold seems like overkill for v1. The client probably knows that anyway.

If you want likits for IDs, look up the limits on the HTML ID attribute.

As @wolever says, don’t count on one user = one email.

This draft doesn’t say how the kvps are going up to the server. form encoded post? While a bit more complex, I would look into POSTing activity streams ATOM, since that will allow distributedness and resyndication to work very well.

Before I got to publishing this a new version of the API spec came out. More comments below.

Authentication: what? I’m reading this, and you have exactly re-invented OAuth. As always when re-inventing a security technology, you have done it poorly (md5? are you on crack?) Use OAuth… it’s the same basic tech you’re suggesting, but is actually good.

Must have for icon is PNG. I think size restrictions would be bad at an API level, though for your implementation you may want to institude a MB limit to prevent abuse.

140 characters is a nice arbitrary limit for SMS, but saying it makes things fit on one line in web designs is just not fair.

In the KvP, are values 8-bit safe? Can I use a 255-byte blob? 8-bit safety is *good*. Make your limits in bytes, not in characters.

users/create_and_retrieve seems like the only one that’s really useful (except maybe retrieve). What’s the use case for users/create? Of course, as above, I think it would be useful to handle user accounts somewhat differently.

Your XML seems like a good candidate for ATOM, as I suggested:


<entry>
	<author><email>dgrace@doomstick.com</email></author> <!-- this could be activity-actor as well/instead -->
	<id>test</id>
	<award:points>10</award:points>
	<title>Test achievement</title>
	<content>Got this for being a guinnea pig.</content> <!-- could use summary for this instead -->
	<link rel="logo" type="image/png" href="data:..." />
	<award:testkey>testvalue</award:testkey>
</entry>

This format (or something similar) has resyndication benefits, and also means that one could pass awards around easily using Salmon or Ostatus.

Generic Achievement System

Posted on

I’ve been thinking a bit recently about generic achievement systems. Basically what I want, is a web service where I’ve got a profile showing off the achievements I’ve unlocked in different ways. This could have some scraping of well-known social web achievements (StackOverflow, FourSquare), but that doesn’t really scale so I’d want an API that any service or application could tie into to publish unlocked achievements to my profile. I’d also want read APIs so I can put achievements I really want to show off on my actual profile page at singpolyma.net. Achievements should be able to take any image+text form, from high scores to badges.

If social web sites can publish achievements to the site and read them back out, that’s great, but there are already a couple of services that sort of serve that purpose (though I’m not sure either serves their own local profile pages). The real benefit here is that applications (especially including games) could easily tie into this as well.

Of course, now that I’ve described it you can see I don’t necessarily want a web service at all. I want an API specification and a reference implementation. Web apps and games should be configurable with my actual “achievement host” (or discover it over LRDD/Webfinger) so that I can just host my own list and not sharecrop on someone else.

Now, on top of all this, I want the same protocol (or something very close to it) to work with a local on-my-machine service that serves a similar purpose: storing scores and achievements for local users. One could then build nice local scoreboard/achievements unlocked display dashboards, which could be very useful in a gaming-heavy environment like the games at www.lionwins.com (some games already do shared local highscores, this is just one up on that).

The right way to build something like this, probably, is to use Activity Streams concepts and publish the content over HTTP as an “achieved” or “scored” event.

Thinking About Aggregators

Posted on

So, I’ve been thinking for awhile about the aggregator experience I want to have for reading my blogs and microblogs. As I’ve increasingly been moving to my own infrastructure for both, my subscription experience has been evolving as well. Right now I have a rather hacky script polling the Twitter and Identi.ca APIs every 2 minutes, putting the content into my WordPress database, and exposing that information over a partial implementation of the Twitter API. I then use a ruby script that polls said Twitter API and sends the messages to me over XMPP (as well as allowing me to post to my self-hosted microblog). I read other feeds using newsbeuter.

This entire setup is a bit suboptimal. Why, for example, is my server polling and then storing content I am subscribed to? Isn’t that normally my aggregator’s job? I’ve realised, that the reason the model has evolved this way elsewhere (such as at status.net) is that in many microblogging services, the service acts as both publisher and aggregator, but this is sort of artificial. My server has no good reason to act as an aggregator, that function is not related to publishing my content.

I thus intend to build a better aggregator infrastructure. My current thought is to build some sort of modular system with data sources and sinks. The primary sources (to start) will be RSS/ATOM feeds and maybe Twitter compatible APIs. On top of this can be built a nice web-feed-reader infrastructure (like Bloglines was), which should support PubSubHubBub for its subscriptions (since it lives on a server asnyway), or an XMPP feed-delivery system (which just connects, delivers the content, and disconnects), or an ncurses or other GUI system for local use.

The system also needs to support replies (via Salmon or Trackback/Pingback) and also cross-posting of said replies (using Twitter-compatible or WordPress-compatible APIs). The public key that goes with the private key the aggregator is using to sign the Salmon slaps needs to be publicly discoverable somewhere (likely at your publishing point), but otherwise the aggregator doesn’t need any kind of public or always-on presence.

Free HTTPS from StartSSL

Posted on

Apparently, some people are still unaware that the can get a TLS/SSL certificate for free from StartCom. StartCom has been providing Class 1 certificates for free for some time now, but they’re certificates were not always widely supported. Having used one of these certificates for my personal site for some time now, I can say with confidence that there are well supported by major browsers and Operating Systems.

Head over to StartSSL Login/Signup and click the “Express Lane” option. The wizard will guide you through creating a new StartSSL account and creating a webserver certificate. You will be asked to give a significant amount of personal information, then asked to verify one of the email addresses associated with your domain name, and finally you will be given the opportunity to have them sign a certificate you have generated (the way to get a CSR is different depending on what webserver you are using or what host you are with, there is good information in the StartSSL FAQ or try Google).

As part of the account creation process a client-side TLS certificate will be generated and stored in your browser or OS keychain. Do not lose this certificate. It is the only way to log into your StartSSL account.

One little gotcha in the way StartCom has chosen to implement their Class 1 CA is that you will not be able to list your main domain as the primary name on the certificate, but will have to list a subdomain. You may list more than one name, and your main domain may be on the certificate, just not as the primary name. I, for example, have www.singpolyma.net as the main name on my certificate, and then have singpolyma.net as a secondary. This is a bit strange, but works fine with most clients (a notable exception being wget).

And that’s about it! Unless you’re on a shared host without a unique IP or host-based TLS support there is no reason for you not to have HTTPS enabled on your personal site.

Signing XML Nodes

Posted on

I’ve been thinking about signing XML nodes. The existing mechanisms are either really complex (XML-DSig) or over-verbose (Magic Sig). This could be useful in RSS/ATOM feeds, XMPP, and other XML-based communication formats. The purpose of this proposal is to provide a lightweight signing (and optionally, encyption) mechanism for embedding inside XML nodes, while not inventing any new XML namespaces, elements, or attributes, not inventing a new envelope format for the signature data, and not suggesting a new way of transmitting octet streams in a text safe way.

Normalization

In order to preserve the form of the XML being signed, an exact textual representation of the XML tree to be signed must be included in the signature packet (“opaque signing”). This is similar to the strategy employed by Magic Sig.

It is recommended that the fragment be encoded as a valid standalone XML document, so that parsers can easily feed the unwrapped content to an XML parser and use the tree that results, without having to graft the text back into the original XML document for parsing.

Envelope format

Rather than inventing a new envelope to mark up what algorithms were used to generate the signature, I suggest using the standard OpenPGP packet format from RFC4880. This standard is well-deployed for use in Email and other cryptosystems, and there are implementations, or partial implementations, in many languages, including PHP.

Inclusion in an XML node

An opaquely signed XML fragment is just an alternative representation of the node it wraps. This relationship is well modelled by the ATOM link element (namespace http://www.w3.org/2005/Atom) with the rel attribute set to alternate.

RFC3156 defines an Internet media type for encrypted and/or signed OpenPGP data as application/pgp-encrypted. This makes an appropriate content for the type attribute.

Text-safe encoding of octets

Protocols may wish to include the OpenPGP packet directly in the XML document, instead of linking to an external resource. In fact, this is probably the normal case. RFC2397 defines a useful mechanism for encoding arbitrary octet streams (such as those used in the OpenPGP binary packet format) as URIs for use anywhere a URI is expected, such as the link element’s href attribute. The media type included in the data URI should be application/pgp-encrypted.

Example

Below is an ATOM fragment demonstrating this recommendation:

<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>Test Feed</title>
<entry>
<title>Test Entry</title>
<link rel="alternate" type="text/html" href="http://example.com/item1" />
<published>2010-03-26T06:47:47+03:00</published>
<link rel="alternate" type="application/pgp-encrypted" href="data:application/pgp-encrypted;base64,owGbwMvMwMF4UUZT8FzgnHuMa/4msZUY6VXk5nhvuC9pYw9kKJSlFhVn5ufZKhnqGSgppOYl56dk5qXbKoWGuOlaKNnbcdmk5pUUVSoA1eYV2ypllJQUWOnrl5eX65Ub6+UXpesbGRiY6juW5OcqAdWWZJbkpNqFpBaXKLiCtNnoQ0S4bHIy87IVilJzbJUSc0pSi/ISS1KVFEoqC1JtlUpSK0r0M0pyc5QUMopS0+CWpFYk5hbkpOol5+fqZ5ak5hoqKegDTSooTcrJLM5ITbEzMjA00DUw1jUyCzEwszIxByJtA2MrAwMbfYQiLht9sA/suDqZZFgYGDkY2FiZQP5n4OIUgAXN+W/8v5h/vXu4cBXbrfUVT0yXTKsI+h1Qw2/zdYZ9h/yJ+/r5L7U1xc/sWMTh4Du1MXjKWsMHP2KZ0ooy7ZVFz0ps011ZaZDUl78y5JbdCbeMNcEcjb0tS1yEu7hfZX6Wsi+qeMDRZO1/1+Lt8c1fqloOLHSqe6YyX8dsp9HmtZlXkxdZVT/67Jcnxig59Rj/Gv0Nr0RPdSg68a/r3vKvV3tavvvyXP6wtWu3NHJrFR25euzI/VdnVMzjuDYt47gjxnm956jU72cvfXj7Dk79+yqfkyPCxfyo/ZFdTJtfK1ncYgur6Dqp47r4EcfZ2pDNwk4n5vYwOfjuWM09Ye6ij+nnbi2Zn2ld/zhx4o+Z2XoF6Twucayxa80urHWeMF0nw/mq2fLFhuv1PWaXRtoHHr9qMc9e+tjdD5KhxQV3Jr49d2YZ2+WjD49UfqtLYVxfxzy3iiG4YWvZHfl2g/yPtx3M+rqVuVbM2Vpzq9q76OytfQsjm769Ely2PGDTmg8ak2frnfL9/49VPKPU8vj0Y0diN213uNkrfeH15TbjM09el/2rCF5v/Uryd3XXB8OfbedesyRJH1s7dWPB04JFvA9Zp6y7IfI2VDo8VHiHVpmqR+aERRt/7Dy1NtmV+48KH8/l7ebCr4KNb7v3SSqfdWDey7lkQ+Nzwdfb1vOlTzXU1zyx0kVX3v2t9h+f1T1px+KuX7rw8gOb17St+69/kvkKAA" />
</entry>
</feed>