[OMB] Questions and concerns about OMB after re-reading the spec

[Evan Prodromou]

Adewale Oshineye wrote:
> I have some suggestions and concerns and suggested changes to the
> current version of the spec. 
Hey, so, thanks a ton for this! Responses inline below.

> 1-First a comment about the terminology used. Listenee and listener
> are really awkward ways of expressing the publisher-subscriber
> relationship. Given that OMB is describing a messaging system could we
> just use standard terminology where the publisher pushes out messages
> to subscribers? We could replace listener with subscriber and listenee
> with publisher.
>   
That's probably for the best. I think "publisher" and "subscriber" may 
sound a little formal for what's 80% of the time a personal and friendly 
relationship, but since it's asymmetrical, unlike "real" friendship, 
it's probably a good idea to use pub and sub. We can just put a note 
about other terms that are used ("friend", "fan", "follower").
> 2-At the moment the spec (on line 269) implies that the number of
> messages exchanged between any 2 OMB implementations is going to be M
> * N (where M is the number of publishers on Server A and N is the
> number of subscribers on Server B). It would be better if the spec
> required ServerA to track the set of remote subscribers for all it's
> users and then send 1 message to Server B per publisher-subscribers
> mapping. It would then be Server B's responsibility to 'fan-out' the
> messages to it's users.
>   
Actually, that's optional and up to the subscriber's server. See 
http://openmicroblogging.org/protocol/0.1/#id7
> 3- The omb_notice_content (line 231) is currently an undefined format.
>   
Ah. That's plain text in all current implementations.
> Can we replace this with an Atom Entry?
That sounds like an amazing idea. It would probably let us leave out a 
lot of the other fields, too.

Are there any existing standards around push-oriented Atom? It may be a 
good idea to fit in with them.
> Problems for which I don't have solutions:
> 1-The OMB spec currently makes no provision for flow control. If a
> server can't keep up with the flow of messages it is receiving it has
> no option but to start dropping messages. The spec has no way for
> ServerB to signal to ServerA that it should reduce the amount of data
> that it is sending. If ServerB isn't as powerful as ServerA or ServerB
> has a poor connection to the internet (for instance if it's running on
> an ADSL connection) then ServerB's users will miss messages.
>   
Would a 503 response ("Service Unavailable") with "Retry-After" make 
sense here? We should maybe call it out. See 
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.5.4
> 2- The spec has no notion of presence. If one of the ServerB falls
> over then ServerA will just keep throwing messages at the place
> ServerB used to occupy. When ServerB comes back it's users will have
> missed a large chunk of messages and the spec currently provides no
> way for ServerB to retrieve all the lost messages or for ServerA to
> detect that ServerB has failed and store the messages till ServerB is
> back  online.
>   
I see a couple of ways to deal with failed service and dropped notices. 
One is to just forget it; this is microblogging, not heart surgery.

Another is that one of the profile entries (like nickname, bio, etc.) 
could be the URL of a notice stream (RSS 1.0 or 2.0 or Atom). Server B 
could pull the streams for all its remote subscriptions. SUP might be 
useful here. http://code.google.com/p/simpleupdateprotocol/
> 3- The previous two points hint at a deeper problem. The current
> version of the spec assumes that all the servers in the OMB federation
> are equally scalable. If a site the size ofTwitter implements the
> current spec then they will be required to post an arbitrary number of
> messages to an arbitrary number of other servers. Some of these
> servers won't have the bandwidth or processing power to handle the
> sheer amount of traffic that could be sent their way if their local
> users subscribed to a few thousand Twitter users.
I'm not sure this concern is justified. I'd compare SMTP email here. 
Yes, there's a lot of email out there, but if I set up a personal server 
or a server for my small company, it only has to handle the volume 
demanded by my limited number of users. Incoming notices will always be 
proportional to number of users.

Every server with N users, where each user has an average M remote 
subscriptions, will have to support N x M subscriptions. If each remote 
user posts on average F times per day, that's N x M x F Web hits per day.

If N = 10 (small site), M = 150 (active, long-time users), and F = 5 
(very high!), that's 7500 Web hits per day for posted notices. Outside 
of Apache on a cell phone, I think that most servers can handle this 
kind of load. Most $5.95/mo. commodity Web hosting services can 
comfortably handle 2 orders of magnitude higher than that, maybe 3. 
Also, these are relatively small requests -- not high-bandwidth ones.
> This kind of
> imbalance is what lead Gnip to shut down their service:
> http://blog.gnipcentral.com/2008/11/03/winding-down-xmpp-for-now/
I think that's an invalid comparison. Servers in the OMB network only 
have to handle hits for their own users, not for every user in the 
entire network.
> 4- It's not possible to have a 1 way relationship according to the
> spec ( line 160). You can only listen to someone with their explicit
> permission.
You misread that section. The /subscriber/ has to authorize the 
subscription. You can only post notices to someone's inbox with their 
explicit permission. I think that's pretty fair.

In fact, 1-way relationships are all that OMB supports. There is space 
in this flow for the /publisher/ to authorize the subscription, although 
I don't think it's explicitly called out. After the subscriber has made 
the request, the publishing server can request authorization from the 
publisher. The publishing server can either post notices, or not, based 
on the publisher's wishes. There's not any way to send a message that 
the subscription has been disallowed or canceled, though.
> 5- The spec doesn't currently address the case where another service
> wants access to an OMB implementation's 'firehose' of all public
> notices from it's users. 
I think that a subscribe-to-whole-server option might make sense for a 
future version, but I think we should leave it explicitly out of scope 
for 0.2.

Altogether, great set of points!

-Evan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroblogging.org/pipermail/omb/attachments/20090131/3153dd8d/attachment.htm>