The question of whether to open source the wireless village code on onesoup came up before, but I thought I should revisit it now to gather more feedback, and open the conversation.

As I have blogged about in the past, I am fully re-writing the system to move to XMPP. All the code, obviously, is now written in Python. The new architecture consists of a twisted HTTP server and TCP server (for CIR) together with client interfaces to XMPP. Actually, in retrospective, the architecture is not very dissimilar to the one I would expect for a BOSH Connection Manager.

I am considering the pros/cons of opening the project under an open source license. The obvious consequence of opening up would be that anybody could setup their own onesoup server, i.e. a Wireless Village Connection Manager against XMPP backend(s). In other words, any XMPP admin could install onesoup and get their mobile users to access XMPP through the built-in IMPS client on the phones.

But there is a much more important aspect to be considered: interoperability between WV domains. Today, the Wireless Village Server to Server Protocol (SSP) is fairly useless. But if onesoup became a common software gateway for the mobile world to access the XMPP federation, we’d then be able to use the XMPP S2S to replace the IMPS SSP, and then achieve true mobile instant messaging interoperability. And, the thing is, one the key reasons, or perhaps the key reason, SMS triumphed as a disruptive messaging technology was interoperability. You can send and receive messages to your friends, regardless of their operator. Interoperability is the quintessential property for open communications.

But there are other reasons I can think as of why to open source something like onesoup. First, there are number of players competing with proprietary mobile instant messaging technologies, and a really great way to compete effectively with them is through openness. Then, there are also pure technological advantages in IMPS for the mobile world vs raw-XMPP or HTTP-XMPP (BOSH), such as transactional message acknowledgements, binary XML and CIR (SUDP, that is). So in a way, an IMPS CSP HTTP connection manager like onesoup, could be the key for spreading XMPP to the mobile world.

Obviously, there are the financial aspects to be considered too. Open sourcing needs to make financial sense, not only to me, but to any other party that could potentially become a contributor. Sure, there are personal reputation, consulting, contractor, support, etc. opportunities associated, but let’s face it, it’s very hard to make a living out of writing open source software.

Because that’s really the trade off, moving from being a service provider, to being a software provider. I believe Jabber did not get acquired by CISCO because of their service (under the jabber.org domain), but because of their software, talent and responsibility for the standard. On the other hand, the success of Jabber Inc. can be largely attributed to the success of the jabber.org domain.

At this point I’d like to open the conversation, both with end-users and potential contributors alike. Do you think there is space, interest and returns for creating an open source wireless village server to interface to the XMPP federation?

I spent some time last night reviewing the logs for the last weeks. I wanted to share with you some numbers about onesoup’s usage so far:

• 513 registered users
• 3,398 chat sessions
• 1,105 upstream login errors (MSN, Yahoo, Gtalk, etc.)
• 401 login errors
• 59,305 messages sent and received

So, while this is good news, it’s also surprising to see so many errors. Almost half the attempts to connect to the service ended up in failure. There is a combination of factors going on here, which I ‘ll take the opportunity to explain.

I consciously decided not to require active registration, but rather to pass on credentials to 3rd party networks. This makes it simpler to use the service, and also avoids me having to store passwords for 3rd party services. I know this is a security feature important for many users.

But passing on credentials has a down-side: not all phones support PWD authentication scheme, and insist on using MD5 (or extremely rarely SHA1). Possibly up to 1/3 of all phones don’t support PWD. That’s a large proportion of users.

I have been thinking how to address this, at the same time that I consider the  move to XMPP. Here’s what I am planning:

• In order to chat through onesoup, users will have to register and pick a username and password. This will allow me to move to support all phones.
• Signing into onesoup will connect you to the onesoup mobile community, on onesoup’s XMPP server.
• As with any XMPP server, users will be able to add XMPP contacts on other servers that are supporting the server-to-server (XMPP federation). Gtalk is allowing it.

By taking these steps, users will be able to chat with other onesoup users, and with any other friends on XMPP. This includes all native XMPP servers with s-2-s enabled, as well as Gtalk and AIM/ICQ.

However, this also means that Yahoo and MSN support will disappear. Yahoo and MSN account today for 284 out of the 513 total users. Even though it’s a difficult decision to make, for me having a reliable service is more important that serving 513 users 50% of the time only. Unfortunately, that means switching off Yahoo and MSN.

Now, these changes won’t happen immediately. I am literally starting to code the new XMPP-powered onesoup IMPS CSP chat service, and it will take weeks until I release anything that works reliably.

I blogged a few weeks ago that IMPS was dead, and that I would be embracing XMPP. Since then I have received many, many, emails from folks asking me to keep the service. I honestly appreciate the support and interest.

I stand by my original posting that IMPS is a failed specification. It is true though that the original arguments for Onesoup, namely being native to the phone and the ability to run on the background, still hold true, and as such, an IMPS service still makes sense.

The issue is that technically IMPS has failed and I am not willing to invest much more time to support a specification that I consider to be broken. I’d much rather spend my time building a mobile IM service on XMPP. But until the manufacturers stop shipping a built-in IMPS client that barely anybody uses, rather than an XMPP client, there is a point in supporting IMPS.

This is the dilemma where I stand:  IMPS is a failed specification yet a widely-spread technology without a short-term viable replacement. My requirement has therefore transformed to providing mobile IM through XMPP, and having the ability to additionally support IMPS at almost no additional cost. The current Onesoup is fully articulated around IMPS, not XMPP, and I need to move away from there.

The current architecture consists of a very light weight Tomcat server implementing the IMPS Data and Client Initiation Request channels. This is fronted by a custom load-balancer written in PHP that examines the XML payload and distributes the request to a particular node based on the session token identifier. Right now there is only one node, but the intent was to be able to scale to multiple nodes this way. The sticky session is unfortunately required to support the CIR STCP (standalone TCP binding), and since the Tomcat server maintains TCP connections to the 3rd party IM services (e.g. AOL, Yahoo, etc.).

In contrast, the proposed architecture is based around the HTTP BOSH binding to XMPP, which avoids having to open bare TCP server sockets. Note that I am selecting ejabberd as a server, which has a built-in http-bind module, but I could use any dedicated BOSH Connection Manager, such as punjab. I’ll decide on this detail as I move further. Also, I am sticking to Apache as web server for now, although for most of the needs it would be more interesting to use nginx as a load balancer, and push the PHP processing to an Apache backend, either through proxy, or a fastcgi processor.

In the proposed architecture, the IMPS data channel support is provided by a stateless IMPS to XMPP BOSH adapter in PHP. The tricky bit is however the CIR channel, which requires the notion of session.

In order to understand how to best support CIR, I decided to scan through the logs looking for CIR usage, and I realized that all devices that support STCP also support SUDP. Unless proven otherwise when I start coding this, I will be dropping support for the STCP binding.

The challenge of moving to XMPP for Onesoup really comes down to implementing a BOSH client in PHP. I’d had loved the device client to help here, but the IMPS specification uses a data channel (HTTP) and a notification channel (TCP, UDP, WAP, SMS) rather than using only long-lived blocking HTTP requests. To adapt IMPS to BOSH, I need then to handle the original IMPS request, and then continue the processing by issuing an asynchronous CURL to the http-bind backend. Since Apache kills long running processes (a good thing), I’ll need some way to catch this and re-launch the request. I’ll also have to avoid holding more than one TCP connection to the http-bind endpoint per client.

I feel this is a step in the right direction. There is certainly further thinking required in terms to solve the CIR. But this architecture would be much easier to adapt going forward into supporting new clients and new applications beyond basic IM.

Over the past few months, I’ve been running this service on alpha for the public. The intent of being alpha was to test the software, and to find out what uptake such service would have. The results are mixed, and I wanted to share them with you.

Our uptake, with absolutely no publicity, has been fantastic. As we speak there are 15 concurrent users chatting using onesoup, and over the last month alone we’ve seen over 200 new users come to onesoup. This is really great. But it’s far from impressive. Yamigo, which was the first free IMPS service provider, built over the years a registered customer base of close to 150,000 users. This may sound a lot, but actually, it’s not.

There are more than 150 million handsets with a built-in IMPS client on them, yet only 0.01% of those are used for chat. That’s a very low penetration rate. There are a few possible explanations for this, operators banning access, expensive data plans, and general lack of consumer appetite for mobile IM — I can list a few reasons why that would be the case, but mainly it’s around the inherent difficulty in maintaining an active communication, and the bad state of the clients –.

The built-in IMPS clients are simply not comparable to what one can get today either from a network provider, e.g. Yahoo! Go, nor aggregators like Fring. Additionally, “features” such as use of XML vs WBXML, lack of support for HTTP CIR bindings, limits in the number of contacts in your roster, and lack of presence integration with the address book, have made the IMPS clients fairly useless and unpopular.

I am convinced that IMPS, Wireless Village, is dead: consumers with IMPS clients do not have a data plan, operators don’t like IMPS, and the native IMPS clients are extremely limiting. This is obvious to me now, but I am sure it has been to the manufacturers for a while. The IMPS v1.3 standard is sort of in a limbo state, and most development is going into other standards, e.g. SIP, XMPP … Manufacturers are not betting on IMPS any longer, and neither is OMA. It’s time to move on, and call it a day.

I strongly believe in mobile communications and that onesoup could provide a great service for users. I have been thinking over the last few months how to evolve onesoup forward. Here’s a few decisions so far:

• I will not write a generic native chat client. I believe this space is now well served with Fring.
• I will not write a generic web chat client. There are many options out there.
• I will not create yet another mobile social network. Nough’ said.
• I will use XMPP.
• I will use geo-location.

So, what could be in the works? The intent is to work on a geo-aware XMPP service. A few ideas so far as of how to execute this:

• Allow connectivity to any XMPP service, and add geo-location extensions on top, i.e. onesoup provides the client, and a BOSH HTTP proxy. This is the closest to the current onesoup service, but on XMPP instead of IMPS.
• Mobile-enable an already established partner, i.e. onesoup provides the client, the BOSH HTTP proxy, and the XMPP servers. The partner provides roster, profiles and authentication.
• Focus on a particular vertical, rather than service general IM needs. Something in my mind is location based mobile dating or games. Again, a partnership with an existing player would be certainly required, or some other means of seeding the system.

What are your thoughts? What would you like to do with mobile messaging?

all services are back to normal now. we only had 60 minutes of downtime, basically for the DNS records to take over. thank you for patience.

heads up guys. we have a new server. the service will be down for the next 2 hours, while we move the service.

today we’ve had a couple of network issues around noon GMT with a faulty router. it’s now fixed and up and running. this is the kind of thing you get with hosting on DSL. but hopefully we’ll move to a new colocation facility within the next few days, where we’ll have redundant kit. for now, please bear with me. thank you guys for your support!

following along the lines of our previous gtalk release, we are now adding support for aim and icq. you can now chat using the embedded chat client in most Nokia, Sony-Ericsson, Siemens or BenQ phones.

behind the scenes in this release, we are also reducing the payload by a factor of 3 for most phones, and a few more phones should be supported now. additionally, not only gtalk, but any jabber account should be supported right now (e.g. myjid@jabber.mobi).

chat client setup instructions here.

again, please give us feedback through the forum.

one of the key virtues of wireless village (aka IMPS) is that it does not require setting up and keeping a TCP/IP connection open throughout a session. IMPS data channel is enabled through HTTP, and as such it does not require to keep the connection open. this is particular important for IM clients running on constraint environments, such as mobile. this is even more important if the server is outside of the fenced operator network.

the issue however is that HTTP is a client-initiated synchronous model, and the only way to ensure responsiveness is to continuously have the client poll for new messages that may be sitting for delivery at the server. with polling comes an implementation trade-off, either the client polls frequently, to ensure quick delivery of messages, which consumes bandwidth, or the client polls less frequently, which saves bandwidth, but delays the delivery of messages (no more “instant” messaging).

the way IMPS solves the issue is by adding a client initiation request (CIR) channel, in addition to the data channel based on HTTP. with the CIR channel, the server notifies the client that it must poll through HTTP for a message. unfortunately, the methods specified within the CIR channel are not really very useful outside of an operator’s network. they are either based on TCP/IP (which defeats the benefit of HTTP) or UDP (standalone or WAP). there is also SMS available, which is clearly only an option for operators, if nothing else, based on cost reasons.

so, in most cases, if the IMPS service access provider for the IM service is outside of the operator’s network it’s very likely that CIR / push will not be available and the client forced to poll regularly. so much for not keeping sockets open …

now, on the IMPS 1.3 specification, still in draft status, OMA adds another CIR transport, based on polling using HTTP. unfortunately the specification draft, not wanting to talk about implementation, misses the real benefit, and keeps talking about polling, which is a pity since i believe this could completely make the TCP/IP transport irrelevant. in reality what OMA has just done is adding Comet or more precisely the BOSH technique of XMPP to IMPS.

the idea is simple: have the client make an HTTP request and have the server hold the request on the queue until there is a message to push. if the TCP/IP connection is lost or times-out, the client will make a new HTTP request, and the delay will only be the round trip time for a request/response pair. the specification unfortunately only talks about the request, and does not mention anything about holding the request. quite the opposite, the specification keeps talking about polling, which is simply wrong and does not solve anything.

this model effectively requires two TCP/IP connections for HTTP, client to server, though none permanent and hence mobile friendly (and also, RFC 2616 recommends only 2 connections to be open at any point in time). one connection is used by the client to send or request information to the server. the other connection is effectively used by the server to push to the client a CIR, where the client will then use the other channel to poll and get the messages

this model requires less bandwidth than polling, provides immediate notifications, and it’s mobile network friendly. there is a trade-off in maintaining the HTTP requests pending on the server side since that consumes server resources, but nothing that can’t be fixed with a separate HTTP server forcing network queueing for the CIR channel. and given that this mechanism is always available, it even makes me wonder whether UDP would be useful any longer, at least considering that it is not reliable.

unfortunately, this is all currently useless for IMPS, since there are barely any 1.3 clients. besides IMPS is now considered by OMA as a legacy messaging protocol, alongside SMS and MMS

anyway, beyond IMPS, this is in all cases a good technique for environments where sockets cannot be open or ensured to be kept open. it also makes server implementation much more simple, since you are now only talking about plain HTTP and there is nothing about having your own daemon running. i very much like the idea and might use it to implement an iPhone client.

i’d have loved to get this out earlier but, finally, here it is. drumroll!! a wireless village server for the embedded chat client in most modern nokia, sony-ericsson, siemens and benq phones. i’ve been working hard to make it as mobile-network friendly as possible, looking at things from transfer volumes, to battery life. but foremost, usability.

if you have a supported phone, you can get gtalk on your cell phone by simply pointing to our IMPS servers as described in the gtalk connection page. you will need to already have a working packet data connection (EDGE/GPRS/3G/UMTS/…). note that not everything you would expect works just yet, but at least you will be able to login, fetch your gtalk/gmail contacts, chat, change your presence and get status updates from your buddies.

on thing that i have tried to keep very visible is the traffic volumes involved. you can see what to expect in our help page. i am also working to provide a simple web interface where you will be able to see your access history, connection time, and transferred data volumes.

now, be friendly to the service. it’s just out of alpha. expect a few interruptions as i build up the infrastructure and get new servers. i’ll blog about it anyway to let you guys know about any upcoming support window.

but, for now, enjoy gtalk on your cell phone, and let others know what your experiences are at the forum.