Scaling a Microblogging Service – Part II

When discussing microblogging scalability, the conversation includes scaling each individual service, but also scaling the network and relationship between services. Part I discussed the challenges of scaling a single microblogging site with focus on dealing with a large and constantly changing content database. In that post I mentioned that the proposal by some critics to build a distributed or federated microblogging service as a scaling solution will actually make thing worse. This second part will elaborate on that claim.

When discussing a distributed microblogging service, the conversation touches the long debate on the future of social networks and linking communities across individual walled gardens. After all, microblogging is one aspect of the social web, and status updates lives side by side sharing photos, videos, and other personal information and experiences. Being able to choose a social network and make friends from another without having to sign up for multiple accounts is one of the visions being offered. Another is the approach being advocated by the Data Portability group, which focuses on being able to move an entire experience off to another network instead, creating multiple identities.

Continue reading

Scaling a Microblogging Service – Part I

When it comes to Twitter, everyone’s a critic.

The irony is, the majority of the technical criticism written about Twitter reveals more about the lack of understanding of the author than anything about Twitter. Creating Nouncer – a developer platform for building microblogs and similar services – provides a firsthand understanding of the inner-working and challenges of microblogging sites. This post is not specifically about Twitter but as the leading microblogging service, it is the best working example of the challenges of scaling in the space.

People don’t seem to get what is so hard about scaling the Twitter service. Some think it has to do with Rails, while others look at the hosting company or other operating factors. Then there are those who incorrectly think a distributed or federated service is the solution (which will only make things worse). The idea that building a large scale web application is trivial or a solved problem is simply ridiculous. In a way it is surprising that there are so few companies out there dealing with commoditizing web developing scaling.

Continue reading

In support of Fragmentation

There seems to be some correlation between Twitter and Pownce usage lately. Looking at Alexa for the past month, the two have similar trends of up and down reach. It’s possible the time period and sample is just too small to reach any conclusions but it might also be that there is a great deal of overlap between the two services, which makes sense. It also seems to show that Pownce has not taken users away from Twitter as the modest decline started before Pownce launched.

Continue reading