organic logo

March 17th, 2011

The Next Wave in Social Content Aggregation

Author

Todd Drake
VP, Technology

Twitter @threeminds

There are a range of currently available apps, loosely termed “social readers” that take the streams of content coming from your Twitter stream RSS feeds, Facebook posts and other random sources, pull them all together and try to make it more relevant for you. There are a couple of different approaches being used to build these social content applications but first, let’s look at a general framework for what they are about.

One way to view intelligent content is by how, as it moves from source to user, it becomes integrated into an application, correlated and aggregated with other content, augmented with richer information and metadata and then filtered for delivery. The smarter we can make each layer, the richer the experience and navigation we can provide to the user.

Given this, how is the current crop of apps trying to solve this problem?

One approach, social aggregation, is to simply beautify the delivery. Apps like Flipboard and Flud take your choice of streams and data sources, dereference the pictures and shortened URLs to get at the original content, and render it in a beautiful, useful form that you can navigate with a richer set of gestures and tools. They strive to remove the “sourciness” and the mechanics behind aggregation and scraping and present the content in a seamless, fluid fashion. Any curation is done beforehand by defining “channels,” which are almost branded sets of data sources. It’s a vast leap up from my RSS reader, but I still have to wade through all the data from the sources. Personally, I have a lot of sources so it’s a lot of wading. How do I find the nuggets of good in all that water?

A second, more interesting approach called social curation, is to try to add more intelligence to the aggregation layer, by applying relevance algorithms to the content, hoping to promote content that is meaningful. Usually the algorithms use social usage data and are very similar in approach to social search. It’s a hard problem for a lot of reasons, the largest of which is that everyone has different definitions of “meaningful,” and those definitions are often dependent on the user’s context. Some people like Reddit, others like Digg, still others like Amazon’s recommendation system – but in different contexts. Digg is great for finding strange new outrages and Amazon for finding excellent t-shirts.

An app announced at SXSWi 2011, Zite, is adding content analytics to the mix. Based on research from the University of British Columbia’s Laboratory for Computational Intelligence, it uses a combination of semantic- and statistically based machine learning to analyze the content of your feeds, understand how you interact (or ignore) stories and start to show you more of the stories you’re interested in. The addition of content analytics fundamentally changes the equation by being able to relate and group content based on the content itself, rather than how it is used or distributed (or tweeted).

Having built one of these types of machine learning systems waaaaay back, the problem that you face is that humans are annoyingly prone to change. Something they thought interesting for a week is no longer interesting and it takes time for the algorithm to believe you and change. This is why after a holiday season buying toys, Amazon thinks you’re all about Legos for a while. Still, it’s great to see someone bring content analytics into the mix. Zite is a brilliant entry into the field with a lot of traction and I’m hoping they keep tweaking the algorithms.

With BroadFeed, we’re taking a hybrid, two-tier approach, adding navigational and filtering tools on-top of a social curation approach within your stream. The algorithm that we’re using to weight content is pretty simple at this point – links that people you follow are retweeting a lot. The more a link is retweeted, the more we think it’s interesting for you. On top of this, we add a lot of navigation and filtering tools that allow you to both change the display of the content and only show the pictures that have been shared, as well as to control the window of time that the algorithm runs over.

BroadFeed is a great test bed for us to try out various relevance algorithms for content, as well as to gather observations about how users interact with stream content on the iPad. Future extensions might include adding, like Zite, content analytics and observation data; adding augmentation of individual stories with related Linked Data from providers like DayLife , the Guardian, even FourSquare or curation data from DataSift; taking feedback from content interactions and simply improving the algorithm using social velocity algorithms we’ve developed in our MI group.

It’s a pretty exciting space and there are lots to do and explore to make it better. Try out BroadFeed and let us know what you think.

0 icon: comments 0 icon: connections + Share

Add to the Conversation