There are a plethora of social media monitoring tools out there now. As they jostle and tousle with each other tantalizing potential clients with colorful charts, demos and trial accounts, I’m still left wondering where this technology will wind up. Will it go the way of the search engines and eventually wind up with just a few mega players? (OK, really one.)

I first started looking at social monitoring tools back in November of 2007. Since then there have been huge advances in the capabilities included in these platforms and some level of convergence in the capabilities in the top players.

Next week I will be representing Organic on an OMMA Social panel called “Choosing a Social Monitoring Tool: How to Find the Right Fit.” At Organic we are are continually reviewing tools, and currently using Sysomos. In the review of numerous tools, I’ve created my dream list for a social monitoring tool:

1. Easy self-service set up of topics and queries.
2. Efficient filtering of topics.
3. Delivery of results in real-time.
4. Sentiment analysis.
5. Trending Data.
6. Comparison to competitive information.
7. Identification of conversation “themes” around a topic.
8. Identification of Influencers or “influential posts.”
9. Demographic Information.
10. Respond to and track engagement.

But what exactly does that top 10 dream list mean?

1. Easy self-service set up of topics and queries. Some platforms require that you submit a “brief” to their customer service team to set up a new topic. Even with the most responsive of teams, this can mean a clunky time lag in getting your hands on the data. The best tools are so intuitively built that you barely need training to figure out how to set up and refine a topic. Most of us have become fairly savvy searchers from using traditional search engines, so social monitoring tools should really leverage this resident knowledge. When was the last time you needed Google training?

Beyond the set up of queries is the easy, dynamic ability to create and export analysis charts and graphs. Unfortunately, many tools have fancy-pants interfaces that do cool stuff, but need a manual to figure out how to use or get to them. In this point in the digital evolution, social media monitoring tools should not be excused from designing intuitive interfaces just because they are part of a new frontier.

2. Efficient filtering of topics. This probably seems stunningly obvious, but often it is still poorly done. The ideal tool is a combination of both depth and breadth–it scans all possible media, yet returns only the most relevant results. Recently a review comparing Scout Labs and Radian6 noted the variations in the search results the two companies returned. One provided more depth (Radian6) while the other did a better job of searching video and images (Scout Labs).

My ideal company scans the full breadth of the web–this should include the ability to turn on or off traditional media as well as include social media. Social media types should include the usual suspects: blogs, message board and forums, microblogging, wikis, social networks (the public areas), video and photo-sharing sites.

When it comes to video, note most companies are only able to crawl the tags of the video to determine relevance. Last year, I saw this ability to “read” video in a proprietary agency product I reviewed. I am looking forward to when the ability to scan actual video content is more widely available.

3. Delivery of results in real-time. A vendor who will remain nameless mentioned to me it took two weeks to a month to set up a new topic and return results (!) and I have also seen immediate set up, but a week for results to “fully populate.” Or there are the services that hang for minutes, even hours if the request is for a broad topic (say “beer.”) In my dream world, all results show up in seconds. Period.

4. Sentiment analysis. Scoring of positive and negative comment sentiment can get a bit controversial. Some companies do this manually, some automate and still others use some combination of the two. This is definitely is getting better, but in my estimation still needs a lot of refinement. The best automated rates for sentiment accuracy I have heard are 72%, but most companies hover in the low 60s.

Manual methods are more accurate, but of course take more time to return results. Automated results are delivered instantaneously, but the inaccuracy can turn into a lot of wasted time spent time reviewing mentions that are not relevant. The hybrid method employs an automated engine that then is corrected manually and thus “learns” to provide results that are more accurate. This “learning” model is probably the most accurate, but is most effective for ongoing monitoring (versus quick research) since the initial results can be off until the algorithm is refined.

5. Trending Data. This applies to both the topic results (number of mentions) as well as sentiment data. The ability to look at the sentiment and mentions over a period of time puts more context around a point in time snapshot. The ability to do live drill-downs into trending charts (all the way to individual mentions if you want) is a huge bonus. This can be critical in trying to determine cause and effect of other factors–like media campaigns, product announcements or scandals.

Which leads me to a related topic–historical data. Until recently, many platforms couldn’t give more than about 30 days worth of retroactive data. Now I’m starting to see claims of 90 days and even years. If you are talking to monitoring vendors, definitely ask how far back their data goes–and test it out.

6. Comparison to competitive information. Trending data is great. It’s even more great when I can look at the same data set for a competitive brand, product or company. When we speak to marketing managers the question is always “how are we doing in the social space?” Raw numbers or trends alone never seem to satisfy the thirst for the answer. Comparison to a rival is what really seems to have resonance. This functionality of course does not always have to have a competitive spirit; the ability to compare two issues or topics to see if their trends correlate can also be valuable.

7. Identification of conversation “themes” around a topic. This can unearth some really cool stuff. Typical methodology is the monitoring tool crawls the topic results, surfacing themes in the conversation you might not have noticed with the naked eye. For one credit card client, it was discovered that the majority of the “buzz” around their brand was driven by their sports sponsorship. It led to a recommendation that they should be doing more to leverage that natural conversation level. Without this cross indexing functionality, this insight would not likely have been discovered.

8. Identification of Influencers or “influential posts.” Every monitoring company has its own algorithm how they define “Influencer” and “influential.” For Influencers, it is usually some combination of relevance of their content to the topic, number of comments around the topic, number of people who view/comment on the content and the number of people who have linked to the content. A similar combination can be used for an influential post–with more focus on the number of views, comments and links related to the post. When talking to companies, I suggest pressing them for how they come up with these rankings and make sure it resonates with how you would approach it yourself.

I love “top ten” Influencers and influential posts. Being able to pinpoint Influencers can be a critical part a broader social media strategy. So the ability to easily find and target them is useful. “Influential posts” are an important addition, because in the new world order of UGC, often it is not just the “Influencers” who create the “influential posts.”

9. Demographic Information. This is a welcome addition that I didn’t see much of when social monitoring tools first came out. This provides the ability to slice and dice your mentions by demographic information like age, gender or even region of the country. How interesting for us to discover that most of the conversation on the web around a sensitive female condition was being conducted by young, male doctors! It led to a recommendation to consider the medical community in the social strategy.

10. Respond to and track engagement. Early on it was only the tools linked to PR agencies, but now it is common for social media monitoring tools to offer the ability to respond directly to comments, track and record responses, or even assign follow-up responsibilities to colleagues. This workflow management capability can be the key to turning a social strategy that is initially just “listening” into one that actively responds to and engages consumers.

The “community manager” role is becoming a growing need for many of our clients. This key player has the responsibility not just for our client’s on site “community” but also seeks out and responds to relevant conversations across the web. The ability to track, quantify and review responses in this uncharted territory is key to tracking the success of this new media role.

I have intentionally stayed away from endorsing (or trashing) any particular tool. But I would be remiss without mentioning some of the players I have reviewed along the way: NetworkedInsights, Radian6, CollectiveIntellect, Scout Labs, Buzzient, and TruCast. I’ve also had the chance to peak at some of the agency proprietary tools like Waggoner Edstrom’s Narrative Networks, FutureGroup’s PropheSEE. More recently I’ve been introduced to Techrigy, My BuzzMetrics and of course Sysomos.

This is by no means a comprehensive list. The landscape is constantly changing as evidenced by an announcement by PR Newswire earlier this month that they are launching Social Media Metrics a tool that “enables communications professionals and marketers to monitor, analyze and measure the impact of what is being said about an organization, brand, spokesperson or competitor across the social media landscape.” Sounds like what we’re all trying to do.

Stay tuned for an updated post after the panel discussion next week.

Anna Banks