Spinn3r 3.1 - Now with Twitter Support and Social Media Ranking

Spinn3r 3.1 just went live today and we're announcing two new features.

Twitter Firehose Support

Spinn3r listens to a new Twitter firehose API which is a sample of the full Twitter feed.

All Twitter content is classified with a new MICROBLOG publisher type which we will be using for Twitter, Identi.ca, Jaiku, and other Microblog systems.

Further, it is language classified (using an algorithm we have developed) and includes all metadata from Twitter including publication time, author name, handle, etc.

All content is real time and indexed and available within Spinn3r a few moments after it is published.

There were a lot of requests for this feature (Social Media is HOT) so I expect a lot of innovation from our customers.

Technically this is still in beta but we feel it's ready for use in production applications (once we get some feedback from our users).

Here's the current breakdown of Twitter vs other social media and blog content in Spinn3r. While Twitter is larger it's important to realize this is much less content since Twitter posts are short.

Social Media Rank

We also published some of our new ranking technology which has been in development for a while now (more than a year).

We're indexing social media sites and computing rank on users based on their social graph.

The results are pretty interesting. Scobleizer OWNS Friendfeed. Techcrunch consistently places high. They're #3 on Friendfeed. #274 on Digg and #24 on Twitter.

It's also interesting how the founders of these social media properties consistently place high, even over celebrities. Ev Williams is still #5 on Twitter. Kevin Rose is still #1 on Digg. Paul is #16 on Friendfeed.

Sources (or nodes) are ranked by authority whereby the more friends or inbound links you have the higher your rank.

Our key differentiator is that we do not consider raw inbound link count to be an accurate representation of authority. This is highly vulnerable to spam and rank errors as users who attract a large number of links (either through black hat methods, link baiting, or viral marketing) can inflate their rankings (and harm other legitimate users).

We consider the quality of inbound links to be far more important. You can observe this in our results as the authority for a source is not a direct function of raw inbounds links. Some users can have high authority but very few (relative) inbound links.

We're really eager for feedback here. If you have any comments on our ranking system feel free to contact us with your thoughts.

Comments

[Robert Scoble]

One problem with Twitter is that the team has "picked" people to rank high by putting them on the suggested user list. That makes these kinds of comparisons difficult to do. For instance @ev had far far fewer followers than I did before he put himself on that list. Now he ranks far far ahead of me. Funny how founders have the power to make themselves popular even when they didn't earn it.

[Kevin Burton]

Hey Robert.

That's a good point.. There was discussion of this back in the days of Userland where Winer would put people on RSS distribution lists ...

I could dampen these users in Twitter but that might not be a good either.

They users DO have authority. Perhaps I could just NOTE this in the rankings....

Kevin

[Robert Scoble]

Kevin: yeah, there's no good way to really talk about this. I do wish that systems like yours could tell us when these people were on the SUL (iJustine, for instance, used to be on the list but isn't anymore and she isn't seeing any follower growth anymore).

[Kevin Burton]

We could post the long term historical growth of the user.

This way you could see how they rise and fall...

This way you could at LEAST see spikes.

If you had raw inbound links you could also see where the traffic came from.

This isn't too hard for us to do but I want to make sure this is valuable to people first :)

Kevin

[Susan Beebe]

FriendFeed rocks - thanks for adding me to your list! I'm #143

The twitter Suggested User List (SUL) is in need of a major overhaul. They need to possibly adopt a similar model as FriendFeed has built so that users are presented with an algorithm based on FOAF user data; not just celebrities or who ever is popular with the twitter management team that day.

Cheers,

http://susanbeebe.com
http://FriendFeed.com/SusanBeebe
http://twitter.com/SusanBeebe