This is Spinn3r's offficial weblog where we discuss new product direction, feature releases, and all our cool news.

Spinn3r is a web service for indexing the blogosphere. We provide raw access to every blog post being published - in real time. We provide the data and you can focus on building your application / mashup.

Spinn3r handles all the difficult tasks of running a spider/crawler including spam prevention, language categorization, ping indexing, and trust ranking.

If you'd like to read more about Spinn3r you could read our Founder's blog or check out Tailrank - our memetracker.

Spinn3r is proudly hosted by ServerBeach.


September 2009
July 2009
June 2009
May 2009
April 2009
February 2009
January 2009
December 2008
October 2008
September 2008

Spinn3r and Social Network Data Portability

200801101243There's been a lot of talk recently about social network data portability with Plaxo, Facebook, and Google now having employees as members of the group.

From Spinn3r's perspective, it's not just about data portability, it's about a fully open social graph.

By open, I mean no restrictions other than copyright and plenty of fair use for public data (private data is another issue altogether which quickly becomes a lot more complicated).

The blogosphere has really paved the way for this with its history of open data thanks to RSS and Atom.

MySpace should be commended for their participation in the blogosphere with their blogging system. They send pings, have RSS feeds, and don't mind that we crawl and build applications on top of their data.

There are certain hosted blogging systems (who shall remain nameless) which, while fully open, have additional restrictions for crawlers. They only allow a finite number of requests to their system. The number is so low that it's mathematically impossible to crawl all their content.

Now, it's their system, they have the right to do what they want and provide access under whatever restrictions they deem fit. However, it's the user's data - not theirs. We don't have any obligation to use their system and customers are going to flock to systems which are more open and have more compelling applications.

Don't believe me? It's not altruism - it's the free market. Users are going to flock to systems with vibrant and compelling applications.

The open content thanks to the blogosphere has brought us companies like Bloglines, Tailrank, Google Reader, Kosmix, Zvents, Powerset - I could go on.

I remember this the other day when I was reading VentureBeat's coverage of Friendfeed and the irony of the fact that Facebook Feeds aren't actually RSS feeds.

This open data is becoming more and more valuable - not just to the company writing the applications that create the open data but to the entire ecosystem. So valuable in fact that NewsGator decided to release all of their applications available for free because they can sell backend appliances that index the data and build compelling applications.

This needs to be solved not from the perspective of user portability but from that of an open content network where all players have equal access to the data.


Matt Harwood

Quote: "MySpace .... don't mind that we crawl and build applications on top of their data."

Maybe I am not current with MySpace's policies (they became defunct to me quite some time ago), but I distinctly remember MySpace blocking external "widgets" and services one after the other?

Nice point overall, though!

The comments to this entry are closed.