real-time – Stay N Alive

Buzz Opens the Firehose With New API Features

Just a few minutes ago, on his Buzz update stream, Google employee DeWitt Clinton announced that Google had opened up their real-time stream of information for Buzz. Now, any application can access, in real-time, all updates across the entire service as they come through. This incredible stream of information will serve useful to Data warehousing and collection apps such as PeopleBrowsr and others that provide images and statistics surrounding the real time streams of Social Networks.

In his update, Clinton describes that the entire API was built around Pubsub Hubbub (PSHB) by Brett Slakin and John Panzer, and also integrates Atom and ActivityStreams standards as part of the integration.  In addition, new API methods to retrieve all the comments and likes of any user were also added to the Buzz API.

Even more significant is the addition of Share counts for any given URL.  Now you can expect to see services such as TweetMeme provide widgets that show and track the number of times that URL has been shared on Buzz and the ability to click to share even further.

In what seems like a long lull of time since any updates on the Buzz API or service, these changes are refreshing.  I think there’s no doubt the Buzz team is working hard to make this platform special and we’ll continue to see results in the near future.

What other API features would you like to see from Buzz and its API?

Speculation: Expect Something BIG in the Area of Real-Time at F8

I don’t do speculative posts like this too often, except around Facebook’s F8 developer events for the most part.  The last one I predicted was that Facebook would announce a Mobile Platform at F8 – the announcement did occur along with Facebook Connect.  The first F8 was the announcement of the Facebook API, which revolutionized Social Development and has left players like Google scrambling to play catch up since.  Now, 2 years since the last F8, the next F8 has been announced, and we are all wondering what the next big announcement will be.  If it is to be in line with the last 2, and, considering they waited 2 years to have another one, they have to be announcing something game-changing.  I predict it directly involves some of the FriendFeed team and it’s directly related to real-time.

First of all, let me preface this with the fact that I am not receiving this data from any inside contacts at Facebook, nor have I been told anything the rest of the world doesn’t already know.  This is pure speculation – I hope it’s taken as such.  I am also certainly not a psychic.  I think if you look at some of the hints though, you can see the potential for something big, perhaps FriendFeed 3.0-like (remember, FriendFeed 2.0 was the advent of their real-time stream you see now) about to happen at Facebook.  Here are my reasons for thinking such:

What is Paul Buchheit Working on?

Paul Buchheit, one of the founders of FriendFeed, creator of Gmail, and now working at Facebook after FriendFeed was acquired, hasn’t yet made it evident exactly what he’s working on.  We know Bret Taylor, also a founder, is now Director of Product for Facebook, and working heavily with the Facebook APIs and the new Roadmap Facebook has laid out for developers.  We know Kevin Fox, pretty much the man behind all the design of FriendFeed, has been working on the new Games and Apps dashboard that Facebook just launched (that you can see on the left-hand side of Facebook).

But what is Paul Buchheit working on?  He recently commented stating he is definitely not working on Facebook’s new e-mail product that they have been rumored to be working on to replace their current inbox structure.  I’m not sure anyone has specifically stated exactly whether he’s working on the Facebook developer platform now or not.  He seems to be doing something big, and he’s certainly been studying Google Buzz recently if you look over his FriendFeed stream lately.

Float Like a Butterfly, Sting Like a Bee

Then there’s that “Butterfly” post.  Paul Buchheit specifically stated when Robert Scoble, Steve Gillmor, and others were all pressuring Facebook to make a statement on what they were going to do with FriendFeed that “the team is working on a couple of longer-term projects that will help bring FriendFeedy goodness to the larger world.”  He then continued, “Transformation is not the end. Consider this the chrysalis stage — if all goes well, a beautiful butterfly will emerge”.

The mystery in all this is that Facebook has not yet released anything even remotely similar to what Paul described yet.  Paul’s a really smart guy.  He’s not just going to work on something mediocre for Facebook – whatever it is, it has to be game changing.  I really believe that whatever it is will blow our minds away when it happens.  The FriendFeed team doesn’t just innovate.  They revolutionize.  I don’t believe they would still be at Facebook if they didn’t have that opportunity.

Facebook’s Needs

Then there’s the lack of any real-time APIs or architecture at Facebook.  I have to click on the page to have it refresh.  Frankly, I think that fits their current audience of 400 million+ “average Joes” well.  It doesn’t tap into the news-seeking, data-mining, and publishing audiences very well though.  That’s what Twitter does well.  It’s what FriendFeed and Buzz also do well.  All of these come with real-time APIs and real-time searches (or “track”).

Facebook needs a real-time interface for developers still.  It needs search.  It needs search to be real-time.  It needs a public view into all of that, supported by the powerful privacy controls Facebook already has in place.  Facebook has already built out the building blocks to launch this with their recent emphasis on encouraging users to open up their posts more and at the same time enabling them to have granularity in who sees those posts.  The next natural step is to finally open up those public posts to developers, and provide a real-time interface to it all.  With a 400 million user audience, that would be game-changing in the realm of real-time data.  We ain’t seen nothin’ yet!

The Lack of any Really Big Known Announcements at F8

Lastly, we know everything else Facebook could announce at F8.  Facebook has already started rolling out credits to developers, so a payment system wouldn’t be much of a “game changer” per se.  I’m sure they’ll talk a lot about it at the conference though.  Facebook has already started rolling out its Ads API to developers.  They’ve already announced the desire to open up websites as virtual 3rd-party “Pages” on the web.  They’ve let us know just about everything in their roadmap, except the fate of FriendFeed.

Doesn’t this seem strange to you that Facebook and the FriendFeed team have been so mute on this in general for almost a year now?  What’s going on behind the scenes?  Even when asking the FriendFeed team about plans to integrate better into Facebook they have remained mute.  How cool would it be if, while everyone is ranting and raving about Google Buzz and calling FriendFeed dead, the FriendFeed team along with the incredible talent that Facebook adds to the mix have all been working on FriendFeed 3.0 behind the scenes?  What if Facebook caught wind Google was working on Buzz and bought FriendFeed in response to that rumor?  Will the “Butterfly” emerge at F8?  The chrysalis stage takes patience – I’m not giving up on FriendFeed yet.

FriendFeed Turns on the Twitter Firehose (Again)

friendfeed-logo.jpgIt seems that some time today, the FriendFeed team has just re-enabled their live Twitter stream (using Twitter’s “Birddog” API) for real-time updates from Twitter.  I noticed the update when posting a cool bookmarklet by Kynetx, and re-tested it again – sure enough the update to Twitter hit FriendFeed almost immediately after I posted it to Twitter.  Looking over FriendFeed, I learned that Paul Buchheit, one of FriendFeed’s founders now working for Facebook, confirmed this earlier today.

Long before many were embracing Twitter’s real-time stream, FriendFeed was one of the first Real-time Twitter stream consumers to take advantage of the platform.  Shortly after the Facebook acquisition the FriendFeed team turned off the real-time updates, others speculating that it was the beginning of the end for FriendFeed.  FriendFeed’s Paul Buchheit assured users that the FriendFeed team was simply working out details with the Facebook lawyers to ensure the real-time stream met up with Facebook’s stringent legal policies.  Others remained skeptical.

Tonight it appears they’ve turned that live stream on for good, and boy is it fast!  FriendFeed continues to remain one of the most powerful Twitter clients and Social Management tools out there.  I think this continues to prove that FriendFeed will continue to improve even after the Facebook acquisition.

If you’re not yet, you can follow me on FriendFeed at http://friendfeed.com/jessestay.

LazyFeed Becomes First Real-Time Web Aggregator for rssCloud

Lazyfeed LogoToday Lazyfeed announced they had officially launched rssCloud and Pub/sub Hubbub (PSHB) support into their real-time RSS aggregator, making them the first major aggregator for rssCloud outside of Dave Winer’s own River2 client, and the first client of its type for the Web (Dave Winer’s client is written for the desktop OS).  What does this mean? It means now you have a way to get the most relevant information you are looking for real-time, as it happens.

Lazyfeed is a new aggregation service that aims to provide real-time news updates on specific topics you want to know about.  You give it the keywords you’re interested in, and it comes back, as the news happens, with the news written about those keywords.  It goes further though and provides additional suggestions for other keywords you might be interested in as they happen, and you can add those to your list as well.  See Louis Gray’s demo here for a great view of how it works.

Now, with rssCloud and PSHB support for real-time news aggregation, they are now one of the most real-time aggregators on the web.  On their blog they mentioned some of the hurdles they had to jump to get through the implementation, and ironically, Feedburner seemed to have the biggest issues with set up (through PSHB) since the Atom protocol wasn’t built natively with any sort of real-time support. No problems were mentioned about rssCloud, showing promise for the protocol developed by Dave Winer. Lazyfeed seemed to think Feedburner wasn’t even real time, based on their experience, showing a delay of a few minutes on each feed published.

Problems aside, seeing aggregators like Lazyfeed implement these technologies is promising, showing we are on the cusp of the 2010 web and real-time news and updates being at our fingertips.  I’ve talked to several other companies also getting ready to embrace these technologies and I’m pretty sure by the end of 2010 it will be an entirely new web and opportunity for entrepreneurs and developers alike.

UPDATE: Brett Slakin, one of the originators of the PSHB protocol, has clarified some of the PSHB and Feedburner issues here: http://blog.lazyfeed.com/2009/09/lazyfeed-rsscloud-pubsubhubbub-real.html#comment-16898143

FriendFeed Opens Up the Firehose to Developers

friendfeed-logo.jpgFriendFeed seems to be staying one (or two or three) step(s) ahead of Twitter in everything they do. Today FriendFeed released their real-time stream of data in beta to any and all developers wishing to write applications. Unlike Twitter, there is no application necessary, no NDA to sign, and all is controlled by simple OAuth. This also means users of FriendFeed-based applications will no longer need to get their special key to manually enter as was previously required.

The real-time stream is based on long-polling techniques to receive near-immediate updates of data from FriendFeed. With Long-polling, developers send a request to a given address, which the server holds open until data is ready for that request. The result is real-time data from the polled source, in this case FriendFeed. It is also less server-intensive as compared to the typical push updates similar to what Twitter is using for their /track and real-time streams, so in theory will scale better (and to me shows the maturity of the FriendFeed team as compared to Twitter’s).

In addition to their real-time stream, FriendFeed released an OAuth solution to developers, enabling users one-click access to the FriendFeed data stream for compatible apps using the platform. SocialToo, my service currently using the Twitter and Facebook platforms, will be using this authentication as well as we integrate FriendFeed into our environment. It will enable simple, one-click login and registration into our system, making it much easier for users to use socially-based applications.

My favorite addition is the integration of social graph data into the stream returned by FriendFeed. Previously, only the list of people a user subscribed to was available via the FriendFeed API. Now, both the list of those subscribed to, and those subscribed to a user are provided, enabling apps like my SocialToo to very soon be able to provide useful analytics around those following you on FriendFeed. Yes, this will also enable auto-follow and auto-unfollow (to keep out spammers) as well if users opt to do so.

Other features released in the API are the ability to upload almost any file attachment to a user’s FriendFeed stream, access to the powerful (and more than 140 character) direct message features of FriendFeed, sharing to multiple streams at once, and more. In addition, FriendFeed is returning the HTML for users and groups, so developers don’t have to differentiate between the two. Hopefully, this will also enable FriendFeed to maintain control of the API and, if you ask me, provide advertising and monetization opportunities via the API in the future as well, which Twitter has completely lost control over.

FriendFeed’s API has proven to have potential as a much more flexible option for developers than Twitter’s in the past, and I think they’re proving that with the new features. In addition to the features launched today, developers can also opt to customize the requests they send to FriendFeed, specifying query parameters about exactly what information they want to retrieve about users, allowing much smaller and much fewer requests to the platform. This is a welcome site as compared to the Twitter platform, which forces entire requests to pull information about a user and their friends, forcing much larger data requests, and higher costs for developers in the end.

FriendFeed is putting the pressure on Twitter with this release. My hope is that developers will see this, and try the platform out, giving Twitter more pressure to fix their own platform issues. If you haven’t tried it, today is the day for Social Platform developers to try FriendFeed’s API.

Reason #552 to Be on FriendFeed: Real-Time Search

friendfeedI’ve talked in the past about how I read your blogs. I rarely subscribe through Google Reader any more – I read all of your blogs through FriendFeed. Therefore if you want me to read your blog, I strongly suggest taking the first step of importing it into FriendFeed. Well, if that weren’t reason enough, FriendFeed just gave you even one more reason to import your blog and other social data into their site: real-time search.

If you’ll look down in the lower-right sidebar of this blog, you’ll see an example of it in action – every mention of “realtime”, “real time”, “social”, “friendfeed”, “twitter”, “facebook”, or “Jesse Stay” anywhere on the web, at any time that has been imported into FriendFeed now appears real-time, as it’s happening.  Go ahead – change the search parameters to something like “earthquake”, or “iran”, or “michael jackson”.  You’ll quickly see the value of having such real-time, on-demand search at your fingertips.

FriendFeed is said to be getting ready to also release notifications, probably in the same way they do your other friend lists and feeds via e-mail and IM for the various search terms you’re trying to find.  As the terms come in real-time, you’ll receive them.  This is powerful stuff!

Over a year ago Twitter had a similar feature – they called it “track”, which they’ve recently re-introduced to developers via their API.  It was the main reason I joined and stuck with Twitter.  As soon as their competition was dead they removed it.  It looks as though FriendFeed has one-upped Twitter once again with this feature.

So if you haven’t already, go to FriendFeed, get an account, import all your Twitter and Facebook friends already on the service (you’ll find most of your active friends probably already are!), and start adding your blogs, Twitter feeds, facebook feeds, photos, videos, and more into your stream so they too can be indexed by this powerful search.  The web just got a whole lot more real-time, and FriendFeed just got a whole lot more powerful.

Track is Back! Steve Gillmor Rejoices

Twitter TrackerOne of the major reasons I joined Twitter over a year ago was the ability to easily subscribe to Tweets, based on search terms I sent over my cell phone or IM client.  Due to scaling issues and load on Twitter as they grew, Twitter removed that ability shortly after Scoble and I visited them last year.  Twitter called it “Track”.

It was simple functionality.  I send “track keyword” to 40404 on my mobile phone, and immediately any Tweet with that keyword would be sent straight to my mobile phone (or IM client if I had it set correctly in my preferences).  It was very useful.  For instance, if I wanted to know every happening at the time going on at Sundance here in Utah, I would simply “track sundance” and all those Tweets would be delivered to me, as they happened.  I could do the same with my name, my brand, or my favorite technologies.

Twitter may have removed that functionality last year, but just over a week ago, they finally released API methods to make the real-time searching of keywords and search terms available again via their limited streaming APIs available in Alpha right now.  The API method is conveniently called “track”.  On June 12, John Kalucki, developer behind the real-time streaming APIs Twitter is now providing developers described it on the Twitter developers mailing list as such:

“The /track resource allows searching the Firehose stream for a list of keywords. This resource may be a useful adjunct to the Twitter Search API. While the predicates are less powerful than the SearchAPI, results are streamed continuously and with low-latency. For common keywords, a more complete set of results can be delivered than is possible by polling the Search API. Consult the Streaming API documentation for limits and details: http://apiwiki.twitter.com/Streaming-API-Documentation.”

Put in simple terms, developers now have access to take a list of keywords for a specific Twitter user, and have Twitter deliver any Tweet that meets those search terms in real-time back to the developer’s app.  That means Push notifications via iPhone applications on the new iPhone 3.0 firmware are now possible, any company utilizing SMS can easily deliver terms via SMS again, and more.

Because the Streaming APIs are in limited Alpha, only certain developers will be allowed access, but I fully expect to see this integrated in the near future.  I expect to see this especially as the code solidifies and moves out of Alpha, into apps such as TweetDeck, Tweetie, and Seesmic Desktop.  Twitter users everywhere can rejoice, as their most requested and most favorite feature taken from Twitter has just been found.

Twitter Tracker logo courtesy NBC Tonight Show.