Home - Stay N Alive

FriendFeed Opens Up the Firehose to Developers

Posted on July 20, 2009 by jessestay

FriendFeed seems to be staying one (or two or three) step(s) ahead of Twitter in everything they do. Today FriendFeed released their real-time stream of data in beta to any and all developers wishing to write applications. Unlike Twitter, there is no application necessary, no NDA to sign, and all is controlled by simple OAuth. This also means users of FriendFeed-based applications will no longer need to get their special key to manually enter as was previously required.

The real-time stream is based on long-polling techniques to receive near-immediate updates of data from FriendFeed. With Long-polling, developers send a request to a given address, which the server holds open until data is ready for that request. The result is real-time data from the polled source, in this case FriendFeed. It is also less server-intensive as compared to the typical push updates similar to what Twitter is using for their /track and real-time streams, so in theory will scale better (and to me shows the maturity of the FriendFeed team as compared to Twitter’s).

In addition to their real-time stream, FriendFeed released an OAuth solution to developers, enabling users one-click access to the FriendFeed data stream for compatible apps using the platform. SocialToo, my service currently using the Twitter and Facebook platforms, will be using this authentication as well as we integrate FriendFeed into our environment. It will enable simple, one-click login and registration into our system, making it much easier for users to use socially-based applications.

My favorite addition is the integration of social graph data into the stream returned by FriendFeed. Previously, only the list of people a user subscribed to was available via the FriendFeed API. Now, both the list of those subscribed to, and those subscribed to a user are provided, enabling apps like my SocialToo to very soon be able to provide useful analytics around those following you on FriendFeed. Yes, this will also enable auto-follow and auto-unfollow (to keep out spammers) as well if users opt to do so.

Other features released in the API are the ability to upload almost any file attachment to a user’s FriendFeed stream, access to the powerful (and more than 140 character) direct message features of FriendFeed, sharing to multiple streams at once, and more. In addition, FriendFeed is returning the HTML for users and groups, so developers don’t have to differentiate between the two. Hopefully, this will also enable FriendFeed to maintain control of the API and, if you ask me, provide advertising and monetization opportunities via the API in the future as well, which Twitter has completely lost control over.

FriendFeed’s API has proven to have potential as a much more flexible option for developers than Twitter’s in the past, and I think they’re proving that with the new features. In addition to the features launched today, developers can also opt to customize the requests they send to FriendFeed, specifying query parameters about exactly what information they want to retrieve about users, allowing much smaller and much fewer requests to the platform. This is a welcome site as compared to the Twitter platform, which forces entire requests to pull information about a user and their friends, forcing much larger data requests, and higher costs for developers in the end.

FriendFeed is putting the pressure on Twitter with this release. My hope is that developers will see this, and try the platform out, giving Twitter more pressure to fix their own platform issues. If you haven’t tried it, today is the day for Social Platform developers to try FriendFeed’s API.

With No Notice, Twitter Adds More Limits – Password Trouble Ensues

Posted on July 17, 2009 by jessestay

Twitter is up to their old antics of adding limits again, changing the API, and not telling developers as they do so. This morning Twitter released into production new limits around their verify_credentials() method in the API, only allowing users to verify their usernames and passwords through Twitter applications 15 times per hour. The problem is they didn’t tell any of the developers.

Sure enough, searching Twitter (the issues are intermittent), users are having password issues across the Twittersphere, wondering what is going on. It even affected my service, SocialToo, as we were using that method as a backup to verify users were indeed authenticated (and hence enabling us to notify them if they forgot to change their password with us). I e-mailed Twitter, and while very respectful as always, they seemed surprised at the issues we were having. When I asked if it had been announced anywhere they responded, “It wasn’t, no, because [we] assumed (apparently incorrectly) that people were only using this method occasionally.” There has still been no announcement by Twitter on the new limits.

Apparently, on June 29th, new text was added to the Developer API Wiki stating (regarding the verify_credentials() method in the API), “Because this method can be a vector for a brute force dictionary attack to determine a user’s password, it is limited to 15 requests per 60 minute period (starting from your first request).” The new limits don’t appear to have been put in place until this morning however, as that is when we noticed it at SocialToo.

So if you’re using the verify_credentials() method in your app, you may want to consider finding some other way to be sure your users are verified – I’m happy to announce it here. It now only takes a few runs by only a few apps to hit that limit for each user, and then users are stuck in the water until the next hour is up until apps begin to adapt to these new limits. That is why we’re seeing the issues across all of Twitter. According to Twitter, the best way is to look for a 401 response code returned in your API calls, as unauthenticated users will return as such when using the API. Twitter only suggests using verify_credentials() for new users. My conversation with Twitter ended with the suggestion from them, “Migrating to OAuth avoids the risk of a user changing her password, FWIW.”

FWIW, OAuth is still in beta and not yet suggested for use in Production. In their exact words, “For us, ‘beta’ really means ‘still in testing, not suitable for production use’.” In other words, use the Twitter API at your own risk.

You can follow the password problems as they happen in real-time on FriendFeed below:

http://friendfeed.com/search?q=password+service%3Atwitter&embed=1

Twitter Looking to Raise the Dead With Previous Tweets

Posted on July 13, 2009 by jessestay

One of the biggest complaints about Twitter is that it is only a “present-tense” service. To pull up a previous conversation or post, I either have to have favorited it, or it has to have been in the previous 3200 Tweets of a user. Anything beyond that disappears forever, or so it would seem. Twitter has previously said they are still archiving these old Tweets, giving comfort to some that maybe their conversations are not gone forever. Today Twitter gave further evidence to that, adding “Get the full archive of a user’s tweets” to their V2 platform roadmap on their developer Wiki. (under “Users”)

While the V2 Roadmap is not set in stone, nor is it intended as an announcement platform for Twitter, it does suggest that some time in the near future we may see access to your previous Tweets open for public consumption. It also suggests that it is something Twitter is currently working on, or has plans to be worked on.

One of the reasons I joined Twitter was that it could be used as a journaling service. It was a public way I could journal the little tidbits of life that perhaps, while insignificant to most, would enlighten and entertain generations to come that would like to learn more about me and my life. It lost that value however when I hit my limit of 3200 (or so) Tweets and could no longer retrieve my past posts, thoughts, and conversations. My hope is that Twitter releases this soon and I can again utilize Twitter as an archival, as well as communications platform.

Have an old conversation you just wish would go away? Looks like getting it off your stream may not be permanent after-all. Twitter seems to be getting ready to bring back those zombies again. Now where’s Buffy when you need her?

Track is Back! Steve Gillmor Rejoices

Posted on June 21, 2009 by jessestay

One of the major reasons I joined Twitter over a year ago was the ability to easily subscribe to Tweets, based on search terms I sent over my cell phone or IM client. Due to scaling issues and load on Twitter as they grew, Twitter removed that ability shortly after Scoble and I visited them last year. Twitter called it “Track”.

It was simple functionality. I send “track keyword” to 40404 on my mobile phone, and immediately any Tweet with that keyword would be sent straight to my mobile phone (or IM client if I had it set correctly in my preferences). It was very useful. For instance, if I wanted to know every happening at the time going on at Sundance here in Utah, I would simply “track sundance” and all those Tweets would be delivered to me, as they happened. I could do the same with my name, my brand, or my favorite technologies.

Twitter may have removed that functionality last year, but just over a week ago, they finally released API methods to make the real-time searching of keywords and search terms available again via their limited streaming APIs available in Alpha right now. The API method is conveniently called “track”. On June 12, John Kalucki, developer behind the real-time streaming APIs Twitter is now providing developers described it on the Twitter developers mailing list as such:

“The /track resource allows searching the Firehose stream for a list of keywords. This resource may be a useful adjunct to the Twitter Search API. While the predicates are less powerful than the SearchAPI, results are streamed continuously and with low-latency. For common keywords, a more complete set of results can be delivered than is possible by polling the Search API. Consult the Streaming API documentation for limits and details: http://apiwiki.twitter.com/Streaming-API-Documentation.”

Put in simple terms, developers now have access to take a list of keywords for a specific Twitter user, and have Twitter deliver any Tweet that meets those search terms in real-time back to the developer’s app. That means Push notifications via iPhone applications on the new iPhone 3.0 firmware are now possible, any company utilizing SMS can easily deliver terms via SMS again, and more.

Because the Streaming APIs are in limited Alpha, only certain developers will be allowed access, but I fully expect to see this integrated in the near future. I expect to see this especially as the code solidifies and moves out of Alpha, into apps such as TweetDeck, Tweetie, and Seesmic Desktop. Twitter users everywhere can rejoice, as their most requested and most favorite feature taken from Twitter has just been found.

Twitter Tracker logo courtesy NBC Tonight Show.

Twitter Declares Checkmate on Twitter Gamers

Posted on June 19, 2009 by jessestay

Just recently Twitter began the much-needed stance to suspend users practicing the controversial “Twitter Game” (as I call it). On the Twitter developers mailing list, Twitter developer Doug Williams told me yesterday that users who “use software to constantly churn followers in a repeated pattern of following and unfollowing will … risk suspension.” This new rule was confirmed to me by several other Twitter users that Twitter customer support is also confirming that this is happening.

The “Twitter Game” was the topic of a podcast I participated in a few months back with Lucretia Pruitt and Jim Turner hosting. The practice centers around following as many people as you can (up to the 1,000 users per day limit that Twitter has imposed), continuing such practice until the ratio of following 10-20% more than those following you is reached. Then users would unfollow the users that don’t follow them back. This practice increases the number of followers of any particular user, theoretically providing a much larger reach for the user or brand doing such. Other similar techniques involve using tools such as Twollo and Hummingbird to find new people to follow that might follow back, and share links under multiple accounts with those individuals.

With Twitter taking a stance against this practice, this means no longer will users be able to quickly inflate their numbers and get around the ratios Twitter has put in place to keep this “gaming” from happening. From users I have talked to, some over major brands, some are not happy. The threat of suspension will come welcome to those getting tired of meaningless followers however. While not a welcome sight to some, it is simply a change in “Game”. Users will need to adapt and use more effective methods of gaining new followers, such as maybe, providing good content? Regardless, I expect people to continue to find new ways to game the system. We’ll see if Twitter can win this “Rat Race”.

Curing Spam on Twitter With Better Follow Limits

Posted on June 10, 2009 by jessestay

I posted this over on the Twitter developer mailing list to try and get a discussion going. I thought I’d post a copy here for my readers to discuss – maybe you have more ideas than I do. I want to make it clear that I do not condone what some users of SocialToo are doing to gain Twitter followers. Will I stop them? I can’t – as long as Twitter allows them to do it, I can’t make a decision one way or another on who is doing this and who is not. No matter what, I have to respect my users, and most (almost all) of them are using Twitter for legitimate reasons. I do think changing the limits to what I suggested in the e-mail (below) will fix the problem Twitter is trying to solve though:

Let’s discuss the follow limits. I feel, as developer of a tool that allows people to auto-follow, I have a bit of insight into this. While there are many, many legitimate users that auto-follow others, and have good reason to do so, some are using it as a way to game the system, build followers quickly, break the Twitter TOS, and reduce the meaning of follower numbers for many other users just using the service legitimately. I see this daily, amongst a few of my own users, and while, due to our privacy policy I can’t share who they are, I do have some suggestions that would make the API follow limits make a little more sense. Maybe you guys can provide more insight.

-Currently the follow per day limit is 1,000 follows per user per day. There is no limit on the number of unfollows a user can do per day (that I know of), and it appears as though there is also a limit of around 10% for the number of users a person can follow more than follow them back. The users taking advantage of Twitter have figured this out. So here’s what they do:

A “gamer”‘s typical activity is that they will follow as many people as they can – most up to the 1,000 limit they’re allowed per day, until they hit the ratio of 10%. The higher the follower base they gain, the longer they’re able to do this. They then hope a good portion of those 1,000 people follow back. Those that don’t use tools like mine (which weren’t intended to be used this way) to unfollow everyone who is not following them back. This is often much greater than 1,000 for the users that are really good at it. The process then starts over. They’ll use tools like Hummingbird (Google it) and Twollo to find people and automatically go out and follow them. This is why I refuse to create auto-follow filters to find new people on my service. It’s way too spammy if you ask me.

Why do they do this? 2 reasons: 1, “supposedly” having more followers means more visits and clicks in whatever you’re trying to promote. (I don’t believe this) and 2, many of these people also have auto-DM set up to send links and messages to each person that follows them back. Back when I offered this service (we disabled it for this exact reason) people told me they were seeing significant clicks on the links they would send to people via DM after they followed them. Therefore, more follows==more clicks==more revenue. I don’t blame them if that’s what they’re really seeing.

So for this reason I think having limits in place is a *good* thing. I don’t think the follow limit is in place due to traffic reasons, since there are many more calls that cause more traffic on the API and there is no limit to unfollows, so I really think Twitter is doing this for the purpose of reducing spam and “gaming” of Twitter. This is a good thing.

However, I think Twitter may be approaching the limits the wrong way. Here’s what I think would be more effective, and beneficial for the legitimate users that want to follow back and at the same time not allow those who want to game the system to use the methods I described. Twitter needs to impose limits based on whether the individual is following the user back or not.

For instance, if I follow @dacort and he is following me back, that shouldn’t count against me as a hit against my follow limit. However, if I try to follow @dacort and he is not following me back, it should count against me as a hit against my limit. With this, users could easily auto-follow back if they choose to, and it would still be difficult for the users trying to game the system and spam Twitter. In fact, you could significantly *reduce* the limit this way and make it virtually impossible for these users to use Twitter in that manner. If you were to look at the relationship between the users when counting against limits, you could probably reduce the follow/day limit all the way to around 200 per day instead of 1,000 per day. I don’t see any reason for the 10% follow/follower ratio with a low limit such as that.

However, as stands, the more followers you get, if you are using Twitter legitimately, you have no way to extend the courtesy back if you choose to do so, since after a certain point you will be following many more than 1,000 users per day. And even if you aren’t, it will take an extremely long time for many individuals to finally catch up to follow those following them if they want to at 1,000 follows per day.

I know there are some that disagree with the auto-follow concept. However, I also know most of you also want Twitter to be an open environment where people can choose to use it as they please. Doug, Alex, etc. I’d love it if you guys could at least consider changing the follow limits as I mentioned. The current limits are doing nothing to prevent the spammers – my suggestions I believe will, and will keep it an open environment for the rest of us.

Sorry for the long discourse – I would really love to hear others thoughts and suggestions.

@Jesse

Feel free to chime in on the developers mailing list, or let’s discuss here – what suggestions do you have? Are there any holes in my proposal?

Twitter a Ubiquitous Utility? Open Up a Little!

Posted on June 7, 2009 by jessestay

I’m completely stumped by the recent media blitz by Twitter co-founders, Ev Williams, Biz Stone, and Jack Dorsey. They seem to be traveling the nation like rock stars, willing to say anything and everything about whatever the media wants to hear, as their service sits broken on the sidelines and the world wonders where their much needed Twitter has gone, or why their favorite app is down. Hey, it’s not my company though, so I’m sure they have their reasons, and good for them for enjoying and celebrating their success.

But then we see statements like that of Jack Dorsey, a reputable coder like myself, and someone I highly respect, stating, “I think Twitter’s a success for us when people stop talking about it, when we stop doing these panels and people just use it as a utility, use it like electricity…It fades into the background, something that’s just a part of communication. We put it on the same level as any communication device. So, e-mail, SMS, phone. That’s where we want to be.” Of course he’s saying this while his co-founders are touring the nation, visiting Oprah and Ashton and every major News outlet on the planet trying to get people to talk about Twitter. The thing is, I don’t care how useful Twitter is or has become (Despite the negatives that I’ve written about Twitter, I’ve written just as many positives), Twitter will never become a “Utility” until they open up a little, and I don’t just mean from a PR perspective (actually, I don’t mean that at all). I mean from an architecture perspective.

My friend (and fellow LouisGray.com co-writer), Rob Diana, thinks that Twitter is on its way to becoming infrastructure. I’m not arguing that’s not possible. In fact, I really think Twitter has the masses to make that happen. However, to make it the caliber of a “Utility”, or like “Electricity”, to me seems far-fetched when in the end, there is always a Twitter brand wrapped around it and no way for the public to have any control of that data. Until they remove those chains, Twitter is and always will be just another communications Service, not a Utility. Twitter will never blend into the background until they open up some.

Let’s look at some example utilities:

Electricity

We’ll start with the most obvious. Electricity is a natural force. It can only be created by utilizing existing energy. Various “Utilities” have found ways to harness this energy to create this electricity and deliver it to your home. However no single company owns that electricity, and each company is using the existing, open, laws of physics to harness and deliver such electricity to your home. No single company controls how electricity gets created or delivered. In fact, depending on your own local laws, you could even create your own electricity, and live independent from any single “Utility” company that also provides such service.

Water

Water is very similar. Each company provides a “service” or “utility” around cleaning the water supplies and routing them to your home. However, there’s nothing stopping any individual (other than government) from collecting their own water, and purifying it themselves for consumption. It’s bound only by the laws of Physics, and lives on a completely open model.

Phone

Perhaps even closer to Twitter’s turf than Electricity, the Phone is an open communications process. A “utility” company delivers the lines to get a signal from the sender’s phone, to a routing operator, and over to the recipient. However the underlying technology sending a phone’s signal from sender to recipient and back is not proprietary to any one single company. It’s simple Electricity (there’s that word again) and open protocols, and based on completely open (the most open) standards, bound simply by the laws of Physics.

So how in the world can a company like Twitter become like “Electricity”, flowing as the infrastructure of communication with little to no knowledge of those using the service? It comes down to the laws of Physics. Twitter needs to stop making the rules. They need to open up 100%, open up their code-base, release a protocol, and start letting people run their own Federated Twitter servers that can be run in any environment, speak the same protocol, and in essence, be invisible to those using the service. Twitter’s current model will never take them there.

Google’s leading the way here with Wave. The funny thing is Google employees aren’t going around bragging that their service will become like “Electricity” (okay, well maybe their name kind of implies that). What will happen is that you’ll be familiar with the Wave Product, which will become like Gmail and many, many users will use it since Google was first-to-market. However, what most users will not be aware of is that the way they communicate elsewhere will also be powered by Google Wave technology. Google Wave is also a Protocol, and built on an Open-Source Architecture. You’ll be able to run your own Wave servers, or even write your own services that speak the Wave protocol. Your users will never even know their communication went through a Google product.

I’m afraid Twitter has some serious competition as long as they want to become a Utility and Google Wave is in open development. Because they’ve waited to enter the Open arena, Google may just beat them to the punch on the “Utility” game. Jack Dorsey’s a smart guy. Twitter’s a great and powerful platform. However, I’m afraid they’re focusing too much on the platform and the product and not enough on what should really be the “Electricity” of their system – the communication.

Twitter, it’s time to open up guys. I’m itching to use some of that “Electricity” of yours.

Twitter Needs to Buy TweetDeck

Posted on June 2, 2009 by jessestay

WSJ today reported that China has blocked Twitter.com, preventing its 1.3 billion citizens from potentially accessing the site. Iain Dodsworth, developer of TweetDeck, was quick to point out however (which I noticed on FriendFeed ironically) that his service is still reportedly accessible in the Mainland. I’ve said it before, and I’ll say it again – Twitter, you need your own desktop client!

With everyone up in arms over how Twitter will monetize and when it will happen, there has been one obvious missing link from the start. Twitter has never had control over how Tweets get to the users. They themselves have admitted the API gets used much more than their own Twitter.com site gets used, showing, to me, that there is a way, currently, Twitter could be monetizing. Twitter has made it clear they do not intend to advertise through Tweets, which would almost certainly be rejected by their current user-base, already annoyed by the abundance of spam on the site. What Twitter could do however, that is assuming they have control over the main methods users receive Tweets, aka their desktop clients, is insert ads into such a desktop client and their own website, unobtrusive to users. Users will probably not even care at such ad placement, considering Google gets away with it, Yahoo and Microsoft do within their own search products.

Other Monetization Strategies

According to a recent TechCrunch article, Iain Dodsworth himself is already looking to monetize via this method. His method doesn’t even involve advertising, and maybe, in a world where advertising is beginning to prove ineffective, his method has even more potentially lucrative than the one I mention. His method is, according to TechCrunch, to charge developers looking for integration into TweetDeck for integration into the product. I think this is a great idea, and something that, with the popularity of TweetDeck, will prove quite profitable for him in the end.

Twitter could be doing this as well and perhaps even more effectively with the strong developer platform they have built. Add to that ability to integrate directories of applications into such an application enable featured app listings, along with perhaps an advertisement here or there Twitter could very quickly become profitable just by having control of such an application.

Why TweetDeck is a Good Choice

This isn’t meant to begrudge any of the other apps I enjoy and use. The fact is that TweetDeck is the most popular and most used Desktop app for Twitter right now (even though we can’t prove that at the moment thanks to Twitter API issues). It has the most users and most adoption – it would be the quickest barrier to entry, and clearly has features their users like, as they seem to continue using the app.

Considering deals are already being made to monetize the application, and considering, based on the recent investment of $500,000 into the company, comparatively speaking to the amount of money Twitter claims to have, TweetDeck shouldn’t be that expensive for Twitter to buy at the moment. Now’s the time Twitter. I wouldn’t wait much longer, as TweetDeck is about to become a whole lot more valuable.

More Distribution Options

Considering the recent issues in China, were Twitter to own the distribution channel, let’s imagine it’s TweetDeck, they would still have control of what goes out to China, and how to monetize off of those users. Currently, TweetDeck, and any client for that matter, have no devotion to Twitter – it’s just the first service they’re servicing in a long line of other services they’re also adding to their clients. Twitter should own this opportunity, and where they are claiming to be the next “AP”, if they’ve learned anything from the News organizations (I’ve worked in this industry), they need to control the distribution and not just the content.

I think Twitter is giving up an important opportunity by not having control of the desktop. I could probably say the same for the mobile phone (especially the iPhone). Perhaps their fear is that it would introduce unfair competition for the other desktop clients out there, but I don’t see any reason the other clients can’t compete as well, if not more than Twitter would provide anyway. This simply provides an important distribution platform where they would be provided many more monetization opportunities. I think it’s win-win.

Now, to get the guys at Twitter to start reading my blog…

Where is Twitter’s Emergency Response System?

Posted on May 31, 2009 by jessestay

The buzz has been swirling around the Twitter developer-sphere about a bug that has been going on for almost a full day now. Louis Gray reported it first at around 12am MST last night, and the first post to the Twitter development mailing list went up at around 2am MST last night. But Twitter is no where to be found, and it’s really starting to hurt some developers. So much that the very popular TweetStats, by Damon Cortesi has completely had to shut down until the service is re-enabled.

The bug is surrounding the display of the source app via both the API and in the Web UI showing which application a Tweet has come from on Twitter. Currently, according to TweetStats, 100% of the messages on Twitter are displaying they are coming from the Web. Developers and bloggers are complaining but no one is being heard.

In fact, according to Twitter, both Evan Williams (founder of Twitter), and Alex Payne (Twitter’s API Lead) are both in Maui on unrelated trips (Alex’s is for vacation – it’s unclear why Ev is there), posting pictures of the frozen drinks they are having and talking about the massages they are getting. Alex even stated he doesn’t have his laptop with him. Of course I don’t expect him to be reading this, and I congratulate him for being able to have some very deserved time-off–but what do we do when the API goes down?

Twitter developers have asked repeatedly for a paid API service which they can be guaranteed more up-time and more API access, along with a higher tier of support. Even Iain Dodsworth, the developer behind TweetDeck has mentioned in conversations on FriendFeed that, with unlimited API access, they would be able to deliver some of their “dream functionality”, and would “pay a lot” for such. As the developer behind SocialToo, I firmly agree with his statement – it would be a cost-savings for me. Regardless, there is still no good way to get Twitter support when their API goes down. Developers need some sort of Emergency Response System, and I think Twitter should charge for this level of service.

In times where developers’ apps go down many livelihoods are at stake. Money is not being made, and with a very poor support system by Twitter as is, and no way to guarantee support during such circumstances, developers are putting a lot on the line writing for such a service. Currently, the only means is via the Developer mailing list, and as we can see there is yet to be a response from Twitter via that means, and at least one entire application has been put out of business because of the issue.

Will there be a time we can see a prioritized service from Twitter that developers can pay for and guarantee service? I think with today’s example this option has just become a lot more important. The free service simply isn’t cutting it any more.

What do you think Twitter should do?