Earlier in the year you may remember Twitter “accidentally” deleting a large portion of a majority of their users’ followers and having to spend days restoring those from tape backup due to their mishap. Well, it would appear that Twitter has done it again. This time may not be as obvious to users.
On SocialToo.com I was noticing we were getting “Rate Limit Exceeded” errors for our whitelisted @socialtoo user. Such errors shouldn’t be happening for whitelisted users so I looked into it. Sure enough, on the Twitter development mailing list I noticed an email from the founder of MrTweet, Yu-Shan Fung, stating they too were seeing the same issue with their @mrtweet whitelited user.
Alex Payne, Twitter API Lead, confirmed the issue, stating, “It looks like some database maintenance inadvertently truncated our table of whitelisted users.” He then continued in a follow-up e-mail making the problem seem worse: ‘The updated estimate I’ve just received from our ops guys is “more
than 15 minutes and less than 12 hours”. They have to restore from a
nightly database backup. Said backups are quite large, and take some
time to get through.’ It would appear that Twitter has inadvertently deleted an entire table of all the whitelisted users – this is the table that enables many of the Twitter Apps you use to continue to talk to the API without breaking. This is what we as developers are relying on to work or our Apps go down.
It was the exact issue of database deletion and insecurity of Twitter being competent enough to not do it again that caused me to go on hiatus from Twitter earlier in the year. Problems like deleting an entire table from a database, especially twice in a year, are inexcusable Twitter! I recommend finding someone that can ensure this doesn’t happen again. It’s time to upgrade your Operations team. As developers, we simply can’t rely on this, and it’s why I called my service SocialToo and not TwitterToo.
In the meantime, many of the Twitter services you use until this gets resolved may have issues. This includes SocialToo.com autofollow being down until the table in the Twitter database is up again. I’ll keep you updated on the @socialtoo Twitter account when this comes up again.
