Over the last two weeks I’ve been blown-away by the take-up of my new site FastFollowers.com. I really didn’t expect people to sign up as quickly as they did. It’s reached the point where I’ve actually had to look seriously at scalability, SEO, and sustainable business models. What started out as a hobby project may actually turn into something more. The last few days has taught me a lot about twitter integration, to the point where I’ll definitely do things differently in future. Luckily, I have managed to put out most of the fires, and am at a point where the site can scale comfortably now for the next few months. I want share a few thoughts here for anybody integrating with twitter. Do these from the start if you want to avoid headaches in the future:
This one almost shut me down in the first week. When I first started, I decided to use basic authentication. This is a perfectly legal way of doing things, but unfortunately has been abused a lot by hackers over recent months. The basic principle is that you ask the user to enter their twitter username and password and then you use this to authenticate them. It’s in fact the same thing twitter does when querying your Google or Hotmail account on sign-up. The problem arises with what you then do with these details thereafter. Phishing sites take those details and use them maliciously. Legitimate sites will either not store the password at all, or will one-way encrypt them so that they cannot be compromised. I started out using basic authentication with one-way encryption, but it was clear this would not be viable going forward. Under risk of being shutdown, I completely rewrote the authentication to use OAuth. This is the preferred method by twitter. It means that you never take a users password, and you instead rely on keys and tokens. Bottom line, if you’re creating a web app, use OAuth. Don’t use basic auth. Twitter doesn’t like it and they will shut you down.
Avoid API Limits…
This can bite you in a nasty way. The limits are real, and you will hit them if you’re not careful. First prize is to try to develop in such a way that isn’t subject to the limits. An example of this is using HTTP POST’s instead of GET’s. Creating follows or status updates via the API uses POSTs, and these won’t run down your API limit. Running many queries for friend ID’s will quickly deplete your limit. It’s obviously not always possible to use the API in this way, but sometimes you can. Where you have to use the limits try and pass this on to the user. When you authenticate the user using OAuth you’ll use the hourly limits assigned to their accounts, and not the one assigned to your server’s IP address. Finally, cache as much as you can. I do one query when the user logs into my site and then update a SQL database with that information.
I’ve realized it’s better to handle a few exceptions rather than query twitter for every bit of information. Too much querying will add delay to your app and will deplete the API limit’s. Rather implement in such a way that you allow the user to do a few things that won’t work, and then explain to them why it didn’t work afterwards. An example here is probably the best way to explain this. Let’s say you don’t want a user to follow a person they’re already following. You could query the API first to see who they’re following, and then prevent them from following them. Alternatively, you could allow them to follow anyone, and then just present a friendly error message when they try re-follow someone. The later does far less querying and is just as resilient, and more scalable.
You’ll save yourself a lot of hassle if you just use twitter to authenticate your users. Don’t create a separate username and password on your own site. Use OAuth, and then based on the twitter ID you get for that user, create or log the user into your site. The alternative is much confusion from users when they change their twitter username or password. They get out of sync with your site’s login details, and they end up doing password recoveries. You’ll also convert more sign-ups with the simplified registration process. It’s too late for me to change to this model, but next time this will be the way I go.
- An alternative Gmail API (petewarden.typepad.com)
- Calling all Twitter tool providers to implement OAuth (thenextweb.com)
- Sign In With Twitter? Twitter Connect? (joedawsons.com)
- Twitter’s OAuth Support Now In Public Beta (techcrunch.com)
- Who’s Typing Your Password? (technologyreview.com)
- Web Slice and Feed Authentication – Developer Guidelines (blogs.msdn.com)
- Enough Already: MLM Invades Twitter’s Trending Topics (mashable.com)
- TwitCam: Live Video Arrives on Twitter (mashable.com)