How I Discovered a Security Vulnerability in Twitter

In addition to being involved in startups, I enjoy finding system vulnerabilities as a side hobby. And I’ve found my fair share (see my about page, scroll down towards the bottom). The startup people I hung out with online in the mid-to-late 90′s were on IRC, before the startup incubators existed. This is where the cyber culture revolved around the discovery and sharing of new information. A little known factoid: before Shawn Fanning created Napster, the music software, he was known as “napster” on IRC. We ran in the same circles, finding and demonstrating security vulnerabilities through software we’d write and share. (I later joined Napster-the-company in 1999).

A couple weeks ago, I found another security vulnerability that impacted 1-1.5 million Twitter accounts.

Discovery

On January 19, 2011, I received a reply to a support ticket that I had filed on one of my business accounts. The support agent needed more information, so I jumped in to my ticket dashboard (everyone on Twitter has a ticket dashboard — just go to http://support.twitter.com). When I went there, I didn’t see my ticket listed. Thinking it’s just a glitch, I looked at an old ticket that was listed and back to the new email. I manipulated a few data fields, hoping it would work. As soon as I pressed enter, the ticket I was looking for showed up. Great, must be a temporary display glitch on my account. In any case, I was happy to be able to work with the ticket. I tried to reply to the ticket on the system. Strange, it didn’t attach my message. That’s when I noticed the account name didn’t match mine – it said @null instead of my business account name. Maybe I wasn’t supposed to see this. I finagled around with the data fields and suddenly I was staring at someone else’s support ticket — one that showed his password (he had wrote it as part of his ticket). This is Not Good.

Impact

If you ever submitted a support ticket for Twitter (and a lot of you did), you were impacted by this. All support tickets – at the time, 1.5+ million! – were exposed.

To protect user privacy, I will not post the screenshots of tickets that contain private information. As with any ticket support system, the tickets included various sensitive information. The content was different in each, but contained a mixture of the following:

• Account passwords
• Contact information (addresses, phone numbers, etc)
• API keys and Consumer Secret keys (for application development)
• VIP requests (related to movie stars)
• Brand impersonations (large multinational companies wanting certain accounts deactivated/removed)
• 419 scams (yes, even Twitter tickets are not immune to offers to help save a dying prince)

Since it doesn’t contain sensitive information, here’s Twitter’s Support Ticket #1 (they are at over #1,500,000 right now) – click to enlarge :

Twitter Security: Support Ticket #1

Response

Twitter uses Zendesk to manage their support tickets. Was this system-wide or user specific — did this work on my main acccount, @Wayne? I tried it — didn’t work. Did this work for Groupon or Rackspace Cloud, who also uses Zendesk to manage tickets? I submitted tickets on those systems, finagled the data in a similar way, trying to replicate the issue. It didn’t work. It must only be affecting a subset of accounts inside Twitter’s account (I learned later that I was right).

I immediately tried to reach out to see who would be best to contact. I spoke with Dave McClure and he gave me two twitter names to try. I tweeted out to them and waited for a response.

Screenshot of @Wayne Twitter: One of Dave McClure's referrals

There must have been more formal ways; it can’t be just hoping and praying someone responds to you via Twitter (even if it’s Twitter itself). After digging, I found RFPolicy, a protocol to follow when discovering a vulnerability. I followed their instructions on sending out emails (to security-alert@, secure@, security@, support@, and info@ for twitter.com). All bounced back, except security@twitter.com. Bob Lord, who works at Twitter Security, responded quickly. We discussed the issue over email, and then a couple days later we had further discussions on the phone. Bob Lord was very professional to work with and seemed genuine when it came to the safety and security of Twitterverse.

Result

After I gave Twitter the instructions, they were able to reproduce the issue. The security hole is now closed and all data that I grabbed from the discovery of this vulnerability has been destroyed.

Bob has also set up new pages on Twitter’s web site and is working on new protocols internally to make it easier for others to submit security issues and for his team at Twitter Security to work with them. Twitter was also kind enough to recognize me in their about page for security.

Twitter's About Page: Recognition for assistance

It was a pleasure to work with Bob and Twitter on this issue.

Interesting notes

• Twitter Support gets about 100-200 tickets per hour
• Twitter Support seems to be addressing every ticket(!)
• Twitter Support receives support tickets in multiple languages
• Twitter Support has support staff responding in those languages

Thanks to Dave McClure (500 Startups), Bob Lord (Twitter), Charles Huang (Spark Capital), Xixi Chen (Dana Farber) and Mark Bao for reviewing drafts and assistance.