In addition to being involved in startups, I enjoy finding system vulnerabilities as a side hobby. And I’ve found my fair share (see my about page, scroll down towards the bottom). The startup people I hung out with online in the mid-to-late 90’s were on IRC, before the startup incubators existed. This is where the cyber culture revolved around the discovery and sharing of new information. A little known factoid: before Shawn Fanning created Napster, the music software, he was known as “napster” on IRC. We ran in the same circles, finding and demonstrating security vulnerabilities through software we’d write and share. (I later joined Napster-the-company in 1999).

A couple weeks ago, I found another security vulnerability that impacted 1-1.5 million Twitter accounts.

Discovery

On January 19, 2011, I received a reply to a support ticket that I had filed on one of my business accounts. The support agent needed more information, so I jumped in to my ticket dashboard (everyone on Twitter has a ticket dashboard — just go to http://support.twitter.com). When I went there, I didn’t see my ticket listed. Thinking it’s just a glitch, I looked at an old ticket that was listed and back to the new email. I manipulated a few data fields, hoping it would work. As soon as I pressed enter, the ticket I was looking for showed up. Great, must be a temporary display glitch on my account. In any case, I was happy to be able to work with the ticket. I tried to reply to the ticket on the system. Strange, it didn’t attach my message. That’s when I noticed the account name didn’t match mine — it said @null instead of my business account name. Maybe I wasn’t supposed to see this. I finagled around with the data fields and suddenly I was staring at someone else’s support ticket — one that showed his password (he had wrote it as part of his ticket). This is Not Good.

Impact

If you ever submitted a support ticket for Twitter (and a lot of you did), you were impacted by this. All support tickets — at the time, 1.5+ million! were exposed.

To protect user privacy, I will not post the screenshots of tickets that contain private information. As with any ticket support system, the tickets included various sensitive information. The content was different in each, but contained a mixture of the following:

  • Account passwords
  • Contact information (addresses, phone numbers, etc)
  • API keys and Consumer Secret keys (for application development)
  • VIP requests (related to movie stars)
  • Brand impersonations (large multinational companies wanting certain accounts deactivated/removed)
  • 419 scams (yes, even Twitter tickets are not immune to offers to help save a dying prince)

Since it doesn’t contain sensitive information, here’s Twitter’s Support Ticket #1 (they are at over #1,500,000 right now) – click to enlarge :

Twitter Security: Support Ticket #1

Response

Twitter uses Zendesk to manage their support tickets. Was this system-wide or user specific — did this work on my main acccount, @Wayne? I tried it — didn’t work. Did this work for Groupon or Rackspace Cloud, who also uses Zendesk to manage tickets? I submitted tickets on those systems, finagled the data in a similar way, trying to replicate the issue. It didn’t work. It must only be affecting a subset of accounts inside Twitter’s account (I learned later that I was right).

I immediately tried to reach out to see who would be best to contact. I spoke with Dave McClure and he gave me two twitter names to try. I tweeted out to them and waited for a response.

Screenshot of @Wayne Twitter: One of Dave McClure's referrals

There must have been more formal ways; it can’t be just hoping and praying someone responds to you via Twitter (even if it’s Twitter itself). After digging, I found RFPolicy, a protocol to follow when discovering a vulnerability. I followed their instructions on sending out emails (to security-alert@, secure@, security@, support@, and info@ for twitter.com). All bounced back, except security@twitter.com. Bob Lord, who works at Twitter Security, responded quickly. We discussed the issue over email, and then a couple days later we had further discussions on the phone. Bob Lord was very professional to work with and seemed genuine when it came to the safety and security of Twitterverse.

Result

After I gave Twitter the instructions, they were able to reproduce the issue. The security hole is now closed and all data that I grabbed from the discovery of this vulnerability has been destroyed.

Bob has also set up new pages on Twitter’s web site and is working on new protocols internally to make it easier for others to submit security issues and for his team at Twitter Security to work with them. Twitter was also kind enough to recognize me in their about page for security.

Twitter's About Page: Recognition for assistance

It was a pleasure to work with Bob and Twitter on this issue.

Interesting notes

  • Twitter Support gets about 100-200 tickets per hour
  • Twitter Support seems to be addressing every ticket(!)
  • Twitter Support receives support tickets in multiple languages
  • Twitter Support has support staff responding in those languages

Thanks to Dave McClure (500 Startups), Bob Lord (Twitter), Charles Huang (Spark Capital), Xixi Chen (Dana Farber) and Mark Bao for reviewing drafts and assistance.

Enjoyed this post? Leave a comment below and Follow me on Twitter
  • Security

    fgdgdfgdfgkgfgjkflgfl;g
    ggp’fdgkfpgkfpgkpfd
    kg;fdkgof;gkf;ll;gkfgkf;kgf;gkf;kgf;kgf
    iorioutututuopfupfuispfljflajflafjaiofjiofjjfjfsfgsdfsdf

  • Anonymous

    thank you.

  • http://mobileaffiliateprofitsx.com Sachy24

    you are great wayne…made twitter to patch up quick :)

  • http://isaacgc.tumblr.com/ isaacgc

    Way to go Wayne!!

  • http://bhavyakamboj.com Bhavyakamboj

    thats nice…..really

  • Dianna Chenevert

    I tried reporting my log in problem with Twitter through their help center, but every time it has said “There was an error preventing ticket submission. Please try again later

  • Dianna Chenevert

    PLEASE HELP!!!
    I haven’t been able to log in to my twitter acct. since March 1st. I’ve reset my password a few times since then, but every time after resetting it, when I go to log in nothing happens~It never lets me log in. I’ve had my twitter acct since 2009 & am in good standing. I can’t figure out what’s going on with it. I sent twitter security a couple of messages (Both through my personal e-mail & tried on twitter’s site), but haven’t received any responses. Something else strange happened today. I thought I’d just make another twitter acct (using family’s last name) so I could hopefully get a message through to security. I set up this new twitter account & password~It allowed me to make the acct, but it’s already following my original twitter acct & I didn’t do it!!!!!! ….And it’s still not letting me log in to the new one either?

    Orginal twitter acct: Dianna Chenevert @MamaDianna
    New one: Dianna Thompson @TweetingMamaD

    • http://www.waynechang.com Wayne Chang

      Dianna, you’ll need to contact them at http://support.twitter.com. I would keep trying until you get through. Sometimes submitting the ticket through a different web browser will fix the error you discussed.

    • Hshsh

      That is probably because you used your parents last name it knows you are related to your old account so it has listed the new account as a follower of the old account. Guess twitter tried to make it easier for you by adding your new account as a follower for your aold account. AAnother thing: Wayne Chang does not work at twitter or investigate it so please don’t ask him

  • http://twtt.ru/ Евгений

    Cool! Hello Wayne Chang. Can i translate to russian languge you article and post on my blog twtt.ru ? Thank you.

    • http://www.waynechang.com Wayne Chang

      Absolutely – just link back to this original article

      • http://twtt.ru/ Евгений

        of course :) thx

  • Pingback: Tweets that mention How I Discovered a Security Vulnerability in Twitter | WayneChang.com -- Topsy.com

  • http://www.LinkedMediaGroup.com Linked Media Group, Inc.

    Thanks for sharing this with the community – wonderful Blog post=Grok Cheers………

  • http://twitter.com/MarkHall123 Mark Hall

    It is great to see immediate action was taken by Twitter; however, it is somewhat surprising that it was so difficult to get in touch with Twitter’s support staff. Every company has its slips, and this is definitely one for Twitter. I’m sure the whole Twitter community would join me in saying thanks for diligently working to solve issues that protect us all.

    • http://www.waynechang.com Wayne Chang

      Once I got a hold of someone at Twitter, it was very easy to work with them. I think it’s insane how they can answer all their support requests.

  • @simther2011

    Good Job! Nice to see people who use there knowledge for good ;-)