Why Do We Ask For Your Phone Number?

A Bit About Spammers

Account Creation By Year

A few weeks ago we re-enabled new account creation on the Open Hub. New Account creation had been shut of for a number of months as it had become impossible to stave off the number of spam accounts. We were desperate and needed a better solution. Back in February of this year, we talked about the problem we were having with spammers and the reality that we had built a Spam Farm, and some of the solutions we were considering. To the right is a chart of how the number of accounts grew each year. I would have loved to believe each of these accounts was someone with interest in Open Source Software (OSS) and the OSS community.

We have over 800,000 accounts. Back in February, when I wrote that post, we had 660,000. That’s 140,000 new accounts in essentially a month and a half. More on that in a moment.

We chose to use Twitter Digits as supplied by Digits.com as the technology to block spammers. This service has a high enough barrier to entry that it would probably not be easily defeated by spammy account bots and “marketing firms”. We initially thought that we could use the service after accounts were created to verify the user and therein not be too intrusive. That really didn’t work. Look at the chart below. See that spike at the right? The line is the percent of spam accounts to valid accounts. For every legitimate account, there were more than 7 spam accounts created. We shut that down quickly.

Spam Account Creation Rate

Hey, we should note that these are detected spam accounts. We choose to err on the side of letting an account that really looks spammy but isn’t violating any Terms Of Usage to remain unblocked. There are probably a lot of accounts that should be flagged as spam but aren’t.

Twitter Digits

The way Twitter Digits works is by presenting the new account applicant with a Digits dialog box prompting for an SMS capable number. Digits.com sends a four digit code to that SMS number. The applicant enters the code into a field we provide and we send that code to Digits, which tells us if the code is valid. At that point we get an ID that we can use to identify that SMS number holder.

Please note that Open Hub does not receive your phone number. The Open Hub can’t call you. We can’t get your phone number. Digits is stand alone service. You don’t need a Twitter account. You don’t need a Digits account. Twitter does not link your number to any other account information. (source: New York Times Blog).

All we get is a unique identifier, but we can’t trace that back to a phone number. Let’s be clear: The Open Hub does not get or store your phone number. (source: Me. I wrote the technical specifications and reviewed the code)

The Numbers

As mentioned, we have over 800,000 accounts. Just over 28,500 accounts have claimed OSS contributions. There could be that many accounts again that are interested in OSS, but haven’t claimed any contributions or haven’t made contributions, or are OSS consumers. That would give us 57,000 accounts that are “legitimate” accounts. Oh, heck. Let’s double that again just to be generous. Now we have almost 120,000 accounts. This still leaves 680,000 illegitimate accounts (spammers!) in our system, using our bandwidth, gumming up our analyses, and impacting the experience of those who seek to use the Open Hub for it’s intended purposes. That means for every one of those legitimate accounts, there are 5.6 accounts that are nothing but junk.

Let’s talk about what happened when we opened up new account creation on August 10, 2015. At that time we let users sign up and then verify their account with Digits. In the next few weeks we had 4,077 new accounts created. We identified that 3807, or 93.4%, of them were spam accounts.

We re-worked the flow of new account creation to ensure that the SMS verification with Digits is done first. Since then we’ve had 391 new accounts created. Of those; only 20, or 5.1%, have been identified as spam.

Twitter Digits is currently successfully controlling new account creation and blocking the vast majority of empty spam accounts. But that does not solve the problem of all those accounts that are currently in the system.  We need a way to get rid of all those worthless accounts.  Which is why we ask for verification using Digits for all Open Hub account holders who are looking to make edits on the Open Hub.

Coming Up

We are looking at expanding our account verification options to include something like a GitHub OAuth verification.

We really hope that existing users will verify their account with these new verification techniques because we are working on the next part of the Spammer Purge, which is to request re-verification of every non-SMS or non-OAuth verified account. Accounts that are have not been SMS or OAuth verified will be deactivated after a few weeks of the request and then deleted after a few months. We understand that this may impact a few legitimate users, which is why we will wait months before deleting any account and why we will send email notifications at each step of the process.

This is only one part of the work that we’re doing at the Open Hub to improve the speed and reliability of our service. Other current work is to move off of our aging crawler infrastructure and update our Analytics Engine so that it is a stand alone application with it’s own database. This will leave the database under the Ohloh-UI responsible only for serving the website. These plans will enable the Open Hub team to invent new analyses and bring in other elements of the open source landscape to support enriched comparisons and conversations about OSS.

The Open Hub Team is grateful to you, our community, for your support and patience as we address these important infrastructure elements. We’re also grateful to Black Duck for providing all the funding for our team, all our crawlers, our web servers, the IT support, and all other support costs that make it possible to provide the Open Hub as a completely free service to the OSS community.

About Peter Degen-Portnoy

Mars-One Round 3 Candidate. Engineer on the Open Hub development team at Black Duck Software. Family man, athlete, inventor
  • Siarhei Siamashka

    Providing a phone number is not an acceptable option for me. You are fighting the spammers, but I do receive occasional SMS spam too and don’t want it to become worse. Especially considering that you are relying on a third-party service for this functionality, which effectively means that I need to trust both you and this third-party. Sorry about it. What would be an approximate ETA for the GitHub OAuth verification introduction?

    Also regarding the existing 28,500 accounts of the long time users, who have claimed OSS contributions (my account belongs to this group too). Would it be possible not to enforce the phone based authentication for them yet? What is the percentage of confirmed spammers among this group?

    • Hi Siarhei;

      Thank you for your comments. We’re working on other authentication models and should have them ready in a few weeks.

      Yes, of our course it would be possible to program a different approach for account holders. After all, we’re programmers –it’s what we do. 😀
      We will be doing something along those lines when we reverify accounts as part of the overall cleanup.

      That said, these issues operate on a continuum. Creating exceptions for ensuring that accounts adhere to consistent standards just lowers the overall level of confidence we have in user accounts.

      Nonetheless, your points are well taken, which is why we are moving quickly to add other forms of authentication.

      Sincerely Yours,

      Peter

      • Eclesia

        Hello, I can understand the problem you have with spammers.

        I tryed to log in after several month to ask for a bit of help since my project didn’t update several weeks (https://www.openhub.net/p/unlicense-lib) and it asked me my phone number even if I was a user of ohloh since 2008.

        I’m just adding my vote here : No way you will have my phone number, and same as Mxx will go see somewhere else if there is no reasonable alternative.

        Thanks for your efforts on finding other ways, still I’m not a social network addict, no disqus,facebook,tweeter, linkedln or google acount. And I’m not a git user either, since all my projects are on bitbucket with mercurial.

        So what about somekind challenge email ? send an email with a question we have to answer back ? well anything, as far as it doesn’t requiere to have some other account elsewhere or personal informations.

        thanks

        • Hi Eclesia;

          Thanks for sharing your thoughts. I do hope you understand that we would not have your phone number.

          We use Digits.com only to authenticate that you are a real, unique person. We don’t get your phone number. We don’t keep your phone number. We won’t call you. We don’t share any of your information we have with anyone. Ever. I promise.

          All of that is only to ensure you are as informed as possible. I respect your choice and recognize that you are one of those edge cases we feared. 🙂

          I will talk over the idea you propose with my team. We are searching to strike a balance between secure, validated accounts and an approach that isn’t too heavy handed, while not implementing too many different ways in that could eliminate the secure mechanism we are putting in place as well as create multiple, similar pieces of functionality to maintain.

          Yours,

          pdp

  • Mxx

    I’m sorry, but this is a deal-breaker for me.
    There’s absolutely no way I will provide my phone number to use this service. I do not trust you with it(yes, i know you will not get it). I do not trust Twitter with it(yes, i know you claim they do not get it, but I also don’t use twitter). I do not trust digits.com with it.
    This is my account https://www.openhub.net/accounts/mxxcon/ As you can see I have a little bit of history in it. But if I’m required to provide my phone number to continue to use it, you lost me as a user.

    • Hi Mxx

      Thanks for sharing your thoughts. We understand that not everyone is comfortable with using their phone for account verification, which is why we are working on other forms of authentication.

      The priority remains to control the overall number of spam accounts because of the additional cost and overhead they incur.

      We hope you will stay with us and are doing what we can to walk the line between a sufficiently secure while free and open system.

  • okapia

    Seems I can’t now do anything on the site without providing a number. I can’t even access basic account settings. I don’t even own a mobile, not that I’d give you the number if I did. This decision is simply idiotic.

    • Hi okapie;

      Thanks for letting us know your thoughts. As mentioned in the post, we are working on additional forms of authentication, though I’d like to point out that the SMS verification has dramatically reduced the volume of spam accounts to a level that is manageable by the team. This means that we have time to do other work, such as add alternative OAuth options, address defects, and build new features.

      Please hang with us a bit longer and let’s see if the next step in account verification doesn’t sufficiently resolve your issue.

      Many thanks, pdp