Page MenuHomePhabricator

Fix and Stress test the Sign Up Captcha
Closed, ResolvedPublic

Description

Update: Currently the Sign Up Captcha does not work at all, see https://phabricator.wikimedia.org/T335769#9067453


When we remove the prerequisite of the invite code, there's a risk we might get spammed with new accounts or have a large amount of signups, that our service cannot yet handle. We need to prepare for these. Note: This concerns account creation, not Wikibase creation, there already exists a limit of 6 for that.
Current sign up rate is max. 4ish new Wikibases created per week. No metrics on accounts.

We already have a Captcha in place, we want to stress test it to make sure it's ready business. We want to check especially:

  • How does it handle spam
  • Is it easily passable
  • How does it affect users from other countries (i.e. is there an auto-blocker on certain countries)

ACs:

  • The Sign Up Captcha is working as intended
  • We know how Captcha performs in case of spam and users from any country signing up, and have a follow up plan to address potential issues

Event Timeline

Evelien_WMDE renamed this task from Introduce Guardrails for Mass account creation to Stress test the Sign Up Captcha.May 4 2023, 2:13 PM
Evelien_WMDE updated the task description. (Show Details)
Evelien_WMDE set the point value for this task to 5.

Context: we are using reCAPTCHA v2 for mediawiki (for creating new wiki accounts) and v3 for the UI (signup and login[?])

The Google reCAPTCHA documentation states that

reCAPTCHA v3 returns a score (1.0 is very likely a good interaction, 0.0 is very likely a bot). Based on the score, you can take variable action in the context of your site.

My current assumption is that we don't use the score to make any differentiation and just let every attempt pass, which may render the captcha useless at worst, but at least less effective.

We tested this by using some free firefox VPN extension (which I assumed should increase suspicious-ness for Google) and indeed today we can see that there was one request rated as "High risk" in the reCAPTCHA Admin Console for new wikibase.cloud v3: https://www.google.com/recaptcha/admin/site/577581332

Next steps could be to

  • find out where we can access this score rating that we should get from the reCAPTCHA response
  • implement some action for more higher rated requests

Currently I suspect that we aren't using the recaptcha v3 correctly - and I think I've got to the same assumption back then when we implemented the contact page (but apparently forgot to document that, maybe this ticket was the outcome of it? can't remember).
Anyway, the UI might do the right thing, but maybe we aren't using the score rating correctly (or at all) in the API. The package that we use for this is https://github.com/albertcht/invisible-recaptcha

Deniz_WMDE removed Rosalie_WMDE as the assignee of this task.EditedAug 3 2023, 5:29 PM
Deniz_WMDE added subscribers: rosalieper, Rosalie_WMDE.

There is an open issue for the recaptcha package we are using mentioning this exact problem https://github.com/albertcht/invisible-recaptcha/issues/164
Although there is a PR from April 23 up for fixing this (I wasn't able to verify that the fix actually works yet), the last commit on the packages repository was in 2022 Feb, so I assume this fix may never make it into the package.

My suggestion is that we try to find a package that provides the recaptcha v3 functionality but is actually working and is not abandoned.

As discussed in the daily today I will change this ticket in favor of creating a new one.
@Rosalie_WMDE I unassigned you because the scope of this task now has changed. Feel free to grab it again if you like!

Deniz_WMDE renamed this task from Stress test the Sign Up Captcha to Fix and Stress test the Sign Up Captcha.Aug 3 2023, 5:31 PM
Deniz_WMDE updated the task description. (Show Details)
Deniz_WMDE removed the point value for this task.
Deniz_WMDE updated the task description. (Show Details)

There is this package https://github.com/biscolab/laravel-recaptcha could be a good replacement for what we are currently using. Last commit was 6 months ago but last activity was last month, so maybe this is not an unmaintained repo.

I think this PR fixes the recaptcha, but further tests are needed since I wasn't able to fully cover the functionality in the unit tests yet. https://github.com/wbstack/api/pull/626

Basically the validation was always removed since this testing hack was faulty - if the env var was not set, the condition still evaluated to true.

I also used the official google recaptcha package to implement a Validation Rule class, because I wasn't able to find a laravel package that worked with our setup and looked trustworthy.

I was able to add some test cases and to get rid of the testing hack mentioned before.

ready for review: https://github.com/wbstack/api/pull/626
can be tested with skaffold and:

dang removed dang as the assignee of this task.Aug 23 2023, 12:15 PM

Deployed to staging. Prod deployment depends on

I deployed this to staging and can confirm sign up works as intended. Will therefore deploy to production as well.

Evelien_WMDE claimed this task.