Page MenuHomePhabricator

Wikimedia Commons Query Service should use Wikimedia url shortener instead of tinyurl
Closed, DuplicatePublic

Description

I wondered why https://tinyurl.com/y4myl8z6 was shared instead of a link like https://w.wiki/rL . Turns out the prototype uses tinyurl. Should probably switched at some point. Not sure how to deal with the temporary domain https://wcqs-beta.wmflabs.org and the final domain. Maybe just do a search replace on the our tinyurl database later when the final domain becomes available?

Event Timeline

Yes, if it goes to *.wikmedia.org or *.commons.org, it'll be shortable in short url but *.wmflabs.org is not among the allowed domain list for security reasons (hiding XSS through open redirects, etc.)

Yes, if it goes to *.wikmedia.org or *.commons.org, it'll be shortable in short url but *.wmflabs.org is not among the allowed domain list for security reasons (hiding XSS through open redirects, etc.)

Can't we just put "wcqs-beta.wmflabs.org" on the whitelist? I agree putting *.wmflabs.org on the whitelist isn't such a good idea, but just one subdomain shouldn't be a problem.

Can't we just put "wcqs-beta.wmflabs.org" on the whitelist? I agree putting *.wmflabs.org on the whitelist isn't such a good idea, but just one subdomain shouldn't be a problem.

That's technically possible. I'd ask security to sign it off before applying it but shouldn't be too hard.

I would really rather not do that. This is a beta service – I think putting it on the official URL shortener whitelist, or even considering to rewrite shortened URLs(!), gives it an undeserved appearance of being more stable than it’s really meant to be.

I would really rather not do that. This is a beta service – I think putting it on the official URL shortener whitelist, or even considering to rewrite shortened URLs(!), gives it an undeserved appearance of being more stable than it’s really meant to be.

Being able to share queries is an integral part of beta. If you don't agree with that, speak up. So being able to do that should be supported in a proper way as part of the beta. Tinyurl is still blacklisted so sharing is painful now.

What about url shortener of beta? It's something like w-beta.wmflabs.org/x12 (it's broken currently but I can fix it).

I would really rather not do that. This is a beta service – I think putting it on the official URL shortener whitelist, or even considering to rewrite shortened URLs(!), gives it an undeserved appearance of being more stable than it’s really meant to be.

Being able to share queries is an integral part of beta. If you don't agree with that, speak up. So being able to do that should be supported in a proper way as part of the beta. Tinyurl is still blacklisted so sharing is painful now.

I never said being able to share queries was not important…? But TinyURL works well enough – not as well as w.wiki, but well enough that I don’t think compromising the integrity of w.wiki is a worthwhile tradeoff.

In some ways it's quite nice that WCQS uses tinyurl rather than the w.wiki shortener -- at least it means there is not such a limit on how long the query can be. (T220703).

Perhaps we could revert WDQS to use tinyurl again, too ?

In some ways it's quite nice that WCQS uses tinyurl rather than the w.wiki shortener -- at least it means there is not such a limit on how long the query can be. (T220703).

Perhaps we could revert WDQS to use tinyurl again, too ?

You can't link to tinyurl in Wikidata/Wikipedia as it's blacklisted. The fact that you disagreed with a decision (which I agreed with you), doesn't mean we should ditch the whole tool in favor of third parties harvesting user's data.

Legoktm added a subscriber: Legoktm.

Can't we just put "wcqs-beta.wmflabs.org" on the whitelist? I agree putting *.wmflabs.org on the whitelist isn't such a good idea, but just one subdomain shouldn't be a problem.

That's technically possible. I'd ask security to sign it off before applying it but shouldn't be too hard.

The general requirements are (we should document this somewhere):

  • No open redirects (allows bypassing the allowed domains list)
  • No dangerous actions on GET requests
  • No reflective XSS
  • Should fall under the Wikimedia privacy policy / or anyone who has access to private information should have signed an NDA.

URL stability explicitly isn't a consideration, e.g. wiki pages can disappear at any time. It's up to each service to maintain stable URLs (Cool URIs don't change!) - I would expect wcqs-beta.wmflabs.org eventually redirects to the production instance.

Gehel triaged this task as Medium priority.Sep 15 2020, 7:43 AM
Legoktm changed the task status from Open to Stalled.Mar 18 2021, 4:59 PM

This is waiting on response from the WCQS maintainers on whether the above criteria are met (now documented at https://wikitech.wikimedia.org/wiki/W.wiki)

This is waiting on response from the WCQS maintainers on whether the above criteria are met (now documented at https://wikitech.wikimedia.org/wiki/W.wiki)

Regarding that "TODO: link phab tickets" at https://wikitech.wikimedia.org/wiki/W.wiki

T231518

The general requirements are (we should document this somewhere):

  • No open redirects (allows bypassing the allowed domains list)
  • No dangerous actions on GET requests
  • No reflective XSS

The code is the same as the production WDQS and should support the requirements above.

  • Should fall under the Wikimedia privacy policy / or anyone who has access to private information should have signed an NDA.

This project has quite a few members. Most of them are WMF staff and are under NDA, but a few are not and even if that was the case, we don't have a good way to enforce NDA for future access. It seems that we'll need to wait for WCQS to be in production to use Wikimedia URL shortener. We plan to be working on that during next quarter, but that's still a few month away.

URL stability explicitly isn't a consideration, e.g. wiki pages can disappear at any time. It's up to each service to maintain stable URLs (Cool URIs don't change!) - I would expect wcqs-beta.wmflabs.org eventually redirects to the production instance.

wcqs-beta.wmflabs.org will disappear once a production service is in place, URLs will be broken and won't be redirected.

  • Should fall under the Wikimedia privacy policy / or anyone who has access to private information should have signed an NDA.

This project has quite a few members. Most of them are WMF staff and are under NDA, but a few are not and even if that was the case, we don't have a good way to enforce NDA for future access. It seems that we'll need to wait for WCQS to be in production to use Wikimedia URL shortener. We plan to be working on that during next quarter, but that's still a few month away.

Yeah :(

URL stability explicitly isn't a consideration, e.g. wiki pages can disappear at any time. It's up to each service to maintain stable URLs (Cool URIs don't change!) - I would expect wcqs-beta.wmflabs.org eventually redirects to the production instance.

wcqs-beta.wmflabs.org will disappear once a production service is in place, URLs will be broken and won't be redirected.

You can use https://wikitech.wikimedia.org/wiki/Nova_Resource:Redirects to set up the redirect without needing to maintain anything extra on your end.

Suggestion. You can't use the production url shortener but we have one in beta cluster. You can simply add it to the list of allowed domains in beta cluster and use it in https://w-beta.wmflabs.org (currently broken but https://gerrit.wikimedia.org/r/c/operations/puppet/+/673389 fixes it). It sorta makes sense, it's the staging environment for production. I'm doing the same for Wikidata query builder (https://query-builder-test.toolforge.org/) that's in labs but will go to production soon.