Page MenuHomePhabricator
Create Task
Maniphest T187538

Provide data for usage metrics
Closed, ResolvedPublic5 Estimated Story Points
Actions

Description

Please measure with at least a one-day granularity

Access step:

Preview step:

  • how often the import fails because of any exception, seperated by the reasons, e.g. - https://gerrit.wikimedia.org/r/#/c/414677/
    • a suppressed file (do we coun't each file once, of if there are multiple attempts, we count it multiple times?)
    • missing info (e.g. first version of file could not be retrieved anymore)
    • any of our max revisions limit is hit

Import Step:

  • how many files were successfully moved
  • how long it takes between users clicking the "import button" and redirecting to the commons page of a successful transfer
  • the number of file revisions, number of text revisions, and total MB moved for files that were attempted to be moved

Misc:

  • add all new metrics variable to a metrics.md file in the docs folder of the respective extension - Done in all patches

Details

SubjectRepoBranchLines +/-
Add another URL parameter to ease detection of imports from exportermediawiki/extensions/FileExportermaster+4 -0
Track how often ImportPlan creation failsmediawiki/extensions/FileImportermaster+17 -0
Track number of users not allowed to access special pagemediawiki/extensions/FileImportermaster+7 -0
Provide special page load data for usage metricsmediawiki/extensions/FileImportermaster+29 -2
Provide import step data for usage metricsmediawiki/extensions/FileImportermaster+113 -5
Customize query in gerrit

Related Objects
Search...

Event Timeline

• Lea_WMDE triaged this task as Medium priority.Feb 16 2018, 12:41 PM
• Lea_WMDE created this task.
• Lea_WMDE mentioned this in T179564: Tracking how often files were (un)successfully imported and other stuff.Feb 16 2018, 12:45 PM
Tobi_WMDE_SW set the point value for this task to 5.Feb 20 2018, 4:33 PM
Tobi_WMDE_SW added a project: WMDE-QWERTY-Sprint-2018-02-20.
Tobi_WMDE_SW moved this task from Proposed to Sprint backlog on the WMDE-QWERTY-Sprint-2018-02-20 board.
Addshore claimed this task.Feb 21 2018, 3:03 PM
Addshore moved this task from Sprint backlog to Doing on the WMDE-QWERTY-Sprint-2018-02-20 board.
Restricted Application added a project: User-Addshore. · View Herald TranscriptFeb 21 2018, 3:03 PM
Addshore moved this task from Unsorted 💣 to Back Burner 🏛️ on the User-Addshore board.Feb 21 2018, 3:04 PM
Addshore added a comment.Feb 21 2018, 4:18 PM
 how often the import fails because of any exception, seperated by the reasons, e.g.
a suppressed file (do we coun't each file once, of if there are multiple attempts, we count it multiple times?)
missing info (e.g. first version of file could not be retrieved anymore)
any of our max revisions limit is hit
a failure of the abuse filter

So all of these things are actually checked before the preview page is shown, rather than at time of import.
I guess we want to track that?
However it should be noted the number of failed imports and the number of times these things occur do not relate at all.

Addshore updated the task description. (Show Details)Feb 21 2018, 4:28 PM
Addshore updated the task description. (Show Details)Feb 21 2018, 4:33 PM
gerritbot subscribed.Feb 21 2018, 4:44 PM

Change 413195 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/FileImporter@master] Provide import step data for usage metrics

https://gerrit.wikimedia.org/r/413195

gerritbot added a project: Patch-For-Review.Feb 21 2018, 4:44 PM
Addshore mentioned this in rEFLI5570dff6aca6: Provide import step data for usage metrics.Feb 21 2018, 4:45 PM
Addshore mentioned this in rEFLI34251a752668: Provide import step data for usage metrics.Feb 21 2018, 4:51 PM
gerritbot added a comment.Feb 21 2018, 5:06 PM

Change 413199 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/FileExporter@master] Add another URL parameter to ease detection of imports from exporter

https://gerrit.wikimedia.org/r/413199

Addshore mentioned this in rEFILEEXPORT273a037a7fa8: Add another URL parameter to ease detection of imports from exporter.Feb 21 2018, 5:09 PM
gerritbot added a comment.Feb 21 2018, 5:12 PM

Change 413203 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/FileImporter@master] Provide special page load data for usage metrics

https://gerrit.wikimedia.org/r/413203

Addshore added a comment.Feb 21 2018, 5:13 PM

Patches above cover the access and import step.
TODO still is the preview step.

Addshore mentioned this in rEFLI5c18eb2ccba4: Provide special page load data for usage metrics.Feb 21 2018, 5:15 PM
WMDE-Fisch mentioned this in rEFLI71b6c73c13bb: Provide import step data for usage metrics.Feb 22 2018, 11:44 AM
WMDE-Fisch mentioned this in rEFLI8328f6e86ef1: Provide special page load data for usage metrics.
WMDE-Fisch mentioned this in rEFLI70c174a527d4: Provide import step data for usage metrics.Feb 22 2018, 12:06 PM
gerritbot added a comment.Feb 22 2018, 12:09 PM

Change 413195 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Provide import step data for usage metrics

https://gerrit.wikimedia.org/r/413195

WMDE-Fisch mentioned this in rEFLIe5fd7e2f219e: Provide special page load data for usage metrics.Feb 22 2018, 2:16 PM
WMDE-Fisch subscribed.EditedFeb 22 2018, 2:39 PM

Some questions that bumped into my head while reviewing patches here:

The SpecialPage on the Importer only shows if the user accessing it has rights to upload a file. ( I think on commons this means he as to have an account and needs to be logged in ). If a user gets to the Importer special page by clicking a link on the Exporter and - for whatever reason - he is not logged in, he will get a message, that he should login to continue.

Q1: Do we want to log theses special events? ( if the user needs to be logged in on the source wiki like en.wp SUL should normally take care of him being logged in on commons as well I guess, but still )
Q2: Sister projects like mediawiki.org / wikivoyage.org do not share the SUL login state with commons so...

  • a) Do we want to allow moving files from there too?
  • b) Would that work?

@Addshore @Lea_WMDE

gerritbot added a comment.Feb 22 2018, 5:34 PM

Change 413203 merged by WMDE-Fisch:
[mediawiki/extensions/FileImporter@master] Provide special page load data for usage metrics

https://gerrit.wikimedia.org/r/413203

• Lea_WMDE added subscribers: • Hanna_Petruschat_WMDE, Charlie_WMDE.Feb 23 2018, 11:37 AM

Q1)Can this really happen? We only show the "move to commons" link to autoconfirmed users, right?
Q2) Let me take this into my next UX meeting (FYI @Charlie_WMDE @Hanna_Petruschat_WMDE) and we will get back about that probably during storytime.
Thanks for the info @WMDE-Fisch!

Addshore mentioned this in rEFILEEXPORT1b599837caa8: Add another URL parameter to ease detection of imports from exporter.Feb 23 2018, 12:38 PM
WMDE-Fisch added a comment.Feb 23 2018, 12:39 PM

Q1)Can this really happen? We only show the "move to commons" link to autoconfirmed users, right?

Yes, the link is only visible for autoconfirmed users. No I think normally this should not happen. But when clicking the link, the user will be redirected to the commons wiki on another domain. There he should be automatically and instantly logged in due to SUL cookie magic and therefore he should be able to see the special page and upload images.

But: We don't really know in how many cases this process of getting automatically logged in to a foreign wiki with SUL breaks. So I thought we might want to try to track that somehow. - Could be disturbing for the users and if it happens often me might want to react to that in some graceful way. ;-)

Addshore added a comment.Feb 23 2018, 12:39 PM
In T187538#3995930, @Lea_WMDE wrote:

Q1)Can this really happen? We only show the "move to commons" link to autoconfirmed users, right?

Permissions on commons and permissions on client projects are different things.
An auto confirmed user on enwiki may not be autoconfirmed on commons.

Yes we do check that the link is only shown to auto confirmed users.

WMDE-Fisch added a comment.EditedFeb 23 2018, 12:43 PM

An auto confirmed user on enwiki may not be autoconfirmed on commons.

But you don't have to be autoconfirmed on commons to use our special page there, right?b

• Lea_WMDE added a comment.Feb 23 2018, 12:53 PM

But: We don't really know in how many cases this process of getting automatically logged in to a foreign wiki with SUL breaks. So I thought we might want to try to track that somehow. - Could be disturbing for the users and if it happens often me might want to react to that in some graceful way. ;-)

Then let's track this and see :)

Addshore added a comment.Feb 23 2018, 2:23 PM
In T187538#3996086, @WMDE-Fisch wrote:

An auto confirmed user on enwiki may not be autoconfirmed on commons.

But you don't have to be autoconfirmed on commons to use our special page there, right?b

No, but you need to be able to upload / be registered :)

Addshore updated the task description. (Show Details)Feb 23 2018, 2:24 PM
gerritbot added a comment.Feb 23 2018, 4:27 PM

Change 413747 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/FileImporter@master] Track number of users not allowed to access special page

https://gerrit.wikimedia.org/r/413747

Addshore updated the task description. (Show Details)Feb 23 2018, 4:28 PM
Addshore mentioned this in rEFLI3c079bc3eac3: Track number of users not allowed to access special page.Feb 23 2018, 4:30 PM
Addshore mentioned this in rEFLIed6b600e7c34: Track number of users not allowed to access special page.
Addshore mentioned this in rEFLI7e9a6c8fcc24: Track number of users not allowed to access special page.Feb 26 2018, 1:54 PM
• Lea_WMDE moved this task from Backlog to Tickets ready for pickup on the Move-Files-To-Commons board.Feb 26 2018, 2:16 PM
• Lea_WMDE moved this task from Tickets ready for pickup to Tickets in sprint on the Move-Files-To-Commons board.
Addshore mentioned this in rEFLI4cbd75463342: Track number of users not allowed to access special page.Feb 26 2018, 2:24 PM
gerritbot added a comment.Feb 26 2018, 2:38 PM

Change 414677 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/FileImporter@master] Track how often ImportPlan creation fails

https://gerrit.wikimedia.org/r/414677

Addshore updated the task description. (Show Details)Feb 26 2018, 2:39 PM
Addshore moved this task from Doing to Review on the WMDE-QWERTY-Sprint-2018-02-20 board.
Addshore moved this task from Back Burner 🏛️ to Active 🚁 on the User-Addshore board.
Addshore mentioned this in rEFLI3aeb68642ac6: Track how often ImportPlan creation fails.Feb 26 2018, 2:39 PM
gerritbot added a comment.Feb 27 2018, 2:37 PM

Change 413747 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Track number of users not allowed to access special page

https://gerrit.wikimedia.org/r/413747

gerritbot added a comment.Feb 28 2018, 10:58 AM

Change 414677 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Track how often ImportPlan creation fails

https://gerrit.wikimedia.org/r/414677

gerritbot added a comment.Feb 28 2018, 2:08 PM

Change 413199 merged by jenkins-bot:
[mediawiki/extensions/FileExporter@master] Add another URL parameter to ease detection of imports from exporter

https://gerrit.wikimedia.org/r/413199

• Lea_WMDE updated the task description. (Show Details)Feb 28 2018, 3:57 PM
WMDE-Fisch added a comment.Mar 1 2018, 2:44 PM

While working with WP and Commons I just realized, that the following seems to be wrong :-/ :

when clicking the link, the user will be redirected to the commons wiki on another domain. There he should be automatically and instantly logged in due to SUL cookie magic and therefore he should be able to see the special page and upload images.

  • when I am logged in on Wikipedia
  • and I am not yet logged in on Commons
  • and I go to https://commons.wikimedia.org/wiki/Special:UploadWizard ( a page that can only be used by logged-in users like our import special page )
  • then I get the message Please log in to upload files.

Usually some JS logic will take care so that I will be logged in an instant but that happens after the PHP is delivered. So users in our use case will probably always run into the "cannot upload" page when not already logged in on commons as well.

WMDE-Fisch added a comment.Mar 1 2018, 2:53 PM

Just tested this on https://en.wikipedia.beta.wmflabs.org with https://commons.wikimedia.beta.wmflabs.org
You must be logged in to upload files. :-/

WMDE-Fisch added a comment.Mar 1 2018, 3:01 PM

Created a task, see T188625

Addshore updated the task description. (Show Details)Mar 1 2018, 3:49 PM
Addshore moved this task from Review to Done on the WMDE-QWERTY-Sprint-2018-02-20 board.
ReleaseTaggerBot added a project: MW-1.31-release-notes (WMF-deploy-2018-03-06 (1.31.0-wmf.24)).Mar 6 2018, 10:01 AM
• Lea_WMDE closed this task as Resolved.Mar 12 2018, 9:02 AM
Addshore moved this task from Active 🚁 to Closing ✔️ on the User-Addshore board.Mar 20 2018, 1:45 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits