Page MenuHomePhabricator

[Commons Impact Metrics] Establish how we represent the allow-list
Closed, ResolvedPublic1 Estimated Story Points

Description

An important decision to make before doing the implementation is establishing how to represent the Commons Impact Metrics category allow-list.
This will help productionizing the data pipeline, since some of its jobs will require the allow-list and will have to read it in its specified format.
It's not important to have the list ready. Its contents can be mocked for the sake of development of the pipeline.
We can finalize the contents of the list later.

Tasks:

  • Discuss with stakeholders and decide how to represent the allow-list

Deliverables:

  • A comment on this task explaining the decision

Event Timeline

On Monday March 4, we had a meeting with @mforns, @VirginiaPoundstone and @FRomeo_WMF were we discussed using GitLab to keep the allow list. I explained briefly how that may work, but here is a detailed proposal:

  1. Find the name of the "top level" category you want to add. For example: Digital Public Library of America.
  1. In GitLab, there will be a file named categories_allowlist.txt. Find it, and click on it. This file will contain all the top level categories we are interested in, one per line, in alphabetic order. For example, we could have the following content in the file already:
A Category
B Category
C Category
D Category
E Category

allowlist step 2.png (1,978×2,014 px, 664 KB)

  1. In GitLab, we will click on the file mentioned in (2), and then click on 'Edit', and then 'Edit single file'. This will present a simple text editor, similar to Notepad in Windows, or to TextEdit in MacOS.

allowlist step 3 and step 4.png (1,978×1,914 px, 580 KB)

  1. Modify the text file such that the new category is added, honoring the alphabetic order. This order is not neccesary for correctness, but just so that it is easy for a human to scan the file. For example, after adding the example category, the file would look like so:
A Category
B Category
C Category
D Category
Digital Public Library of America
E Category

In the area that reads 'Commit message', write something in the lines of "Adding top level category 'Digital Public Library of America'". In the area that reads "Target Branch", something in the lines of 'add-dpla', but really, this could be anything. Make sure "Start a new merge request" is checked. Click "Commit changes".

allowlist step 5.png (1,978×2,662 px, 826 KB)

  1. GitHub will redirect you to the "Merge request" flow. On reviewer field, choose either xcollazo or mforns. Leave everything else with its defaults. Click on "create merge request".

Allowlist step 6.png (1,978×3,082 px, 1 MB)

  1. Either xcollazo or mforns will review the change and accept it if it looks good. That's it.

Assumptions:
a) Top level category names seldom change. In the event that they do, we would need to remove the old category name and add the new category name to this list. All previously computed data would be lost.

xcollazo changed the task status from Open to In Progress.Mar 5 2024, 5:13 PM

This is clear and is a workable solution.

@xcollazo Are these instructions for the community at large, or for WMF? It seems like there should ideally be a Commons interface way of doing this, unless part of the goal/expectation is to limit the number of categories on the list. Currently, we have about 2000 Commons categories tracked, at least, in BaGLAMa and similar tools already. Would this process handle that number of requests?

These are instructions for Wikimedia Foundation staff.

We're building a list of the categories already tracked by the GLAM Wiki Dashboard, Cassandra Dashboard, and BaGLAMa, and they will be part of the initial set-up. Are there other tools we should be looking at?

The process described here will just be used to add new partnerships/categories.

mforns set the point value for this task to 1.Mar 21 2024, 2:06 PM

These are instructions for Wikimedia Foundation staff.

We're building a list of the categories already tracked by the GLAM Wiki Dashboard, Cassandra Dashboard, and BaGLAMa, and they will be part of the initial set-up. Are there other tools we should be looking at?

Also the ~500 categories at https://commons.wikimedia.org/wiki/Category:Category_with_page_views_table are another set of categories tracked by another tool, if they can be in the initial setup. Thanks!