Page MenuHomePhabricator

See actual data for new editors, pages created, improved etc.
Closed, InvalidPublic

Description

(From the user feedback interview.)

User should be able to see actual data than just the numbers.

Mock 1:

image.png (1×1 px, 352 KB)

Basically add a div in each table cell and populate it on demand (see this for example). We can perhaps show a loading animation if it turns out to be slow. Note:

  • The entire row will be clickable and we add the collapse/expand markers on the left to let people know the rows are expandable.
  • Multiple rows can be expanded at the same time.
  • Show a sane number of rows (say 10 for starters) and make the div scrollable.
  • Totals row is expandable as well.
  • User names and page titles are links (all left aligned in the div).
Mock 2:

image.png (1×1 px, 380 KB)

We don't touch the Event page tables and add another button for letting users see a more granular view of data which takes them to this page. All except one wiki are collapsed by default and we run the queries when user clicks a section open. This still needs us to run all five queries for a wiki at the same time, like in above mock, so we still need to do the performance testing .

This gives us the advantage of not having to complicate an existing UI that works well. The sections shown in the mock are all placeholders and can/will change depending on what data is deemed important. The other advantage is that we can show a lot more data - more freedom of playing with UI elements.

Open question:
I think it makes sense (with this UI in mind) to go with Leon's idea in T182885#3949875 to show participants, new editors and retention on per-wiki basis. This also came up in a one of the user interviews where a program organizer would like to keep track of which participants are working on which wikis.
I would however split that into a separate task if we decide to go with it.

Event Timeline

Nice idea! This data probably shouldn't be precomputed because that would involve a lot of storage. Looking at production Grant Metrics, aside from our test events, it seems the number of participants and pages improved for events is never that high. So as it stands now I think we'd be able to run these queries (pages created, etc.) on-demand without worry.

For some of these, we could provide other neat stats beyond just a raw list. For instance "pages created" could also show the date it was created and the author, and perhaps the assessment, similar to https://xtools.wmflabs.org/pages/en.wikipedia.org/L235

Nice idea! This data probably shouldn't be precomputed because that would involve a lot of storage. Looking at production Grant Metrics, aside from our test events, it seems the number of participants and pages improved for events is never that high. So as it stands now I think we'd be able to run these queries (pages created, etc.) on-demand without worry.

Yeah, I was thinking the same. I think the big question here is how do we show that data without the UI becoming too complex.

For some of these, we could provide other neat stats beyond just a raw list. For instance "pages created" could also show the date it was created and the author, and perhaps the assessment, similar to https://xtools.wmflabs.org/pages/en.wikipedia.org/L235

That's a great idea!

Niharika added a subscriber: Shouston_WMF.

@Shouston_WMF @MusikAnimal I added a mock and some details. What do you think?

I couldn't see how we can fit in the extra stats about page creation and users in the UI, with what we have currently though.

@Shouston_WMF @MusikAnimal I added a mock and some details. What do you think?

I couldn't see how we can fit in the extra stats about page creation and users in the UI, with what we have currently though.

Looks good! This would mean running all queries at once for a given wiki, which by itself I don't think is bad, but might want to run some performance tests...

If they in general only want to look at single stats (all the pages created for a given wiki), you might consider a modal popup. E.g. see https://tools.wmflabs.org/pageviews/meta and click on a "Application" name in the table. It runs the query to get all the unique projects/page loads and shows it in a modal. So we could do the same when clicking on a given statistic. This would also allot you the room to add more data, like the date the page was created and the author (this data specifically wouldn't slow down the query).

Totals row is expandable as well.

This goes against the whole system of precomputing the stats, which we've put a lot of work into. If we want this, we might as well store everything (page titles, users, etc.). At the very least, we'd need to put all of these queries into the job queue (and frankly the same for the revision browser, which apparently is the slowest). Otherwise I don't see us having any room for growth without asking for higher quota, which the DBAs don't appreciate =P

I still wonder where we draw the line for a simple, straightforward Grant Metrics versus feature-rich things you can get with the Dashboard.

@MusikAnimal I want to clarify that everything I wrote is up for discussion and debate. I want your input on everything. :)

@Shouston_WMF @MusikAnimal I added a mock and some details. What do you think?

I couldn't see how we can fit in the extra stats about page creation and users in the UI, with what we have currently though.

Looks good! This would mean running all queries at once for a given wiki, which by itself I don't think is bad, but might want to run some performance tests...

If they in general only want to look at single stats (all the pages created for a given wiki), you might consider a modal popup. E.g. see https://tools.wmflabs.org/pageviews/meta and click on a "Application" name in the table. It runs the query to get all the unique projects/page loads and shows it in a modal. So we could do the same when clicking on a given statistic. This would also allot you the room to add more data, like the date the page was created and the author (this data specifically wouldn't slow down the query).

I thought about having a popup for showing the data. There is the obvious benefit of being able to show a lot more data like you said. But I think that will lead to a lot of repetitive popup opening. For example - if the organizer wants to know which wikis User:A and User:B are working on. They'd end up opening the popups and writing that down somewhere. If they have to compare data across wikis , that might be harder to do. Also, there will have to be a lot of popups. Say if the table has 15 rows or so - we get 30 popups to begin with (if we exclude users retained, new editors and participants).

About doing queries on one wiki - I imagined that it wouldn't take too long given how fast our queries against multiple projects are, but you're right that we should do some testing for it first.

Totals row is expandable as well.

This goes against the whole system of precomputing the stats, which we've put a lot of work into. If we want this, we might as well store everything (page titles, users, etc.). At the very least, we'd need to put all of these queries into the job queue (and frankly the same for the revision browser, which apparently is the slowest). Otherwise I don't see us having any room for growth without asking for higher quota, which the DBAs don't appreciate =P

Yeah, you're right. I didn't think about that. Okay, let's make totals row not expandable then.

I still wonder where we draw the line for a simple, straightforward Grant Metrics versus feature-rich things you can get with the Dashboard.

I don't think there is a clear line there. The gap we're trying to fill with Grant Metrics is for Wikimetrics. That's the current plan, to the best of my knowledge. :)

Okay, I added another mock. @MusikAnimal, what do you think of this one?

I personally like the first one more. With the second, we still run all the queries at once, but moreover, the username lists can be quite large and would pollute the interface. We also don't have the per-wiki totals at a glance (without having to expand), and you'd lose the table sorting capability. Also, not sure if the "active editors", and two separate retention thresholds are a new thing, or perhaps taken from an older mock? This may be slow to compute on-demand.

I realize event organizers are very data-hungry, but I still think we should give careful thought to performance. I still haven't done any tests, but I think this could go slow-ish... especially with the job queue (long-term necessary evil). Within the UI, we're merely talking about expanding a row in a table, something you wouldn't expect to take long. In my opinion extra clicks are OK if it means it will load quickly, and perhaps provide a cleaner UI. If organizers want to know which wikis a user is working on, I would somehow present this as a separate feature ("search by user", maybe), or offer some "sort by" options in the revision browser.

Can't they get all this data from the CSV export anyway, at least after running some equations (which we might could pre-supply)? Does Wikimetrics show the per-wiki raw data on demand? I guess I should actually spend some time seeing what that tool does =P

I'm glad we're having this discussion. We should think this through thoroughly and figure out exactly what stats we want, and how organizers prefer they be presented, and try to find a balance between performance, satisfying our users, and minimizing developer effort.

I guess the other big question is, long-term, how much time do we plan to devote to Grant Metrics?

Sorry for not responding earlier - I was out at training last week. Since I have carpal-tunnel I can't type a bunch, so let's talk about this at our meeting on Thursday :)

Here are a few thoughts based on what was discussed so far:

Instead of creating a whole new UI (or complicating one that we know works), I'm wondering if we could simply update the "View Data" section to show data by metric. Meaning, when you click "View All Data" you still get the edit table we currently have. But if you clicked on a metric, you would get a different table that shows the list of New Editors, or the list of pages created, etc. Would this make the querying / performance better, since you only show the table if the person clicks on the metric? (If you can't tell I'm all for extra clicks, as long as people know it's there to be clicked ;) )

For the participant metrics though, I still think we should keep them as totals, not by wiki. Looking at how the Dashboard handles this breakdown of a particular metric, they too don't distinguish by "home" wiki. From what I understood, I think what people were looking for when they clicked the metrics was a simple quick snapshot, not a place to do extra calculations (e.g. if they click on the New Editors metric, they get a page that just lists the new editors; if they click on the New Editor Retention metric, they get a graph of retention over time and the list of retained editors under that like in T189917: Make it possible to see retention over time (graph maybe)). Wikimetrics (when it works right for multiple wikis) does show raw wiki info, but the CSV is *hella* confusing, and I know a lot of people (including myself) who give up.

In general, I agree with @MusikAnimal that the CSV should be the most granular view of data (and tangent: I also think pre-populating with some calculations would make the CSV more useful).

In the Event Metrics expansion, we are creating two completely new reports, for Pages Created and Pages Improved. See the details here. For this reason I'm closing this ticket as no longer valid.