Page MenuHomePhabricator

Develop EventLogging schema for documentation feedback gadget
Closed, ResolvedPublic

Description

What's requested: Review the Eventlogging schema designed for documentation feedback gadget. See T195119: Create user feedback gadget for technical documentation on Wikitech pages. The first version of the gadget would display a user feedback form with a "YES" and "No" button and the question: Was this documentation helpful? The plan is to deploy the Gadget on Wikitech/MediaWiki. Here is the first draft of the schema: https://www.mediawiki.org/wiki/User:SSethi_(WMF)/Sandbox/user_feedback_schema.json

Why it's requested: From the EventLogging/Guide, it seems like it might be a good idea to get the schema reviewed from an analyst and ensure there aren't any privacy concerns associated with the data we are planning on collecting, format is right, and the metrics we are currently considering (as listed in the schema) are sufficient and will help us understand whether or not the technical documentation is useful to our users.

When it's requested: There isn't a hard deadline, but ideally, we have the schema reviewed by early January, that's when we might be interested in deploying the Gadget on Wikitech.

Any other helpful information:
The script for the gadget is almost ready here in case that might help understand what the flow for this process would be like.

Event Timeline

srishakatux triaged this task as Medium priority.Dec 10 2018, 10:26 PM
srishakatux created this task.

Ideally, we want to collect users votes on a document and page ids associated with it. Based on this, I've created a first draft of the schema here: https://www.mediawiki.org/wiki/User:SSethi_(WMF)/Sandbox/user_feedback_schema.json.

Is this the right approach? Is there anything that I'm missing here?

The script for the gadget is getting ready here in case that might help understand what the flow for this process would be like cc @chelsyx

Hey @srishakatux , to help us prioritize this task, can you provide and organize the task description in the following format?

  • What's requested. If you know what data you want, be specific! If you have a question but don't know how it can be answered, say what you've already tried.
  • Why it's requested. You don't have to write an essay, but give enough context that the analyst can interpret, adapt, and prioritize your request.
  • When it's requested. If you have a hard deadline, say what it is and what it's tied to.
  • Any other helpful information, like relevant documentation.

See "How to request analysis" section at https://phabricator.wikimedia.org/project/profile/3313/

srishakatux updated the task description. (Show Details)Dec 11 2018, 8:17 PM

Thanks @chelsyx for sharing this information! I've updated the task description accordingly.

fdans added a subscriber: fdans.Dec 13 2018, 6:03 PM

If you need any data from this schema to be kept beyond the 90 day window, you need to add it to the whitelist:

https://github.com/wikimedia/analytics-refinery/blob/master/static_data/eventlogging/whitelist.yaml

fdans moved this task from Incoming to Radar on the Analytics board.Dec 13 2018, 6:03 PM

@fdans Thanks! We don't need to keep the data beyond 90-day window for now. Does the schema look good to you and is it ready to be published on meta-wiki and tested?

fdans added a comment.Dec 14 2018, 7:46 AM

@srishakatux we reviewed the schema during our tasking meeting and it looks good! Feel free to put it in metawiki.

Nuria added a subscriber: Nuria.EditedDec 14 2018, 10:34 PM

@srishakatux something to think about is overall participation versus overall impressions of survey. Example: "how many people have seen this question and have not answered" this will help you weight the data a bit better as 90% of "no" answers when only 1% of people have actually answered question is of little meaning . 90% of "yes" answers with 90% participation (impression-wise) is however very meaningful. Hopefully this makes sense, let us know otherwise.

Aklapper edited projects, added Analytics-Radar; removed Analytics.Jun 10 2020, 6:44 AM