I think that EventLogging could be made to work as the data collection backend for a gadget like this. In addition to feedback results we may want to have a "gadget shown" type of data point collected that we could use to determine percentage of users giving any feedback at all. This would be useful in attempts to unbias the data we receive (I am assuming that response rate will be low generally and biased towards negative feedback).

Once the data is captured by EventLogging we will need to do additional work to turn the raw data into something that we can visualize and act on, but I would consider that a second phase of the project. Until we can get some data we don't need to be too concerned about visualizations, etc.

For background research, I looked into existing gadgets or extensions around rating/voting and found a couple interesting ones:

• Extension:ArticleFeedback5 - Made a lot of progress with different iterations before it was disabled on Wikimedia wikis in ~2013. The extension was disabled a) due to philosophical reasons and b) there weren’t more resources to put in development efforts. It supported anonymous data collection and for that relied heavily on AbuseFliter extension. The work on AbuseFilter was on halt for a while. Also, it wasn't clear how to handle the amount of data collected via the extension and how to use it. There were some major bugs with the archiving process too.

• Extension:VoteNy - Allows voting on articles. The design might need a bit of tweak, but it is exactly what we need. The voting widget can be embedded on a page with a wiki markup. <vote /> outputs a green square voting box and <vote type=1 /> outputs five voting stars.

From work on ArticleFeedback, it seems like it might be too complicated to collect anonymous qualitative feedback, and to archive and process it. The simple solution would be a thumbs up and thumbs down button that allows (un)voting. We can also display number of votes that an article has received. If a user clicks on thumbs down button, we redirect them to provide feedback on the talk page.

EventLogging seems like a robust approach to log data entries, JSON schema, probably easy to update entries, but the data can only be stored for ~90 days. I am inclined towards using Extension:VoteNy, as it is functional and works. But, also wondering what would be the advantage of using EventLogging over database logging and extension over gadget for this purpose? How about the idea of using a customized version of VoteNy on Wikitech, to begin?

In T195119#4798468, @srishakatux wrote:

EventLogging seems like a robust approach to log data entries, JSON schema, probably easy to update entries, but the data can only be stored for ~90 days. I am inclined towards using Extension:VoteNy, as it is functional and works. But, also wondering what would be the advantage of using EventLogging over database logging and extension over gadget for this purpose? How about the idea of using a customized version of VoteNy on Wikitech, to begin?

First, thank you for doing some "build vs buy" research. It is always a good idea to stop and think about how to reuse or extend existing things before diving into building new things from scratch.

Deploying a new extension to a Wikimedia wiki is not a trivial undertaking. "A customized version" also seems to imply a fork of the existing project which is almost never a good idea in my opinion either.

Here are my reasons for suggesting a gadget + eventlogging approach for the initial project:

• EventLogging works as a data collection pipeline and as pointed out in T195119#4798468 already deals with cleaning up historical data which I consider to be a feature rather than a problem
• Gadgets do not require security review and are easy to tweak and to uninstall
• We really do not know what we will settle on for the UI/UX of this and would like to be able to experiment
• We don't need large amounts of historically data, most of what we will be interested in is recent data and historical trends

The gadget + eventlogging parts are only a partial solution. We would still need to build some backend scripts against the eventlogging data store to produce whatever reporting we need to analyze the votes. What kind of reporting we want/need should be fleshed out a bit more before anything is built as well so we can test the assumption that any of these things would/could work. I think we should consider some of the lessons of ArticleFeedback in that collecting anonymous text feedback could be problematic. It might be a much better idea to have the gadget make it easy/obvious to add suggestions for improving a page to its talk page or a central talk page that might make responding to feedback easier and more public.

@bd808 This is a helpful summary! Also, learned from Gilles that their team used QuickSurveys to understand site performance on wikis: T188503: Identify two wikis to run a research study and get approval from their respective communities

It looks like QuickSurveys is easy to use, but the constraint is that for asking questions we can only stick to a specific format. It looks like our initial survey format is going to be similar to the one followed by the performance team: one question with a yes and no choice. If not, there are Microsurveys too. Plus, it relies on EventLogging and we can use it to host external surveys via Google/Qualtrics service.

Though, in its current state and as per @Quiddity's comment T194099#4189182, we are able to show the survey to % of users, we could consider enhancing the extension’s capabilities to add support for specific pages.

Not trying to be lazy (:P). Any opinions on this?

In T195119#4806572, @srishakatux wrote:

we could consider enhancing the extension’s capabilities to add support for specific pages.

We could, but again this is high cost development (changing an existing deployed extension to serve a new use case) rather than the hoped for 'light' development cost of a gadget that can be changed with relatively little oversight or friction with other use cases. I haven't looked at the implementation of QuickSurveys to see how invasive the changes would need to be to make it something that we could configure to present to 100% of people looking at pages in the Help namespace on Wikitech, but since this deviates in multiple ways from the documented use of the extension I am assuming that it is non-trivial.

Update > User script is ready and is in testing mode. Some technical details:

• Feedback form is displayed at the very bottom of a content page that is in a namespace supported by the script. Feedback form screenshots are here.
• For a “thumbs-up” or “thumbs-down” vote:
  • User’s vote gets recorded as per Schema:UserFeedback
  • Cookie set with vote value and expire in 10 minutes. A user can vote again only after cookie clears.
  • Confirmation messages get shown; and in case of a thumbs-down vote, a user is being asked to provide more feedback on the article's talk page.

Current status: Tried testing on the EventLogging devserver and reported the errors I ran into here T153641#48252231.

Interesting external example of a "Help and feedback" footer section: 4 tabs, each with 2-3 entries - https://docs.gitlab.com/ee/development/i18n/translation.html#help-and-feedback