Page MenuHomePhabricator

[EPIC] Technical plan for Wishlist Survey
Open, Needs TriagePublic

Description

This task documents on a high-level how all the pieces of the Future of the Wishlist project tie together. This includes the intake form, voting, the bot, and on-wiki templates and pages.

This all subject to debate.

Technical goals

  • Write our code so that could easily be moved to an extension (if/when that happens)
  • The survey should work on any MediaWiki installation, especially local environments
  • We should be able to develop and test without deploying as a gadget, i.e. wishlist-intake is essentially loaded like a user script.
  • Use the new package feature of MediaWiki-extensions-Gadgets so that production has separate JS pages for each component. This should not complicate local development in any way.
  • "Installation" of the survey should be as simple as using Special:Import with an XML file (or running a script) to create the various pages, install the gadget with the deploy script, and starting the bot
    • On local envs, this can all happen with a single command
  • The bot should be written in Node (so it can live in the same repo as wishlist-intake) and contributors with a username other than MusikAnimal should also be able to contribute to it (no more Ruby / MusikBot framework) (T361067)
  • Try to get everything that is needed into the same git repo
  • Everything should be highly configurable i.e. setting the wishlist root to "Community Wishlist", and defining the "focus areas" (formerly known as categories) should live as a JSON config file, TBD if this lives on-wiki or in the repo (n.b. T363229)
  • The bot goes off this same config, and will live in the same repo as a script so that you can run it during development on your local MW installation
  • All the wishlist pages, templates, and other on-wiki content should be in version control, and the deploy script handles shipping them (with CLI flags to control what gets shipped)
    • This is mainly for local envs and new wishlist installations. Production wiki pages can be edited by anyone, so after initial deployment we likely won't touch normal wikitext pages again.

Nice to haves

  • Make the wishlist database-driven, build an API for it and let the gadget generate dynamic content instead of the bot editing those pages.

Workflow

  • User browses to the survey home (T363241)
  • User clicks button to create a proposal
  • User is redirected to Special:CreateProposal where the gadget will create a full-page form (no dialogs or anything!)
  • Upon submission (T362761, T363223), the user is redirected to the proposal page that was created, i.e. Community Wishlist/My_proposal
    • Note we no longer will put proposals as a subpage of a category
  • If/when desired, use the CWS Manager gadget to set up the proposal with MediaWiki-extensions-Translate
  • Staff will add the proposal to a focus area (formerly known as proposal categories)
    • TBD how -- maybe a field in the intake form shown only to staff (?)
  • Users can edit a proposal using the normal edit links, which will be intercepted and the user redirected to Special:EditProposal/My_proposal using the same intake form
  • Users can browse to various other pages such as focus areas (T363240,

Meanwhile:

  • The bot (T361067) picks up that a new proposal was created and updates the corresponding lists and count pages, just like in older surveys
    • It also process changes to any existing proposals, and updates counts accordingly
    • If time allows: the bot should instead write the data to a database
  • Pages that list wishes or have dynamic content (T363241, T363236, T363237, maybe T363240) are managed by the bot
    • If time allows: the content can instead populated by the gadget, pulling data from an API that reads from the bot's database (note we will get complaints about CSP warnings for external API calls)

Voting works just like it has in the past. Focus areas will also be vote-able.

Roadmap

The durations shown are conservative estimates of implementation time for a single engineer.

  • Implement wishlist intake dev environment (~1 week)
  • Finish up intake form, with or without submission (3-5 weeks)
  • Implement module to parse proposals -- to be used by the bot and (possibly) the gadget (1-3 days)
  • T362809 Create and package survey pages as XML dump and/or script (~1 week)
  • T361067 Rewrite bot to work on new survey structure (1-2 weeks)
  • Update CWS Manager as needed (~1 week)
  • Rework voting gadget as needed, or merge into intake gadget (1-3 days)
  • Rework AbuseFilters (1-2 days)

From here we go one of two routes:

Bot-powered approach (the planned route for now)

  • Make the bot populate the dynamic content on focus areas and index pages (3-5 days)
  • No changes needed to gadget for this

Database driven approach (if we have time)

  • Expand bot to write to database (1-3 days)
  • Write micro app with REST API to serve the data (~1 week)
  • Write modules to populate dynamic content on focus area and index pages (1-2 weeks)

In the future

The future of the Future of the Wishlist

  • Any production wiki should be able to install the survey software (again with minimal effort), and all proposals etc. still go to Meta. I.e. we'd make use of mw.ForeignApi.
  • Survey dashboard with fancy visualizations and such.
  • Possibly move everything to a proper MediaWiki extension.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
MusikAnimal updated the task description. (Show Details)
MusikAnimal updated the task description. (Show Details)

For T362761 I'm experimenting with how to load the form for a given page (or a new page). Currently wondering if it'd be good to an action entry point to the form, so that e.g. /Wishlist_Survey/Propsoals/Lorem_ipsum?action=wishlistedit can be used to edit or create the Lorem_ipsum page. The issue I'm wondering about is how to handle the Title field, given that it's part of the page name — if we allow it to be edited (other than during the initial proposal creation) then we'll have to implement a page move etc. as part of the proposal-saving process, and this feels a bit complicated.

Actually, my assumption above is wrong isn't it? We do want to include the title parameter in the template, and have it be independent of the page title, because it'll be translated. The page title will (I think?) always be in English, but could be different to the Wish Title either for clarity or translation purposes, and so there's no need for the intake form to worry about renaming the page if the Wish Title changes.

Actually, my assumption above is wrong isn't it? We do want to include the title parameter in the template, and have it be independent of the page title, because it'll be translated. The page title will (I think?) always be in English, but could be different to the Wish Title either for clarity or translation purposes, and so there's no need for the intake form to worry about renaming the page if the Wish Title changes.

Correct! No need for intake-form to handle page moves, but this may still be done by either the community or with CWS Manager, and I'm sure it will given $wgRestrictDisplayTitle on our cluster prevents us from forcing a specific page title. If the titles changes too much then a page move is likely to happen, and that's okay.