Page MenuHomePhabricator

UX Testing Environment (Tracking) {panda}
Closed, DeclinedPublic

Description

Test MVP
What: 1-5 basic mediawiki test instances (on WMF Labs) to test between UserZoom/loop11
Needed:

Production Testing
User requirements:

  • given 100 snapshot test instances
  • email 100 users who have opted in to testing
  • distribute users to their own test servers
  • know when links are clicked (email)
  • know when tasks are done (funnel)

Tech requirements:

  • way to expire email links for testing
  • snapshots should be up-to-date with production on a weekly(?) basis
  • wipe instances on a daily/weekly/monthly basis
    • instances could wipe after task is done?
      • to do this, you'd need a lot more logic somewhere, and that is probably something that the ex-Growth team would be able to tell you if it's possible or not. Realistically, it's just time based.
  • be able to include specific non-production patches in snapshots
    • what kind of patches?
  • include tracking js (UserZoom/loop11) probably as an extension
    • maybe, it might be easier than an extension, but I'll (Greg) defer to someone else.

Revisions and Commits

Event Timeline

greg raised the priority of this task from to Medium.
greg updated the task description. (Show Details)
greg added a project: Release-Engineering-Team.
greg moved this task to INBOX on the Release-Engineering-Team board.
greg changed Security from none to None.
greg added subscribers: greg, Jaredzimmerman-WMF.

Making a comment instead of just editing the description (so people can more easily follow along):

A full copy of enwiki isn't possible for the test MVP instances (5) on WMF labs. We need a subset. Someone needs to define that subset for us. Please make it as small as possible but still meet the needs (which is just to test basic functionality).

@greg I'm not sure how to define a subset that is meaningful (or rather how to define a subset in a meaningful and understandable way) do you have a recommendation?

Can you take a look at the Proposed Tasks section of https://docs.google.com/a/wikimedia.org/document/d/1YKaCwhR_RW-iCe6zJvuST5NBVP01eE_seNyc-v7rvxs/edit#heading=h.2fxajlbxqbi1

and based on those tasks propose a subset that would make sense, or be usable, e.g. needs functioning search, categories, history, article content etc.

As I'm not the researcher doing the testing, I'm going to have to request the UI/UX team do that part :) (I'm likely to miss some nuance that you're shooting for). My suggestion:

  1. Determine which subset of those proposed tasks you absolutely need to test the two providers on the 1-5 instances ("test MVP")
  2. Then, go through those tasks and determine what minimum of content you need to test them.

After that, we need to talk with someone from Ops to get that information exported and imported correctly.

For the full on "Production Testing" requirements we'll be able to do more.

Jaredzimmerman-WMF raised the priority of this task from Medium to High.
Jaredzimmerman-WMF removed a subscriber: aripstra.
greg lowered the priority of this task from High to Medium.Oct 28 2014, 11:26 PM

(sorry Jared, just lowered prio since this is on my team's workboard. If you want to make a board for your team and add it there, that's fine. But this is not a high priority for us at this time.)

@greg I guess there is no concept of a task being high priority for one team and lower for another.

@Jaredzimmerman-WMF Yeah, for now. But we could make this task (the tracking one) in a Design/UX project board and then put the appropriate sub-tasks in the appropriate workboards for other teams/projects (including Release-Engineering-Team ). Do you all have a project yet?

@greg Re: "subset" - are there any docs giving examples/details of what previous people/teams have used? Partially for the sake of this particular task, but also because I've long been curious as to what subset of content exists on test, test2, beta, etc. and what could potentially be requested at other test-instances.
I have no idea whether a "subset" implies 10, or 500, or 10,000 pages! Nor if there are obvious-in-hindsight pros/cons to selecting certain subsets. E.g. One of my first hypothetical suggestions (given what Jared asks for above), would be "Everything 2 degrees away from Arctic tern" (a recently Featured-class article) but that's probably in the 10,000 range (or more).

(I tried a bit of searching, and found nothing useful at wikitech or test/test2/beta, and way too many results at mw.o, so possibly it's already documented somewhere? Nothing at the obvious suspects, like [[mw:Beta cluster]] or [[mw:Test wikis]])
Thanks.

greg raised the priority of this task from Medium to High.Oct 29 2014, 5:16 PM
greg removed a project: Release-Engineering-Team.

(Removed the releng project and reset to High, since this task is more of a task for UX to drive, it's now project-less, but that's ok until UX/Design create a project here.)

Maybe "UX-Testing-Environment" deserves a project on its on? Is this a short one-off shot, or an actual project?

@Qgil I don't know what scale something has to have to get a "project" ;) if it becomes a Q3 goal, probably.

This is more about semantics ;) than about sale. If it is a project, then it deserves a Phabricator project. If it's just a group of tasks belonging to something bigger, then not.

It is also probably a matter of timing. If this is just preliminary work to propose a project, then not yet, if you are certain that you will work on this and the only question is when and with how much resources, then yes.

We are defining the guidelines as we go. In case of doubt, I would create a project. They are cheap, and are better than having tasks in the pool of orphans (as these tasks are now).

If we're making a project it should be "State of the User" UX testing
environment is an aspect of that larger project.

epriestley closed this task as Resolved by committing Unknown Object (Diffusion Commit).Mar 4 2015, 8:18 AM
epriestley added a commit: Unknown Object (Diffusion Commit).

(Mirroring of Phab's code seems to be closing local bugs that match bug numbers with upstream issues. Alerting the Phab team.)

Related tasks (precursors to this task):

This task is blocked until we get an engineering resource

Related tasks (precursors to this task):

So should those be listed as blocking this task under "Blocked By" here, or are they "just" related?

@Aklapper not sure. This task/project has been significantly re-scoped post-reorg. I'll follow up with the team at our next backlog grooming session and update accordingly.

aripstra renamed this task from UX Testing Environment (Tracking) to UX Testing Environment (Tracking) {panda}.Dec 18 2015, 6:07 PM