Page MenuHomePhabricator

MPIC: Determine language/copy for combined application
Closed, ResolvedPublic3 Estimated Story Points

Assigned To
Authored By
cjming
Aug 27 2024, 6:10 PM
Referenced Files
F57643383: Screenshot 2024-10-25 at 21.20.48.png
Oct 25 2024, 7:26 PM
F57643381: Screenshot 2024-10-25 at 21.19.28.png
Oct 25 2024, 7:26 PM
F57643109: Screenshot 2024-10-25 at 1.13.01 PM.png
Oct 25 2024, 5:15 PM
F57606261: image.png
Oct 11 2024, 1:13 PM
F57582275: image.png
Oct 2 2024, 10:07 AM
F57516099: Screenshot 2024-09-16 at 22.22.06.png
Sep 16 2024, 8:25 PM
F57514742: image.png
Sep 16 2024, 11:46 AM
F57464722: image.png
Sep 5 2024, 6:11 PM

Description

T370880

Description

In order for MPIC to manage both instrument and experiment configuration, the references to instrument should be updated to a more generic term (inclusive of experiments). We will still keep the basic functionality of the instrument form (which will retain the references to instrument) but in all the other user interfaces where we reference EITHER instrument or experiment, a new name/phrase will be needed.

Copy proposal

Only the text elements that still need to be adjusted in the context of the combined application are included in this table and illustrated in the screenshot below:

IDElementCurrent copyProposalObservations
1Tool's name (header)"Metrics Platform Instrument Configuration""Metrics Platform Experimentation Lab"Temporary until the team resumes naming activities
2Main page's title"Experimentation dashboard""Experimentation directory"Following the feedback that the term "Dashboard" has connotations of data visualization. "Directory" is suggested instead of "Registry" for being a slightly more common term in digital contexts
3Main page's description"Configure and manage your data collection instruments and A/B experiments""Review and manage your data collection instruments and A/B tests"This reduces the amount of mentions we do to "experiment" or "experimentation" on the UI. "Configure" is replaced by "Review" because configuration rather takes place in the forms, not in the main page
4Table action"Turn Baseline instrument on/off", "Turn A/B test on/off""Turn on/off"Using more generic copy makes the option clearer and more scalable. The context makes the action understandable
5Table action"Delete Baseline instrument", "Delete A/B test""Delete"Using more generic copy makes the option clearer and more scalable. The context makes the action understandable
6Detailed view's "Edit" action"Edit Baseline instrument" , "Edit A/B test""Edit"Using more generic copy makes the option more scalable. The context makes the action sufficiently understandable

image.png (1×2 px, 177 KB)
image.png (1×2 px, 83 KB)

What doesn't need to change as part of the scope of this task:

  • Status change/Deletion dialogs: e.g. "Are you sure you want to delete this Baseline instrument? This will permanently delete the Baseline instrument. You cannot undo this action". – Indicating the artifact type can bring clarity in this context, where the table's information is not available. For full clarity, it'd be a great idea to display the name of the artifact affected by the action. A new dialog copy proposal will be made in a separate task (TBD)
  • Status change/Deletion confirmation message: e.g. "The Baseline instrument {name} has been successfully updated" – Like with dialogs, the artifact type can be considered more necessary information in this context, given that the message is not positioned within or next to the table (users cannot easily see what type of artifact they deleted)

Changes that are being evaluated:

  • The use of the term "Baseline": During the ongoing usability testing round (T376120), at least 2 users (test outcomes are still being processed) expressed confusion around the term "baseline". They either mentioned that: 1. they weren't sure of the meaning of said term, or 2. that they didn't think it applied to all instrument use cases (i.e. instruments can be used to measure the outcome of making a change, which doesn't theoretically correspond to a baseline anymore on the user's opinion).

Observations

  • An original design proposal mapped some of the places where this new name/copy will need to be updated.

Acceptance Criteria

  • Design mocks are created/updated with new language/copy
  • Update application according to the mocks regarding the new language

Event Timeline

While I continue to work on a proposal, there's a foundational question that I need some help to clarify. Understanding this would be key to define the terminology and the tool's flows, so please allow me to double-check: Are MPIC users setting up experiments (A/B tests for now, but maybe cohort analysis or Multivariant tests in the future), or are they still “instrumenting” A/B tests (etc.)? Is the product of using the instrument v. the experiment configuration forms different? In which way?

My interpretation right now is that users are creating baseline and experimentation instruments, and that experimentation instruments can have diverse applications, one of which is A/B testing. So, in my head, I still group all outcomes under an “instrument” category, and just differentiate between the products based on their application (default measurement, experimentation). I'd be really grateful if someone could help me correct this most probable misconception 🙏🏻 Thank you!

Hello @VirginiaPoundstone, @Sfaci and @cjming 👋🏻 Sharing a first copy exploration here to spark the conversation. Please find a screenshot and descriptions below:

image.png (976×2 px, 122 KB)

A. "Experimentation Hub": A placeholder! With the introduction of experiment configuration capabilities, I assume that the name "Instrument configuration" does no longer convey the tool's purpose. What would be a new, more accurate name for MPIC? What are the user needs that the platform is looking to address long term? How do we chose a name that can still apply to the tool as its features evolve?

B. "Experimentation catalog": This might be a valid, broad enough option only if we're fine with the association of instruments to experimentation. If this is definitely not the case, and instruments won't commonly be connected to experiments, then we might want to consider alternatives. Just using "Experiments and instruments catalog" might also be fine if we don't want to risk causing confusion. I'm avoiding the use of terms that might imply complexity, like "Telemetry".

C. "Track and manage your data instruments and A/B experiments". It would be a nice touch to quickly describe the purpose of this section to users.

D, E. Experiments and Instruments tabs and specific table titles (related to T373466: MPIC: Determine user flow between instrument and experiment forms): I understand that this is not entirely aligned with our "be water" approach 😁 but tabs would be an organized way to structure information, differentiate clearly between types of artifacts and cater to the different use cases. Key questions: is the plan to provide capabilities to create other sorts of experiment? Otherwise, we could just refer to A/B experiment here (and remove the "Type" column).

Regarding form titles. I think it's safe to stick to "Launch new instrument" or "Modify instrument" on the one hand. For experimentation, it depends: if the focus is solely on A/B testing right now, the form title could very well say "Launch new A/B experiment" or "Modify A/B experiment".

As mentioned, feedback on this early exploration would be extremely welcome. Is there any information you noticed I'm not considering? Please share! Thank you 🙏🏻

Great work on this @Sarai-WMF! You managed to dig into the heart of the complex web we have woven.

A. "Experimentation Hub" I like it! We had a whole rename metrics platform project that got paused because it was a distraction and HARD. I really like your approach of sticking with Metrics Platform (it's a known entity), but calling this as a specific offering. It allows our work to expand over time as the interface for metrics. Or if it doesn't expand, we can build the identity around the new name and phase out metrics platform over time.

B. "Experimentation catalog" vs "Experiment and Instrumentation Catalog" Let's try the most simple one and see if people are confused. It's an easy thing to iterate on based on user feedback.

C. Really appreciate adding this descriptor. Minor tweak for clarification: "Track and manage your data collection instruments and A/B experiments".
Once we add additional types of experimentation capabilities we can drop the A/B, but for now it is a good signifier.

D, E. Experiments and Instruments tabs

is the plan to provide capabilities to create other sorts of experiment?

Yes, in the future we will be able to conduct more complex experiments like multivariate and funnel tests. But we are pretty far way from that now, that is future or even horizon work.

I like the tab idea. It may not get implemented right away depending on capacity, but it's a smart clean way to keep things organized.

Ok, new iteration proposal:

image.png (1×2 px, 184 KB)

A. Tool's name: This will require following-up on the naming discussions initiated by the team. For now, I suggest we go for one of the options that were suggested at the time, "METRICS PLATFORM Experimentation Lab", just as a placeholder.
B. Home page title: During a recent discussion, it was brought up that "Catalog" has a somewhat passive meaning attached to it. A term like "dashboard" conveys a more hands-on connotation, hopefully more aligned with the fact that users can not only track, but also manage their experiments by modifying their status or deleting them.
C. Description: Adjusted following Virginia's recommendation 🙏🏻
D. Experiment/instrument forms' entry point: Now signified by the two buttons with the copies "New A/B test" and "New baseline instrument" (See T373475: MPIC: Update application to include experiment user flow)
E. Table's title: Refers to both artifact types, like in the previous iteration. The "type" column in the table will allow differentiating between them.
F. Turn on/off menu option: At the moment, this action on the menu reads "Turn Instrument On/Turn Instrument Off" with the incorporation of experiments, we could go for a simpler, more generic option like "Turn off" and "Turn on". Thanks @Sfaci for the catch!

Questions:

  1. The terms 'experiment' and 'experimentation' appear perhaps too redundantly on the UI. This might improve if we set on a different name for the tool, but any suggestions to remove or replace any instances of these terms would be much appreciated.
  2. We would now use the terms 'experiment', 'A/B experiment' and 'A/B test' interchangeably, even though they have different levels of granularity. Is this acceptable?

Nice iteration on this @Sarai-WMF. As we flesh out the user flow overall, these wrinkles will get ironed out. So I agree, that we will continue to iterate on this.

B. dashboard has more connotation of data visualization... what about "experimentation registry" or "data collection registry"?

1.& 2. if we wanted to cut down on the number of times the terms 'experiment' and 'experimentation' show up, which I think we should, We could go with something like "data collection registry" and the description (C) could be simplified to: "Configure and manage your baseline instruments and A/B tests").

Part of what makes this tricky is the preexisting culture around instrumentation... in the future we won't really be talking about instruments so directly. We will be talking about metrics (that the baseline instruments measure) and experiments (which make use of instruments).

@mpopov and @Nettrom what do you think?

@Sarai-WMF Are we including in the scope of this ticket any message an user can find while using the MPIC UI? For example, every time a user launch/update/enable/disable/delete an instrument/experiment there is a message that confirms that action to the user. At this time we are using a Message component when registering/updating something and, in addition to that, we are showing some non-codex Toast messages for these and other actions with messages like "The instrument has been enabled/deleted/......".

Should we use always the same message structure we are using for the Message. We show something like The baseline instrument test Desktop UI Interactions has been successfully updated. (screenshot below)

Screenshot 2024-09-16 at 22.22.06.png (76×1 px, 10 KB)

Thanks for all the feedback, @VirginiaPoundstone! I'll reflect those changes in the new version 💯

Hey @Sfaci. Let's hear from Virginia about the scope, but I would say that it sounds like a good idea to adjust all copy as part of the same effort?
The message structure sounds quite right to me! 👍🏻 It would just be a matter of shifting the sentence's object depending on the scenario.

Sarai-WMF updated the task description. (Show Details)
Milimetric updated Other Assignee, added: Sarai-WMF.
Milimetric added a subscriber: Sarai-WMF.

@Sarai-WMF it looks good! I'll create the engineering ticket based on your work here. Thanks!

Sfaci updated Other Assignee, removed: Sarai-WMF.

Hi @VirginiaPoundstone! It'd be great to get your review on this proposal. Thank you! 🙏🏻
Regarding logistics, @Sfaci and I were discussing whether we could save some time by adjusting and reusing this task as an implementation ticket. What do you think?

Sfaci updated the task description. (Show Details)

Based on our discussion I have rewritten the engineering AC (from creating a separate ticket to implement using this) and I have moved this task to the backlog so someone can pick it to start working on

The task description was updated to document the need to also change the "Edit" option included in the experiments' and instruments' Detailed view page (see 6). Big thanks to @Sfaci for the catch and quick fix 🙏🏻

Sfaci set the point value for this task to 3.Oct 14 2024, 2:28 PM

Change #1080816 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/deployment-charts@master] Metrics Platform Instrument Configuration: Deploying to staging

https://gerrit.wikimedia.org/r/1080816

Change #1080817 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/deployment-charts@master] Metrics Platform Instrument Configuration: Deploying to production

https://gerrit.wikimedia.org/r/1080817

Change #1080817 merged by jenkins-bot:

[operations/deployment-charts@master] Metrics Platform Instrument Configuration: Deploying to production

https://gerrit.wikimedia.org/r/1080817

Change #1080816 merged by jenkins-bot:

[operations/deployment-charts@master] Metrics Platform Instrument Configuration: Deploying to staging

https://gerrit.wikimedia.org/r/1080816

  1. two notes for the header:

a. the title is stacked in the design, can we match it?
b. I would like to add a clear indicator to the header that this is experimental software. Maybe the cdxIconLabFlask? And/or a tag that says "alpha" similar to the beta tag on codex?

Screenshot 2024-10-25 at 1.13.01 PM.png (62×208 px, 5 KB)

@Sarai-WMF can you help with this after the copy change work?

  1. Based on definition is https://www.mediawiki.org/wiki/Product_Analytics/Glossary, Let's try "Instrumentation Directory" for the heading.

a. the title is stacked in the design, can we match it?
b. I would like to add a clear indicator to the header that this is experimental software. Maybe the cdxIconLabFlask? And/or a tag that says "alpha" similar to the beta tag on codex?

a. This is specified as part of the header task, so it should probably be covered there: T374979: MPIC: Update app header

b. The fastest way to add the indicator would be to use a notice InfoChip component with the cdxIconLabFlask. I'm not sure it looks great with our logo. Thoughts on adjustments welcome:

Screenshot 2024-10-25 at 21.19.28.png (356×1 px, 45 KB)

Screenshot 2024-10-25 at 21.20.48.png (254×1 px, 29 KB)

I can live with an imperfect InfoChip for now. Let's do it!
Thank you for mocking it up @Sarai-WMF

CC @Sfaci and @cjming