Page MenuHomePhabricator

Wikimedia Technical Conference 2018 Session - Improving frontend testing and logging
Open, Needs TriagePublic

Description

Session Themes and Topics

  • Theme: Enabling our engineers, staff, and volunteers to achieve our goals easier and faster
  • Topic: Testing

Session Leader

  • Brion Vibber

Facilitator

  • Josh Minor

Description

This session will examine our front-end testing infrastructure, with the goal of identifying gaps and areas for improvement. We will also discuss logging of client-side errors, and attempt to outline an acceptable solution for client-side error logging.

Questions to answer during this session

QuestionSignificance: Why is this question important? What is blocked by it remaining unanswered?
What works, and doesn’t work, about our front-end testing infrastructure? What are the known gaps or failure conditions in our current testing methodology?Before deciding what should be changed, it’s important to understand the current state of our front-end testing infrastructure. This will help to enumerate the issues, and the things that do not need to be addressed.
What tools or methods should we be looking at to enhance (or replace) parts of our front-end testing environment?Front-end testing has developed significantly over the last few years, including some advances that may not obviously apply to MediaWiki. This will help to identify how these changes apply.
What confidence do you have that a change that passes our existing tests will be safe to deploy to production? What changes or enhancements to our existing test methodology or coverage would make you more confident?This will expand on the lived experience of developers who are extending the front-end of MediaWiki, which will help to define priority for various enhancements.
Once front-end code has been deployed, how should we log errors? What are the tradeoffs of building/maintaining a client side error logging solution, versus integrating with existing solutions?Client side logging is the primary mechanism for identifying bugs that have been introduced to production. There are several ways that this type of logging could potentially be implemented, and we need to understand the tradeoffs involved.

Facilitator and Scribe notes

Facilitator reminders

Session Structure

  • Define session scope, clarify desired outcomes, present agenda
  • Discuss Focus Areas
    • Discuss and Adjust. ''Note that we are not trying to come to a final agreement, we are just prioritizing and assigning responsibilities!''
    • For each proposition [add etherpad link here]
      • Decides whether there is (mostly) agreement or disagreement and the proposition(s).
      • Decide whether there is more need for discussion on the topic, and how urgent or important that is.
      • Identify any open questions that need answering from others, and from who (product, ops, etc)
      • Decides who will drive the further discussion/decision process (ie: a four month deadline)
  • Discuss additional strategy questions [add etherpad link here]. For each question:
    • Decide whether it is considered important.
    • Discuss who should answer it.
    • Decide who will follow up on it.
  • Wrap up

Session Leaders please:

  • Add more details to this task description.
  • Coordinate any pre-event discussions (here on Phab, IRC, email, hangout, etc).
  • Outline the plan for discussing this topic at the event.
  • Optionally, include what it will not try to solve.
  • Update this task with summaries of any pre-event discussions.
  • Include ways for people not attending to be involved in discussions before the event and afterwards.

Post-event Summary:

Action items:

  • Interested parties should join QA SIG - https://www.mediawiki.org/wiki/Quality_Assurance_SIG
  • Provide per-deploy versioning info that can be used to filter logs property
  • Must decide whether to build on EventLogging or build on Sentry with our own infrastructure (sentry.io) (SIG)

Event Timeline

debt created this task.Oct 2 2018, 10:59 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 2 2018, 10:59 PM
kchapman renamed this task from Wikimedia Technical Conference 2018 Session - Improving the front end development process to Wikimedia Technical Conference 2018 Session - Improving frontend testing and logging.Oct 3 2018, 2:35 AM
debt updated the task description. (Show Details)Oct 4 2018, 11:44 PM
debt updated the task description. (Show Details)Oct 10 2018, 1:50 PM
debt added a subscriber: brion.
debt updated the task description. (Show Details)Oct 17 2018, 11:26 PM
debt added a subscriber: JMinor.
Quiddity updated the task description. (Show Details)Oct 20 2018, 12:49 AM
Tgr added a subscriber: Tgr.Oct 21 2018, 4:43 AM
brion updated the task description. (Show Details)Oct 25 2018, 11:23 PM
brion updated the task description. (Show Details)Oct 25 2018, 11:27 PM

Hello! We are starting to ramp up on session creation for the 2019 Wikimedia Technical Conference. If there is no longer anything remaining to do here please close this task to avoid confusion.