Session Themes and Topics
- Theme: Enabling our engineers, staff, and volunteers to achieve our goals easier and faster
- Topic: Testing
Session Leader
- Brion Vibber
Facilitator
- Josh Minor
Description
This session will examine our front-end testing infrastructure, with the goal of identifying gaps and areas for improvement. We will also discuss logging of client-side errors, and attempt to outline an acceptable solution for client-side error logging.
Questions to answer during this session
| Question | Significance: Why is this question important? What is blocked by it remaining unanswered? |
| What works, and doesn’t work, about our front-end testing infrastructure? What are the known gaps or failure conditions in our current testing methodology? | Before deciding what should be changed, it’s important to understand the current state of our front-end testing infrastructure. This will help to enumerate the issues, and the things that do not need to be addressed. |
| What tools or methods should we be looking at to enhance (or replace) parts of our front-end testing environment? | Front-end testing has developed significantly over the last few years, including some advances that may not obviously apply to MediaWiki. This will help to identify how these changes apply. |
| What confidence do you have that a change that passes our existing tests will be safe to deploy to production? What changes or enhancements to our existing test methodology or coverage would make you more confident? | This will expand on the lived experience of developers who are extending the front-end of MediaWiki, which will help to define priority for various enhancements. |
| Once front-end code has been deployed, how should we log errors? What are the tradeoffs of building/maintaining a client side error logging solution, versus integrating with existing solutions? | Client side logging is the primary mechanism for identifying bugs that have been introduced to production. There are several ways that this type of logging could potentially be implemented, and we need to understand the tradeoffs involved. |
Facilitator and Scribe notes
Facilitator reminders
Session Structure
- Define session scope, clarify desired outcomes, present agenda
- Discuss Focus Areas
- Discuss and Adjust. ''Note that we are not trying to come to a final agreement, we are just prioritizing and assigning responsibilities!''
- For each proposition [add etherpad link here]
- Decides whether there is (mostly) agreement or disagreement and the proposition(s).
- Decide whether there is more need for discussion on the topic, and how urgent or important that is.
- Identify any open questions that need answering from others, and from who (product, ops, etc)
- Decides who will drive the further discussion/decision process (ie: a four month deadline)
- Discuss additional strategy questions [add etherpad link here]. For each question:
- Decide whether it is considered important.
- Discuss who should answer it.
- Decide who will follow up on it.
- Wrap up
Session Leaders please:
- Add more details to this task description.
- Coordinate any pre-event discussions (here on Phab, IRC, email, hangout, etc).
- Outline the plan for discussing this topic at the event.
- Optionally, include what it will not try to solve.
- Update this task with summaries of any pre-event discussions.
- Include ways for people not attending to be involved in discussions before the event and afterwards.
Post-event Summary:
Action items:
- Interested parties should join QA SIG - https://www.mediawiki.org/wiki/Quality_Assurance_SIG
- Provide per-deploy versioning info that can be used to filter logs property
- Must decide whether to build on EventLogging or build on Sentry with our own infrastructure (sentry.io) (SIG)