Change Details

An idea from the RelEng team day at the Dec 2024 offsite: track what tests get run for a given code change. We know that in many cases we run the same tests for the same pieces of code. What we don't know is whether the configuration for those tests is materially different or if we're running the same tests in the same way multiple times. Details: - It takes less than 10 minutes to branch MediaWiki -- We know this because we create a `wmf/next` branch nightly—`wmf/next` is equivalent to the new version we branch each week for the train, except it happens nightly - It takes about 30 minutes for the branching job to complete - This is due to the 20 minutes spent waiting for tests to complete - If these tests are unnecessary, we'd like to avoid running those tests - The first step in knowing if those tests are unnecessary is monitoring Acceptance criteria: - Build a plan for collecting information about test runs for a given patchset in a granular way - The plan should include what information we need to collect and how to collect it against a unique identifier and how to query it in a reasonable way.

An idea from the RelEng team day at the Dec 2024 offsite: track what tests get run for a given code change. We know that in many cases we run the same tests for the same pieces of code. What we don't know is whether the configuration for those tests is materially different or if we're running the same tests in the same way multiple times. Details: - It takes less than 10 minutes to branch MediaWiki -- We know this because we create a `wmf/next` branch nightly—`wmf/next` is equivalent to the new version we branch each week for the train, except it happens nightly - It takes about 30 minutes for the branching job to complete - This is due to the 20 minutes spent waiting for tests to complete - If these tests are unnecessary, we'd like to avoid running those tests - The first step in knowing if those tests are unnecessary is monitoring Acceptance criteria: - Build a plan for collecting information about test runs for a given patchset in a granular way - The plan should include what information we need to collect and how to collect it against a unique identifier and how to query it in a reasonable way. - Stretch: tracking how many hits we get against a particular hashed key

An idea from the RelEng team day at the Dec 2024 offsite: track what tests get run for a given code change. We know that in many cases we run the same tests for the same pieces of code. What we don't know is whether the configuration for those tests is materially different or if we're running the same tests in the same way multiple times. Details: - It takes less than 10 minutes to branch MediaWiki -- We know this because we create a `wmf/next` branch nightly—`wmf/next` is equivalent to the new version we branch each week for the train, except it happens nightly - It takes about 30 minutes for the branching job to complete - This is due to the 20 minutes spent waiting for tests to complete - If these tests are unnecessary, we'd like to avoid running those tests - The first step in knowing if those tests are unnecessary is monitoring Acceptance criteria: - Build a plan for collecting information about test runs for a given patchset in a granular way - The plan should include what information we need to collect and how to collect it against a unique identifier and how to query it in a reasonable way. - Stretch: tracking how many hits we get against a particular hashed key