• Any streams that were submitted to by mw.eventLog.dispatch() and associated datasets are marked as deprecated in DataHub

In T411356#11425050, @JVanderhoop-WMF wrote:

What is the impact of not doing? What teams would this block (and in what time frame)?

UploadWizard uses EventLogging to validate content on a wiki page against a schema. The Experiment Platform team would very much prefer this not to be the case (see T317793: [EPIC] Deprecate EventLogging::schemaValidate()). I would be happy to work with the maintainers of the extension to use the justinrainbow/json-schema or opis/json-schema, both of which are used in production.

In T411677#11433160, @mpopov wrote:

So maybe we don't produce these automatically but do require developers to produce their own, because they would know where to produce them from in their feature code. I'm just worried about relying on developers to remember this, but at the same time I can envision problems with trying to automate it. Particularly getting false positives – logging exposure when there hasn't actually been one.

@JVanderhoop-WMF: I've taken a quick swing at what parts of the platform I consider stable and unstable. I was going to hand over the list to the team and ask them for more input. Cool?

In T405716#11425101, @JVanderhoop-WMF wrote:

@phuedx Should this happen after renaming?

The Test Kitchen SDK is being rewritten in Kotlin and will track the JS and PHP Test Kitchen SDKs.

In T366807#11425118, @JVanderhoop-WMF wrote:

@mpopov Is this still relevant with mw.xLab.getInstrument( ) ?

I don't think that the complications around when to enroll search sessions in experiments highlighted above requires any significant changes to our notion of an Enrollment Authority or the conclusions in the task description. Rather, it means that we need to work closely with the Search team in order to ship an Enrollment Coordinator for search sessions that allows Product teams to experiment on the various autocomplete implementations and the Search team to experiment on the backend. I'll create a new task.

In T408133#11388078, @EBernhardson wrote:

In T408133#11370867, @phuedx wrote:

2. Is the experiment enrollment always performed by the same code path in these scenarios?

Sort of? The enrollment decision is always made in the CirrusSearch\UserTestingEngine class. The typical entrypoint is via invoking CirrusSearch\UserTestingStatus::getInstance() which happens during search engine initialization. The main variance is how the user gets to that code path. Special:Search, as mentioned above, embeds the enrollment decision into the page java script variables, whereas autocomplete has to invoke a special configuration api, as we don't have a pleasant way to inject custom values into the api responses, and even if we did there are multiple javascript autocomplete implementations and they would all have to be adjusted to give us access to that raw response.

In T407642#11411420, @JVanderhoop-WMF wrote:

Reviewed! Some questions in there. Who else from DPE should review? @GGoncalves-WMF ?

In T407804#11413581, @Sfaci wrote:

• @phuedx: At https://wikitech.wikimedia.org/wiki/Test_Kitchen/Troubleshooting and https://wikitech.wikimedia.org/wiki/Test_Kitchen/Architecture we are using something similar to https://miro.com/app/board/uXjVMtGrgVc=/?moveToWidget=3458764610125332534&cot=14. Is that the official one we should change to use there?

In T407804#11409499, @Sfaci wrote:

• Experimentation Lab -> TestKitchen

The fix has been deployed and the validation error rate is returning to normal:

Thanks @dr0ptp4kt for the additional info. The patch above should fix this for now, while we work on the correct fix for the issue.

In T408142#11400662, @STran wrote:

-Next steps would be to deprecate our custom event schema, update the ingest code to pipe to the default schema.

In T384506#11400635, @Sfaci wrote:

@phuedx @Milimetric Can I assume that the replacement is the following?

• mw.log() -> console.log
• mw.log.warn()-> console.warn()
• mw.log.deprecate() -> console.warn()
• mw.log.error() -> console.error()

In T267217#11393091, @Milimetric wrote:

sounds like we'd need to re-vet Edge Uniques design if we derive more IDs from it.

In T409965#11373906, @Sfaci wrote:

Something like adding some header in the response with the experiment and variant where the request has been enrolled so that consumers know when their request are running as a part of an A/B test.

In T405249#11373456, @Sfaci wrote:

I can see a few cases where page_namespace is still spelled as page_namespace in some json files that contain some configuration for java test cases. Aren't they relevant?

In T406261#11373309, @phuedx wrote:

@mpopov: Earlier this year, we were thinking about Clickthrough Rate and "Page Lifecycle". The latter was/is meant to capture how long the page is visible for. Is that something that we want to add to the Metrics/Components Suite?

I declined those tasks because, even if we don't end up bringing Page Lifecycle into the Metrics/Components Suite, the API is being designed here.

@mpopov: Earlier this year, we were thinking about Clickthrough Rate and "Page Lifecycle". The latter was/is meant to capture how long the page is visible for. Is that something that we want to add to the Metrics/Components Suite?

Superseded by T406261: [EPIC] Create a suite of standard metrics to use in experiments.

Superseded by T406261: [EPIC] Create a suite of standard metrics to use in experiments.

Once the patch above is merged, I think we can call this Done™ per https://codesearch.wmcloud.org/search/?q=page_namespace&files=&excludeFiles=swift%24&repos=Extension%3AMetricsPlatform%2Crepos%2Fdata-engineering%2Fmetrics-platform.

In T408133#11366050, @pfischer wrote:

Regarding CirrusSearch: Do you already have particular open questions/obstacles related to the A/B test scenarios covered in the backend?

• Session starting by following a link to blank Special:Search
• Session starting by following a link to Special:Search with a query
• Session starting at Special:Search with a 'go'