This ticket has two goals:
a) Create an end-to-end test repo
b) Write the first end-to-end testcase that covers the functionality of following services
{F69932266}
**Testcase description**
//Input to the testcase://
```
{
"sample_config" = [
{"project": "enwiki", "namespace": 0, "event_type": "update", "samples": 50},
{"project": "ruwiki", "namespace": 0, "event_type": "update", "samples": 50},
....
],
"wait_duration" = "5m",
"retries" = 5
}
```
//Test setup://
- Connect to `page_change` eventstream. Collect samples based on `sample_config`. (Do not collect `page.is_redirect`). Maintain `article_name`, `project`, `namespace`, `revision_id`, `revision_timestamp` for each sample.
- Also maintain the start time of collection.
- Get access token for WME API calls
//Test://
1. After waiting for wait duration, check main API `/articles` for each article in memory. The `version.identifier` returned for each article should be >= revision_id in memory. If the project is one of the structured-contents one, additionally, check `/structured-contents` for the same. (Tip: use project filter and fields to only get few relevant fields).
2. If, for all samples the revision_id checks out, call it PASS.
3. If some articles are missing or revision_id is lower, wait and retry step 1-2 (only for the failed articles), until `retries` are exhausted.
4. If, not all samples met the revision_id criteria after retries were exhausted, connect to structured-data error topic. Consume from the start time of collection. If the failed articles were found in the error topic, call the testcase PASSX (pass w/ exception). Note: for the failed articles on `/structured-contents`, we will analyse the structured-contents error topic, additionally.
5. Report test result with details on failed articles.
Test teardown:
- Revoke token
- Close kafka consumer
'''
**To do**
[] Create the end-to-end test repo. (Easier with a Go test framework. We can reuse our kafka library code, etc.)
[] Build the testcase as per above draft.
**Acceptance criteria**
[] End-to-end test repo with this intial testcase. Make it easier to run, update input. Functional reporting.