Page MenuHomePhabricator

QuickSurveysResponses Event Platform Migration
Closed, ResolvedPublic

Description

See: https://wikitech.wikimedia.org/wiki/Event_Platform/EventLogging_legacy

Unless otherwise notified, client IP and consequently geocoded data will no longer be collected for this event data after this migration. Please let us know if this should continue to be captured. See also T262626.

Status
  • 2021-02-23 - DONE!

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

@ovasileva @phuedx let us know if this schema needs client IP and/or geocoded data. If not, it will be removed as part of this migration.

@Gilles @bmansurov do either of you know if this schema needs client IP and/or geocoded data? If it doesn't, it will be removed as part of this migration.

Not strictly needed for the survey I run on that extension, but I imagine that given how versatile QuickSurveys can be, someone would want to know where survey respondents are from for some surveys.

Not strictly needed for the survey I run on that extension, but I imagine that given how versatile QuickSurveys can be, someone would want to know where survey respondents are from for some surveys.

I would also lean towards keeping it, if possible.

Thanks for adding me @ovasileva.

For our surveys we haven't needed it, but would be fine to keep it if it's useful for others.

Ok, we will keep client_ip and geocoded_data for this.

FYI: This schema will be migrated during the week of Jan 18 - Jan 22.

Change 657341 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[schemas/event/secondary@master] Add leggacy quicksurveyinitiation and quicksurveysresponses schemas

https://gerrit.wikimedia.org/r/657341

Change 657341 merged by Ottomata:
[schemas/event/secondary@master] Add leggacy quicksurveyinitiation and quicksurveysresponses schemas

https://gerrit.wikimedia.org/r/657341

Change 657343 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Migrate QuickSurveyInitiation and QuickSurveysResponses to eventgate on testwiki

https://gerrit.wikimedia.org/r/657343

Change 657343 merged by Ottomata:
[operations/mediawiki-config@master] Migrate QuickSurveyInitiation and QuickSurveysResponses to eventgate on testwiki

https://gerrit.wikimedia.org/r/657343

Change 657346 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Declare streams QuickSurveysResponses and QuickSurveyInitiation

https://gerrit.wikimedia.org/r/657346

Change 657346 merged by Ottomata:
[operations/mediawiki-config@master] Declare streams QuickSurveysResponses and QuickSurveyInitiation

https://gerrit.wikimedia.org/r/657346

Mentioned in SAL (#wikimedia-operations) [2021-01-20T14:47:50Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Migrate QuickSurveys schemas to EventGate on testwiki - T271165, T271166 (duration: 01m 06s)

Change 657351 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Migrate QuickSurveyInitiation and QuickSurveysResponses to eventgate on all wikis

https://gerrit.wikimedia.org/r/657351

Change 657351 merged by Ottomata:
[operations/mediawiki-config@master] Migrate QuickSurveyInitiation and QuickSurveysResponses to eventgate on all wikis

https://gerrit.wikimedia.org/r/657351

Mentioned in SAL (#wikimedia-operations) [2021-01-20T14:57:09Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Migrate QuickSurveys schemas to EventGate on all wikis - T271165, T271166 (duration: 01m 05s)

Change 657355 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[mediawiki/extensions/QuickSurveys@master] Finialize migration of EventLogging schemas to Event Platform

https://gerrit.wikimedia.org/r/657355

Change 657355 merged by jenkins-bot:
[mediawiki/extensions/QuickSurveys@master] Finialize migration of EventLogging schemas to Event Platform

https://gerrit.wikimedia.org/r/657355

@Ottomata the migration appears to have broken our daemon subscribed to Kafka updates for these schemas:

https://github.com/wikimedia/performance-navtiming/blob/ba6cd0d970d18aa71d3745eaa49fa09dea7dbe21/navtiming/__init__.py#L103-L104

Screenshot 2021-01-21 at 17.01.15.png (530×546 px, 57 KB)

Should we do this differently now?

? Weird. no your client should be working just fine with the change. The events are flowing into Kafka just fine:

e.g. https://grafana-rw.wikimedia.org/d/000000018/eventlogging-schema?orgId=1&var-schema=QuickSurveyInitiation

Here's an example of a QuickSurveyInitiation I see now:

{
  "event": {
    "surveySessionToken": "xxxx-quicksurveys",
    "pageviewToken": "xxxxx",
    "surveyCodeName": "perceived-performance-survey",
    "eventName": "impression",
    "performanceNow": 27310
  },
  "schema": "QuickSurveyInitiation",
  "webHost": "es.m.wikipedia.org",
  "wiki": "eswiki",
  "$schema": "/analytics/legacy/quicksurveyinitiation/1.0.0",
  "client_dt": "2021-01-21T16:38:01.469Z",
  "meta": {
    "stream": "eventlogging_QuickSurveyInitiation",
    "domain": "es.m.wikipedia.org",
    "id": "xxxxxxx",
    "dt": "2021-01-21T16:38:21.486Z",
    "request_id": "xxxxxx"
  },
  "dt": "2021-01-21T16:38:01.470Z",
  "http": {
    "client_ip": "xxxxx",
    "request_headers": {
      "user-agent": "xxxxxx"
    }
  }
}

Change 657681 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Finalize QuickSurvey* Event Platform migration

https://gerrit.wikimedia.org/r/657681

Change 657681 merged by Ottomata:
[operations/puppet@production] Finalize QuickSurvey* Event Platform migration

https://gerrit.wikimedia.org/r/657681

Change 666372 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Remove wgEventLoggingSchemas overrides for QuickSurvey and NavigationTiming

https://gerrit.wikimedia.org/r/666372

Change 666372 merged by Ottomata:
[operations/mediawiki-config@master] Remove wgEventLoggingSchemas overrides for QuickSurvey and NavigationTiming

https://gerrit.wikimedia.org/r/666372

Ottomata updated the task description. (Show Details)