Page MenuHomePhabricator

Create a elasticsearch/kibana index with queries to allow query completion candidate research
Open, Needs TriagePublic8 Estimated Story Points

Description

As a search developer I want to be able to analyze queries so that I can find a viable query completion candidates.

Since relforge is available through analytics network, we should be able to ship queries into kibana and analyze them.

AC:

  • search events from backend logs and frontend logs are joined via request_id
  • queries are normalized (to be determine how extensively)
  • joined events are available for analysis in Kibana

Event Timeline

Zbyszko set the point value for this task to 8.

Change 665310 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[wikimedia/discovery/analytics@master] Add export_queries_to_relforge dag

https://gerrit.wikimedia.org/r/665310

Change 665310 merged by jenkins-bot:
[wikimedia/discovery/analytics@master] Add export_queries_to_relforge dag

https://gerrit.wikimedia.org/r/665310

Mentioned in SAL (#wikimedia-operations) [2021-03-11T20:44:48Z] <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@cc478d4]: T273847 export queries to relforge dag deployment

Mentioned in SAL (#wikimedia-operations) [2021-03-11T20:46:57Z] <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@cc478d4]: T273847 export queries to relforge dag deployment (duration: 02m 09s)

Change 670942 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[wikimedia/discovery/analytics@master] Set correct start date for export_queries_to_relforge

https://gerrit.wikimedia.org/r/670942

Change 670942 merged by ZPapierski:
[wikimedia/discovery/analytics@master] Set correct start date for export_queries_to_relforge

https://gerrit.wikimedia.org/r/670942

Mentioned in SAL (#wikimedia-operations) [2021-03-11T21:12:04Z] <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@3810277]: T273847 export queries to relforge dag deployment - correct start date

Mentioned in SAL (#wikimedia-operations) [2021-03-11T21:13:57Z] <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@3810277]: T273847 export queries to relforge dag deployment - correct start date (duration: 01m 53s)

Change 670955 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[wikimedia/discovery/analytics@master] Add elastic-template handling

https://gerrit.wikimedia.org/r/670955

Change 670955 merged by ZPapierski:
[wikimedia/discovery/analytics@master] Add elastic-template handling

https://gerrit.wikimedia.org/r/670955

Mentioned in SAL (#wikimedia-operations) [2021-03-12T08:59:33Z] <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@9a408b2]: T273847 export queries to relforge dag deployment - elastic-template handling

Mentioned in SAL (#wikimedia-operations) [2021-03-12T08:59:43Z] <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@9a408b2]: T273847 export queries to relforge dag deployment - elastic-template handling (duration: 00m 10s)

Mentioned in SAL (#wikimedia-operations) [2021-03-12T09:00:34Z] <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@9a408b2]: T273847 export queries to relforge dag deployment - elastic-template handling

Mentioned in SAL (#wikimedia-operations) [2021-03-12T09:00:44Z] <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@9a408b2]: T273847 export queries to relforge dag deployment - elastic-template handling (duration: 00m 09s)

Mentioned in SAL (#wikimedia-operations) [2021-03-12T09:05:26Z] <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@9a408b2]: T273847 export queries to relforge dag deployment - elastic-template handling

Mentioned in SAL (#wikimedia-operations) [2021-03-12T09:07:01Z] <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@9a408b2]: T273847 export queries to relforge dag deployment - elastic-template handling (duration: 01m 35s)

Change 671115 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[wikimedia/discovery/analytics@master] Deploy the index setting file

https://gerrit.wikimedia.org/r/671115

Change 672463 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[wikimedia/discovery/analytics@master] Add code coverage to Sonarqube report

https://gerrit.wikimedia.org/r/672463

Change 671115 merged by ZPapierski:
[wikimedia/discovery/analytics@master] Deploy the index setting file

https://gerrit.wikimedia.org/r/671115

Change 672463 merged by ZPapierski:
[wikimedia/discovery/analytics@master] Add code coverage to Sonarqube report

https://gerrit.wikimedia.org/r/672463

Mentioned in SAL (#wikimedia-operations) [2021-04-08T09:25:48Z] <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@d098717]: T273847 export queries to relforge dag deployment - sensor name fix

Mentioned in SAL (#wikimedia-operations) [2021-04-08T09:27:36Z] <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@d098717]: T273847 export queries to relforge dag deployment - sensor name fix (duration: 01m 48s)

Mentioned in SAL (#wikimedia-operations) [2021-04-08T10:09:44Z] <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@ff0137d]: T273847 export queries to relforge dag deployment - start date update

Mentioned in SAL (#wikimedia-operations) [2021-04-08T10:11:28Z] <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@ff0137d]: T273847 export queries to relforge dag deployment - start date update (duration: 01m 37s)

Mentioned in SAL (#wikimedia-operations) [2021-04-08T11:47:30Z] <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@25dad72]: T273847 export queries to relforge dag deployment - elastic index name fix

Mentioned in SAL (#wikimedia-operations) [2021-04-08T11:49:10Z] <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@25dad72]: T273847 export queries to relforge dag deployment - elastic index name fix (duration: 01m 39s)

Mentioned in SAL (#wikimedia-operations) [2021-04-08T11:57:34Z] <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@25dad72]: T273847 export queries to relforge dag deployment - elastic index name fix

Mentioned in SAL (#wikimedia-operations) [2021-04-08T11:57:43Z] <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@25dad72]: T273847 export queries to relforge dag deployment - elastic index name fix (duration: 00m 09s)

Mentioned in SAL (#wikimedia-operations) [2021-04-14T13:27:05Z] <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@825c60a]: T273847 export queries to relforge dag deployment - schedule change

Mentioned in SAL (#wikimedia-operations) [2021-04-14T13:29:14Z] <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@825c60a]: T273847 export queries to relforge dag deployment - schedule change (duration: 02m 08s)

Mentioned in SAL (#wikimedia-operations) [2021-04-14T14:09:06Z] <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@8ae53e3]: T273847 export queries to relforge dag deployment - start date update

Mentioned in SAL (#wikimedia-operations) [2021-04-14T14:11:20Z] <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@8ae53e3]: T273847 export queries to relforge dag deployment - start date update (duration: 02m 14s)