AndyRussG (Andrew Green)
User

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Oct 28 2014, 2:36 PM (189 w, 5 d)
Availability
Available
LDAP User
Unknown
MediaWiki User
AGreen (WMF)

Recent Activity

Thu, Jun 7

AndyRussG moved T195908: FundraiserLandingPage: Parse EL sample rate URL param as number from Pending Deployment to Done on the Fundraising Sprint Lactose is unusually tolerant board.
Thu, Jun 7, 1:27 AM · Fundraising Sprint Lactose is unusually tolerant, MW-1.32-release-notes (WMF-deploy-2018-06-05 (1.32.0-wmf.7)), Fundraising Sprint Karma chameleons hide amongst us, Patch-For-Review, Fundraising-Backlog

Wed, Jun 6

AndyRussG placed T196564: DB schemas (production changes and test DB) and SQL commands to run for new banner and LP impression data from EL up for grabs.
Wed, Jun 6, 3:54 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog
AndyRussG claimed T196563: Write a specification for mapping banner/landing page impression event properties -> database schema.
Wed, Jun 6, 3:54 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog
AndyRussG moved T196563: Write a specification for mapping banner/landing page impression event properties -> database schema from Backlog to Doing on the Fundraising Sprint Lactose is unusually tolerant board.
Wed, Jun 6, 3:53 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog
AndyRussG added a comment to T195594: New scripts to ingress data from Kafkatee into MySQL.

Spinning out a couple related tasks:

Wed, Jun 6, 3:50 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog
AndyRussG triaged T196564: DB schemas (production changes and test DB) and SQL commands to run for new banner and LP impression data from EL as Normal priority.
Wed, Jun 6, 3:49 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog
AndyRussG triaged T196563: Write a specification for mapping banner/landing page impression event properties -> database schema as Normal priority.
Wed, Jun 6, 3:20 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog
AndyRussG closed T186047: centralnotice_analytics: adapt ImpressionsQuery for EventLogging-based impressions recording as Invalid.
Wed, Jun 6, 2:59 PM · Fundraising-Backlog
AndyRussG closed T186047: centralnotice_analytics: adapt ImpressionsQuery for EventLogging-based impressions recording, a subtask of T183978: [Epic] Kafkatee changes, as Invalid.
Wed, Jun 6, 2:59 PM · Epic, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Asymmetrical Earth Theory, Fundraising-Backlog
AndyRussG added a comment to T186047: centralnotice_analytics: adapt ImpressionsQuery for EventLogging-based impressions recording.

As per T186048#4189098, dimensions and measures available in Druid seem unlikely to change. So we shouldn't actually have to change anything in centralnotice_analytics. So I'll close this task as invalid. (If things change, we can re-open it...) Thanks!!!

Wed, Jun 6, 2:58 PM · Fundraising-Backlog

Tue, Jun 5

AndyRussG moved T178930: CentralNotice: Consolidate typical analytics queries (for example, for impression rates) into a library from Backlog to Review on the Fundraising Sprint Lactose is unusually tolerant board.
Tue, Jun 5, 9:54 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Asymmetrical Earth Theory, Fundraising Sprint Winter Wanderland, Fundraising Sprint Vaporwerewolf, Fundraising Sprint Uptight Piano, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG removed a project from T195276: CentralNotice: Some URL params break EventLogging impression schema: Fundraising Sprint Lactose is unusually tolerant.
Tue, Jun 5, 9:53 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Patch-For-Review, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG closed T195276: CentralNotice: Some URL params break EventLogging impression schema as Resolved.
Tue, Jun 5, 9:53 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Patch-For-Review, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG closed T195276: CentralNotice: Some URL params break EventLogging impression schema, a subtask of T183978: [Epic] Kafkatee changes, as Resolved.
Tue, Jun 5, 9:53 PM · Epic, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Asymmetrical Earth Theory, Fundraising-Backlog
AndyRussG moved T195908: FundraiserLandingPage: Parse EL sample rate URL param as number from Backlog to Pending Deployment on the Fundraising Sprint Lactose is unusually tolerant board.
Tue, Jun 5, 9:31 PM · Fundraising Sprint Lactose is unusually tolerant, MW-1.32-release-notes (WMF-deploy-2018-06-05 (1.32.0-wmf.7)), Fundraising Sprint Karma chameleons hide amongst us, Patch-For-Review, Fundraising-Backlog
AndyRussG moved T195276: CentralNotice: Some URL params break EventLogging impression schema from Pending Deployment to Done on the Fundraising Sprint Karma chameleons hide amongst us board.
Tue, Jun 5, 8:12 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Patch-For-Review, MediaWiki-extensions-CentralNotice, Fundraising-Backlog

Wed, May 30

AndyRussG added a comment to T193601: CentralNotice shouldn't load JS mixins for campaigns targeting other Geos.

On topic of CentralNotice, the internal review has spawned T192623 which is going to eliminate 1 of the serial roundtrips required for CentralNotice banner display (as well as for everything else!).

Wed, May 30, 3:17 AM · MediaWiki-extensions-CentralNotice, Performance-Team
AndyRussG added a parent task for T193601: CentralNotice shouldn't load JS mixins for campaigns targeting other Geos: T192828: [Epic] CentralNotice performance.
Wed, May 30, 1:31 AM · MediaWiki-extensions-CentralNotice, Performance-Team
AndyRussG added a subtask for T192828: [Epic] CentralNotice performance: T193601: CentralNotice shouldn't load JS mixins for campaigns targeting other Geos.
Wed, May 30, 1:31 AM · Epic, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG added a comment to T193332: CentralNotice - CSP overreach.

CSP warnings have now landed in core, should go out on the train pretty soon: https://gerrit.wikimedia.org/r/#/c/253969

Wed, May 30, 12:17 AM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising-Backlog, MediaWiki-extensions-CentralNotice
AndyRussG added a comment to T195840: Track when a CentralNotice banner was displayed to the user in NavTiming.

For CentralNotice, it should be pretty simple to set up a hook to run for this (or some similar mechanism). :)

Wed, May 30, 12:11 AM · Patch-For-Review, MediaWiki-extensions-NavigationTiming, Performance-Team

Tue, May 29

AndyRussG moved T195908: FundraiserLandingPage: Parse EL sample rate URL param as number from Backlog to Review on the Fundraising Sprint Karma chameleons hide amongst us board.
Tue, May 29, 7:47 PM · Fundraising Sprint Lactose is unusually tolerant, MW-1.32-release-notes (WMF-deploy-2018-06-05 (1.32.0-wmf.7)), Fundraising Sprint Karma chameleons hide amongst us, Patch-For-Review, Fundraising-Backlog
AndyRussG added a project to T195908: FundraiserLandingPage: Parse EL sample rate URL param as number: Fundraising Sprint Karma chameleons hide amongst us.
Tue, May 29, 7:46 PM · Fundraising Sprint Lactose is unusually tolerant, MW-1.32-release-notes (WMF-deploy-2018-06-05 (1.32.0-wmf.7)), Fundraising Sprint Karma chameleons hide amongst us, Patch-For-Review, Fundraising-Backlog
AndyRussG created T195908: FundraiserLandingPage: Parse EL sample rate URL param as number.
Tue, May 29, 7:38 PM · Fundraising Sprint Lactose is unusually tolerant, MW-1.32-release-notes (WMF-deploy-2018-06-05 (1.32.0-wmf.7)), Fundraising Sprint Karma chameleons hide amongst us, Patch-For-Review, Fundraising-Backlog
AndyRussG moved T113428: CentralNotice: make-wmf-branch doesn't work for named extension deployment branches from Blocked & not fr-tech to Analysis on the Fundraising-Backlog board.
Tue, May 29, 3:09 AM · MediaWiki-Release-Tools, Fundraising-Backlog

Fri, May 25

AndyRussG claimed T195594: New scripts to ingress data from Kafkatee into MySQL.
Fri, May 25, 3:18 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog
AndyRussG moved T195594: New scripts to ingress data from Kafkatee into MySQL from Backlog to Doing on the Fundraising Sprint Karma chameleons hide amongst us board.
Fri, May 25, 3:17 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog
AndyRussG renamed T195594: New scripts to ingress data from Kafkatee into MySQL from New scripts to ingress Kafkatee into MySQL to New scripts to ingress data from Kafkatee into MySQL.
Fri, May 25, 3:14 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog
AndyRussG added a parent task for T195594: New scripts to ingress data from Kafkatee into MySQL: T183978: [Epic] Kafkatee changes.
Fri, May 25, 3:13 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog
AndyRussG added a subtask for T183978: [Epic] Kafkatee changes: T195594: New scripts to ingress data from Kafkatee into MySQL.
Fri, May 25, 3:13 PM · Epic, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Asymmetrical Earth Theory, Fundraising-Backlog
AndyRussG created T195594: New scripts to ingress data from Kafkatee into MySQL.
Fri, May 25, 3:08 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog

Tue, May 22

AndyRussG moved T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data from Backlog to Done on the Fundraising Sprint Karma chameleons hide amongst us board.
Tue, May 22, 10:44 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG moved T178930: CentralNotice: Consolidate typical analytics queries (for example, for impression rates) into a library from Backlog to Review on the Fundraising Sprint Karma chameleons hide amongst us board.
Tue, May 22, 10:43 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Asymmetrical Earth Theory, Fundraising Sprint Winter Wanderland, Fundraising Sprint Vaporwerewolf, Fundraising Sprint Uptight Piano, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG moved T195276: CentralNotice: Some URL params break EventLogging impression schema from Backlog to Pending Deployment on the Fundraising Sprint Karma chameleons hide amongst us board.
Tue, May 22, 10:43 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Patch-For-Review, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG moved T195276: CentralNotice: Some URL params break EventLogging impression schema from Backlog to Review on the Fundraising Sprint Junebugs prefer July board.
Tue, May 22, 3:57 AM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Patch-For-Review, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG added a project to T195276: CentralNotice: Some URL params break EventLogging impression schema: Fundraising Sprint Junebugs prefer July.
Tue, May 22, 3:56 AM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Patch-For-Review, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG added a parent task for T195276: CentralNotice: Some URL params break EventLogging impression schema: T183978: [Epic] Kafkatee changes.
Tue, May 22, 3:36 AM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Patch-For-Review, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG added a subtask for T183978: [Epic] Kafkatee changes: T195276: CentralNotice: Some URL params break EventLogging impression schema.
Tue, May 22, 3:36 AM · Epic, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Asymmetrical Earth Theory, Fundraising-Backlog
AndyRussG created T195276: CentralNotice: Some URL params break EventLogging impression schema.
Tue, May 22, 3:35 AM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Patch-For-Review, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG added a comment to T195259: Testing infrastructure for EventLogging ingress of banner impression and landing page data.

I think this involves:

  • a test Kafkatee instance,
  • a test database, and
  • a test DjangoBannerStats job and config to slurp up the files.
Tue, May 22, 3:07 AM · Fundraising-Backlog

Mon, May 21

AndyRussG created T195259: Testing infrastructure for EventLogging ingress of banner impression and landing page data.
Mon, May 21, 7:39 PM · Fundraising-Backlog

May 16 2018

AndyRussG moved T193332: CentralNotice - CSP overreach from Backlog to Doing on the Fundraising Sprint Junebugs prefer July board.
May 16 2018, 4:04 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising-Backlog, MediaWiki-extensions-CentralNotice
AndyRussG updated subscribers of T186048: Adapt Druid banenr_activity jobs to EventLogging-based impression recording.

For the realtime job, are there any similar examples that have EventLogging sources that I could more or less copy?

May 16 2018, 4:02 PM · Fundraising Sprint Junebugs prefer July, Patch-For-Review, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising-Backlog

May 15 2018

AndyRussG moved T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data from Review to Done on the Fundraising Sprint Junebugs prefer July board.
May 15 2018, 7:18 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG added a comment to T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data.

OK! So we'll go with (A), that is, continue to use Kafkatee to write files, and slurp them with the python script.

May 15 2018, 7:17 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG added a comment to T186048: Adapt Druid banenr_activity jobs to EventLogging-based impression recording.

As mentioned on Gerrit, the patch uploaded is a rough attempt in need of review by those familiar with these systems.

May 15 2018, 2:21 PM · Fundraising Sprint Junebugs prefer July, Patch-For-Review, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising-Backlog

May 10 2018

AndyRussG moved T186048: Adapt Druid banenr_activity jobs to EventLogging-based impression recording from Backlog to Doing on the Fundraising Sprint Junebugs prefer July board.
May 10 2018, 5:07 PM · Fundraising Sprint Junebugs prefer July, Patch-For-Review, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising-Backlog
AndyRussG moved T178930: CentralNotice: Consolidate typical analytics queries (for example, for impression rates) into a library from Backlog to Review on the Fundraising Sprint Junebugs prefer July board.
May 10 2018, 5:07 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Asymmetrical Earth Theory, Fundraising Sprint Winter Wanderland, Fundraising Sprint Vaporwerewolf, Fundraising Sprint Uptight Piano, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG moved T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data from Backlog to Review on the Fundraising Sprint Junebugs prefer July board.
May 10 2018, 5:07 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog

May 8 2018

AndyRussG updated subscribers of T186048: Adapt Druid banenr_activity jobs to EventLogging-based impression recording.

There are two fields that in Druid that we can't currently get with the pure event Data. One is country_matches_geocode, which indicates whether server-based geolocation gave the same result as the Geo cookie. (In a small number of cases, it doesn't.) The other is region.

May 8 2018, 2:58 AM · Fundraising Sprint Junebugs prefer July, Patch-For-Review, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising-Backlog

May 4 2018

AndyRussG added a comment to T186048: Adapt Druid banenr_activity jobs to EventLogging-based impression recording.

[mediawiki/extensions/CentralNotice@master] Add geo region to client-side data and impression event

Schema updated, too: https://meta.wikimedia.org/wiki/Schema:CentralNoticeImpression

May 4 2018, 4:08 PM · Fundraising Sprint Junebugs prefer July, Patch-For-Review, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising-Backlog

May 3 2018

AndyRussG added a comment to T193601: CentralNotice shouldn't load JS mixins for campaigns targeting other Geos.

Mmm just one further note, if we could split the cache on country and get that data into ResourceLoaderContext, we could also send a lot less choices to each client, which would also be a win performancewise.

May 3 2018, 11:52 PM · MediaWiki-extensions-CentralNotice, Performance-Team
AndyRussG added a comment to T193601: CentralNotice shouldn't load JS mixins for campaigns targeting other Geos.

I don't know if mixins can affect the targeting logic. If they don't, they should only be loaded after all targeting logic has run

May 3 2018, 11:16 PM · MediaWiki-extensions-CentralNotice, Performance-Team
AndyRussG updated subscribers of T193601: CentralNotice shouldn't load JS mixins for campaigns targeting other Geos.
May 3 2018, 11:13 PM · MediaWiki-extensions-CentralNotice, Performance-Team
AndyRussG added a comment to T193601: CentralNotice shouldn't load JS mixins for campaigns targeting other Geos.

Thanks!! :) This is correct, however it's blocked by 1) lack of geolocation information from the load.php entry point and 2) inability to split the RL cache on country. If those limitations are solved, it'd be pretty simple to do this.

May 3 2018, 11:13 PM · MediaWiki-extensions-CentralNotice, Performance-Team
AndyRussG added a comment to T186048: Adapt Druid banenr_activity jobs to EventLogging-based impression recording.

There are two fields that in Druid that we can't currently get with the pure event Data. One is country_matches_geocode, which indicates whether server-based geolocation gave the same result as the Geo cookie. (In a small number of cases, it doesn't.) The other is region.

May 3 2018, 7:38 PM · Fundraising Sprint Junebugs prefer July, Patch-For-Review, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising-Backlog
AndyRussG removed a project from T183978: [Epic] Kafkatee changes: Patch-For-Review.
May 3 2018, 4:43 PM · Epic, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Asymmetrical Earth Theory, Fundraising-Backlog
AndyRussG updated the task description for T186048: Adapt Druid banenr_activity jobs to EventLogging-based impression recording.
May 3 2018, 3:16 PM · Fundraising Sprint Junebugs prefer July, Patch-For-Review, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising-Backlog
AndyRussG added a comment to T186048: Adapt Druid banenr_activity jobs to EventLogging-based impression recording.

(Note that Pivot is not working properly now for Banner impressions, due to some update somewhere, and new versions of Pivot no longer being FOSS. Replacement: https://superset.wikimedia.org/ . See also https://wikitech.wikimedia.org/wiki/Analytics/Systems/Superset .)

May 3 2018, 3:16 PM · Fundraising Sprint Junebugs prefer July, Patch-For-Review, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising-Backlog

May 2 2018

AndyRussG moved T186048: Adapt Druid banenr_activity jobs to EventLogging-based impression recording from Backlog to Doing on the Fundraising Sprint Ivory and eggshell white are the same color board.
May 2 2018, 2:34 PM · Fundraising Sprint Junebugs prefer July, Patch-For-Review, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising-Backlog
AndyRussG moved T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data from Doing to Review on the Fundraising Sprint Ivory and eggshell white are the same color board.
May 2 2018, 2:30 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG added a comment to T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data.

Just curious though. Are you sure you want to just write this stuff to a log file? Are you just putting it there for a bit? The less hacky pipeline for yall would be kafka -> into MySQL directly.

May 2 2018, 2:29 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog

May 1 2018

AndyRussG added a comment to T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data.

Yes, please let us know the schema and timeline for this, thank youuuu

May 1 2018, 7:09 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog

Apr 30 2018

AndyRussG added a comment to T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data.

The main downside is you don't have control over the name of the tables (they will be named SchemaName_<REVISION_ID>), and any new schema revisions will result in new tables being created.

Apr 30 2018, 6:31 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog

Apr 29 2018

AndyRussG added a project to T193332: CentralNotice - CSP overreach: Fundraising-Backlog.
Apr 29 2018, 12:57 AM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising-Backlog, MediaWiki-extensions-CentralNotice
AndyRussG added a comment to T193332: CentralNotice - CSP overreach.

@Jseddon Thanks!!

Apr 29 2018, 12:57 AM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising-Backlog, MediaWiki-extensions-CentralNotice

Apr 25 2018

AndyRussG moved T189613: Make a rough timeline/roadmap to replace usage of Kafkatee by FR with an event-based system to count pageviews to donatewiki from Doing to Backlog on the Fundraising Sprint Ivory and eggshell white are the same color board.
Apr 25 2018, 4:01 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog
AndyRussG added a comment to T189613: Make a rough timeline/roadmap to replace usage of Kafkatee by FR with an event-based system to count pageviews to donatewiki.

Remaining to-do:

Apr 25 2018, 4:01 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog
AndyRussG added a comment to T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data.

Some of discussion on this same topic here: T189820: Create job to deliver the eventlogging_CentralNoticeImpression topic.

Apr 25 2018, 3:52 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG added a comment to T189613: Make a rough timeline/roadmap to replace usage of Kafkatee by FR with an event-based system to count pageviews to donatewiki.

(Sorry, removed the previous comment since I put it on the wrong task... ;p )

Apr 25 2018, 3:52 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog
AndyRussG added a comment to T189613: Make a rough timeline/roadmap to replace usage of Kafkatee by FR with an event-based system to count pageviews to donatewiki.
Apr 25 2018, 3:50 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog
AndyRussG added subtasks for T183978: [Epic] Kafkatee changes: T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data, T189613: Make a rough timeline/roadmap to replace usage of Kafkatee by FR with an event-based system to count pageviews to donatewiki, T185932: CentralNotice: use EventLogging instead of custom beacon, T185933: Donatewiki: use EventLogging to log pageloads, T186047: centralnotice_analytics: adapt ImpressionsQuery for EventLogging-based impressions recording, T186048: Adapt Druid banenr_activity jobs to EventLogging-based impression recording.
Apr 25 2018, 3:47 PM · Epic, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Asymmetrical Earth Theory, Fundraising-Backlog
AndyRussG added a parent task for T185932: CentralNotice: use EventLogging instead of custom beacon: T183978: [Epic] Kafkatee changes.
Apr 25 2018, 3:47 PM · Fundraising Sprint Fhabricator is spelled with an "F", Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Patch-For-Review, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising-Backlog, MediaWiki-extensions-CentralNotice
AndyRussG added a parent task for T185933: Donatewiki: use EventLogging to log pageloads: T183978: [Epic] Kafkatee changes.
Apr 25 2018, 3:47 PM · Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, MW-1.31-release-notes (WMF-deploy-2018-04-10 (1.31.0-wmf.29)), Patch-For-Review, Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising-Backlog
AndyRussG added a parent task for T186047: centralnotice_analytics: adapt ImpressionsQuery for EventLogging-based impressions recording: T183978: [Epic] Kafkatee changes.
Apr 25 2018, 3:47 PM · Fundraising-Backlog
AndyRussG added a parent task for T186048: Adapt Druid banenr_activity jobs to EventLogging-based impression recording: T183978: [Epic] Kafkatee changes.
Apr 25 2018, 3:47 PM · Fundraising Sprint Junebugs prefer July, Patch-For-Review, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising-Backlog
AndyRussG added a parent task for T189613: Make a rough timeline/roadmap to replace usage of Kafkatee by FR with an event-based system to count pageviews to donatewiki: T183978: [Epic] Kafkatee changes.
Apr 25 2018, 3:47 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog
AndyRussG closed T190100: Option to enforce CSP on banner previews and flag errors as Resolved.
Apr 25 2018, 3:35 PM · Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, Unplanned-Sprint-Work, Patch-For-Review, Fundraising-Backlog, MediaWiki-extensions-CentralNotice
AndyRussG moved T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data from Review to Doing on the Fundraising Sprint Ivory and eggshell white are the same color board.
Apr 25 2018, 3:34 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG added a comment to T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data.

Seems one option is to use Apache Flume with a JDBC sink...

Apr 25 2018, 3:34 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG updated subscribers of T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data.
Apr 25 2018, 3:10 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG added a comment to T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data.

Here's one possible issue: backfill in case of failure. In the current system, I don't think files with this data are ever deleted from the FR cluster, so if there's a problem with any part of the pipeline downstream of the files, we have practically no time limit on our ability to backfill.

Apr 25 2018, 3:09 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG moved T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data from Backlog to Review on the Fundraising Sprint Ivory and eggshell white are the same color board.
Apr 25 2018, 2:29 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG closed T185933: Donatewiki: use EventLogging to log pageloads as Resolved.
Apr 25 2018, 2:01 PM · Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, MW-1.31-release-notes (WMF-deploy-2018-04-10 (1.31.0-wmf.29)), Patch-For-Review, Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising-Backlog
AndyRussG moved T178930: CentralNotice: Consolidate typical analytics queries (for example, for impression rates) into a library from Backlog to Review on the Fundraising Sprint Ivory and eggshell white are the same color board.
Apr 25 2018, 1:59 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Asymmetrical Earth Theory, Fundraising Sprint Winter Wanderland, Fundraising Sprint Vaporwerewolf, Fundraising Sprint Uptight Piano, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG moved T189613: Make a rough timeline/roadmap to replace usage of Kafkatee by FR with an event-based system to count pageviews to donatewiki from Backlog to Doing on the Fundraising Sprint Ivory and eggshell white are the same color board.
Apr 25 2018, 1:58 PM · Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog

Apr 24 2018

AndyRussG moved T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data from Doing to Review on the Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol board.
Apr 24 2018, 7:34 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG set the point value for T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data to 1.
Apr 24 2018, 7:34 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG added a comment to T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data.

It looks like python-kafka is definitely the way to go for receiving data from a Kafka producer and processing it in Python. It looks like we can adapt the current python script to received data this way and still keep much of the processing code as-is. Also probably a good opportunity to remove cruft and refactor parts of the script.

Apr 24 2018, 7:29 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG moved T190089: Add Banner Guidelines link to banner edit screen from Pending Deployment to Done on the Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol board.
Apr 24 2018, 3:51 AM · Patch-For-Review, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG moved T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data from Backlog to Doing on the Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol board.
Apr 24 2018, 3:50 AM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog

Apr 23 2018

AndyRussG moved T190100: Option to enforce CSP on banner previews and flag errors from Pending Deployment to Done on the Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol board.
Apr 23 2018, 10:30 PM · Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, Unplanned-Sprint-Work, Patch-For-Review, Fundraising-Backlog, MediaWiki-extensions-CentralNotice
AndyRussG added a comment to T190089: Add Banner Guidelines link to banner edit screen.

The patch is now deployed, so it should be possible to add a link to the message using wikitext.

Apr 23 2018, 7:49 PM · Patch-For-Review, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising Sprint Gravity wasn't always this pushy, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG added parent tasks for T192842: Banner History: Include some banner performance metrics in log: T192564: [Epic] Banner history improvements, T192828: [Epic] CentralNotice performance.
Apr 23 2018, 7:46 PM · MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG added a subtask for T192564: [Epic] Banner history improvements: T192842: Banner History: Include some banner performance metrics in log.
Apr 23 2018, 7:46 PM · Epic, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG added a subtask for T192828: [Epic] CentralNotice performance: T192842: Banner History: Include some banner performance metrics in log.
Apr 23 2018, 7:46 PM · Epic, MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG created T192842: Banner History: Include some banner performance metrics in log.
Apr 23 2018, 7:45 PM · MediaWiki-extensions-CentralNotice, Fundraising-Backlog
AndyRussG added a comment to T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data.

It has been suggested that we use python-kafka now... Would this as a complement to, or instead of, the current Python scripts? How would they interact?

Apr 23 2018, 7:31 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog
AndyRussG added a comment to T183978: [Epic] Kafkatee changes.

Also: T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data.

Apr 23 2018, 7:28 PM · Epic, Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Asymmetrical Earth Theory, Fundraising-Backlog
AndyRussG created T192839: [Spike] Plan ingress system to use with new Kafka topic for landing page and impression data.
Apr 23 2018, 7:27 PM · Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Junebugs prefer July, Fundraising Sprint Ivory and eggshell white are the same color, Fundraising Sprint HTTP originally stood for Happy Turtle Transfer Protocol, Fundraising-Backlog