Page MenuHomePhabricator
Create Task
Maniphest T213741

Use MaxMind DB in piwik geo-location
Closed, ResolvedPublic
Actions

Description

Via puppet we should copy the maxmmind db into the misc Matomo subdirectory

This is a real good first bug for a junior SRE

Details

SubjectRepoBranchLines +/-
profile::piwik::instance: actually copy Maxmind data filesoperations/puppetproduction+1 -0
Add MaxMind data files to Matomo /misc directoryoperations/puppetproduction+17 -1
Add types to profile::analytics classesoperations/puppetproduction+49 -49
Add types to profile::analytics classesoperations/puppetproduction+49 -49
Add geoip::data::puppet to profile::piwik::instanceoperations/puppetproduction+8 -1
profile::piwik::webserver: Add geoip data to Matomooperations/puppetproduction+11 -0
Customize query in gerrit

Event Timeline

• Nuria created this task.Jan 14 2019, 5:40 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 14 2019, 5:40 PM
mforns triaged this task as Low priority.Jan 17 2019, 6:01 PM
mforns moved this task from Incoming to Operational Excellence on the Analytics board.
• Nuria added a project: good first task.Aug 25 2019, 9:50 AM
• Nuria updated the task description. (Show Details)
• Nuria moved this task from Operational Excellence to Mentoring on the Analytics board.
Aklapper removed a project: good first task.May 12 2020, 2:31 PM

@Nuria: A good first task is a self-contained, non-controversial task with a clear approach. It should be well-described with pointers to help a completely new contributor. Given the current short task description (no links to documentation, no explanation what all these terms mean, no info which code base this is about or where the code base is located) I'm removing the good first task tag. Please add details what exactly has to happen where and how for a new contributor, and then add back the good first task project tag. Thanks a lot in advance!

elukey subscribed.Aug 11 2020, 1:55 PM

@Aklapper the description is too generic, what we meant was "good first task for new Analytics SREs" since we are onboarding two very soon. We'll avoid using the tag sorry!

• Nuria added a comment.Aug 11 2020, 3:21 PM
  • For an entry level task this probably needs to happen in labs first
Ottomata added a subscriber: • razzi.Sep 3 2020, 9:24 PM
gerritbot added a comment.Sep 10 2020, 7:36 PM

Change 626481 had a related patch set uploaded (by Razzi; owner: Razzi):
[operations/puppet@production] Add geoip::data::puppet to profile::piwik::instance

https://gerrit.wikimedia.org/r/626481

gerritbot added a project: Patch-For-Review.Sep 10 2020, 7:36 PM
• Nuria added a comment.Sep 14 2020, 3:14 PM

Let's make sure to test this is working once deployed, what does our data on piwik say? http://piwik.wikimedia.org

• Nuria added a comment.Sep 14 2020, 3:16 PM

For reference: https://matomo.org/docs/geo-locate/

• Nuria assigned this task to • razzi.Sep 14 2020, 3:36 PM
• Nuria added a project: Analytics-Kanban.
• Nuria moved this task from Next Up to In Progress on the Analytics-Kanban board.
gerritbot added a comment.Sep 15 2020, 5:58 PM

Change 627577 had a related patch set uploaded (by Razzi; owner: Razzi):
[operations/puppet@production] profile::piwik::webserver: Add geoip data to Matomo

https://gerrit.wikimedia.org/r/627577

gerritbot added a comment.Sep 15 2020, 6:03 PM

Change 627577 merged by Elukey:
[operations/puppet@production] profile::piwik::webserver: Add geoip data to Matomo

https://gerrit.wikimedia.org/r/627577

gerritbot added a comment.Sep 15 2020, 6:52 PM

Change 626481 merged by Elukey:
[operations/puppet@production] Add geoip::data::puppet to profile::piwik::instance

https://gerrit.wikimedia.org/r/626481

elukey added a comment.Sep 15 2020, 7:08 PM

After merging the second change this was the output:

Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoIP.dat]/ensure: defined content as '{md5}995a730183f160290ed8289c47370bf5'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoIP2-City.mmdb]/ensure: defined content as '{md5}a4fb0591977df10a0d89376773917140'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoIP2-Connection-Type.mmdb]/ensure: defined content as '{md5}a39126c5f3104bbef529952732a08a63'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoIP2-Country.mmdb]/ensure: defined content as '{md5}6d439338330793f2d63c97461d8becbc'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoIP2-ISP.mmdb]/ensure: defined content as '{md5}584cb3187861004cad1f57f45d90b1a9'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoIPASNum.dat]/ensure: defined content as '{md5}d5672dde5cd81286aaaf87ab7defc690'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoIPASNumv6.dat]/ensure: defined content as '{md5}e60631f5f91ca2da15c092e378f275e9'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoIPCity.dat]/ensure: defined content as '{md5}0bd3859ad2c599e7033430cf68f4c6d7'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoIPNetSpeed.dat]/ensure: defined content as '{md5}6c1c98bf79b333f77168c7d5105f8d1d'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoIPNetSpeedCell.dat]/ensure: defined content as '{md5}2e2d9cb9aea70da9eeda92702baa25a1'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoIPRegion.dat]/ensure: defined content as '{md5}264a2818599d65ac0d6254808ba22365'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoIPv6.dat]/ensure: defined content as '{md5}043a59361de6dfd9b6cde077f07d28db'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoLite.dat]/ensure: defined content as '{md5}0701a01cb50cec28c5ff643912bf05e3'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoLite2-City.mmdb]/ensure: defined content as '{md5}ab00b342809f695016810418905b5646'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoLiteCity.dat]/ensure: defined content as '{md5}5c29a58ef988fdd4359c814287ff6c0b'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/GeoLiteCityv6.dat]/ensure: defined content as '{md5}e5f41164a659cf1dc67f63a65cdfb0f9'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/composer/build-xhprof.sh]/mode: mode changed '0755' to '0644'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/composer/clean-xhprof.sh]/mode: mode changed '0755' to '0644'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/cron/archive.sh]/mode: mode changed '0755' to '0644'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/user]/group: group changed 'www-data' to 'root'
Notice: /Stage[main]/Geoip::Data::Puppet/File[/usr/share/matomo/misc/user]/mode: mode changed '0775' to '0755'

So we don't really want this, we have to fix it somehow.

Maintenance_bot removed a project: Patch-For-Review.Sep 15 2020, 7:10 PM
elukey added a comment.Sep 15 2020, 7:14 PM

The change was reverted, we have a couple of ideas about how to proceed, new patches will follow.

gerritbot added a comment.Sep 16 2020, 4:23 PM

Change 627878 had a related patch set uploaded (by Razzi; owner: Razzi):
[operations/puppet@production] Add types to profile::analytics classes

https://gerrit.wikimedia.org/r/627878

gerritbot added a project: Patch-For-Review.Sep 16 2020, 4:23 PM
gerritbot added a comment.Sep 16 2020, 6:59 PM

Change 627903 had a related patch set uploaded (by Razzi; owner: Razzi):
[operations/puppet@production] Add types to profile::analytics classes

https://gerrit.wikimedia.org/r/627903

gerritbot added a comment.Sep 16 2020, 7:03 PM

Change 627903 abandoned by Razzi:
[operations/puppet@production] Add types to profile::analytics classes

Reason:
Duplicate of https://gerrit.wikimedia.org/r/c/operations/puppet/ /627878

https://gerrit.wikimedia.org/r/627903

gerritbot added a comment.Sep 16 2020, 7:15 PM

Change 627878 merged by Ottomata:
[operations/puppet@production] Add types to profile::analytics classes

https://gerrit.wikimedia.org/r/627878

gerritbot added a comment.Sep 17 2020, 5:13 PM

Change 628146 had a related patch set uploaded (by Razzi; owner: Razzi):
[operations/puppet@production] Add MaxMind data files to Matomo /misc directory

https://gerrit.wikimedia.org/r/628146

gerritbot added a comment.Sep 17 2020, 5:49 PM

Change 628146 merged by Razzi:
[operations/puppet@production] Add MaxMind data files to Matomo /misc directory

https://gerrit.wikimedia.org/r/628146

gerritbot added a comment.Sep 17 2020, 6:08 PM

Change 628164 had a related patch set uploaded (by Razzi; owner: Razzi):
[operations/puppet@production] profile::piwik::instance: actually copy Maxmind data files

https://gerrit.wikimedia.org/r/628164

gerritbot added a comment.Sep 17 2020, 6:14 PM

Change 628164 merged by Razzi:
[operations/puppet@production] profile::piwik::instance: actually copy Maxmind data files

https://gerrit.wikimedia.org/r/628164

• razzi added a comment.Sep 17 2020, 6:22 PM

This is done and configured on Matomo: https://piwik.wikimedia.org

@Nuria could you please check that everything is working as expected?

• Nuria moved this task from In Progress to Done on the Analytics-Kanban board.Sep 18 2020, 7:51 PM
• Nuria added a comment.Sep 18 2020, 7:58 PM

piwik seems to be reading this well and i can see files on nuria@matomo1002:/usr/share/matomo/misc, closing

• Nuria closed this task as Resolved.Sep 18 2020, 7:58 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL