Page MenuHomePhabricator
Create Task
Maniphest T339284

Deploy ipoid
Closed, ResolvedPublic
Actions

Description

This exposes data to MediaWiki, ready for ingestion by IPInfo.

Details

SubjectRepoBranchLines +/-
ipoid: Re-enable daily updatesoperations/deployment-chartsmaster+2 -2
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 In ProgressNiharikaT324492 Temporary accounts - MVP
 OpenNoneT340895 [Epic] IP Info accommodations for temporary accounts
 OpenSTranT341395 Display Spur data on IPInfo infobox
 ResolvedkostajhT339284 Deploy ipoid
 ResolvedkostajhT340984 Fix problems with data import
 ResolvedBUG REPORT• AGueyteT340178 "Column 'anonymous' cannot be null"
 DeclinedBUG REPORT• TThoabalaT340176 "Data too long for column 'behavior' at row 1"
 DeclinedBUG REPORTNoneT339837 "Data too long for column 'anonymous' at row 1"
 ResolvedBUG REPORT• TThoabalaT339359 'Parameter at position 2 is undefined\n' if tunnels.type is not set
 ResolvedBUG REPORT• AGueyteT339345 "Duplicate entry '0' for key 'behavior'"
 ResolvedBUG REPORT• TThoabalaT339343 "Column 'type' cannot be null"
 DeclinedBUG REPORT• TThoabalaT341605 Ipoid: ER_DATA_TOO_LONG
 DeclinedBUG REPORT• AGueyteT339325 "Duplicate entry '0' for key 'proxy'"
 InvalidBUG REPORTNoneT339324 "Data too long for column 'type' at row 1"
 InvalidNoneT345107 Update Tunnels Type schema VARBINARY value
 ResolvedBUG REPORT• TThoabalaT339073 Density imported incorrectly
 ResolvedSTranT344749 Refactor import-db and update-db into one script
 ResolvedSTranT341122 Implement daily data update routine
 ResolvedTchandersT344517 Fix statements generated for updated IP data
 ResolvedTchandersT344520 Remove temporary files to avoid taking up unnecessary disk space
 ResolvedSTranT344272 Use batching for daily data import
 ResolvedSTranT325630 Implement call to data vendor
 ResolvedTchandersT344499 Move behaviors/proxies/tunnels processing into an independent script
 DeclinedNoneT344939 Log when the lookup fails on attempting to INSERT to actor_data (because the proxy/behaviour doesn't exist)
 ResolvedTchandersT344921 Ensure dependent SQL statements are all run in the same batch
 ResolvedTchandersT344981 Map tables should not have null for any of their values
 ResolvedSTranT345757 Create orchestration script for daily data update
 ResolvedNoneT345761 Turn scripts into modules that export
 ResolvedSTranT345684 Implement batch tracking for imports
 OpenBUG REPORTTchandersT341660 Investigate non-deterministic behaviour of import
 ResolvedkostajhT341129 Setup spec.yaml
 ResolvedjijikiT341326 Deploy ipoid to staging on Kuberenetes
 ResolvedkostajhT339331 Prepare for initial data import on production servers
 ResolvedSTranT344941 Respond to data update and import errors
 DuplicateNoneT355429 data pipeline: Don't attempt to re-import current day if full import already existed
 ResolvedSTranT345156 DBA audit of ipoid database
 DuplicateSTranT345237 Investigate: Recovering from skipped updates when pulling from the daily feed
 ResolvedkostajhT346861 [M] Write CronJob configuration
 ResolvedjijikiT350903 Rename FEED_API_KEY
 ResolvedkostajhT348477 Update mariadb library to latest version 3 release
 ResolvedBUG REPORTkostajhT349056 [S] split: too many files
 ResolvedkostajhT349171 [M] Implement proxy configuration for kubernetes deployment

Event Timeline

Tchanders created this task.Jun 15 2023, 6:03 PM
Tchanders added a subtask: T340984: Fix problems with data import.Jul 3 2023, 1:09 PM
kostajh closed subtask T341129: Setup spec.yaml as Resolved.Jul 7 2023, 8:25 AM
jijiki closed subtask T341326: Deploy ipoid to staging on Kuberenetes as Resolved.Jul 7 2023, 10:19 AM
jijiki added a subtask: T339331: Prepare for initial data import on production servers.Jul 7 2023, 4:16 PM
• AGueyte moved this task from Untriaged to Similar Editors on the Anti-Harassment board.Jul 12 2023, 6:34 PM
• AGueyte moved this task from Similar Editors to Untriaged on the Anti-Harassment board.Jul 13 2023, 11:42 AM
Tchanders removed a subtask: T341122: Implement daily data update routine.Aug 23 2023, 1:57 PM
kostajh mentioned this in T289579: <Security Initiative> P2P Proxy API.Aug 24 2023, 7:19 AM
GeneralNotability subscribed.Aug 24 2023, 4:17 PM
kostajh changed the status of subtask T345151: Support capability to inform WMF tools about new, removed, and updated information from Open to Stalled.Aug 29 2023, 12:48 PM
Tchanders added a subtask: T345237: Investigate: Recovering from skipped updates when pulling from the daily feed.Sep 6 2023, 5:07 PM
kostajh changed the status of subtask T345151: Support capability to inform WMF tools about new, removed, and updated information from Stalled to Open.Sep 18 2023, 8:31 AM
kostajh added a subtask: T348477: Update mariadb library to latest version 3 release.Oct 10 2023, 7:08 AM
kostajh added a subtask: T349056: [S] split: too many files.Oct 17 2023, 6:28 AM
kostajh added a subtask: T349171: [M] Implement proxy configuration for kubernetes deployment.Oct 18 2023, 8:34 AM
jijiki changed the status of subtask T346861: [M] Write CronJob configuration from Open to In Progress.Nov 7 2023, 3:36 PM
jijiki closed subtask T346861: [M] Write CronJob configuration as Resolved.Nov 10 2023, 8:34 AM
kostajh closed subtask T349056: [S] split: too many files as Resolved.Nov 10 2023, 10:07 AM
kostajh closed subtask T349171: [M] Implement proxy configuration for kubernetes deployment as Resolved.Nov 10 2023, 10:09 AM
kostajh closed subtask T339331: Prepare for initial data import on production servers as Resolved.Nov 14 2023, 9:51 AM
kostajh closed subtask T345156: DBA audit of ipoid database as Resolved.Nov 16 2023, 9:21 AM
gerritbot added a comment.Dec 12 2023, 3:56 PM

Change 982427 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/deployment-charts@master] ipoid: Re-enable daily updates

https://gerrit.wikimedia.org/r/982427

gerritbot added a project: Patch-For-Review.Dec 12 2023, 3:56 PM

Change 982427 merged by jenkins-bot:

[operations/deployment-charts@master] ipoid: Re-enable daily updates

https://gerrit.wikimedia.org/r/982427

Maintenance_bot removed a project: Patch-For-Review.Dec 12 2023, 4:10 PM
kostajh closed this task as Resolved.Jan 29 2024, 1:39 PM
kostajh claimed this task.
kostajh subscribed.

ipoid is running in production, with daily updates, and serving requests.

kostajh closed subtask T340984: Fix problems with data import as Resolved.Jan 29 2024, 1:40 PM
kostajh removed a subtask: T345151: Support capability to inform WMF tools about new, removed, and updated information .Jan 29 2024, 1:44 PM
Tchanders added a parent task: T341395: Display Spur data on IPInfo infobox.Feb 7 2024, 5:16 PM
Tchanders removed a parent task: T339279: [Epic] Display data from ipoid in IPInfo infobox.
Dreamy_Jazz closed subtask T344941: Respond to data update and import errors as Resolved.Mon, May 13, 9:19 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL