Page MenuHomePhabricator
Create Task
Maniphest T344749

Refactor import-db and update-db into one script
Closed, ResolvedPublic
Actions

Description

These two scripts essentially do the same thing, so should be combined. Then bugs like T341660 can be fixed just once.

Details

Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Add 'import' mode to the script that outputs the .sql filerepos/mediawiki/services/ipoid!61tchandersrefactor-import-datamain
Refactor import-data.jsrepos/mediawiki/services/ipoid!59stranrefactor-import-datamain
Customize query in GitLab

Related Objects
Search...

StatusSubtypeAssignedTask
Restricted Task
 ResolvedkostajhT294511 2021 Security Team wikireplicas audit
 DeclinedNoneT284948 Raw IPs of logged-out users disclosed in wiki-replicas
 ResolvedNiharikaT324492 Temporary accounts - MVP
 ResolvedkostajhT340895 [Epic] IP Info accommodations for temporary accounts
 ResolvedSTranT341395 Display Spur data on IPInfo infobox
 ResolvedkostajhT339284 Deploy ipoid
 ResolvedkostajhT340984 Fix problems with data import
 ResolvedSTranT344749 Refactor import-db and update-db into one script
 ResolvedSTranT341122 Implement daily data update routine
 ResolvedTchandersT344517 Fix statements generated for updated IP data
 ResolvedTchandersT344520 Remove temporary files to avoid taking up unnecessary disk space
 ResolvedSTranT344272 Use batching for daily data import
 ResolvedSTranT325630 Implement call to data vendor
 ResolvedTchandersT344499 Move behaviors/proxies/tunnels processing into an independent script
 DeclinedNoneT344939 Log when the lookup fails on attempting to INSERT to actor_data (because the proxy/behaviour doesn't exist)
 ResolvedTchandersT344921 Ensure dependent SQL statements are all run in the same batch
 ResolvedTchandersT344981 Map tables should not have null for any of their values
 ResolvedSTranT345757 Create orchestration script for daily data update
 ResolvedNoneT345761 Turn scripts into modules that export
 ResolvedSTranT345684 Implement batch tracking for imports
 ResolvedBUG REPORTTchandersT341660 Investigate non-deterministic behaviour of import

Event Timeline

Tchanders created this task.Aug 22 2023, 5:13 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 22 2023, 5:13 PM
Tchanders mentioned this in T341660: Investigate non-deterministic behaviour of import.Aug 22 2023, 5:13 PM
Tchanders added a parent task: T340984: Fix problems with data import.Aug 23 2023, 1:51 PM
Tchanders added a subtask: T341122: Implement daily data update routine.Aug 23 2023, 1:57 PM
STran claimed this task.Sep 1 2023, 9:37 AM
STran moved this task from Ready 🎬 (ONLY IF YOU HAVE NO MORE CODE TO REVIEW) to In Progress 💪 on the Anti-Harassment-Team (AHaT Sprint 32 - Baseball Cap) board.
STran moved this task from In Progress 💪 to Code Review 🔍 on the Anti-Harassment-Team (AHaT Sprint 32 - Baseball Cap) board.Sep 1 2023, 9:59 AM
CodeReviewBot added a comment.Sep 4 2023, 6:00 PM

stran updated https://gitlab.wikimedia.org/repos/mediawiki/services/ipoid/-/merge_requests/59

Refactor import-data.js

CodeReviewBot added a comment.Sep 7 2023, 3:38 PM

tchanders merged https://gitlab.wikimedia.org/repos/mediawiki/services/ipoid/-/merge_requests/59

Refactor import-data.js

CodeReviewBot added a project: Patch-For-Review.Sep 7 2023, 3:43 PM

tchanders opened https://gitlab.wikimedia.org/repos/mediawiki/services/ipoid/-/merge_requests/61

Add 'import' mode to the script that outputs the .sql file

CodeReviewBot added a comment.Sep 7 2023, 5:39 PM

tchanders opened https://gitlab.wikimedia.org/repos/mediawiki/services/ipoid/-/merge_requests/62

Add 'import' mode to the script that outputs the .sql file

CodeReviewBot added a comment.Sep 7 2023, 5:39 PM

tchanders opened https://gitlab.wikimedia.org/repos/mediawiki/services/ipoid/-/merge_requests/63

Add 'import' mode to the script that outputs the .sql file

CodeReviewBot added a comment.Sep 11 2023, 6:21 PM

stran merged https://gitlab.wikimedia.org/repos/mediawiki/services/ipoid/-/merge_requests/61

Add 'import' mode to the script that outputs the .sql file

Tchanders closed this task as Resolved.Sep 11 2023, 6:25 PM
Tchanders moved this task from Backlog to Done on the iPoid-Service board.
STran closed subtask T341122: Implement daily data update routine as Resolved.Dec 12 2023, 8:40 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits