Page MenuHomePhabricator
Create Task
Maniphest T313251

Collect and fix schema drifts of wmf-deployed extension tables in production
Open, Needs TriagePublic
Actions

Description

We are mostly done with core drifts, now it's time to take a look at extensions.

This is now possible with T261912: Convert WMF Deployed Extensions to Abstract Schema being done.

You just need to add paths to extension schema files in https://github.com/Ladsgroup/db-analyzor-tools/blob/master/abstract_paths.json and in valid_categories: https://github.com/Ladsgroup/db-analyzor-tools/blob/master/drift-tracker/app.py and then one run would add the list.

  • AbuseFilter
  • AntiSpoof
  • Babel
  • BetaFeatures
  • BounceHandler
  • CentralAuth
  • CentralNotice
  • CheckUser T321063
  • Cognate
  • ContentTranslation
  • CreditsSource T326156
  • DiscussionTools
  • Echo
  • EntitySchema
  • FlaggedRevs T313253
  • Flow
  • GeoData
  • GlobalBlocking
  • GlobalPreferences
  • GlobalUsage
  • GrowthExperiments
  • LdapAuthentication
  • Linter
  • LiquidThreads
  • MachineVision
  • Math
  • Newsletter
  • OATHAuth
  • OAuth
  • OAuthRateLimiter
  • ORES
  • PageAssessments
  • PageTriage
  • ProofreadPage
  • PropertySuggester
  • ReadingLists
  • SecurePoll
  • ShortUrl
  • TimedMediaHandler
  • Translate
  • UploadWizard
  • UrlShortener
  • WikiLove
  • Wikibase
  • WikibaseQualityConstraints
  • WikimediaEditorTasks

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenNoneT313251 Collect and fix schema drifts of wmf-deployed extension tables in production
 ResolvedLadsgroupT313253 Fix Flaggedrevs database schema drifts in production
 ResolvedLadsgroupT318950 Fix renamed indexes of flaggedrevs table in production
 ResolvedLadsgroupT318955 Drop fr_comment and fr_text from flaggedrevs table in production
 ResolvedLadsgroupT318959 Add fr_user index on flaggedrevs in production
 ResolvedLadsgroupT322618 Fix renamed indexes of flaggedrevs_tracking table in production
 ResolvedLadsgroupT323214 Fix unsigned drifts in flaggedrevs caused by 4c0b3c7b9b0
 ResolvedLadsgroupT323827 Finish timestamp schema changes in flaggedrevs
 ResolvedLadsgroupT323907 Make fr_user unsigned
 ResolvedLadsgroupT333918 Switchover s2 master (db1162 -> db1122)
 ResolvedLadsgroupT251191 FlaggedRevs has lots of database drifts but only in s1 and s5
 OpenNoneT321063 Fix CheckUser database schema drifts in production
 Resolved• MarosteguiT321123 Drop old index cuc_user_time on cu_changes table for wmf wikis
 Resolved• MarosteguiT321127 Add index cuc_actor_ip_time to cu_changes on wmf wikis
 Resolved• MarosteguiT321126 Add column 'cul_actor' and index cul_actor_time to cu_log on wmf wikis
 Resolved• MarosteguiT321130 Add column cuc_private to cu_changes on wmf wikis
 ResolvedPapaulT322988 db2173 HW errors
 Resolved• MarosteguiT343174 Add missing column cuc_only_for_read_old to testcommonswiki
 Resolved• MarosteguiT343175 Remove old fields 'cuc_user' and 'cuc_user_text' as well as index 'cuc_user_ip_time' from a few production wikis
 OpenNoneT356631 Column cu_log.cul_timestamp has an incorrect type and incorrectly has a default value set
 Resolved• MarosteguiT356634 Column cu_log.cul_type incorrectly has a default value set on s1
 ResolvedLadsgroupT326156 Fix CreditsSource drifts
 ResolvedLadsgroupT326662 Drop unused tables of CreditsSource

Event Timeline

Ladsgroup created this task.Jul 18 2022, 6:19 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 18 2022, 6:19 PM
Ladsgroup moved this task from Inbox to Epic - Database on the Data-Persistence board.Jul 18 2022, 6:19 PM
Ladsgroup updated the task description. (Show Details)Jul 18 2022, 6:24 PM
Umherirrender updated the task description. (Show Details)Jul 18 2022, 8:01 PM
Daimona subscribed.Jul 19 2022, 12:21 AM
Umherirrender added a comment.Jul 29 2022, 5:08 PM

First collection with the abstract schema file names (as of now) and some dblist entries

Needs revisit on which databases the check needs to run. Not sure how to tag the tables only exists once (central) and some other special handling for databases (like x1)

I am not sure how big the overhead for new dblists is, but extensions not exists on all wiki possible needs its own list to make the check easier in the drift tracker (like Translate, ContentTranslation, LiquidThreads, WikiLove, ReadingLists, PageTriage, etc.)

"abusefilter": {
    "path": [
        "mediawiki/extensions/AbuseFilter/+/master/db_patches/tables.json"
    ]
},
"antispoof": {
    "path": [
        "mediawiki/extensions/AntiSpoof/+/master/sql/tables.json"
    ]
},
"babel": {
    "path": [
        "mediawiki/extensions/Babel/+/master/sql/tables.json"
    ]
},
"betafeatures": {
    "path": [
        "mediawiki/extensions/BetaFeatures/+/master/sql/tables.json"
    ]
},
"bouncehandler": {
    "path": [
        "mediawiki/extensions/BounceHandler/+/master/sql/tables.json"
    ]
},
"centralauth": {
    "path": [
        "mediawiki/extensions/CentralAuth/+/master/schema/tables.json"
    ]
},
"centralnotice": {
    "path": [
        "mediawiki/extensions/CentralNotice/+/master/sql/tables.json"
    ]
},
"checkuser": {
    "path": [
        "mediawiki/extensions/CheckUser/+/master/schema/tables.json"
    ]
},
"cognate": {
    "path": [
        "mediawiki/extensions/Cognate/+/master/sql/tables.json"
    ],
    "dblist": "wiktionary"
},
"contenttranslation": {
    "path": [
        "mediawiki/extensions/ContentTranslation/+/master/sql/tables.json"
    ]
},
"creditssource": {
    "path": [
        "mediawiki/extensions/CreditsSource/+/master/schema/tables.json"
    ],
    "dblist": "wikivoyage"
},
"discussiontools": {
    "path": [
        "mediawiki/extensions/DiscussionTools/+/master/sql/discussiontools_subscription.json"
    ]
},
"echo": {
    "path": [
        "mediawiki/extensions/Echo/+/master/sql/tables.json"
    ],
    "dblist": "echo"
},
"echo_central": {
    "path": [
        "mediawiki/extensions/Echo/+/master/sql/tables-sharedtracking.json"
    ]
},
"entityschema": {
    "path": [
        "mediawiki/extensions/EntitySchema/+/master/sql/tables.json"
    ],
    "dblist": "wikibaserepo"
},
"flow": {
    "path": [
        "mediawiki/extensions/Flow/+/master/sql/tables.json"
    ],
    "dblist": "flow"
},
"geodata": {
    "path": [
        "mediawiki/extensions/GeoData/+/master/sql/tables.json"
    ]
},
"globalblocking": {
    "path": [
        "mediawiki/extensions/GlobalBlocking/+/master/sql/tables-global_block_whitelist.json"
    ]
},
"globalblocking_central": {
    "path": [
        "mediawiki/extensions/GlobalBlocking/+/master/sql/tables-globalblocks.json"
    ]
},
"globalpreferences": {
    "path": [
        "mediawiki/extensions/GlobalPreferences/+/master/sql/tables.json"
    ]
},
"globalusage": {
    "path": [
        "mediawiki/extensions/GlobalUsage/+/master/sql/tables.json"
    ]
},
"growthexperiments": {
    "path": [
        "mediawiki/extensions/GrowthExperiments/+/master/sql/growthexperiments_link_recommendations.json",
        "mediawiki/extensions/GrowthExperiments/+/master/sql/growthexperiments_link_submissions.json",
        "mediawiki/extensions/GrowthExperiments/+/master/sql/growthexperiments_mentee_data.json",
        "mediawiki/extensions/GrowthExperiments/+/master/sql/growthexperiments_mentor_mentee.json"
    ],
    "dblist": "growthexperiments"
},
"ldapauthentication": {
    "path": [
        "mediawiki/extensions/LdapAuthentication/+/master/schema/tables.json"
    ]
},
"linter": {
    "path": [
        "mediawiki/extensions/Linter/+/master/sql/tables.json"
    ]
},
"liquidthreads": {
    "path": [
        "mediawiki/extensions/LiquidThreads/+/master/sql/tables.json"
    ]
},
"machinevision": {
    "path": [
        "mediawiki/extensions/MachineVision/+/master/sql/tables.json"
    ]
},
"math": {
    "path": [
        "mediawiki/extensions/Math/+/master/sql/mathlatexml.json",
        "mediawiki/extensions/Math/+/master/sql/mathoid.json"
    ]
},
"newsletter": {
    "path": [
        "mediawiki/extensions/Newsletter/+/master/sql/tables.json"
    ]
},
"oathauth": {
    "path": [
        "mediawiki/extensions/OATHAuth/+/master/sql/tables.json"
    ]
},
"oauth": {
    "path": [
        "mediawiki/extensions/OAuth/+/master/schema/tables.json"
    ]
},
"oauthratelimiter": {
    "path": [
        "mediawiki/extensions/OAuthRateLimiter/+/master/schema/tables.json"
    ]
},
"ores": {
    "path": [
        "mediawiki/extensions/ORES/+/master/sql/tables.json"
    ]
},
"pageassessments": {
    "path": [
        "mediawiki/extensions/PageAssessments/+/master/db/tables.json"
    ],
    "dblist": "pageassessments"
},
"pagetriage": {
    "path": [
        "mediawiki/extensions/PageTriage/+/master/sql/tables.json"
    ]
},
"proofreadpage": {
    "path": [
        "mediawiki/extensions/ProofreadPage/+/master/sql/tables.json"
    ],
    "dblist": "wikisource"
},
"propertysuggester": {
    "path": [
        "mediawiki/extensions/PropertySuggester/+/master/sql/tables.json"
    ],
    "dblist": "wikibaserepo"
},
"readinglists": {
    "path": [
        "mediawiki/extensions/ReadingLists/+/master/sql/tables.json"
    ]
},
"securepoll": {
    "path": [
        "mediawiki/extensions/SecurePoll/+/master/sql/tables.json"
    ]
},
"shorturl": {
    "path": [
        "mediawiki/extensions/ShortUrl/+/master/schemas/tables.json"
    ]
},
"timedmediahandler": {
    "path": [
        "mediawiki/extensions/TimedMediaHandler/+/master/sql/tables.json"
    ]
},
"translate": {
    "path": [
        "mediawiki/extensions/Translate/+/master/sql/tables.json"
    ]
},
"uploadwizard": {
    "path": [
        "mediawiki/extensions/UploadWizard/+/master/sql/tables.json"
    ]
},
"urlshortener": {
    "path": [
        "mediawiki/extensions/UrlShortener/+/master/schemas/table.json"
    ]
},
"wikibasequalityconstraints": {
    "path": [
        "mediawiki/extensions/WikibaseQualityConstraints/+/master/sql/tables.json"
    ]
},
"wikilove": {
    "path": [
        "mediawiki/extensions/WikiLove/+/master/patches/tables.json"
    ]
},
"wikimediaeditortasks": {
    "path": [
        "mediawiki/extensions/WikimediaEditorTasks/+/master/sql/tables.json"
    ]
},
Lucas_Werkmeister_WMDE added a subtask: T321063: Fix CheckUser database schema drifts in production.Oct 18 2022, 2:23 PM
Lucas_Werkmeister_WMDE updated the task description. (Show Details)
Ladsgroup added a subtask: T326156: Fix CreditsSource drifts.Jan 3 2023, 3:38 PM
Ladsgroup updated the task description. (Show Details)
Ladsgroup closed subtask T326156: Fix CreditsSource drifts as Resolved.Jan 30 2023, 7:42 AM
Ladsgroup updated the task description. (Show Details)
Ladsgroup closed subtask T313253: Fix Flaggedrevs database schema drifts in production as Resolved.Apr 4 2023, 8:11 AM
Ladsgroup updated the task description. (Show Details)
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL