Page MenuHomePhabricator
Create Task
Maniphest T212460

Adopt static array files for local disk storage of values (epic)
Closed, ResolvedPublic
Actions

Description

... instead of CDB files, JSON files, serialised PHP files, or other custom formats.

This is a tracking task to cover instances of these anti-patterns within the MediaWiki run-time in Wikimedia production.

Details

SubjectRepoBranchLines +/-
Use PHP key-value lookup instead of bloomfilter with JSON filesmediawiki/libs/CommonPasswordsmaster+100 K -48
Upgrade wikimedia/common-passwords to 0.5.0mediawiki/vendormaster+100 K -774
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedKrinkleT212460 Adopt static array files for local disk storage of values (epic)
 ResolvedoriT207055 Replace Babel CDB files with static array files
 OpenNoneT99740 Use static php array files for l10n cache at WMF (instead of CDB)
 ResolvedJoeT103886 Translation cache exhaustion caused by changes to PHP code in file scope
 ResolvedJoeT119637 Update HHVM package to recent release
 ResolvedJoeT129467 HHVM 3.12 has a race-condition when starting up
 ResolvedLadsgroupT105683 Add Scap support for static-array format of LCStore
 Resolved• LarsWirzeniusT243358 Tag a new release of scap
 ResolvedjijikiT245530 Deploy scap 3.13.0-1
 ResolvedKrinkleT253673 Avoid php-opcache corruption in WMF production
 ResolvedjijikiT261009 Reproduce opcache corruptions in production
ResolvedJoeT266055 Update Scap to perform rolling restart for all MW deploy
 ResolveddancyT243009 Add option in Scap to restart php-fpm for emergency deployments, and skip depooling/pooling servers
 ResolvedthciprianiT264362 Scap feature: restart php-fpm on deployment
 ResolveddancyT290038 scap sync-file --force warns "sudo: no tty present and no askpass program specified"
 ResolveddancyT237033 Scap can't clear opcache on mw servers in Beta Cluster
 ResolvedKrinkleT311788 MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off)
 ResolveddancyT263872 Localisation cache must be purged after or during train deploy, not (just) before
 ResolvedNoneT314947 Remove old refreshMessageBlobs.php script from WikimediaMaintenance
 ResolvedoriT108638 Convert wikiversions from CDB to static array file
 ResolvedoriT141120 Move TrustedXFF away from CDB to static PHP array
 ResolvedLegoktmT200626 Standardize on a single PHP static array writer implementation
 ResolvedoriT122362 Use static arrays for Interwiki cache in Wikimedia production (instead of cdb files)
 ResolvedoriT127780 Convert Wikihiero to use static array files instead of ".ser" files (serialised php text)
 ResolvedKrinkleT212461 Update TrustedXFF to use core's StaticArrayWriter implementation
 ResolvedUmherirrenderT334655 Analyse if Equivset should use static array file instead of ".ser" file (serialised php text)

Event Timeline

Krinkle created this task.Dec 20 2018, 9:44 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 20 2018, 9:44 PM
Krinkle added subtasks: T207055: Replace Babel CDB files with static array files, T99740: Use static php array files for l10n cache at WMF (instead of CDB), T108638: Convert wikiversions from CDB to static array file, T141120: Move TrustedXFF away from CDB to static PHP array, T200626: Standardize on a single PHP static array writer implementation, T122362: Use static arrays for Interwiki cache in Wikimedia production (instead of cdb files), T127780: Convert Wikihiero to use static array files instead of ".ser" files (serialised php text), T129499: Move ZhConversion conversion tables from static class to array file.Dec 20 2018, 9:51 PM
Krinkle added a subtask: T212461: Update TrustedXFF to use core's StaticArrayWriter implementation.
Krinkle renamed this task from Adopt static array files for local disk storage of scaler values (tracking) to Adopt static array files for local disk storage of values (tracking).Dec 20 2018, 10:00 PM
Legoktm awarded a token.Dec 20 2018, 10:05 PM
• Imarlier moved this task from Inbox, needs triage to To-do: Goals, prioritized next 4 Quarters on the Performance-Team board.Jan 7 2019, 9:04 PM
Jdforrester-WMF subscribed.Mar 5 2019, 5:20 PM
Krinkle mentioned this in T218207: Use disk-based LCStore by default in MediaWiki.Apr 25 2019, 5:26 PM
Krinkle triaged this task as Medium priority.Jul 13 2019, 12:30 AM
Anomie mentioned this in T223602: Define variant Wikimedia production config in compiled, static files.Sep 16 2019, 7:25 PM
Ricordisamoa subscribed.Dec 24 2019, 1:40 PM
Jdforrester-WMF renamed this task from Adopt static array files for local disk storage of values (tracking) to Adopt static array files for local disk storage of values (epic).Feb 21 2020, 11:29 PM
Reedy closed subtask T207055: Replace Babel CDB files with static array files as Resolved.Feb 26 2020, 7:09 PM
Reedy mentioned this in T246244: Make StaticArrayWriter able to wrap things in a PHP class.
Krinkle added a project: Wikimedia-Performance-publish.Mar 2 2020, 9:52 PM
Krinkle moved this task from Untriaged to Not yet on the Wikimedia-Performance-publish board.
Krinkle closed subtask T129499: Move ZhConversion conversion tables from static class to array file as Declined.Mar 2 2020, 9:56 PM
Krinkle mentioned this in T129499: Move ZhConversion conversion tables from static class to array file.
Krinkle removed a subtask: T129499: Move ZhConversion conversion tables from static class to array file.
Krinkle changed the status of subtask T99740: Use static php array files for l10n cache at WMF (instead of CDB) from Open to Stalled.May 26 2020, 6:30 PM
Krinkle closed subtask T212461: Update TrustedXFF to use core's StaticArrayWriter implementation as Resolved.Jun 24 2020, 1:15 AM
Krinkle changed the status of subtask T99740: Use static php array files for l10n cache at WMF (instead of CDB) from Stalled to Open.Jul 28 2022, 4:05 AM
Krinkle closed this task as Resolved.Oct 4 2022, 7:51 PM
Krinkle claimed this task.

The only remaining here is localisation, which is big enough that it's its own project. Marking this as resolved to reduce clutter on open "future goals".

gerritbot added a comment.Jan 22 2023, 8:56 PM

Change 882234 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/libs/CommonPasswords@master] Use static PHP array instead of JSON for improved latency

https://gerrit.wikimedia.org/r/882234

gerritbot added a project: Patch-For-Review.Jan 22 2023, 8:56 PM

Change 882235 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/libs/CommonPasswords@master] Use simple key-value lookup instead of bloomfilter

https://gerrit.wikimedia.org/r/882235

gerritbot added a comment.Jan 22 2023, 10:02 PM

Change 882234 merged by jenkins-bot:

[mediawiki/libs/CommonPasswords@master] Use PHP key-value lookup instead of bloomfilter with JSON files

https://gerrit.wikimedia.org/r/882234

Umherirrender mentioned this in T334655: Analyse if Equivset should use static array file instead of ".ser" file (serialised php text).Apr 13 2023, 10:09 AM
Umherirrender added a subtask: T334655: Analyse if Equivset should use static array file instead of ".ser" file (serialised php text).
Krinkle moved this task from Not yet to Ready for write-up on the Wikimedia-Performance-publish board.May 28 2023, 4:00 PM
gerritbot added a comment.Jun 6 2023, 12:44 PM

Change 927650 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/vendor@master] Upgrade wikimedia/common-passwords to 0.5.0

https://gerrit.wikimedia.org/r/927650

gerritbot added a comment.Jun 6 2023, 5:45 PM

Change 927650 merged by jenkins-bot:

[mediawiki/vendor@master] Upgrade wikimedia/common-passwords to 0.5.0

https://gerrit.wikimedia.org/r/927650

ReleaseTaggerBot added a project: MW-1.41-notes (1.41.0-wmf.13; 2023-06-13).Jun 6 2023, 6:00 PM
Reedy mentioned this in T273220: Deploy StopForumSpam extension to production.Nov 27 2023, 11:47 PM
Umherirrender closed subtask T334655: Analyse if Equivset should use static array file instead of ".ser" file (serialised php text) as Resolved.Feb 29 2024, 10:32 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL