@Eileenmcnaughton and @MDemosWMF I reviewed the Engage group, and we can certainly assist by deduping records using the first+last name and address rule, as this appears to be the primary cause of many duplicates.

@DTorsani-WMF one more thing...obviously, the colors being output are quite different than the ones in your design. The new color categories (orange, lime, and pink) will show up automatically once they're added. All of the colors are output in the order they appear in the wikimedia-ui.json file, and a color is outputted if it begins with one of the allowed categories (see the colorCategories variable in this file):

@DTorsani-WMF This was so fun to work on :D Here's the demo. Some questions/comments for you:

Eh, I see... is there any public info anywhere? :) Maybe https://meta.wikimedia.org/wiki/Wikimedia_Foundation/Advancement/Community_Growth comes close?

I've asked the powers that be if I can create a page for my sub-team like the other sub-teams on that Community Growth page, and I am waiting for their answer. If the link to the public info in Phab can be updated later, I am happy to use https://meta.wikimedia.org/wiki/Wikimedia_Foundation/Advancement/Community_Growth for now as a placeholder.

Change #1068883 merged by jenkins-bot:

[mediawiki/extensions/DonationInterface@master] Set address2/supplimental address to blank if N/A

https://gerrit.wikimedia.org/r/1068883

Change #1070884 merged by jenkins-bot:

[mediawiki/core@master] phpunit: Set much smaller defaults for RandomImageGenerator

https://gerrit.wikimedia.org/r/1070884

Change #1070896 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Use fixed font size for UI componenets in Vector 22

https://gerrit.wikimedia.org/r/1070896

Change #1071023 had a related patch set uploaded (by Anne Tomasevich; author: Anne Tomasevich):

[design/codex@main] docs: Update Colors page of the Style Guide

https://gerrit.wikimedia.org/r/1071023

Change #1071022 had a related patch set uploaded (by Andrea Denisse; author: Andrea Denisse):

[operations/puppet@production] ldap: Add JJMC89's wmf_prod SSH key

https://gerrit.wikimedia.org/r/1071022

Change #1070965 merged by Dzahn:

[operations/puppet@production] stewards: Enable profile::auto_restarts::service for rsync

https://gerrit.wikimedia.org/r/1070965

Change #1064801 merged by jenkins-bot:

[wikimedia/fundraising/SmashPig@master] Set address2/supplimental address to blank if N/A

https://gerrit.wikimedia.org/r/1064801

I wrote a python script to read using a bloom filter: P68726

Now https://phabricator.wikimedia.org/F57465429

Change #1071021 had a related patch set uploaded (by Sbisson; author: Sbisson):

[mediawiki/extensions/ContentTranslation@master] SX: translation recommendations based on topics

https://gerrit.wikimedia.org/r/1071021

Change #1071020 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] puppet8: ensure cassandra passwords are defined

https://gerrit.wikimedia.org/r/1071020

Seems like all we need is a uniquifier in the PRIMARY KEY then. Instead of using an existing Cassandra column to achieve this, which will vary per table, I propose we simply add the row_number() as an additional column, which we are already calculating when generating the top N tables, but that we do not currently persist in Cassandra.

Cookbook cookbooks.sre.hosts.reimage started by jclark@cumin1002 for host dse-k8s-worker1009.eqiad.wmnet with OS bookworm completed:

• dse-k8s-worker1009 (PASS)
  • Downtimed on Icinga/Alertmanager
  • Disabled Puppet
  • Removed from Puppet and PuppetDB if present and deleted any certificates
  • Removed from Debmonitor if present
  • Forced PXE for next reboot
  • Host rebooted via IPMI
  • Host up (Debian installer)
  • Add puppet_version metadata to Debian installer
  • Checked BIOS boot parameters are back to normal
  • Host up (new fresh bookworm OS)
  • Generated Puppet certificate
  • Signed new Puppet certificate
  • Run Puppet in NOOP mode to populate exported resources in PuppetDB
  • Found Nagios_host resource for this host in PuppetDB
  • Downtimed the new host on Icinga/Alertmanager
  • Removed previous downtime on Alertmanager (old OS)
  • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202409051936_jclark_2650778_dse-k8s-worker1009.out
  • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
  • Rebooted
  • Automatic Puppet run was successful
  • Forced a re-check of all Icinga services for the host
  • Icinga status is optimal
  • Icinga downtime removed
  • Updated Netbox data from PuppetDB

Change #1070884 had a related patch set uploaded (by Krinkle; author: Thiemo Kreuz (WMDE)):

[mediawiki/core@master] phpunit: Set much smaller defaults for RandomImageGenerator

https://gerrit.wikimedia.org/r/1070884

If we can’t figure out the underlying issue, I suppose I could:

I’m leaving the background runner in its current state for a bit in case someone else wants to take a look, but at some point I’ll restart it again to get the edits unstuck.

On August 3rd, overall citoid requests from MyBib dropped substantially. I re-ran this analysis to include 30 days post August 3rd and did not identify any significant changes to Citoid feature use due to this change in request volume.

Change #1070660 merged by jenkins-bot:

[mediawiki/core@master] ParserOutput::collectMetadata: Properly handle non-scalar page properties

https://gerrit.wikimedia.org/r/1070660

Change #1070660 merged by jenkins-bot:

[mediawiki/core@master] ParserOutput::collectMetadata: Properly handle non-scalar page properties

https://gerrit.wikimedia.org/r/1070660

In T372498#10122832, @rook wrote:

My hoped for end result is that we can apply the helm charts from https://gerrit.wikimedia.org/g/operations/deployment-charts to this cluster.

I'm seeing about 80 helm charts in that repo. Is each of these deployed to prod?

Change #1070666 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Replace use of ParserOutput::setPageProperty, deprecated in 1.42

https://gerrit.wikimedia.org/r/1070666

In T370470#10119267, @mforns wrote:

After discussing with @xcollazo, we agreed that we like the second solution best:

• Dropping all top tables from Cassandra (8 tables)
• Recreating them with the modified clustering keys (adding the ranked entity, i.e.: user_name, category, media_file, etc.)
• Reloading all the data to Cassandra (8 tables for 10 months each)

Some considerations:

• Note that the only re-runs necessary are for Cassandra loading, the underlying data in the data lake is correct.
• Not sure how difficult is to drop and recreate the 8 Cassandra tables, but I imagine it's going to be a light change. I imagine the table creation statements are in a repository and have to be modified, tested, reviewed, etc. I imagine there won't be a lot of other changes cascading from that, because the only change is the clustering key. The table names, fields and types would still be the same. So, I'm pretty sure no tests will need to change, etc.
• From what I could read about Cassandra tables, I don't think we can do a seamless table switch without downtime. So, I imagine we'd need some downtime for all top-based CIM endpoints. Looking at the Airflow DAG, it takes approximately 30mins to load all top endpoints for 1 month. For the 10 months, we'd need about 5 hours downtime.

Maybe, @Eevans, do you see a better solution re. replacing the tables? 🙏

Change #1056568 merged by jenkins-bot:

[mediawiki/extensions/DonationInterface@master] Put Number on same line as Street

https://gerrit.wikimedia.org/r/1056568

Closing this out as all the core work to support dark mode last fiscal year is complete! I've spun out the remaining open subtasks and any new Codex-related dark mode tasks can just be filed normally.

Mentioned in SAL (#wikimedia-operations) [2024-09-05T19:21:25Z] <topranks> add interface qos scheduler config to codfw switches T339850

Cookbook cookbooks.sre.hosts.reimage was started by jclark@cumin1002 for host dse-k8s-worker1009.eqiad.wmnet with OS bookworm

Change #1069590 merged by jenkins-bot:

[mediawiki/extensions/UploadWizard@master] Set parser for image gallery in CampaignPageFormatter

https://gerrit.wikimedia.org/r/1069590