Page MenuHomePhabricator

Consider ways to reduce the row count in cu_useragent_clienthints_map
Closed, DeclinedPublic

Description

To store Client Hints data, two new tables were added. Now that data collection is enabled on enwiki, the amount of rows in the cu_useragent_clienthints_map table has reached around 4 million. Once data deletion starts in 3 months time, there will be around 48 million rows.

This task has two objectives:

  • Do the DBAs think this many rows will cause issues for display and/or continued collection
  • Whether any optimisations can be made to reduce the row count

On the second point some thoughts have been discussed, but mostly this cannot be done until T344800 is completed. Ideas so far are:

  1. Only store uach_name of brands when the same name isn't being used for rows with uach_name as fullVersionList
  2. Don't store data that has little entropy and therefore isn't displayed

Some discussion about adding an extra column cu_useragent_clienthints to store the version number instead of combining it with a space was had, but this is likely not going to be done for the MVP.

Event Timeline

The thought behind suggestion 1 is that the brands values store the less specific version number than is already stored in the associated fullVersionList value. Not storing the row for brands when the data exists in a more specific form in another row would cut the row count by potentially up to 4 rows per reference ID (i.e. revision ID).

From DB point of view, it's not that big. It's currently 297MB and if it grows to the ten times the original size, it'll be only 3GB. In comparison, our biggest table is externallinks of commons which is 300GB or watchlist of enwiki that is 34GB. Any improvement on those will drastically offset clienthints changes and such by a wide-margin.

i.e. If there is a low-hanging fruit, sure. Otherwise, don't bother or better spend the time fixing another table instead (e.g. user_properties: T54777) offsetting clinethints and more.

From DB point of view, it's not that big. It's currently 297MB and if it grows to the ten times the original size, it'll be only 3GB. In comparison, our biggest table is externallinks of commons which is 300GB or watchlist of enwiki that is 34GB. Any improvement on those will drastically offset clienthints changes and such by a wide-margin.

i.e. If there is a low-hanging fruit, sure. Otherwise, don't bother or better spend the time fixing another table instead (e.g. user_properties: T54777) offsetting clinethints and more.

Thanks for the thoughts.

Some of the reduction will be "low-hanging fruit", as the data we don't display can simply not be collected. However, it's good to know that the status-quo is fine.

Dreamy_Jazz moved this task from CheckUser to Inbox on the CheckUser board.

Given the DBA comment from 2023 and that this feature is now stable, I think we can call this declined as no action needed.