Baseline:

mysql:research@dbstore1007.eqiad.wmnet [ruwikinews]> select count(*) from externallinks;
+----------+
| count(*) |
+----------+
| 60609082 |
+----------+
1 row in set (28.480 sec)

First: https://ru.wikinews.org/w/index.php?title=%D0%A8%D0%B0%D0%B1%D0%BB%D0%BE%D0%BD:%D0%A1%D0%BE%D0%B4%D0%B5%D1%80%D0%B6%D0%B8%D1%82_%D0%BC%D0%B0%D1%82%D0%B5%D1%80%D0%B8%D0%B0%D0%BB%D1%8B&diff=prev&oldid=16808235

Second: https://ru.wikinews.org/w/index.php?title=%D0%A8%D0%B0%D0%B1%D0%BB%D0%BE%D0%BD:%D0%A1%D1%82%D0%B0%D1%82%D0%B8%D1%81%D1%82%D0%B8%D0%BA%D0%B0_%D1%81%D1%82%D1%80%D0%B0%D0%BD%D0%B8%D1%86%D1%8B&diff=prev&oldid=16808237

Asking to see if these pages are actually needed: https://ru.wikinews.org/w/index.php?title=%D0%9E%D0%B1%D1%81%D1%83%D0%B6%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5_%D0%92%D0%B8%D0%BA%D0%B8%D0%BD%D0%BE%D0%B2%D0%BE%D1%81%D1%82%D0%B5%D0%B9%3A%D0%A1%D1%82%D0%B0%D1%82%D0%B8%D1%81%D1%82%D0%B8%D0%BA%D0%B0_%D1%81%D1%82%D1%80%D0%B0%D0%BD%D0%B8%D1%86%2F%D0%9A%D0%B0%D1%82%D0%B5%D0%B3%D0%BE%D1%80%D0%B8%D1%8F%3A%D0%9F%D0%BE%D0%BB_%D0%9A%D1%80%D0%B5%D0%B9%D0%B3_%D0%A0%D0%BE%D0%B1%D0%B5%D1%80%D1%82%D1%81#Are_these_pages_really_needed?

Third: https://ru.wikinews.org/w/index.php?title=%D0%A8%D0%B0%D0%B1%D0%BB%D0%BE%D0%BD:%D0%A1%D1%82%D0%B0%D1%82%D0%B8%D1%81%D1%82%D0%B8%D0%BA%D0%B0_%D1%81%D1%82%D1%80%D0%B0%D0%BD%D0%B8%D1%86%D1%8B&diff=prev&oldid=16808245

This one should remove 15M rows.

Asking to see if these pages are actually needed

I wonder whether there is a way to provide such download links on page info page. I remember looking into how to add a custom row there myself, for an unrelated thing, and found it difficult.

Third

This one feels like some API design flaw to me. It should not matter whether you are using action API or REST API to access info about the very wiki you are linking from.

In T403397#11136756, @Base wrote:

Asking to see if these pages are actually needed

I wonder whether there is a way to provide such download links on page info page. I remember looking into how to add a custom row there myself, for an unrelated thing, and found it difficult.

At this point a custom js in common.js would be much better than creating 7M pages.

In T403397#11136756, @Base wrote:

Asking to see if these pages are actually needed

I wonder whether there is a way to provide such download links on page info page. I remember looking into how to add a custom row there myself, for an unrelated thing, and found it difficult.

Do you mean the "Page information" (action=info) page? I suppose that could work. The pageinfo-header and pageinfo-footer interface messages allow you to add a section with links above or below the rest of the page. That could work through the MediaWiki:-namespace on the wiki.

Example:

• https://en.wikipedia.org/w/index.php?title=Banana&action=info (enwiki interface mods)
• https://test2.wikipedia.org/w/index.php?title=Banana&action=info (default-ish)

If you want it to be inside the "Basic information" table, then this could be done via the PageViewInfo extension. This is the extension that adds the "Page views in the past 30 days" row today. One could write a patch in Gerrit to add a link to the API for JSON data from there.

This would mean that these "Statistic" pages don't need to exist, as the default info action of the software would provide these links instead — would requiring millions of individually created, editable, and database-indexed wiki pages.

FWIW, At least one user thinks these pages in Russian Wikinews are useless and we can simply delete all of them. I'll ask in the main forum before moving forward.

So far, the community is on the side of just deleting these pages: https://ru.wikinews.org/wiki/%D0%92%D0%B8%D0%BA%D0%B8%D0%BD%D0%BE%D0%B2%D0%BE%D1%81%D1%82%D0%B8:%D0%A4%D0%BE%D1%80%D1%83%D0%BC/%D0%9E%D0%B1%D1%89%D0%B8%D0%B9

Given the clear community consensus, I've started deleting the pages: https://ru.wikinews.org/wiki/%D0%A1%D0%BB%D1%83%D0%B6%D0%B5%D0%B1%D0%BD%D0%B0%D1%8F:%D0%96%D1%83%D1%80%D0%BD%D0%B0%D0%BB%D1%8B?type=&user=Ladsgroup&page=&excludetempacct=1&wpdate=&tagfilter=&wpfilters%5B%5D=newusers&wpFormIdentifier=logeventslist&issubmitted=1

It's 6.8M of them, it's going to take a while.

With the current speed of deleting around 60 every minute. It'll take 71 days to finish.

@jcrespo this might cause some reduction in backup sizes of s3.

Progress report: Now 6.1M pages left. Around 10% have been deleted. Here is the size of externallinks table:

mysql:research@dbstore1007.eqiad.wmnet [ruwikinews]> select count(*) from externallinks;
+----------+
| count(*) |
+----------+
| 55219649 |
+----------+
1 row in set (20.300 sec)

Progress update: We have deleted around 2M so far. 5M more left:

mysql:research@dbstore1007.eqiad.wmnet [ruwikinews]> select count(*) from externallinks;
+----------+
| count(*) |
+----------+
| 49028285 |
+----------+
1 row in set (21.682 sec)

Nitpick: as I see, there are 20M and 50M accordingly.

In my opinion, the next best way to reduce the number of links is to replace the https://ru.wikinews.org/wiki/Шаблон:Социальные_закладки template with a script that dynamically adds these links to the page.

In T403397#11245696, @putnik wrote:

Nitpick: as I see, there are 20M and 50M accordingly.

20M rows of externallinks but 2M pages deleted. I should have been clearer that I'm talking about pages and not rows of externallinks

In my opinion, the next best way to reduce the number of links is to replace the https://ru.wikinews.org/wiki/Шаблон:Социальные_закладки template with a script that dynamically adds these links to the page.

Yeah, that'd be nice too. Will get to it after the deletion is done.

There is only 31 pages left in that category.

The number of rows have been dropped to 27M:

mysql:research@dbstore1007.eqiad.wmnet [ruwikinews]> select count(*) from externallinks;
+----------+
| count(*) |
+----------+
| 26122047 |
+----------+
1 row in set (10.476 sec)

I tried changing the template into using interwiki link but due to double escaping, it makes invalid url and there is no way out of it. I think the better option would be T405005: Implement mechanism to exclude a domain from externallinks database (LinkSearch)

I'm optimizing all the tables everywhere in T410589: Optimize all core tables, late 2025 so the table will become much smaller soon.

FTR:

root@db1157:/srv/sqldata/ruwikinews# ls -Ssh | head
total 81G
 17G externallinks.ibd
 14G logging.ibd
9.7G categorylinks.ibd

Compared to another host that isn't optimized yet:

root@db1175:/srv/sqldata/ruwikinews# ls -Ssh
total 256G
130G externallinks.ibd
 31G categorylinks.ibd
 22G page_props.ibd
 16G logging.ibd