Page MenuHomePhabricator

ReplaceText should ignore translation pages
Closed, ResolvedPublic4 Estimated Story PointsFeature

Description

Feature summary:

Pages handled by Translate extension should be ignored and not presented in the search results. ReplaceText is not allowed to touch them anyway, so why present them?

Use case(s):

The translation results clutter the list.

Benefits:

When having a maximum limit for replacements, the cluttering of results can be very annoying and slow down the replacement work as multiple passes need to be done while waiting for the job queue to be processed each time.

Implementation details

  • When titles are being renamed using ReplaceText, translatable pages and translation pages are skipped.
  • When content in pages are being replaced using ReplaceText, translation pages are skipped.

Event Timeline

I did some experiments and will share here what worked. It's not a finished solution as it has no checks for the presence of Translate etc., but I hope someone could consider the idea and create a good quality patch:

 diff --git a/src/Search.php b/src/Search.php
index 86263a2..4b7cc79 100644
--- a/src/Search.php
+++ b/src/Search.php
@@ -22,6 +22,7 @@ namespace MediaWiki\Extension\ReplaceText;
 use Title;
 use Wikimedia\Rdbms\IDatabase;
 use Wikimedia\Rdbms\IResultWrapper;
+use MediaWiki\Extension\Translate\PageTranslation\TranslatablePage;
 
 class Search {
 
@@ -62,8 +63,16 @@ class Search {
                        'ORDER BY' => 'page_namespace, page_title',
                        'LIMIT' => $wgReplaceTextResultsLimit
                ];
+               $result = $dbr->select( $tables, $vars, $conds, __METHOD__, $options );
+               $pages = [];
+               foreach ( $result as $row ) {
+                       $title = Title::newFromText( $row->page_title );
+                       if ( !TranslatablePage::isTranslationPage( $title ) ) {
+                               $pages[] = $row;
+                       }
+               }
 
-               return $dbr->select( $tables, $vars, $conds, __METHOD__, $options );
+               return $pages;
        }
 
        /**
diff --git a/src/SpecialReplaceText.php b/src/SpecialReplaceText.php
index 3885957..a913bbd 100644
--- a/src/SpecialReplaceText.php
+++ b/src/SpecialReplaceText.php
@@ -470,7 +470,7 @@ class SpecialReplaceText extends SpecialPage {
                                $this->prefix,
                                $this->use_regex
                        );
-                       $count = $res->numRows();
+                       $count = count( $res );
                        if ( $count > 0 ) {
                                return $this->msg( 'replacetext_warning' )->numParams( $count )
                                        ->params( "<code><nowiki>{$this->replacement}</nowiki></code>" )->parse();
@@ -483,7 +483,7 @@ class SpecialReplaceText extends SpecialPage {
                                $this->prefix,
                                $this->use_regex
                        );
-                       $count = $res->numRows();
+                       $count = count( $res );
                        if ( $count > 0 ) {
                                return $this->msg( 'replacetext_warning' )->numParams( $count )
                                        ->params( $this->replacement )->parse();

Change 891258 had a related patch set uploaded (by Jdlrobson; author: Ilmari Lauhakangas):

[mediawiki/extensions/ReplaceText@master] Ignore translation pages

https://gerrit.wikimedia.org/r/891258

abi_ triaged this task as Low priority.

Change 921345 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/ReplaceText@master] Add ReplaceTextFilterTitleForEdit hook to filter pages to not rename

https://gerrit.wikimedia.org/r/921345

Change 895721 had a related patch set uploaded (by Abijeet Patro; author: Ilmari Lauhakangas):

[mediawiki/extensions/Translate@master] Implement ReplaceTextFilterTitleForEdit

https://gerrit.wikimedia.org/r/895721

abi_ set the point value for this task to 4.May 19 2023, 2:10 PM

Change 922801 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/ReplaceText@master] Add ReplaceTextFilterTitleForEdit hook to filter out title names

https://gerrit.wikimedia.org/r/922801

Change 922802 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] Implement ReplaceTextFilterTitleForEdit

https://gerrit.wikimedia.org/r/922802

I've updated the code to add two new hooks to ReplaceText extension:

  1. To allow extensions to skip a page from being edited based on its content
  2. To allow extensions to skip a page from being moved based on its title

The Translate extension listens to these hooks and removes translatable, translation pages, and translation units from this list.

Here's a screenshot of how things look. Note that Translations:DEV Wiki:Translatable and Translations:Heading Translate are not marked as translatable.

image.png (2×1 px, 468 KB)

I'm guessing that this was an Athens hackathon project. But what, if anything, is going on with it now? Should I be reviewing any of these five or so patches?

I'm guessing that this was an Athens hackathon project. But what, if anything, is going on with it now? Should I be reviewing any of these five or so patches?

The initial patch was contributed by a volunteer a while ago but I took it over due to the complexity and yes, we could use your inputs on these patches.

The following two patches add two new hooks to ReplaceText extension:

  1. 921345: Add ReplaceTextFilterPageTitlesForEdit hook to avoid pages for edit | https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ReplaceText/+/921345
  2. 922801: Add ReplaceTextFilterPageTitlesForRename hook to avoid pages to rename | https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ReplaceText/+/922801

These are then implemented in the Translate extension to hide certain pages from being edited / renamed:

  1. 895721: Implement ReplaceTextFilterPageTitlesForEdit | https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/895721
  2. 922802: Implement ReplaceTextFilterPageTitlesForRename | https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/922802

Okay, that makes sense. I looked through the code, and the overall concept, and the implementation, both look good to me. Let me know when/if the two Approved Revs patches are ready for merging, and I will approve them.

Okay, that makes sense. I looked through the code, and the overall concept, and the implementation, both look good to me. Let me know when/if the two Approved Revs patches are ready for merging, and I will approve them.

@Yaron_Koren The patches for the ReplaceText extension are ready to be merged. The Translate extension pages can be merged in later.

Change 921345 merged by jenkins-bot:

[mediawiki/extensions/ReplaceText@master] Add ReplaceTextFilterPageTitlesForEdit hook to avoid pages for edit

https://gerrit.wikimedia.org/r/921345

Change 922801 merged by jenkins-bot:

[mediawiki/extensions/ReplaceText@master] Add ReplaceTextFilterPageTitlesForRename hook to avoid pages to rename

https://gerrit.wikimedia.org/r/922801

Change 895721 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Implement ReplaceTextFilterPageTitlesForEdit

https://gerrit.wikimedia.org/r/895721

Change 922802 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Implement ReplaceTextFilterPageTitlesForRename

https://gerrit.wikimedia.org/r/922802

Below is an image where we are replacing "Help us" with "Help them" on translatewiki.net

The contents are replaced in the source translation page and other normal pages where the text appears. None of the translation pages appear in the list of pages that will be edited.

When moving / renaming pages, the translatable source page, and then translation pages are skipped from being renamed.

image.png (1×1 px, 198 KB)

Moving this to the done column, but leaving it open for few days to gather any feedback

Thanks a lot for implementing this!