Page MenuHomePhabricator

Unhandled Exception ("PhutilAggregateException") when using Phabricator in languages that translated "AM"
Closed, ResolvedPublicBUG REPORT

Description

Steps to replicate the issue (include links if applicable):

  • Set Phabricator Locale to Finnish (presumably)
  • Try searching something from the top right corner

What happens?:

Unhandled Exception ("PhutilAggregateException")	
All of the configured Fulltext Search services failed.
    - Exception: DateTime::__construct(): Failed to parse time string (2024-04-23 12:00 epp.) at position 17 (e): The timezone could not be found in the database

What should have happened instead?:
Search results.

Software version (on Special:Version page; skip for WMF-hosted wikis like Wikipedia):

Other information (browser name/version, screenshots, etc.):

Started happening after the latest update.

[2024-04-23 19:28:57] EXCEPTION: (Exception) DateTime::__construct(): Failed to parse time string (2024-04-23 12:00 epp.) at position 17 (e): The timezone could not be found in the database at [<phorge>/src/view/form/control/AphrontFormDateControlValue.php:267]
arcanist(), ava(), phorge(), translations(), wmf-ext-misc()
  #0 <#2> DateTime::__construct(string, DateTimeZone) called at [<phorge>/src/view/form/control/AphrontFormDateControlValue.php:267]
  #1 <#2> AphrontFormDateControlValue::getFormattedDateFromParts(string, string, string, string) called at [<phorge>/src/view/form/control/AphrontFormDateControlValue.php:110]
  #2 <#2> AphrontFormDateControlValue::newFromEpoch(PhabricatorUser, string) called at [<phorge>/src/applications/calendar/query/PhabricatorCalendarEventSearchEngine.php:590]
  #3 <#2> PhabricatorCalendarEventSearchEngine::getSafeDate(NULL) called at [<phorge>/src/applications/calendar/query/PhabricatorCalendarEventSearchEngine.php:155]
  #4 <#2> PhabricatorCalendarEventSearchEngine::getQueryDateRange(NULL, NULL, string, boolean) called at [<phorge>/src/applications/calendar/query/PhabricatorCalendarEventSearchEngine.php:120]
  #5 <#2> PhabricatorCalendarEventSearchEngine::buildQueryFromParameters(array) called at [<phorge>/src/applications/search/engine/PhabricatorApplicationSearchEngine.php:168]
  #6 <#2> PhabricatorApplicationSearchEngine::buildQueryFromSavedQuery(PhabricatorSavedQuery) called at [<phorge>/src/applications/calendar/query/PhabricatorCalendarEventSearchEngine.php:86]
  #7 <#2> PhabricatorCalendarEventSearchEngine::buildQueryFromSavedQuery(PhabricatorSavedQuery) called at [<phorge>/src/applications/search/fulltextstorage/PhabricatorFerretFulltextStorageEngine.php:77]
  #8 <#2> PhabricatorFerretFulltextStorageEngine::executeSearch(PhabricatorSavedQuery) called at [<phorge>/src/infrastructure/cluster/search/PhabricatorSearchService.php:266]
  #9 phlog(Exception) called at [<phorge>/src/infrastructure/cluster/search/PhabricatorSearchService.php:275]
  #10 PhabricatorSearchService::newResultSet(PhabricatorSavedQuery, PhabricatorSearchDocumentQuery) called at [<phorge>/src/applications/search/query/PhabricatorSearchDocumentQuery.php:52]
  #11 PhabricatorSearchDocumentQuery::loadPage() called at [<phorge>/src/infrastructure/query/policy/PhabricatorPolicyAwareQuery.php:251]
  #12 PhabricatorPolicyAwareQuery::execute() called at [<phorge>/src/infrastructure/query/PhabricatorOffsetPagedQuery.php:46]
  #13 PhabricatorOffsetPagedQuery::executeWithOffsetPager(PHUIPagerView) called at [<phorge>/src/applications/search/engine/PhabricatorApplicationSearchEngine.php:1036]
  #14 PhabricatorApplicationSearchEngine::executeQuery(PhabricatorSearchDocumentQuery, PHUIPagerView) called at [<phorge>/src/applications/search/controller/PhabricatorApplicationSearchController.php:280]
  #15 PhabricatorApplicationSearchController::processSearchRequest() called at [<phorge>/src/applications/search/controller/PhabricatorApplicationSearchController.php:91]
  #16 PhabricatorApplicationSearchController::processRequest() called at [<phorge>/src/aphront/AphrontController.php:29]
  #17 AphrontController::handleRequest(AphrontRequest) called at [<phorge>/src/aphront/AphrontController.php:71]
  #18 AphrontController::delegateToController(PhabricatorApplicationSearchController) called at [<phorge>/src/applications/search/controller/PhabricatorSearchController.php:96]
  #19 PhabricatorSearchController::handleRequest(AphrontRequest) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:284]
  #20 AphrontApplicationConfiguration::processRequest(AphrontRequest, PhutilDeferredLog, AphrontPHPHTTPSink, MultimeterControl) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:204]
  #21 AphrontApplicationConfiguration::runHTTPRequest(AphrontPHPHTTPSink) called at [<phorge>/webroot/index.php:35]
[2024-04-23 19:28:57] EXCEPTION: (PhutilAggregateException) All of the configured Fulltext Search services failed.\n    - Exception: DateTime::__construct(): Failed to parse time string (2024-04-23 12:00 epp.) at position 17 (e): The timezone could not be found in the database at [<phorge>/src/infrastructure/cluster/search/PhabricatorSearchService.php:279]
arcanist(), ava(), phorge(), translations(), wmf-ext-misc()
  #0 <#3> DateTime::__construct(string, DateTimeZone) called at [<phorge>/src/view/form/control/AphrontFormDateControlValue.php:267]
  #1 <#3> AphrontFormDateControlValue::getFormattedDateFromParts(string, string, string, string) called at [<phorge>/src/view/form/control/AphrontFormDateControlValue.php:110]
  #2 <#3> AphrontFormDateControlValue::newFromEpoch(PhabricatorUser, string) called at [<phorge>/src/applications/calendar/query/PhabricatorCalendarEventSearchEngine.php:590]
  #3 <#3> PhabricatorCalendarEventSearchEngine::getSafeDate(NULL) called at [<phorge>/src/applications/calendar/query/PhabricatorCalendarEventSearchEngine.php:155]
  #4 <#3> PhabricatorCalendarEventSearchEngine::getQueryDateRange(NULL, NULL, string, boolean) called at [<phorge>/src/applications/calendar/query/PhabricatorCalendarEventSearchEngine.php:120]
  #5 <#3> PhabricatorCalendarEventSearchEngine::buildQueryFromParameters(array) called at [<phorge>/src/applications/search/engine/PhabricatorApplicationSearchEngine.php:168]
  #6 <#3> PhabricatorApplicationSearchEngine::buildQueryFromSavedQuery(PhabricatorSavedQuery) called at [<phorge>/src/applications/calendar/query/PhabricatorCalendarEventSearchEngine.php:86]
  #7 <#3> PhabricatorCalendarEventSearchEngine::buildQueryFromSavedQuery(PhabricatorSavedQuery) called at [<phorge>/src/applications/search/fulltextstorage/PhabricatorFerretFulltextStorageEngine.php:77]
  #8 <#3> PhabricatorFerretFulltextStorageEngine::executeSearch(PhabricatorSavedQuery) called at [<phorge>/src/infrastructure/cluster/search/PhabricatorSearchService.php:266]
  #9 <#2> PhabricatorSearchService::newResultSet(PhabricatorSavedQuery, PhabricatorSearchDocumentQuery) called at [<phorge>/src/applications/search/query/PhabricatorSearchDocumentQuery.php:52]
  #10 <#2> PhabricatorSearchDocumentQuery::loadPage() called at [<phorge>/src/infrastructure/query/policy/PhabricatorPolicyAwareQuery.php:251]
  #11 <#2> PhabricatorPolicyAwareQuery::execute() called at [<phorge>/src/infrastructure/query/PhabricatorOffsetPagedQuery.php:46]
  #12 <#2> PhabricatorOffsetPagedQuery::executeWithOffsetPager(PHUIPagerView) called at [<phorge>/src/applications/search/engine/PhabricatorApplicationSearchEngine.php:1036]
  #13 <#2> PhabricatorApplicationSearchEngine::executeQuery(PhabricatorSearchDocumentQuery, PHUIPagerView) called at [<phorge>/src/applications/search/controller/PhabricatorApplicationSearchController.php:280]
  #14 <#2> PhabricatorApplicationSearchController::processSearchRequest() called at [<phorge>/src/applications/search/controller/PhabricatorApplicationSearchController.php:91]
  #15 <#2> PhabricatorApplicationSearchController::processRequest() called at [<phorge>/src/aphront/AphrontController.php:29]
  #16 <#2> AphrontController::handleRequest(AphrontRequest) called at [<phorge>/src/aphront/AphrontController.php:71]
  #17 <#2> AphrontController::delegateToController(PhabricatorApplicationSearchController) called at [<phorge>/src/applications/search/controller/PhabricatorSearchController.php:96]
...

Event Timeline

Aklapper updated the task description. (Show Details)

Set Phabricator Locale to Finnish (presumably)

How to do that? There is a Translation setting at https://phabricator.wikimedia.org/settings/panel/language/ and there are Timezone, Date Format, Time Format settings at https://phabricator.wikimedia.org/settings/panel/datetime/ .

Could you share all your four settings which in combination trigger this issue, please?

Try searching something from the top right corner

What is something and on which full URL? Search is context dependent. Please always provide clear steps to reproduce which leave no room for interpretation.

Probably triggered because

./src/translations/ArcanistCoreFi.php:      'AM' => 'epp.',
./src/translations/ArcanistCoreFi.php:      'am' => 'epp.',
./projects/arcanist/core/fi.json:	"80d305c58f97edfa": "epp.",
./projects/arcanist/core/fi.json:	"96e8155732e8324a": "epp.",

should not be marked as translatable and user setting for Time Format is not set to 24 Hour, 14:34 but to another option using AM/PM stuff

Anything on any URL. E.g. this query fails for me https://phabricator.wikimedia.org/search/query/m03cFmqV18Sl/#R while logged in. I typed anything on the box on this page.

In my preferences I have Language set to Finnish and date and time settings are as follows:

image.png (517×894 px, 38 KB)
(i.e. AM/PM option is not selected)

The issue is the code is given 2024-04-23 12:00 epp. and attempt to use epp. as a timezone for the PHP DateTimeZone object construction. But that is not possible:

From phab1004.eqiad.wmnet:

PHP 7.4.33 (cli) (built: Apr 12 2024 00:02:16) ( NTS )
php > DateTimeZone( 'epp' );
PHP Warning:  Uncaught Error: Call to undefined function DateTimeZone() in php shell code:1
Stack trace:
#0 {main}
  thrown in php shell code on line 1

Warning: Uncaught Error: Call to undefined function DateTimeZone() in php shell code:1
Stack trace:
#0 {main}
  thrown in php shell code on line 1

I have no idea what epp. means, but I echo what Andre said: that should not be transformed and left as-is unless AphrontFormDateControlValue supports the translation?

Aklapper renamed this task from Unhandled Exception ("PhutilAggregateException") when using Phabricator in Finnish to Unhandled Exception ("PhutilAggregateException") when using Phabricator in languages that translated "AM".Apr 24 2024, 2:18 PM

__tests__ files should probably not be included in phabricator-translations' exports in the first place. But I doubt that's the cause of this issue, since the string coming from a testcase shouldn't be used in the first place.

It would be nice to see what the stack trace for this error is, if one is present.

It would be nice to see what the stack trace for this error is, if one is present.

On the Phab side it's in the task description

Pppery claimed this task.

Looking at this more closely, probably it is the test case string, and the weird way in which Phabricator's translations are keyed by proto-English strings rather than any message key means that the message extracted from a test case ends up being picked up by the date formatting code. I'll submit a patch to unbreak this issue soon, and will file another ticket to not extract test case strings in the first place.

(oops, did not mean to resolve)

Oddly I could never reproduce this, but the stack trace was good enough to let me understand what happens by looking through the code:

  • PhabricatorCalendarEventSearchEngine::getSafeDate(NULL) ends up calling AphrontFormDateControlValue::newFromEpoch
  • This then calls AphrontFormDateControlValue:formatTime with the hardcoded date format string ''Y!m!d!g:i A'
  • This format is passed through to phabricator_format_local_time and then PhutilTranslator->translateDate unchanged.
  • translateDate splits it into Y!m!d!g:i and 'A', since the latter is in the hardcoded list of translatable time elements and the former isn't. The first part results in a string like 2024!04!24!5:21 , and the second part results in a translation of AM or PM (which is in the translatewiki list because of the above bug, although I guess it should be for unrelated reasons)
  • This gets passed down the call stack back to AphrontFormDateControlValue::newFromEpoch, which splits it by the !s into year, month, day, and time. The translation of "AM" or "PM" is treated as part of the time
    • This then gets passed down to getFormattedDateFromParts, which then tries to interpret it as a date again to render it in a different format, and fails since it doesn't recognize "E.P.P"

Suggested fixes:

  • Immediately: delete the AM string from phabricator-translations and deploy it.
  • For Phabricator (Upstream) : fix the above code to use some user with the default internationalization settings rather than the current viewer since bad translations shouldn't cause internal code to break.
  • For Phabricator (Upstream) : Consider redesigning the above workflow, it seems very complicated and brittle for what should be a simple task.
  • For Phabricator (Upstream) : Reconsider the way date translation is supported - including a hardcoded list of date elements whose output can be translated seems contrary to i18n best practices - it would be better to give translators more control over the ordering of date elements etc. (Edit June 2024: this is wrong - upstream date handling is confusing and poorly documented but not fundamentally broken in the way I thought)
  • Phabricator-translations should not extract strings from test cases
  • Phabricator-translations should support some better way of handling dates, once upstream does.

Change #1023926 had a related patch set uploaded (by Pppery; author: Pppery):

[phabricator/translations@wmf/stable] Delete "AM" and "PM" translations breaking search

https://gerrit.wikimedia.org/r/1023926

(the missing step that was stopping me from reproducing was that the "Document Types" has to include "Event", either explicitly or implicitly by being empty. It won't reproduce if only "task" is selected, which is one of the possible defaults)

Aklapper raised the priority of this task from Low to Medium.Apr 25 2024, 9:12 AM
Aklapper moved this task from To Triage to Misc on the Phabricator board.

Change #1023926 merged by Brennen Bearnes:

[phabricator/translations@wmf/stable] Delete "AM" and "PM" translations breaking search

https://gerrit.wikimedia.org/r/1023926

  • For Phabricator (Upstream) : fix the above code to use some user with the default internationalization settings rather than the current viewer since bad translations shouldn't cause internal code to break.

I think this might do that job but I do not know if it maybe 'resets' localized date formats to a default one in some UI places:

1commit a42564511067cc3339818dbd790be06e367963b9 (HEAD -> master)
2Author: Andre Klapper <aklapper@wikimedia.org>
3Date: Fri May 3 18:42:38 2024 +0200
4
5 When creating a DateTime, do not localize the `A` format via `formatTime()` here. With translations of AM and PM in place, it blows up search for calendar events for translation letters not included in $translatable in https://we.phorge.it/source/arcanist/browse/master/src/internationalization/PhutilTranslator.php$222
6
7diff --git a/src/view/form/control/AphrontFormDateControlValue.php b/src/view/form/control/AphrontFormDateControlValue.php
8index 873ad44fe4..b55e2616c3 100644
9--- a/src/view/form/control/AphrontFormDateControlValue.php
10+++ b/src/view/form/control/AphrontFormDateControlValue.php
11@@ -97,7 +97,8 @@ final class AphrontFormDateControlValue extends Phobject {
12 return $value;
13 }
14
15- $readable = $value->formatTime($epoch, 'Y!m!d!g:i A');
16+ $dt = new DateTime("@$epoch");
17+ $readable = $dt->format('Y!m!d!g:i A');
18 $readable = explode('!', $readable, 4);
19
20 $year = $readable[0];
21diff --git a/src/view/viewutils.php b/src/view/viewutils.php
22index 956f6ca2c2..96f555bcd3 100644
23--- a/src/view/viewutils.php
24+++ b/src/view/viewutils.php
25@@ -132,8 +132,8 @@ function phabricator_format_local_time($epoch, $user, $format) {
26 try {
27 $date = new DateTime('@'.$epoch);
28 } catch (Exception $ex) {
29- // NOTE: DateTime throws an empty exception if the format is invalid,
30- // just replace it with a useful one.
31+ // NOTE: Before PHP 8.3, DateTime throws an empty exception if the format
32+ // is invalid, just replace it with a useful one.
33 throw new Exception(
34 pht("Construction of a DateTime() with epoch '%s' ".
35 "raised an exception.", $epoch));

Looks reasonable in principle to me. And I don't think it would reset localized time formats - the code below parses it out into year, month, day, which means any further uses have to go through DateTime again. This would also resolve "Consider redesigning the above workflow, it seems very complicated and brittle for what should be a simple task." IMO.

There still should be an upstream task to reconsider how date translation works entirely. But that's lower-priority and doesn't need to block this task.

And the commit message of your commit is wrong - it blows up all searches for calendar events AFAICS.

  • For Phabricator (Upstream) : fix the above code to use some user with the default internationalization settings rather than the current viewer since bad translations shouldn't cause internal code to break.

FYI Filed upstream as https://we.phorge.it/T15811 / https://we.phorge.it/D25618

Deployed on 2024-05-05.