Page MenuHomePhabricator
Create Task
Maniphest T184698

EventBus rejecting events because of malformed characters in the comment
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

The following event could not be sent because of encoding problems:

{
  "rev_content_changed": false,
  "performer": {
    "user_is_bot": false,
    "user_id": 2904,
    "user_edit_count": 19670,
    "user_text": "Zscout370",
    "user_groups": [
      "autopatrolled",
      "*",
      "user",
      "autoconfirmed"
    ]
  },
  "rev_content_model": "wikitext",
  "rev_minor_edit": false,
  "page_title": "File:Flag_of_Serbia.svg",
  "rev_sha1": "djwhgd8j5rthgcns9q8wzqluoz6ubl6",
  "page_namespace": 6,
  "page_id": 65419036,
  "rev_content_format": "wikitext",
  "database": "commonswiki",
  "rev_len": 2078,
  "rev_parent_id": 45834940,
  "meta": {
    "dt": "2018-01-11T01:05:45+00:00",
    "domain": "commons.wikimedia.org",
    "topic": "mediawiki.revision-create",
    "id": "8dc9b98b-f66b-11e7-ad25-1418776135a3",
    "uri": "https://commons.wikimedia.org/wiki/File:Flag_of_Serbia.svg",
    "request_id": "283a040e-2a0b-4814-84ff-f13663e1e842"
  },
  "rev_id": 47386029,
  "rev_timestamp": "2010-12-24T08:11:32Z",
  "comment": "uploaded a new version of File:Flag of Serbia.svg: 2010 update, based from http://commons.wikimedia.org/wiki/File:Flag_of_Serbia_updated.svg and using the Pantone colors stated in \"П ­утвр­ђО­ва­њу ­Оз­вПр­МО­ка ­Ве­лО­Ð",
  "page_is_redirect": false,
  "parsedcomment": "uploaded a new version of File:Flag of Serbia.svg: 2010 update, based from http://commons.wikimedia.org/wiki/File:Flag_of_Serbia_updated.svg and using the Pantone colors stated in "о ­утвр­ђи­ва­њу ­из­вор­ни­ка ­Ве­ли­�"
}

Error:

{
  "trace": "#0 /srv/mediawiki/php-1.31.0-wmf.16/extensions/EventBus/EventBus.php(78): EventBus::serializeEvents(array)\n#1 /srv/mediawiki/php-1.31.0-wmf.16/extensions/EventBus/EventBus.hooks.php(123): EventBus->send(array)\n#2 /srv/mediawiki/php-1.31.0-wmf.16/includes/deferred/MWCallableUpdate.php(30): Closure$EventBusHooks::onRevisionInsertComplete()\n#3 /srv/mediawiki/php-1.31.0-wmf.16/includes/deferred/DeferredUpdates.php(259): MWCallableUpdate->doUpdate()\n#4 /srv/mediawiki/php-1.31.0-wmf.16/includes/deferred/DeferredUpdates.php(210): DeferredUpdates::runUpdate(MWCallableUpdate, Wikimedia\\Rdbms\\LBFactoryMulti, string, integer)\n#5 /srv/mediawiki/php-1.31.0-wmf.16/includes/deferred/DeferredUpdates.php(131): DeferredUpdates::execute(array, string, integer)\n#6 /srv/mediawiki/php-1.31.0-wmf.16/includes/MediaWiki.php(900): DeferredUpdates::doUpdates(string)\n#7 /srv/mediawiki/php-1.31.0-wmf.16/includes/MediaWiki.php(720): MediaWiki->restInPeace(string, boolean)\n#8 [internal function]: Closure$MediaWiki::doPostOutputShutdown()\n#9 {main}",
  "code": 0,
  "file": "/srv/mediawiki/php-1.31.0-wmf.16/extensions/EventBus/EventBus.php:134",
  "message": "FormatJson::encode($events) failed: Malformed UTF-8 characters, possibly incorrectly encoded. Aborting send.",
  "class": "Exception"
}

Related Objects

Event Timeline

• mobrovac triaged this task as Medium priority.Jan 11 2018, 11:15 AM
• mobrovac created this task.
Restricted Application added a project: Analytics. · View Herald TranscriptJan 11 2018, 11:15 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
• Nuria moved this task from Incoming to Radar on the Analytics board.Jan 11 2018, 6:01 PM
• Pchelolo added a comment.Jan 16 2018, 7:27 PM

I've found several more similar issues, around 50 over the last 60 days.

What all of them have in common is that the comment/parsedcomment end with \uFFFD which is a Unicode replacement character used to replace invalid Unicode codepoints. However, that symbol should be perfectly OK to be serialized as JSON. Perhaps we're trying to serialize before the replacement was done by some other code?

• Pchelolo added a comment.EditedFeb 27 2018, 1:47 PM

Found another one in the logs today:

{
  "rev_content_changed": true,
  "performer": {
    "user_is_bot": false,
    "user_id": 677441,
    "user_edit_count": 33965,
    "user_text": "Тара-Амингу",
    "user_groups": [
      "editor",
      "sysop",
      "*",
      "user",
      "autoconfirmed"
    ],
    "user_registration_dt": "2011-05-31T18:23:07Z"
  },
  "rev_content_model": "wikitext",
  "rev_minor_edit": true,
  "page_title": "Военный_комиссариат",
  "rev_sha1": "9dxdwu7074evvia62pd1tkv1my3ejmu",
  "page_namespace": 0,
  "page_id": 1062791,
  "rev_content_format": "wikitext",
  "database": "ruwiki",
  "rev_len": 14390,
  "rev_parent_id": 54912654,
  "meta": {
    "dt": "2018-02-27T09:30:47+00:00",
    "domain": "ru.wikipedia.org",
    "topic": "mediawiki.revision-create",
    "id": "e49adb9b-1ba0-11e8-aa8c-141877613a22",
    "uri": "https://ru.wikipedia.org/wiki/%D0%92%D0%BE%D0%B5%D0%BD%D0%BD%D1%8B%D0%B9_%D0%BA%D0%BE%D0%BC%D0%B8%D1%81%D1%81%D0%B0%D1%80%D0%B8%D0%B0%D1%82",
    "request_id": "8ffae445-9beb-4bc5-999b-0a11a06ce760"
  },
  "rev_id": 54912857,
  "rev_timestamp": "2013-04-21T01:41:19Z",
  "comment": "ЗащОщеМа ВПеММый кПЌОссарОат: частый [[ВП:В|ваМЎалОзЌ]] ([РеЎактОрПваМОе=тПлькП автПпПЎтвержЎёММые] (Остекает 01:41, 28 апреля 2013 (UTC)) [ПереОЌеМПÐ",
  "page_is_redirect": false,
  "parsedcomment": "Защищена Военный комиссариат: частый <a href=\"/wiki/%D0%92%D0%B8%D0%BA%D0%B8%D0%BF%D0%B5%D0%B4%D0%B8%D1%8F:%D0%92\" class=\"mw-redirect\" title=\"Википедия:В\">вандализм</a> ([Редактирование=только автоподтверждённые] (истекает 01:41, 28 апреля 2013 (UTC)) [Переимено�"
}

The most interesting about this one is the rev_timestamp value - it's in 2013! Looking at the page history it's not entirely clear why this event was posted now.

Update: just confirmed that it's due to the page undeletion T166351

• mobrovac added a project: Platform Team Legacy (Later).Dec 20 2018, 12:17 PM
dcausse mentioned this in T225200: Fatal error during CirrusSearch-LinksUpdate job (CirrusTitleJob) from JobQueueGroup->push.Jun 6 2019, 1:09 PM
Krinkle added a project: Wikimedia-production-error.Jun 8 2019, 2:38 PM
Krinkle moved this task from Untriaged to Older on the Wikimedia-production-error board.
• Pchelolo edited projects, added Platform Engineering (Modern Event Platform (TEC2)); removed Platform Team Legacy (Later), Services (next).Jul 10 2019, 9:48 PM
WDoranWMF moved this task from Modern Event Platform (TEC2) to mop on the Platform Engineering board.Jul 26 2019, 6:30 PM
WDoranWMF edited projects, added Platform Team Initiatives (Modern Event Platform (TEC2)); removed Platform Engineering (Modern Event Platform (TEC2)).
• mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:09 PM
CCicalese_WMF subscribed.Mar 24 2020, 10:15 PM

@Pchelolo Is this still a problem? If not, can you please resolve this task? Thanks.

• Pchelolo closed this task as Resolved.Mar 24 2020, 10:18 PM
• Pchelolo claimed this task.

We haven't done anything to fix it, but gripping the logs for the last month doesn't reveal any new instances of this. Please reopen if it reoccurs.

Aklapper edited projects, added Analytics-Radar; removed Analytics.Jun 10 2020, 6:44 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL