Page MenuHomePhabricator
Create Task
Maniphest T105890

Missing database prefix for echo_event
Closed, ResolvedPublic
Actions

Description

2015-07-15 14:28:51 translatewiki.net translatewiki_net-bw_: [5ae27dd7] /w/api.php?action=query&format=json&meta=notifications&notsections=alert%7Cmessage&notgroupbysection=1&notmessageunreadfirst=1&notformat=flyout&notlimit=25&notprop=index%7Clist%7Ccount&uselang=en&_=1436979483197   DBQueryError from line 1276 of /srv/mediawiki/tags/2015-07-15_00:50:47/includes/db/Database.php: A database error has occurred. Did you forget to run maintenance/update.php after upgrading?  See: https://www.mediawiki.org/wiki/Manual:Upgrading#Run_the_update_script
Query: SELECT  echo_event.*  FROM `bw_echo_notification`,`bw_echo_event`   WHERE (notification_event=event_id) AND notification_user = '18167' AND notification_bundle_base = '0' AND notification_bundle_display_hash = '5cc2369d7dff9026524720d0d3ce4b81'  ORDER BY notification_timestamp DESC LIMIT 250  
Function: EchoEventMapper::fetchByUserBundleHash
Error: 1051 Unknown table 'echo_event' (127.0.0.1:3306)

#0 /srv/mediawiki/tags/2015-07-15_00:50:47/includes/db/Database.php(1231): DatabaseBase->reportQueryError('Unknown table '...', 1051, 'SELECT  echo_ev...', 'EchoEventMapper...', false)
#1 /srv/mediawiki/tags/2015-07-15_00:50:47/includes/db/Database.php(1757): DatabaseBase->query('SELECT  echo_ev...', 'EchoEventMapper...')
#2 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/mapper/EventMapper.php(90): DatabaseBase->select(Array, Array, Array, 'EchoEventMapper...', Array)
#3 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/formatters/BasicFormatter.php(569): EchoEventMapper->fetchByUserBundleHash(Object(User), '5cc2369d7dff902...', 'web', 'DESC', 250)
#4 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/formatters/BasicFormatter.php(594): EchoBasicFormatter->getRawBundleData(Object(EchoEvent), Object(User), 'web')
#5 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/formatters/BasicFormatter.php(156): EchoBasicFormatter->generateBundleData(Object(EchoEvent), Object(User), 'web')
#6 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/formatters/EditUserTalkFormatter.php(12): EchoBasicFormatter->applyChangeBeforeFormatting(Object(EchoEvent), Object(User), 'web')
#7 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/formatters/BasicFormatter.php(174): EchoEditUserTalkFormatter->applyChangeBeforeFormatting(Object(EchoEvent), Object(User), 'web')
#8 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/controller/NotificationController.php(374): EchoBasicFormatter->format(Object(EchoEvent), Object(User), 'web')
#9 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/DataOutputFormatter.php(104): EchoNotificationController::formatNotification(Object(EchoEvent), Object(User), 'flyout')
#10 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/api/ApiEchoNotifications.php(155): EchoDataOutputFormatter::formatOutput(Object(EchoNotification), 'flyout', Object(User))
#11 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/api/ApiEchoNotifications.php(94): ApiEchoNotifications->getPropList(Object(User), Array, 25, NULL, 'flyout', false)
#12 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/api/ApiEchoNotifications.php(30): ApiEchoNotifications->getSectionPropList(Object(User), 'alert', 25, NULL, 'flyout', false)
#13 /srv/mediawiki/tags/2015-07-15_00:50:47/includes/api/ApiQuery.php(286): ApiEchoNotifications->execute()
#14 /srv/mediawiki/tags/2015-07-15_00:50:47/includes/api/ApiMain.php(1095): ApiQuery->execute()
#15 /srv/mediawiki/tags/2015-07-15_00:50:47/includes/api/ApiMain.php(432): ApiMain->executeAction()
#16 /srv/mediawiki/tags/2015-07-15_00:50:47/includes/api/ApiMain.php(405): ApiMain->executeActionWithErrorHandling()
#17 /srv/mediawiki/tags/2015-07-15_00:50:47/api.php(88): ApiMain->execute()
#18 {main}

Details

SubjectRepoBranchLines +/-
Don't select echo_event.*mediawiki/extensions/Echomaster+1 -29
Select specific rows in EchoEventMapper::fetchByUserBundleHash()mediawiki/extensions/Echomaster+6 -2
Fix fatal DB exceptionmediawiki/extensions/Echomaster+2 -4
Fix fatal DB exceptionmediawiki/extensions/Echowmf/1.26wmf20+2 -4
Quick fix for missing table prefixmediawiki/extensions/Echomaster+3 -2
Customize query in gerrit

Related Objects

Event Timeline

Nikerabbit created this task.Jul 15 2015, 2:37 PM
Nikerabbit raised the priority of this task from to Needs Triage.
Nikerabbit updated the task description. (Show Details)
Nikerabbit added a project: Notifications.
Nikerabbit subscribed.
Restricted Application added a project: Collaboration-Team-Triage. · View Herald TranscriptJul 15 2015, 2:37 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Nemo_bis added a parent task: T41480: [DO NOT USE] Issues affecting translatewiki.net [superseded by #affects-translatewiki.net].Jul 15 2015, 5:49 PM
Legoktm subscribed.Jul 15 2015, 9:13 PM
			$res = $dbr->select(
				array( 'echo_notification', 'echo_event' ),
				array( 'echo_event.*' ),

It shouldn't be doing echo_event.*, it should have a hardcoded list of columns...

Catrope triaged this task as High priority.Jul 22 2015, 11:41 PM
Catrope set Security to None.
Catrope moved this task from Untriaged to Near-Term Interest on the Collaboration-Team-Triage board.
• DannyH moved this task from Near-Term Interest to Freezer on the Collaboration-Team-Triage board.Aug 21 2015, 9:18 PM
gerritbot subscribed.Aug 22 2015, 8:06 AM

Change 233093 had a related patch set uploaded (by Nemo bis):
Select specific rows in EchoEventMapper::fetchByUserBundleHash()

https://gerrit.wikimedia.org/r/233093

gerritbot added a project: Patch-For-Review.Aug 22 2015, 8:06 AM

Change 233094 had a related patch set uploaded (by Nikerabbit):
Quick fix for missing table prefix

https://gerrit.wikimedia.org/r/233094

gerritbot added a comment.Aug 22 2015, 3:56 PM

Change 233094 abandoned by Alex Monk:
Quick fix for missing table prefix

Reason:
I6c956738 solved this

https://gerrit.wikimedia.org/r/233094

gerritbot added a comment.Aug 22 2015, 3:57 PM

Change 233093 merged by jenkins-bot:
Select specific rows in EchoEventMapper::fetchByUserBundleHash()

https://gerrit.wikimedia.org/r/233093

Nemo_bis mentioned this in rMEXT4c0c7c85b21d: Updated mediawiki/extensions Project: mediawiki/extensions/Echo….Aug 22 2015, 3:57 PM
Nemo_bis mentioned this in rECHOc94c3f3dad66: Select specific rows in EchoEventMapper::fetchByUserBundleHash().
• Forrestbot added a project: WMF-deploy-2015-08-25_(1.26wmf20).Aug 23 2015, 1:01 AM
jcrespo added a project: Wikimedia-production-error.Aug 26 2015, 10:25 AM
jcrespo subscribed.

I think the issue I saw is related to this commit, please tell me otherwise:

I've seen some (not a huge) number of EchoEventMapper::fetchByUserBundleHash - Unknown column 'notification_timestamp' in 'field list' on mediawikiwiki while executing /rpc/RunJobs.php?wiki=mediawikiwiki&type=MWEchoNotificationEmailBundleJob&maxtime=30&maxmem=300M.

There seems to be a missmatch between the columns selected and the ones on production.

Full error trace:

{
  "_index": "logstash-2015.08.26",
  "_type": "mediawiki",
  "_id": "AU9pXK46hYw1KCY3mMep",
  "_score": null,
  "_source": {
    "message": "EchoEventMapper::fetchByUserBundleHash\t10.64.16.24\t1054\tUnknown column 'notification_timestamp' in 'field list' (10.64.16.24)\tSELECT  event_agent_id,event_agent_ip,event_extra,event_id,event_page_id,event_type,event_variant,notification_timestamp  FROM `echo_email_batch`,`echo_event`   WHERE (eeb_event_id=event_id) AND eeb_user_id = '[SANITIZED]' AND eeb_event_hash = '[SANITIZED]'  ORDER BY eeb_event_id DESC LIMIT 250  ",
    "@version": 1,
    "@timestamp": "2015-08-26T09:35:24.492Z",
    "type": "mediawiki",
    "host": "mw1004",
    "level": "ERROR",
    "tags": [
      "syslog",
      "es",
      "es",
      "normalized_message_trimmed"
    ],
    "channel": "wfLogDBError",
    "url": "/rpc/RunJobs.php?wiki=mediawikiwiki&type=MWEchoNotificationEmailBundleJob&maxtime=30&maxmem=300M",
    "ip": "127.0.0.1",
    "http_method": "POST",
    "server": "127.0.0.1",
    "referrer": null,
    "uid": "743bdb9",
    "process_id": 21155,
    "wiki": "mediawikiwiki",
    "db_server": "10.64.16.24",
    "db_name": "mediawikiwiki",
    "db_user": "wikiuser",
    "method": "DatabaseBase::reportQueryError",
    "errno": 1054,
    "error": "Unknown column 'notification_timestamp' in 'field list' (10.64.16.24)",
    "sql1line": "SELECT  event_agent_id,event_agent_ip,event_extra,event_id,event_page_id,event_type,event_variant,notification_timestamp  FROM `echo_email_batch`,`echo_event`   WHERE (eeb_event_id=event_id) AND eeb_user_id = '[SANITIZED]' AND eeb_event_hash = '[SANITIZED]'  ORDER BY eeb_event_id DESC LIMIT 250  ",
    "fname": "EchoEventMapper::fetchByUserBundleHash",
    "normalized_message": "EchoEventMapper::fetchByUserBundleHash\t10.64.16.24\t1054\tUnknown column 'notification_timestamp' in 'field list' (10.64.16.24)\tSELECT  event_agent_id,event_agent_ip,event_extra,event_id,event_page_id,event_type,event_variant,notification_timestamp  FROM `e"
  },
  "sort": [
    1440581724492
  ]
}

(query contents have been sanitized to avoid potential private data being exposed)

gerritbot added a comment.Aug 26 2015, 1:30 PM

Change 233964 had a related patch set uploaded (by Alex Monk):
Fix fatal DB exception

https://gerrit.wikimedia.org/r/233964

• Mattflaschen-WMF raised the priority of this task from High to Unbreak Now!.Aug 26 2015, 7:31 PM
• Mattflaschen-WMF subscribed.
gerritbot added a comment.Aug 26 2015, 7:46 PM

Change 234082 had a related patch set uploaded (by Alex Monk):
Fix fatal DB exception

https://gerrit.wikimedia.org/r/234082

gerritbot added a comment.Aug 26 2015, 7:47 PM

Change 233964 merged by jenkins-bot:
Fix fatal DB exception

https://gerrit.wikimedia.org/r/233964

Krenair mentioned this in rMEXT8eaa49a38d21: Updated mediawiki/extensions Project: mediawiki/extensions/Echo….Aug 26 2015, 7:47 PM
Krenair mentioned this in rECHO92dc3a0d3ce4: Fix fatal DB exception.
gerritbot added a comment.Aug 26 2015, 7:49 PM

Change 234082 merged by jenkins-bot:
Fix fatal DB exception

https://gerrit.wikimedia.org/r/234082

Krenair mentioned this in rMWd7498d443386: Updated mediawiki/core Project: mediawiki/extensions/Echo….Aug 26 2015, 7:49 PM
Krenair mentioned this in rECHO9289f8f35b63: Fix fatal DB exception.
• Forrestbot added a project: WMF-deploy-2015-09-01_(1.26wmf21).Aug 26 2015, 8:00 PM
• Mattflaschen-WMF edited projects, added Collaboration-Team-Archive-2015-2016; removed Collaboration-Team-Triage.Aug 26 2015, 8:10 PM
• Mattflaschen-WMF moved this task from Untriaged to QA Review on the Collaboration-Team-Archive-2015-2016 board.Aug 26 2015, 8:14 PM
• DannyH added a project: OKR-Work.Aug 27 2015, 7:03 PM
Etonkovidova subscribed.Aug 28 2015, 4:27 PM

Since in betalabs

SELECT * FROM information_schema.tables  WHERE  table_name = 'bw_echo_notification';
Empty set (0.01 sec)

just checked for the absence from fatal db errors and EchoEventMapper errors.

Etonkovidova moved this task from QA Review to Product Review on the Collaboration-Team-Archive-2015-2016 board.Aug 28 2015, 4:28 PM
• DannyH closed this task as Resolved.Aug 28 2015, 5:22 PM
• DannyH claimed this task.
• demon moved this task from Untriaged to Resolved on the Wikimedia-production-error board.Sep 1 2015, 5:33 PM
Nikerabbit reopened this task as Open.Aug 24 2016, 4:40 PM
Nikerabbit removed a project: Patch-For-Review.

Apperently https://gerrit.wikimedia.org/r/#/c/301382/ re-introduces this issue, in addition to another instance introduced earlier:
git blame EventMapper.php | grep 'echo_event\.\*'
24caf50f (Stephane Bisson 2016-03-04 14:23:02 -0500 160) array( 'echo_event.*' ),
6e2236db (Stephane Bisson 2016-07-27 10:52:18 -0400 189) array( 'echo_event.*' ),

Restricted Application added subscribers: Luke081515, TerraCodes. · View Herald TranscriptAug 24 2016, 4:40 PM
Paladox subscribed.Aug 24 2016, 4:44 PM
gerritbot added a comment.Aug 24 2016, 4:58 PM

Change 306465 had a related patch set uploaded (by Sbisson):
Don't select echo_event.*

https://gerrit.wikimedia.org/r/306465

gerritbot added a project: Patch-For-Review.Aug 24 2016, 4:58 PM
SBisson added a project: Collab-Team-Q1-July-Sep-2016.Aug 24 2016, 4:59 PM
SBisson moved this task from Untriaged to Needs Review on the Collab-Team-Q1-July-Sep-2016 board.
gerritbot added a comment.Aug 24 2016, 5:46 PM

Change 306465 merged by jenkins-bot:
Don't select echo_event.*

https://gerrit.wikimedia.org/r/306465

ReleaseTaggerBot added a project: MW-1.28-release (WMF-deploy-2016-08-30_(1.28.0-wmf.17)).Aug 24 2016, 6:00 PM
• DannyH removed • DannyH as the assignee of this task.Aug 25 2016, 11:04 PM
• DannyH subscribed.
• DannyH unsubscribed.
greg subscribed.Aug 29 2016, 6:23 PM

Status? Is this re-opened task still UBN! ? the associated patch is merged.

What needs to happen here?

Restricted Application added a subscriber: Jay8g. · View Herald TranscriptAug 29 2016, 6:23 PM
• Mattflaschen-WMF lowered the priority of this task from Unbreak Now! to High.Aug 29 2016, 8:05 PM

Will be QA'ed then resolved. Changing to High.

SBisson moved this task from Needs Review to QA Review on the Collab-Team-Q1-July-Sep-2016 board.Aug 30 2016, 11:10 AM
Etonkovidova added a comment.Aug 30 2016, 11:37 PM

Checked for log errors and the presence of code.

Etonkovidova moved this task from QA Review to Product Review on the Collab-Team-Q1-July-Sep-2016 board.Aug 30 2016, 11:37 PM
• jmatazzoni closed this task as Resolved.Sep 12 2016, 8:44 PM
• jmatazzoni claimed this task.
• Phabricator_maintenance added a project: affects-translatewiki.net.Sep 26 2019, 1:53 PM
• Phabricator_maintenance removed a parent task: T41480: [DO NOT USE] Issues affecting translatewiki.net [superseded by #affects-translatewiki.net].Sep 26 2019, 1:55 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL