Page MenuHomePhabricator

Missing database prefix for echo_event
Closed, ResolvedPublic

Description

2015-07-15 14:28:51 translatewiki.net translatewiki_net-bw_: [5ae27dd7] /w/api.php?action=query&format=json&meta=notifications&notsections=alert%7Cmessage&notgroupbysection=1&notmessageunreadfirst=1&notformat=flyout&notlimit=25&notprop=index%7Clist%7Ccount&uselang=en&_=1436979483197   DBQueryError from line 1276 of /srv/mediawiki/tags/2015-07-15_00:50:47/includes/db/Database.php: A database error has occurred. Did you forget to run maintenance/update.php after upgrading?  See: https://www.mediawiki.org/wiki/Manual:Upgrading#Run_the_update_script
Query: SELECT  echo_event.*  FROM `bw_echo_notification`,`bw_echo_event`   WHERE (notification_event=event_id) AND notification_user = '18167' AND notification_bundle_base = '0' AND notification_bundle_display_hash = '5cc2369d7dff9026524720d0d3ce4b81'  ORDER BY notification_timestamp DESC LIMIT 250  
Function: EchoEventMapper::fetchByUserBundleHash
Error: 1051 Unknown table 'echo_event' (127.0.0.1:3306)

#0 /srv/mediawiki/tags/2015-07-15_00:50:47/includes/db/Database.php(1231): DatabaseBase->reportQueryError('Unknown table '...', 1051, 'SELECT  echo_ev...', 'EchoEventMapper...', false)
#1 /srv/mediawiki/tags/2015-07-15_00:50:47/includes/db/Database.php(1757): DatabaseBase->query('SELECT  echo_ev...', 'EchoEventMapper...')
#2 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/mapper/EventMapper.php(90): DatabaseBase->select(Array, Array, Array, 'EchoEventMapper...', Array)
#3 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/formatters/BasicFormatter.php(569): EchoEventMapper->fetchByUserBundleHash(Object(User), '5cc2369d7dff902...', 'web', 'DESC', 250)
#4 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/formatters/BasicFormatter.php(594): EchoBasicFormatter->getRawBundleData(Object(EchoEvent), Object(User), 'web')
#5 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/formatters/BasicFormatter.php(156): EchoBasicFormatter->generateBundleData(Object(EchoEvent), Object(User), 'web')
#6 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/formatters/EditUserTalkFormatter.php(12): EchoBasicFormatter->applyChangeBeforeFormatting(Object(EchoEvent), Object(User), 'web')
#7 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/formatters/BasicFormatter.php(174): EchoEditUserTalkFormatter->applyChangeBeforeFormatting(Object(EchoEvent), Object(User), 'web')
#8 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/controller/NotificationController.php(374): EchoBasicFormatter->format(Object(EchoEvent), Object(User), 'web')
#9 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/DataOutputFormatter.php(104): EchoNotificationController::formatNotification(Object(EchoEvent), Object(User), 'flyout')
#10 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/api/ApiEchoNotifications.php(155): EchoDataOutputFormatter::formatOutput(Object(EchoNotification), 'flyout', Object(User))
#11 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/api/ApiEchoNotifications.php(94): ApiEchoNotifications->getPropList(Object(User), Array, 25, NULL, 'flyout', false)
#12 /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/Echo/includes/api/ApiEchoNotifications.php(30): ApiEchoNotifications->getSectionPropList(Object(User), 'alert', 25, NULL, 'flyout', false)
#13 /srv/mediawiki/tags/2015-07-15_00:50:47/includes/api/ApiQuery.php(286): ApiEchoNotifications->execute()
#14 /srv/mediawiki/tags/2015-07-15_00:50:47/includes/api/ApiMain.php(1095): ApiQuery->execute()
#15 /srv/mediawiki/tags/2015-07-15_00:50:47/includes/api/ApiMain.php(432): ApiMain->executeAction()
#16 /srv/mediawiki/tags/2015-07-15_00:50:47/includes/api/ApiMain.php(405): ApiMain->executeActionWithErrorHandling()
#17 /srv/mediawiki/tags/2015-07-15_00:50:47/api.php(88): ApiMain->execute()
#18 {main}

Details

Related Gerrit Patches:
mediawiki/extensions/Echo : masterDon't select echo_event.*
mediawiki/extensions/Echo : masterSelect specific rows in EchoEventMapper::fetchByUserBundleHash()
mediawiki/extensions/Echo : masterFix fatal DB exception
mediawiki/extensions/Echo : wmf/1.26wmf20Fix fatal DB exception
mediawiki/extensions/Echo : masterQuick fix for missing table prefix

Event Timeline

Nikerabbit raised the priority of this task from to Needs Triage.
Nikerabbit updated the task description. (Show Details)
Nikerabbit added a project: Notifications.
Nikerabbit added a subscriber: Nikerabbit.
Restricted Application added a project: Collaboration-Team-Triage. · View Herald TranscriptJul 15 2015, 2:37 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
			$res = $dbr->select(
				array( 'echo_notification', 'echo_event' ),
				array( 'echo_event.*' ),

It shouldn't be doing echo_event.*, it should have a hardcoded list of columns...

Catrope triaged this task as High priority.Jul 22 2015, 11:41 PM
Catrope set Security to None.
Catrope moved this task from Untriaged to Near-Term Interest on the Collaboration-Team-Triage board.

Change 233093 had a related patch set uploaded (by Nemo bis):
Select specific rows in EchoEventMapper::fetchByUserBundleHash()

https://gerrit.wikimedia.org/r/233093

Change 233094 had a related patch set uploaded (by Nikerabbit):
Quick fix for missing table prefix

https://gerrit.wikimedia.org/r/233094

Change 233094 abandoned by Alex Monk:
Quick fix for missing table prefix

Reason:
I6c956738 solved this

https://gerrit.wikimedia.org/r/233094

Change 233093 merged by jenkins-bot:
Select specific rows in EchoEventMapper::fetchByUserBundleHash()

https://gerrit.wikimedia.org/r/233093

jcrespo added a subscriber: jcrespo.

I think the issue I saw is related to this commit, please tell me otherwise:

I've seen some (not a huge) number of EchoEventMapper::fetchByUserBundleHash - Unknown column 'notification_timestamp' in 'field list' on mediawikiwiki while executing /rpc/RunJobs.php?wiki=mediawikiwiki&type=MWEchoNotificationEmailBundleJob&maxtime=30&maxmem=300M.

There seems to be a missmatch between the columns selected and the ones on production.

Full error trace:

{
  "_index": "logstash-2015.08.26",
  "_type": "mediawiki",
  "_id": "AU9pXK46hYw1KCY3mMep",
  "_score": null,
  "_source": {
    "message": "EchoEventMapper::fetchByUserBundleHash\t10.64.16.24\t1054\tUnknown column 'notification_timestamp' in 'field list' (10.64.16.24)\tSELECT  event_agent_id,event_agent_ip,event_extra,event_id,event_page_id,event_type,event_variant,notification_timestamp  FROM `echo_email_batch`,`echo_event`   WHERE (eeb_event_id=event_id) AND eeb_user_id = '[SANITIZED]' AND eeb_event_hash = '[SANITIZED]'  ORDER BY eeb_event_id DESC LIMIT 250  ",
    "@version": 1,
    "@timestamp": "2015-08-26T09:35:24.492Z",
    "type": "mediawiki",
    "host": "mw1004",
    "level": "ERROR",
    "tags": [
      "syslog",
      "es",
      "es",
      "normalized_message_trimmed"
    ],
    "channel": "wfLogDBError",
    "url": "/rpc/RunJobs.php?wiki=mediawikiwiki&type=MWEchoNotificationEmailBundleJob&maxtime=30&maxmem=300M",
    "ip": "127.0.0.1",
    "http_method": "POST",
    "server": "127.0.0.1",
    "referrer": null,
    "uid": "743bdb9",
    "process_id": 21155,
    "wiki": "mediawikiwiki",
    "db_server": "10.64.16.24",
    "db_name": "mediawikiwiki",
    "db_user": "wikiuser",
    "method": "DatabaseBase::reportQueryError",
    "errno": 1054,
    "error": "Unknown column 'notification_timestamp' in 'field list' (10.64.16.24)",
    "sql1line": "SELECT  event_agent_id,event_agent_ip,event_extra,event_id,event_page_id,event_type,event_variant,notification_timestamp  FROM `echo_email_batch`,`echo_event`   WHERE (eeb_event_id=event_id) AND eeb_user_id = '[SANITIZED]' AND eeb_event_hash = '[SANITIZED]'  ORDER BY eeb_event_id DESC LIMIT 250  ",
    "fname": "EchoEventMapper::fetchByUserBundleHash",
    "normalized_message": "EchoEventMapper::fetchByUserBundleHash\t10.64.16.24\t1054\tUnknown column 'notification_timestamp' in 'field list' (10.64.16.24)\tSELECT  event_agent_id,event_agent_ip,event_extra,event_id,event_page_id,event_type,event_variant,notification_timestamp  FROM `e"
  },
  "sort": [
    1440581724492
  ]
}

(query contents have been sanitized to avoid potential private data being exposed)

Change 233964 had a related patch set uploaded (by Alex Monk):
Fix fatal DB exception

https://gerrit.wikimedia.org/r/233964

Mattflaschen-WMF raised the priority of this task from High to Unbreak Now!.Aug 26 2015, 7:31 PM

Change 234082 had a related patch set uploaded (by Alex Monk):
Fix fatal DB exception

https://gerrit.wikimedia.org/r/234082

Change 233964 merged by jenkins-bot:
Fix fatal DB exception

https://gerrit.wikimedia.org/r/233964

Change 234082 merged by jenkins-bot:
Fix fatal DB exception

https://gerrit.wikimedia.org/r/234082

Since in betalabs

SELECT * FROM information_schema.tables  WHERE  table_name = 'bw_echo_notification';
Empty set (0.01 sec)

just checked for the absence from fatal db errors and EchoEventMapper errors.

DannyH closed this task as Resolved.Aug 28 2015, 5:22 PM
DannyH claimed this task.
Nikerabbit reopened this task as Open.Aug 24 2016, 4:40 PM
Nikerabbit removed a project: Patch-For-Review.

Apperently https://gerrit.wikimedia.org/r/#/c/301382/ re-introduces this issue, in addition to another instance introduced earlier:
git blame EventMapper.php | grep 'echo_event\.\*'
24caf50f (Stephane Bisson 2016-03-04 14:23:02 -0500 160) array( 'echo_event.*' ),
6e2236db (Stephane Bisson 2016-07-27 10:52:18 -0400 189) array( 'echo_event.*' ),

Restricted Application added subscribers: Luke081515, TerraCodes. · View Herald TranscriptAug 24 2016, 4:40 PM

Change 306465 had a related patch set uploaded (by Sbisson):
Don't select echo_event.*

https://gerrit.wikimedia.org/r/306465

Change 306465 merged by jenkins-bot:
Don't select echo_event.*

https://gerrit.wikimedia.org/r/306465

DannyH removed DannyH as the assignee of this task.Aug 25 2016, 11:04 PM
DannyH added a subscriber: DannyH.
DannyH removed a subscriber: DannyH.
greg added a subscriber: greg.Aug 29 2016, 6:23 PM

Status? Is this re-opened task still UBN! ? the associated patch is merged.

What needs to happen here?

Restricted Application added a subscriber: Jay8g. · View Herald TranscriptAug 29 2016, 6:23 PM
Mattflaschen-WMF lowered the priority of this task from Unbreak Now! to High.Aug 29 2016, 8:05 PM

Will be QA'ed then resolved. Changing to High.

Checked for log errors and the presence of code.

jmatazzoni closed this task as Resolved.Sep 12 2016, 8:44 PM
jmatazzoni claimed this task.