Lots of logs: "PHP Notice: Undefined Index: PATH_INFO"
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	colewhite
	Jun 4 2024, 11:57 PM

Description

Seeing ~15k events/sec in Logstash:

{
  "normalized_message": "PHP Notice:  Undefined index: PATH_INFO in /srv/mediawiki/multiversion/MWMultiVersion.php on line 644",
  "kubernetes": {
    "labels": {
      "deployment": "mw-api-int",
      "release": "main"
    },
    "pod_name": "mw-api-int.eqiad.main-8444d5896d-r26hs",
    "host": "kubernetes1013.eqiad.wmnet",
    "namespace_name": "mw-api-int"
  },
  "program": "php-fpm-error",
  "logsource": "mw-api-int",
  "level": "NOTICE",
  "message": "PHP Notice:  Undefined index: PATH_INFO in /srv/mediawiki/multiversion/MWMultiVersion.php on line 644",
  "metadata": {
    "filename": "/var/log/php-fpm/error.log",
    "fileoffset": "99459"
  }
}

Seems to have shown up since this was deployed.

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	MWMultiVersion: Fix "Undefined index: PATH_INFO" warnings	operations/mediawiki-config	master	+1 -1
	logstash: add drop for php notice unedefined index issue	operations/puppet	production	+6 -0

Customize query in gerrit

Related Objects

Mentioned In: T361402: 1.43.0-wmf.8 deployment blockers
Mentioned Here: T355018: missing project handler does not handle short-form URLs properly

Event Timeline

colewhite created this task.Jun 4 2024, 11:57 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 4 2024, 11:57 PM

Change #1038790 had a related patch set uploaded (by Cwhite; author: Cwhite):

[operations/puppet@production] logstash: add drop for php notice unedefined index issue

https://gerrit.wikimedia.org/r/1038790

gerritbot added a project: Patch-For-Review.Jun 5 2024, 12:04 AM

It must be caused by https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1037930, not https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1037929 (error points to line 644).

That change can be reverted if it's causing issues.

Change #1038790 merged by Cwhite:

[operations/puppet@production] logstash: add drop for php notice unedefined index issue

https://gerrit.wikimedia.org/r/1038790

Maintenance_bot removed a project: Patch-For-Review.Jun 5 2024, 12:30 AM

See also T355018

colewhite renamed this task from Lots of PHP Notice: Undefined Index: PATH_INFO to Lots of logs: "PHP Notice: Undefined Index: PATH_INFO".Jun 5 2024, 3:52 AM

Aklapper added a project: Wikimedia-production-error.Jun 5 2024, 7:30 AM

		if ( $wiki === null ) {
			$scriptName = @$_SERVER['SCRIPT_NAME'];
			$serverName = @$_SERVER['SERVER_NAME'];
			$pathInfo = $_SERVER['PATH_INFO'];
			$multiVersion = self::initializeFromServerData( $serverName, $scriptName, $pathInfo );
		}

The first two have @ error suppression...

It wasn't there in the old code, but the use of $_SERVER['PATH_INFO'] was guarded by the script/server name check, but that went into the new initializeFromServerData function.

Since initializeFromServerData() expects $pathInfo to be a string, it's probably not doing what is expected. The situation has worsened for logging. I think we should roll back the patch.

colewhite triaged this task as Unbreak Now! priority.Jun 5 2024, 1:07 PM

Change #1039212 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[operations/mediawiki-config@master] MWMultiVersion: Fix "Undefined index: PATH_INFO" warnings

https://gerrit.wikimedia.org/r/1039212

gerritbot added a project: Patch-For-Review.Jun 5 2024, 1:10 PM

Aklapper mentioned this in T361402: 1.43.0-wmf.8 deployment blockers.Jun 5 2024, 1:22 PM

Change #1039212 merged by jenkins-bot:

[operations/mediawiki-config@master] MWMultiVersion: Fix "Undefined index: PATH_INFO" warnings

https://gerrit.wikimedia.org/r/1039212

Mentioned in SAL (#wikimedia-operations) [2024-06-05T14:46:35Z] <cwhite@deploy1002> Started scap: Backport for [[gerrit:1039212|MWMultiVersion: Fix "Undefined index: PATH_INFO" warnings (T366657)]]

Pppery unsubscribed.Jun 5 2024, 2:46 PM

Mentioned in SAL (#wikimedia-operations) [2024-06-05T14:49:08Z] <cwhite@deploy1002> matmarex and cwhite: Backport for [[gerrit:1039212|MWMultiVersion: Fix "Undefined index: PATH_INFO" warnings (T366657)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-06-05T14:59:07Z] <cwhite@deploy1002> Finished scap: Backport for [[gerrit:1039212|MWMultiVersion: Fix "Undefined index: PATH_INFO" warnings (T366657)]] (duration: 12m 32s)

The log rate has dropped back to normal levels post-merge. Thanks, all!

Maintenance_bot removed a project: Patch-For-Review.Jun 5 2024, 3:30 PM

Sorry for the trouble & thanks for the fix @Batorsz!

Um I mean @matmarex.

Lots of logs: "PHP Notice: Undefined Index: PATH_INFO"Closed, ResolvedPublicActions

Description

Details

Related Objects

Event Timeline

Lots of logs: "PHP Notice: Undefined Index: PATH_INFO"
Closed, ResolvedPublic
Actions