Include the request ID in API request logs
Open, NormalPublic

Description

RESTBase is making sure that we have a unique X-Request-ID header on each request, and sets this on each outgoing request. Other services are now starting to forward this header in their own outgoing requests as well (built into the service template, T97207 for parsoid).

To leverage this header to trace incoming requests to the PHP API back to their origin(s), the PHP API should log the X-Request-ID header along with other log data. Since internal requests to the PHP API typically don't pass through Varnish, it probably makes sense to tackle this at the API (Apache?) layer first.

GWicke created this task.Apr 25 2015, 5:04 PM
GWicke updated the task description. (Show Details)
GWicke raised the priority of this task from to Needs Triage.
GWicke added a project: MediaWiki-API.
GWicke added subscribers: GWicke, Anomie, ori, BBlack.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 25 2015, 5:04 PM
GWicke renamed this task from Include the request ID in request logs to Include the request ID in API request logs.Apr 25 2015, 5:04 PM
GWicke set Security to None.
Anomie added a subscriber: bd808.Apr 25 2015, 9:27 PM

Why not use the requestid URL parameter for your requests, which is already included in api.log?

Or if you're talking about logging in general, @bd808 might be able to tell us of a way to get it added to all log messages that go into logstash. Possibly WikiProcessor.php, if I had to guess, or a new processor added to the list in $wgMWLoggerDefaultSpi.

bd808 added a comment.Apr 26 2015, 4:23 PM

Or if you're talking about logging in general, @bd808 might be able to tell us of a way to get it added to all log messages that go into logstash. Possibly WikiProcessor.php, if I had to guess, or a new processor added to the list in $wgMWLoggerDefaultSpi.

The current Monolog config for production is using Monolog's UidProcessor to add a correlating id to all log messages for a given PHP request. The right thing to do would probably be to create a new subclass of UidProcessor that looks for the X-Request-ID header and uses its value when found instead.

To make propagating an X-Request-ID among layers easier we should introduce some easy way to get the value that also transparently generates and remembers a conforming value if none in present for the current request. This could then be wired into not only the logging layer but also used to decorate all outbound curl requests.

Aklapper triaged this task as Normal priority.Apr 27 2015, 9:34 AM