Page MenuHomePhabricator

Use a Proper Logging Module in Orchestrator
Closed, ResolvedPublic

Event Timeline

The service-runner framework provides a logger object that uses bunyan, a proper logging library. The problem is how to make this logger object available to code throughout the orchestrator.

One approach would be to pass around a context object for all request-scoped values and services, including a logger. This would mean changing the signature of most functions in the orchestrator, making it a pretty invasive and involved change.

I propose we add continuation-local storage for this. We'd have a request-local namespace that gets created in initAndLogRequest, and store / retrieve the logger object from there.

Perhaps I am lacking context, but why do we need to pass the logger object around? Could we use separate logger for each class/script?

Change 824772 had a related patch set uploaded (by Ori; author: Ori):

[mediawiki/services/function-orchestrator@master] Centralize access to a shared logger object

https://gerrit.wikimedia.org/r/824772

@ori Thank you for providing options! I agree with the continuation-local storage suggestion.

I'll note that we do have an object which is passed around to every function in the orchestrator (helpfully called invariants), so we could put a logger there, but it's an unpleasant solution (and also won't let us log anything in imported functions, e.g. from function-schemata).

Unfortunately continuation-local-storage and its more modern counterpart, AsyncLocalStorage come with a substantial performance cost, particularly for workloads with a lot of async/await calls. I don't think we can afford the performance penalty.

I went for a much simpler global logger object approach in I7126a1d49. I think it's simple enough that if it becomes cheaper to annotate the logger object with additional tags to identify the current request context, we could add it there quite easily.

Change 824772 had a related patch set uploaded (by Ori; author: Ori):

[mediawiki/services/function-orchestrator@master] Centralize access to a shared logger object

https://gerrit.wikimedia.org/r/824772

Change 824772 merged by jenkins-bot:

[mediawiki/services/function-orchestrator@master] Centralize access to a shared logger object

https://gerrit.wikimedia.org/r/824772

ori claimed this task.