What the stack trace shows:

• The failure starts in getMwVersionForDomain at MediaWikiHostResolver.php:47, which throws UnknownWikiDomainException at MediaWikiHostResolver.php:51
• That is called from getBackendUrlForDomain at MediaWikiHostResolver.php:43, then from hasPendingJobs in PollForMediaWikiJobsJob.php:27-32
• PollForMediaWikiJobsJob does not catch UnknownWikiDomainException around that call, so it bubbles out of handle at PollForMediaWikiJobsJob.php:17-25
• checking some of the domains, they really are soft-deleted wikis

> $w = Wiki::where('domain', 'testtemptest.wikibase.dev')->withTrashed()->first()

= App\Wiki {#8452
    id: 7,
    domain: "testtemptest.wikibase.dev",
    sitename: "TestTempTest",
    deleted_at: "2021-11-18 15:29:51",
    created_at: "2021-11-18 15:26:45",
    updated_at: "2021-11-18 15:29:51",
    description: null,
    is_featured: 0,
    wiki_deletion_reason: null,
    +domain_decoded: "testtemptest.wikibase.dev",
  }

Why this can happen

• Some wikis were soft-deleted. (using Wiki's SoftDeletes)
• PollForMediaWikiJobsJob first loads all domains with Wiki::all()->pluck('domain') at PollForMediaWikiJobsJob.php:19.
• Resolver then does a second lookup by domain with default scope (non-trashed only) at MediaWikiHostResolver.php:48.
• If a wiki is deleted between those two queries, or the domain row is otherwise missing/inconsistent at that moment, the second query returns null and throws this exact exception.

Solution

• use Wiki::whereNull('deleted_at')->pluck('domain') so soft-deleted rows are not considered in the first place
• catch UnknownWikiDomainException inside per-wiki check and log any warnings instead of crashing the whole job

I wrote this comment as we discussed it in a recent daily but forgot to submit it 😅

For the reviewer:

• Read through the ticket and the PR for context
• Once deployed to staging check that the jobs continue to run successfully
• Once successfully tested on staging, do the same on production

The error still persists earlier this morning. Check the error log

Taking a look at the error, I believe that the resolver call is simply too early in the loop:
https://github.com/wbstack/api/blob/48104e2c80052fb9890ec8540c002c832b6d89d2/app/Jobs/PlatformStatsSummaryJob.php#L83

dati18 opened https://github.com/wbstack/api/pull/1246

fix(api): skip deleted wikis before resolving backend URLs

So I did some quick investigation, and here are the results:

Why the error still persists?

• PlatformStatsSummaryJob resolves a backend URL for every wiki, even deleted ones
• The resolver’s domain lookup uses Wiki::where(), which ignores soft-deleted rows, while this job intentionally includes trashed wikis

Added fix:

• The resolver call is simply too early in the loop. I moved it behind the deleted-wiki logic
• Added regression test

dati18 merged https://github.com/wbstack/api/pull/1246

fix: skip deleted wikis before resolving URLs in PlatformStatsSummaryJob

I went and confirmed that I saw no more errors since Monday! Nice! I know it took a while but this kind of cleaning and maintenance does have lots a value. Less logspam makes real issues much easier to identify.