@RickiJay-WMDE mentioned if: github.event.pull_request.merged == 'true' might help to fix that issue.

Updates:

Changed the Github workflow build_test_publish_release.yml, which is now triggered by closed PRs on the mw-* branches:

on:
  pull_request:
    types:
      - closed
    branches:
      - "mw-*"

Each job on the workflow has a check that the PR was *merged*, not just closed:

if: github.event.pull_request.merged == true