Z28602 often, but not always, fails with a Z570
Open, HighPublicBUG REPORT
Actions

Assigned To

Authored By

	DVrandecic
	Fri, Oct 10, 11:32 AM

Description

Steps to replicate the issue (include links if applicable):

go to https://www.wikifunctions.org/wiki/Z28602
run it with e.g. L226850/Herz or some other German noun

What happens?:
It fails with a Z570/Reached rate limit in orchestrator with orchestrator rate limit: "600"

It doesn't do so consistently for the same noun. Sometimes a noun passes, sometimes it doesn't (and on repeat, I hit the cache).

What should have happened instead?:
Hopefully pass and show the HTML

Software version (on Special:Version page; skip for WMF-hosted wikis like Wikipedia):

Other information (browser name/version, screenshots, etc.):

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	wikifunctions: Upgrade orchestrator from 2025-10-28-205854 to 2025-11-04-215809	operations/deployment-charts	master	+1 -1

Customize query in gerrit

Related Changes in GitLab:

	Title	Reference	Author	Source Branch	Dest Branch
	Make maximum simultaneous executions controllable by an environment variable.	repos/abstract-wiki/wikifunctions/function-orchestrator!475	apine	apine-rate-limits	main

Customize query in GitLab

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		cmassaro	T406346 [26Q2] Orchestrator memory squashing
		Open	BUG REPORT	cmassaro	T406976 Z28602 often, but not always, fails with a Z570

Event Timeline

DVrandecic created this task.Fri, Oct 10, 11:32 AM

Jdforrester-WMF triaged this task as High priority.Wed, Oct 15, 5:50 PM

Jdforrester-WMF added a parent task: T406346: [26Q2] Orchestrator memory squashing.

Jdforrester-WMF edited projects, added Abstract Wikipedia team (26Q2 (Oct–Dec)); removed Abstract Wikipedia team.

Jdforrester-WMF added a project: OKR-Work.

Jdforrester-WMF moved this task from Incoming to Ready on the Abstract Wikipedia team (26Q2 (Oct–Dec)) board.

This is being debugged by committee. I'm assigning it to myself, but it's actually a group effort among several of us.

cmassaro claimed this task.Wed, Oct 15, 5:52 PM

cmassaro moved this task from Ready to In Engineering on the Abstract Wikipedia team (26Q2 (Oct–Dec)) board.

I hope I am not messing up the debugging effort, but I created a code implementation ( Z28931 ) that works much faster than the previous implementation ( Z28604 ).

The mentioned error is still reproducible on implementation Z28604

https://www.wikifunctions.org/view/en/Z28604

apine opened https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-orchestrator/-/merge_requests/475

Make maximum simultaneous executions controllable by an environment variable.

ecarg merged https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-orchestrator/-/merge_requests/475

Make maximum simultaneous executions controllable by an environment variable.

Maintenance_bot removed a project: Patch-For-Review.Mon, Nov 3, 5:31 PM

Change #1202149 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2025-10-28-205854 to 2025-11-04-215809

https://gerrit.wikimedia.org/r/1202149

gerritbot added a project: Patch-For-Review.Wed, Nov 5, 2:25 PM

Change #1202149 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2025-10-28-205854 to 2025-11-04-215809

https://gerrit.wikimedia.org/r/1202149

Maintenance_bot removed a project: Patch-For-Review.Wed, Nov 5, 3:31 PM

After the deploy, I've just triggered Z28602 for Herz in ~6 rare languages in a row, and all worked. Is this just happenstance/luck, or is this ameliorated/fixed?

Z28602 often, but not always, fails with a Z570Open, HighPublicBUG REPORTActions

Description

Details

Related ObjectsSearch...

Event Timeline

Z28602 often, but not always, fails with a Z570
Open, HighPublicBUG REPORT
Actions

Related Objects
Search...