Page MenuHomePhabricator

Jenkins console output freezes with 100% CPU usage after collapsible sections update
Closed, ResolvedPublic

Description

In T378327, the plugin for collapsible sections has been updated. There seems to be a bug in the new code that's causing the browser tab to freeze when reading console output. For example, https://integration.wikimedia.org/ci/job/composer-package-php74/1881/console won't load for me -- as soon as I open the page in chrome, CPU usage for that tab goes to 100%, it freezes and I have to kill the tab. The same happens in a semi-empty profile in firefox, so it shouldn't be browser- or plugins-related. However, it does seem to be job-related: I cannot reproduce with https://integration.wikimedia.org/ci/job/mwgate-node18/78239/console, https://integration.wikimedia.org/ci/job/mwext-node18-rundoc/11476/console freezes, and yesterday I had times when a tab would freeze, but then work normally upon refresh.

I tried profiling it in devtools, and basically 100% of the CPU time is spent in handleSidePanelChanges, added in https://github.com/jenkinsci/collapsing-console-sections-plugin/pull/35/files. Without looking further, I suspect it's entering an infinite call sequence.

Event Timeline

Daimona triaged this task as Unbreak Now! priority.Nov 2 2024, 1:09 AM

(I should also note that this can be worked around by reading the plaintext version, which you can get by replacing console with consoleText in the URL. Still, not being able to read the output immediately is serious enough IMO.)

That is due to a change I have made to ensure the collapsible sections panel is always last. The code does:

function handleSidePanelChanges() {
    const sidePanel = document.getElementById("side-panel");
    const collapsePanel = sidePanel.children.namedItem("console-section-container");

    // Move the collapsible section headers to the last position
    if (sidePanel.lastElementChild !== collapsePanel) {
        sidePanel.lastElementChild.after(collapsePanel);
    }
}

new MutationObserver(handleSidePanelChanges)
    .observe(document.getElementById("side-panel"), { childList: true });

When the side-panel childs change, if the console-section-container child is not the last one, it is moved to be after the last child. That in turns trigger the observer and the code runs again. Looks like that does not quite work :/

I have found it. I have not tested it when there is no collapsible section. When the timestamper plugin adds its panel to the sidebar that triggers the observer, namedItem('console-section-container') returns null and never fulfills the condition that the last element is console-section-container. The code should abort early when the panel does not exist (`if (collapsePanel === null) return;).

Mentioned in SAL (#wikimedia-releng) [2024-11-04T11:42:47Z] <hashar> Restarted CI Jenkins to update the Collapsible Sections plugin due to T378864 # T378327

Thank you for the detailed report, I have updated the code, rebuilt the plugin and deployed it.