Page MenuHomePhabricator
Create Task
Maniphest T347241

Tox verbose outputs has poor contrast in Jenkins console output
Closed, ResolvedPublic
Actions

Description

The Jenkins console output has grey lines over grey background and is not readable:

image.png (763×1 px, 157 KB)

Details

SubjectRepoBranchLines +/-
jjb: update tox v4 jobs to decrease verbosityintegration/configmaster+12 -12
dockerfiles: lower tox 4 verbosityintegration/configmaster+49 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

Xqt created this task.Sep 24 2023, 11:07 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 24 2023, 11:07 AM
Xqt removed a project: Release-Engineering-Team (CI & Testing services).Sep 24 2023, 11:28 AM
RhinosF1 renamed this task from Jenkins console output is not readably to Jenkins console output is not readable.Sep 24 2023, 11:30 AM
Restricted Application added a subscriber: RhinosF1. · View Herald TranscriptSep 24 2023, 11:30 AM
RhinosF1 renamed this task from Jenkins console output is not readable to Jenkins console output is not readable due to poor contrast.Sep 24 2023, 11:30 AM
hashar renamed this task from Jenkins console output is not readable due to poor contrast to Tox verbose outputs has poor contrast in Jenkins console output.Oct 2 2023, 8:07 AM
hashar added a parent task: T345695: Deal with tox 4 upgrading.Oct 2 2023, 8:21 AM

Then ANSI sequences are rendered by the Jenkins ANSI Color Plugin. I has some several mappings between the ANSI colors and the HTML/CSS rendered color. We use xterm and the defaults are defined at https://github.com/jenkinsci/ansicolor-plugin/blob/master/src/main/java/hudson/plugins/ansicolor/AnsiColorMap.java#L79

When looking at the Jenkins configuration, it states that got edited, I am not sure what got edited but here is our plugin XML configuration (only including the xterm mapping which is the one globally enabled:

/var/lib/jenkins/hudson.plugins.ansicolor.AnsiColorBuildWrapper.xml
<hudson.plugins.ansicolor.AnsiColorBuildWrapper_-DescriptorImpl plugin="ansicolor@0.7.5">
  <globalColorMapName>xterm</globalColorMapName>
  <colorMaps>
    <hudson.plugins.ansicolor.AnsiColorMap>
      <name>xterm</name>
      <normalMap class="enum-map" enum-type="hudson.plugins.ansicolor.AnsiColorMap$Color">
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>BLACK</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#000000</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>RED</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#CD0000</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>GREEN</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#00CD00</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>YELLOW</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#CDCD00</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>BLUE</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#1E90FF</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>MAGENTA</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#CD00CD</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>CYAN</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#00CDCD</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>WHITE</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#E5E5E5</string>
        </entry>
      </normalMap>
      <brightMap class="enum-map" enum-type="hudson.plugins.ansicolor.AnsiColorMap$Color">
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>BLACK</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#4C4C4C</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>RED</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#FF0000</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>GREEN</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#00FF00</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>YELLOW</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#FFFF00</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>BLUE</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#4682B4</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>MAGENTA</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#FF00FF</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>CYAN</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#00FFFF</string>
        </entry>
        <entry>
          <hudson.plugins.ansicolor.AnsiColorMap_-Color>WHITE</hudson.plugins.ansicolor.AnsiColorMap_-Color>
          <string>#FFFFFF</string>
        </entry>
      </brightMap>
    </hudson.plugins.ansicolor.AnsiColorMap>
hashar added a comment.Oct 2 2023, 8:33 AM

If I look at the Jenkins console HTML, the poor contrast lines have style="color: #E5E5E5" which is a light grey. In the XML configuration that is WHITE from the Normal map (and would be a #FFFFFF in the Bright map).

Tox uses colorama and there is a single occurrence of Fore.WHITE in v4.8.0 code:

src/tox/report.py
class ToxHandler(logging.StreamHandler):
...
    @staticmethod
    def _get_formatter(level: int, enabled_level: int, is_colored: bool) -> logging.Formatter:  # noqa: FBT001
        color: int | str = ""
        if is_colored:
            if level >= logging.ERROR:
                color = Fore.RED
            elif level >= logging.WARNING:
                color = Fore.CYAN
            else:
                color = Fore.WHITE

The xterm map uses the Jenkins defaults colors for Background/Foreground (respectively black and some light grey). Maybe we can change the color map to gnome-terminal which uses a Black background and white foreground.

hashar added a comment.Oct 2 2023, 8:35 AM

The built-in defaults for reference:

public static final AnsiColorMap XTerm = new AnsiColorMap(
        "xterm",
        "#000000", "#CD0000", "#00CD00", "#CDCD00", "#1E90FF", "#CD00CD", "#00CDCD", "#E5E5E5",
        "#4C4C4C", "#FF0000", "#00FF00", "#FFFF00", "#4682B4", "#FF00FF", "#00FFFF", "#FFFFFF",
        null,
        null
);

public static final AnsiColorMap VGA = new AnsiColorMap(
        "vga",
        "#000000", "#AA0000", "#00AA00", "#AA5500", "#0000AA", "#AA00AA", "#00AAAA", "#AAAAAA",
        "#555555", "#FF5555", "#55FF55", "#FFFF55", "#5555FF", "#FF55FF", "#55FFFF", "#FFFFFF",
        Color.WHITE.ordinal(),
        Color.BLACK.ordinal()
);

public static final AnsiColorMap CSS = new AnsiColorMap(
        "css",
        "black", "red", "green", "yellow", "blue", "magenta", "cyan", "white",
        "black", "red", "green", "yellow", "blue", "magenta", "cyan", "white",
        null,
        null
);

public static final AnsiColorMap GnomeTerminal = new AnsiColorMap(
        "gnome-terminal",
        "#2E3436", "#CC0000", "#4E9A06", "#C4A000", "#3465A4", "#75507B", "#06989A", "#D3D7CF",
        "#2E3436", "#CC0000", "#4E9A06", "#C4A000", "#3465A4", "#75507B", "#06989A", "#D3D7CF",
        Color.WHITE.ordinal(),
        Color.BLACK.ordinal()
);
hashar added a comment.Oct 5 2023, 8:43 AM

I have manually edited https://integration.wikimedia.org/ci/job/pywikibot-core-tox-docker/ to change the color map to gnome-terminal. That did not reflect on the existing build. I have rebuild one that used tox v4 but the output has extra newlines:

ansi-color-gnome-terminal.png (304×699 px, 52 KB)

Possibly due to a poor interaction with the timestamper plugin which looks like https://github.com/jenkinsci/ansicolor-plugin/issues/255 Disable Empty new lines before and after ANSI Color Block.

hashar added a comment.Oct 6 2023, 10:50 AM

An alternative, we run tox in verbose mode tox -v, I guess the tox 4 image can remove the verbose flag which kind of work around the debug output being white on white.

gerritbot added a comment.Oct 6 2023, 10:57 AM

Change 963992 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] dockerfiles: lower tox 4 verbosity

https://gerrit.wikimedia.org/r/963992

gerritbot added a project: Patch-For-Review.Oct 6 2023, 10:57 AM
gerritbot added a comment.Oct 6 2023, 7:39 PM

Change 963992 merged by jenkins-bot:

[integration/config@master] dockerfiles: lower tox 4 verbosity

https://gerrit.wikimedia.org/r/963992

Stashbot added a comment.Oct 6 2023, 7:40 PM

Mentioned in SAL (#wikimedia-releng) [2023-10-06T19:40:01Z] <hashar> Updating docker-pkg files for https://gerrit.wikimedia.org/r/c/integration/config/+/963992/ "lower tox 4 verbosity" # T347241

hashar added a comment.Oct 6 2023, 7:49 PM

Successfully published image docker-registry.wikimedia.org/releng/tox-pywikibot:4.8.0-2
Successfully published image docker-registry.wikimedia.org/releng/tox-acme-chief:4.8.0-2
Successfully published image docker-registry.wikimedia.org/releng/tox-mysqld:4.8.0-2
Successfully published image docker-registry.wikimedia.org/releng/zuul-cloner:0.3.0-s4
Successfully published image docker-registry.wikimedia.org/releng/tox-buster:4.8.0-2
Successfully published image docker-registry.wikimedia.org/releng/tox-poolcounter:4.8.0-2
Successfully published image docker-registry.wikimedia.org/releng/tox-java8:4.8.0-2

gerritbot added a comment.Oct 6 2023, 7:55 PM

Change 964075 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: update tox v4 jobs to decrease verbosity

https://gerrit.wikimedia.org/r/964075

hashar added a comment.Oct 6 2023, 7:57 PM

Among other jobs, two pywikibot related jobs got updated to drop -v when invoking tox:

gerritbot added a comment.Oct 6 2023, 7:58 PM

Change 964075 merged by jenkins-bot:

[integration/config@master] jjb: update tox v4 jobs to decrease verbosity

https://gerrit.wikimedia.org/r/964075

Maintenance_bot removed a project: Patch-For-Review.Oct 6 2023, 8:11 PM
hashar closed this task as Resolved.Dec 13 2023, 12:53 PM
hashar claimed this task.

Worked around by removing tox verbose messages (which are written in light gray)

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL