Page MenuHomePhabricator

[ci][builds-cli][envvars-cli] Investigate discrepancy between different CI envs
Open, MediumPublic

Description

There is some discrepancy when running CI in gitlab (passes), via utils/run_ci_locally.sh (some unit tests fail but pre-commit passes) and via tox (unit tests pass but pre-commit mypy fails)

tox:

py3-unit: OK ✔ in 3.21 seconds
py3-pre-commit: recreate env because env type changed from {'name': 'py3-unit', 'type': 'VirtualEnvRunner'} to {'name': 'py3-pre-commit', 'type': 'VirtualEnvRunner'}
py3-pre-commit: remove tox env folder /Users/sstefanova/repos/work/toolforge/builds-cli/.tox/py3-tests
py3-pre-commit: commands_pre[0]> poetry install
Installing dependencies from lock file

No dependencies to install or update

Installing the current project: toolforge-builds-cli (0.0.7)
py3-pre-commit: commands[0]> poetry run pre-commit run --all-files
black....................................................................Passed
isort....................................................................Passed
flake8...................................................................Passed
mypy.....................................................................Failed
- hook id: mypy
- duration: 0.63s
- exit code: 1

toolforge_builds_cli/build.py:6: error: Module "toolforge_weld.api_client" has no attribute "ConnectionError"  [attr-defined]
toolforge_builds_cli/build.py:74: error: Unexpected keyword argument "connect_exception_handler" for "__init__" of "ToolforgeClient"; did you mean "exception_handler"?  [call-arg]
Found 2 errors in 1 file (checked 4 source files)

trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check toml...............................................................Passed
check yaml...............................................................Passed
py3-pre-commit: exit 1 (2.87 seconds) /Users/sstefanova/repos/work/toolforge/builds-cli> poetry run pre-commit run --all-files pid=30759
  py3-unit: OK (3.21=setup[0.41]+cmd[1.19,1.62] seconds)
  py3-pre-commit: FAIL code 1 (4.34=setup[0.36]+cmd[1.12,2.87] seconds)
  evaluation failed :( (7.85 seconds)

utils/run_ci_locally.sh:

============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-6.2.5, py-1.11.0, pluggy-1.2.0
cachedir: .tox/py3-tests/.pytest_cache
rootdir: /src
plugins: cov-3.0.0
collected 23 items

tests/unit/test_cli.py ...F...F...............                           [100%]

=================================== FAILURES ===================================
_ BuildStartTestCase.test__invokes_build_logs_command_on_successful_build_creation_by_default _

self = <test_cli.BuildStartTestCase testMethod=test__invokes_build_logs_command_on_successful_build_creation_by_default>
mocked_get_raw_lines = <MagicMock name='get_raw_lines' id='140363227613728'>
__ = <MagicMock name='post' id='140363227675112'>

    @mock.patch.object(
        BuildClient,
        "post",
        return_value={"name": "dummy-build-name", "parameters": {"source_url": "https://dummy-url"}},
    )
    @mock.patch.object(BuildClient, "get_raw_lines")
    def test__invokes_build_logs_command_on_successful_build_creation_by_default(self, mocked_get_raw_lines, __):
        self.cli_runner.invoke(toolforge_build, ["start", "https://dummy-url"])
        call_args = mocked_get_raw_lines.call_args
        expected_log_endpoint = "/build/dummy-build-name/logs?follow=True"
>       self.assertEqual(call_args.args[0], expected_log_endpoint)
E       AssertionError: 'args' != '/build/dummy-build-name/logs?follow=True'
E       - args
E       + /build/dummy-build-name/logs?follow=True

tests/unit/test_cli.py:103: AssertionError
_ BuildStartTestCase.test__stream_logs_on_successful_post_request_if_detach_false _

self = <test_cli.BuildStartTestCase testMethod=test__stream_logs_on_successful_post_request_if_detach_false>
mocked_build_logs = <MagicMock name='build_logs' id='140363227062400'>
mocked_post = <MagicMock name='post' id='140363226558592'>

    @mock.patch.object(BuildClient, "post")
    @mock.patch("toolforge_builds_cli.cli.build_logs")
    def test__stream_logs_on_successful_post_request_if_detach_false(self, mocked_build_logs, mocked_post):
        build_name = "dummy-build-name"
        source_url = "https://dummy-url"
        mocked_post.return_value = {"name": build_name, "parameters": {"source_url": source_url}}
        result = self.cli_runner.invoke(toolforge_build, ["start", source_url])
        self.assertEqual(result.exit_code, 0)
        self.assertTrue(mocked_build_logs.called)
        kwargs = mocked_build_logs.call_args.kwargs
>       self.assertEqual(kwargs["build_name"], build_name)
E       TypeError: tuple indices must be integers or slices, not str

tests/unit/test_cli.py:90: TypeError
=============================== warnings summary ===============================
.tox/py3-tests/lib/python3.7/site-packages/coverage/inorout.py:472
  /src/.tox/py3-tests/lib/python3.7/site-packages/coverage/inorout.py:472: CoverageWarning: --include is ignored because --source is set (include-ignored)
    self.warn("--include is ignored because --source is set", slug="include-ignored")

.tox/py3-tests/lib/python3.7/site-packages/toolforge_weld/__init__.py:1
  /src/.tox/py3-tests/lib/python3.7/site-packages/toolforge_weld/__init__.py:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

.tox/py3-tests/lib/python3.7/site-packages/toolforge_weld/api_client.py:11
  /src/.tox/py3-tests/lib/python3.7/site-packages/toolforge_weld/api_client.py:11: DeprecationWarning: 'urllib3.contrib.pyopenssl' module is deprecated and will be removed in a future release of urllib3 2.x. Read more in this issue: https://github.com/urllib3/urllib3/issues/2680
    from urllib3.contrib.pyopenssl import PyOpenSSLContext

-- Docs: https://docs.pytest.org/en/stable/warnings.html

(output shortened to show failures only)

Details

TitleReferenceAuthorSource BranchDest Branch
dev: update toolforge-weld to 1.4.0+repos/cloud/toolforge/builds-cli!46sstefanovaslavina/update-toolforge-weldmain
dev: update toolforge-weld versionrepos/cloud/toolforge/envvars-cli!18sstefanovaslavina/update-toolforge-weldmain
Customize query in GitLab

Event Timeline

Slst2020 renamed this task from [builds-cli][ci] Investigate discrepancy between different CI envs to [ci] Investigate discrepancy between different CI envs.Jan 4 2024, 2:47 PM

Has been observed in builds-cli and envvars-cli

One source of this is the two different sources of truth for installed packages, poetry.toml and .pre-commit-config.yaml, specifically for toolforge-weld as they are de-synchronized.

Another issues is that we are requiring an old version of toolforge-weld, when we should be requiring >=1.4,<2.0 in both files

The issue for the script shown here is due to the old version of python (3.7) that comes with an old version of unittest.mock.

We still have python3.7 on the bastions, so we still should support that (though the tests don't need to run on python3.7, the code should).

Slst2020 renamed this task from [ci] Investigate discrepancy between different CI envs to [ci][builds-cli][envvars-cli] Investigate discrepancy between different CI envs.Jan 12 2024, 11:18 AM
Slst2020 edited projects, added Toolforge Build Service; removed Toolforge.
dcaro triaged this task as Medium priority.Mar 5 2024, 11:52 AM
dcaro edited projects, added Toolforge; removed Toolforge Build Service.
dcaro moved this task from Backlog to Ready to be worked on on the Toolforge board.

We should double check that this is still an issue, and that we are using the same image for CI and run_local_ci (I think we are on python 3.9 already)