Page MenuHomePhabricator

GitLab CI acting funny for image_suggestions
Closed, ResolvedPublic

Description

From @mfossati :

Hey @xcollazo, code looks good to me, but the test CI pipeline is not getting triggered for some reason, and I'm not able to manually run the pytest suite:

I looked into GitLab docs, but I don't understand why we should add a rule to our CI YAML? I mean, section alignment suggestions seems to work fine without any addition 🤷

a manual tox -e pytest shot cries for every test with py4j.protocol.Py4JError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext

Would you be so kind to look into that?

Event Timeline

The culprit seems to be the definition of the bump_on_airflow_dags CI step. I deleted it temporarily and now MRs open up with a proper pipeline run triggered by the MR creation.

Will play with the bump_on_airflow_dags code to see what is wrong with it. Gitlab's own CI validator is certainly happy with it...

xcollazo changed the task status from Open to In Progress.Mar 17 2023, 3:41 PM
xcollazo triaged this task as High priority.

Ok this one was fun. Explanation:

We were missing the following GitLab CI rule definition as part of bump_on_airflow_dags:

  # Don't allow bumping from the MR pipeline.
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
  when: never

For our purposes, there are two different pipeline triggers: push (when we push to a branch) and merge_request_event (when we create or otherwise modify an MR).

When we define the above rule on the bump_on_airflow_dags job, we are implicitly telling Gitlab that there is no need to create a separate pipeline when the MR is created. Thus Gitlab behavior is to bind the pipeline run for the push event with the MR creation. If we do *not* have the above flag, then Gitlab would create separate pipeline runs, but, for some strange reason, never run but 'skip' the one associated with the MR. Go figure. ¯\_(ツ)_/¯

Anyhow, I've fixed this on both image_suggestions and on section_topics.