Page MenuHomePhabricator

Create repository with configuration files for common CI use cases
Closed, ResolvedPublic

Description

Create a GitLab repository that will serve as a how-to guide, and will contain:

  • a README file with an overview of files in the repository and basic instructions on how to use them
  • Blubber and GitLab CI configuration files with configuration of common CI elements for different projects and with comments explaining the configuration options in use
  • Application code to test the configuration files

Use cases to consider:

  • basic GitLab CI tests
  • building container images using Dockerfiles and Blubber, and pushing them to different registries
  • generating build artifacts, for example a Debian package, and pushing these artifacts to centralized repositories (for example an apt repository)
  • running extra services necessary for pipeline jobs, for example a database used in tests (implemented in the end-to-end test stage)

Other considerations:

  • use these configuration files in custom GitLab templates?
  • ensure code and configuration files in this repository are tested automatically

Summary

(largely copied from this comment)

The repository is available on GitLab: https://gitlab.wikimedia.org/repos/technical-documentation/deployment-pipeline-sandbox

The repository contains:

  • A README file with instructions for testing Blubber variants locally
  • Five examples of web applications with the same functionality (display a plain text response which includes a random number - changed on every reload).
    • Languages used: Java, NodeJS, PHP, Python, and Rust
    • Each application has tests for its functionality
    • Each application uses standard and popular (to my knowledge) build mechanisms, libraries, and tools for the given language.
  • Separate Blubber variants for building, testing, and running these applications (in .pipeline/blubber.yaml)
  • GitLab CI job configurations for unit tests and end-to-end tests (in .gitlab-ci.yml)

Different variants and configurations showcase:

  • how to use different builder configurations
  • how to use Blubber and GitLab CI with different project directory structures (the NodeJS application uses a different configuration from the rest)
  • how to use images from the Wikimedia registry (all applications) and from Docker Hub (Java and Rust applications)
  • how to use a multi-stage image and transfer build artifacts between image stages (Rust application)
  • how to run an application as a service in GitLab CI
  • how to specify stages and job dependencies in GitLab CI
  • how to use Kokkuri in your project

Content in the repository is automatically tested on merge/commit. Tests also run periodically, once a month.

Merge requests and comments welcome!

Event Timeline

KBach created this task.
KBach changed the task status from Open to In Progress.Dec 1 2023, 11:45 AM
KBach moved this task from Backlog to In progress on the Tech-Docs-Team board.

A sandbox repository for this task is available on GitLab: https://gitlab.wikimedia.org/repos/technical-documentation/deployment-pipeline-sandbox

Currently working on a multi-stage Rust build.

Submitted a merge request for the example in Rust.

First version of the repository is now ready (https://gitlab.wikimedia.org/repos/technical-documentation/deployment-pipeline-sandbox). This version contains:

  • A README file with instructions for testing Blubber variants locally
  • Five examples of web applications with the same functionality (display a plain text response which includes a random number - changed on every reload).
    • Languages used: Java, NodeJS, PHP, Python, and Rust
    • Each application has tests for its functionality
    • Each application uses standard and popular (to my knowledge) build mechanisms, libraries, and tools for the given language.
  • Separate Blubber variants for building, testing, and running these applications (in .pipeline/blubber.yaml)
  • GitLab CI job configurations for unit tests and end-to-end tests (in .gitlab-ci.yml)

Different variants and configurations showcase:

  • how to use different builder configurations
  • how to use Blubber and GitLab CI with different project directory structures (the NodeJS application uses a different configuration from the rest)
  • how to use images from the Wikimedia registry (all applications) and from Docker Hub (Java and Rust applications)
  • how to use a multi-stage image and transfer build artifacts between image stages (Rust application)
  • how to run an application as a service in GitLab CI
  • how to specify stages and job dependencies in GitLab CI
  • how to use Kokkuri in your project

This version of the repository can serve as a starting point for working on the requirements listed in task description. Contributions to Blubber/GitLab CI configuration files and to the application code are welcome!

I cloned the repo and was able to successfully run the tests and the sample services for each of the different language examples. I wish it was possible to run the NodeJS commands in the same format as all the others :-(

I cloned the repo and was able to successfully run the tests and the sample services for each of the different language examples. I wish it was possible to run the NodeJS commands in the same format as all the others :-(

The idea behind this was to demonstrate different configuration options. I realize this might be a bit inconvenient but some of the applications deliberately have slightly different settings to serve as examples.

KBach updated the task description. (Show Details)