Change Details

== Problem == There's an agreed high level flow for toolforge component deployment outlined here: https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Toolforge_Kubernetes_component_workflow_improvements But at implementation time there's some details that need deciding/refining as those showed to be controversial. This is a decision request to decide on those details. Note that the focus is on the continuous delivery of toolforge components, not of toolforge itself (that might require another discussion). * What artifacts do we want to ship? (image, helm chart, etc.) * How to version those artifacts * When to ship those artifacts Some assumptions (though feel free to challenge later): * We want to do this on gitlab * We want to use gitlab ci * We want to store the artifacts in harbor * The deployment of the artifacts to toolforge (tools or toolsbeta) is handled by a different repo, assuming here that there is or will be human validation at that stage * We want to automate the building of the artifacts, even if the trigger to build them is not * We publish the artifacts at the same time in tools and toolsbeta (note that this does not mean deployment on toolforge, just publishing the artifacts on harbor) === Constraints and risks === If delivered late: * Some increased maintenance on our side until that is done to manually deliver the current components * Might become harder to adopt due to the increase of components and lack of prioritization If not all the full flow is delivered: * Possible increased maintenance to build and release components * Mix of processes flows might make release of code error-prone (wmcs.toolforge.component.build + ./deploy.sh vs toolforge-deploy + helm chart bump) == Decision record == In progress https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Decision_record_T302593_Toolforge_component_deployment_flow_details == Partial Options == Note that these three decisions will be dependent on each other, so for a different publish flow you might want a specific versioning scheme and set of artifacts that is different than another publish flow. So take that into account when suggesting solutions. === Versioning scheme === ==== VS Option 1 ==== 0.0.X-TTTTT-HHHHH, where: * X is the number of commits since the origin of the repository or the previous tag * TTTTT is the datestamp (YYYYmmddHHMMSS, ex. 202306151038) * HHHHH is the short git hash of the commit used to build Pros: * Identifies the commit it was built on top of * Sequential code-wise (if A has more commits than B, A>B) * Sequential time-wise (for the same commit, a build today has priority over a build from yesterday) Cons: * No semantic versioning * Can't be in the source code itself as it depends on the time + git hash ==== VS Option 2 ==== A.B.C-TTTTT-HHHHH, where: * A.B.C is as semantic version, generated from the git history: ** Extract the list of commits in historical order: **** For each commit that has `Sem-Ver: major` bump the major version **** For each commit that has `Sem-Ver: feature` bump the feature version **** For any other commit, bump the bug version * TTTTT is the datestamp (YYYYmmddHHMMSS, ex. 202306151038) * HHHHH is the short git hash of the commit used to build Pros: * Identifies the commit it was built on top of * Sequential code-wise (if A has more commits than B, A>B) * Sequential time-wise (for the same commit, a build today has priority over a build from yesterday) * Semantic versioning from git source (developer flags non-backwards compatible commits at review time) Cons: * Can't be in the source code itself as it depends on the time + git hash * The version generation is a bit more complicated ==== VS Option 3 ==== 0.0.0, where: * 0.0.0 is a static or hardcoded string in the code Pros: * Manual control of the version * Version comes from the code itself, no generation needed Cons: * Needs manual updating ** You might forget to bump (version does not identify code anymore) ** You might bump to the wrong version (version does not identify code anymore, no semantic versioning, not sequential) ** You have to review the history to decide if commits are major/feature or minor if you want to follow semantic versioning ==== VS Option N ==== ADD MORE HERE! === Artifacts === ==== Arts option 1 ==== Helm and image are delivered separately Pros: * Strong control of each artifact * No need to rebuild image if only chart changes (in most cases) * No need to update chart if only code changes (in most cases) Cons: * More complex release process/flow, harder to automate * Need to keep track of two versions, and updating the correct one in the right place * Chart will not ensure it works without overriding the image version it comes with by default ==== Arts option 2 ==== Helm and image are delivered at the same time Pros: * Only one artifact/version to manage * The chart will always work by itself (as the image it comes with is built for it) * Easier to automate/simpler workflow Cons: * Loose control of when the publish happens as both image and chart are now tied up * Sometimes the image rebuild will be done but not needed (as no code changes happened) * Sometimes the helm chart rebuild will be done but not needed (as no code chart changes happened) ==== Arts option N ==== ADD MORE HERE! === When to ship === Here "when to ship" means "when to build a container image and the helm chart and push them to harbor". ==== Shipping Option 1 ==== Ship on manual tag Pros: * Control when the shipping happens * Easy to see from the git tags history when a shipping happened Cons: * Might be unable to pin-point which commit was shipped with which tag (as tags are not immutable, they can be moved around, and are local to each git clone) * Needs a manual step * No peer-review on what is going to be shipped (can't review tag pushes) ==== Shipping Option 2 ==== Ship on manual tag after version bump commit Pros: * Control when the shipping happens * Easy to see from the git tags history when a shipping happened * Half-review of what is going to be shipped (at least the commit bump) Cons: * Needs two manual steps (mr to bump + tag) * Might be unable to pin-point which commit was shipped with which tag (as tags are not immutable, they can be moved around, and are local to each git clone) * Only half-peer-review on what is going to be shipped (can't review tag pushes) ==== Shipping Option 3 ==== Ship on every merge/push to main Pros: * No manual steps * Shipping happens all the time, so hard to pin-point a single one (no tag to flag it, or special commit) * Review of what's going to be shipped is the same as code review before merge Cons: * Shipping on every merge of a merge request, even if that version might not be deployed ==== Shipping Option N ==== ADD MORE HERE! == Options == === Option 1 === Version option: 1 Artifacts option: 1 Shipping option: 1 Using 0.0.X-TTTTT-HHHHH, triggered on manual tag (one for artifact). Pros: * Control on when publish happens * Version is sequential code-wise * Version is sequential time-wise * Version identifies the commit it was first built on Cons: * Several manual steps, one per artifact (image/chart) * Lack of review on what's going to ship (can't review tags) * Chart might get out of sync with image * Tags are local to the repo and might move around, so might not identify the commit the version was generated for * Not following semantic versioning === Option 2 === Version option: 2 Artifacts option: 2 Shipping option: 2 Using A.B.C-X-TTTTT-HHHHH, triggered by manual commit + tag (shared for both image and chart) Pros: * Control on when publish happens * Version is sequential code-wise * Version is sequential time-wise * Version identifies the commit it was first built on * Version follows semantic versioning * Partial of review on what's going to ship (bump commit) Cons: * Several manual steps (bump version commit + tag) * Partial of review on what's going to ship (can't review tags) * Tags are local to the repo and might move around, so might not identify the commit the version was generated for === Option 3 === Version option: 2 Artifacts option: 2 Shipping option: 3 Using A.B.C-X-TTTTT-HHHHH, triggered by mr merge or direct push to main Pros: * Version is sequential code-wise * Version is sequential time-wise * Version identifies the commit it was first built on * Version follows semantic versioning (non-backwards compatible commits labeled at development time) * Review on what's going to ship (each mr) * No manual steps Cons: * No control of when a publish happens (well, it happens once per merge) === Option N === ADD MORE HERE!

== Problem == There's an agreed high level flow for toolforge component deployment outlined here: https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Toolforge_Kubernetes_component_workflow_improvements But at implementation time there's some details that need deciding/refining as those showed to be controversial. This is a decision request to decide on those details. Note that the focus is on the continuous delivery of toolforge components, not of toolforge itself (that might require another discussion). * What artifacts do we want to ship? (image, helm chart, etc.) * How to version those artifacts * When to ship those artifacts Some assumptions (though feel free to challenge later): * We want to do this on gitlab * We want to use gitlab ci * We want to store the artifacts in harbor * The deployment of the artifacts to toolforge (tools or toolsbeta) is handled by a different repo, assuming here that there is or will be human validation at that stage * We want to automate the building of the artifacts, even if the trigger to build them is not * We publish the artifacts at the same time in tools and toolsbeta (note that this does not mean deployment on toolforge, just publishing the artifacts on harbor) === Constraints and risks === If delivered late: * Some increased maintenance on our side until that is done to manually deliver the current components * Might become harder to adopt due to the increase of components and lack of prioritization If not all the full flow is delivered: * Possible increased maintenance to build and release components * Mix of processes flows might make release of code error-prone (wmcs.toolforge.component.build + ./deploy.sh vs toolforge-deploy + helm chart bump) == Decision record == In progress https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Decision_record_T302593_Toolforge_component_deployment_flow_details == Partial Options == Note that these three decisions will be dependent on each other, so for a different publish flow you might want a specific versioning scheme and set of artifacts that is different than another publish flow. So take that into account when suggesting solutions. === Versioning scheme === ==== VS Option 1 ==== 0.0.X-TTTTT-HHHHH, where: * X is the number of commits since the origin of the repository or the previous tag * TTTTT is the datestamp (YYYYmmddHHMMSS, ex. 202306151038) * HHHHH is the short git hash of the commit used to build Pros: * Identifies the commit it was built on top of * Sequential code-wise (if A has more commits than B, A>B) * Sequential time-wise (for the same commit, a build today has priority over a build from yesterday) Cons: * No semantic versioning * Can't be in the source code itself as it depends on the time + git hash ==== VS Option 2 ==== A.B.C-TTTTT-HHHHH, where: * A.B.C is as semantic version, generated from the git history: ** Extract the list of commits in historical order: **** For each commit that has `Sem-Ver: major` bump the major version **** For each commit that has `Sem-Ver: feature` bump the feature version **** For any other commit, bump the bug version * TTTTT is the datestamp (YYYYmmddHHMMSS, ex. 202306151038) * HHHHH is the short git hash of the commit used to build Pros: * Identifies the commit it was built on top of * Sequential code-wise (if A has more commits than B, A>B) * Sequential time-wise (for the same commit, a build today has priority over a build from yesterday) * Semantic versioning from git source (developer flags non-backwards compatible commits at review time) Cons: * Can't be in the source code itself as it depends on the time + git hash * The version generation is a bit more complicated ==== VS Option 3 ==== 0.0.0, where: * 0.0.0 is a static or hardcoded string in the code Pros: * Manual control of the version * Version comes from the code itself, no generation needed Cons: * Needs manual updating ** You might forget to bump (version does not identify code anymore) ** You might bump to the wrong version (version does not identify code anymore, no semantic versioning, not sequential) ** You have to review the history to decide if commits are major/feature or minor if you want to follow semantic versioning ==== VS Option N ==== ADD MORE HERE! === Artifacts === ==== Arts option 1 ==== Helm and image are delivered separately Pros: * Strong control of each artifact * No need to rebuild image if only chart changes (in most cases) * No need to update chart if only code changes (in most cases) Cons: * More complex release process/flow, harder to automate * Need to keep track of two versions, and updating the correct one in the right place * Chart will not ensure it works without overriding the image version it comes with by default ==== Arts option 2 ==== Helm and image are delivered at the same time Pros: * Only one artifact/version to manage * The chart will always work by itself (as the image it comes with is built for it) * Easier to automate/simpler workflow Cons: * Loose control of when the publish happens as both image and chart are now tied up * Sometimes the image rebuild will be done but not needed (as no code changes happened) * Sometimes the helm chart rebuild will be done but not needed (as no code chart changes happened) ==== Arts option N ==== ADD MORE HERE! === When to ship === Here "when to ship" means "when to build a container image and the helm chart and push them to harbor". ==== Shipping Option 1 ==== Ship on manual tag Pros: * Control when the shipping happens * Easy to see from the git tags history when a shipping happened Cons: * Might be unable to pin-point which commit was shipped with which tag (as tags are not immutable, they can be moved around, and are local to each git clone) * Needs a manual step * No peer-review on what is going to be shipped (can't review tag pushes) ==== Shipping Option 2 ==== Ship on manual tag after version bump commit Pros: * Control when the shipping happens * Easy to see from the git tags history when a shipping happened * Half-review of what is going to be shipped (at least the commit bump) Cons: * Needs two manual steps (mr to bump + tag) * Might be unable to pin-point which commit was shipped with which tag (as tags are not immutable, they can be moved around, and are local to each git clone) * Only half-peer-review on what is going to be shipped (can't review tag pushes) ==== Shipping Option 3 ==== Ship on every merge/push to main Pros: * No manual steps * Shipping happens all the time, so hard to pin-point a single one (no tag to flag it, or special commit) * Review of what's going to be shipped is the same as code review before merge Cons: * Shipping on every merge of a merge request, even if that version might not be deployed ==== Shipping Option 4 ==== Ship on version bump. The Git tag is a nice addition, and it helps identify things, but is not a hard requirement (or, the workflow doesn't depend on it or break because lack of git tag). This is similar to what happens with debian packages. The tag mostly works to help backtrace in a short way which version was deployed (can be figured if no git tag anyway with a bit more work). Pros: * Control when the shipping happens * Easy to see from the git history when a shipping happened Cons: * TBD ==== Shipping Option 2 ==== Ship on manual tag after version bump commit Pros: * Control when the shipping happens * Easy to see from the git tags history when a shipping happened * Half-review of what is going to be shipped (at least the commit bump) Cons: * Needs two manual steps (mr to bump + tag) * Might be unable to pin-point which commit was shipped with which tag (as tags are not immutable, they can be moved around, and are local to each git clone) * Only half-peer-review on what is going to be shipped (can't review tag pushes) * Needs a commit to bump the version * The git tag part may be forgotten (but also, harmless if forgotten anyway) ==== Shipping Option N ==== ADD MORE HERE! == Options == === Option 1 === Version option: 1 Artifacts option: 1 Shipping option: 1 Using 0.0.X-TTTTT-HHHHH, triggered on manual tag (one for artifact). Pros: * Control on when publish happens * Version is sequential code-wise * Version is sequential time-wise * Version identifies the commit it was first built on Cons: * Several manual steps, one per artifact (image/chart) * Lack of review on what's going to ship (can't review tags) * Chart might get out of sync with image * Tags are local to the repo and might move around, so might not identify the commit the version was generated for * Not following semantic versioning === Option 2 === Version option: 2 Artifacts option: 2 Shipping option: 2 Using A.B.C-X-TTTTT-HHHHH, triggered by manual commit + tag (shared for both image and chart) Pros: * Control on when publish happens * Version is sequential code-wise * Version is sequential time-wise * Version identifies the commit it was first built on * Version follows semantic versioning * Partial of review on what's going to ship (bump commit) Cons: * Several manual steps (bump version commit + tag) * Partial of review on what's going to ship (can't review tags) * Tags are local to the repo and might move around, so might not identify the commit the version was generated for === Option 3 === Version option: 2 Artifacts option: 2 Shipping option: 3 Using A.B.C-X-TTTTT-HHHHH, triggered by mr merge or direct push to main Pros: * Version is sequential code-wise * Version is sequential time-wise * Version identifies the commit it was first built on * Version follows semantic versioning (non-backwards compatible commits labeled at development time) * Review on what's going to ship (each mr) * No manual steps Cons: * No control of when a publish happens (well, it happens once per merge) === Option N === ADD MORE HERE!

== Problem == There's an agreed high level flow for toolforge component deployment outlined here: https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Toolforge_Kubernetes_component_workflow_improvements But at implementation time there's some details that need deciding/refining as those showed to be controversial. This is a decision request to decide on those details. Note that the focus is on the continuous delivery of toolforge components, not of toolforge itself (that might require another discussion). * What artifacts do we want to ship? (image, helm chart, etc.) * How to version those artifacts * When to ship those artifacts Some assumptions (though feel free to challenge later): * We want to do this on gitlab * We want to use gitlab ci * We want to store the artifacts in harbor * The deployment of the artifacts to toolforge (tools or toolsbeta) is handled by a different repo, assuming here that there is or will be human validation at that stage * We want to automate the building of the artifacts, even if the trigger to build them is not * We publish the artifacts at the same time in tools and toolsbeta (note that this does not mean deployment on toolforge, just publishing the artifacts on harbor) === Constraints and risks === If delivered late: * Some increased maintenance on our side until that is done to manually deliver the current components * Might become harder to adopt due to the increase of components and lack of prioritization If not all the full flow is delivered: * Possible increased maintenance to build and release components * Mix of processes flows might make release of code error-prone (wmcs.toolforge.component.build + ./deploy.sh vs toolforge-deploy + helm chart bump) == Decision record == In progress https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Decision_record_T302593_Toolforge_component_deployment_flow_details == Partial Options == Note that these three decisions will be dependent on each other, so for a different publish flow you might want a specific versioning scheme and set of artifacts that is different than another publish flow. So take that into account when suggesting solutions. === Versioning scheme === ==== VS Option 1 ==== 0.0.X-TTTTT-HHHHH, where: * X is the number of commits since the origin of the repository or the previous tag * TTTTT is the datestamp (YYYYmmddHHMMSS, ex. 202306151038) * HHHHH is the short git hash of the commit used to build Pros: * Identifies the commit it was built on top of * Sequential code-wise (if A has more commits than B, A>B) * Sequential time-wise (for the same commit, a build today has priority over a build from yesterday) Cons: * No semantic versioning * Can't be in the source code itself as it depends on the time + git hash ==== VS Option 2 ==== A.B.C-TTTTT-HHHHH, where: * A.B.C is as semantic version, generated from the git history: ** Extract the list of commits in historical order: **** For each commit that has `Sem-Ver: major` bump the major version **** For each commit that has `Sem-Ver: feature` bump the feature version **** For any other commit, bump the bug version * TTTTT is the datestamp (YYYYmmddHHMMSS, ex. 202306151038) * HHHHH is the short git hash of the commit used to build Pros: * Identifies the commit it was built on top of * Sequential code-wise (if A has more commits than B, A>B) * Sequential time-wise (for the same commit, a build today has priority over a build from yesterday) * Semantic versioning from git source (developer flags non-backwards compatible commits at review time) Cons: * Can't be in the source code itself as it depends on the time + git hash * The version generation is a bit more complicated ==== VS Option 3 ==== 0.0.0, where: * 0.0.0 is a static or hardcoded string in the code Pros: * Manual control of the version * Version comes from the code itself, no generation needed Cons: * Needs manual updating ** You might forget to bump (version does not identify code anymore) ** You might bump to the wrong version (version does not identify code anymore, no semantic versioning, not sequential) ** You have to review the history to decide if commits are major/feature or minor if you want to follow semantic versioning ==== VS Option N ==== ADD MORE HERE! === Artifacts === ==== Arts option 1 ==== Helm and image are delivered separately Pros: * Strong control of each artifact * No need to rebuild image if only chart changes (in most cases) * No need to update chart if only code changes (in most cases) Cons: * More complex release process/flow, harder to automate * Need to keep track of two versions, and updating the correct one in the right place * Chart will not ensure it works without overriding the image version it comes with by default ==== Arts option 2 ==== Helm and image are delivered at the same time Pros: * Only one artifact/version to manage * The chart will always work by itself (as the image it comes with is built for it) * Easier to automate/simpler workflow Cons: * Loose control of when the publish happens as both image and chart are now tied up * Sometimes the image rebuild will be done but not needed (as no code changes happened) * Sometimes the helm chart rebuild will be done but not needed (as no code chart changes happened) ==== Arts option N ==== ADD MORE HERE! === When to ship === Here "when to ship" means "when to build a container image and the helm chart and push them to harbor". ==== Shipping Option 1 ==== Ship on manual tag Pros: * Control when the shipping happens * Easy to see from the git tags history when a shipping happened Cons: * Might be unable to pin-point which commit was shipped with which tag (as tags are not immutable, they can be moved around, and are local to each git clone) * Needs a manual step * No peer-review on what is going to be shipped (can't review tag pushes) ==== Shipping Option 2 ==== Ship on manual tag after version bump commit Pros: * Control when the shipping happens * Easy to see from the git tags history when a shipping happened * Half-review of what is going to be shipped (at least the commit bump) Cons: * Needs two manual steps (mr to bump + tag) * Might be unable to pin-point which commit was shipped with which tag (as tags are not immutable, they can be moved around, and are local to each git clone) * Only half-peer-review on what is going to be shipped (can't review tag pushes) ==== Shipping Option 3 ==== Ship on every merge/push to main Pros: * No manual steps * Shipping happens all the time, so hard to pin-point a single one (no tag to flag it, or special commit) * Review of what's going to be shipped is the same as code review before merge Cons: * Shipping on every merge of a merge request, even if that version might not be deployed ==== Shipping Option 4 ==== Ship on version bump. The Git tag is a nice addition, and it helps identify things, but is not a hard requirement (or, the workflow doesn't depend on it or break because lack of git tag). This is similar to what happens with debian packages. The tag mostly works to help backtrace in a short way which version was deployed (can be figured if no git tag anyway with a bit more work). Pros: * Control when the shipping happens * Easy to see from the git history when a shipping happened Cons: * TBD ==== Shipping Option 2 ==== Ship on manual tag after version bump commit Pros: * Control when the shipping happens * Easy to see from the git tags history when a shipping happened * Half-review of what is going to be shipped (at least the commit bump) Cons: * Needs two manual steps (mr to bump + tag) * Might be unable to pin-point which commit was shipped with which tag (as tags are not immutable, they can be moved around, and are local to each git clone) * Only half-peer-review on what is going to be shipped (can't review tag pushes) * Needs a commit to bump the version * The git tag part may be forgotten (but also, harmless if forgotten anyway) ==== Shipping Option N ==== ADD MORE HERE! == Options == === Option 1 === Version option: 1 Artifacts option: 1 Shipping option: 1 Using 0.0.X-TTTTT-HHHHH, triggered on manual tag (one for artifact). Pros: * Control on when publish happens * Version is sequential code-wise * Version is sequential time-wise * Version identifies the commit it was first built on Cons: * Several manual steps, one per artifact (image/chart) * Lack of review on what's going to ship (can't review tags) * Chart might get out of sync with image * Tags are local to the repo and might move around, so might not identify the commit the version was generated for * Not following semantic versioning === Option 2 === Version option: 2 Artifacts option: 2 Shipping option: 2 Using A.B.C-X-TTTTT-HHHHH, triggered by manual commit + tag (shared for both image and chart) Pros: * Control on when publish happens * Version is sequential code-wise * Version is sequential time-wise * Version identifies the commit it was first built on * Version follows semantic versioning * Partial of review on what's going to ship (bump commit) Cons: * Several manual steps (bump version commit + tag) * Partial of review on what's going to ship (can't review tags) * Tags are local to the repo and might move around, so might not identify the commit the version was generated for === Option 3 === Version option: 2 Artifacts option: 2 Shipping option: 3 Using A.B.C-X-TTTTT-HHHHH, triggered by mr merge or direct push to main Pros: * Version is sequential code-wise * Version is sequential time-wise * Version identifies the commit it was first built on * Version follows semantic versioning (non-backwards compatible commits labeled at development time) * Review on what's going to ship (each mr) * No manual steps Cons: * No control of when a publish happens (well, it happens once per merge) === Option N === ADD MORE HERE!