Fetch submodules from the deploy host
Closed, ResolvedPublic1 Story Points

Description

While the deploy repo is currently fetched from the deploy host on targets, submodules remain unmodified and will end up fetching from upstream. This is less than ideal for a couple of reasons, mainly that it precludes us from safely integrating security patches on the deploy host (tin), and secondly that it poses a possible bottleneck under high concurrency. It may also be a blocker for future fanout implementation.

So far, we've discussed two options for implementing this.

  1. Rewrite .gitmodules. Doing this at a single level seems simple enough, but handling it recursively (correctly) would add a lot of complexity to the fetch stage.
  2. Configure deploy targets to use git insteadOf to munge remote URLs. This should cover all submodules recursively without actually having to perform the recursive rewrites ourselves, and seems like a "cleaner" option overall simply because it's non-destructive.

The second option seems like the best approach for now.

Related Objects

dduvall created this task.Sep 25 2015, 6:18 PM
dduvall updated the task description. (Show Details)
dduvall raised the priority of this task from to Normal.
dduvall claimed this task.
dduvall added projects: Scap, Deployment-Systems.
dduvall added subscribers: dduvall, thcipriani, 20after4, demon.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 25 2015, 6:18 PM
dduvall moved this task from Needs triage to Services MVP on the Scap board.Sep 25 2015, 6:18 PM

From today's cabal meeting: There will likely be edge cases where upstream submodule remotes aren't mapped but we're willing to accept those cases for now. We're primarily concerned with pulling in locally applied security patches and only secondarily concerned with performance bottlenecks (for now).

However, a warning should be emitted for upstream remotes that aren't mapped.

dduvall removed dduvall as the assignee of this task.Sep 30 2015, 6:19 PM
dduvall set Security to None.
dduvall moved this task from Services MVP to Needs triage on the Scap board.
thcipriani moved this task from Needs triage to Services MVP on the Scap board.

D9 is the patch for review in this case. There's probably a fancier way to do this.

mmodell edited a custom field.Oct 12 2015, 4:15 PM
thcipriani moved this task from Services MVP to Done on the Scap board.Oct 21 2015, 6:01 PM
thcipriani closed this task as Resolved.