Page MenuHomePhabricator

Epic: feature parity with vagrant on docker
Open, Needs TriagePublic

Description

user story?

subtasks

  • setting up payments wiki
  • getting civi running
  • queues
  • fundraising tools
  • process control
  • document how to use

Related Objects

StatusSubtypeAssignedTask
OpenNone
ResolvedAndyRussG
ResolvedAndyRussG
ResolvedAndyRussG
OpenAndyRussG
ResolvedAndyRussG
ResolvedAndyRussG
OpenNone
ResolvedEjegg
ResolvedNone
ResolvedAndyRussG
OpenNone
OpenNone
OpenNone
ResolvedAndyRussG
ResolvedAndyRussG
Openjgleeson
Openjgleeson
OpenAndyRussG
OpenNone
OpenNone
DuplicateNone
ResolvedAndyRussG
Openjgleeson
OpenEileenmcnaughton
DuplicateNone
OpenNone
OpenNone
ResolvedAndyRussG

Event Timeline

DStrine renamed this task from get payments wiki setup on docker to feature parity with vagrant on docker.Sep 15 2020, 7:53 PM
DStrine moved this task from Triage to Current Sprint on the Fundraising-Backlog board.
DStrine updated the task description. (Show Details)
DStrine renamed this task from feature parity with vagrant on docker to Epic: feature parity with vagrant on docker.Sep 15 2020, 8:01 PM
DStrine added a project: Epic.
DStrine updated the task description. (Show Details)
DStrine updated the task description. (Show Details)

Here are some notes about goals, requirements and nice-to-haves:

  • 1-2 steps to get a full dev setup
  • All system tools and libraries at the same version as production, whenever possible
  • Keep setup clean, don't include anything that's not needed
  • If something goes wrong on someone's local dev setup:
    • Easy to reproduce elsewhere
    • Easy to fix and share reliable fixes
    • Quick and easy to create new dev environment from scratch
    • Easy to create patches for deployment
  • Configurability:
    • Ports exposed on host machine
  • Dev tools
    • XDebug (for Web and CLI)
    • Mailcatcher
    • Reception and processing of IPN callbacks (ngrep)
    • Organized logs, easy to access
    • Easy setup of payment processor config with private credentials
    • Page with links to test donations, real donations, consoles, documentation
    • Test data for everything, that can be easily reset to initial state
    • Tests, linting and other CI checks run locally out-of-the-box
    • Automated batch test donation submission to investigate performance
    • Easy connection with MySQL Workbench and/or other graphical db tools
  • Full FR stack
    • Payments wiki
    • Civicrm
    • Queues
    • Queue consumers
    • Process control
    • IPN listeners
    • Audit processors
    • TY pages
    • Any other useful stuff from donatewiki?
    • CentralNotice Use standard MW dev images, separate setup procedure
    • Tools
      • silverpop export
      • django banner stats
      • fruec
    • Grafana
    • Prometheus
    • Logdog
  • Coordination when feasible with dev setup work across the WMF
    • Same tools for creating base images
    • Same repository for base images
    • Try to stay close to general MediaWiki setup for developers
    • Share base images (like buster-python3, for example) when appropriate
AndyRussG closed subtask Restricted Task as Resolved.Nov 24 2020, 8:05 PM

@AndyRussG
This is a tremendous feat! As an armchair observer, I love what I found in https://gerrit.wikimedia.org/r/plugins/gitiles/wikimedia/fundraising/dev/ , and I hope to find an excuse to run the stack one day. Also really impressed that this is happening near December, maybe that means the seasonal crunch has levelled out a bit? (Or maybe just that everyone is going bonkers and looking for side projects :p )

I have more than a little interest in this project, since WMDE Technical Wishes just kicked off our own docker-dev environment with (admittedly much less) complex requirements (T225826). In our case, the biggest obstacle was simply running a multi-wiki farm, the contextual challenge being that we need to work with many different combinations of MediaWiki extension and some cross wiki boundaries. I see that your problems will be more the heterogenous and unique software stack, and many interoperating services. But because of exactly that, I'm starting to think that you might be curious to look at the modular framework we began here, it might be even better for you than for us: https://gitlab.com/wmde/wmde-technicalwishes-docker-dev/ . For fr-tech, the variability might not be so important (you would probably want to set DEFAULT_MODULES to include everything), but the benefit is that each service can be logically encapsulated, yet still interact. It's mostly just a trick of directory structure, but it seems to be helping us keep concerns well separated for the moment. Would you like to compare notes, in real time even?

Hey @awight! Many, many apologies for the long delay in replying here, and thanks so much for sharing WMDE docker dev work, that looks amazing! I really like the modular setup and also how clean the bash code is.

Work on FR Docker setup was partially on hold for a little while, but it's expected that soon we'll be able to focus on it more fully. I'll check out the work you shared in detail, and I'm sure there's stuff we should draw on.

Would you like to compare notes, in real time even?

Yes that'd be fantastic, and thanks so much for the offer!!

P.S. Thanks also for reviewing the process-control Docker stuff :)