CI for Civi: provision and run tests under Jenkins/Zuul
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• atgo
	Jan 8 2015, 12:40 AM

Description

Set up a labs instance of the WMF CI system. [not in the cards]
Create repos for all the tools. Also for vendor/ subrepos :(
Integrate the civibuild work.
Merge to production and deploy.
The following job will check out repos, run civicrm-buildkit, and phpunit:

https://integration.wikimedia.org/ci/view/WMF/job/wikimedia-fundraising-civicrm/

Status

Jenkins slave scripts pending review, https://gerrit.wikimedia.org/r/#/c/158554/

Jenkins job is manually configured to run the slave scripts and tests, but we need to move this into a file-backed Jenkins Job Builder config.
Precreated db type for amp pending review: https://github.com/totten/amp/pull/18 -- Currently, one db is reused for all three connections, not great, but an acceptable starting point.
We must circumvent Composer. Repos and submodules are provisioning nicely, and we’re disabling the builtin composer run by setting the $OFFLINE environment var.

https://gerrit.wikimedia.org/r/179366
https://gerrit.wikimedia.org/r/179371
https://gerrit.wikimedia.org/r/179372
https://gerrit.wikimedia.org/r/179373
https://gerrit.wikimedia.org/r/179374
https://gerrit.wikimedia.org/r/179377
https://gerrit.wikimedia.org/r/179378
https://gerrit.wikimedia.org/r/179379
https://gerrit.wikimedia.org/r/179380
https://gerrit.wikimedia.org/r/179381
https://gerrit.wikimedia.org/r/179384
https://gerrit.wikimedia.org/r/179534
https://gerrit.wikimedia.org/r/179535

May need more code to remove the ~/.amp config at the end of a run, and any other statefulness being stored outside the WORKSPACE directory.
Would like to improve the nasty symlink workaround for missing --web-root option to buildkit.
Minor configuration and application-level issues to clean up before the phpunit tests will pass.

Related Objects
Search...

Status	Assigned	Task
Resolved	Ejegg	T89404 Create unit and integration tests for Fundraising extensions to identify breaking MediaWiki changes
Resolved	awight	T78100 Continuous integration - CiviCRM
Declined	None	T90630 Recurring Payments Reporting
Resolved	• mepps	T97372 Enable ability to look up contacts by phone number
Resolved	None	T77910 [epic] Upgrade Civi to 4.6 & integrate new reporting
Resolved	awight	T86103 CI for Civi: provision and run tests under Jenkins/Zuul
Resolved	awight	T86374 Deploy CiviCRM integration job to WMF integration server
Declined	awight	T88599 Create new labs project: fundraising-integration
Invalid	awight	T89894 Create and provision CI slave instance for CiviCRM testing
Resolved	awight	T90472 Add Fundraising Tech team to the labs Integration project
Resolved	awight	T89895 Configure Jenkins to run CiviCRM builds on Fundraising CI slave instance
Resolved	awight	T89896 Run CiviCRM testing scripts during CI
Resolved	awight	T86686 Ressurect phpunit tests for the CRM modules
Declined	None	T88379 Skip tests when modules not enabled
Resolved	awight	T91895 Write Jenkins job builder definition for CiviCRM CI job
Resolved	awight	T91905 Make Civi CI job run on civicrm, drupal, and vendor (DonationInterface and SmashPig) repos
Resolved	awight	T93707 Mysterious failure to zuul-clone drupal repo
Resolved	awight	T94586 Disable CI job on CRM deployment branch
Resolved	Eileenmcnaughton	T91911 Point amp cache to $WORKSPACE

Event Timeline

• atgo created this task.Jan 8 2015, 12:40 AM

• atgo changed the task status from Open to Stalled.

• atgo assigned this task to awight.

• atgo raised the priority of this task from to Medium.

• atgo updated the task description. (Show Details)

• atgo added a project: § Fundraising Sprint Abba.

• atgo set Security to None.

• atgo edited a custom field.

• atgo subscribed.

• atgo moved this task from In Analysis to Pending Code Review on the § Fundraising Sprint Abba board.Jan 8 2015, 12:43 AM

awight moved this task from Pending Code Review to Pending Deployment on the § Fundraising Sprint Abba board.Jan 13 2015, 6:56 PM

• atgo added a subtask: T86374: Deploy CiviCRM integration job to WMF integration server.Jan 14 2015, 11:21 PM

• atgo added a project: § Fundraising Sprint Beastie Boys.Jan 15 2015, 12:12 AM

• atgo moved this task from In Analysis to Pending Deployment on the § Fundraising Sprint Beastie Boys board.

awight removed projects: § Fundraising Sprint Beastie Boys, § Fundraising Sprint Abba.Jan 16 2015, 9:15 PM

Aklapper added projects: Wikimedia-Fundraising-CiviCRM, Continuous-Integration-Infrastructure.Jan 24 2015, 5:54 AM

Aklapper edited a custom field.

• atgo moved this task from Backlog to ContributionTracking on the Wikimedia-Fundraising-CiviCRM board.Jan 29 2015, 9:22 PM

• atgo added a parent task: T78100: Continuous integration - CiviCRM.

awight added a subtask: T86686: Ressurect phpunit tests for the CRM modules.Feb 5 2015, 7:42 AM

awight added a parent task: T77910: [epic] Upgrade Civi to 4.6 & integrate new reporting.Feb 5 2015, 7:56 AM

awight closed subtask T86686: Ressurect phpunit tests for the CRM modules as Resolved.Feb 19 2015, 5:55 PM

Ejegg added a project: Scrum-of-Scrums.Feb 25 2015, 6:26 PM

Ejegg moved this task from Scheduled to blocked by Release Engineering on the Scrum-of-Scrums board.

• atgo moved this task from ContributionTracking to UserRequests (misc) on the Wikimedia-Fundraising-CiviCRM board.Feb 27 2015, 12:56 AM

Ejegg removed a project: Scrum-of-Scrums.Mar 11 2015, 5:30 PM

awight closed subtask T91905: Make Civi CI job run on civicrm, drupal, and vendor (DonationInterface and SmashPig) repos as Resolved.Mar 19 2015, 6:42 PM

awight closed subtask T86374: Deploy CiviCRM integration job to WMF integration server as Resolved.Mar 19 2015, 6:50 PM

hashar closed subtask T91895: Write Jenkins job builder definition for CiviCRM CI job as Resolved.Apr 13 2015, 11:18 AM

Krinkle moved this task from Untriaged to Backlog on the Continuous-Integration-Infrastructure board.Apr 13 2015, 4:52 PM

Per blocked tasks, the only outstanding work here is to prevent CI jobs on the deployment branch, since we've removed the test classes there.

awight closed subtask T94586: Disable CI job on CRM deployment branch as Resolved.May 11 2015, 8:38 PM

@awight should you add a blocking task to have the job to run concurrently? IIRC there is some cache in $HOME which needs to vary between build in case two builds happens on the same node.

Would be nice to have imho but I have no knowledge about the framework CiviCRM is using.

hashar moved this task from Backlog to In-progress on the Continuous-Integration-Infrastructure board.May 13 2015, 7:55 PM

awight changed the task status from Stalled to Open.May 13 2015, 8:31 PM

awight added a subtask: T91911: Point amp cache to $WORKSPACE.

@awight are you keep this task open solely because it is blocked by T91911 ? Seems most of the work is complete now. Just want to clarify.

Seems this is just waiting for T91911: Point amp cache to $WORKSPACE.

hashar changed the task status from Open to Stalled.Oct 6 2015, 11:26 AM

hashar lowered the priority of this task from Medium to Low.

awight closed this task as Resolved.Oct 14 2015, 10:58 PM

A left over task is T91911: Point amp cache to $WORKSPACE it has a pull request with upstream.

@awight let us know if you need any more support / tweak :-)

Eileenmcnaughton closed subtask T91911: Point amp cache to $WORKSPACE as Resolved.Feb 3 2016, 11:17 PM

CI for Civi: provision and run tests under Jenkins/ZuulClosed, ResolvedPublicActions

Description

Related ObjectsSearch...

Event Timeline

CI for Civi: provision and run tests under Jenkins/Zuul
Closed, ResolvedPublic
Actions

Related Objects
Search...