Page MenuHomePhabricator
Create Task
Maniphest T170904

Create a Wikimedia fork of the Jenkins android emulator plugin
Closed, ResolvedPublic
Actions

Description

The Android team relies on the Jenkins Android emulator plugin for CI emulator testing, but it is increasingly a stretch to call it actively maintained. The GitHub repo has 13 open pull requests and its last commit was May 23, 2016. Its constants file has not been updated since Android 6.0 (for context, 7.1 is the current version and 8.0 will likely ship in the next few months). Due to some no-longer-valid hardcoded bits in SdkInstaller.java it is now blocking T150623.

If we are going to continue relying on this piece of software I propose that we fork it so that we can update it as needed to provide us the functionality we need on an ongoing basis.

Details

SubjectRepoBranchLines +/-
Maven job for Jenkins pluginsintegration/configmaster+34 -0
Customize query in gerrit

Related Objects

Event Timeline

• Mholloway created this task.Jul 17 2017, 11:56 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 17 2017, 11:56 PM
• Mholloway updated the task description. (Show Details)Jul 17 2017, 11:56 PM
• Mholloway updated the task description. (Show Details)Jul 17 2017, 11:58 PM
• Mholloway updated the task description. (Show Details)
hashar added a comment.Jul 18 2017, 8:59 AM

Surely we could create a repo in Gerrit integration/jenkinsci/android-emulator-plugin, fork from git hub and then pill up patches in that fork. Then mvn package should produce a .hpi file that we can then upload to our Jenkins instance.

I asked on the IRC channel #jenkins and that plugin author (orrc) actually replied! So at least he is still around :]

Could you try sending a pull request on Github? Then during European time zone I guess I can poke him to get it reviewed/merged/released.

And I guess at some point you might well end up being added to the Github project. That would save us the effort of maintaining our own fork.

hashar added a comment.Jul 18 2017, 10:29 AM

From upstream developer (orrc):

The SdkConstants file doesn't actually need to be up-to-date in order to use newer Android versions. You can use "android-25" even if the plugin doesn't currently understand "7.1"

So it seems in JJB you can:

jjb/mobile.yaml
 - job-template:
     name: 'apps-android-wikipedia-periodic-test'
     ...
     wrappers:
-         os: android-24
+         os: android-25
     screen-density: 240
hashar added a comment.Jul 18 2017, 10:41 AM

Another point mentionned by orrc: we already use android-24 which the plugin does not know about :]

hashar added a comment.Jul 18 2017, 10:45 AM

So our job https://integration.wikimedia.org/ci/job/apps-android-wikipedia-periodic-test/ actually got changed to android-25 but the build eventually fails:

$ /srv/jenkins-workspace/tools/android-sdk/tools/android update sdk -u -a -t sys-img-x86-android-25
Refresh Sources:
  Fetching https://dl.google.com/android/repository/addons_list-2.xml
  Validate XML
  Parse XML
  Fetched Add-ons List successfully
  Refresh Sources
  Fetching URL: https://dl.google.com/android/repository/repository-11.xml
  Validate XML: https://dl.google.com/android/repository/repository-11.xml
  Parse XML:    https://dl.google.com/android/repository/repository-11.xml
  Fetching URL: https://dl.google.com/android/repository/addon.xml
  Validate XML: https://dl.google.com/android/repository/addon.xml
  Parse XML:    https://dl.google.com/android/repository/addon.xml
  Fetching URL: https://dl.google.com/android/repository/glass/addon.xml
  Validate XML: https://dl.google.com/android/repository/glass/addon.xml
  Parse XML:    https://dl.google.com/android/repository/glass/addon.xml
  Fetching URL: https://dl.google.com/android/repository/extras/intel/addon.xml
  Validate XML: https://dl.google.com/android/repository/extras/intel/addon.xml
  Parse XML:    https://dl.google.com/android/repository/extras/intel/addon.xml
  Fetching URL: https://dl.google.com/android/repository/sys-img/android/sys-img.xml
  Validate XML: https://dl.google.com/android/repository/sys-img/android/sys-img.xml
  Parse XML:    https://dl.google.com/android/repository/sys-img/android/sys-img.xml
  Fetching URL: https://dl.google.com/android/repository/sys-img/android-wear/sys-img.xml
  Validate XML: https://dl.google.com/android/repository/sys-img/android-wear/sys-img.xml
  Parse XML:    https://dl.google.com/android/repository/sys-img/android-wear/sys-img.xml
  Fetching URL: https://dl.google.com/android/repository/sys-img/android-tv/sys-img.xml
  Validate XML: https://dl.google.com/android/repository/sys-img/android-tv/sys-img.xml
  Parse XML:    https://dl.google.com/android/repository/sys-img/android-tv/sys-img.xml
  Fetching URL: https://dl.google.com/android/repository/sys-img/google_apis/sys-img.xml
  Validate XML: https://dl.google.com/android/repository/sys-img/google_apis/sys-img.xml
  Parse XML:    https://dl.google.com/android/repository/sys-img/google_apis/sys-img.xml
  Fetching URL: https://dl.google.com/android/repository/sys-img/google_apis_playstore/sys-img.xml
  Validate XML: https://dl.google.com/android/repository/sys-img/google_apis_playstore/sys-img.xml
  Parse XML:    https://dl.google.com/android/repository/sys-img/google_apis_playstore/sys-img.xml
Error: Ignoring unknown package filter 'sys-img-x86-android-25'
Warning: The package filter removed all packages. There is nothing to install.
         Please consider trying to update again without a package filter.
[android] Using Android SDK: /srv/jenkins-workspace/tools/android-sdk
[android] Creating Android AVD: /mnt/home/jenkins-deploy/.android/avd/hudson_en-US_240_WVGA_android-25_x86.avd
[android] /srv/jenkins-workspace/tools/android-sdk/tools/android create avd -f -a -s WVGA800 -n hudson_en-US_240_WVGA_android-25_x86 -t android-25 --abi x86
[android] Could not create Android emulator: Failed to parse AVD config file
Archiving artifacts
• Mholloway added a comment.EditedJul 18 2017, 3:59 PM
In T170904#3447425, @hashar wrote:

So our job https://integration.wikimedia.org/ci/job/apps-android-wikipedia-periodic-test/ actually got changed to android-25 but the build eventually fails:
[snip]

Error: Ignoring unknown package filter 'sys-img-x86-android-25'
Warning: The package filter removed all packages. There is nothing to install.
         Please consider trying to update again without a package filter.

Hey @hashar, thanks for investigating! Yep, I saw from looking at the code earlier that (as you mentioned above) the plugin is able to parse the desired SDK version number from an arbitrary android-xx string. That comment about the constants was just to illustrate my general concerns about the maintenance level of the plugin. The real problem is a little later on, where the plugin constructs the component name for the system image:

// Add system image, if required
// Even if a system image doesn't exist for this platform, the installer silently ignores it
if (dependentPlatform >= 10 && abi != null) {
  if (sdk.supportsSystemImageNewFormat()) {
    String tag = "android";
    [...]
    components.add(String.format("sys-img-%s-%s-%d", abi, tag, dependentPlatform));
  } else {
    components.add(String.format("sysimg-%d", dependentPlatform));
  }
}

(source)

As the output says, the sys-img-x86-android-25 component name constructed here doesn't actually exist. sys-img-x86-google_apis-25 does exist, but at present there's no way to compel the plugin to construct that name without injecting the unwanted android substring.

I'll do an upstream PR later, then.

hashar added a comment.Jul 19 2017, 12:49 PM

Ah nice. I did play with the job yesterday and it got confused because android-25 apparently has no ABI dependent platform.

As soon as you have a pull request we can pick it in Gerrit, rebuild the plugin from the fork and update our Jenkins setup.

Stashbot subscribed.Jul 19 2017, 12:54 PM

Mentioned in SAL (#wikimedia-releng) [2017-07-19T12:54:43Z] <hashar> Gerrit: created repo integration/jenkinsci/android-emulator-plugin.git owned by access group integration-jenkinsci-android-emulator-plugin which has Mholloway - T170904

hashar added a comment.Jul 19 2017, 12:59 PM

And I made a fork of the repository in Gerrit. Should be clonal via:

git clone ssh://gerrit.wikimedia.org:29418/integration/jenkinsci/android-emulator-plugin

I guess we can close this task and follow up on the parent one T150623 ?

• Mholloway closed this task as Resolved.Jul 19 2017, 1:11 PM
• Mholloway claimed this task.

Got it! Yes, let's follow up there. Thank you, @hashar!

gerritbot subscribed.Jul 19 2017, 2:12 PM

Change 366257 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Maven job for Jenkins plugins

https://gerrit.wikimedia.org/r/366257

gerritbot added a project: Patch-For-Review.Jul 19 2017, 2:12 PM
gerritbot added a comment.Jul 19 2017, 2:19 PM

Change 366257 merged by jenkins-bot:
[integration/config@master] Maven job for Jenkins plugins

https://gerrit.wikimedia.org/r/366257

greg reassigned this task from • Mholloway to hashar.Jul 19 2017, 4:13 PM
greg subscribed.

No offense, @Mholloway, but I'll re-assign to Antoine to give him credit for the investigation (and for us to track work) ;)

Restricted Application added a project: Release-Engineering-Team (Kanban). · View Herald TranscriptJul 19 2017, 4:13 PM
• Mholloway added a comment.Jul 19 2017, 4:14 PM

Ah, no worries, looks like Phab just auto-assigned this to me when I marked it resolved...

greg added a comment.Jul 19 2017, 4:14 PM

No worries, that usually is a good approximate :)

• Phabricator_maintenance edited projects, added RelEng-Archive-FY201718-Q1; removed Release-Engineering-Team (Kanban).Sep 26 2017, 11:45 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL