When doing the WMF update patches, I start fresh each time. There is no inherited committed results from previous times (not intentionally at least).

Afaik what is comitted currently is the result of cloning upstream, ensuring no untracked files exist (incl old vendor), pinning platform deps to PHP 5.6 etc, and then letting Composer fetch the dependencies as best it can. I suspect the platform pinning for tungsten's PHP 5.6 is the main reason Composer ends up settling on fairly old package versions.

Change 602203 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[performance/debs/xhgui@master] [WIP] Package XHGui

https://gerrit.wikimedia.org/r/602203

Change 602205 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[performance/debs/php-slim@master] [WIP] Package php-slim

https://gerrit.wikimedia.org/r/602205

Change 602206 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[performance/debs/php-psr-http-server-middleware@master] [WIP] Package php-psr-http-server-middleware

https://gerrit.wikimedia.org/r/602206

Change 602207 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[performance/debs/php-psr-http-server-handler@master] [WIP] Package php-psr-http-server-handler

https://gerrit.wikimedia.org/r/602207

Change 602514 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[performance/debs/php-slim-views@master] [WIP] Package php-slim-views

https://gerrit.wikimedia.org/r/602514

Change 602516 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[performance/debs/php-perftools-xhgui-collector@master] [WIP] Package php-perftools-xhgui-collector

https://gerrit.wikimedia.org/r/602516

The ancient versions are pinned in composer.json because the API has changed between versions. I spent some time today trying to update the code to support the latest version of each dependency, but this became a bit of a yak-shaving exercise, so I abandoned it in favor of pointing the packaging at the old releases. I'll revisit this once my other Xhgui changes (to fully support non-MongoDB backends) are merged.

Change 610426 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[performance/debs/php-pimple@master] Package php-pimple

https://gerrit.wikimedia.org/r/610426

The seven Gerrit changes linked to this ticket have been verified to build correctly with pbuilder, and are ready for +2 and upload. All of the significant Lintian warnings have been fixed.

Following these instructions, I created a host (deployment-boron01.deployment-prep.eqiad.wmflabs) with role::package::builder. I then configured ~/.gbp.conf to use pbuilder, checked out each proposed package from Gerrit, and ran gbp buildpackage -uc -us.

There is one additional package from stretch, php-twig which needs to be backported. I did:

$ dget -x http://deb.debian.org/debian/pool/main/t/twig/twig_1.24.0-2+deb9u1.dsc
$ cd twig-1.24.0
$ DEB_BUILD_OPTIONS=nocheck pdebuild

nocheck was needed because buster's phpunit failed due to API changes since stretch. I can do a proper backport (forwardport?) if this is a show-stopper, but the package works just fine as-is. (Longer-term, we want to submit upstream changes to accommodate more modern versions of XHGui's dependencies.)

I then did:

# cd /var/cache/pbuilder/result/buster-amd64
# dpkg-scanpackages . | gzip -c > Packages.gz

...and copied the resulting directory to a fresh host (deployment-xhgui03.deployment-prep.eqiad.wmflabs). To simulate everything being in reprepro, I did:

# echo "deb [trusted=yes] file:///home/dpifke/result/buster-amd64 ./" >>/etc/apt/sources.list
# apt-get update
# apt-get install xhgui php-twig=1.24.0-2+deb9u1

A puppet patch to pin php-twig is forthcoming.

XHGui was confirmed to work with the freshly-built packages.

Change 610446 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[operations/puppet@production] xhgui: Pin php-twig at version 1.*

https://gerrit.wikimedia.org/r/610446

Would SRE have any objection to us (Performance) setting up our own reprepro instance, with upload rights for members of performance-team? (It would be added to apt only on hosts where we have root.)

Change 610446 merged by Dzahn:
[operations/puppet@production] xhgui: Pin php-twig at version 1.*

https://gerrit.wikimedia.org/r/610446

In T254310#6291975, @dpifke wrote:

(If you're not the best person to do the upload, can you please reassign to the appropriate SRE? Thanks!)

While I can upload packages to apt.wikimedia.org and import them with reprepro, I do not have the +2 in Gerrit to merge those pending changes.

Regarding the php packages it would be best to talk to @Muehlenhoff

In T254310#6303117, @dpifke wrote:

Would SRE have any objection to us (Performance) setting up our own reprepro instance, with upload rights for members of performance-team? (It would be added to apt only on hosts where we have root.)

It would be best to bring this up on the ops mailing list or a separate ticket.

In T254310#6303189, @Dzahn wrote:

While I can upload packages to apt.wikimedia.org and import them with reprepro, I do not have the +2 in Gerrit to merge those pending changes.

I've added wmf-sre-serviceops to performance/debs/* on Gerrit.

Change 602203 merged by Dzahn:
[performance/debs/xhgui@master] Package XHGui

https://gerrit.wikimedia.org/r/602203

Change 602514 merged by Dzahn:
[performance/debs/php-slim-views@master] Package php-slim-views

https://gerrit.wikimedia.org/r/602514

Change 602207 merged by Dzahn:
[performance/debs/php-psr-http-server-handler@master] Package php-psr-http-server-handler

https://gerrit.wikimedia.org/r/602207

Change 602206 merged by Dzahn:
[performance/debs/php-psr-http-server-middleware@master] Package php-psr-http-server-middleware

https://gerrit.wikimedia.org/r/602206

Change 610426 merged by Dzahn:
[performance/debs/php-pimple@master] Package php-pimple

https://gerrit.wikimedia.org/r/610426

Change 602516 merged by Dzahn:
[performance/debs/php-perftools-xhgui-collector@master] Package php-perftools-xhgui-collector

https://gerrit.wikimedia.org/r/602516

Change 602205 merged by Dzahn:
[performance/debs/php-slim@master] Package php-slim 2.6.3

https://gerrit.wikimedia.org/r/602205

I built the following packages on deneb.codfw.wmnet using DIST=buster pdebuild.

• php-slim-views_0.1.3-wmf1 (https://gerrit.wikimedia.org/r/c/performance/debs/php-slim-views/+/602514)
• php-psr-http-server-handler_1.0.1-wmf1 (https://gerrit.wikimedia.org/r/c/performance/debs/php-psr-http-server-handler/+/602207)
• php-psr-http-server-middleware_1.0.1-wmf1 (https://gerrit.wikimedia.org/r/c/performance/debs/php-psr-http-server-middleware/+/602206)
• php-pimple_1.0.2-wmf1 (https://gerrit.wikimedia.org/r/c/performance/debs/php-pimple/+/610426)
• php-perftools-xhgui-collector_1.8.0-1-wmf1 (https://gerrit.wikimedia.org/r/c/performance/debs/php-perftools-xhgui-collector/+/602516)
• php-slim_2.6.3-wmf1 (https://gerrit.wikimedia.org/r/c/performance/debs/php-slim/+/602205)

I got the upstream tarballs from the cloud instance used by Dave.

example:

scp deployment-boron01.deployment-prep:/home/dpifke/php-*.tar.gz .
scp php-*.tar.gz deneb.codfw.wmnet:~

[deneb:~] $ git clone "https://gerrit.wikimedia.org/r/performance/debs/php-slim"

[deneb:~/php-slim] $ DIST=buster pdebuild

..
[apt1001:~] $ rsync -av rsync://deneb.codfw.wmnet/pbuilder-result/buster-amd64/php-slim* .

..
[apt1001:~] $ sudo -E reprepro -C main include buster-wikimedia php-slim_2.6.3-wmf1_amd64.changes

From a random buster host i can now simulate to install, f.e. php-slim:

[planet1002:~] $ sudo apt-get -s install php-slim

Use 'sudo apt autoremove' to remove it.
The following additional packages will be installed:
  php-common
Suggested packages:
  php-mcrypt php-phpseclib-mcrypt-compat
The following NEW packages will be installed:
  php-common php-slim

The xhgui* hosts are on stretch though because in the previous version using MongoDB buster was an issue.

Now we want buster of course to install these packages, so we need to reimage the xhgui hosts for which i made T259206.

xhgui also done!

[apt1001:~] $ for package in php-slim-views php-psr-http-server-handler php-psr-http-server-middleware php-pimple php-perftools-xhgui-collector php-slim xhgui; do sudo -E reprepro ls $package; done

php-slim-views | 0.1.3-wmf1 | buster-wikimedia | amd64, i386, source
php-psr-http-server-handler | 1.0.1-wmf1 | buster-wikimedia | amd64, i386, source
php-psr-http-server-middleware | 1.0.1-wmf1 | buster-wikimedia | amd64, i386, source
php-pimple | 1.0.2-wmf1 | buster-wikimedia | amd64, i386, source
php-perftools-xhgui-collector | 1.8.0-1-wmf1 | buster-wikimedia | amd64, i386, source
php-slim | 2.6.3-wmf1 | buster-wikimedia | amd64, i386, source
xhgui | 0.9.0-1-wmf1 | buster-wikimedia | amd64, i386, source

[apt1001:~] $ apt-cache show xhgui
Package: xhgui
Version: 0.9.0-1-wmf1
Architecture: all
Maintainer: Dave Pifke <dpifke@wikimedia.org>
Installed-Size: 1055
Depends: apache2 | httpd, php-common, php-slim (>= 2.6.3), php-slim (<< 3~~), php-slim-views (>= 0.1.0), php-slim-views (<< 0.2~~), php-twig (>= 1.17), php-twig (<< 2~~), php-pimple (>= 1.0.2), php-pimple (<< 2~~), php-perftools-xhgui-collector (>= 1.7.0), php-perftools-xhgui-collector (<< 2~~), libjs-bootstrap, libjs-jquery, libjs-jquery-tablesorter
Recommends: apache2 (>= 2.4.6-4~) | httpd
Suggests: php-xhprof, php-uprofiler, php-mongo, php-mongodb, php-alcaeus-mongo-php-adapter
Priority: optional
Section: php
Filename: pool/main/x/xhgui/xhgui_0.9.0-1-wmf1_all.deb
Size: 216740
SHA256: acfb2f1b4f728c9047168218287fc6969440d113f914bf0ded908c3efd32ce7b
SHA1: 465f34a4cfc8f5966958740776d891de5e9e28fc
MD5sum: 1abcbec95d203ed6e78135aaada1391e
Description: A web based interface for viewing profile data collected by XHProf
 XHGui takes profiling information from XHProf and provides a
 convenient GUI for working with it.
Description-md5: 5231b8d3ec82acb186e87569cf9fd198

When simulating an install we still have this though:

The following packages have unmet dependencies:
 xhgui : Depends: php-twig (< 2~~) but 2.6.2-2 is to be installed

Re: php-twig, from an earlier comment:

In T254310#6291955, @dpifke wrote:

There is one additional package from stretch, php-twig which needs to be backported. I did:

$ dget -x http://deb.debian.org/debian/pool/main/t/twig/twig_1.24.0-2+deb9u1.dsc
$ cd twig-1.24.0
$ DEB_BUILD_OPTIONS=nocheck pdebuild

nocheck was needed because buster's phpunit failed due to API changes since stretch. I can do a proper backport (forwardport?) if this is a show-stopper, but the package works just fine as-is. (Longer-term, we want to submit upstream changes to accommodate more modern versions of XHGui's dependencies.)

(Presumably the output of the above can be uploaded as-is; worst-case you may have to update the target in debian/changelog.)

...

A puppet patch to pin php-twig is forthcoming.

(That patch was https://gerrit.wikimedia.org/r/c/operations/puppet/+/610446.)

twig/php-twig has been imported to apt.wikimedia.org as well

[apt1001:~] $ sudo -E reprepro ls twig
twig | 1.24.0-2+deb9u1 | buster-wikimedia | source
[apt1001:~] $ sudo -E reprepro ls php-twig
php-twig | 1.24.0-2+deb9u1 | buster-wikimedia | amd64, i386

I used dget as suggested and could also confirm that DEB_BUILD_OPTIONS=nocheck was needed to make it work.

[deneb:~] $ dget -x http://deb.debian.org/debian/pool/main/t/twig/twig_1.24.0-2+deb9u1.dsc
..
[deneb:~/twig-1.24.0] $ DEB_BUILD_OPTIONS=nocheck DIST=buster pdebuild

Had to rsync both php-twig* and twig* files.

Then also manually scp the orig.tar.gz from my home dir on deneb to apt1001.

[apt1001:~] $ rsync -av rsync://deneb.codfw.wmnet/pbuilder-result/buster-amd64/php-twig* .
[apt1001:~] $ rsync -av rsync://deneb.codfw.wmnet/pbuilder-result/buster-amd64/twig* .

 scp twig_1.24.0* apt1001.wikimedia.org:~
scp deneb.codfw.wmnet:~/twig_* .

Finally tell reprepro to "ignore missingfile" which means it will look for the orig.tar.gz in the current working dir to get around Unable to find pool/main/t/twig/twig_1.24.0.orig.tar.gz needed by twig_1.24.0-2+deb9u1.dsc. Put the twig_1.24.0.orig.tar.gz in the current working dir and it finds it with:

[apt1001:~] $ sudo -E reprepro --ignore=missingfile -C main include buster-wikimedia twig_1.24.0-2+deb9u1_amd64.changes