Fixes T125685
Details
- Maniphest Tasks
- T125685: refreshCdbJsonFiles should be rewritten in python
- Reviewers
dduvall mmodell bd808 demon - Group Reviewers
Release-Engineering-Team - Commits
- rMSCAba8e257037e0: Rewrite refreshCdbJsonFiles in python
- Patch without arc
- git checkout -b D133 && curl -L https://phabricator.wikimedia.org/D133?download=true | git apply
regenerate json files and compare
Diff Detail
- Repository
- rMSCA Scap
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Event Timeline
Build is green https://integration.wikimedia.org/ci/job/harbormaster-test/303/ for more details.
Looks pretty good. A few things noted inline about the json file formatting.
The new entry point script should be added to docs/scap2/commands.rst as well.
scap/main.py | ||
---|---|---|
732 | "Directory containing" | |
734 | uppercase Number | |
scap/tasks.py | ||
620 | Why not just test and return here? | |
637 | json.dumps should use indent=0 to add newlines so the files are readable by humans too. To match the hand built json from the PHP version you should set separators=(',', ':') as well. |
Logic tweaks and doc updates
- Spelling changes in help text
- Add json.dumps options to make json human readable
- Return as early as possible in refresh_cdb_json_file
- Add refreshCdbJsonFiles to scap2 docs
Build is green https://integration.wikimedia.org/ci/job/harbormaster-test/304/ for more details.
Not tested (my scap dev vm is badly broken) but the code looks good. This should be tested on beta cluster for a bit before rolling out to production just to be sure that it works as expected. My main concern would be ensuring that the N+1th run doesn't vary the the JSON output unnecessarily. I would expect that the first run following deployment will dirty all the JSON blobs due to dict ordering differences from the PHP insert order behavior.
Build is green https://integration.wikimedia.org/ci/job/harbormaster-test/329/ for more details.
Build is unstable
W: scap source: binary-nmu-debian-revision-in-source 3.0.3-1+0~20160317174845.32~1.gbpad5a3c
N:
N: The version number of your source package ends in +b and a number or has
N: a Debian revision containing three parts. These version numbers are used
N: by binary NMUs and should not be used as the source version. (The +b
N: form is the current standard; the three-part version number now
N: obsolete.)
N:
N: Refer to Debian Developer's Reference section 5.10.2.1 (Recompilation or
N: binary-only NMU) for details.
N:
N: Severity: normal, Certainty: certain
N:
N: Check: fields, Type: binary, udeb, source
N:
W: scap source: newer-standards-version 3.9.6 (current is 3.9.5)
N:
N: The source package refers to a Standards-Version which is newer than the
N: highest one lintian is programmed to check. If the source package is
N: correct, then please upgrade lintian to the newest version. (If there is
N: no newer lintian version, then please bug li
Link to build: https://integration.wikimedia.org/ci/job/beta-build-deb/32/
See console output for more information: https://integration.wikimedia.org/ci/job/beta-build-deb/32/console
Build is green https://integration.wikimedia.org/ci/job/harbormaster-test/331/ for more details.
Build is unstable
W: scap source: binary-nmu-debian-revision-in-source 3.0.3-1+0~20160317223557.34~1.gbp86dce1
N:
N: The version number of your source package ends in +b and a number or has
N: a Debian revision containing three parts. These version numbers are used
N: by binary NMUs and should not be used as the source version. (The +b
N: form is the current standard; the three-part version number now
N: obsolete.)
N:
N: Refer to Debian Developer's Reference section 5.10.2.1 (Recompilation or
N: binary-only NMU) for details.
N:
N: Severity: normal, Certainty: certain
N:
N: Check: fields, Type: binary, udeb, source
N:
W: scap source: newer-standards-version 3.9.6 (current is 3.9.5)
N:
N: The source package refers to a Standards-Version which is newer than the
N: highest one lintian is programmed to check. If the source package is
N: correct, then please upgrade lintian to the newest version. (If there is
N: no newer lintian version, then please bug li
Link to build: https://integration.wikimedia.org/ci/job/beta-build-deb/34/
See console output for more information: https://integration.wikimedia.org/ci/job/beta-build-deb/34/console
scap/tasks.py | ||
---|---|---|
612 | Nit: Python's logger magically does msg % args interpolation. |