scap logs sometimes have incorrect date
Open, LowPublic
Actions

Assigned To

None

Authored By

	awight
	Nov 22 2017, 5:11 PM

Description

While investigating my own mess, I discovered that some scap logs had the wrong date:

ls -ltr scap/log
....
-rw-rw-r-- 1 awight    wikidev 10096 Nov  8 16:49 scap-sync-2017-11-08-0001.log
-rw-rw-r-- 1 awight    wikidev 83060 Nov 20 22:27 scap-sync-2017-11-08-0002-1-g5084251.log
-rw-rw-r-- 1 awight    wikidev 25178 Nov 20 22:55 scap-sync-2017-11-20-0001.log
-rw-rw-r-- 1 awight    wikidev  7161 Nov 20 23:19 95cd523.log
-rw-rw-r-- 1 awight    wikidev 32871 Nov 20 23:38 scap-sync-2017-11-08-0003.log

I remember that I had run scap deploy-log on those 11-08 files, and that they existed before I ran my 11-20 deployments. It seems that scap deploy picked up the last filename given to deploy-log, and overwrote it.

Related Objects

Mentioned In: T181661: Connection timeout from tin to new ores servers
Mentioned Here: T179711: ORES 500 errors on a threshold lookup request
T181006: Watchlist and RecentChanges failure due to ORES on frwiki and ruwiki

Event Timeline

awight created this task.Nov 22 2017, 5:11 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 22 2017, 5:11 PM

More context: I was attempting rollbacks, here are the SAL lines from that day:

21:37 <awight@tin> Started deploy [ores/deploy@5084251]: Updating ORES to revscoring 2.0.10, T179711
22:54 <awight@tin> Started deploy [ores/deploy@5084251]: Rollback ORES; T179711
23:18 awight@tin: Started deploy [ores/deploy@95cd523]: Rollback ORES (take 2); 181006
23:25 <awight@tin> Started deploy [ores/deploy@82a13ae]: Rollback ORES (take 3); T181006

Looks like we name the log file with git describe --always which could make for some funky names indeed: https://github.com/wikimedia/scap/blob/master/scap/deploy.py#L1000-L1001

We should probably just use the scap tag, although we make that much later in deploy, we could probably move that logic a bit since it's pretty straight-forward/deterministic.

awight mentioned this in T181661: Connection timeout from tin to new ores servers.Dec 12 2017, 3:38 PM

greg lowered the priority of this task from Medium to Low.Jan 22 2018, 5:57 PM

greg edited projects, added Release-Engineering-Team (Next); removed Release-Engineering-Team (Kanban).

• mmodell subscribed.Feb 1 2018, 12:14 AM

• mmodell moved this task from Needs triage to Debt on the Scap board.Feb 1 2018, 12:23 AM

awight unsubscribed.Mar 21 2019, 4:01 PM

• Phabricator_maintenance edited projects, added Release-Engineering-Team-TODO; removed Release-Engineering-Team (Next).Jun 14 2019, 9:45 PM

• Phabricator_maintenance moved this task from Should be empty (use Release-Engineering-Team) to Soon-ish on the Release-Engineering-Team-TODO board.Jun 14 2019, 9:46 PM

greg added a project: Release-Engineering-Team.Jun 21 2019, 10:35 PM

greg edited projects, added Release-Engineering-Team (Deployment services); removed Release-Engineering-Team.Jun 24 2019, 9:20 PM

thcipriani removed a project: Release-Engineering-Team (Deployment services).Apr 20 2021, 1:10 AM

thcipriani edited projects, added Release-Engineering-Team (thcipriani-workboard-fiddling); removed Release-Engineering-Team-TODO.Apr 20 2021, 3:42 AM