Page MenuHomePhabricator
Create Task
Maniphest T219333

apt-get update broken on jessie: jessie-updates and jessie-backports removed by Debian
Closed, ResolvedPublic
Actions

Description

On jessie hosts apt-get update is failing to download some indices due to 404 Not Found, in particular:

jessie-updates/main Sources
jessie-updates/contrib Sources
jessie-updates/non-free Sources
jessie-updates/main amd64 Packages
jessie-updates/contrib amd64 Packages
jessie-updates/non-free amd64 Packages
jessie-backports/main Sources
jessie-backports/contrib Sources
jessie-backports/non-free Sources
jessie-backports/main amd64 Packages
jessie-backports/contrib amd64 Packages
jessie-backports/non-free amd64 Packages

apt-get update ends with the usual error:

E: Some index files failed to download. They have been ignored, or old ones used instead.

This seems to be due because Debian removed those upstream, from the topic in #debian:

Oldstable: Debian Jessie, jessie-updates and jessie-backports REMOVED 2019-03-24

A way to keep track of them is to look at the list of hosts in Debmonitor and sort by Last Update (although it might include some additional few "broken" host):
https://debmonitor.wikimedia.org/hosts/

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

Volans created this task.Mar 26 2019, 10:01 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 26 2019, 10:01 PM
Volans renamed this task from jessie-updates and jessie-backports removed by Debian to apt-get update broken on jessie: jessie-updates and jessie-backports removed by Debian.Mar 26 2019, 10:01 PM
Volans triaged this task as High priority.
Paladox added a subscriber: Paladox.Mar 26 2019, 10:03 PM
bd808 edited projects, added cloud-services-team (Kanban); removed cloud-services-team.Mar 26 2019, 10:13 PM
bd808 moved this task from Inbox to Needs discussion on the cloud-services-team (Kanban) board.
• cwdent added a subscriber: • cwdent.Mar 26 2019, 11:09 PM
• cwdent added a subtask: T219280: sort out jessie vs jesse-backports vs openssl pinning issue .Mar 26 2019, 11:17 PM
fgiunchedi added a subscriber: fgiunchedi.Mar 27 2019, 8:59 AM

This also affects package_builder role on boron when trying to build packages for jessie-backports

jbond claimed this task.Mar 27 2019, 9:58 AM
gerritbot added a comment.Mar 27 2019, 11:59 AM

Change 499453 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] jessie-backports: Remove unsued pins

https://gerrit.wikimedia.org/r/499453

gerritbot added a project: Patch-For-Review.Mar 27 2019, 11:59 AM
jbond added a comment.EditedMar 27 2019, 1:23 PM

once the jessie-backport repos has been remove i suggest running the following on cumin

sudo cumin -m async 'F:lsbdistcodename = jessie' 'rm -f $(grep -l "release a=jessie-backports" /etc/apt/preferences.d/*) /etc/apt/sources.list.d/debian-backports.list' 'sed -i '/jessie-updates/d' /etc/apt/sources.list' 'apt-get update'

i have already checked the above sed doesn't remove two much with the following

sudo cumin 'F:lsbdistcodename = jessie' 'grep jessie-updates /etc/apt/sources.list
gerritbot added a comment.Mar 27 2019, 3:09 PM

Change 499505 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] jessie-backports: warn users if the try to use backports on jessie

https://gerrit.wikimedia.org/r/499505

fgiunchedi added a comment.Mar 27 2019, 4:01 PM
In T219333#5061125, @fgiunchedi wrote:

This also affects package_builder role on boron when trying to build packages for jessie-backports

FYI I have worked around this today to be able to build rsyslog using jessie-backports with this hook (/var/cache/pbuilder/hooks/jessie/D02backports) (notice the [check-valid-until=no] and apt command line) and setting http_proxy=http://webproxy.eqiad.wmnet:8080 in the environment.

#!/bin/sh

# Avoid running hooks if backports is not specified
if [ "${BACKPORTS}" = "yes" ]; then
	cat > /etc/apt/sources.list.d/jessie-backports.list <<-'EOF'
  deb [check-valid-until=no] http://archive.debian.org/debian jessie-backports main contrib non-free
  deb-src [check-valid-until=no] http://archive.debian.org/debian jessie-backports main contrib non-free
EOF
	cat > /etc/apt/preferences.d/jessie-backports.pref <<-'EOF'
	Package: *
	Pin: release a=jessie-backports
	Pin-Priority: 500
EOF
  apt-get -o Acquire::Check-Valid-Until=false update
fi
gerritbot added a comment.Mar 28 2019, 11:02 AM

Change 499745 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] jessie-backport: Remove the jessie backport

https://gerrit.wikimedia.org/r/499745

gerritbot added a comment.Mar 28 2019, 11:10 AM

Change 499745 merged by Jbond:
[operations/puppet@production] jessie-backport: Remove the jessie backport

https://gerrit.wikimedia.org/r/499745

gerritbot added a comment.Mar 28 2019, 11:25 AM

Change 499749 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] jessie-backports: remove jessie pinning.

https://gerrit.wikimedia.org/r/499749

gerritbot added a comment.Mar 28 2019, 11:38 AM

Change 499749 merged by Jbond:
[operations/puppet@production] jessie-backports: remove jessie pinning.

https://gerrit.wikimedia.org/r/499749

gerritbot added a comment.Mar 28 2019, 11:39 AM

Change 499751 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] jessie-backports: remove pins files as its redundent

https://gerrit.wikimedia.org/r/499751

gerritbot added a comment.Mar 28 2019, 11:57 AM

Change 499751 merged by Jbond:
[operations/puppet@production] jessie-backports: remove pins files as its redundent

https://gerrit.wikimedia.org/r/499751

gerritbot added a comment.Mar 28 2019, 12:01 PM

Change 499757 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] jessie-backports: remove pins for packages in openstack-mitaka-jessie

https://gerrit.wikimedia.org/r/499757

gerritbot added a comment.Mar 28 2019, 12:21 PM

Change 499757 merged by Jbond:
[operations/puppet@production] jessie-backports: remove pins for packages in openstack-mitaka-jessie

https://gerrit.wikimedia.org/r/499757

gerritbot added a comment.Mar 28 2019, 12:24 PM

Change 499758 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] jessie-backports: add components/ci repository

https://gerrit.wikimedia.org/r/499758

gerritbot added a comment.Mar 28 2019, 2:03 PM

Change 499758 merged by Jbond:
[operations/puppet@production] jessie-backports: add components/ci repository

https://gerrit.wikimedia.org/r/499758

gerritbot added a comment.Mar 28 2019, 2:05 PM

Change 499773 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] jessie-backports: remove pining from packages

https://gerrit.wikimedia.org/r/499773

gerritbot added a comment.Mar 28 2019, 2:15 PM

Change 499773 merged by Jbond:
[operations/puppet@production] jessie-backports: remove pining from packages

https://gerrit.wikimedia.org/r/499773

gerritbot added a comment.Mar 28 2019, 2:20 PM

Change 499778 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] jessie-backports: remove pins for jessie-backports

https://gerrit.wikimedia.org/r/499778

gerritbot added a comment.Mar 28 2019, 2:32 PM

Change 499453 abandoned by Jbond:
jessie-backports: Remove unsued pins

Reason:
created as seperate CR's

https://gerrit.wikimedia.org/r/499453

gerritbot added a comment.Mar 28 2019, 4:09 PM

Change 499803 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] jessie-backports: create new component for kube2proxy

https://gerrit.wikimedia.org/r/499803

gerritbot added a comment.Mar 28 2019, 4:24 PM

Change 499803 merged by Jbond:
[operations/puppet@production] jessie-backports: create new component for kube2proxy

https://gerrit.wikimedia.org/r/499803

gerritbot added a comment.Mar 28 2019, 4:29 PM

Change 499808 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] jessie-backports: remove redundant pins

https://gerrit.wikimedia.org/r/499808

gerritbot added a comment.Mar 28 2019, 4:46 PM

Change 499815 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] jessie-backports: add component/kube2proxy apt repository

https://gerrit.wikimedia.org/r/499815

gerritbot added a comment.Mar 28 2019, 7:50 PM

Change 499815 merged by Jbond:
[operations/puppet@production] jessie-backports: add component/kube2proxy apt repository

https://gerrit.wikimedia.org/r/499815

gerritbot added a comment.Mar 28 2019, 7:55 PM

Change 499778 merged by Jbond:
[operations/puppet@production] jessie-backports: remove pins for jessie-backports

https://gerrit.wikimedia.org/r/499778

MoritzMuehlenhoff mentioned this in T216711: Audit our puppet tree for uses of jessie-backports.Mar 28 2019, 9:09 PM
jijiki mentioned this in T219580: Remove backports from wikimedia-jessie.Mar 29 2019, 12:27 AM
gerritbot added a comment.Mar 29 2019, 1:55 PM

Change 499505 merged by Jbond:
[operations/puppet@production] jessie-backports: warn users if they try to use backports on jessie

https://gerrit.wikimedia.org/r/499505

Andrew moved this task from Needs discussion to Doing on the cloud-services-team (Kanban) board.Apr 2 2019, 4:43 PM
Andrew removed a project: cloud-services-team (Kanban).
jbond closed this task as Resolved.Apr 4 2019, 2:03 PM
gerritbot added a comment.Apr 5 2019, 10:08 AM

Change 499808 merged by Jbond:
[operations/puppet@production] jessie-backports: remove redundant pins

https://gerrit.wikimedia.org/r/499808

Jgreen closed subtask T219280: sort out jessie vs jesse-backports vs openssl pinning issue as Resolved.Apr 16 2019, 3:27 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL