Page MenuHomePhabricator

Decommission svn.wikimedia.org server (import SVN into Phabricator)
Closed, ResolvedPublic

Description

It's been around long enough, should put the svn service itself out to pasture somehow.

Update:
The SVN repo should be rolled into phabricator and then the old services shut down. SVN will only need to be accessible over web.

Event Timeline

yuvipanda raised the priority of this task from to Needs Triage.
yuvipanda triaged this task as Low priority.
yuvipanda updated the task description. (Show Details)
yuvipanda added a project: acl*sre-team.
yuvipanda added a subscriber: demon.
yuvipanda added a subscriber: yuvipanda.

Would that mean that extension codebases in SVN that have not been migrated to Git would be lost and links to codebases on extension homepages on mediawiki.org would break? If so, any idea how many (ancient and unmaintained) extensions would actually be affected by that?

@Aklapper: No, they would probably be migrated to phabricator (or some other such tool) in a readonly fashion.

I think you just identified a separate task that is blocking this one? ;)

I still semi-regularly (every few months) use viewvc to figure out history of files for pywikibot, as not all branches were converted during the git migration -- mainly because it was too complicated to do this (entire branches being moved around and later deleted in svn).

There's definitely stuff in there that hasn't been migrated yet indeed. I referred to it just today as well.

For that reason I'm inclined to import SVN repos to Phab so they can be usable from here. The history is important.

we have 17 days left until the SSL cert for svn.wm.org expires

https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=antimony&service=HTTPS

until then we need to either buy a new cert, or resolve this and decom it completely, or, what i have been thinking before:

remove or just firewall the actual svn protocol part and keep ViewVC but move that behind misc-web to not have the cert issue anymore because there the regular *.wikimedia.org cert would match

Dzahn raised the priority of this task from Low to High.Jan 14 2015, 2:44 AM

Chad: If it is imported to Phab, will it cause an issue requiring odd ports or will it simply present on web ports?

In T86655#990319, @RobH wrote:

Chad: If it is imported to Phab, will it cause an issue requiring odd ports or will it simply present on web ports?

It should just be the web ports, nothing special. We won't be serving it over SSH anymore.

importing into phab would be a great solution as well. then we should also get rid of viewvc completely and can delete the entire puppet role

So then I suppose this task should be modified to import SVN into phab?

RobH renamed this task from Decomission svn.wikimedia.org to Decomission (server) svn.wikimedia.org - by importing svn into phab.Jan 26 2015, 11:48 PM
RobH updated the task description. (Show Details)
RobH set Security to None.

@Chad,

Would you be the person to import this in, or should an ops person take point on this?

MZMcBride renamed this task from Decomission (server) svn.wikimedia.org - by importing svn into phab to Decommission svn.wikimedia.org server (import SVN into Phabricator).Jan 27 2015, 12:36 AM
In T86655#995479, @RobH wrote:

@Chad,

Would you be the person to import this in, or should an ops person take point on this?

I'm working on it, as soon as we verify the fix from upstream is good. See T87282.

mark lowered the priority of this task from High to Medium.Jan 30 2015, 2:57 PM

We're making this a High priority just because the HTTPS cert expires? I don't know, do we care that much for a service essentially fallen in disuse? Sure, let's migrate it but perhaps it doesn't need to be that urgent.

FWIW I cherry-picked the relevant upstream commit on phab-01 for my man @Chad to test if the SVN issue is actually resolved. This week being crazy I think that still needs to be done? If it isn't resolved then we have to figure out what's going on, but either way I don't see us having this all wrapped up by tomorrow when the cert expires.

The certificate has now expired; the alert remains CRITICAL in Icinga, as it has been for 60 days now. My opinion is that if it is not of a high priority, that check should not exist (or at minimum should not be at a CRITICAL priority). We shouldn't just be used to having certificate alerts (or any alerts for that matter) staying in that state for a prolonged time period — it just trains everyone to ignore alerts.

I also do think that instead of wasting so many man hours we could just pay the minimal one-off fee for 1 year and defer this discussion for a year when we're possibly going to have code review & git in Phabricator anyway and an SVN migration might be easier at that point.

what Faidon said, just pay for it. but let's not keep ignoring the icinga check (or remove it if we don't consider it critical)

Does this include importing CodeReview comments into Audit?
Can all viewvc URLs be safely redirected?
Can the SVN imports be clearly separated from non-archived repos?
Will checkout still work?
What's the benefit of the import compared to the costs?

Does this include importing CodeReview comments into Audit?

No. But worth discussing.

Can all viewvc URLs be safely redirected?

Sure why not?

Can the SVN imports be clearly separated from non-archived repos?

Phabricator repos that are "old" are sorted to the bottom by default. Same thing with Git.

Will checkout still work?

We can make it work :)

What's the benefit of the import compared to the costs?

Not maintaining a one-off service for a thing we don't use actively is kind of a burden. Getting things into one place is nice :)

now we only need to have consenus that the actual cloning with the svn protocol can be disabled

now we only need to have consenus that the actual cloning with the svn protocol can be disabled

-1

Cool! Did you/could you also import the other repositories (most importantly for me, the pywikipedia repo)?

merged https://gerrit.wikimedia.org/r/#/c/219228/

tested on mw1033:

[terbium:~] $ apache-fast-test svn.urls mw1033
testing 10 urls on 1 servers, totalling 10 requests
spawning threads..

http://svn.mediawiki.org
 * 301 Moved Permanently https://phabricator.wikimedia.org/diffusion/
http://svn.mediawiki.org/doc
 * 301 Moved Permanently https://doc.wikimedia.org/mediawiki-core/master/php/
http://svn.mediawiki.org/viewvc/mediawiki
 * 301 Moved Permanently https://phabricator.wikimedia.org/diffusion/SVN/
http://svn.mediawiki.org/viewvc/mysql
 * 301 Moved Permanently https://phabricator.wikimedia.org/diffusion/SVNM/
http://svn.mediawiki.org/viewvc/pywikipedia
 * 301 Moved Permanently https://phabricator.wikimedia.org/diffusion/SVNP/
http://svn.wikimedia.org
 * 301 Moved Permanently https://phabricator.wikimedia.org/diffusion/
http://svn.wikimedia.org/doc
 * 301 Moved Permanently https://doc.wikimedia.org/mediawiki-core/master/php/
http://svn.wikimedia.org/viewvc/mediawiki
 * 301 Moved Permanently https://phabricator.wikimedia.org/diffusion/SVN/
http://svn.wikimedia.org/viewvc/mysql
 * 301 Moved Permanently https://phabricator.wikimedia.org/diffusion/SVNM/
http://svn.wikimedia.org/viewvc/pywikipedia
 * 301 Moved Permanently https://phabricator.wikimedia.org/diffusion/SVNP/

Cool! Did you/could you also import the other repositories (most importantly for me, the pywikipedia repo)?

I didn't, Chad did, here it is: https://phabricator.wikimedia.org/diffusion/SVNP/

Change 219234 had a related patch set uploaded (by Dzahn):
Point svn.wikimedia.org at text-lb

https://gerrit.wikimedia.org/r/219234

Change 219234 merged by Dzahn:
Point svn.wikimedia.org at text-lb

https://gerrit.wikimedia.org/r/219234

Change 219240 had a related patch set uploaded (by Dzahn):
Remove subversion server support

https://gerrit.wikimedia.org/r/219240

Change 219240 merged by Dzahn:
Remove subversion server support

https://gerrit.wikimedia.org/r/219240

Change 220967 had a related patch set uploaded (by Dzahn):
svn: delete svn.wm.org SSL cert

https://gerrit.wikimedia.org/r/220967

Change 220967 merged by Dzahn:
svn: delete svn.wm.org SSL cert

https://gerrit.wikimedia.org/r/220967

deleted monitor for svn from watchmouse /nimsoft

deleted cert and key from antimony

The recent changes seem to have broken even Phabricator's Subversion viewer:

Unhandled Exception ("CommandException")
Command failed with error #1!
COMMAND
svn --non-interactive --no-auth-cache --trust-server-cert cat 'http://svn.wikimedia.org/svnroot/mediawiki/GOODBYE@115794'

STDOUT
(empty)

STDERR
svn: E175013: Unable to connect to a repository at URL 'http://svn.wikimedia.org/svnroot/mediawiki/GOODBYE'
svn: E175013: Access to '/svnroot/mediawiki/GOODBYE' forbidden

Swapped them to locally hosted, which is better since it's not trying to hit svn.wm.o. Similar failure though.

Had to import the full history from the dumps since they're now locally hosted (duh?!). Pywikipedia and Mysql repos imported and all working just fine.

MediaWiki repo will take the better part of 6 more hours probably to finish but you can already see it working in eg: rSVN863.