Also, I'm thinking we may want to change the default assignee for OTRS-related requests from Tim to Fred?

bastique.bz wrote:

I've put in a request for this internally, this is a duplicate of that request. I'd like to see it done sooner rather than later, though. :-)

Bump this bug. Version 3 was released in November. Rumors said it's much better and faster ;-)

I've been thinking about testing the upgrade myself on a db snapshot or some subset of data.

Is there sample data available?
Maybe all Junk messages for a month. (I'm thinking it would be ok privacy wise to give out a subset of junk messages to someone that already has access through the web interface)

Also, what version exactly is in use now? a specific tag or date? (web says it's 2.x CVS) Is there a comprehensive list of all local patches? (e.g. 1 click junking)

Also, OTRS already puppetized? Is it a dedicated web server? (I know it's a shared DB)

Seems to be williams.wikimedia.org is OTRS only, and i'd guess it's not puppetised.

http://wikitech.wikimedia.org/view/OTRS

Knock yourself out ;)

(In reply to comment #6)

http://wikitech.wikimedia.org/view/OTRS

Knock yourself out ;)

does anyone know if wikitech is actually up to date wrt OTRS?

I think all of the questions in comment 4 still stand.

It's still on williams. The rest of the page is probably quite ok.

I would too, be interested in that OTRS sample data.

Dumped a couple of tickets in RT for the sample data and the upgrade

(In reply to comment #9)

Dumped a couple of tickets in RT for the sample data and the upgrade

http://rt.wikimedia.org/Ticket/Display.html?id=452
http://rt.wikimedia.org/Ticket/Display.html?id=456

mhershberger wrote:

Unassigning default assignments. http://article.gmane.org/gmane.science.linguistics.wikipedia.technical/54734

(In reply to comment #10)

http://rt.wikimedia.org/Ticket/Display.html?id=452
http://rt.wikimedia.org/Ticket/Display.html?id=456

Jeff Green is slated to work on these, but it may take a while.

I should add that right now Jeff Green is investigating the feasibility of this upgrade. We've gotten an indication that this may be more involved than you would initially think.

me+w wrote:

Hi Guys,

my name is Martin Edenhofer (I'm the Inventor of OTRS).

I would offer some assistance for OTRS upgrading for free (personal giving from
me to wikimedia).

Just drop me a note.

-Martin

(In reply to comment #14)

Hi Guys,

my name is Martin Edenhofer (I'm the Inventor of OTRS).

I would offer some assistance for OTRS upgrading for free (personal giving from
me to wikimedia).

Just drop me a note.

-Martin

Hi Martin,
Thank you very much for the offer of some help on behalf of the Wikimedia Foundation.

I'm going to add you onto one of the rt operations tickets, and hopefully we can take it from there with our operations staff.

Thanks

Sam

Just wondering if there is any update to this. Additionally, what version are we (at this point) talking about upgrading to? This bug was opened some time ago and requesting 2.4.7 ... I believe OTRS is up to 3.1.

(In reply to comment #16)

Just wondering if there is any update to this. Additionally, what version are
we (at this point) talking about upgrading to? This bug was opened some time
ago and requesting 2.4.7 ... I believe OTRS is up to 3.1.

The RT was logged as v3. I'm not sure how long 3.1 has been out for, but there's little point going part of the way I guess (though, it might have to go via 3.0 as an intermediary).

It's currently blocked on some internal legal related WMF processes.

No, it's not blocked on legal. We've done everything we needed to. We're waiting on the NDA to be returned.

(In reply to comment #18)

No, it's not blocked on legal. We've done everything we needed to. We're
waiting on the NDA to be returned.

Oh. No-one updated the RT ticket. I'll do that now

Not to rush, nag or anything like that -- is this still progressing, Philippe?

Thehelpfulonewiki wrote:

As far as I am aware, the current situation is that we are still waiting for Martin to return the NDA before he can help with the upgrade.

Correct. The NDA was sent to him, and I'll prod him again, just to be sure. Just waiting to hear.

3.1.8 is the latest version available.

Hello everyone,

I understand it can be an important task to migrate from 2.4.x to 3.x, but are we at least using the last version of the 2.4 branch (2.4.14 right now)? Because the version number displayed (2.4.x CVS) does not give much information and there are serious security issues with older versions (see CVE-2012-4600).

Thank you and best regards,

Thehelpfulonewiki wrote:

Thanks for this - the last updates to OTRS can be seen https://svn.wikimedia.org/viewvc/mediawiki/trunk/otrs/README.wikimedia?view=log

The last CVS update and refresh was:

Modified Mon Feb 2 06:30:25 2009 UTC (3 years, 6 months ago) by tstarling

See https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2012-4600.

Just as an update, we still do not have an NDA on Martin. Until we do, there's no way for us to proceed with this, unfortunately, as there are currently no resources allocated toward this on tech's road map.

(In reply to comment #26)

Just as an update, we still do not have an NDA on Martin. Until we do, there's
no way for us to proceed with this, unfortunately, as there are currently no
resources allocated toward this on tech's road map.

Do you believe this situation is acceptable? What is the plan going forward?

me wrote:

Update, I resent the NDA to Philippe now.

To close the security issue on OTRS 2.4, you can do the following:

a) Disable rich text via sysconfig

or:

b) install the following package via Admin -> Package Mananager

https://github.com/downloads/znuny/Znuny4OTRS-CVE-2012-2582/Znuny4OTRS-CVE-2012-2582-1.2.3.opm
and replace the following files:
o Kernel/Modules/CustomerTicketAttachment.pm with version 1.17.2.7
   http://source.otrs.org/viewvc.cgi/otrs/Kernel/Modules/CustomerTicketAttachment.pm?revision=1.17.2.7&view=co
o Kernel/Modules/AgentTicketAttachment.pm with version 1.22.2.7
   http://source.otrs.org/viewvc.cgi/otrs/Kernel/Modules/AgentTicketAttachment.pm?revision=1.22.2.7&view=co

How can we proceed with the OTRS 3.1 upgrade? Who is the contact person?

PS: In case you need more details to the security issues, see http://znuny.com/#!/advisory/ZSA-2012-01 and http://znuny.com/#!/advisory/ZSA-2012-02

(In reply to comment #28)

Update, I resent the NDA to Philippe now.

Tell "us" (probably better on the RT ticket) what you need; A copy of our current otrs database and a copy of the OTRS files as current is?

From there, I suppose you can do a test upgrade locally, and see if you can create updates for our "patches" that are in SVN.

Next, a test install on the WMF cluster, db upgrade etc. Probably some testing on our side with our current OTRS users

And after that, schedule some maintenance/downtime for our current OTRS install, update files, update the database and make it live again.

I know Ops were keen to get this also puppeti[sz]ed, but I don't think this is a blocker from the point of upgrading.

You will need an ops point of contact too, I believe this was going to be Jeff Green originally, however we're getting close to fundraising season, so I'm not sure how much time he would have available. Daniel (among others) seem reasonably comfortable working with Perl for Bugzilla. This would need to be discussed with CT first.

For the record; confirming the receipt of Martin's NDA (thank you!)

FYI, there is a serious security issue with 2.4.14 as well (see CVE-2012-4751).

Best regards,

Indeed «Today 3.1.11 has been released with a few bugfixes and one security fix for a XSS vulnerability on viewing special prepared HTML e-mails, which leads to that the browser executes JavaScript code (as described in CVE-2012-4751 and OTRS security announcement OSA-2012-03).»
http://www.linux-dev.org/2012/10/package-updates-from-the-middle-of-october/

me wrote:

JFI, I still stay tuned to support you for upgrading.

Is there a responsible OTRS admin accessible?

-Martin

If you'll let me know what you need from an OTRS admin, I'll ping one. :-)

Also, we should discuss whether we want any changes to configuration once it's upgraded, and test those out on the new install.

Martin, I'm one of the active OTRS admins (I've been following this bug - I created it years ago). Perhaps you want somebody in ops? What specifically might you need? I'm happy to help in any way possible.

Copying Erik and CT on this. This issue is becoming more urgent (mostly as this bug has become long in the tooth) and there's a volunteer willing to help out here. Anything the two of you could do to move this forward would be great.

According to RT #452 setting up infrastructure for Martin is work in progress.

RT tickets states that this first requires setting up a new VLAN in eqiad for an OTRS developer.

[not "blocking general development" => resetting severity]

(In reply to comment #39)

[not "blocking general development" => resetting severity]

I disagree and I think https://bugzilla.wikimedia.org/buglist.cgi?resolution=---&query_format=advanced&component=OTRS&product=Wikimedia easily proves it (even if many issues are not tracked).
Please let the relevant manager (Philippe? Erik? Geoff?) confirm or disconfirm it.

closing the linked RT ticket. This should continue here and be assigned to Martin Edenhofer.

I have a few updates on the server assignment for this upgrade. I spun up and assigned a misc server 'colby' for this task, but now am aware that a db class machine is needed for testing.

As such, colby will be shut down and wiped, and server 'db61' will be assigned to this task. I'll be reinstalling db62 for this task, and placing it in the development sandbox (as colby was.)

I emailed Martin off-ticket with the login info for Colby, but after two emails I have not heard back. Once this system is installed, I'll need Martin's ssh public key, which is not presently in any of our systems that I can find.

db61 is now online and in the sandbox vlan, ready for Martin to use for development work on OTRS.

Jeff Green and myself have our keys in the root auth key file, so we can login and setup Martin's key once we get it.

Martin: We need a copy of your public ssh key so we can place it in the authorized keys file for this system.

Thehelpfulonewiki wrote:

Martin: Did you manage to send your SSH key through to Rob and Jeff?

(In reply to comment #41)

This should continue here and be assigned to Martin Edenhofer.

I'm not sure if Martin has an account in Wikimedia Bugzilla, so not sure either if asking him here about his SSH key makes sense.

(In reply to comment #45)

(In reply to comment #41)

This should continue here and be assigned to Martin Edenhofer.

I'm not sure if Martin has an account in Wikimedia Bugzilla, so not sure
either
if asking him here about his SSH key makes sense.

He has. See comment #33 and other.

(In reply to comment #40)

I disagree and I think
https://bugzilla.wikimedia.org/buglist.cgi?resolution=---
&query_format=advanced&component=OTRS&product=Wikimedia
easily proves it

Nemo:
I've just retriaged all open OTRS tickets in Bugzilla.
Out of 15 open tickets (some of them unresolved upstream issues, two local configuration issues) there are 2 known problems (see "Blocks" field) that would get solved by upgrading.
I don't think this justifies "blocker" severity ("blocks development work", see http://www.mediawiki.org/wiki/Bugzilla/Fields ) yet, so I propose resetting this to "major".
This does not mean that we would not vastly benefit from an OTRS update to make life of our community in OTRS easier.

(even if many issues are not tracked).

If there are more unreported issues with OTRS they are extremely welcome to be reported in Bugzilla (bonus points for finding out if an OTRS software upgrade would influence them and marking them as depending on bug 22622 - upstream bugtracker is at http://bugs.otrs.org/ for queries). And if there are further good reasons why an OTRS upgrade blocks development work, I kindly request to bring them up in Bugzilla.
Both helps everybody to get a better impression on how severe and urgent the OTRS upgrade request is (how many and which issues would get solved by it), and would lead to reevaluating this request.

Again: I don't see this as a "blocker" based on what's known.

Has the javascript injection been fixed? (cf comment 32)

That seems pretty critical to me.

Thanks for your triaging.

(In reply to comment #47)

If there are more unreported issues with OTRS they are extremely welcome to
be
reported in Bugzilla (bonus points for finding out if an OTRS software
upgrade
would influence them and marking them as depending on bug 22622 - upstream
bugtracker is at http://bugs.otrs.org/ for queries). And if there are further
good reasons why an OTRS upgrade blocks development work, I kindly request to
bring them up in Bugzilla.

Would you ask someone to triage, test, assess and file any problem they have with MediaWiki on a MediaWiki 1.15 instance, and to verify if they're fixed in MediaWiki 1.20 (and then at every release: 1.21, 1.22 and so on for a few years), based on sparse documentation?
If someone does so, high praise is due, but I doubt any volunteer is going to do it. I've never seen any FLOSS project spending time debugging 3-4 years old releases.

me wrote:

Hi Guys,

I see there is some action on this bug. I still stay tuned to support the upgrading to OTRS 3.

Who is the guy which is responsible for this task/project?

Who is the OTRS owner at WMF?

-Martin

(In reply to comment #50)

Who is the guy which is responsible for this task/project?

I think not really anyone at this point. Except maybe Martin. :)

Who is the OTRS owner at WMF?

I guess Philippe? On the ops side there's not a specific owner AFAIK.

Was there a specific question you had? What is the status on your end?

Any public questions/comments can go directly on this bug here. (or for a faster response you could try #wikimedia-operations on Freenode and if that's unsuccessful then come comment here)

If there's something that should not be public for some reason then you can send mail to 452@rt.wikimedia.org

(of course there are some questions that may end up getting redirected to other places like the OTRS users with access to the admin web interface. but the last 2 paragraphs should cover the bulk of cases)

Thanks!

bugs wrote:

Martin: see comment 43. Jeff Green and Rob Halsell are working on this on the WMF side. They said they need your SSH key before they can move forward, though.

(In reply to comment #48)

Has the javascript injection been fixed? (cf comment 32)

That seems pretty critical to me.

I concur with this: remember that there's a lot of *confidential* information on OTRS. It's not acceptable to have almost every single user account hijackable, no matter what rights they have, with a single email using an exploit that is easily available and ready to use on the Internet.

Best regards,

me wrote:

(In reply to comment #53)

(In reply to comment #48)

Has the javascript injection been fixed? (cf comment 32)

That seems pretty critical to me.

I concur with this: remember that there's a lot of *confidential* information
on OTRS. It's not acceptable to have almost every single user account
hijackable, no matter what rights they have, with a single email using an
exploit that is easily available and ready to use on the Internet.

JFI: Here would be a hot fix.

https://bugzilla.wikimedia.org/show_bug.cgi?id=22622#c28

PS: If you have "rich text" disabled, you are save. Do you use currently "rich text" in your system?

-Martin

Martin,

Since you're all NDA'd, I just created you an account for our OTRS installation. You should have the password in your email; if not, email me at philippe@wikimedia.org and I'll get it reset. That should let you check anything like this.

(In reply to comment #54)

(In reply to comment #53)

(In reply to comment #48)

Has the javascript injection been fixed? (cf comment 32)

That seems pretty critical to me.

I concur with this: remember that there's a lot of *confidential* information
on OTRS. It's not acceptable to have almost every single user account
hijackable, no matter what rights they have, with a single email using an
exploit that is easily available and ready to use on the Internet.

JFI: Here would be a hot fix.

https://bugzilla.wikimedia.org/show_bug.cgi?id=22622#c28

Thanks Martin.

PS: If you have "rich text" disabled, you are save. Do you use currently
"rich text" in your system?

It seems we don't. Does it mean that we are immune against any security issue involving XSS like CVE-2012-4600 and CVE-2012-4751 ?

Thanks again,

me wrote:

(In reply to comment #56)

PS: If you have "rich text" disabled, you are save. Do you use currently
"rich text" in your system?

It seems we don't. Does it mean that we are immune against any security issue
involving XSS like CVE-2012-4600 and CVE-2012-4751 ?

Exactly. If "rich text is disable" you "do not have a security issue" regarding the XSS stuff like CVE-2012-2582/CVE-2012-4600/CVE-2012-4751.

Have a nice day! :)

(In reply to comment #57)

(In reply to comment #56)

PS: If you have "rich text" disabled, you are save. Do you use currently
"rich text" in your system?

It seems we don't. Does it mean that we are immune against any security issue
involving XSS like CVE-2012-4600 and CVE-2012-4751 ?

Exactly. If "rich text is disable" you "do not have a security issue"
regarding
the XSS stuff like CVE-2012-2582/CVE-2012-4600/CVE-2012-4751.

Now, that's really good news.

Have a nice day! :)

Thanks, you too! :)

sumanah wrote:

Martin, as you work, it would be great if you could also update our wikitech.wikimedia.org page to correct anything that's obsolete: https://wikitech.wikimedia.org/view/OTRS Thanks!

(In reply to comment #59)

Martin, as you work, it would be great if you could also update our
wikitech.wikimedia.org page to correct anything that's obsolete:
https://wikitech.wikimedia.org/view/OTRS Thanks!

Assuming he doesn't already have a wikitech account, he'll need to be given one. Account creation is restricted on that wiki.

Thehelpfulonewiki wrote:

(In reply to comment #60)

(In reply to comment #59)

Martin, as you work, it would be great if you could also update our
wikitech.wikimedia.org page to correct anything that's obsolete:
https://wikitech.wikimedia.org/view/OTRS Thanks!

Assuming he doesn't already have a wikitech account, he'll need to be given
one. Account creation is restricted on that wiki.

I chased this up with CT to get this request fast tracked (sometimes these requests can take a couple of weeks to process). He's created User:Martin on wikitech and is sending password details directly to Martin.

(In reply to comment #61)

I chased this up with CT to get this request fast tracked (sometimes these
requests can take a couple of weeks to process). He's created User:Martin on
wikitech and is sending password details directly to Martin.

Hmmm, I don't see an account:

• https://wikitech.wikimedia.org/wiki/Special:ListUsers/Martin

• https://wikitech.wikimedia.org/wiki/User:Martin

Is there a status update about this bug? Has Martin been able to assess the current database and estimate when we can upgrade?

Thehelpfulonewiki wrote:

(In reply to comment #62)

(In reply to comment #61)

I chased this up with CT to get this request fast tracked (sometimes these
requests can take a couple of weeks to process). He's created User:Martin on
wikitech and is sending password details directly to Martin.

Hmmm, I don't see an account:

• https://wikitech.wikimedia.org/wiki/Special:ListUsers/Martin

• https://wikitech.wikimedia.org/wiki/User:Martin

Is there a status update about this bug? Has Martin been able to assess the
current database and estimate when we can upgrade?

Yeah a couple of people have brought this up so a full explanation for anyone else, with extra background info:

Wikitech is a wiki that is outside of the usual Wikipedia cluster so that if the site goes down, tech staff still have that wiki up to be able to access technical documentation etc. The problem with this is that the old wikitech wiki was rarely updated or maintained (it's on MediaWiki 1.19.2 right now), and it also had restricted editing.

With the introduction of Labs, ops decided that it would be good to merge the labs wiki and wikitech together, and so after a long time waiting they finally merged the two wikis at the beginning of this month.

The old wiki is still accessible at http://wikitech-old.wikimedia.org/index.php?title=Special%3ALog&type=newusers&user=&page=&year=&month=-1 but I believe it's read only now, as we should be using wikitech.wikimedia.org.

So for Martin's account on the new wiki, I think he will already have an account if he has a Labs/Gerrit/Git account, else I think he should be able to create one himself (the new wiki is open account creation, but not integrated with SUL).

He's not in the "OTRS" project on Labs/wikitech though, https://wikitech.wikimedia.org/wiki/Nova_Resource:Otrs, so I don't know if he's been able to do any testing.

(In reply to comment #63)

He's not in the "OTRS" project on Labs/wikitech though,
https://wikitech.wikimedia.org/wiki/Nova_Resource:Otrs, so I don't know if
he's been able to do any testing.

That's not an immediate concern. He can't do any testing with prod data in labs anyway. (no NDA data in labs)

you don't need to worry about the account setup and if we got his key. quoting RobH: "Your key is in the root's authorized key file, so you can ssh in directly as root. You can then setup a user for yourself or whatever else you need on that server to get things going."

Any progress on this yet Martin? :)

me wrote:

(In reply to comment #66)

Any progress on this yet Martin? :)

JFI: Test-System is upgraded. Waiting for feedback of OTRS-Admins.

(In reply to comment #67)

(In reply to comment #66)

Any progress on this yet Martin? :)

JFI: Test-System is upgraded. Waiting for feedback of OTRS-Admins.

To update further: The admins have had access to the system for quite some time now, and have been testing/writing documentation/getting things re-"fixed", etc. Currently, we are waiting for another round of "fixes" to the test install - from there, we'll see.

Hopefully we're getting close here.

Martin - If anything else is needed from the OTRS admins, feel free to post here or add a note to the document you have been accessing and working from. Thanks for your help so far.

To be precise: Martin, we've added some new notes regarding open issues to that Google Docs document. Vielen Dank für deine Hilfe!

As an update: The OTRS team (the people who created the software) continue to work on our test install of the 3.2.3 version. The most recent updates to said install (that I know of) were done on July 4.

There are a couple more bugs to work out on the test install ... and then we'll see what the next step is to actually upgrading.

The upgrade to OTRS version 3.2 is scheduled for 10:00 UTC on Tuesday, 6 August, and is expect to take up to eight hours to complete.

The upgrade is to 3.2.9 and is in progress.

Quoting Ryan's (Rjd0060) message here:

"As you may have noticed, the ticket system (https://ticket.wikimedia.org/otrs/index.pl) is back up, with a new and improved look! You may find some "bugs" as the upgrade process is *still ongoing* as we finish customizing OTRS 3.2.9 to make it as familiar as it was before.

We have set up a page on OTRS wiki at https://otrs-wiki.wikimedia.org/wiki/Upgrade_issues. There, you will find the most up-to-date information that we have on the upgrade process, as well as known issues/bugs that are being addressed. We will continue to update that page as changes are made and new information becomes available. If you happen to notice any other issues that aren't listed, feel free to leave a note on the talk page."

As the upgrade to 3.2.9 has happened, closing this bug report as FIXED. \o/
Big big thanks to everybody who was involved in getting this done, plus for everybody's patience.