Page MenuHomePhabricator

((OTRS)) Community Edition 6 is end-of-life; no FOSS replacement provided
Closed, ResolvedPublic

Description

As of December 23, 2020, OTRS AG released a security advisory declaring that OTRS 6, including ((OTRS)) Community Edition 6 (the version run on ticket.wikimedia.org), is end of life. No further security updates will be provided.

Previous statements from OTRS AG implied that ((OTRS)) Community Edition 7 would be released when OTRS 8 was released, but this has not happened. OTRS AG has also archived https://github.com/OTRS/otrs and most of their other public GPL repos.

Potential solutions:

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Here are some of the alternatives I found with some quick research:

I'm particularly attracted to zammad. It's run by a nonprofit (https://zammad-foundation.org/), so not as much risk as being discontinued like OTRS, and the user interface looks a lot cleaner (see https://zammad.org/screenshots). The interface is a lot different from OTRS, but if we were to switch, we might as well make a big jump. It has all the staples of OTRS, such as queues, templates, etc., and we could even experiment with some of the other support platforms it allows. With integration of twitter/instagram/facebook (converting direct messages into tickets), we could communicate with people over those platforms. Bonus: It allows OTRS migration! See: https://docs.zammad.org/en/latest/migration/otrs.html

osTicket is a more conservative alternative. The user interface is quite similar to OTRS. It's owned by a commercial entity so who knows how long it will be maintained for. It does seem to be slightly more well-established than zammad though.

Those two options seem to be the most popular. The others seem to be not as popular nor as feature rich.

My general suggestion is not to only look at FOSS, but to look at what would fit our needs best. That could be something that’s open source, but it might be something else, and I don’t think we should limit ourselves.

My general suggestion is not to only look at FOSS, but to look at what would fit our needs best. That could be something that’s open source, but it might be something else, and I don’t think we should limit ourselves.

I disagree with this because non-free (in terms of FOSS, not price) solutions are essentially timed lock-ins and because it goes against the ethos of what Wikimedia is all about. None of the Wikimedia-related stack works on non-free software (notably the Foundation site uses a paid subscription, but is still technically free software), I don't think it's time to change that. I do agree that this is an opportunity to switch to something more user-friendly, so perhaps in a year or two, it'd be a good time to make the switch and the Wikimedia team handling this will also need time to carve a suitable roadmap going forward (for e.g. see GitLab roadmap).

My general suggestion is not to only look at FOSS, but to look at what would fit our needs best. That could be something that’s open source, but it might be something else, and I don’t think we should limit ourselves.

Just to be clear, this is a non-starter. We don't deploy non-free software. This is covered in https://foundation.wikimedia.org/wiki/Resolution:Wikimedia_Foundation_Guiding_Principles#Freedom_and_open_source and not really worth rehashing IMO.

But that doesn't preclude us from paying for/sponsoring feature development in free software so it fits our needs. Given how valuable OTRS is to the movement, it seems like a worthy investment.

My general suggestion is not to only look at FOSS, but to look at what would fit our needs best. That could be something that’s open source, but it might be something else, and I don’t think we should limit ourselves.

Just to be clear, this is a non-starter. We don't deploy non-free software. This is covered in https://foundation.wikimedia.org/wiki/Resolution:Wikimedia_Foundation_Guiding_Principles#Freedom_and_open_source and not really worth rehashing IMO.

But that doesn't preclude us from paying for/sponsoring feature development in free software so it fits our needs. Given how valuable OTRS is to the movement, it seems like a worthy investment.

There’s some wiggle room in that. If you make the argument that no free software can adequately meet our needs you can use non-free software. I think the Foundation and en.wiki ArbCom use or at one point used Google Groups for incoming email correspondence.

Looking at what T&S uses already to see if it’s a good option or affordable is where I would start— whether it is free or non-free. I’m not tied to the non-free idea. I just suspect we’d be writing off some of the better solutions if we categorically excluded them.

Edit: asked around and it appears they use Zen Desk, though might want to confirm. Using a consistent system might have benefits, though.

@TonyBallioni's original comment before deletion:

Is there any non-open source proprietary software that will function well and we wouldn’t be picking for ideological reasons?

I’d strongly support one of those.

My general suggestion is not to only look at FOSS, but to look at what would fit our needs best. That could be something that’s open source, but it might be something else, and I don’t think we should limit ourselves.

Just to be clear, this is a non-starter. We don't deploy non-free software. This is covered in https://foundation.wikimedia.org/wiki/Resolution:Wikimedia_Foundation_Guiding_Principles#Freedom_and_open_source and not really worth rehashing IMO.

But that doesn't preclude us from paying for/sponsoring feature development in free software so it fits our needs. Given how valuable OTRS is to the movement, it seems like a worthy investment.

There’s some wiggle room in that. If you make the argument that no free software can adequately meet our needs you can use non-free software.

That's likely going to be a tough argument to make in the case of an OTRS replacement.

It may be worth baring in mind that trying to find loopholes in this way may indicate that the spirit of the resolution may not be being followed.

Here are some of the alternatives I found with some quick research:
[...]

I think wikimedia may still have a zombie instance of that kicking around somewhere for some reason.

It may be worth baring in mind that trying to find loopholes in this way may indicate that the spirit of the resolution may not be being followed.

My point was that there has already been a determination made that non-FOSS software was acceptable for email ticket and email list systems within multiple areas of Wikimedia. en.wiki ArbCom uses Google Groups that was provided by the WMF, and as I mentioned above, the impression multiple people have is that the WMF uses ZenDesk already.

If we’re already using what’s one of the industry standards, it’s worth investigating. Anyway, I’m not going to say anything more on this, but I think raising it so others are aware that it’s an option we can look at is worthwhile.

[...]

I think wikimedia may still have a zombie instance of that kicking around somewhere for some reason.

Yes, there is a read-only version still running and maintained(?) by SRE as not all in queues were transferred over to Phab from memory. T38: Migrate RT to Phabricator has some info on it.

«As an organization, we strive to use open source tools over proprietary ones, although we use proprietary or closed tools (such as software, operating systems, etc.) where there is currently no open-source tool that will effectively meet our needs.»
This is from here: https://foundation.wikimedia.org/wiki/Resolution:Wikimedia_Foundation_Guiding_Principles#Freedom_and_open_source
and I believe it correctly describes how we should address a choice made by the Movement. There are open source alternatives providing what we were using now, we are not in a case of "no open-source tool that will effectively meet our needs". So no need to pay for the "commercial" OTRS. Let's decide whether the current OTRS should be replaced by a *better* FOSS, or if we are fine with the current one. In the latter (easier, quicker) case, let's choose a good fork to which we'd better think if we can offer help or other support (localizations or whatever useful for them).

(Besides, time should have come, now, to slow down a bit with all this use of non-free goods and services - I understand it was a strange year, the last one, but I haven't seen an online meeting, a counted one, on a free platform: let's bring back home our good ol' habits)

(I just want to state the obvious that in case some replacement is provided all thousands of links to the existing tickets as well as all the ticket content both body and attachments have to be preserved. This is especially essential for permissions and photosubmissions queues for which that is kind of the whole point.)

(I just want to state the obvious that in case some replacement is provided all thousands of links to the existing tickets as well as all the ticket content both body and attachments have to be preserved. This is especially essential for permissions and photosubmissions queues for which that is kind of the whole point.)

Wouldn't it be easy to bulk import all the content on a fork?

The transition to an OTRS fork would be much the same as any other OTRS upgrade. They're time-consuming, especially when database upgrades are concerned, and have some pitfalls, but they're not too difficult.

Migrating to non-OTRS software would likely require a significant engineering effort to import history, especially if the new software doesn't already have a good way to import OTRS data. The OTRS database has an absolutely huge amount of data T138915: OTRS database is "too large", which is important to underscore.

@akosiaris did the most recent major OTRS upgrade (T187984), they may be able to provide more information on the feasibility of various solutions.

As a sidenote: I have checked a lot of alternatives in the past and have had a second round when otrs ag pulled the plug but found no real replacement. Some offers were dead simple : way too simple for real use. Some were tailored for very specific needs (usually tied with computer code management). Some had horrible source code (which, when mixed with php usually leads to really bad things). RT got lot of facelift in the recent years but I am still not convinced of that UI.
Generally I see two problems:

  1. The system shall be a problem ticketing system with multiple agent and queues, handling mainly email. This is not the case in many systems.
  2. The UI shall be good (eg. easy to use, functional, fast). I don't really recall many of those.

I support opening a wiki page somewhere and start collecting alternatives, with short (linked to detailed) analysis about pros and cons, but it's a lot of work and bit of matter of taste. (Like, I try to avoid php whenever I can, while many of you obviously does not have this aversion.)
I am very much against a closed or SaaS solution, had pretty bad experience with companies using zendesk (response quality went down the drain while the used needed more and more external systems to login into); and closed source equals vendor lockin. Otrs database is a known format, it is not particularly hard to translate in any form, and it is not very hard to create any translation from old URLs to new ones (provided the tickets are the same). It is work, but can be done in foreseeable timeframe.
Yet, "upgrading" OTRS community edition to any fork is the same as OTRS upgrade.

(Database is big since ticket attachments are stored in db instead of the filesystem storage. It can be converted, making the db way smaller and use the same space from plain filesystem storage.)

FWIW, Znuny has helped us before, in 2013 (T24622). Znuny was founded by Martin Edenhofer, the original author of OTRS.

Our OTRS install was stuck on something like 2.4 for years, and OTRS AG wanted to charge the WMF a significant sum of money to assist us in upgrading to (IIRC) 3.2 as it was a major upgrade. Znuny stepped in to help us for free, providing an engineer to work with an engineer from the WMF to run the upgrade for us through the course of a weekend and get us setup for easy, future upgrades (ref: WMF blog post on the upgrade).

I think one of the reasons Znuny offered to help was because OTRS AG was unwilling to do anything at all for us; I got the sense the broader OTRS community isn't too happy with how OTRS AG has gone and this decision shouldn't be too surprising to the OTRS community. We should be able to find help elsewhere from good partners is the point I'm getting at.

The transition to an OTRS fork would be much the same as any other OTRS upgrade. They're time-consuming, especially when database upgrades are concerned, and have some pitfalls, but they're not too difficult.

Migrating to non-OTRS software would likely require a significant engineering effort to import history, especially if the new software doesn't already have a good way to import OTRS data. The OTRS database has an absolutely huge amount of data T138915: OTRS database is "too large", which is important to underscore.

@akosiaris did the most recent major OTRS upgrade (T187984), they may be able to provide more information on the feasibility of various solutions.

The field is still "murky" and very active after the, not well communicated, "death" of OTRS community edition and the feasibility of the various paths out of this isn't currently clear to me at all. Researching those alone currently will require a considerable amount of effort. Drafting and executing a plan will also require effort and depending on the chosen solution it could be anything from relatively small to enormous. Given all that and the fact as a community are just a user of the software and not a contributor, my 2c right now say to wait out a bit for the dust to settle and reach out to others for input on how they plan to address this issue in their systems. Security wise, we should be able to cherry pick patches from the various forks for a while.

(Database is big since ticket attachments are stored in db instead of the filesystem storage. It can be converted, making the db way smaller and use the same space from plain filesystem storage.)

The reason for the database storage usage instead of the filesystem storage usage (which is the recommended way for large setups) is that we don't currently have a highly available and fault tolerant way to provide a filesystem (we 've been moving away from solutions doing so e.g. SANs for years). We do have those characteristics in our database offerings internally and thus can rely on that. We 've been planning to address this but the "death" of OTRS community edition is making this harder to plan for now.

jbond triaged this task as Medium priority.Feb 23 2021, 12:21 PM

We don't use the Debian OTRS packages for ticket.wikimedia.org, but adding this as an additional data point: Debian switched to the Znuny fork: https://tracker.debian.org/news/1234381/accepted-otrs2-6032-1-source-into-experimental/

There's now a group of companies related to OTRS which will be collaborating on Znuny: https://www.otter-alliance.de/en/die-allianz.html

There's now a group of companies related to OTRS which will be collaborating on Znuny: https://www.otter-alliance.de/en/die-allianz.html

Thank you for this link, Moritz.

After careful consideration, the Foundation is going to migrate over to Znuny's fork. This will provide some near-term stability and security to the system, and should be a pretty painless process from the standpoint of both engineering and for the community. I had a meeting with Johannes Nickel from Znuny and Alexandros from the WMF Site Reliability Engineering to go through some of the details of the fork and Znuny's future plans.

The migration consists almost wholly of branding change, there will be no functional or usability impact on the agents. Everything will work exactly the same after the fork as it did before. We're scheduling this to take place on Tuesday, 13 April, and I'm in touch with the OTRS admins about working with the community to manage through the change.

Znuny is committed to providing LTS support for the software formerly known as OTRS 6, they will publish a roadmap soon and will begin providing version updates. So we have at least a year or two to decide what we want the long-term future of the volunteer response team's software to be. We might want to stick with Znuny indefinitely, they are planning an update to the GUI and are dedicated FOSS partners, but we should also explore other options once we have some breathing room.

I would prefer to wait a week until this is published in Tech/News, I'm meeting with the admins and we're discussing and planning a larger re-branding effort. For now the migration is simply a re-branding our software internals, communicating this change needs to be done carefully to avoid confusion.

I would prefer to wait a week until this is published in Tech/News, I'm meeting with the admins and we're discussing and planning a larger re-branding effort. For now the migration is simply a re-branding our software internals, communicating this change needs to be done carefully to avoid confusion.

So, April 20th? Fine by me.

I would prefer to wait a week until this is published in Tech/News, I'm meeting with the admins and we're discussing and planning a larger re-branding effort. For now the migration is simply a re-branding our software internals, communicating this change needs to be done carefully to avoid confusion.

So, April 20th? Fine by me.

@akosiaris apologies, I was referring to waiting to publish the notice in this week's Tech/News instead of last week. I'd still like to do this on 13 April. Do you have time window we can schedule this for, and would you like me to make a task for the migration?

I would prefer to wait a week until this is published in Tech/News, I'm meeting with the admins and we're discussing and planning a larger re-branding effort. For now the migration is simply a re-branding our software internals, communicating this change needs to be done carefully to avoid confusion.

So, April 20th? Fine by me.

@akosiaris apologies, I was referring to waiting to publish the notice in this week's Tech/News instead of last week. I'd still like to do this on 13 April. Do you have time window we can schedule this for, and would you like me to make a task for the migration?

Oh, sorry my bad. I had reserved 2 hours on the 13th, from 07:00 UTC to 09:00 UTC, for the main migration. Judging from the input from Znuny, it should be sufficient, major issues aside. If any minor issues show up up after the migration, we can handle them outside that window of course.

+1 on the task. We probably want to have the most technically inclined agents aware and able to quickly provide input.

7:00-900 UTC on 13 April sounds good, I'll communicate that along. FWIW, I learned this weekend that WMDE has already forked their OTRS to Znuny LTS, it went quickly and painlessly.

I'll get the task made today.

I would prefer to wait a week until this is published in Tech/News, I'm meeting with the admins and we're discussing and planning a larger re-branding effort. For now the migration is simply a re-branding our software internals, communicating this change needs to be done carefully to avoid confusion.

So, April 20th? Fine by me.

@akosiaris apologies, I was referring to waiting to publish the notice in this week's Tech/News instead of last week. I'd still like to do this on 13 April. Do you have time window we can schedule this for, and would you like me to make a task for the migration?

Oh, sorry my bad. I had reserved 2 hours on the 13th, from 07:00 UTC to 09:00 UTC, for the main migration. Judging from the input from Znuny, it should be sufficient, major issues aside. If any minor issues show up up after the migration, we can handle them outside that window of course.

+1 on the task. We probably want to have the most technically inclined agents aware and able to quickly provide input.

I can be around during that window I think.

I can be around during that window I think.

Thanks! You might be interested in something else we may need to do in the near future, move OTRS wiki to a new name. It doesn't have the Wikidata problem blocking it, so perhaps it's feasible? Anyway, that's a ticket to look out for eventually and it can be discussed there when it's created.

Okay, so to sum this all up:

  • The Foundation is migrating over to Znuny LTS as our FOSS replacement for now, Znuny is promising support for at least two years to maintain and improve Znuny LTS. This will take place on 13 April, 7:00-9:00 UTC.
  • Back in late 2012/early 2013 the OTRS administrators decided to change the English name of the team to the "Volunteer Response Team." This name was never promoted or put into common usage, but now seems like the time to do so. Which leads to...
  • This will necessitate a rebranding effort on our Wikimedia projects as well. I'm working closely with the Volunteer Response Team leaders (formerly OTRS admins) to coordinate this effort, we've established a migration project team (first meeting notes):
    • User:Krd
    • User:DCB
    • User:Emufarmers
    • User:Ruthven
    • User:Raymond
    • User:Keegan (WMF)
  • Please do not rename anything on your local wiki, the migration project team will handle renamings as needed. Many non-English communities do not use "OTRS" as a common name for the team anyway, so local names for the team that already do not use OTRS will be unaffected.
    • "Please do not rename anything" includes templates on Commons and other wikis, the migration team will handle new logos and any replacement needed.
    • Further details about the renaming/rebranding project will be posted on OTRSwiki and where ever else might be relevant as they develop.
    • Deadlines for renaming/rebranding are self-imposed for now, the team will use this opportunity to be thorough as the work is defined.

This task can probably be closed after the migration is complete next week, we'll set up something else for the renaming project.

Please do not rename anything on your local wiki, the migration project team will handle renamings as needed.

I think local projects, especially non-English ones, are better placed to figure out how to rename things. OTRS is the name of a software so using that made some sense (although it was always a bit unhelpful since it's not a widely known piece of software) but using "Volunteer Response Team" / "VRT" on a non-English project doesn't and a mirror translation might not work well either. Providing some sort of "branding guidelines" (e.g. one might want to make clear this is something run by volunteers and not Wikipedia Inc.) would be a better approach.

Please do not rename anything on your local wiki, the migration project team will handle renamings as needed.

I think local projects, especially non-English ones, are better placed to figure out how to rename things. OTRS is the name of a software so using that made some sense (although it was always a bit unhelpful since it's not a widely known piece of software) but using "Volunteer Response Team" / "VRT" on a non-English project doesn't and a mirror translation might not work well either. Providing some sort of "branding guidelines" (e.g. one might want to make clear this is something run by volunteers and not Wikipedia Inc.) would be a better approach.

Yes, re-reading what I wrote it is unclear on this point but what you're saying is what's being planned to some extent. Local community names are certainly going to be preferable. In this instance I was referring to finding old references to "OTRS" on wikis that might not even have active agents on the team, that sort of find-and-replace work in addition to what's needed for templates. I think we'll put together some guidelines/naming conventions but for sure there's no intention to be a mandate for "VRT" for non-English communities.

something else we may need to do in the near future, move OTRS wiki to a new name

Should become a separate ticket under https://phabricator.wikimedia.org/project/view/2943/ I guess, and this ticket should be a subtask of that new ticket.

something else we may need to do in the near future, move OTRS wiki to a new name

Should become a separate ticket under https://phabricator.wikimedia.org/project/view/2943/ I guess, and this ticket should be a subtask of that new ticket.

Thanks, I'll get something going there in the near future.

You might be interested in something else we may need to do in the near future, move OTRS wiki to a new name. It doesn't have the Wikidata problem blocking it, so perhaps it's feasible? Anyway, that's a ticket to look out for eventually and it can be discussed there when it's created.

Uh oh. Yeah... I haven't been a Wikimedia deployer for over 4 years, and the sheer number of incomplete followup tasks from T11823 (that often don't even seem to be linked in phab :() would make me nervous to support such an operation. T172035 says to this day that renaming wikis is not advised.
Still, probably easier for a private/special wiki like otrs_wikiwiki than for a language-project wiki like be_x_oldwiki.

Hello! Pls add my account User:Niklitov from OTRS, Wikimedia RU! Best Regards!

something else we may need to do in the near future, move OTRS wiki to a new name

Should become a separate ticket under https://phabricator.wikimedia.org/project/view/2943/ I guess, and this ticket should be a subtask of that new ticket.

Thanks, I'll get something going there in the near future.

And here is an example of request for the global group rename: T256299

I can be around during that window I think.

Thanks!

Hey, I'm around but don't know where the discussion is happening. #wikimedia-operations seems quiet.

I can be around during that window I think.

Thanks!

Hey, I'm around but don't know where the discussion is happening. #wikimedia-operations seems quiet.

That's cause if there was a discussion, it would be me talking to myself. But I am logging stuff in #wikimedia-operations and updating T279303. I 've let other SREs be aware though.

The migration to Znuny LTS 6.0 is complete.

The volunteer admins and I are meeting on Thursday to discuss and plan the next steps in regards to community rebranding and renaming projects. I'll open a new epic to map out all the work that's needed for the next steps after we've met, otherwise this task is complete and I'll resolve it once I've created the new epic.