Page MenuHomePhabricator

Plain Format view does not work on some tickets
Open, LowPublic

Description

When clicking "plain format" on certain tickets (such as https://ticket.wikimedia.org/otrs/index.pl?Action=AgentTicketZoom&TicketNumber=2015020910006071), an error pops up stating "Error Message: Can't read plain article! Maybe there is no plain email in backend! "

First reported approximately 10 days ago, approximately 1 dozen tickets are known to be affected (though it's not that common to need to check the plain format of an email so this may be a low reflection of the actual number of tickets).

Error details as follows:

Backend ERROR: OTRS-CGI-10 Perl: 5.14.2 OS: linux Time: Tue Feb 10 01:52:40 2015 Message: No plain article (article id 9635260) in database! RemoteAddress: 69.47.113.56 RequestURI: /otrs/index.pl?Action=AgentTicketPlain;TicketID=8123662;ArticleID=9635260 Traceback (31465): Module: Kernel::System::Ticket::ArticleStorageDB::ArticlePlain (OTRS 3.2.14) Line: 341 Module: Kernel::Modules::AgentTicketPlain::Run (OTRS 3.2.14) Line: 61 Module: Kernel::System::Web::InterfaceAgent::Run (OTRS 3.2.14) Line: 863 Module: ModPerl::ROOT::ModPerl::Registry::opt_otrs_bin_cgi_2dbin_index_2epl::handler (unknown version) Line: 41 Module: (eval) (v1.99) Line: 204 Module: ModPerl::RegistryCooker::run (v1.99) Line: 204 Module: ModPerl::RegistryCooker::default_handler (v1.99) Line: 170 Module: ModPerl::Registry::handler (v1.99) Line: 31

Event Timeline

Rjd0060 created this task.Feb 10 2015, 1:59 AM
Rjd0060 raised the priority of this task from to Needs Triage.
Rjd0060 updated the task description. (Show Details)
Rjd0060 added a project: OTRS.
Rjd0060 added subscribers: Rjd0060, Jgreen, pajz, Krenair.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 10 2015, 1:59 AM
Rjd0060 updated the task description. (Show Details)Feb 10 2015, 2:14 AM
Rjd0060 set Security to None.

A bounce was delivered in case of ticket 2015020910006133 with following error message:

This message was created automatically by mail delivery software.               
                                                                                
A message that you sent could not be delivered to one or more of its            
recipients. This is a permanent error. The following address(es) failed:        
                                                                                
permissions-commons@wikimedia.org                                               
local delivery failed                                                           
                                                                                
------ This is a copy of the message, including all the headers. ------         
------ The body of the message is 51320413 characters long; only the first      
------ 106496 or so are included here.

According to this error message the original message was quite voluminous, apparently through huge attachments with c. 51 MB in total. This probably hit some threshold, probably that of "ulimit -f" while saving the file to disk. However, I find it unfortunate that this limit is checked at a point where a ticket within the OTRS system is already created in the database, leaving all this in an inconsistent state. It seems better to

  • generate an appropriate error message in such cases (like email too large, exceeding the limit of ... MB) and to make
  • make sure that the OTRS database and file storage are consistent, i.e. check the size first, then attempt delivery to the OTRS system.
Aklapper triaged this task as Low priority.Feb 10 2015, 12:47 PM
Steinsplitter moved this task from Incoming to Backlog on the OTRS board.Mar 12 2015, 12:37 PM
Base added a subscriber: Base.Sep 15 2015, 10:33 PM
Restricted Application added a subscriber: Steinsplitter. · View Herald TranscriptSep 15 2015, 10:33 PM

Looking at upstream OTRS in their bug tracker, I get this list.

"No plain email in backend" is marked as a duplicate of "Better handling and feedback of too big mails/attachements" which is closed because it is considered an enhancement request to be handled in http://otrsteam.ideascale.com instead of their bug tracker. There is also "Plain article not saved if mail couldn't be sent".

pajz added a comment.EditedSep 16 2015, 10:20 PM

Thanks for checking. However, the problems described there seem to differ from the issue addressed with this report. In our case, the problem does not occure when a (too large) email is sent (outbound), but when we receive one with a large attachment.

First thing I'd be interested to know is where this threshold comes from that large emails seem to hit. Two notes:

  • OTRS itself is currently configured to "store all [attachment] data in the database" as opposed to on the filesystem. This is a setting in SysConfig, called Ticket::StorageModule; according to their description, this option is "not recommended for storing big attachments [...] 'FS' stores the data on the filesystem; this is faster but the webserver should run under the OTRS user. You can switch between the modules even on a system that is already in production without any loss of data."
  • There is also a PostMasterMaxEmailSize setting in SysConfig, described as "Maximal size in KBytes for mails that can be fetched via POP3/POP3S/IMAP/IMAPS (KBytes)." This is currently set to the default value of 16384.

So, as far as I can see, we're probably dealing with multiple thresholds here -- WMF mailservers probably have some (how high?), and likely there's also a limit to how much can be written in the DB (how much?). Could the current issues be related to inconsistencies between these thresholds?

Not sure who can answer the WMF mail configuration parts here. @Jgreen: Do you know?

Not sure who can answer the WMF mail configuration parts here. @Jgreen: Do you know?

There are a lot of moving parts in the mail chain, but I think a message being truncated happen would happen once a message is delivered to OTRS. Limits set elsewhere along the mail route would result in non-delivery, not truncation.

We don't use POP/IMAP for message delivery so I don't ~think~ PostMasterMaxEmailSize is relevant if that description is accurate. But there may be another setting that effects the postmaster script.

Since the last update to OTRS 5, is this bug still existing?