Application Error #401 [cyrillian Email]

This plugin allows you to report an issue in MantisBT by sending an email to a particular mail account

Moderators: Developer, Contributor

Post Reply
oboeck

Application Error #401 [cyrillian Email]

Post by oboeck »

Hello,

I hope you could help me to find a solution for my problem.

We installed the Plugin on a Mantis Version 1.2.3 and it runs smooth. It's some kind of windows server with an inetpub-folder (I don't have real access to this, because it's an customer).
The database has UTF-8 and in the plugin-setting is UTF-8 too.
Email Reporting Version is 0.8.2. and on the server runs PHP 5.2.14.

But at the time an email with cyrillian characters is collected from the postbox, the plugin crashes and throw this error Message.
The message is:

APPLICATION ERROR 0000401

Database query failed. Error received from database was #1366: Incorrect string value: '\x93junk...' for column 'description' at row 1 for the query: INSERT INTO mantis_bug_text_table
( description, steps_to_reproduce, additional_information) VALUES ( ?,?,?).
Please use the "Back" button in your web browser to return to the previous page. There you can correct whatever problems were identified in this error or select another action. You can also click an option from the menu bar to go directly to a new section.


Previous non-fatal errors occurred. Page content follow.
[...
SYSTEM WARNING: mb_convert_encoding() [..]: Unable to detect the character encoding
....]

I don't know if this here relates to my problem, but there "SL-Gundam (Manager)" fixed the problem via mail.
http://www.mantisbt.org/bugs/view.php?id=4286#c25072 (post 25072 to 25134)
http://www.mantisbt.org/bugs/view.php?id=11770 (this is the same just seperatly)

Kind Regards
oboeck
SL-Gundam
Posts: 722
Joined: 06 Jul 2011, 14:17

Re: Application Error #401 [cyrillian Email]

Post by SL-Gundam »

Referenced issues are not related to your problem i'm afraid

Your problem is the following. The email itself does not specify its character encoding or it could not be found by the EmailReporting plugin

If the character encoding is not found it falls back on "auto". Which is basically means ASCII,JIS,UTF-8,EUC-JP,SJIS (http://www.php.net/manual/en/mbstring.s ... odings.php)

Apparently this does not include cyrillian.

Now there are 2 options.
1. Find out why the character encoding is missing in your emails. Microsoft Outlook for example has the option to define it here for new emails: Tools -> Options -> Mail Format -> International options -> Encoding options
2. Wait for the release of EmailReporting 0.9.0 (which requires MantisBT 1.2.6+). I'm going to try to improve the detection of the character sets in that version
oboeck

Re: Application Error #401 [cyrillian Email]

Post by oboeck »

Hello SL-Gundam,

big thanks for that fast response.

I will try it with the encoding in the mailsettings from outlook and give a feedback afterwards.
But it is a bit strange too that the fallback "utf-8" or "windows-1250" as it says on the list from the mbstring apparently doesn't support cyrillic characters? I though cyrillic is part of UTF-8 or windows-1251.

Kind regards
oboeck
SL-Gundam
Posts: 722
Joined: 06 Jul 2011, 14:17

Re: Application Error #401 [cyrillian Email]

Post by SL-Gundam »

the fallback "auto" is only the short list i posted, not all the character sets in the link i gave. Thats the thing that needs to be fixed in 0.9.0
oboeck

Re: Application Error #401 [cyrillian Email]

Post by oboeck »

Hello SL-Gundam,

now I gotcha!
I tried to set the enconding into outlook but it's not possible in my version. And the customer said, that the mail come from the different countries from his customers and he couldn't really check what encodings they use.
So we decided to wait for the next release. Could you please expand the "auto" as you already suggested? We need this really, to get the thing runnig right.

What do you thing when the next release is coming??

Thank you for all
Kind regards
oboeck
SL-Gundam
Posts: 722
Joined: 06 Jul 2011, 14:17

Re: Application Error #401 [cyrillian Email]

Post by SL-Gundam »

0.9.0 is still in development. I'm not really sure when it will be released as i've still got a pretty big todo list before a stable release is possible.

If you are in a hurry then i can post here ones the changes have been committed into git and then you can get the 0.9.0-DEV. But since that is a development release it could contain any number of bugs
indancesky
Posts: 1
Joined: 02 Aug 2012, 02:10

Re: Application Error #401 [cyrillian Email]

Post by indancesky »

Apparently this does not include cyrillian.

Now there are 2 options.
1. Find out why the character encoding is missing in your emails. Microsoft Outlook for example has the option to define it here for new emails: Tools -> Options -> Mail Format -> International options -> Encoding options
2. Wait for the release of EmailReporting 0.9.0 (which requires MantisBT 1.2.6+). I'm going to try to improve the detection of the character sets in that version
.
Rc Helicopters Reviews|Rc Helicopter
athirat
Posts: 1
Joined: 02 Nov 2012, 08:03

Re: Application Error #401 [cyrillian Email]

Post by athirat »

hi
i am running Mantis 1.2.11 with Emailreporting 0.9.0-dev. I found the same problem when trying to get email which is not standard encoding. After investigate, the problem will happen when emailreporting try to convert email with "TIS-620" or "windows-874". those charset are using for Thai encoding. Unfortunately, both charset are not support by function mb_convert_encoding as mentioned at http://www.php.net/manual/en/mbstring.s ... odings.php

Cause of problem & solution:

1. in original Parser.php, it will check whether the charset in mail heading is support by function:mb_convert_encoding or not, if not it will set charset to 'auto' and leave function:mb_convert_encoding handle the conversion which end up with error.

Solution: i have decided to to use charset in email header for conversion instead.

Original function :
private function setCharset( $charset )
{
if ( extension_loaded( 'mbstring' ) && $this->_charset === 'auto' )
{
$t_arraysearch_result = array_search( strtolower( $charset ), $this->_mb_list_encodings, TRUE );
$this->_charset = ( ( $t_arraysearch_result !== FALSE ) ? $t_arraysearch_result : 'auto' );
}
}

Modified function:
private function setCharset( $charset )
{
if ( extension_loaded( 'mbstring' ) && $this->_charset === 'auto' )
{
$t_arraysearch_result = array_search( strtolower( $charset ), $this->_mb_list_encodings, TRUE );
// $this->_charset = ( ( $t_arraysearch_result !== FALSE ) ? $t_arraysearch_result : 'auto' );
$this->_charset = ( ( $t_arraysearch_result !== FALSE ) ? $t_arraysearch_result : $charset );

}
}

2. as mentioned, the function:mb_convert_encoding still not support Thai encode but function: iconv does. so i have change function:process_encodeing as below

Original:
private function process_encoding( $encode )
{
if ( extension_loaded( 'mbstring' ) && $this->_encoding !== $this->_charset )
{
$encode = mb_convert_encoding( $encode, $this->_encoding, $this->_charset );
}

return( $encode );
}

Modified :
private function process_encoding( $encode )
{
if ( extension_loaded( 'mbstring' ) && $this->_encoding !== $this->_charset )
{

//$encode = mb_convert_encoding( $encode, $this->_encoding, $this->_charset );
$encode = iconv( $this->_charset, $this->_encoding, $encode );
}

return( $encode );
}

after the changes, Mantis now can handle both encode and show proper text in ticket.

I hope this can help someone who has problem like me.

Cheers,
Athirat
SL-Gundam
Posts: 722
Joined: 06 Jul 2011, 14:17

Re: Application Error #401 [cyrillian Email]

Post by SL-Gundam »

Thanks i will see about integrating a choice to use iconv or mb_string and maybe integrate part of your code
mdiallo
Posts: 1
Joined: 02 Jun 2021, 09:47

Re: Application Error #401 [cyrillian Email]

Post by mdiallo »

Hello, where is located the file of these function
s please i have litteraly the same problem and i want to try to fix it.

thank you
athirat wrote: 02 Nov 2012, 08:27 hi
i am running Mantis 1.2.11 with Emailreporting 0.9.0-dev. I found the same problem when trying to get email which is not standard encoding. After investigate, the problem will happen when emailreporting try to convert email with "TIS-620" or "windows-874". those charset are using for Thai encoding. Unfortunately, both charset are not support by function mb_convert_encoding as mentioned at http://www.php.net/manual/en/mbstring.s ... odings.php

Cause of problem & solution:

1. in original Parser.php, it will check whether the charset in mail heading is support by function:mb_convert_encoding or not, if not it will set charset to 'auto' and leave function:mb_convert_encoding handle the conversion which end up with error.

Solution: i have decided to to use charset in email header for conversion instead.

Original function :
private function setCharset( $charset )
{
if ( extension_loaded( 'mbstring' ) && $this->_charset === 'auto' )
{
$t_arraysearch_result = array_search( strtolower( $charset ), $this->_mb_list_encodings, TRUE );
$this->_charset = ( ( $t_arraysearch_result !== FALSE ) ? $t_arraysearch_result : 'auto' );
}
}

Modified function:
private function setCharset( $charset )
{
if ( extension_loaded( 'mbstring' ) && $this->_charset === 'auto' )
{
$t_arraysearch_result = array_search( strtolower( $charset ), $this->_mb_list_encodings, TRUE );
// $this->_charset = ( ( $t_arraysearch_result !== FALSE ) ? $t_arraysearch_result : 'auto' );
$this->_charset = ( ( $t_arraysearch_result !== FALSE ) ? $t_arraysearch_result : $charset );

}
}

2. as mentioned, the function:mb_convert_encoding still not support Thai encode but function: iconv does. so i have change function:process_encodeing as below

Original:
private function process_encoding( $encode )
{
if ( extension_loaded( 'mbstring' ) && $this->_encoding !== $this->_charset )
{
$encode = mb_convert_encoding( $encode, $this->_encoding, $this->_charset );
}

return( $encode );
}

Modified :
private function process_encoding( $encode )
{
if ( extension_loaded( 'mbstring' ) && $this->_encoding !== $this->_charset )
{

//$encode = mb_convert_encoding( $encode, $this->_encoding, $this->_charset );
$encode = iconv( $this->_charset, $this->_encoding, $encode );
}

return( $encode );
}

after the changes, Mantis now can handle both encode and show proper text in ticket.

I hope this can help someone who has problem like me.

Cheers,
Athirat
SL-Gundam
Posts: 722
Joined: 06 Jul 2011, 14:17

Re: Application Error #401 [cyrillian Email]

Post by SL-Gundam »

https://github.com/mantisbt-plugins/Ema ... Parser.php
ICONV will not be used by the base project because MantisBT uses MBSTRING and could therefore result in side effects.
Post Reply