Page MenuHomePhabricator

Huggle doesn't show Unicode characters correctly in Webkit + Ubuntu
Open, LowestPublic

Description

Hi, I received latest Snap release of Huggle and got it running on Ubuntu 16.04, I tried using it on Fa Wikipedia but it doesn't show characters correctly, it's default encoding seems to be non UTF-8.
Any solutions?
Here's an example of what I see:
محمد باقری در رزمایش امنیتی پیامبر اعظم سپØ

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 7 2017, 8:52 AM
Petrb added a subscriber: Petrb.Sep 7 2017, 9:33 AM

Hi,

This is probably related to https://github.com/huggle/huggle3-qt-lx/issues/244 I am now looking into it.

Petrb triaged this task as High priority.Sep 7 2017, 9:34 AM
Petrb added a subscriber: Ijon.Sep 8 2017, 9:35 AM
Petrb added a comment.Sep 8 2017, 9:48 AM

Interesting part is that on my Linux it display OK but when I try to dump the page it results in garbage, so the problem here may be with the function that dumps the traffic to file. In my case I am using web-engine though, instead of WebKit.

On Ubuntu I can reproduce this (under WebKit). On Windows it works ok.

Function that retrieves the data from mediawiki is here: https://github.com/huggle/huggle3-qt-lx/blob/master/huggle/apiquery.cpp#L398

it essentially is all about this->temp += this->reply->readAll(); which is a function inherited from https://doc.qt.io/qt-5/qiodevice.html#readAll

That function returns QByteArray which is appended to this->temp, that variable is later converted into QString in function Finish() on line 269:

result->Data = QString(this->temp);

This is probably where the conversion issues happen. But it's just my guess.

Petrb added a comment.Sep 8 2017, 11:48 AM

It seems like a bug in the webkit, when I compiled Huggle with web-engine on Ubuntu this worked just fine. Problem is that this component is so new that it's not yet in Ubuntu repository, so it needs to be compiled using Qt libraries downloaded from qt community portal.

Configuration script looks like this:

./configure --qt5 --extension --qtpath ~/Qt/5.9.1/gcc_64/ --web-engine
Petrb added a comment.Sep 8 2017, 11:48 AM

Unfortunately I am not yet aware of any sane way to include this into Canonical's snap auto-builder, I might be able to compile some custom binary package though on my virtual box as a workaround for people who have this problem.

Framawiki renamed this task from Huggle doesn't show Persian characters correctly in Linux to Huggle doesn't show Unicode characters correctly in Webkit + Ubuntu.Sep 11 2017, 7:07 PM
Framawiki added a subscriber: Framawiki.
Restricted Application added a subscriber: RichSmith. · View Herald TranscriptMay 19 2018, 1:08 PM
Petrb added a comment.Jun 10 2018, 5:16 PM

Given that WebKit is de-facto deprecated I would rather focus on making Ubuntu snaps based on web engine

Petrb moved this task from Backlog to Code / Bug on the Huggle board.Oct 10 2018, 6:19 PM
Petrb lowered the priority of this task from High to Lowest.Dec 13 2018, 9:37 PM

we are not waiting for snap system to support newer Qt as well which has web engine, then we can finally get rid of webkit. Nothing I can do besides waiting.