User Details
- User Since
- Sep 24 2020, 9:17 AM (129 w, 4 d)
- Availability
- Available
- LDAP User
- Vlad.shapik
- MediaWiki User
- Unknown
Jan 31 2023
Jan 17 2023
Jan 16 2023
Jan 13 2023
Jan 9 2023
As I understood from the Avoiding arcanist paragraph, also this repo should be deleted from Phabricator Diffusion in terms of this process. Am I right?
Hello @greg
I saw you were the author of the change that introduced Arcanist into the thumbor-plugins project.
As I noticed Arcanist hasn't been actively used since 2019. Now, all the code review process is on Gerrit. Shall we need to support .arcconfig, .arclint in the project since it isn't used? Or is it the strict rule to have and maintain it in the repo?
@hnowlan maybe you have some ideas about it.
Please feel free to share your thoughts.
Jan 6 2023
Jan 3 2023
Dec 22 2022
Dec 21 2022
The ticket was moved to Blocked on the Thumbor Migration board since the ticket requires decisions from the WMF management/engineering staff related to MediaWiki core, Structured Data, etc.
The ticket was moved to Blocked on the Thumbor Migration board since the ticket requires decisions from the WMF management/engineering staff related to MediaWiki core, Structured Data, etc.
The ticket was moved to Blocked on the Thumbor Migration board since the ticket requires decisions from the WMF management/engineering staff related to MediaWiki core, Structured Data, etc.
@TheDJ hello.
Thank you for the really good catch.
Does it happen on the current production version of Thumbor?
Hello @Jeff_G. It was interesting to work on this ticket because it was not really obvious what is going on with the mentioned files. I found nine buggy files.
I tested all files mentioned in the description of the ticket and the following link which goes to Commons_talk:CropTool.
I figured out that all thumbnails that we can't see even the preview fail with the next error on the production version of Thumbor -> RGB color space not permitted on grayscale PNG. I mean the thumbnails that look as in the screenshot below.
But on the Debian:Buster and Python3.7 (it will be a production version of Thumbor but now it is only on staging) I don't see this error locally and I successfully get the thumbnails of all these mentioned files. ImageMagick can already automatically convert RGB to grayscale than the ImageMagick version for Debian:Stretch - it is the version of OS that the current Thumbor production is based on.
The solution is to wait until the current version of Thumbor will be in production and this problem will disappear.
Dec 20 2022
Dec 15 2022
It seems that I found where is the trick. As it turned out the failed SVG file has a small body, as a result, the source in the prepare_source function can be created from a buffer and the buffer is a byte string. So it requires opening a file to write bytes.
Other files I took to test this bug have a bigger body, so the source was taken from the context and there is no need to create a temporary file and write to this file in the prepare_source function.
Dec 14 2022
Dec 13 2022
@hnowlan has checked the mentioned file on the staging and he didn't get any error as on the current production version of Thumbor. Since we will soon replace the current Thumbor production version(Debian Stretch/Python2.7) with the current staging one(Debian Buster/Python3.7), this ticket can be moved to done.
Dec 12 2022
I got the same error when I tried to thumbnail this file on the production version of Thumbor with Debian:Stretch and Python2.7 locally.
Here is the error message from libvips which causes the break of the code -> vipspng: profile 'icc': 'RGB ': RGB color space not permitted on grayscale PNG vips2png: unable to write.
Dec 8 2022
I have uploaded a TIF file to commons.wikimedia.beta.wmflabs.org with round brackets in the filename -> https://commons.wikimedia.beta.wmflabs.org/wiki/File:67-0372-C_Bann_Van_Hddds_(left)_and_Lai_Tio_Truffn.tif. And everything is ok. I believe my suspicion of round brackets in the filename is wrong.
@TheDJ thank you for your research. I am looking into it.
I just found that the mentioned file can be thumbnailed only if the name doesn't have any round brackets. I was able to reproduce it locally.
I successfully got an image for localhost:8800/thumbor/unsafe/800x/67-0372-C_Tran_Van_Huong_left_and_Mai_Tho_Truyen.tif
but
It failed for localhost:8800/thumbor/unsafe/800x/67-0372-C_Tran_Van_Huong_(left)_and_Mai_Tho_Truyen.tif
67-0372-C_Tran_Van_Huong_(left)_and_Mai_Tho_Truyen.tif is the original name of the mentioned file in the description of the ticket. The only difference is that the word left is enclosed in round brackets in the original filename.
Python 3.7 says that the file is not found.
Dec 7 2022
Dec 5 2022
Dec 2 2022
I worked within the Per-file control ticket. I researched the following subtasks: PNG thumbnail..., TIF less focused..., Allow PDF's to be rendered... I found several useful comments and links to the related Phab tickets.
There is an idea to create a new URL scheme for getting thumbnails. From my point of view, it is not a blocker for us to add an opportunity to configure creating thumbnails, but the status of this ticket should be controlled since a new URL scheme can influence the logic of per-file control for thumbnail generation parameters based on the plan below. This comment describes a valuable approach of using structured data and checking the header which Thumbor will process and decide what to do with one or another user config parameter.
Here is the short plan which can help to add per-file control for some thumbnail generation parameters:
- The community should define a standard for how file configs information is stored. As mentioned by @Tgr it could be a new property that needs to be proposed on Wikidata or could use some existing property.
- We also need a Mediawiki side logic which will detect when some config parameter of a file is changed in structured data, and push the info to a Swift header.
- This information would have to be stored in Swift alongside the original object as a special header, as access to the database from the thumbnailing layer is undesirable for security reasons.
- It should be also possible to configure only per file not per thumbnail because it helps to do this without inflating storage needs. There needs to be a mechanism to keep those particular flags in sync in Swift. Switching the flag on and off would need to purge all thumbnails. Once one of the flags exists in Swift, it's trivial for Thumbor to generate thumbnails accordingly.
- And at the end, Thumbor should check the header and apply a config parameter appropriately.
It seems like a lot of work should be done there.
Nov 29 2022
The DPI value will need to be stored with other image meta-data somehow, hence the suggestion of storing ti as a property or field.
The front-end of this task is basically a matter of querying the stuctured data for the DPI value that should be used. The data is stored in a json file, but there are functions to get the information you need, like the Page Banner extension uses.
Nov 23 2022
Hello @AntiCompositeNumber.
I am working on the BE solution to make it possible to configure the DPI option for PDF files. It is the sub-ticket T256959 of this one.
As I understood correctly you proposed that we have these configuration options already put in the structured data tab on the file page. And then thumbor will take possible configuration options and use them to generate the file on a case-by-case basis.
Did I get your point? Could you please share your ideas with me?
Hello @ShakespeareFan00
I am working on this ticket now. Could you please share with me the location of a 'property' for the File:'s entry in the image links table which you mentioned as a possible way to implement the DPI option in the FE?
And also will you be able to describe the idea of a 'modifier' template for use on sites like Wikimedia Commons and Wikisource?
For now, I just can't understand the connection between these two steps because I do not how the DPI option should look on these sites. Maybe you could give me some examples of already implemented options on these sites or something like that?
I just need to know if the front-end team(for now I don't know who will implement it in the FE) can use my BE solution to implement this configurator on the file page, for instance, that's why I am asking.
Nov 21 2022
Nov 15 2022
Nov 3 2022
Oct 28 2022
Oct 27 2022
Oct 25 2022
The code coverage has increased from 75% to 76%. It means that 11 lines of errors related code have been covered by tests.
Oct 24 2022
Oct 14 2022
Oct 11 2022
Sep 23 2022
Sep 13 2022
Aug 25 2022
Aug 19 2022
Hi @ori.
I will create a new patch with reverted changes related to https://gerrit.wikimedia.org/r/c/operations/software/thumbor-plugins/+/489022/6
Aug 17 2022
The results of the investigation:
- In general current tests work as expected and it doesn't seem that they are written wrong.
- Regarding the integration testing, most of the tests use one test case for testing - run_and_check_ssim_and_size. The number of testing parameters in this function can be extended which could bring better test cases. Additionally, I’ve noticed that “tests_3d.py” is not an online test, it has a local URL. And what's more, it can be as one of the variants to create more tests using different filters:(parameter), format:(parameter), etc of the URL for the Thumbor service to get a converted image.
- As for unit tests, it also can be taken into consideration because very few functions of the source code are used in the existing tests. For now, it is only an 80% assumption which has appeared after working with tests. But the picture will be clearer after the code coverage check.