Page MenuHomePhabricator

Vlad.shapik (Vlad.shapik)
User

Projects

User does not belong to any projects.

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Monday

  • Clear sailing ahead.

User Details

User Since
Sep 24 2020, 9:17 AM (186 w, 2 d)
Availability
Available
LDAP User
Vlad.shapik
MediaWiki User
Unknown

Recent Activity

Jan 31 2023

Vlad.shapik reassigned T325770: Add ability to specify default filters such as sharpening and etc. for TIFF format from Vlad.shapik to Atieno.
Jan 31 2023, 2:06 PM · Thumbor Migration, Thumbor
Vlad.shapik reassigned T325811: Update Thumbor documentation and repository from Vlad.shapik to Atieno.
Jan 31 2023, 2:06 PM · Thumbor, Thumbor Migration
Vlad.shapik reassigned T253280: Per-file control for some thumbnail generation parameters from Vlad.shapik to Atieno.
Jan 31 2023, 2:06 PM · Thumbor Migration, Commons, MediaWiki-File-management, SDC General, Wikidata, Thumbor
Vlad.shapik reassigned T325771: Add the ability to specify the default DPI value for PDF files from Vlad.shapik to Atieno.
Jan 31 2023, 2:06 PM · Thumbor Migration, Thumbor
Vlad.shapik reassigned T256959: Allow PDF's to be rendered at higher (or user specified DPI) from Vlad.shapik to Atieno.
Jan 31 2023, 2:05 PM · MediaWiki-extensions-PdfHandler, Thumbor Migration, Patch-For-Review, Commons, All-and-every-Wikisource, Thumbor
Vlad.shapik closed T318548: Incorrect orientation of some thumbnails as Resolved.
Jan 31 2023, 2:02 PM · Thumbor Migration, Thumbor
Vlad.shapik closed T313016: Investigate Code Coverage with tests for all the code as Resolved.
Jan 31 2023, 2:02 PM · Test-Coverage, Thumbor Migration
Vlad.shapik closed T252719: Upgrade thumbor to Thumbor 7 and python3, a subtask of T294484: <Code Stewardship Review> Thumbor, as Resolved.
Jan 31 2023, 2:01 PM · Thumbor, Code-Stewardship-Reviews, Foundational Technology Requests
Vlad.shapik closed T252719: Upgrade thumbor to Thumbor 7 and python3 as Resolved.
Jan 31 2023, 2:01 PM · Patch-For-Review, Thumbor Migration, Python3-Porting
Vlad.shapik closed T308561: Create a branch with tc core dependency for Wikimedia Thumbor as Resolved.
Jan 31 2023, 2:00 PM · Patch-For-Review, Thumbor Migration
Vlad.shapik closed T312103: Update "Online" unit tests as Resolved.
Jan 31 2023, 2:00 PM · Thumbor Migration
Vlad.shapik closed T312919: Investigate whether these tests are currently correct and fit for purpose as Resolved.
Jan 31 2023, 2:00 PM · Thumbor Migration
Vlad.shapik closed T314657: Setup Debugger for Thumbor as Resolved.
Jan 31 2023, 2:00 PM · Thumbor Migration
Vlad.shapik closed T314393: Division operation fix for Python 3+ as Resolved.
Jan 31 2023, 2:00 PM · Python3-Porting, Thumbor Migration
Vlad.shapik closed T318406: Providing additional coverage based on the code coverage research results as Resolved.
Jan 31 2023, 1:59 PM · Test-Coverage, Thumbor Migration
Vlad.shapik closed T314015: HTTP 404 error for specific TIF thumbnail on Commons as Resolved.
Jan 31 2023, 1:59 PM · Thumbor Migration, Commons, Thumbor
Vlad.shapik closed T311004: HTTP 429 returned when rendering thumbnails for grayscale TIFF files as Resolved.
Jan 31 2023, 1:59 PM · Thumbor Migration, Thumbor, Commons

Jan 17 2023

Vlad.shapik updated the task description for T325811: Update Thumbor documentation and repository .
Jan 17 2023, 3:43 PM · Thumbor, Thumbor Migration

Jan 16 2023

Vlad.shapik updated the task description for T325811: Update Thumbor documentation and repository .
Jan 16 2023, 11:44 AM · Thumbor, Thumbor Migration

Jan 13 2023

Vlad.shapik updated the task description for T325811: Update Thumbor documentation and repository .
Jan 13 2023, 4:23 PM · Thumbor, Thumbor Migration
Vlad.shapik updated the task description for T325811: Update Thumbor documentation and repository .
Jan 13 2023, 4:12 PM · Thumbor, Thumbor Migration

Jan 9 2023

Vlad.shapik added a comment to T325811: Update Thumbor documentation and repository .

different at that time (move from Gerrit to Phab Differential instead of GitLab).

As I understood from the Avoiding arcanist paragraph, also this repo should be deleted from Phabricator Diffusion in terms of this process. Am I right?

Jan 9 2023, 3:13 PM · Thumbor, Thumbor Migration
Vlad.shapik moved T314015: HTTP 404 error for specific TIF thumbnail on Commons from In Review to Done on the Thumbor Migration board.
Jan 9 2023, 2:48 PM · Thumbor Migration, Commons, Thumbor
Vlad.shapik moved T325150: byte/str TypeError during svg conversion from In Review to Done on the Thumbor Migration board.
Jan 9 2023, 2:47 PM · Platform Team Workboards (Platform Engineering Reliability), Thumbor Migration, serviceops, Thumbor
Vlad.shapik added a comment to T325811: Update Thumbor documentation and repository .

Arcanist can be removed. That change was merged 6.5y ago and plans were different at that time (move from Gerrit to Phab Differential instead of GitLab).

Jan 9 2023, 1:21 PM · Thumbor, Thumbor Migration
Vlad.shapik updated subscribers of T325811: Update Thumbor documentation and repository .

Hello @greg
I saw you were the author of the change that introduced Arcanist into the thumbor-plugins project.
As I noticed Arcanist hasn't been actively used since 2019. Now, all the code review process is on Gerrit. Shall we need to support .arcconfig, .arclint in the project since it isn't used? Or is it the strict rule to have and maintain it in the repo?
@hnowlan maybe you have some ideas about it.
Please feel free to share your thoughts.

Jan 9 2023, 12:47 PM · Thumbor, Thumbor Migration
Vlad.shapik updated the task description for T325811: Update Thumbor documentation and repository .
Jan 9 2023, 10:50 AM · Thumbor, Thumbor Migration
Vlad.shapik updated the task description for T325811: Update Thumbor documentation and repository .
Jan 9 2023, 10:39 AM · Thumbor, Thumbor Migration

Jan 6 2023

Vlad.shapik added a comment to T253280: Per-file control for some thumbnail generation parameters .

T325948: Provide an API for accessing alt text (and possibly other structured data) stored with MediaWiki file uploads and T325949: Allow access to (some) structured image metadata across wikis is my vague idea of how to access Commons structured file metadata during thumbnail rendering. (The use case I'm personally interested in is alt text, but as I said above I think it is a fairly generic problem.)

Jan 6 2023, 11:47 AM · Thumbor Migration, Commons, MediaWiki-File-management, SDC General, Wikidata, Thumbor

Jan 3 2023

Vlad.shapik updated the task description for T325811: Update Thumbor documentation and repository .
Jan 3 2023, 5:09 PM · Thumbor, Thumbor Migration
Vlad.shapik updated the task description for T325811: Update Thumbor documentation and repository .
Jan 3 2023, 1:56 PM · Thumbor, Thumbor Migration

Dec 22 2022

Vlad.shapik moved T325811: Update Thumbor documentation and repository from Backlog to In Progress on the Thumbor Migration board.
Dec 22 2022, 12:34 PM · Thumbor, Thumbor Migration
Vlad.shapik created T325811: Update Thumbor documentation and repository .
Dec 22 2022, 12:34 PM · Thumbor, Thumbor Migration

Dec 21 2022

Vlad.shapik moved T325770: Add ability to specify default filters such as sharpening and etc. for TIFF format from Backlog to In Review on the Thumbor Migration board.
Dec 21 2022, 7:50 PM · Thumbor Migration, Thumbor
Vlad.shapik moved T325771: Add the ability to specify the default DPI value for PDF files from Backlog to In Review on the Thumbor Migration board.
Dec 21 2022, 7:49 PM · Thumbor Migration, Thumbor
Vlad.shapik created T325771: Add the ability to specify the default DPI value for PDF files.
Dec 21 2022, 7:47 PM · Thumbor Migration, Thumbor
Vlad.shapik renamed T325770: Add ability to specify default filters such as sharpening and etc. for TIFF format from Add ability to specify filters such as sharpening and etc. for TIFF format to Add ability to specify default filters such as sharpening and etc. for TIFF format.
Dec 21 2022, 7:44 PM · Thumbor Migration, Thumbor
Vlad.shapik updated the task description for T325770: Add ability to specify default filters such as sharpening and etc. for TIFF format.
Dec 21 2022, 7:40 PM · Thumbor Migration, Thumbor
Vlad.shapik created T325770: Add ability to specify default filters such as sharpening and etc. for TIFF format.
Dec 21 2022, 7:32 PM · Thumbor Migration, Thumbor
Vlad.shapik added a comment to T256959: Allow PDF's to be rendered at higher (or user specified DPI).

The ticket was moved to Blocked on the Thumbor Migration board since the ticket requires decisions from the WMF management/engineering staff related to MediaWiki core, Structured Data, etc.

Dec 21 2022, 7:08 PM · MediaWiki-extensions-PdfHandler, Thumbor Migration, Patch-For-Review, Commons, All-and-every-Wikisource, Thumbor
Vlad.shapik added a comment to T47212: TIF less focused than JPG equivalent (due to conditional sharpening applied to JPEGs and not TIFFS).

The ticket was moved to Blocked on the Thumbor Migration board since the ticket requires decisions from the WMF management/engineering staff related to MediaWiki core, Structured Data, etc.

Dec 21 2022, 7:06 PM · Thumbor Migration, Thumbor, Commons, MediaWiki-File-management
Vlad.shapik added a comment to T253280: Per-file control for some thumbnail generation parameters .

The ticket was moved to Blocked on the Thumbor Migration board since the ticket requires decisions from the WMF management/engineering staff related to MediaWiki core, Structured Data, etc.

Dec 21 2022, 7:06 PM · Thumbor Migration, Commons, MediaWiki-File-management, SDC General, Wikidata, Thumbor
Vlad.shapik moved T253280: Per-file control for some thumbnail generation parameters from In Progress to Blocked on the Thumbor Migration board.
Dec 21 2022, 7:01 PM · Thumbor Migration, Commons, MediaWiki-File-management, SDC General, Wikidata, Thumbor
Vlad.shapik moved T47212: TIF less focused than JPG equivalent (due to conditional sharpening applied to JPEGs and not TIFFS) from In Progress to Blocked on the Thumbor Migration board.
Dec 21 2022, 7:01 PM · Thumbor Migration, Thumbor, Commons, MediaWiki-File-management
Vlad.shapik moved T256959: Allow PDF's to be rendered at higher (or user specified DPI) from In Progress to Blocked on the Thumbor Migration board.
Dec 21 2022, 7:01 PM · MediaWiki-extensions-PdfHandler, Thumbor Migration, Patch-For-Review, Commons, All-and-every-Wikisource, Thumbor
Vlad.shapik added a comment to T325656: [SWIFT_STORAGE] put exception: AttributeError("'ClientException' object has no attribute 'startswith'",) for files in office wiki.

@TheDJ hello.
Thank you for the really good catch.
Does it happen on the current production version of Thumbor?

Dec 21 2022, 6:58 PM · Thumbor, Wikimedia-production-error
Vlad.shapik moved T325150: byte/str TypeError during svg conversion from In Progress to In Review on the Thumbor Migration board.
Dec 21 2022, 6:54 PM · Platform Team Workboards (Platform Engineering Reliability), Thumbor Migration, serviceops, Thumbor
Vlad.shapik moved T314015: HTTP 404 error for specific TIF thumbnail on Commons from In Progress to In Review on the Thumbor Migration board.
Dec 21 2022, 6:53 PM · Thumbor Migration, Commons, Thumbor
Vlad.shapik added a comment to T314015: HTTP 404 error for specific TIF thumbnail on Commons.

Hello @Jeff_G. It was interesting to work on this ticket because it was not really obvious what is going on with the mentioned files. I found nine buggy files.
I tested all files mentioned in the description of the ticket and the following link which goes to Commons_talk:CropTool.
I figured out that all thumbnails that we can't see even the preview fail with the next error on the production version of Thumbor -> RGB color space not permitted on grayscale PNG. I mean the thumbnails that look as in the screenshot below.

tempsnip.png (888×1 px, 230 KB)

But on the Debian:Buster and Python3.7 (it will be a production version of Thumbor but now it is only on staging) I don't see this error locally and I successfully get the thumbnails of all these mentioned files. ImageMagick can already automatically convert RGB to grayscale than the ImageMagick version for Debian:Stretch - it is the version of OS that the current Thumbor production is based on.
The solution is to wait until the current version of Thumbor will be in production and this problem will disappear.

Dec 21 2022, 6:46 PM · Thumbor Migration, Commons, Thumbor

Dec 20 2022

Vlad.shapik claimed T314015: HTTP 404 error for specific TIF thumbnail on Commons.
Dec 20 2022, 1:08 PM · Thumbor Migration, Commons, Thumbor

Dec 15 2022

Vlad.shapik added a comment to T325150: byte/str TypeError during svg conversion .

It seems that I found where is the trick. As it turned out the failed SVG file has a small body, as a result, the source in the prepare_source function can be created from a buffer and the buffer is a byte string. So it requires opening a file to write bytes.
Other files I took to test this bug have a bigger body, so the source was taken from the context and there is no need to create a temporary file and write to this file in the prepare_source function.

Dec 15 2022, 6:38 PM · Platform Team Workboards (Platform Engineering Reliability), Thumbor Migration, serviceops, Thumbor
Vlad.shapik claimed T325150: byte/str TypeError during svg conversion .
Dec 15 2022, 4:04 PM · Platform Team Workboards (Platform Engineering Reliability), Thumbor Migration, serviceops, Thumbor

Dec 14 2022

Vlad.shapik moved T314015: HTTP 404 error for specific TIF thumbnail on Commons from To Do to In Progress on the Thumbor Migration board.
Dec 14 2022, 5:58 PM · Thumbor Migration, Commons, Thumbor
Vlad.shapik moved T47212: TIF less focused than JPG equivalent (due to conditional sharpening applied to JPEGs and not TIFFS) from To Do to In Progress on the Thumbor Migration board.
Dec 14 2022, 5:58 PM · Thumbor Migration, Thumbor, Commons, MediaWiki-File-management
Vlad.shapik moved T325150: byte/str TypeError during svg conversion from Backlog to In Progress on the Thumbor Migration board.
Dec 14 2022, 5:56 PM · Platform Team Workboards (Platform Engineering Reliability), Thumbor Migration, serviceops, Thumbor
Vlad.shapik moved T311004: HTTP 429 returned when rendering thumbnails for grayscale TIFF files from In Progress to Done on the Thumbor Migration board.
Dec 14 2022, 5:52 PM · Thumbor Migration, Thumbor, Commons

Dec 13 2022

Vlad.shapik updated subscribers of T311004: HTTP 429 returned when rendering thumbnails for grayscale TIFF files.

@hnowlan has checked the mentioned file on the staging and he didn't get any error as on the current production version of Thumbor. Since we will soon replace the current Thumbor production version(Debian Stretch/Python2.7) with the current staging one(Debian Buster/Python3.7), this ticket can be moved to done.

Dec 13 2022, 5:49 PM · Thumbor Migration, Thumbor, Commons

Dec 12 2022

Vlad.shapik added a comment to T311004: HTTP 429 returned when rendering thumbnails for grayscale TIFF files.
File "/usr/lib/python2.7/dist-packages/wikimedia_thumbor/engine/imagemagick/imagemagick.py", line 62, in create_image
  temp_file.write(buffer)

I got the same error when I tried to thumbnail this file on the production version of Thumbor with Debian:Stretch and Python2.7 locally.
Here is the error message from libvips which causes the break of the code -> vipspng: profile 'icc': 'RGB ': RGB color space not permitted on grayscale PNG vips2png: unable to write.

Dec 12 2022, 10:12 PM · Thumbor Migration, Thumbor, Commons

Dec 8 2022

Vlad.shapik renamed T311004: HTTP 429 returned when rendering thumbnails for grayscale TIFF files from HTTP 429 returned when rendering thumbnails for tiff file with ( and or ) in the filename to HTTP 429 returned when rendering thumbnails for certain TIFF file.
Dec 8 2022, 6:23 PM · Thumbor Migration, Thumbor, Commons
Vlad.shapik added a comment to T311004: HTTP 429 returned when rendering thumbnails for grayscale TIFF files.

I just found that the mentioned file can be thumbnailed only if the name doesn't have any round brackets. I was able to reproduce it locally.

I have uploaded a TIF file to commons.wikimedia.beta.wmflabs.org with round brackets in the filename -> https://commons.wikimedia.beta.wmflabs.org/wiki/File:67-0372-C_Bann_Van_Hddds_(left)_and_Lai_Tio_Truffn.tif. And everything is ok. I believe my suspicion of round brackets in the filename is wrong.

Dec 8 2022, 5:59 PM · Thumbor Migration, Thumbor, Commons
Vlad.shapik added a comment to T311004: HTTP 429 returned when rendering thumbnails for grayscale TIFF files.

@TheDJ thank you for your research. I am looking into it.
I just found that the mentioned file can be thumbnailed only if the name doesn't have any round brackets. I was able to reproduce it locally.
I successfully got an image for localhost:8800/thumbor/unsafe/800x/67-0372-C_Tran_Van_Huong_left_and_Mai_Tho_Truyen.tif
but
It failed for localhost:8800/thumbor/unsafe/800x/67-0372-C_Tran_Van_Huong_(left)_and_Mai_Tho_Truyen.tif
67-0372-C_Tran_Van_Huong_(left)_and_Mai_Tho_Truyen.tif is the original name of the mentioned file in the description of the ticket. The only difference is that the word left is enclosed in round brackets in the original filename.
Python 3.7 says that the file is not found.

Dec 8 2022, 12:11 PM · Thumbor Migration, Thumbor, Commons

Dec 7 2022

Vlad.shapik claimed T311004: HTTP 429 returned when rendering thumbnails for grayscale TIFF files.
Dec 7 2022, 4:37 PM · Thumbor Migration, Thumbor, Commons

Dec 5 2022

Vlad.shapik added a comment to T256959: Allow PDF's to be rendered at higher (or user specified DPI).

Change 853402 had a related patch set uploaded (by Vlad.shapik; author: Vlad.shapik):

[operations/software/thumbor-plugins@master] Add ability to specify a DPI value for PDF

https://gerrit.wikimedia.org/r/853402

Dec 5 2022, 9:17 PM · MediaWiki-extensions-PdfHandler, Thumbor Migration, Patch-For-Review, Commons, All-and-every-Wikisource, Thumbor
Vlad.shapik added a comment to T47212: TIF less focused than JPG equivalent (due to conditional sharpening applied to JPEGs and not TIFFS).

Change 863399 had a related patch set uploaded (by Vlad.shapik; author: Vlad.shapik):

[operations/software/thumbor-plugins@master] WP: Add ability to specify filters such as sharpening and etc. for TIFF format

https://gerrit.wikimedia.org/r/863399

Dec 5 2022, 9:14 PM · Thumbor Migration, Thumbor, Commons, MediaWiki-File-management

Dec 2 2022

Vlad.shapik updated subscribers of T253280: Per-file control for some thumbnail generation parameters .

I worked within the Per-file control ticket. I researched the following subtasks: PNG thumbnail..., TIF less focused..., Allow PDF's to be rendered... I found several useful comments and links to the related Phab tickets.
There is an idea to create a new URL scheme for getting thumbnails. From my point of view, it is not a blocker for us to add an opportunity to configure creating thumbnails, but the status of this ticket should be controlled since a new URL scheme can influence the logic of per-file control for thumbnail generation parameters based on the plan below. This comment describes a valuable approach of using structured data and checking the header which Thumbor will process and decide what to do with one or another user config parameter.
Here is the short plan which can help to add per-file control for some thumbnail generation parameters:

  1. The community should define a standard for how file configs information is stored. As mentioned by @Tgr it could be a new property that needs to be proposed on Wikidata or could use some existing property.
  2. We also need a Mediawiki side logic which will detect when some config parameter of a file is changed in structured data, and push the info to a Swift header.
  3. This information would have to be stored in Swift alongside the original object as a special header, as access to the database from the thumbnailing layer is undesirable for security reasons.
  4. It should be also possible to configure only per file not per thumbnail because it helps to do this without inflating storage needs. There needs to be a mechanism to keep those particular flags in sync in Swift. Switching the flag on and off would need to purge all thumbnails. Once one of the flags exists in Swift, it's trivial for Thumbor to generate thumbnails accordingly.
  5. And at the end, Thumbor should check the header and apply a config parameter appropriately.

It seems like a lot of work should be done there.

Dec 2 2022, 3:25 PM · Thumbor Migration, Commons, MediaWiki-File-management, SDC General, Wikidata, Thumbor

Nov 29 2022

Vlad.shapik added a comment to T256959: Allow PDF's to be rendered at higher (or user specified DPI).

The DPI value will need to be stored with other image meta-data somehow, hence the suggestion of storing ti as a property or field.

Nov 29 2022, 7:09 PM · MediaWiki-extensions-PdfHandler, Thumbor Migration, Patch-For-Review, Commons, All-and-every-Wikisource, Thumbor
Vlad.shapik added a comment to T256959: Allow PDF's to be rendered at higher (or user specified DPI).

The front-end of this task is basically a matter of querying the stuctured data for the DPI value that should be used. The data is stored in a json file, but there are functions to get the information you need, like the Page Banner extension uses.

Nov 29 2022, 7:05 PM · MediaWiki-extensions-PdfHandler, Thumbor Migration, Patch-For-Review, Commons, All-and-every-Wikisource, Thumbor

Nov 23 2022

Vlad.shapik added a comment to T253280: Per-file control for some thumbnail generation parameters .

Hello @AntiCompositeNumber.
I am working on the BE solution to make it possible to configure the DPI option for PDF files. It is the sub-ticket T256959 of this one.
As I understood correctly you proposed that we have these configuration options already put in the structured data tab on the file page. And then thumbor will take possible configuration options and use them to generate the file on a case-by-case basis.
Did I get your point? Could you please share your ideas with me?

Nov 23 2022, 5:48 PM · Thumbor Migration, Commons, MediaWiki-File-management, SDC General, Wikidata, Thumbor
Vlad.shapik claimed T256959: Allow PDF's to be rendered at higher (or user specified DPI).
Nov 23 2022, 5:45 PM · MediaWiki-extensions-PdfHandler, Thumbor Migration, Patch-For-Review, Commons, All-and-every-Wikisource, Thumbor
Vlad.shapik added a comment to T256959: Allow PDF's to be rendered at higher (or user specified DPI).

Hello @ShakespeareFan00
I am working on this ticket now. Could you please share with me the location of a 'property' for the File:'s entry in the image links table which you mentioned as a possible way to implement the DPI option in the FE?
And also will you be able to describe the idea of a 'modifier' template for use on sites like Wikimedia Commons and Wikisource?
For now, I just can't understand the connection between these two steps because I do not how the DPI option should look on these sites. Maybe you could give me some examples of already implemented options on these sites or something like that?
I just need to know if the front-end team(for now I don't know who will implement it in the FE) can use my BE solution to implement this configurator on the file page, for instance, that's why I am asking.

Nov 23 2022, 5:35 PM · MediaWiki-extensions-PdfHandler, Thumbor Migration, Patch-For-Review, Commons, All-and-every-Wikisource, Thumbor

Nov 21 2022

Vlad.shapik updated the task description for T323530: Cover encoding of headers in the images handler with a test/tests.
Nov 21 2022, 7:46 PM · Platform Team Workboards (Platform Engineering Reliability), Thumbor, Thumbor Migration
Vlad.shapik added a project to T323530: Cover encoding of headers in the images handler with a test/tests: Thumbor.
Nov 21 2022, 7:44 PM · Platform Team Workboards (Platform Engineering Reliability), Thumbor, Thumbor Migration
Vlad.shapik created T323530: Cover encoding of headers in the images handler with a test/tests.
Nov 21 2022, 7:42 PM · Platform Team Workboards (Platform Engineering Reliability), Thumbor, Thumbor Migration
Vlad.shapik added a comment to T323114: Encoding issues when handling unicode characters in filenames.

This appears to be fixed. The issue relates to us calling Tornado's set_header with a string that contains non-ascii characters. Encoding these strings to be utf-8 bytestrings fixes the issue.

Nov 21 2022, 6:10 PM · Patch-For-Review, Platform Team Workboards (Platform Engineering Reliability), Thumbor Migration, SRE, serviceops, Thumbor

Nov 15 2022

Vlad.shapik moved T323114: Encoding issues when handling unicode characters in filenames from Backlog to In Progress on the Thumbor Migration board.
Nov 15 2022, 4:02 PM · Patch-For-Review, Platform Team Workboards (Platform Engineering Reliability), Thumbor Migration, SRE, serviceops, Thumbor

Nov 3 2022

Vlad.shapik claimed T253280: Per-file control for some thumbnail generation parameters .
Nov 3 2022, 5:46 PM · Thumbor Migration, Commons, MediaWiki-File-management, SDC General, Wikidata, Thumbor
Vlad.shapik moved T318406: Providing additional coverage based on the code coverage research results from In Progress to Done on the Thumbor Migration board.

Change 841183 merged by jenkins-bot:

[operations/software/thumbor-plugins@master] Provide additional tests to cover errors caused by wrong engine commands

https://gerrit.wikimedia.org/r/841183

Nov 3 2022, 3:18 PM · Test-Coverage, Thumbor Migration

Oct 28 2022

Vlad.shapik moved T253280: Per-file control for some thumbnail generation parameters from Backlog to In Progress on the Thumbor Migration board.
Oct 28 2022, 11:36 AM · Thumbor Migration, Commons, MediaWiki-File-management, SDC General, Wikidata, Thumbor
Vlad.shapik moved T318548: Incorrect orientation of some thumbnails from In Progress to Done on the Thumbor Migration board.
Oct 28 2022, 11:36 AM · Thumbor Migration, Thumbor

Oct 27 2022

Vlad.shapik added a comment to T318548: Incorrect orientation of some thumbnails.

So, I think the difference I am seeing is because I had not set RESPECT_ORIENTATION = True in my local thumbor config file. That option is set on beta. The default for that option is False.

When I add that option to my local thumbor config, the 4 files mentioned in the description are rendered in the correct orientation.

@roman-stolar @Vlad.shapik please double-check, but I think this bug is a false alarm.

Oct 27 2022, 6:26 PM · Thumbor Migration, Thumbor

Oct 25 2022

Vlad.shapik added a comment to T318406: Providing additional coverage based on the code coverage research results .

The code coverage has increased from 75% to 76%. It means that 11 lines of errors related code have been covered by tests.

Oct 25 2022, 11:21 AM · Test-Coverage, Thumbor Migration

Oct 24 2022

Vlad.shapik claimed T318548: Incorrect orientation of some thumbnails.
Oct 24 2022, 4:41 PM · Thumbor Migration, Thumbor

Oct 14 2022

Vlad.shapik renamed T318406: Providing additional coverage based on the code coverage research results from Providing additional coverage based on the test coverage research results to Providing additional coverage based on the code coverage research results .
Oct 14 2022, 11:36 AM · Test-Coverage, Thumbor Migration

Oct 11 2022

Vlad.shapik moved T313016: Investigate Code Coverage with tests for all the code from In Progress to Done on the Thumbor Migration board.
Oct 11 2022, 1:41 PM · Test-Coverage, Thumbor Migration

Sep 23 2022

Vlad.shapik added a comment to T313016: Investigate Code Coverage with tests for all the code.

@pwangai and I have been reviewing the code coverage. We found we could run coverage run -m pytest || coverage html where you can see line-by-line code coverage.

The areas we seem to be missing code coverage:

  • PoolCounter (https://wikitech.wikimedia.org/wiki/Thumbor#Poolcounter-based; code wikimedia_thumbor/poolcounter/__init__.py)
  • Swift (code wikimedia_thumbor/loader/swift/__init__.py, wikimedia_thumbor/result_storage/swift/swift.py)
  • Error/exception handling
  • Several files in wikimedia_thumbor/logging/
  • Several files in wikimedia_thumbor/error_handlers/
  • wikimedia_thumbor/storage/request/request.py (not sure what this does)
  • wikimedia_thumbor/core/importer.py (not sure what this does)
  • wikimedia_thumbor/core/context_importer.py (not sure what this does)

Testing PoolCounter and Swift might not be possible without getting these services running locally. This might be hard, particularly with Swift.

Sep 23 2022, 1:09 PM · Test-Coverage, Thumbor Migration
Vlad.shapik claimed T313016: Investigate Code Coverage with tests for all the code.
Sep 23 2022, 6:59 AM · Test-Coverage, Thumbor Migration

Sep 13 2022

Vlad.shapik moved T313016: Investigate Code Coverage with tests for all the code from To Do to In Progress on the Thumbor Migration board.
Sep 13 2022, 4:47 PM · Test-Coverage, Thumbor Migration
Vlad.shapik moved T314393: Division operation fix for Python 3+ from In Progress to Done on the Thumbor Migration board.
Sep 13 2022, 4:47 PM · Python3-Porting, Thumbor Migration

Aug 25 2022

Vlad.shapik renamed T316279: Fix not triggering mock_get_object() in test_swift.py from Fix not triggering of mock_get_object() in test_swift.py to Fix not triggering mock_get_object() in test_swift.py.
Aug 25 2022, 6:29 PM · Thumbor, Thumbor Migration
Vlad.shapik added a comment to T252719: Upgrade thumbor to Thumbor 7 and python3.

Hi, I'm trying to understand https://gerrit.wikimedia.org/r/c/operations/software/thumbor-plugins/+/800170/ a bit better.

  • What parts of this change are coming from upstream and what parts are new?
  • How did https://gerrit.wikimedia.org/r/c/operations/software/thumbor-plugins/+/489022/6 ended up getting included, when it has not (AFAICT) been merged?
  • Did the new code get thoroughly reviewed? I only looked at one file, tests/integration/test_swift.py, and it looks like the change made at least some of the test code unreachable -- adding assert False to mock_get_object() does not result in a test failure.
Aug 25 2022, 6:25 PM · Patch-For-Review, Thumbor Migration, Python3-Porting
Vlad.shapik renamed T316279: Fix not triggering mock_get_object() in test_swift.py from Fix not triggering mock_get_object() in test_swift.py to Fix not triggering of mock_get_object() in test_swift.py.
Aug 25 2022, 6:11 PM · Thumbor, Thumbor Migration
Vlad.shapik updated the task description for T316279: Fix not triggering mock_get_object() in test_swift.py.
Aug 25 2022, 6:05 PM · Thumbor, Thumbor Migration
Vlad.shapik updated the task description for T316279: Fix not triggering mock_get_object() in test_swift.py.
Aug 25 2022, 6:04 PM · Thumbor, Thumbor Migration
Vlad.shapik created T316279: Fix not triggering mock_get_object() in test_swift.py.
Aug 25 2022, 6:03 PM · Thumbor, Thumbor Migration
Vlad.shapik moved T314393: Division operation fix for Python 3+ from To Do to In Progress on the Thumbor Migration board.
Aug 25 2022, 5:45 PM · Python3-Porting, Thumbor Migration

Aug 19 2022

Vlad.shapik added a comment to T252719: Upgrade thumbor to Thumbor 7 and python3.

Hi @ori.
I will create a new patch with reverted changes related to https://gerrit.wikimedia.org/r/c/operations/software/thumbor-plugins/+/489022/6

Aug 19 2022, 2:07 PM · Patch-For-Review, Thumbor Migration, Python3-Porting
Vlad.shapik renamed T314657: Setup Debugger for Thumbor from Setup debugpy package for Thumbor to Setup Debugger for Thumbor.
Aug 19 2022, 9:47 AM · Thumbor Migration
Vlad.shapik moved T314657: Setup Debugger for Thumbor from In Progress to Done on the Thumbor Migration board.
Aug 19 2022, 9:46 AM · Thumbor Migration

Aug 17 2022

Vlad.shapik reopened T312919: Investigate whether these tests are currently correct and fit for purpose as "Open".
Aug 17 2022, 12:59 PM · Thumbor Migration
Vlad.shapik closed T312919: Investigate whether these tests are currently correct and fit for purpose as Resolved.

The results of the investigation:

  1. In general current tests work as expected and it doesn't seem that they are written wrong.
  2. Regarding the integration testing, most of the tests use one test case for testing - run_and_check_ssim_and_size. The number of testing parameters in this function can be extended which could bring better test cases. Additionally, I’ve noticed that “tests_3d.py” is not an online test, it has a local URL. And what's more, it can be as one of the variants to create more tests using different filters:(parameter), format:(parameter), etc of the URL for the Thumbor service to get a converted image.
  3. As for unit tests, it also can be taken into consideration because very few functions of the source code are used in the existing tests. For now, it is only an 80% assumption which has appeared after working with tests. But the picture will be clearer after the code coverage check.
Aug 17 2022, 12:54 PM · Thumbor Migration