Page MenuHomePhabricator

Create a tool to upload to Commons icons previously downloaded from The Noun Project
Open, MediumPublic

Description

The Noun Project holds over 1 million openly licensed icons, however the options for mass importing them are very limited (terms of use for API).

A solution would be to create a tool that helps people upload to Commons icons previously downloaded from The Noun Project. The tool should allow to upload one or many icons as needed whilst making sure the upload adds relevant information to the file on Commons. See an example of information added here with this file imported through a bot.

Also, CC BY-SA icons have the credit within the file written under the icon. This should be removed as the template added to the icon during upload already includes credit: name, author, link, etc.

Use the uploader: https://thenoun-uploader.toolforge.org

Next steps:

  • Wait for the Oauth consumer to be approved.
  • Troubleshoot autoloading issue in ToolForge related to Composer
  • Test the tool in production and update the source code
  • Switch from writing on sandbox to uploading the actual SVG file

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 10 2018, 11:02 AM
samuelguebo updated the task description. (Show Details)Mar 11 2018, 12:44 PM

Thanks @Mrjohncummings , I updated the description.

samuelguebo renamed this task from Create a tool to upload images from The Noun Project to Create a tool to upload icons from The Noun Project to Commons.Mar 11 2018, 12:47 PM
samuelguebo renamed this task from Create a tool to upload icons from The Noun Project to Commons to Create a tool to upload to Commons icons previously downloaded from The Noun Project.Mar 11 2018, 12:54 PM
samuelguebo updated the task description. (Show Details)
samuelguebo updated the task description. (Show Details)Mar 11 2018, 7:45 PM
bd808 awarded a token.Mar 15 2018, 3:52 AM
samuelguebo updated the task description. (Show Details)Mar 26 2018, 10:08 PM
samuelguebo added a subscriber: Eugene233.
samuelguebo updated the task description. (Show Details)Mar 26 2018, 10:10 PM

@samuelguebo: Hi! This task has been assigned to you a while ago. Could you maybe share an update? Do you still plan to work on this task, or do you need any help?

If this task has been resolved in the meantime: Please update the task status (via Add Action...Change Status in the dropdown menu).
If this task is not resolved and only if you do not plan to work on this task anymore: Please consider removing yourself as assignee (via Add Action...Assign / Claim in the dropdown menu): That would allow others to work on this (in theory), as others won't think that someone is already working on this. Thanks! :)

samuelguebo added a comment.EditedMay 17 2020, 6:30 PM

@samuelguebo: Hi! This task has been assigned to you a while ago. Could you maybe share an update? Do you still plan to work on this task, or do you need any help?

If this task has been resolved in the meantime: Please update the task status (via Add Action...Change Status in the dropdown menu).
If this task is not resolved and only if you do not plan to work on this task anymore: Please consider removing yourself as assignee (via Add Action...Assign / Claim in the dropdown menu): That would allow others to work on this (in theory), as others won't think that someone is already working on this. Thanks! :)

Thanks for the follow up @Aklapper, really appreciated. I resumed working on it a few weeks ago. The project is now in a public repo: https://github.com/samuelguebo/thenoun-uploader

I will make sure to update this ticket as work progresses.

samuelguebo triaged this task as Medium priority.May 17 2020, 6:32 PM

@samuelguebo: Hi! Thank you so much for the effort to build this tool. However, I tried to check your repo and noticed that there's no test and the current code is not at all ES6+. Would you accept any help?

Thanks.

@samuelguebo: Hi! Thank you so much for the effort to build this tool. However, I tried to check your repo and noticed that there's no test and the current code is not at all ES6+. Would you accept any help?

Thanks.

Hey @BamLifa, sure! The more the merrier. I'm focusing on the Oauth part for now but any help with the test part or any other aspects is welcome.

Aklapper removed samuelguebo as the assignee of this task.Jun 19 2020, 4:19 PM
Aklapper added a subscriber: samuelguebo.

This task has been assigned to the same task owner for more than two years. Resetting task assignee due to inactivity, to decrease task cookie-licking and to get a slightly more realistic overview of plans. Please feel free to assign this task to yourself again if you still realistically work or plan to work on this task - it would be welcome!

For tips how to manage individual work in Phabricator (noisy notifications, lists of task, etc.), see https://phabricator.wikimedia.org/T228575#6237124 for available options.
(For the records, two emails were sent to assignee addresses before resetting assignees. See T228575 for more info and for potential feedback. Thanks!)

Whoops, readding per T189368#6143295 - sorry!

Quick update: I managed to have a proof-of-concept version that worked locally. I'm in the process of deploying it to ToolForge (https://thenoun-uploader.toolforge.org/).

For now, the uploader writes the details of the uploaded icons on test sandbox (https://commons.wikimedia.org/wiki/User:African_Hope/TheNounProject).
Once the tests in production are conclusive, the logic will be replaced with the actual upload process, so as not to disrupt Commons.

Next steps:

  • Wait for the Oauth consumer to be approved.
  • Test the tool in production and update the source code which is available here: https://github.com/samuelguebo/thenoun-uploader
  • Troubleshoot an autoloading issue in ToolForge related to Composer.
samuelguebo updated the task description. (Show Details)Jul 24 2020, 10:00 PM

@samuelguebo this is really great, please let me know where I can help as a non technical person, happy to be a tester

One note to add about the upload, it would be helpful if the borders around the icons could be consistent so they could be used together, the credit that is included in the file means the border spacing at the bottom at least will be a bit weird if not fixed

In T189368#6344141, @Mrjohncummings wrote:

@samuelguebo this is really great, please let me know where I can help as a non technical person, happy to be a tester

Hello @Mrjohncummings, for the test part you can try to download a couple of icons from thenounproject and try to simulate an import to Commons by using the new uploader available here : https://thenoun-uploader.toolforge.org

I said simulate because, for now, when the upload is successful, the SVG file isn't directly available on Commons. Instead the content (template, description, etc) is inserted in a sandbox: https://commons.wikimedia.org/wiki/User:African_Hope/TheNounProject

Once we're comfortable with the metadata, I can adjust the workflow to actually import the icons to Commons. I did not want to flood Commons with test files.

In T189368#6344144, @Mrjohncummings wrote:

One note to add about the upload, it would be helpful if the borders around the icons could be consistent so they could be used together, the credit that is included in the file means the border spacing at the bottom at least will be a bit weird if not fixed

Although I managed to remove the in-line credits, I haven't figured out yet how to adjust the spacing as it means changing dynamically the dimension/shapes SVG. I'll do so digging later.

Cool, I wonder if this has already been worked out by someone or if there is a bot for it, I assume there are a lot of SVGs uploaded to commons with weird white spaces

In T189368#6372898, @Mrjohncummings wrote:

Cool, I wonder if this has already been worked out by someone or if there is a bot for it, I assume there are a lot of SVGs uploaded to commons with weird white spaces

I found a way to work around the white space issue by using a built-in feature of SVG (Viewbox).

I'll update the code base to actually upload SVG properly now. @Mrjohncummings, do you mind looking here and making sure this template tag generated during the test looks fine: https://commons.wikimedia.org/w/index.php?title=User:African_Hope/TheNounProject&oldid=444213781

samuelguebo updated the task description. (Show Details)Aug 27 2020, 11:55 PM
samuelguebo updated the task description. (Show Details)
samuelguebo updated the task description. (Show Details)Sep 19 2020, 4:22 PM

@Mrjohncummings, I've run a little batch test today with 5 icons I downloaded from Thenounproject.com some minutes earlier. All the uploads went fine. Still seeing some minor white space but overall the SVG look decent to me: https://commons.wikimedia.org/wiki/Special:ListFiles?limit=20&user=African+Hope

Feel free to give the tool a try and if it works as appropriate, we can close this task (and file any eventual bugs later).