• create github user: https://github.com/drtrigon

• google calendar... email invite !

T129611: [GSoC 2016 Proposal] Port catimages.py to pywikibot-core, notes:

• about replacing packages code with others; one main concer was to have an infrastructure that allows to use the newest experimental packages as well (e.g. the C code make and compile functionallity) that is available some specific interface (e.g. C) only - we should still support this in an python module that allows to do that for any chosen code
• the same holds for the package install and patch apply functionallity

• move meeting to fri

• learn about:

• T132122: Enable isort for pywikibot import rules
• http://github.com/coala-analyzer/coala

• read letter/mail:

Hi DrTrigon,

Some people didn't receive the initial email sent about 15 hours ago so we are resending this to all OAs and Mentors, apologies for the duplicate emails.

Thank you all for being Mentors and OAs for GSoC 2016! We value your commitment to GSoC and your enthusiasm for helping students learn more about open source. All of you are the reason this program has continued to grow and evolve since 2005. Thank you.

We rely on you to help enforce the rules and to keep your students on course over the next 4 months. We want to help you build long term contributors to your projects and have made some changes/clarifications to the community bonding period that should help everyone be more clear on what is expected of them.

This is a very long email but contains very important information, please read it carefully.

Community Bonding Period

As part of the student’s acceptance into GSoC they are expected to actively participate in the Community Bonding period (April 22 - May 22). The Community Bonding period is intended to get students ready to start contributing to your organization full time in May.

Unfortunately, some students think their acceptance into GSoC guarantees them the initial $500 payment. That is not the case. A student does not receive $500 just for writing a good proposal. They must be active in the Community Bonding period to earn that $500.

Community Bonding activities may involve:

• Becoming familiar with the community practices and processes. (This often involves a mix of observation and participation.)
• Participating on Mailing Lists / IRC / etc. (Not just lurking.)
• Setting up their development environment.
• Small (or large) patches/bug fixes. (These do not need to be directly related to their GSoC project.)
• Participating in code reviews for others. (Even someone who isn't familiar with the project can contribute by pointing out potential inefficiencies, bad error handling, etc.)
• Working with their mentor and other org members on refining their project plan. This might include finalizing deadlines and milestones, adding more detail, figuring out potential issues, etc.
• If the student is already familiar with the organization, they could be helping others get involved in the community.
• Reading (and updating!) documentation they will need to understand to complete their project.
• Reporting or replicating bugs.

Active means active. Students have committed to the program schedule, and we would like you to hold them to it. There is no simple standard, as every org is different, every student has different time constraints, and there are many different ways to interact. Some students may require coaxing and encouragement in order to get them to actively participate.

If you do not see regular public interaction from the student, you should strongly encourage it. Public interaction is important -- it is a key principle of open source -- work happens where everyone can see it. Similarly, all work done by the students should be shared in a publicly available repository.

By May 16th if the student has not been active in Community Bonding please notify Google at gsoc-support@google.com to let us know. After a brief investigation, we may remove the student from the program. They will not receive any payments.

Do not feel bad about "failing" a student.
The past eleven years of GSoC have demonstrated that students who don't interact early and often are more likely to fail later. Often they just disappear. We don't want you to waste your time on students who don't care about the project/organization and don’t even attempt to show interest these first few weeks of the program. Small contributions early on are often a very positive signal.

Organizations will not be penalized for failing students who are not fulfilling their responsibilities during the Community Bonding period. We expect there to be students who fail this Community Bonding period, just like we expect there to be some students who fail the midterm and others that fail the final. This is completely normal.

If you have questions as to whether a student has met the guidelines, don't hesitate to ask. Email gsoc-support@google.com with a summary of the student's activities during the Community Bonding period, and we will review it.

Best,
Stephanie, Cat, Mary, Robert, Helen and Josh

http://www.sitepoint.com/google-summer-code-10-minutes-crash-course/

• learn about:

• https://travis-ci.org/:
• https://docs.travis-ci.com/user/languages/python
• http://docs.python-guide.org/en/latest/scenarios/ci/

• dig up my latest (most recent) catimages.py etc. code:

• https://phabricator.wikimedia.org/diffusion/PWBO/browse/master/catimages.py
• https://phabricator.wikimedia.org/diffusion/PWBO/browse/master/externals/
• and found it on old notebook again, copy put onto server as backup as well

• read examples from community bonding by jayvdb from last year:

• T100097: Bonding period report for "Implement OAuth Support for Pywikibot"
• T100149: Community bonding evaluation report for "Implement Flow support in Pywikibot"

• figure out what's wrong with skype: the connection issue seams not to be on my side
• test mumble/plumble (fdroid)

• read about PAWS:

• https://www.mediawiki.org/wiki/Manual:Pywikibot/PAWS
• https://wikitech.wikimedia.org/wiki/PAWS

• jayvdb; how is this related to matplotlib?

• PAWS notebooks are Jupyter notebooks (=IPython) as can be seen from:
  • https://wikitech.wikimedia.org/wiki/PAWS/Introduction#Motivation_for_notebooks
  • http://jupyter.org/
  • https://ipython.org/#jupyter-and-the-future-of-ipython

• check and extend if necessary:

https://etherpad.wikimedia.org/p/Zl7V7KuK7J

Finish by the end of this week:

• T136409: Pywikibot-catimages: Meeting 6 - Fri 3 June 2016 - 12:30 UTC
  • review/install code from pip (e.g. on travis) and report bugs on github: https://pypi.python.org/pypi/file-metadata (do a beta-test)
  • review https://commons.wikimedia.org/wiki/User:AbdealiJKTravis/logs (and look closer at community needs)
    • find good category (as DB)? e.g. faces?
    • https://commons.wikimedia.org/wiki/Commons:Village_pump
  • Consensus on plan and timeline: we have now the outcome of Week 2 (!)

Prepare:

• T135835: Meeting about handling non-pip packages / replace externals module

• note mumble instead of skype as new audio chat option (T135834: Pywikibot-catimages: Meeting 5 - Fri 27 May 2016 - 12:30 UTC, T136409: Pywikibot-catimages: Meeting 6 - Fri 3 June 2016 - 12:30 UTC)
• solve issue with lot of echoes in group chat (caused by me?)
  • try headphones

http://www.worldtimebuddy.com/?qm=1&lid=30,100,1642911,2657896&h=30&date=2016-6-6&sln=21.5-22.5

• read about php bot using exif data:
• Z360#5392:
  • https://commons.wikimedia.org/w/index.php?title=File:Cut_Cat_5_Cable_2.jpg&diff=0&oldid=119285624
  • https://commons.wikimedia.org/wiki/Commons:Bots/Requests/ArndBot

• check email draft:
• Z360#5483: https://commons.wikimedia.org/wiki/User:AbdealiJK/file-metadata/Email

• find uses for barcode - qr code - natural language processing - qrpedia on commons - need a bussines case (see links)
  • check with the people from https://en.wikipedia.org/wiki/Wikipedia:WikiProject_QRpedia
  • mail written by @jayvdb: Z441#5698

• decide last 50% of the gsoc - project plan !!! irc - T129611: [GSoC 2016 Proposal] Port catimages.py to pywikibot-core
  • what from the original proposal do we need (must have) and is essential?
  • what from the original bot is still missing?
  • rest: free!
• do beta-test (in vm - on labs?) - T136985: file-metadata testing log
• think about training for specific database uploads
  • do not train a haarcascade form a set specific to a upload to be done
  • but create a set of e.g. persons that are likely to appear and use image comparing algorithms instead of feature detection (haarcascade, etc.)
  • what was this algorithm for the video copyright project called again?
• think about training in general
  • Z441#5657: SVM for line drawing / graphics detection?
• do facial landmarks give more info than just the face position? e.g. "landmarks"? yes
• Z441#5705

• enable irc channel logging:

T139024: Permission denied when trying to add a channel to log

Test:

• Categorize based on metadata: in progress, needs to output categorization hints to console
• Graphics detection code of catimages implemented and improved but not tested on a mixed/random set yet
• @jayvdb and @DrTrigon need to define "categorization": accuracy, etc.
• define clean-up final tasks finish project (docu, test, bugfixes, etc) -> who will be future maintainer? AbdealiJK & DrTrigon ?
• Dependencies, Script -> DrTrigon (other beta-testers?) needs to test install and all scripts again! test also new install hints ;)
  • https://gist.githubusercontent.com/drtrigon/2dcbc5fbac1e00f0f89dec9343994e48/raw/d755448ebfbcbf156ee0b01289c34583db31b1d6/tasks.py
• @jayvdb and @DrTrigon need to define "categorization": accuracy etc.
• check and write code

• video2copyright project meeting
  • 80
  • splitted videos
  • python wrapper vs .net ones
  • pydotnet or else ironpython idea still alive

• afraid but do not have time today for "video2copyright project meeting" - could we find another date? else I try to make it

-> problem - doch heute

• next want to check scripts

(run it on eth-bib set!!)
(check and write code - fork and update README.rst in file-metadata)
(change usage syntax in simple_bot.py docu string)
(check my todos!)

osboxes@osboxes:~/core$ python bulk.py -search:'eth-bib' -logname:ETH-Bib -> move this and output to testing log

pywikibot search page generator should output number of matches before starting

bot script and/or bulk.py:

• does the bot script and/or bulk.py write local logs e.g. conatining errors, warnings, etc. occured?
  • use pywikibot logging for the bot script
  • use tee for bulk.py
• retry downloading if errors occur (or check filesize)

$ python -u bulk.py -search:'eth-bib' -logname:ETH-Bib 2>&1 | tee bulk.log

osboxes@osboxes:~/core$ ps -aux | grep "python"
root 1960 0.0 0.2 76308 4580 ? S 14:11 0:00 /usr/bin/python /usr/share/apt-xapian-index/update-apt-xapian-index-dbus
osboxes 21607 34.5 8.8 1045072 182028 pts/9 Sl+ 15:45 44:13 python bulk.py -search:eth-bib -logname:ETH-Bib
osboxes 23479 0.0 0.1 11756 2188 pts/0 S+ 17:53 0:00 grep --color=auto python
osboxes@osboxes:~/core$ kill -STOP 21607
osboxes@osboxes:~/core$ kill -CONT 21607

• abdelaijk's docker has open-cv isues:
  • https://travis-ci.org/pywikibot-catfiles/docker-file-metadata/jobs/145192714
  • https://github.com/pywikibot-catfiles/docker-file-metadata/pull/6
  • switch default branch to that one and try to use my dockerfiles
• have still some install issues with the scripts in the catimages-gsoc docker and direct install through spm or pip: https://travis-ci.org/drtrigon/catimages-gsoc/builds/145395518
  • #53.3 failed
  • #53.4 failed
• was able to run for eth-bib partially but not completely - need to run that on labs
  • find key files on old notebook
• http://zulip.readthedocs.io/en/latest/mypy.html

• need categories stats - discriminate the file type cats from others
• graphics detection into other direction, lot of shades, gradients/contours, etc. -> Category:Photographs_by_camera

• opencv text detection and recognition (for car plates e.g. - may be needs cv3):
  • http://www.pyimagesearch.com/2015/06/22/install-opencv-3-0-and-python-2-7-on-ubuntu/
  • http://rodrigoberriel.com/2014/10/installing-opencv-3-0-0-on-ubuntu-14-04/
  • http://www.samontab.com/web/2014/06/installing-opencv-2-4-9-in-ubuntu-14-04-lts/

• enable conda opencv install on labs and re-run eth-bib job
  • https://bots.wmflabs.org/~wm-bot/logs/%23gsoc-catimages/20160803.txt
  • jsub -o $HOME/log_bot_eth-bib.log -j y -N log_bot_eth-bib python $HOME/file-metadata/file_metadata/wikibot/log_bot.py -search:'eth-bib' -logname:eth-bib
• adopt catimages-gsoc docker according to how the test is run on labs and renamed scripts
  • https://phabricator.wikimedia.org/Z441#6320
• Questions for @AbdealiJK:
  • unittest: why and when are they skipped? what about skipping the ones for opencv?
  • implement haarcascade in skilearn; what about implementing T141633 (the wavelet paper) too?
  • do we have any kind of segmentation implemented in file-metadata?
  • what about writing a qt client for file-metadata? can that be combined with the web-client @AbdealiJK plans to write?

From https://github.com/pywikibot-catfiles/file-metadata/blob/master/.travis.yml#L27:

drtrigon@tools-bastion-03:~$ wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh -O miniconda.sh
drtrigon@tools-bastion-03:~$ bash miniconda.sh -f -b -p $HOME/miniconda
...
$ conda list
...

see also http://conda.pydata.org/docs/using/pkgs.html

Switch bot login to OAuth: https://www.mediawiki.org/wiki/Manual:Pywikibot/OAuth/Wikimedia

• https://tools.wmflabs.org/wikidata-game/#
• https://tools.wmflabs.org/widar/?oauth_verifier=42b3aa4e6461e282da652d8059d3901b&oauth_token=0723c0ea224eed2b3a082f38a988e18a

• https://lists.wikimedia.org/pipermail/commons-l/2016-August/007770.html
  • https://cloud.google.com/blog/big-data/2016/05/explore-the-galaxy-of-images-with-cloud-vision-api
  • Google Cloud Vision API is NOT free - our code IS free and gives developpers a framwork to integrate other algorithms easily too
• https://etherpad.wikimedia.org/p/gsoc-catimages-MVP
  • T142818: Assess pywikibot-catfiles MVP
  • MVP lockdown: Mo-Tue
  • MVP sign DrTrigon then check jayvdb
• https://developers.google.com/open-source/gsoc/help/work-product

gsoc dates ... see mail