Page MenuHomePhabricator

Move out opencv from pywikibot-compat
Closed, DeclinedPublic

Description

Compat depends on some external packages which do not appear to be hosted elsewhere, and are only used by compat script catimages.py.

One of them is :
http://git.wikimedia.org/log/pywikibot%2Fopencv.git

Ideally these packages should be published as externally maintained packages, on https://pypi.python.org/ , as this would help with the porting of the catimages.py script.

Microtask for https://phabricator.wikimedia.org/T66838
Related to https://phabricator.wikimedia.org/T76211

Event Timeline

I was taking a look at this issue, and as discussed am making another issue to have an independent discussion on this.
I'm wondering why we even have this package. It seems easier to just use the opencv bindings for python ?

I've used the Pascal dataset before (with matlab). Would it be alright to use VOCDevkit directly rather than write our own C version of it ? We could run it on octave or matlab.
Plus I notice a lot of math functions which are easier in matlab/octave.

Could someone give me a brief overview of what exactly this package aims to do and some background on why it's C ?

To me this seems to be a package that can be replaced by VOCDevkit (for handling images from VOC Pascal) and sklearn (for BoW and precision recall, etc) at the moment.

This will be much harder to package properly, as it is c++.

Now the important question -- which bits of this repo (if any) are needed for catimages.py.

There are two Python classes, in two different .cpp files. bagofwords_classification_python.cpp and posit_python.cpp

If we need both of them, should they each be in a separate repo & separate package?

If bagofwords_classification_python.cpp is needed, then I think it needs to be under the same license as https://github.com/Itseez/opencv/blob/master/LICENSE as code was copied from opencv (which was removed in https://github.com/Itseez/opencv/commit/31df47b6 and is now in https://github.com/Itseez/opencv_contrib ) .

I dont see posit used by catimages.py

@AbdealiJK , if you think the same functionality can be achieved with another library, then we do not need to do this task ;-)

Ah, I see posit is likely used in method _JpegFile._util_get_Pose_POSIT.

Lets expand this tasks scope a little, to include investigating cv2 used by catimages.

catimages.py 's first revision (4e302bae) used cv2 described in the source as

# TS: nonofficial cv2.so backport of the testing-version of
# python-opencv because of missing build-host, done by DaB

My guess is that this "testing-version of python-opencv" is now part of the main distributed library, and there is no need for a custom cv2. But it would be good to find out what "missing build-host" means.

There are a bunch of Python packages that provide access to opencv : https://pypi.python.org/pypi?%3Aaction=search&term=opencv&submit=search

It looks like none of them are the 'real' cv2 , according to http://answers.opencv.org/question/67725/opencv-in-pypi/ , which means catimages.py in core will need a custom ImportError message to inform the user how to install it.

On Fedora, they recommend using yum/dnf to install the operating system package opencv-python. In ubuntu it appears to be called python-opencv. Debian doesnt appear to include it, but Wikimedia doesnt usually use Debian.

@Multichill , the intention is to organise the catimages dependencies, so that it can be ported to core without the old externals system that compat used, and that all dependencies are properly maintained packages with unit tests, etc. i.e. all dependencies should be available via pip and/or yum/apt/dnf/etc.

This will be much harder to package properly, as it is c++.

True, this is part of the reason why I never did it myself. I appreciate your effort!

Now the important question -- which bits of this repo (if any) are needed for catimages.py.

Have you ever had the bot running? That would make answering those questions easier as I did not run it for quite some time and things might have changed.

I dont see posit used by catimages.py

I found 3 appearances:

def _detect_Faces(self):
def _util_get_Pose_POSIT(self, D3points, D2points):
def _detect_Chessboard(self):

@AbdealiJK , if you think the same functionality can be achieved with another library, then we do not need to do this task ;-)

This actually is THE crucial point, see next comment. And another reason why I asked whether you ran the bot once.

Lets expand this tasks scope a little, to include investigating cv2 used by catimages.

catimages.py 's first revision (4e302bae) used cv2 described in the source as

# TS: nonofficial cv2.so backport of the testing-version of
# python-opencv because of missing build-host, done by DaB

My guess is that this "testing-version of python-opencv" is now part of the main distributed library, and there is no need for a custom cv2. But it would be good to find out what "missing build-host" means.

There are a bunch of Python packages that provide access to opencv : https://pypi.python.org/pypi?%3Aaction=search&term=opencv&submit=search

It looks like none of them are the 'real' cv2 , according to http://answers.opencv.org/question/67725/opencv-in-pypi/ , which means catimages.py in core will need a custom ImportError message to inform the user how to install it.

On Fedora, they recommend using yum/dnf to install the operating system package opencv-python. In ubuntu it appears to be called python-opencv. Debian doesnt appear to include it, but Wikimedia doesnt usually use Debian.

As far as I remember we had 2 issues back then:

  1. The ubuntu version used on the toolserver was older than the one on my local machine and thus contained other packages. DaB did this compile manually due to a missing build-host.
  2. No matter how recent the ubuntu version used was, the opencv-python wrapper always had a lack of support for ALL cv2 functions, so I had to write diry/cheap and nasty wrappers myself. This gives the freedom to use THE WHOLE cv2 functionality as the c++ implementation is complete. Here is where e.g. the need for an automated build process came up; I had to make sure that the code got recompiled in case the toolserver would have been updated and thus included libraries changed.
Xqt subscribed.

compat is closed and no longer supported

Dvorapa edited projects, added Pywikibot-compat-to-core; removed Pywikibot-compat.
Dvorapa subscribed.

compat-to-core task

Multichill edited projects, added Pywikibot-compat; removed Pywikibot-compat-to-core.

Declining again. We're not going to do anything here