Page MenuHomePhabricator

Document how to install Python modules in a tool's home directory/virtual environment
Open, LowPublic


I'm building a script to upload into Internet Archive pdf files that need some previous painful manipulation (converting into tiffs, then cropped, splitted and sometines rotated); a good set of metadata (those needed for a good Book template) could be uploaded too.

Reproducible: Didn't try

Version: unspecified
Severity: enhancement



Event Timeline

bzimport raised the priority of this task from to Lowest.Nov 22 2014, 3:05 AM
bzimport added a project: Toolforge.
bzimport set Reference to bz61824.

I forgot to report the main information! Python package that allows Internet Archive management can be found here:

After migrating into eqiad, local installation of internetarchive gives an ImportError and doesn't run any more.

The new message into eqiad is:

from requests.auth import AuthBase
ImportError: cannot import name AuthBase

pip doesn't run in eqiad so we can't even try to fix the issue; careful global installation is needed.

Have you tried installing the Python package in your home directory? Is that possible?

I tried, but pip seems lacking into eqiad; internetarchive installation is not simple, i.e. I got often a (random!) ImportError message when running my test python script with jsub, while I never got it running the same by itsource prompt.

So, a global, careful installation of internetarchive module is a good idea; I think that a better idea could be, to integrate it into pywikibot, given the interest of IA for wikisource projects and possible automatic alignment of IA-Common-Wikidata-Wikisource metadata.

I see that the bug is now assigned to marc Pelletier: thanks for attention! :-)

virtualenv --no-site-packages env
echo "source $HOME/env/bin/activate" >> .bashrc
source $HOME/env/bin/activate
pip install internetarchive

This will work, if you're interested in using virtualenv.

I furiusly browsed python doc to see what virtualenv is; it doesn't seems something so alarming, I'll try. Thanks MZMcBride.

@MZMcBride: Great: env, pip, internetarchive, running! Thanks again.

coren added a comment.Mar 25 2014, 5:59 PM

Re-prioritizing since this only needs documentation.

coren moved this task from Triage to Backlog on the Toolforge board.Nov 25 2014, 4:22 PM
jayvdb added a subscriber: jayvdb.Dec 22 2014, 10:44 AM
coren added a comment.Dec 22 2014, 3:57 PM

I don't know enough about python to be certain how reusable that bit is, but there are instructions on setting up pywikibot in a virtualenv at .

How general is that information?

coren removed coren as the assignee of this task.Mar 25 2015, 8:12 PM

Leaving for a volunteer virtualenv guru.

Restricted Application added a project: Internet-Archive. · View Herald TranscriptOct 24 2015, 8:02 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
bd808 added a subscriber: bd808.Apr 26 2017, 11:01 PM

There are some instructions on setting up a virtualenv related to a webservice at:

These could be combined and cleaned up with some additional information such as links to good external documentation on using virtualenv generally and a note the the www/python/venv is only magical because the uwsgi webservice script will use the directory if it is found.

srodlund claimed this task.Aug 29 2018, 6:31 PM
srodlund added a subscriber: srodlund.

Reviewing as part of initiative to improve Toolforge / Cloud Services Technical Documentation

srodlund raised the priority of this task from Lowest to Low.Aug 29 2018, 10:32 PM