Page MenuHomePhabricator

Running data_ingestion.py depends on tests/data/csv_ingestion.csv
Closed, ResolvedPublic

Description

Running data_ingestion requires the file 'tests/data/csv_ingestion.csv' relative to CWD, which doesnt exist except when the bot operator is calling the script whilst CWD is the scripts directory.

$ python pwb.py scripts/data_ingestion.py
Traceback (most recent call last):

File "pwb.py", line 157, in <module>
  run_python_file(fn, argv, argvu)
File "pwb.py", line 67, in run_python_file
  exec(compile(source, filename, "exec"), main_mod.__dict__)
File "scripts/data_ingestion.py", line 144, in <module>
  reader = CSVReader(open('tests/data/csv_ingestion.csv'), 'url')

IOError: [Errno 2] No such file or directory: 'tests/data/csv_ingestion.csv'
<type 'exceptions.IOError'>
CRITICAL: Waiting for 1 network thread(s) to finish. Press ctrl-c to abort

There is a test, which isnt run as part of the test suite:
scripts/tests/test_data_ingestion.py

And the data file does exist as:
scripts/tests/data/csv_ingestion.csv


Version: core-(2.0)
Severity: normal

Details

Reference
bz68611

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 3:36 AM
bzimport set Reference to bz68611.
bzimport added a subscriber: Unknown Object (????).

audiodude wrote:

I'd like to work on this bug. Do you have instructions on how to run the failing test?

Change 151997 had a related patch set uploaded by Audiodude:
data_ingestion.py: Fix data path to be relative to script.

https://gerrit.wikimedia.org/r/151997

gerritbot subscribed.

Change 185666 had a related patch set uploaded (by John Vandenberg):
Normalise data_ingestion script

https://gerrit.wikimedia.org/r/185666

Patch-For-Review

Change 185666 merged by jenkins-bot:
Normalise data_ingestion script

https://gerrit.wikimedia.org/r/185666

jayvdb claimed this task.