Page MenuHomePhabricator

populate_image_table job crashes with ImportError: No module named builtins
Closed, ResolvedPublic

Description

This has been failing for a year with

Traceback (most recent call last):
  File "/data/project/heritage/erfgoedbot/populate_image_table.py", line 35, in <module>
    import common as common
  File "/mnt/nfs/labstore-secondary-tools-project/heritage/heritage/erfgoedbot/common.py", line 8, in <module>
    from builtins import open
ImportError: No module named builtins
CRITICAL: Closing network session.
<type 'exceptions.ImportError'>

Event Timeline

Can’t reproduce locally…

$ docker-compose run --rm bot python erfgoedbot/populate_image_table.py
Starting labs-tools-heritage_db_1 ... done
Working on all countrycodes
Found 96 countries with monument tracker templates to work on

Oh, of course, this does not make use of the virtual environment >_>

Lokal_Profil subscribed.

This one was solved using the run-any-script bash, wasn't it?

Although looking in the logs I see <type 'exceptions.KeyboardInterrupt'> which suggests a similar problem to T203417: update_database crashing due to KeyboardInterrupt

Although looking in the logs I see <type 'exceptions.KeyboardInterrupt'> which suggests a similar problem to T203417: update_database crashing due to KeyboardInterrupt

Ah, no, this one is me killing the script :) I realized with @Multichill that 4473c945 was a bad idea − having one API call for every single image makes it so slow that it is unusable.

I commented out the cron tab, need to remove that commit :-(

Ah. Is that then also the reason why so few countries are being (image) harvested? (see T206043#4636525)

Change 468928 had a related patch set uploaded (by Lokal Profil; owner: Lokal Profil):
[labs/tools/heritage@master] Revert "Harvest whether an image is geolocated in the image table"

https://gerrit.wikimedia.org/r/468928

@Braveheart pinging you as the one who originally requested we add whether image is geocoded. With this revert that will unfortunately be lost.

@Lokal_Profil thanks for the ping - I think that request was in order to evaluate the pictures we already have, but I don't think this was very important to begin with (or maybe I'm forgetting something obvious/important... ).

@JeanFred looks like this was resolved?

The original problem was resolved, I believe there are still some issues but they are tracked in other tasks

Change 468928 abandoned by Lokal Profil:
[labs/tools/heritage@master] Revert "Harvest whether an image is geolocated in the image table"

Reason:
2 years old and obviously the revert can't have been that urgent

https://gerrit.wikimedia.org/r/468928