Page MenuHomePhabricator

Kernel consistently crashing
Closed, ResolvedPublic

Description

I've created a PAWS notebook (https://paws.wmflabs.org/paws/user/Nettrom/notebooks/Article%20Importance%20Feasibility.ipynb) as part of our article importance project (which is further documented here: https://meta.wikimedia.org/wiki/Research:Automated_classification_of_article_importance). It is now at the stage where the underlying kernel consistently crashes during data import. As I do not know how to figure out why it crashes, I'm opening this ticket.

Note that the notebook does contain remnants of earlier work where the dataset was not imported correctly, so please ignore anything except the first two cells of R code.

Event Timeline

Nettrom renamed this task from Kernel consistently crashing (out of memory issue?) to Kernel consistently crashing.Jan 4 2017, 8:10 PM
Nettrom created this task.

I'm pretty sure it's just running out of memory. How big is the data being imported?

3,746,600 rows. The file I'm importing is 259MiB when unzipped.

Chicocvenancio claimed this task.
Chicocvenancio subscribed.

@Nettrom I'm marking this as resolved as I see you used a subset of the data on the notebook. The larger issues "how much ram can each notebook use", and "how to show resource overconsumption to users" might need further treatment.