Page MenuHomePhabricator

Global memory leak in lib/Scraper.js
Closed, ResolvedPublic8 Estimated Story Points

Description

Detected when running test in this unmerged change: https://gerrit.wikimedia.org/r/#/c/198491/

Likely has something to do with scraping open graph metadata.

Event Timeline

Mvolz raised the priority of this task from to Needs Triage.
Mvolz updated the task description. (Show Details)
Mvolz added a project: Citoid.
Mvolz moved this task to Service on the Citoid board.
Mvolz subscribed.

Mocha detects global memory leak when scraping the following url: http://www.pbs.org/newshour/making-sense/care-peoples-kids/

Error: global leak detected: content
    at Runner.checkGlobals (/home/marielle/Code/Node/citoid/node_modules/grunt-simple-mocha/node_modules/mocha/lib/runner.js:181:21)
    at Runner.<anonymous> (/home/marielle/Code/Node/citoid/node_modules/grunt-simple-mocha/node_modules/mocha/lib/runner.js:60:44)
    at Runner.EventEmitter.emit (events.js:117:20)
    at /home/marielle/Code/Node/citoid/node_modules/grunt-simple-mocha/node_modules/mocha/lib/runner.js:463:14
    at done (/home/marielle/Code/Node/citoid/node_modules/grunt-simple-mocha/node_modules/mocha/lib/runnable.js:207:5)
    at /home/marielle/Code/Node/citoid/node_modules/grunt-simple-mocha/node_modules/mocha/lib/runnable.js:227:9
    at /home/marielle/Code/Node/citoid/test/index.js:227:4
    at /home/marielle/Code/Node/citoid/lib/Scraper.js:68:7
    at Scraper.parseHTML (/home/marielle/Code/Node/citoid/lib/Scraper.js:161:2)
    at Request._callback (/home/marielle/Code/Node/citoid/lib/Scraper.js:65:14)
Mvolz triaged this task as Unbreak Now! priority.
Mvolz set Security to None.

This was caused by running an outdated version of html-metadata.

Need to a) bump html-metadata HEAD to new version

b) Update citoid package.json to use new version

Change 198553 had a related patch set uploaded (by Mvolz):
Fix global memory leak in open graph

https://gerrit.wikimedia.org/r/198553

Change 198553 merged by jenkins-bot:
Fix global memory leak when scraping OG data

https://gerrit.wikimedia.org/r/198553

mobrovac removed a project: Patch-For-Review.
mobrovac subscribed.

In master, resolving. To be deployed once T93337 is fixed.

@mobrovac This can (should) be deployed before we validate language codes, as a temporary fix.

@mobrovac, sorry, so that bug has multiple patches, I was referring to the only one of them; the WIP change doesn't need to be deployed immediately.

@mobrovac, sorry, so that bug has multiple patches, I was referring to the only one of them; the WIP change doesn't need to be deployed immediately.

I think the WIP can be merged soon, with just some little adjustments, so better to deploy them together.