Page MenuHomePhabricator
Create Task
Maniphest T93473

Global memory leak in lib/Scraper.js
Closed, ResolvedPublic8 Estimated Story Points
Actions

Description

Detected when running test in this unmerged change: https://gerrit.wikimedia.org/r/#/c/198491/

Likely has something to do with scraping open graph metadata.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Fix global memory leak when scraping OG datamediawiki/services/citoidmaster+24 -2
Customize query in gerrit

Related Objects

Event Timeline

Mvolz created this task.Mar 21 2015, 4:01 PM
Mvolz raised the priority of this task from to Needs Triage.
Mvolz updated the task description. (Show Details)
Mvolz added a project: Citoid.
Mvolz moved this task to Service on the Citoid board.
Mvolz subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 21 2015, 4:01 PM
Mvolz added a comment.Mar 21 2015, 4:17 PM

Mocha detects global memory leak when scraping the following url: http://www.pbs.org/newshour/making-sense/care-peoples-kids/

Error: global leak detected: content
    at Runner.checkGlobals (/home/marielle/Code/Node/citoid/node_modules/grunt-simple-mocha/node_modules/mocha/lib/runner.js:181:21)
    at Runner.<anonymous> (/home/marielle/Code/Node/citoid/node_modules/grunt-simple-mocha/node_modules/mocha/lib/runner.js:60:44)
    at Runner.EventEmitter.emit (events.js:117:20)
    at /home/marielle/Code/Node/citoid/node_modules/grunt-simple-mocha/node_modules/mocha/lib/runner.js:463:14
    at done (/home/marielle/Code/Node/citoid/node_modules/grunt-simple-mocha/node_modules/mocha/lib/runnable.js:207:5)
    at /home/marielle/Code/Node/citoid/node_modules/grunt-simple-mocha/node_modules/mocha/lib/runnable.js:227:9
    at /home/marielle/Code/Node/citoid/test/index.js:227:4
    at /home/marielle/Code/Node/citoid/lib/Scraper.js:68:7
    at Scraper.parseHTML (/home/marielle/Code/Node/citoid/lib/Scraper.js:161:2)
    at Request._callback (/home/marielle/Code/Node/citoid/lib/Scraper.js:65:14)
Mvolz claimed this task.Mar 22 2015, 11:22 AM
Mvolz triaged this task as Unbreak Now! priority.
Mvolz set Security to None.
Mvolz added a comment.Mar 22 2015, 12:25 PM

This was caused by running an outdated version of html-metadata.

Need to a) bump html-metadata HEAD to new version

b) Update citoid package.json to use new version

gerritbot subscribed.Mar 22 2015, 1:08 PM

Change 198553 had a related patch set uploaded (by Mvolz):
Fix global memory leak in open graph

https://gerrit.wikimedia.org/r/198553

gerritbot added a project: Patch-For-Review.Mar 22 2015, 1:08 PM
gerritbot added a comment.Mar 22 2015, 5:47 PM

Change 198553 merged by jenkins-bot:
Fix global memory leak when scraping OG data

https://gerrit.wikimedia.org/r/198553

• mobrovac closed this task as Resolved.Mar 22 2015, 6:12 PM
• mobrovac removed a project: Patch-For-Review.
• mobrovac subscribed.

In master, resolving. To be deployed once T93337 is fixed.

Mvolz added a comment.Mar 22 2015, 6:49 PM

@mobrovac This can (should) be deployed before we validate language codes, as a temporary fix.

Mvolz added a comment.Mar 22 2015, 6:50 PM

@mobrovac, sorry, so that bug has multiple patches, I was referring to the only one of them; the WIP change doesn't need to be deployed immediately.

• mobrovac added a comment.Mar 22 2015, 6:53 PM
In T93473#1139372, @Mvolz wrote:

@mobrovac, sorry, so that bug has multiple patches, I was referring to the only one of them; the WIP change doesn't need to be deployed immediately.

I think the WIP can be merged soon, with just some little adjustments, so better to deploy them together.

Jdforrester-WMF added a project: WMF-deploy-2015-03-25_(1.25wmf23).Mar 22 2015, 7:03 PM
Jdforrester-WMF added a project: VisualEditor 2014/15 Q3 blockers.Mar 29 2015, 11:41 PM
Jdforrester-WMF edited a custom field.Mar 29 2015, 11:47 PM
Jdforrester-WMF moved this task from Nominated to Done on the VisualEditor 2014/15 Q3 blockers board.
Mvolz mentioned this in rGCIT168578b38279: Fix global memory leak when scraping OG data.Apr 8 2015, 4:11 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits