Page MenuHomePhabricator

Build w_cache data in an intermediate file to allow reuse
Closed, ResolvedPublic

Description

Feature extraction is expensive, and sometimes we want to re-run training and data collection with only slight variations. In these cases, it's convenient that any existing w_cache file be supplied to extraction, and we can reuse most of the data.

I'm imagining something like,

aawiki.labeled.w_cache: aawiki.labeled
    revscoring extract --cache=$@ --input=$^ > .tmp.$@
    && mv .tmp.$@ $@

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Halfak reopened this task as Open.
Halfak closed this task as Resolved.