Page Menu
Home
Phabricator
Search
Configure Global Search
Log In
Paste
P5017
Recommendation API profiled
Active
Public
Actions
Authored by
•
schana
on Mar 6 2017, 4:05 PM.
Edit Paste
Archive Paste
View Raw File
Subscribe
Mute Notifications
Award Token
Flag For Later
Tags
Recommendation-API
Referenced Files
F6246258: Recommendation API profiled
Mar 6 2017, 4:05 PM
2017-03-06 16:05:42 (UTC+0)
Subscribers
None
============================= test session starts ==============================
platform darwin -- Python 3.6.0, pytest-3.0.5, py-1.4.32, pluggy-0.4.0 -- /Users/nschaaf/.virtualenvs/recommendation/bin/python
cachedir: .cache
rootdir: /Users/nschaaf/git/recommendation-api, inifile:
collecting ... collected 7 items
recommendation/data/profile_performance.py::test_translation_without_seed 2017-03-06 17:02:44,330 recommendation.api.types.related_articles.candidate_finder initialize():97 INFO -- starting to load embedding
2017-03-06 17:02:46,726 recommendation.api.types.related_articles.candidate_finder initialize():108 INFO -- embedding loaded in 2.395980 seconds
2017-03-06 17:02:46,727 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://wikimedia.org/api/rest_v1/metrics/pageviews/top/en.wikipedia/all-access/2017/03/04
2017-03-06 17:02:46,922 recommendation.api.types.translation.filters apply_filters():45 DEBUG -- Number of candidates: 500
2017-03-06 17:02:46,922 recommendation.api.types.translation.filters apply_filters():47 DEBUG -- Number of candidates after title: 455
2017-03-06 17:02:46,925 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:46,927 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:46,928 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:46,929 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:46,931 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:46,932 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:46,932 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:46,933 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:46,934 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:46,934 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:48,401 recommendation.api.types.translation.filters apply_filters():49 DEBUG -- Number of candidates after missing: 77
2017-03-06 17:02:48,401 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:48,689 recommendation.api.types.translation.filters apply_filters():51 DEBUG -- Number of candidates after disambiguation: 77
2017-03-06 17:02:48,776 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://wikimedia.org/api/rest_v1/metrics/pageviews/top/en.wikipedia/all-access/2017/03/04
2017-03-06 17:02:48,974 recommendation.api.types.translation.filters apply_filters():45 DEBUG -- Number of candidates: 500
2017-03-06 17:02:48,975 recommendation.api.types.translation.filters apply_filters():47 DEBUG -- Number of candidates after title: 461
2017-03-06 17:02:48,982 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:48,983 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:48,984 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:48,984 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:48,985 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:48,985 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:48,985 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:48,985 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:48,985 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:48,985 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:50,257 recommendation.api.types.translation.filters apply_filters():49 DEBUG -- Number of candidates after missing: 67
2017-03-06 17:02:50,258 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:50,530 recommendation.api.types.translation.filters apply_filters():51 DEBUG -- Number of candidates after disambiguation: 66
26591 function calls (26364 primitive calls) in 1.962 seconds
Ordered by: internal time
List reduced from 860 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
107 1.468 0.014 1.468 0.014 {method 'acquire' of '_thread.lock' objects}
26 0.239 0.009 0.239 0.009 {method 'read' of '_ssl._SSLSocket' objects}
2 0.115 0.058 0.115 0.058 {method 'do_handshake' of '_ssl._SSLSocket' objects}
2 0.056 0.028 0.056 0.028 {method 'connect' of '_socket.socket' objects}
2 0.026 0.013 0.026 0.013 {method 'load_verify_locations' of '_ssl._SSLContext' objects}
1 0.007 0.007 1.478 1.478 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
2 0.006 0.003 0.006 0.003 {built-in method _scproxy._get_proxy_settings}
2 0.006 0.003 0.006 0.003 /Users/nschaaf/.virtualenvs/recommendation/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py:414(close)
2 0.004 0.002 0.004 0.002 {built-in method marshal.loads}
2 0.002 0.001 0.002 0.001 {built-in method _socket.gethostbyname}
26591 function calls (26364 primitive calls) in 1.962 seconds
Ordered by: internal time
List reduced from 860 to 34 due to restriction <'recommendation-api'>
List reduced from 34 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.007 0.007 1.478 1.478 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
1 0.001 0.001 0.194 0.194 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/candidate_finders.py:10(get_top_pageview_candidates)
1000 0.001 0.000 0.001 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/recommendation.py:2(__init__)
1 0.001 0.001 0.192 0.192 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/fetcher.py:104(get_most_popular_articles)
455 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:17(<lambda>)
1000 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/candidate_finders.py:14(<lambda>)
1 0.000 0.000 1.478 1.478 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:9(filter_by_missing)
80 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:114(extract_from_raw)
77 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/recommendation.py:10(__dict__)
1 0.000 0.000 0.001 0.001 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:41(<listcomp>)
26591 function calls (26364 primitive calls) in 1.962 seconds
Ordered by: cumulative time
List reduced from 860 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 1.962 1.962 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/translation.py:193(recommend)
1 0.000 0.000 1.768 1.768 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:44(apply_filters)
1 0.000 0.000 1.478 1.478 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:9(filter_by_missing)
1 0.000 0.000 1.478 1.478 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:15(get_items_in_source_missing_in_target_by_titles)
1 0.007 0.007 1.478 1.478 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
1 0.000 0.000 1.470 1.470 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:46(chunk_query_for_parameter)
107 1.468 0.014 1.468 0.014 {method 'acquire' of '_thread.lock' objects}
14 0.000 0.000 1.459 0.104 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py:533(wait)
14 0.000 0.000 1.459 0.104 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py:263(wait)
1 0.000 0.000 1.458 1.458 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py:255(map)
26591 function calls (26364 primitive calls) in 1.962 seconds
Ordered by: cumulative time
List reduced from 860 to 34 due to restriction <'recommendation-api'>
List reduced from 34 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 1.962 1.962 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/translation.py:193(recommend)
1 0.000 0.000 1.768 1.768 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:44(apply_filters)
1 0.000 0.000 1.478 1.478 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:9(filter_by_missing)
1 0.000 0.000 1.478 1.478 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:15(get_items_in_source_missing_in_target_by_titles)
1 0.007 0.007 1.478 1.478 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
1 0.000 0.000 1.470 1.470 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:46(chunk_query_for_parameter)
1 0.000 0.000 0.288 0.288 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:28(filter_by_disambiguation)
1 0.000 0.000 0.288 0.288 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/fetcher.py:33(get_disambiguation_pages)
1 0.000 0.000 0.288 0.288 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/fetcher.py:22(post)
1 0.001 0.001 0.194 0.194 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/candidate_finders.py:10(get_top_pageview_candidates)
Filename: /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/translation.py
Line # Mem usage Increment Line Contents
================================================
193 2554.3203 MiB 0.0000 MiB def recommend(source, target, search, seed, count, include_pageviews, rank_method='default', max_candidates=500):
194 """
195 1. Use finder to select a set of candidate articles
196 2. Filter out candidates that are not missing, are disambiguation pages, etc
197 3. get pageview info for each passing candidate if desired
198 """
199
200 2554.3203 MiB 0.0000 MiB candidates = []
201
202 2554.3203 MiB 0.0000 MiB if seed:
203 finder = finder_map[search]
204 for seed in seed.split('|'):
205 candidates.extend(finder(source, seed, max_candidates))
206 else:
207 2554.9336 MiB 0.6133 MiB candidates.extend(finder_map['mostpopular'](source, seed, max_candidates))
208
209 2566.5625 MiB 11.6289 MiB recs = filters.apply_filters(source, target, candidates)
210
211 2566.5625 MiB 0.0000 MiB recs = sorted(recs, key=lambda x: x.rank, reverse=True)
212
213 2566.5625 MiB 0.0000 MiB if rank_method == 'sitelinks':
214 for rec in recs:
215 rec.rank = rec.sitelink_count
216 recs = sorted(recs, key=lambda x: x.rank, reverse=True)
217
218 2566.5625 MiB 0.0000 MiB recs = recs[:count]
219
220 2566.5625 MiB 0.0000 MiB if recs and include_pageviews:
221 recs = pageviews.set_pageview_data(source, recs)
222
223 2566.5625 MiB 0.0000 MiB return [r.__dict__() for r in recs]
2017-03-06 17:02:50,557 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://wikimedia.org/api/rest_v1/metrics/pageviews/top/en.wikipedia/all-access/2017/03/04
2017-03-06 17:02:50,782 recommendation.api.types.translation.filters apply_filters():45 DEBUG -- Number of candidates: 500
2017-03-06 17:02:50,782 recommendation.api.types.translation.filters apply_filters():47 DEBUG -- Number of candidates after title: 459
2017-03-06 17:02:50,786 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:50,786 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:50,786 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:50,786 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:50,786 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:50,786 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:50,786 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:50,786 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:50,786 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:50,787 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:52,459 recommendation.api.types.translation.filters apply_filters():49 DEBUG -- Number of candidates after missing: 72
2017-03-06 17:02:52,459 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:52,841 recommendation.api.types.translation.filters apply_filters():51 DEBUG -- Number of candidates after disambiguation: 71
PASSED
recommendation/data/profile_performance.py::test_translation_with_seed[Apple] 2017-03-06 17:02:52,851 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:53,149 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:53,566 recommendation.api.types.translation.candidate_finders get_morelike_candidates():39 INFO -- morelike returned 501 results
2017-03-06 17:02:53,567 recommendation.api.types.translation.filters apply_filters():45 DEBUG -- Number of candidates: 500
2017-03-06 17:02:53,567 recommendation.api.types.translation.filters apply_filters():47 DEBUG -- Number of candidates after title: 494
2017-03-06 17:02:53,570 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:53,570 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:53,570 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:53,570 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:53,570 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:53,570 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:53,571 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:53,571 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:53,571 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:53,571 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:54,715 recommendation.api.types.translation.filters apply_filters():49 DEBUG -- Number of candidates after missing: 215
2017-03-06 17:02:54,715 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:54,993 recommendation.api.types.translation.filters apply_filters():51 DEBUG -- Number of candidates after disambiguation: 215
2017-03-06 17:02:55,012 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:55,354 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:55,753 recommendation.api.types.translation.candidate_finders get_morelike_candidates():39 INFO -- morelike returned 501 results
2017-03-06 17:02:55,758 recommendation.api.types.translation.filters apply_filters():45 DEBUG -- Number of candidates: 500
2017-03-06 17:02:55,759 recommendation.api.types.translation.filters apply_filters():47 DEBUG -- Number of candidates after title: 494
2017-03-06 17:02:55,767 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:55,768 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:55,769 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:55,769 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:55,769 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:55,769 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:55,770 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:55,770 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:55,770 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:55,770 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:56,717 recommendation.api.types.translation.filters apply_filters():49 DEBUG -- Number of candidates after missing: 215
2017-03-06 17:02:56,718 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:56,994 recommendation.api.types.translation.filters apply_filters():51 DEBUG -- Number of candidates after disambiguation: 215
27177 function calls (27140 primitive calls) in 2.143 seconds
Ordered by: internal time
List reduced from 698 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
129 1.139 0.009 1.139 0.009 {method 'acquire' of '_thread.lock' objects}
15 0.660 0.044 0.660 0.044 {method 'read' of '_ssl._SSLSocket' objects}
3 0.179 0.060 0.179 0.060 {method 'do_handshake' of '_ssl._SSLSocket' objects}
3 0.083 0.028 0.083 0.028 {method 'connect' of '_socket.socket' objects}
3 0.031 0.010 0.031 0.010 {method 'load_verify_locations' of '_ssl._SSLContext' objects}
3 0.008 0.003 0.009 0.003 /Users/nschaaf/.virtualenvs/recommendation/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py:414(close)
1 0.004 0.004 1.147 1.147 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
3 0.002 0.001 0.002 0.001 {built-in method _socket.gethostbyname}
3 0.002 0.001 0.002 0.001 {built-in method _socket.getaddrinfo}
3 0.001 0.000 0.001 0.000 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/decoder.py:345(raw_decode)
27177 function calls (27140 primitive calls) in 2.143 seconds
Ordered by: internal time
List reduced from 698 to 35 due to restriction <'recommendation-api'>
List reduced from 35 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.004 0.004 1.147 1.147 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
1 0.001 0.001 0.716 0.716 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/candidate_finders.py:27(get_morelike_candidates)
215 0.001 0.000 0.001 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:114(extract_from_raw)
1 0.000 0.000 1.147 1.147 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:9(filter_by_missing)
494 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:17(<lambda>)
215 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/recommendation.py:10(__dict__)
501 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/recommendation.py:2(__init__)
1 0.000 0.000 0.001 0.001 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:41(<listcomp>)
1 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:41(<listcomp>)
215 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/recommendation.py:18(incorporate_wikidata_item)
27177 function calls (27140 primitive calls) in 2.143 seconds
Ordered by: cumulative time
List reduced from 698 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 2.143 2.143 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/translation.py:193(recommend)
1 0.000 0.000 1.426 1.426 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:44(apply_filters)
1 0.000 0.000 1.147 1.147 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:9(filter_by_missing)
1 0.000 0.000 1.147 1.147 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:15(get_items_in_source_missing_in_target_by_titles)
1 0.004 0.004 1.147 1.147 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
1 0.000 0.000 1.141 1.141 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:46(chunk_query_for_parameter)
129 1.139 0.009 1.139 0.009 {method 'acquire' of '_thread.lock' objects}
14 0.000 0.000 1.061 0.076 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py:533(wait)
14 0.000 0.000 1.061 0.076 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py:263(wait)
1 0.000 0.000 1.060 1.060 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py:255(map)
27177 function calls (27140 primitive calls) in 2.143 seconds
Ordered by: cumulative time
List reduced from 698 to 35 due to restriction <'recommendation-api'>
List reduced from 35 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 2.143 2.143 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/translation.py:193(recommend)
1 0.000 0.000 1.426 1.426 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:44(apply_filters)
1 0.000 0.000 1.147 1.147 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:9(filter_by_missing)
1 0.000 0.000 1.147 1.147 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:15(get_items_in_source_missing_in_target_by_titles)
1 0.004 0.004 1.147 1.147 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
1 0.000 0.000 1.141 1.141 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:46(chunk_query_for_parameter)
1 0.001 0.001 0.716 0.716 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/candidate_finders.py:27(get_morelike_candidates)
2 0.000 0.000 0.715 0.358 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/fetcher.py:80(wiki_search)
2 0.000 0.000 0.714 0.357 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/fetcher.py:11(get)
1 0.000 0.000 0.278 0.278 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:28(filter_by_disambiguation)
Filename: /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/translation.py
Line # Mem usage Increment Line Contents
================================================
193 2562.3203 MiB 0.0000 MiB def recommend(source, target, search, seed, count, include_pageviews, rank_method='default', max_candidates=500):
194 """
195 1. Use finder to select a set of candidate articles
196 2. Filter out candidates that are not missing, are disambiguation pages, etc
197 3. get pageview info for each passing candidate if desired
198 """
199
200 2562.3203 MiB 0.0000 MiB candidates = []
201
202 2562.3203 MiB 0.0000 MiB if seed:
203 2562.3203 MiB 0.0000 MiB finder = finder_map[search]
204 2562.3203 MiB 0.0000 MiB for seed in seed.split('|'):
205 2562.3203 MiB 0.0000 MiB candidates.extend(finder(source, seed, max_candidates))
206 else:
207 candidates.extend(finder_map['mostpopular'](source, seed, max_candidates))
208
209 2562.4062 MiB 0.0859 MiB recs = filters.apply_filters(source, target, candidates)
210
211 2562.4062 MiB 0.0000 MiB recs = sorted(recs, key=lambda x: x.rank, reverse=True)
212
213 2562.4062 MiB 0.0000 MiB if rank_method == 'sitelinks':
214 for rec in recs:
215 rec.rank = rec.sitelink_count
216 recs = sorted(recs, key=lambda x: x.rank, reverse=True)
217
218 2562.4062 MiB 0.0000 MiB recs = recs[:count]
219
220 2562.4062 MiB 0.0000 MiB if recs and include_pageviews:
221 recs = pageviews.set_pageview_data(source, recs)
222
223 2562.4062 MiB 0.0000 MiB return [r.__dict__() for r in recs]
2017-03-06 17:02:57,019 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:57,373 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:57,844 recommendation.api.types.translation.candidate_finders get_morelike_candidates():39 INFO -- morelike returned 501 results
2017-03-06 17:02:57,845 recommendation.api.types.translation.filters apply_filters():45 DEBUG -- Number of candidates: 500
2017-03-06 17:02:57,846 recommendation.api.types.translation.filters apply_filters():47 DEBUG -- Number of candidates after title: 494
2017-03-06 17:02:57,849 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:57,850 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:57,850 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:57,850 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:57,850 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:57,850 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:57,851 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:57,851 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:57,851 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:57,851 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:02:59,003 recommendation.api.types.translation.filters apply_filters():49 DEBUG -- Number of candidates after missing: 215
2017-03-06 17:02:59,003 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:59,278 recommendation.api.types.translation.filters apply_filters():51 DEBUG -- Number of candidates after disambiguation: 215
PASSED
recommendation/data/profile_performance.py::test_translation_with_seed[Banana] 2017-03-06 17:02:59,287 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:02:59,608 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:00,054 recommendation.api.types.translation.candidate_finders get_morelike_candidates():39 INFO -- morelike returned 501 results
2017-03-06 17:03:00,055 recommendation.api.types.translation.filters apply_filters():45 DEBUG -- Number of candidates: 500
2017-03-06 17:03:00,055 recommendation.api.types.translation.filters apply_filters():47 DEBUG -- Number of candidates after title: 486
2017-03-06 17:03:00,058 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:00,058 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:00,058 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:00,058 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:00,058 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:00,058 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:00,058 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:00,058 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:00,059 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:00,059 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:01,198 recommendation.api.types.translation.filters apply_filters():49 DEBUG -- Number of candidates after missing: 235
2017-03-06 17:03:01,198 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:01,490 recommendation.api.types.translation.filters apply_filters():51 DEBUG -- Number of candidates after disambiguation: 234
2017-03-06 17:03:01,507 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:01,871 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:02,396 recommendation.api.types.translation.candidate_finders get_morelike_candidates():39 INFO -- morelike returned 501 results
2017-03-06 17:03:02,403 recommendation.api.types.translation.filters apply_filters():45 DEBUG -- Number of candidates: 500
2017-03-06 17:03:02,404 recommendation.api.types.translation.filters apply_filters():47 DEBUG -- Number of candidates after title: 486
2017-03-06 17:03:02,412 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:02,414 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:02,415 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:02,415 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:02,417 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:02,418 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:02,419 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:02,422 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:02,425 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:02,429 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:03,369 recommendation.api.types.translation.filters apply_filters():49 DEBUG -- Number of candidates after missing: 235
2017-03-06 17:03:03,371 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:03,674 recommendation.api.types.translation.filters apply_filters():51 DEBUG -- Number of candidates after disambiguation: 234
27693 function calls (27656 primitive calls) in 2.205 seconds
Ordered by: internal time
List reduced from 697 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
129 1.134 0.009 1.134 0.009 {method 'acquire' of '_thread.lock' objects}
15 0.724 0.048 0.724 0.048 {method 'read' of '_ssl._SSLSocket' objects}
3 0.186 0.062 0.186 0.062 {method 'do_handshake' of '_ssl._SSLSocket' objects}
3 0.078 0.026 0.078 0.026 {method 'connect' of '_socket.socket' objects}
3 0.031 0.010 0.031 0.010 {method 'load_verify_locations' of '_ssl._SSLContext' objects}
3 0.008 0.003 0.009 0.003 /Users/nschaaf/.virtualenvs/recommendation/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py:414(close)
1 0.004 0.004 1.142 1.142 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
3 0.002 0.001 0.002 0.001 {built-in method _socket.gethostbyname}
3 0.001 0.000 0.001 0.000 {built-in method _socket.getaddrinfo}
3 0.001 0.000 0.001 0.000 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/decoder.py:345(raw_decode)
27693 function calls (27656 primitive calls) in 2.205 seconds
Ordered by: internal time
List reduced from 697 to 35 due to restriction <'recommendation-api'>
List reduced from 35 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.004 0.004 1.142 1.142 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
236 0.001 0.000 0.001 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:114(extract_from_raw)
1 0.000 0.000 0.768 0.768 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/candidate_finders.py:27(get_morelike_candidates)
501 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/recommendation.py:2(__init__)
1 0.000 0.000 1.143 1.143 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:9(filter_by_missing)
486 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:17(<lambda>)
234 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/recommendation.py:10(__dict__)
1 0.000 0.000 0.002 0.002 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:41(<listcomp>)
235 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/recommendation.py:18(incorporate_wikidata_item)
1 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:41(<listcomp>)
27693 function calls (27656 primitive calls) in 2.205 seconds
Ordered by: cumulative time
List reduced from 697 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 2.205 2.205 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/translation.py:193(recommend)
1 0.000 0.000 1.436 1.436 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:44(apply_filters)
1 0.000 0.000 1.143 1.143 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:9(filter_by_missing)
1 0.000 0.000 1.142 1.142 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:15(get_items_in_source_missing_in_target_by_titles)
1 0.004 0.004 1.142 1.142 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
1 0.000 0.000 1.136 1.136 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:46(chunk_query_for_parameter)
129 1.134 0.009 1.134 0.009 {method 'acquire' of '_thread.lock' objects}
14 0.000 0.000 1.084 0.077 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py:533(wait)
14 0.000 0.000 1.084 0.077 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py:263(wait)
1 0.000 0.000 1.083 1.083 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py:255(map)
27693 function calls (27656 primitive calls) in 2.205 seconds
Ordered by: cumulative time
List reduced from 697 to 35 due to restriction <'recommendation-api'>
List reduced from 35 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 2.205 2.205 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/translation.py:193(recommend)
1 0.000 0.000 1.436 1.436 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:44(apply_filters)
1 0.000 0.000 1.143 1.143 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:9(filter_by_missing)
1 0.000 0.000 1.142 1.142 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:15(get_items_in_source_missing_in_target_by_titles)
1 0.004 0.004 1.142 1.142 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
1 0.000 0.000 1.136 1.136 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:46(chunk_query_for_parameter)
1 0.000 0.000 0.768 0.768 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/candidate_finders.py:27(get_morelike_candidates)
2 0.000 0.000 0.767 0.384 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/fetcher.py:80(wiki_search)
2 0.000 0.000 0.766 0.383 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/fetcher.py:11(get)
1 0.000 0.000 0.292 0.292 /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/filters.py:28(filter_by_disambiguation)
Filename: /Users/nschaaf/git/recommendation-api/recommendation/api/types/translation/translation.py
Line # Mem usage Increment Line Contents
================================================
193 2561.7383 MiB 0.0000 MiB def recommend(source, target, search, seed, count, include_pageviews, rank_method='default', max_candidates=500):
194 """
195 1. Use finder to select a set of candidate articles
196 2. Filter out candidates that are not missing, are disambiguation pages, etc
197 3. get pageview info for each passing candidate if desired
198 """
199
200 2561.7383 MiB 0.0000 MiB candidates = []
201
202 2561.7383 MiB 0.0000 MiB if seed:
203 2561.7383 MiB 0.0000 MiB finder = finder_map[search]
204 2561.7383 MiB 0.0000 MiB for seed in seed.split('|'):
205 2561.7383 MiB 0.0000 MiB candidates.extend(finder(source, seed, max_candidates))
206 else:
207 candidates.extend(finder_map['mostpopular'](source, seed, max_candidates))
208
209 2561.7383 MiB 0.0000 MiB recs = filters.apply_filters(source, target, candidates)
210
211 2561.7383 MiB 0.0000 MiB recs = sorted(recs, key=lambda x: x.rank, reverse=True)
212
213 2561.7383 MiB 0.0000 MiB if rank_method == 'sitelinks':
214 for rec in recs:
215 rec.rank = rec.sitelink_count
216 recs = sorted(recs, key=lambda x: x.rank, reverse=True)
217
218 2561.7383 MiB 0.0000 MiB recs = recs[:count]
219
220 2561.7383 MiB 0.0000 MiB if recs and include_pageviews:
221 recs = pageviews.set_pageview_data(source, recs)
222
223 2561.7383 MiB 0.0000 MiB return [r.__dict__() for r in recs]
2017-03-06 17:03:03,703 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:04,039 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:04,501 recommendation.api.types.translation.candidate_finders get_morelike_candidates():39 INFO -- morelike returned 501 results
2017-03-06 17:03:04,502 recommendation.api.types.translation.filters apply_filters():45 DEBUG -- Number of candidates: 500
2017-03-06 17:03:04,502 recommendation.api.types.translation.filters apply_filters():47 DEBUG -- Number of candidates after title: 486
2017-03-06 17:03:04,505 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:04,506 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:04,506 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:04,506 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:04,506 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:04,506 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:04,506 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:04,506 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:04,506 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:04,506 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:05,441 recommendation.api.types.translation.filters apply_filters():49 DEBUG -- Number of candidates after missing: 235
2017-03-06 17:03:05,442 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:05,689 recommendation.api.types.translation.filters apply_filters():51 DEBUG -- Number of candidates after disambiguation: 234
PASSED
recommendation/data/profile_performance.py::test_related_articles[Apple] 2017-03-06 17:03:05,699 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:06,104 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:06,760 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:06,760 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:06,760 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:06,760 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:06,760 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:06,760 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:06,761 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:06,761 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:06,761 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:06,761 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:07,697 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:07,697 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:07,697 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:07,697 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:07,697 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:07,697 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:07,697 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:07,697 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:07,697 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:07,697 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:08,437 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:08,842 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:09,352 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:09,352 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:09,352 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:09,352 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:09,352 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:09,352 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:09,352 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:09,352 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:09,352 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:09,352 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:10,309 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:10,309 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:10,309 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:10,309 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:10,309 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:10,309 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:10,310 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:10,310 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:10,310 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:10,310 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
16902 function calls (16874 primitive calls) in 2.721 seconds
Ordered by: internal time
List reduced from 674 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
175 2.057 0.012 2.057 0.012 {method 'acquire' of '_thread.lock' objects}
2 0.283 0.142 0.283 0.142 {method 'read' of '_ssl._SSLSocket' objects}
1 0.189 0.189 0.189 0.189 {method 'dot' of 'numpy.ndarray' objects}
1 0.061 0.061 0.061 0.061 {method 'do_handshake' of '_ssl._SSLSocket' objects}
1 0.048 0.048 0.238 0.238 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:158(most_similar)
1 0.037 0.037 0.037 0.037 {method 'connect' of '_socket.socket' objects}
1 0.012 0.012 0.012 0.012 {method 'load_verify_locations' of '_ssl._SSLContext' objects}
3 0.005 0.002 2.072 0.691 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
1 0.003 0.003 0.241 0.241 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:66(get_nearest_neighbors)
1 0.003 0.003 0.003 0.003 /Users/nschaaf/.virtualenvs/recommendation/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py:414(close)
16902 function calls (16874 primitive calls) in 2.721 seconds
Ordered by: internal time
List reduced from 674 to 28 due to restriction <'recommendation-api'>
List reduced from 28 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.048 0.048 0.238 0.238 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:158(most_similar)
3 0.005 0.002 2.072 0.691 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
1 0.003 0.003 0.241 0.241 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:66(get_nearest_neighbors)
1001 0.002 0.000 0.003 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:114(extract_from_raw)
2 0.001 0.000 0.001 0.001 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:48(<listcomp>)
3 0.001 0.000 0.004 0.001 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:41(<listcomp>)
1 0.001 0.001 0.002 0.002 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:134(<listcomp>)
3 0.000 0.000 2.062 0.687 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:46(chunk_query_for_parameter)
1 0.000 0.000 2.722 2.722 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:128(recommend)
1002 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:47(<genexpr>)
16902 function calls (16874 primitive calls) in 2.721 seconds
Ordered by: cumulative time
List reduced from 674 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 2.722 2.722 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:128(recommend)
1 0.000 0.000 2.720 2.720 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:21(get_candidates)
3 0.005 0.002 2.072 0.691 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
3 0.000 0.000 2.062 0.687 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:46(chunk_query_for_parameter)
175 2.057 0.012 2.057 0.012 {method 'acquire' of '_thread.lock' objects}
33 0.000 0.000 1.948 0.059 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py:533(wait)
33 0.000 0.000 1.947 0.059 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py:263(wait)
3 0.000 0.000 1.946 0.649 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py:255(map)
3 0.000 0.000 1.945 0.648 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py:601(get)
3 0.000 0.000 1.945 0.648 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py:598(wait)
16902 function calls (16874 primitive calls) in 2.721 seconds
Ordered by: cumulative time
List reduced from 674 to 28 due to restriction <'recommendation-api'>
List reduced from 28 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 2.722 2.722 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:128(recommend)
1 0.000 0.000 2.720 2.720 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:21(get_candidates)
3 0.005 0.002 2.072 0.691 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
3 0.000 0.000 2.062 0.687 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:46(chunk_query_for_parameter)
2 0.000 0.000 1.660 0.830 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:25(get_titles_from_wikidata_items)
1 0.000 0.000 0.412 0.412 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:21(get_wikidata_items_from_titles)
1 0.000 0.000 0.405 0.405 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:58(resolve_seed)
1 0.000 0.000 0.405 0.405 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/fetcher.py:80(wiki_search)
1 0.000 0.000 0.404 0.404 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/fetcher.py:11(get)
1 0.003 0.003 0.241 0.241 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:66(get_nearest_neighbors)
Filename: /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py
Line # Mem usage Increment Line Contents
================================================
128 2587.8281 MiB 0.0000 MiB def recommend(source, seed, count):
129 2587.8281 MiB 0.0000 MiB recs = []
130
131 2587.8516 MiB 0.0234 MiB for seed in seed.split('|'):
132 2587.8516 MiB 0.0000 MiB recs.extend(candidate_finder.get_candidates(source, seed, count))
133
134 2587.8516 MiB 0.0000 MiB return [ArticleSpec(title=rec.title,
135 wikidata_id=rec.wikidata_id,
136 url=rec.url,
137 2587.8516 MiB 0.0000 MiB score=rec.score)._asdict() for rec in recs]
2017-03-06 17:03:11,081 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:11,428 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:11,960 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:11,961 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:11,961 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:11,961 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:11,962 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:11,962 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:11,962 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:11,962 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:11,963 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:11,967 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:13,003 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:13,004 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:13,006 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:13,007 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:13,007 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:13,007 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:13,007 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:13,008 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:13,008 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:13,013 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
PASSED
recommendation/data/profile_performance.py::test_related_articles[Banana] 2017-03-06 17:03:13,955 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:14,270 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:14,780 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:14,780 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:14,780 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:14,780 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:14,780 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:14,780 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:14,780 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:14,780 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:14,780 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:14,780 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:15,422 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:15,422 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:15,422 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:15,422 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:15,422 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:15,422 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:15,422 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:15,423 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:15,423 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:15,423 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:16,480 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:16,822 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:17,434 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:17,434 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:17,434 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:17,434 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:17,434 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:17,435 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:17,435 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:17,435 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:17,435 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:17,435 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:18,395 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:18,396 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:18,397 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:18,398 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:18,399 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:18,401 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:18,402 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:18,402 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:18,403 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:18,406 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
17706 function calls (17678 primitive calls) in 2.512 seconds
Ordered by: internal time
List reduced from 674 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
175 1.984 0.011 1.984 0.011 {method 'acquire' of '_thread.lock' objects}
2 0.201 0.100 0.201 0.100 {method 'read' of '_ssl._SSLSocket' objects}
1 0.148 0.148 0.148 0.148 {method 'dot' of 'numpy.ndarray' objects}
1 0.060 0.060 0.060 0.060 {method 'do_handshake' of '_ssl._SSLSocket' objects}
1 0.041 0.041 0.191 0.191 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:158(most_similar)
1 0.030 0.030 0.030 0.030 {method 'connect' of '_socket.socket' objects}
1 0.011 0.011 0.011 0.011 {method 'load_verify_locations' of '_ssl._SSLContext' objects}
3 0.004 0.001 1.998 0.666 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
1 0.004 0.004 0.195 0.195 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:66(get_nearest_neighbors)
1 0.003 0.003 0.003 0.003 /Users/nschaaf/.virtualenvs/recommendation/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py:414(close)
17706 function calls (17678 primitive calls) in 2.512 seconds
Ordered by: internal time
List reduced from 674 to 28 due to restriction <'recommendation-api'>
List reduced from 28 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.041 0.041 0.191 0.191 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:158(most_similar)
3 0.004 0.001 1.998 0.666 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
1 0.004 0.004 0.195 0.195 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:66(get_nearest_neighbors)
1001 0.002 0.000 0.005 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:114(extract_from_raw)
2 0.001 0.001 0.002 0.001 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:48(<listcomp>)
3 0.001 0.000 0.005 0.002 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:41(<listcomp>)
1 0.001 0.001 0.001 0.001 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:134(<listcomp>)
1 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:36(<dictcomp>)
3 0.000 0.000 1.989 0.663 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:46(chunk_query_for_parameter)
1 0.000 0.000 2.512 2.512 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:128(recommend)
17706 function calls (17678 primitive calls) in 2.512 seconds
Ordered by: cumulative time
List reduced from 674 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 2.512 2.512 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:128(recommend)
1 0.000 0.000 2.510 2.510 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:21(get_candidates)
3 0.004 0.001 1.998 0.666 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
3 0.000 0.000 1.989 0.663 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:46(chunk_query_for_parameter)
175 1.984 0.011 1.984 0.011 {method 'acquire' of '_thread.lock' objects}
33 0.000 0.000 1.896 0.057 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py:533(wait)
33 0.000 0.000 1.896 0.057 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py:263(wait)
3 0.000 0.000 1.895 0.632 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py:255(map)
3 0.000 0.000 1.894 0.631 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py:601(get)
3 0.000 0.000 1.894 0.631 /usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py:598(wait)
17706 function calls (17678 primitive calls) in 2.512 seconds
Ordered by: cumulative time
List reduced from 674 to 28 due to restriction <'recommendation-api'>
List reduced from 28 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 2.512 2.512 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:128(recommend)
1 0.000 0.000 2.510 2.510 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:21(get_candidates)
3 0.004 0.001 1.998 0.666 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:33(get_items)
3 0.000 0.000 1.989 0.663 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:46(chunk_query_for_parameter)
2 0.000 0.000 1.686 0.843 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:25(get_titles_from_wikidata_items)
1 0.000 0.000 0.314 0.314 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:58(resolve_seed)
1 0.000 0.000 0.314 0.314 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/fetcher.py:80(wiki_search)
1 0.000 0.000 0.314 0.314 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/fetcher.py:11(get)
1 0.000 0.000 0.313 0.313 /Users/nschaaf/git/recommendation-api/recommendation/api/external_data/wikidata.py:21(get_wikidata_items_from_titles)
1 0.004 0.004 0.195 0.195 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:66(get_nearest_neighbors)
Filename: /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py
Line # Mem usage Increment Line Contents
================================================
128 2586.0469 MiB 0.0000 MiB def recommend(source, seed, count):
129 2586.0469 MiB 0.0000 MiB recs = []
130
131 2586.0508 MiB 0.0039 MiB for seed in seed.split('|'):
132 2586.0508 MiB 0.0000 MiB recs.extend(candidate_finder.get_candidates(source, seed, count))
133
134 2586.0508 MiB 0.0000 MiB return [ArticleSpec(title=rec.title,
135 wikidata_id=rec.wikidata_id,
136 url=rec.url,
137 2586.0508 MiB 0.0000 MiB score=rec.score)._asdict() for rec in recs]
2017-03-06 17:03:19,268 recommendation.api.external_data.fetcher get():12 DEBUG -- Get: https://en.wikipedia.org/w/api.php
2017-03-06 17:03:19,578 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,129 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,130 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,130 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,130 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,130 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,130 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,130 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,130 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,130 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,130 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,953 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,954 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,956 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,956 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,956 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,956 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,956 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,956 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,957 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
2017-03-06 17:03:20,960 recommendation.api.external_data.fetcher post():23 DEBUG -- Post: https://www.wikidata.org/w/api.php
PASSED
recommendation/data/profile_performance.py::test_related_articles_by_items[Q89] 3174 function calls in 0.231 seconds
Ordered by: internal time
List reduced from 13 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.186 0.186 0.186 0.186 {method 'dot' of 'numpy.ndarray' objects}
1 0.035 0.035 0.223 0.223 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:158(most_similar)
1 0.004 0.004 0.231 0.231 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:106(recommend_items)
1054 0.002 0.000 0.002 0.000 <string>:35(_asdict)
2 0.002 0.001 0.002 0.001 {built-in method numpy.core.multiarray.where}
1 0.001 0.001 0.004 0.004 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:108(<listcomp>)
1054 0.000 0.000 0.001 0.000 <string>:12(__new__)
1054 0.000 0.000 0.000 0.000 {built-in method __new__ of type object at 0x1026e2660}
2 0.000 0.000 0.000 0.000 {method 'tolist' of 'numpy.ndarray' objects}
1 0.000 0.000 0.000 0.000 {method 'argsort' of 'numpy.ndarray' objects}
3174 function calls in 0.231 seconds
Ordered by: internal time
List reduced from 13 to 4 due to restriction <'recommendation-api'>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.035 0.035 0.223 0.223 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:158(most_similar)
1 0.004 0.004 0.231 0.231 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:106(recommend_items)
1 0.001 0.001 0.004 0.004 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:108(<listcomp>)
1 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:82(get_embedding)
3174 function calls in 0.231 seconds
Ordered by: cumulative time
List reduced from 13 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.004 0.004 0.231 0.231 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:106(recommend_items)
1 0.035 0.035 0.223 0.223 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:158(most_similar)
1 0.186 0.186 0.186 0.186 {method 'dot' of 'numpy.ndarray' objects}
1 0.001 0.001 0.004 0.004 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:108(<listcomp>)
1054 0.002 0.000 0.002 0.000 <string>:35(_asdict)
2 0.002 0.001 0.002 0.001 {built-in method numpy.core.multiarray.where}
1054 0.000 0.000 0.001 0.000 <string>:12(__new__)
1054 0.000 0.000 0.000 0.000 {built-in method __new__ of type object at 0x1026e2660}
2 0.000 0.000 0.000 0.000 {method 'tolist' of 'numpy.ndarray' objects}
1 0.000 0.000 0.000 0.000 /Users/nschaaf/.virtualenvs/recommendation/lib/python3.6/site-packages/numpy/core/fromnumeric.py:828(argsort)
3174 function calls in 0.231 seconds
Ordered by: cumulative time
List reduced from 13 to 4 due to restriction <'recommendation-api'>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.004 0.004 0.231 0.231 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:106(recommend_items)
1 0.035 0.035 0.223 0.223 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:158(most_similar)
1 0.001 0.001 0.004 0.004 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:108(<listcomp>)
1 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:82(get_embedding)
Filename: /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py
Line # Mem usage Increment Line Contents
================================================
106 2586.0664 MiB 0.0000 MiB def recommend_items(seed):
107 2586.0664 MiB 0.0000 MiB recs = candidate_finder.get_embedding().most_similar(word=seed)
108 2586.0664 MiB 0.0000 MiB return [ItemSpec(wikidata_id=rec[0], score=rec[1])._asdict() for rec in recs]
PASSED
recommendation/data/profile_performance.py::test_related_articles_by_items[Q90] 5430 function calls in 0.220 seconds
Ordered by: internal time
List reduced from 13 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.174 0.174 0.174 0.174 {method 'dot' of 'numpy.ndarray' objects}
1 0.036 0.036 0.212 0.212 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:158(most_similar)
1806 0.003 0.000 0.003 0.000 <string>:35(_asdict)
1 0.003 0.003 0.220 0.220 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:106(recommend_items)
1 0.002 0.002 0.006 0.006 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:108(<listcomp>)
2 0.001 0.001 0.001 0.001 {built-in method numpy.core.multiarray.where}
1806 0.000 0.000 0.001 0.000 <string>:12(__new__)
1806 0.000 0.000 0.000 0.000 {built-in method __new__ of type object at 0x1026e2660}
2 0.000 0.000 0.000 0.000 {method 'tolist' of 'numpy.ndarray' objects}
1 0.000 0.000 0.000 0.000 {method 'argsort' of 'numpy.ndarray' objects}
5430 function calls in 0.220 seconds
Ordered by: internal time
List reduced from 13 to 4 due to restriction <'recommendation-api'>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.036 0.036 0.212 0.212 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:158(most_similar)
1 0.003 0.003 0.220 0.220 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:106(recommend_items)
1 0.002 0.002 0.006 0.006 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:108(<listcomp>)
1 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:82(get_embedding)
5430 function calls in 0.220 seconds
Ordered by: cumulative time
List reduced from 13 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.003 0.003 0.220 0.220 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:106(recommend_items)
1 0.036 0.036 0.212 0.212 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:158(most_similar)
1 0.174 0.174 0.174 0.174 {method 'dot' of 'numpy.ndarray' objects}
1 0.002 0.002 0.006 0.006 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:108(<listcomp>)
1806 0.003 0.000 0.003 0.000 <string>:35(_asdict)
2 0.001 0.001 0.001 0.001 {built-in method numpy.core.multiarray.where}
1806 0.000 0.000 0.001 0.000 <string>:12(__new__)
1806 0.000 0.000 0.000 0.000 {built-in method __new__ of type object at 0x1026e2660}
2 0.000 0.000 0.000 0.000 {method 'tolist' of 'numpy.ndarray' objects}
1 0.000 0.000 0.000 0.000 /Users/nschaaf/.virtualenvs/recommendation/lib/python3.6/site-packages/numpy/core/fromnumeric.py:828(argsort)
5430 function calls in 0.220 seconds
Ordered by: cumulative time
List reduced from 13 to 4 due to restriction <'recommendation-api'>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.003 0.003 0.220 0.220 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:106(recommend_items)
1 0.036 0.036 0.212 0.212 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:158(most_similar)
1 0.002 0.002 0.006 0.006 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py:108(<listcomp>)
1 0.000 0.000 0.000 0.000 /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/candidate_finder.py:82(get_embedding)
Filename: /Users/nschaaf/git/recommendation-api/recommendation/api/types/related_articles/related_articles.py
Line # Mem usage Increment Line Contents
================================================
106 2586.0664 MiB 0.0000 MiB def recommend_items(seed):
107 2586.0703 MiB 0.0039 MiB recs = candidate_finder.get_embedding().most_similar(word=seed)
108 2586.0703 MiB 0.0000 MiB return [ItemSpec(wikidata_id=rec[0], score=rec[1])._asdict() for rec in recs]
PASSED
========================== 7 passed in 39.70 seconds ===========================
Event Timeline
•
schana
created this paste.
Mar 6 2017, 4:05 PM
2017-03-06 16:05:42 (UTC+0)
•
schana
mentioned this in
T159528: Gather performance info for the Related Articles type in Recommendation API
.
•
schana
mentioned this in
T159544: Gather performance info for Translation type in Recommendation API
.
Log In to Comment