Maniphest T219505

Recommendation API: output source language
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• bmansurov
	Mar 28 2019, 2:02 PM

Description

When you visit https://en.wikipedia.org/api/rest_v1/data/recommendation/article/creation/morelike/Book you'll see something like this:

[{"wikidata_id":"Q998687","normalized_rank":0.0000016773},{"wikidata_id":"Q1306010","normalized_rank":0.0000016773},{"wikidata_id":"Q997969","normalized_rank":0.0000016773},{"wikidata_id":"Q998851","normalized_rank":0.0000016773},{"wikidata_id":"Q1489555","normalized_rank":0.0000016773},{"wikidata_id":"Q1811594","normalized_rank":0.0000016773},{"wikidata_id":"Q998758","normalized_rank":0.0000016773},{"wikidata_id":"Q1648227","normalized_rank":0.0000016773},{"wikidata_id":"Q1811586","normalized_rank":0.0000016773},{"wikidata_id":"Q1238507","normalized_rank":0.0000016773}]

It would be useful to output source language for each recommended item. Along with the wikidata_id and normalized_rank properties, output a source_language property whose value is the language code for that item, e.g. 'es' for Spanish.

Codebase is here: https://gerrit.wikimedia.org/r/admin/projects/mediawiki/services/recommendation-api

Details

	Subject	Repo	Branch	Lines +/-
	Morelike: output source language	mediawiki/services/recommendation-api	master	+7 -5

Customize query in gerrit

Related Objects

Mentioned In: T220329: GSoC 2019 Proposal: Build statistics toolset to support WM-HU editor retention grant
rMSRAf7134d1673c2: Morelike: output source language
rMSRAe8f45645554c: Morelike: output source language
rMSRA07fcd6e37f83: Morelike: output source language
rMSRA3b0e0779def9: Morelike: output source language
rMSRAdd889721de39: Morelike: output source language
T218971: GSoC 2019 Proposal: Improve article recommendation pipeline

Event Timeline

• bmansurov created this task.Mar 28 2019, 2:02 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 28 2019, 2:02 PM

• bmansurov updated the task description. (Show Details)Mar 28 2019, 2:04 PM

• bmansurov mentioned this in T218971: GSoC 2019 Proposal: Improve article recommendation pipeline.

Usmanmuhd claimed this task.Mar 28 2019, 2:54 PM

@Usmanmuhd how's the progress? Do you have any questions?

I followed all the steps, and when I run npm start server.js | bunyan I get

usman@lenovo:~/recommendation-api$ npm start server.js | bunyan
bunyan: command not found
events.js:174
      throw er; // Unhandled 'error' event
      ^

Error [ERR_STREAM_DESTROYED]: Cannot call write after a stream was destroyed
    at doWrite (_stream_writable.js:406:19)
    at writeOrBuffer (_stream_writable.js:394:5)
    at Socket.Writable.write (_stream_writable.js:294:11)
    at Logger._emit (/home/usman/recommendation-api/node_modules/bunyan/lib/bunyan.js:923:22)
    at Logger.fatal (/home/usman/recommendation-api/node_modules/bunyan/lib/bunyan.js:1045:24)
    at Logger._log (/home/usman/recommendation-api/node_modules/service-runner/lib/logger.js:331:26)
    at Logger.log (/home/usman/recommendation-api/node_modules/service-runner/lib/logger.js:365:18)
    at logUnhandledException (/home/usman/recommendation-api/node_modules/service-runner/lib/logger.js:209:22)
    at process.on (/home/usman/recommendation-api/node_modules/service-runner/lib/logger.js:219:13)
    at process.emit (events.js:189:13)
    at process._fatalException (internal/bootstrap/node.js:496:27)
npm ERR! code ELIFECYCLE
npm ERR! errno 7
npm ERR! recommendation-api@0.5.0 start: `service-runner "server.js"`
npm ERR! Exit status 7
npm ERR! 
npm ERR! Failed at the recommendation-api@0.5.0 start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/usman/.npm/_logs/2019-03-30T03_39_12_117Z-debug.log

Edit: Fixed it by npm install -g bunyan. Shall I include this in documentation?

When I navigate to http://localhost:6927/en/v1/article/creation/morelike/book I get:

[2019-03-30T03:53:56.675Z] TRACE: recommendation-api/26082 on lenovo: incoming request (request_id=715d7a20-529f-11e9-8f5e-3f15249dbade, levelPath=trace/req)
    request: {
      "url": "/en/v1/article/creation/morelike/book",
      "headers": {
        "user-agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0",
        "cache-control": "max-age=0",
        "x-request-id": "715d7a20-529f-11e9-8f5e-3f15249dbade"
      },
      "method": "GET",
      "params": {
        "0": "/en/v1/article/creation/morelike/book"
      },
      "query": {},
      "remoteAddress": "127.0.0.1",
      "remotePort": 36146
    }
[2019-03-30T03:53:56.676Z] DEBUG: recommendation-api/26082 on lenovo: Getting article's Wikidata ID from MW API. (articleTitle=book, levelPath=debug/article.creation.morelike)
[2019-03-30T03:53:56.682Z] ERROR: recommendation-api/26082 on lenovo: 504: internal_http_error (message="504: internal_http_error", status=504, type=internal_http_error, detail="504: internal_http_error", request_id=715d7a20-529f-11e9-8f5e-3f15249dbade, levelPath=error/504)
    request: {
      "url": "/en/v1/article/creation/morelike/book",
      "headers": {
        "user-agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0",
        "cache-control": "max-age=0",
        "x-request-id": "715d7a20-529f-11e9-8f5e-3f15249dbade"
      },
      "method": "GET",
      "params": {
        "0": "/en/v1/article/creation/morelike/book"
      },
      "query": {},
      "remoteAddress": "127.0.0.1",
      "remotePort": 36146
    }

Edit: Fixed it by npm install -g bunyan. Shall I include this in documentation?

Sure that'd be good.

When I navigate to http://localhost:6927/en/v1/article/creation/morelike/book I get:

Try this instead: http://localhost:6927/en.wikipedia.org/v1/article/creation/morelike/Book

Thanks, it works. Documentation changes needed for this?

Sure.

Change 500234 had a related patch set uploaded (by Usmanmuhd; owner: Usmanmuhd):
[mediawiki/services/recommendation-api@master] Morelike: output source language

https://gerrit.wikimedia.org/r/500234

gerritbot added a project: Patch-For-Review.Mar 31 2019, 2:51 PM

Output:

[{"wikidata_id":"Q48781","normalized_rank":0.0000216333,"source_language":"de"},{"wikidata_id":"Q1304185","normalized_rank":0.00000783086,"source_language":"de"},{"wikidata_id":"Q386797","normalized_rank":0.00000783086,"source_language":"de"},{"wikidata_id":"Q2598040","normalized_rank":0.0000016773,"source_language":"de"},{"wikidata_id":"Q23924990","normalized_rank":0.0000016773,"source_language":"de"},{"wikidata_id":"Q2569863","normalized_rank":0.0000016773,"source_language":"de"},{"wikidata_id":"Q296063","normalized_rank":0.0000016773,"source_language":"de"},{"wikidata_id":"Q1120683","normalized_rank":0.0000016773,"source_language":"de"},{"wikidata_id":"Q1496250","normalized_rank":0.0000016773,"source_language":"de"},{"wikidata_id":"Q1106915","normalized_rank":0.0000016773,"source_language":"de"}]

Usmanmuhd mentioned this in rMSRAdd889721de39: Morelike: output source language.Mar 31 2019, 3:55 PM

Usmanmuhd mentioned this in rMSRA3b0e0779def9: Morelike: output source language.

Usmanmuhd mentioned this in rMSRA07fcd6e37f83: Morelike: output source language.

Usmanmuhd mentioned this in rMSRAe8f45645554c: Morelike: output source language.Apr 1 2019, 2:38 PM

Usmanmuhd mentioned this in rMSRAf7134d1673c2: Morelike: output source language.

Change 500234 merged by Bmansurov:
[mediawiki/services/recommendation-api@master] Morelike: output source language

https://gerrit.wikimedia.org/r/500234