Currently, MCS uses the following settings for logging in production:
logging:
level: warn
name: mobileapps
streams:
- host: logstash.svc.eqiad.wmnet
port: 12201
type: gelf
- level: info
path: /srv/log/mobileapps/main.log
type: fileHowever, it doesn't seem like all the error level messages actually reach logstash. For example, the following error can not be found in logstash:
[2019-08-05T15:29:23.817Z] ERROR: mobileapps/123 on scb1002: 500: invalid_request_parameters (message="500: invalid_request_parameters", status=500, type=invalid_request_parameters, detail="500: invalid_request_parameters", request_id=cd668950-b795-11e9-bab3-0bbfa50eb22e, levelPath=error/500)
stack: HTTPError: 500: invalid_request_parameters
at Object.mUtil.createDocument (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/src/lib/mobile-util.js:261:25)
at Object.createDocumentFromLeadSection (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/src/lib/sections/parsoidSections.js:126:18)
at buildExtracts (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/src/lib/summary.js:105:32)
at Object.buildSummary (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/src/lib/summary.js:127:12)
at BBPromise.join (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/src/routes/page/summary.js:31:24)
at tryCatcher (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/util.js:16:23)
at Holder$3._callFunction (eval at generateHolderClass (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/join.js:1:0), <anonymous>:14:44)
at Holder$3.checkFulfillment (eval at generateHolderClass (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/join.js:1:0), <anonymous>:29:30)
at Promise.eval (eval at thenCallback (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/join.js:1:0), <anonymous>:6:20)
at Promise._settlePromise (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/promise.js:571:21)
at Promise._settlePromise0 (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/promise.js:619:10)
at Promise._settlePromises (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/promise.js:699:18)
at _drainQueueStep (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/async.js:138:12)
at _drainQueue (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/async.js:131:9)
at Async._drainQueues (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/async.js:147:5)
at Immediate.Async.drainQueues (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/async.js:17:14)
at runCallback (timers.js:672:20)
at tryOnImmediate (timers.js:645:5)
at processImmediate [as _immediateCallback] (timers.js:617:5)
--
request: {
"url": "/pt.wikipedia.org/v1/page/summary/Temporada_da_United_SportsCar_Championship_de_2017",
"headers": {
"x-request-id": "cd668950-b795-11e9-bab3-0bbfa50eb22e",
"content-length": "0"
},
"method": "GET",
"params": {
"0": "/pt.wikipedia.org/v1/page/summary/Temporada_da_United_SportsCar_Championship_de_2017"
},
"query": {},
"remoteAddress": "10.64.16.117",
"remotePort": 58188
}
[2019-08-05T15:29:24.083Z] ERROR: mobileapps/79 on scb1002: 500: invalid_request_parameters (message="500: invalid_request_parameters", status=500, type=invalid_request_parameters, detail="500: invalid_request_parameters", request_id=cd668950-b795-11e9-bab3-0bbfa50eb22e, levelPath=error/500)
stack: HTTPError: 500: invalid_request_parameters
at Object.mUtil.createDocument (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/src/lib/mobile-util.js:261:25)
at Object.createDocumentFromLeadSection (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/src/lib/sections/parsoidSections.js:126:18)
at buildExtracts (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/src/lib/summary.js:105:32)
at Object.buildSummary (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/src/lib/summary.js:127:12)
at BBPromise.join (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/src/routes/page/summary.js:31:24)
at tryCatcher (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/util.js:16:23)
at Holder$3._callFunction (eval at generateHolderClass (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/join.js:1:0), <anonymous>:14:44)
at Holder$3.checkFulfillment (eval at generateHolderClass (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/join.js:1:0), <anonymous>:29:30)
at Promise.eval (eval at thenCallback (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/join.js:1:0), <anonymous>:6:20)
at Promise._settlePromise (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/promise.js:571:21)
at Promise._settlePromise0 (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/promise.js:619:10)
at Promise._settlePromises (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/promise.js:699:18)
at _drainQueueStep (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/async.js:138:12)
at _drainQueue (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/async.js:131:9)
at Async._drainQueues (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/async.js:147:5)
at Immediate.Async.drainQueues (/srv/deployment/mobileapps/deploy-cache/revs/250f7117a30b5f716653907e3557407fca57f416/node_modules/bluebird/js/release/async.js:17:14)
at runCallback (timers.js:672:20)
at tryOnImmediate (timers.js:645:5)
at processImmediate [as _immediateCallback] (timers.js:617:5)
--
request: {
"url": "/pt.wikipedia.org/v1/page/summary/Temporada_da_United_SportsCar_Championship_de_2017",
"headers": {
"x-request-id": "cd668950-b795-11e9-bab3-0bbfa50eb22e",
"content-length": "0"
},
"method": "GET",
"params": {
"0": "/pt.wikipedia.org/v1/page/summary/Temporada_da_United_SportsCar_Championship_de_2017"
},
"query": {},
"remoteAddress": "10.64.16.117",
"remotePort": 58200
}Searching for invalid_request_parameters or mobileapps only gives me results from RESTBase and Change-Prop, but not mobileapps itself. We need to find out what's happening. One possibility being that there's a conflict in property names in logstash for this kind of logs.