Page MenuHomePhabricator

RESTBase with VisualEditor gives TypeError: Cannot set property 'etag' of undefined
Closed, ResolvedPublic

Description

Steps to reproduce:

Results (domain with "fake.example.net" for privacy):

{"name":"restbase","hostname":"bones","pid":51675,"level":40,"msg":"Client did not supply etag, fallback to mw:TimeUuid meta element","root_req":{"method":"post","uri":"/fake.example.net/v1/transform/html/to/wikitext/MediaWiki_Server/24","headers":{"content-length":"21866","content-type":"application/x-www-form-urlencoded","user-agent":"VisualEditor-MediaWiki/1.32.0","x-client-ip":"::1","x-forwarded-for":"::1","x-request-id":"7e754840-a45d-11e9-95db-4df3c493cdc6","x-request-class":"internal"}},"request_id":"7e754840-a45d-11e9-95db-4df3c493cdc6","api_path":"/fake.example.net/v1/transform/html/to/wikitext/{title}/{revision}","levelPath":"warn/parsoid/etag","time":"2019-07-12T04:28:27.598Z","v":0}

{"name":"restbase","hostname":"bones","pid":51675,"level":50,"message":"500: internal_error","res":{"name":"HTTPError","message":"500: internal_error","status":500,"body":{"type":"internal_error","description":"TypeError: Cannot read property 'body' of undefined"},"innerError":{"requestName":"get_from_backend"},"headers":{"access-control-allow-origin":"*","access-control-allow-methods":"GET,HEAD","access-control-allow-headers":"accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding","access-control-expose-headers":"etag","x-content-type-options":"nosniff","x-frame-options":"SAMEORIGIN","referrer-policy":"origin-when-cross-origin","x-xss-protection":"1; mode=block","content-security-policy":"default-src 'none'; frame-ancestors 'none'","x-content-security-policy":"default-src 'none'; frame-ancestors 'none'","x-webkit-csp":"default-src 'none'; frame-ancestors 'none'","cache-control":"private, max-age=0, s-maxage=0, must-revalidate","x-request-id":"7e754840-a45d-11e9-95db-4df3c493cdc6","server":"bones"}},"stack":"TypeError: Cannot read property 'body' of undefined
    at contentPromise.tap (/usr/home/wikiuser/restbase-1.0.0/sys/parsoid.js:585:47)
    at PassThroughHandlerContext.finallyHandler (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/finally.js:57:23)
    at PassThroughHandlerContext.tryCatcher (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:517:31)
    at Promise._settlePromise (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:574:18)
    at Promise._settlePromise0 (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:619:10)
    at Promise._settlePromises (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:699:18)
    at Promise._fulfill (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:643:18)
    at Promise._resolveCallback (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:437:57)
    at Promise._settlePromiseFromHandler (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:529:17)
    at Promise._settlePromise (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:574:18)
    at Promise._settlePromise0 (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:619:10)
    at Promise._settlePromises (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:699:18)
    at Promise._fulfill (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:643:18)
    at Promise._resolveCallback (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:437:57)
    at Promise._settlePromiseFromHandler (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:529:17)","latency":404,"root_req":{"method":"post","uri":"/fake.example.net/v1/transform/html/to/wikitext/MediaWiki_Server/24","headers":{"content-length":"21866","content-type":"application/x-www-form-urlencoded","user-agent":"VisualEditor-MediaWiki/1.32.0","x-client-ip":"::1","x-forwarded-for":"::1","x-request-id":"7e754840-a45d-11e9-95db-4df3c493cdc6","x-request-class":"internal"}},"request_id":"7e754840-a45d-11e9-95db-4df3c493cdc6","levelPath":"error/request","msg":"500: internal_error","time":"2019-07-12T04:28:27.608Z","v":0}

{"name":"restbase","hostname":"bones","pid":51674,"level":50,"message":"500: internal_error","res":{"name":"HTTPError","message":"500: internal_error","status":500,"body":{"type":"internal_error","description":"TypeError: Cannot set property 'etag' of undefined"},"innerError":{"requestName":"get_from_backend"},"headers":{"access-control-allow-origin":"*","access-control-allow-methods":"GET,HEAD","access-control-allow-headers":"accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding","access-control-expose-headers":"etag","x-content-type-options":"nosniff","x-frame-options":"SAMEORIGIN","referrer-policy":"origin-when-cross-origin","x-xss-protection":"1; mode=block","content-security-policy":"default-src 'none'; frame-ancestors 'none'","x-content-security-policy":"default-src 'none'; frame-ancestors 'none'","x-webkit-csp":"default-src 'none'; frame-ancestors 'none'","cache-control":"private, max-age=0, s-maxage=0, must-revalidate","x-request-id":"86f41e10-a45d-11e9-bc34-0b5288621277","server":"bones"}},"stack":"TypeError: Cannot set property 'etag' of undefined
    at contentReq.then.then (/usr/home/wikiuser/restbase-1.0.0/sys/parsoid.js:511:30)
    at tryCatcher (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:517:31)
    at Promise._settlePromise (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:574:18)
    at Promise._settlePromise0 (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:619:10)
    at Promise._settlePromises (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:699:18)
    at Promise._fulfill (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:643:18)
    at Promise._resolveCallback (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:437:57)
    at Promise._settlePromiseFromHandler (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:529:17)
    at Promise._settlePromise (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:574:18)
    at Promise._settlePromise0 (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:619:10)
    at Promise._settlePromises (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:699:18)
    at Promise._fulfill (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:643:18)
    at Promise._settlePromise (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:587:21)
    at Promise._settlePromise0 (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:619:10)
    at Promise._settlePromises (/usr/home/wikiuser/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:699:18)","latency":8,"root_req":{"method":"get","uri":"/fake.example.net/v1/page/html/MediaWiki_Server/24?redirect=false","headers":{"user-agent":"VisualEditor-MediaWiki/1.32.0","x-client-ip":"::1","x-forwarded-for":"::1","x-request-id":"86f41e10-a45d-11e9-bc34-0b5288621277","x-request-class":"internal"}},"request_id":"86f41e10-a45d-11e9-bc34-0b5288621277","levelPath":"error/request","msg":"500: internal_error","time":"2019-07-12T04:28:41.465Z","v":0}

Similar errors result from using curl. I have tried RESTBase 1.0.0 and current git clone. Node is version 11. OS is FreeBSD 11. MediaWiki is 1.32

UPDATE:

I played around with the code in parsoid.js:511 and added some console.logs to get more info:

.then((res) => {
    console.log('--res');
    console.log(res);
    console.log('--format');
    console.log(format);
    console.log('--res.body[format]');
    console.log(res.body[format]);
    const etag = res.headers.etag;
    // Chop off the correct format to return.
    res = Object.assign({ status: res.status }, res.body[format]);
    console.log('--res');
    console.log(res);
    res.headers.etag = etag;
    mwUtil.normalizeContentType(res);
    res.headers = res.headers || {};
    if (req.query.stash) {
        // The stash is used by clients that want further support
        // for transforming the content. If the content is stored in caches,
        // subsequent requests might not even reach RESTBase and the stash
        // will expire, thus no-cache.
        res.headers['cache-control'] = 'no-cache';
    } else if (this.options.response_cache_control) {
        res.headers['cache-control'] = this.options.response_cache_control;
    }
    if (/^null$/.test(res.headers.etag)) {
        hyper.logger.log('error/parsoid/response_etag_missing', {
            msg: 'Detected a null etag in the response!'
        });
    }

    return res;
});

The resulting output is:

--res
{ status: 200,
  headers:
   { etag: '"1/3cc18770-ac09-11e9-b7bd-5102a7d962e9"',
     'content-type': 'application/json' },
  body:
   <Buffer 7b 22 64 61 74 61 2d 70 61 72 73 6f 69 64 22 3a 7b 22 68 65 61 64 65 72 73 22 3a 7b 22 63 6f 6e 74 65 6e 74 2d 74 79 70 65 22 3a 22 61 70 70 6c 69 63 ... 4425 more bytes> }
--format
html
--res.body[format]
undefined
--res
{ status: 200 }
{"name":"restbase","hostname":"bones","pid":46414,"level":50,"message":"500: internal_error","res":{"name":"HTTPError","message":"500: internal_error","status":500,"body":{"type":"internal_error","description":"TypeError: Cannot set property 'etag' of undefined"},"innerError":{"requestName":"get_from_backend"},"headers":{"access-control-allow-origin":"*","access-control-allow-methods":"GET,HEAD","access-control-allow-headers":"accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding","access-control-expose-headers":"etag","x-content-type-options":"nosniff","x-frame-options":"SAMEORIGIN","referrer-policy":"origin-when-cross-origin","x-xss-protection":"1; mode=block","content-security-policy":"default-src 'none'; frame-ancestors 'none'","x-content-security-policy":"default-src 'none'; frame-ancestors 'none'","x-webkit-csp":"default-src 'none'; frame-ancestors 'none'","cache-control":"private, max-age=0, s-maxage=0, must-revalidate","x-request-id":"ec9a7890-ac23-11e9-8ecb-d9f672f5d2fd","server":"bones"}},"stack":"TypeError: Cannot set property 'etag' of undefined
    at contentReq.then.then (/usr/home/dave/restbase-1.0.0/sys/parsoid.js:519:30)
    at tryCatcher (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:517:31)
    at Promise._settlePromise (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:574:18)
    at Promise._settlePromise0 (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:619:10)
    at Promise._settlePromises (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:699:18)
    at Promise._fulfill (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:643:18)
    at Promise._resolveCallback (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:437:57)
    at Promise._settlePromiseFromHandler (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:529:17)
    at Promise._settlePromise (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:574:18)
    at Promise._settlePromise0 (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:619:10)
    at Promise._settlePromises (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:699:18)
    at Promise._fulfill (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:643:18)
    at Promise._settlePromise (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:587:21)
    at Promise._settlePromise0 (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:619:10)
    at Promise._settlePromises (/usr/home/dave/restbase-1.0.0/node_modules/bluebird/js/release/promise.js:699:18)","latency":283,"root_req":{"method":"get","uri":"/bones.dhowland.net/v1/page/html/Main_Page/1?redirect=false","headers":{"user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0","api-user-agent":"VisualEditor-MediaWiki/1.32.0","x-client-ip":"::ffff:192.168.1.22","x-forwarded-for":"::ffff:192.168.1.22","x-request-id":"ec9a7890-ac23-11e9-8ecb-d9f672f5d2fd","x-request-class":"external"}},"request_id":"ec9a7890-ac23-11e9-8ecb-d9f672f5d2fd","levelPath":"error/request","msg":"500: internal_error","time":"2019-07-22T01:56:30.772Z","v":0}

I'm afraid I really don't know anything about Node or js in general, but it appears that the data in this response is not expected to be a buffer.

UPDATE:

The unprocessed string made me think that something was going wrong with the json stringify package, so I manually undid the change that replaced json-stable-stringify with fast-json-stable-stringify, and the error went away.

However now I get a new error and I've run out of energy to try and get this working.

Event Timeline

Metalliqaz renamed this task from TypeError: Cannot set property 'etag' of undefined to RESTBase with VisualEditor gives TypeError: Cannot set property 'etag' of undefined.Jul 15 2019, 4:31 PM

I updated the description with some crude investigative work.

@Pchelolo it appears that the code in question was committed by you back in may. Do you happen to recognize what is going on here?

Thank you for the investigation. Sorry about this, https://github.com/wikimedia/restbase/pull/1173 should fix it.

Pchelolo claimed this task.