Page MenuHomePhabricator

Cassandra JSON fields getting reversed
Closed, InvalidPublic

Description

We think this dependency change: https://gerrit.wikimedia.org/r/#/c/analytics/aqs/deploy/+/474705/1/node_modules/restbase-mod-table-cassandra/node_modules/json-stable-stringify/package.json might have caused our arrays to come out of Cassandra JSON fields reversed. We added a sort as a fix for T210091, which is the right thing to do anyway, but you probably need to look at this in case it affects other services. The schema that endpoint pulls from is this: https://phabricator.wikimedia.org/diffusion/AAQS/browse/master/sys/pageviews.js$91

Event Timeline

Relying on the order of things in JSON was clearly a bug and it's nice that it's been fixed. However, I do not think json-stable-stringify was the one to blame - it's not used anywhere on the actual response data - it's used more on various metadata to compile hashes of, for example, table schemas. I do not quite understand which exact change has made the JSON properties reverse order, but since it's not specifyed, it could be literally anything. I'm inclined to close the task as Invalid.

However, thank you for filing it, cause it made me discover a fast-json-stable-stringify library that we should probably try using.

@Pchelolo, weird! I just blamed the first thing I saw that made any sense, and now we have no explanation. But, it doesn't matter, like you said the bug was not sorting in the first place. Agreed with Invalid, thanks.