Page MenuHomePhabricator

A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer, Echo, Flow, RelatedArticles, MobileFrontend and Wikibase browser tests since Aug 2, 2016 12:01 PM
Closed, ResolvedPublic

Description

Various jobs are seeing mass failures on https://integration.wikimedia.org/ci/view/Selenium/ with the error
A JSON text must at least contain two octets!

...
00:00:09.090 Feature: Multimedia Viewer performance
00:00:09.090 
00:00:09.090   Background:                                                      # features/mmv.performance.feature:4

00:00:11.259     Given I am using a custom user agent                           # features/step_definitions/mmv_performance_steps.rb:26
00:00:11.259     And I am at a wiki article with at least two embedded pictures # features/step_definitions/mmv_steps.rb:3
00:00:11.259       A JSON text must at least contain two octets! (JSON::ParserError)
00:00:11.259       /mnt/home/jenkins-deploy/.gem/2.0.0/gems/json-1.8.3/lib/json/common.rb:155:in `initialize'
...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
zeljkofilipin closed this task as Resolved.May 10 2016, 1:44 PM
zeljkofilipin claimed this task.

Resolved by T128190.

phuedx added a subscriber: phuedx.EditedAug 3 2016, 11:18 AM

I'm seeing this same error on the MobileFrontend browser tests: https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/103/

The error first started appearing yesterday (Tuesday, 2nd August).

Tgr added a subscriber: Tgr.Aug 3 2016, 5:37 PM

CentralAuth as well.

Change 302766 had a related patch set uploaded (by Dduvall):
Support automatic redirects

https://gerrit.wikimedia.org/r/302766

Change 302766 merged by jenkins-bot:
Support automatic redirects

https://gerrit.wikimedia.org/r/302766

Jdlrobson renamed this task from A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer browser tests to A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer, RelatedArticles, MobileFrontend and Wikibase browser tests.Aug 3 2016, 7:42 PM
Jdlrobson added a subscriber: thcipriani.

@thcipriani this is somewhat different from the original bug. Tests are passing when run locally and via Jenkins on patch upload but recently have started failing all over the place with this issue. My feeling is there is a common browser test step shared by these extensions that has suddenly started failing or there is an issue with something in the beta cluster infrastructure. I'm a bit lost right now tbh on how to fix this.

Jdlrobson renamed this task from A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer, RelatedArticles, MobileFrontend and Wikibase browser tests to A JSON text must at least contain two octets! (JSON::ParserError) in MultimediaViewer, Echo, Flow, RelatedArticles, MobileFrontend and Wikibase browser tests since Aug 2, 2016 12:01 PM.Aug 3 2016, 7:48 PM

Issue seems to have been introduced some time around Aug 2, 2016 12:01 PM.

@dduvall and I paired on this a bit. It seems like the faraday gem does not support redirect in some instances from http → https (hence, https://gerrit.wikimedia.org/r/302766).

There are a number of updates that have to happen to get that update in to place. One of which is updating the beta mediawiki_api in environments.yml to use https rather than http, which may fix the symptom of the problem you you're seeing with mobilefrontend @Jdlrobson (https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/blob/master/tests/browser/environments.yml#L31).

Change 302775 had a related patch set uploaded (by Jdlrobson):
API should use https on the beta cluster

https://gerrit.wikimedia.org/r/302775

Jdlrobson added a comment.EditedAug 3 2016, 8:00 PM

That would make sense. mediawiki_api is defined in integration/config and used by various extensions. Does the above patch make sense as a good starting point?

Thanks @thcipriani and @dduvall for your investigations so far :)

Change 302775 merged by jenkins-bot:
API should use https on the beta cluster

https://gerrit.wikimedia.org/r/302775

dduvall claimed this task.Aug 4 2016, 8:25 PM

In addition to the above integration/config patch, we'll need to update the mediawiki_selenium dependency for each of the following extensions.

  • CentralAuth
  • CentralNotice
  • CirrusSearch
  • Core
  • Echo
  • Flow
  • GettingStarted
  • Math
  • MobileFrontend
  • MultimediaViewer
  • PageTriage
  • QuickSurveys
  • RelatedArticles
  • VisualEditor
  • WikiLove
  • Wikibase
  • Wikidata

Simply edit the Gemfile to specify the newest version (1.7.2) and run bundle install to update the Gemfile.lock. This will pull in the newest version of mediawiki_api which supports automatic redirects.

Change 303007 had a related patch set uploaded (by Jdlrobson):
Upgrade Gemfile

https://gerrit.wikimedia.org/r/303007

Change 303007 merged by jenkins-bot:
Upgrade Gemfile

https://gerrit.wikimedia.org/r/303007

@Jdlrobson, thanks for taking the initiative on MobileFrontend though your patch doesn't seem to include any changes to Gemfile.lock—the lock file is authoritative when it comes to installing dependencies. Did you run bundle install after bumping the version in Gemfile and before committing?

@Jdlrobson, thanks for taking the initiative on MobileFrontend though your patch doesn't seem to include any changes to Gemfile.lock—the lock file is authoritative when it comes to installing dependencies. Did you run bundle install after bumping the version in Gemfile and before committing?

See my comments in the subtask about this being a nitpick. The CI jobs should work after bumping the version in Gemfile because the version spec is strict enough to trigger an update. Brain...

Change 303091 had a related patch set uploaded (by Jdlrobson):
Run bundle install

https://gerrit.wikimedia.org/r/303091

Pinging test maintainers per {T134492: Ownership of Selenium tests} for repositories that have not yet been updated. It would be great to get these updated ASAP. Let me know if you need direction. Also, in addition to updating the gem dependency, please update the beta mediawiki_url in environments.yaml to use 'https` as well. This will avoid the overhead of following redirects.

Repositorycontact
mediawiki/extensions/CentralAuth@Tgr
mediawiki/extensions/CentralNotice@AndyRussG
mediawiki/extensions/CirrusSearch@dcausse
mediawiki/extensions/Echo@SBisson
mediawiki/extensions/Flow@SBisson
mediawiki/extensions/GettingStarted@Mattflaschen-WMF @phuedx
mediawiki/extensions/Math@Physikerwelt
mediawiki/extensions/PageTriage@SBisson
mediawiki/extensions/VisualEditor@Jdforrester-WMF
mediawiki/extensions/WikibaseWikidata @adrianheine
wmde/WikidataBrowserTestsWikidata @adrianheine
mediawiki/extensions/WikiLove@SBisson

Change 303581 had a related patch set uploaded (by Jforrester):
Gemfile: Bump mediawiki_api from 1.7.1 to 1.7.2

https://gerrit.wikimedia.org/r/303581

Change 303584 had a related patch set uploaded (by DCausse):
Update browser test config

https://gerrit.wikimedia.org/r/303584

Change 303584 merged by jenkins-bot:
Update browser test config

https://gerrit.wikimedia.org/r/303584

Tgr added a comment.Aug 8 2016, 9:05 PM

CentralAuth done in https://gerrit.wikimedia.org/r/#/c/303603/

IMO for mechanical tasks like this it would be more efficient for one person to do it, instead of each maintainer having to figure out what to do. A bash script with a few sed and git commands can iterate through all the repos with tests and create changesets.

Working on Flow and GettingStarted.

Change 303702 had a related patch set uploaded (by Mattflaschen):
Update mediawiki-selenium to 1.7.2 to fix bug

https://gerrit.wikimedia.org/r/303702

Change 303704 had a related patch set uploaded (by Mattflaschen):
Update mediawiki-selenium to 1.7.2 to fix bug

https://gerrit.wikimedia.org/r/303704

Change 303704 merged by jenkins-bot:
Update mediawiki-selenium to 1.7.2 to fix bug

https://gerrit.wikimedia.org/r/303704

Change 303702 merged by jenkins-bot:
Update mediawiki-selenium to 1.7.2 to fix bug

https://gerrit.wikimedia.org/r/303702

Change 303581 merged by jenkins-bot:
Gemfile: Bump mediawiki_api from 1.7.1 to 1.7.2

https://gerrit.wikimedia.org/r/303581

Change 303091 merged by jenkins-bot:
Run bundle install

https://gerrit.wikimedia.org/r/303091

I think everything is merged? Can we resolve this?

Change 304305 had a related patch set uploaded (by Sbisson):
Browser tests: update to mw-selenium 1.7.2 and https

https://gerrit.wikimedia.org/r/304305

Change 304307 had a related patch set uploaded (by Sbisson):
Browser tests: update to mw-selenium 1.7.2 and https

https://gerrit.wikimedia.org/r/304307

Change 304305 merged by jenkins-bot:
Browser tests: update to mw-selenium 1.7.2 and https

https://gerrit.wikimedia.org/r/304305

Change 304307 merged by jenkins-bot:
Browser tests: update to mw-selenium 1.7.2 and https

https://gerrit.wikimedia.org/r/304307

Change 304467 had a related patch set uploaded (by Sbisson):
Browser tests: update to mw-selenium 1.7.2 and https

https://gerrit.wikimedia.org/r/304467

Change 304467 merged by jenkins-bot:
Browser tests: update to mw-selenium 1.7.2 and https

https://gerrit.wikimedia.org/r/304467

Jdlrobson added subscribers: Mattflaschen-WMF, greg.

@greg who's responsible for mediawiki/core ? It's the only thing that didn't get patched. If someone can merge my patch I'm about to post I believe this task can be resolved. @Mattflaschen-WMF can you help?

Change 304881 had a related patch set uploaded (by Jdlrobson):
Browser tests: update to mw-selenium 1.7.2 and https

https://gerrit.wikimedia.org/r/304881

greg added a comment.Aug 15 2016, 9:32 PM

@greg who's responsible for mediawiki/core ? It's the only thing that didn't get patched. If someone can merge my patch I'm about to post I believe this task can be resolved. @Mattflaschen-WMF can you help?

For now, @zeljkofilipin / RelEng

@zeljkofilipin I've added you to the patch for review so we can try to get the core browser tests green again.

I +2'ed core. I agree with Tgr though:

IMO for mechanical tasks like this it would be more efficient for one person to do it, instead of each maintainer having to figure out what to do. A bash script with a few sed and git commands can iterate through all the repos with tests and create changesets.

This is often done (though I don't know how automatically) for browser test changes already.

Change 304881 merged by jenkins-bot:
Browser tests: update to mw-selenium 1.7.2 and https

https://gerrit.wikimedia.org/r/304881

I have just took a look at all selenium-* jobs and the only one still failing with JSON::ParserError is selenium-Wikidata.

Jdlrobson closed this task as Resolved.EditedAug 17 2016, 3:14 PM

This is fixed. Remaining browser test issues are covered in T142600 Various tests failing due to login .

Jdlrobson reopened this task as Open.Aug 17 2016, 3:14 PM

My bad yes there is that one still.

Change 305271 had a related patch set uploaded (by Zfilipin):
WIP Fix JSON::ParserError

https://gerrit.wikimedia.org/r/305271

I have tried updating wikidata/browsertests repository (305271) but got this:

$ bundle
Fetching gem metadata from https://rubygems.org/..........
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Resolving dependencies...
Bundler could not find compatible versions for gem "mediawiki_api":
  In Gemfile:
    mediawiki_selenium (>= 1.7.2, ~> 1.7) ruby depends on
      mediawiki_api (>= 0.7.0, ~> 0.7) ruby

    mediawiki_api-wikidata (>= 0.2.1, ~> 0.2.0) ruby depends on
      mediawiki_api (0.6.0)
zeljkofilipin closed this task as Resolved.Aug 17 2016, 3:43 PM

The last one is resolved in 305271.

Change 305271 merged by jenkins-bot:
Fix JSON::ParserError

https://gerrit.wikimedia.org/r/305271