Page MenuHomePhabricator

Internal server error when accessing change
Closed, ResolvedPublic

Description

I managed to create a change that cannot be accessed: https://gerrit.wikimedia.org/r/359241. I was editing it with no problems, then I suddenly got an Internal Server Error and nobody can access it anymore.

From Paladox T168012#3353152

@Dzahn kindly looked in the logs and found this error for my change

[2017-06-15 21:25:28,800] [HTTP-67550] ERROR com.google.gerrit.httpd.restapi.RestApiServlet : Error in GET /r/changes/356181/revisions/c0f8da0e921aca0f8178ac561ea2987fc6f1c29e/files?reviewed
java.lang.NullPointerException

this error is fixed in gerrit 2.14 as i got this error upstream and reported it too.

Here's the bug upstream https://bugs.chromium.org/p/gerrit/issues/detail?id=6176

Here's the fix https://gerrit-review.googlesource.com/#/c/106613/

Event Timeline

Hi, was this your change?

If not was this a draft?

If it was someone may have either deleted it or removed you as a reviewer.

It was my change. I created it through the Gerrit user interface and used it to add a simple text file to my extension. After I tried to publish the edit, the change became inaccessible. Nobody else was editing the change.

Interesting - I can access yours with no problem.

Really? It's meant to be a draft. So i guess there's a bug. (lucky drafts are being replaced with something stable)

That's so strange! Yours is not listed as a draft when I look at it, but mine is marked as a draft in the list of changes. Should I just give up on this change and start over? Is there a way to delete it so I don't see it in my list of changes taunting me?

Oh maybe reindexing the change will fix this. (but needs to be done either through ssh using online index or on the server (you need access to do both which only the admin currently has)

Paladox triaged this task as High priority.EditedJun 15 2017, 8:54 PM

Three users have reported this. Ktinkle also reported this on irc as having same problem, inaccessible for them but accessible by everyone else. Triaging.

ah https://gerrit.wikimedia.org/r/#/c/356181/ that's not a draft whoops, thought it was since i have several similar with same name but are a draft. But i get 500 when visiting it logged in.

Being logged in and going to https://gerrit.wikimedia.org/r/359241 I get:

Code Review - Error
The page you requested was not found, or you do not have permission to view this page.

No "Internal server error" or such.

I can access https://gerrit.wikimedia.org/r/#/c/356181/ without any problems.

Being logged in and going to https://gerrit.wikimedia.org/r/359241 I get:

Code Review - Error
The page you requested was not found, or you do not have permission to view this page.

No "Internal server error" or such.

I can access https://gerrit.wikimedia.org/r/#/c/356181/ without any problems.

I get the same as you for the first one, the second one i get 500, but logging out i can view it.

Strange within the last minute 500 has shown when logged out for my change now.

Update: Shows again when logged out.

@Dzahn kindly looked in the logs and found this error for my change

[2017-06-15 21:25:28,800] [HTTP-67550] ERROR com.google.gerrit.httpd.restapi.RestApiServlet : Error in GET /r/changes/356181/revisions/c0f8da0e921aca0f8178ac561ea2987fc6f1c29e/files?reviewed
java.lang.NullPointerException

this error is fixed in gerrit 2.14 as i got this error upstream and reported it too.

Here's the bug upstream https://bugs.chromium.org/p/gerrit/issues/detail?id=6176

Here's the fix https://gerrit-review.googlesource.com/#/c/106613/

@cicalese your error sounds different to mine. Yours sounds like the change is deleted or somehow gerrit does not think your the owner of the change.

though you say you created a change through the ui and as you publish it, it threw an error and you can now not see your change right?

@cicalese ah, i can confirm his problem is the same as mine now. Creating changes through web ui and trying to publish with changes done to file is throwing 500 for me too.

I filled this https://bugs.chromium.org/p/gerrit/issues/detail?id=6519 upstream. Upstream are not planning for any more 2.13 releases so to fix this. we can a: build from the 2.13 branch with the change back ported which i did here https://gerrit-review.googlesource.com/#/c/110410/ or upgrade to 2.14.1 which includes the fix.

Adding releng project as this is a gerrit source code problem.

@demon would you mind cherry picking https://gerrit-review.googlesource.com/c/110410 to our 2.13 Gerrit package? That is the backport Paladox did for stable-2.13 but upstream is no more releasing new versions for that branch. Thx!

I'll have a look Tuesday

I've got the full log here

[2017-06-18 16:42:32,415] [HTTP-77] ERROR com.google.gerrit.httpd.restapi.RestApiServlet : Error in GET /gerrit/changes/2/revisions/60d4559483c72ce7a1f830098150bd6f3bbd51ca/files?reviewed
java.lang.NullPointerException
        at com.google.gerrit.server.patch.PatchListCacheImpl.get(PatchListCacheImpl.java:118)
        at com.google.gerrit.server.patch.PatchListCacheImpl.get(PatchListCacheImpl.java:106)
        at com.google.gerrit.server.change.Files$ListFiles.copy(Files.java:262)
        at com.google.gerrit.server.change.Files$ListFiles.reviewed(Files.java:244)
        at com.google.gerrit.server.change.Files$ListFiles.apply(Files.java:148)
        at com.google.gerrit.server.change.Files$ListFiles.apply(Files.java:94)
        at com.google.gerrit.httpd.restapi.RestApiServlet.service(RestApiServlet.java:334)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
        at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
        at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
        at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
        at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
        at com.google.gerrit.httpd.GetUserFilter.doFilter(GetUserFilter.java:82)
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
        at com.google.gwtexpui.server.CacheControlFilter.doFilter(CacheControlFilter.java:73)
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
        at com.google.gerrit.httpd.RunAsFilter.doFilter(RunAsFilter.java:122)
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
        at com.google.gerrit.httpd.RequestMetricsFilter.doFilter(RequestMetricsFilter.java:60)
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
        at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:136)
        at com.google.gerrit.httpd.AllRequestFilter$FilterProxy.doFilter(AllRequestFilter.java:105)
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
        at com.google.gerrit.httpd.RequestContextFilter.doFilter(RequestContextFilter.java:75)
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
        at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:120)
        at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:499)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:748)

from a test install of gerrit 2.13.8 to reproduce this error.

I now know how to reproduce this:

  1. Create change through inline edit then edit a file and save the file (do not press publish or anything yet)
  2. then try to review the file.
  3. then try going to patchset 1. it will throw a 500.
  4. then try publishing patchset 5. it will now throw 500.

Note: That 2.14 seemes to be affected by this too but was fixed in 2.14.1. Also note that 2.14 did not prevent viewing the change it only prevented you from seeing whats in the file list.

Also tested with https://gerrit-review.googlesource.com/110410 now and it fixes my issue.

@demon could you pull in the 2.13 branch instead please.

Also we will need to re cherry pick https://gerrit-review.googlesource.com/#/c/92830/ to prevent auth errors again please.

Also us pulling in stable-2.13 will hopefully fix T161206 as it has the fix.

Is it possible for an admin to delete my draft change https://gerrit.wikimedia.org/r/359241 from the backend? I still cannot access that change, and I no longer need it since I created another change to replace it. I would like to have it removed from my list of open changes. Thanks!

Change 360684 had a related patch set uploaded (by Chad; owner: Chad):
[operations/debs/gerrit@master] gerrit (2.13.8 git1-wmf.6) jessie-wikimedia; urgency=medium

https://gerrit.wikimedia.org/r/360684

Change 360684 merged by Dzahn:
[operations/debs/gerrit@master] gerrit (2.13.8 git1-wmf.6) jessie-wikimedia; urgency=medium

https://gerrit.wikimedia.org/r/360684

Is it possible for an admin to delete my draft change https://gerrit.wikimedia.org/r/359241 from the backend? I still cannot access that change, and I no longer need it since I created another change to replace it. I would like to have it removed from my list of open changes. Thanks!

Have the same problem, please delete https://gerrit.wikimedia.org/r/#/c/360697

I'm upgrading today to include the upstream patches for this, no need for deletion.

demon claimed this task.

Thank you, @demon! I was able to access my draft change and delete it.