Page MenuHomePhabricator

Openrefine: problems logging in
Closed, ResolvedPublic

Description

Hi!

I cannot seen to login to Openrefine.

I never tried to access it before, but I know from other Wikimedia related applications that sometimes my 2FA causes problems.

The screen simply says "invalid credentials".

Event Timeline

Ciell created this task.Jun 13 2020, 1:53 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 13 2020, 1:53 PM
Reedy renamed this task from Openrefine: problems lgging in with 2FA? to Openrefine: problems logging in with 2FA?.Jun 13 2020, 2:01 PM
Reedy added a subscriber: Reedy.

If you have 2FA enabled on your account, you'll have to use BotPasswords to login to third party tools like OpenRefine

Ciell added a comment.Jun 13 2020, 2:15 PM

Ah, thank you!

We would be interested in supporting 2FA login directly in OpenRefine, for this we need to improve Wikidata-Toolkit first:
https://github.com/Wikidata/Wikidata-Toolkit/issues/412

Reedy added a comment.Jun 14 2020, 1:51 PM

We would be interested in supporting 2FA login directly in OpenRefine, for this we need to improve Wikidata-Toolkit first:
https://github.com/Wikidata/Wikidata-Toolkit/issues/412

Just to point out literally, you cannot support it in OpenRefine. Users have to use BotPasswords or if applicable using OAuth. As a "desktop" type app (ie installing it yourself), OAuth 1.0 won't work, 2.0 might.

Ah interesting! I thought the MediaWiki API could be used to do 2FA log in:
https://www.wikidata.org/w/api.php?action=help&modules=clientlogin
but if not that simplifies our life even more :)

Reedy added a comment.EditedJun 14 2020, 2:22 PM

Continue logging in after a UI response for two-factor auth, supplying an OATHToken of 987654.

api.php?action=clientlogin&logincontinue=1&OATHToken=987654&logintoken=123ABC

Hmm. Does that actually work?

I will note, that this will definitely not work for WebAuthn based 2FA

It seems the message is in MW core

	"apihelp-clientlogin-example-login2": "Continue logging in after a <samp>UI</samp> response for two-factor auth, supplying an <var>OATHToken</var> of <kbd>987654</kbd>.",

Which seems a bad place to actually have it, because it's basically misleading.. It will show on wikis without OATHAuth enabled (I'll file a task about clearing that up/moving it).

As for if it actually works.. I'm honestly not sure. I'm guessing it does/did if it's in an example

The Android Commons App seems to be using it if I understand this issue correctly:
https://github.com/commons-app/apps-android-commons/issues/3614

The Android Commons App seems to be using it if I understand this issue correctly:
https://github.com/commons-app/apps-android-commons/issues/3614

Seemingly so...

https://github.com/commons-app/apps-android-commons/blob/32ee0b4f9a65ec854243f666b13476ec30a2fced/data-client/src/main/java/org/wikipedia/dataclient/Service.java#L229-L239

Which seems to be a copy of the Wikipedia Android App...

https://github.com/wikimedia/apps-android-wikipedia/blob/b4778290cf7e21dd3d6b8afc53710aa95db7d01e/app/src/main/java/org/wikipedia/dataclient/Service.java#L158-L172

From T150582: Support two-factor authentication in AutoWikiBrowser (I'm sorry for CC'ing people as a result of quoting messages)

I don't think the API supports 2FA yet

The API supports everything AuthManager supports via action=clientlogin. Using action=login for anything except BotPasswords is deprecated, and AWB should use OAuth instead of BotPasswords for non-interactive login.

This came up at WikiConf North America today. I think there's been a bit of misunderstanding of this ticket. There's no need for OAuth support in AWB (nor does it make sense). Instead, AWB needs to switch to action=clientlogin, which will return a response prompting for 2FA if necessary, which the user should provide to complete the login process.

Support for owner-only OAuth does makes sense, it's basically a different type of password that is more secure and does not require 2FA. It's not user-friendly though and you are probably better off with bot passwords (which do not require any code change, other than maybe explaining the user how to use them).

action=clientlogin requires the app to implement an open-ended dialog system which is not really specified anywhere. It's a nontrivial amount of work, and I'm not sure it is worth the effort, nor that it is good security practice to train users to put their password and 2FA into random desktop apps they have downloaded from the internet.

Ok. So my retained knowledge (even though I was part of that discussion, but it is a few years ago) is obviously out of date. So, sorry for the incorrect information!

I think Gergo's point probably matches up with my mental expectation though

nor that it is good security practice to train users to put their password and 2FA into random desktop apps they have downloaded from the internet.

I guess in the case of the Mobile Apps... The barrier to entry of owner only OAuth apps (because the code is open source), or creating and using a bot password... (though, you could argue if they have 2FA enabled, they're probably above average technical experience), so handling this in the mobile apps was the "easiest" way forward.

Obviously, if an app is being given a users TOTP, they could intercept them (after already having been given their username and password too) and you could login before they did, pretend it didn't work and then get them to give another to log themselves in... That'd be quite easy to spot in an open source application if someone was to audit the code.

Going to file another task about documenting something along these lines as a case of best practices.

Ciell added a comment.Sat, Aug 1, 7:46 PM

Open Refine is still not connecting.

I only have to create the password, and use it to log into OR rigt? I do not need to create a separate account in order to use it?

Reedy added a comment.Sat, Aug 1, 10:44 PM

Open Refine is still not connecting.

I only have to create the password, and use it to log into OR rigt? I do not need to create a separate account in order to use it?

For AWB (for example, I got) after creating the Bot Password:

The new password to log in with Reedy@AWB is <redactedpassword>. Please record this for future reference.

I login with a username of "Reedy@AWB" rather than just "Reedy", and then use the generated password. No new account is created

@Ciell looking at your screenshot, you have successfully logged in. OpenRefine is not making any edits because Wikidata is overloaded - that is independent from OpenRefine itself.
See T243701.

Ciell added a comment.Sun, Aug 2, 1:15 PM

Thank you for the clarification Pintoch, and I'm glad it is not about the log in.
Is there any work arounds, are there times that the load on the server is less high, for instance the European morning weekdays?

I am not sure! Perhaps you could ask on T243701, people might have identified patterns?

Ciell added a comment.Sun, Aug 2, 6:15 PM

Thank you, I did. I wonder if this is the problem though: on Telegram people tell me their bots don't have trouble editing today, but I restarted Open Refine manually again three times because it just keeps stuck at the '0%' as shown in my screenshot from yesterday. And still nothing gets uploaded.

Ciell added a comment.Sun, Aug 2, 8:05 PM

I had an idea and am trying to do the edits though OR with my sock puppet, but am looking at the same '0%' for over half an hour now.
What could I be coding wrong...?

Ciell renamed this task from Openrefine: problems logging in with 2FA? to Openrefine: problems logging in.Sun, Aug 2, 8:10 PM

You could have a look at OpenRefine's logs:

If you could paste here what you see in these logs, it would probably be helpful!

Ciell added a comment.EditedMon, Aug 3, 8:16 AM

wait, I'll try again

Ciell added a comment.Mon, Aug 3, 6:21 PM

So.... it's on 10%...! But not moving any more in the past half hour, so here's the log:

19:47:06.901 [ refine_server] Starting Server bound to '127.0.0.1:3333' (0ms)
19:47:06.972 [ refine_server] Initializing context: '/' from 'C:\Users\C\Documents\Openrefine\openrefine-3.3\webapp' (71ms)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Users/C/Documents/Openrefine/openrefine-3.3/server/target/lib/slf4j-log4j12-1.7.18.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/C/Documents/Openrefine/openrefine-3.3/webapp/WEB-INF/lib/slf4j-log4j12-1.7.18.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
19:47:38.457 [ refine] Starting OpenRefine 3.3 [58b839b]... (31485ms)
19:47:38.457 [ refine] initializing FileProjectManager with dir (0ms)
19:47:38.457 [ refine] C:\Users\C\AppData\Local\OpenRefine (0ms)
19:48:07.703 [ refine] POST /command/core/load-language (29246ms)
19:48:07.781 [ refine] GET /command/core/get-preference (78ms)
19:48:07.847 [ refine] POST /command/core/load-language (66ms)
19:48:07.874 [ refine] POST /command/core/load-language (27ms)
19:48:07.978 [ org.mortbay.log] /images/favicon.png (104ms)
java.lang.IllegalStateException: Committed

at org.mortbay.jetty.Response.resetBuffer(Response.java:1024)
at javax.servlet.ServletResponseWrapper.resetBuffer(ServletResponseWrapper.java:202)
at org.mortbay.servlet.GzipFilter$GZIPResponseWrapper.resetBuffer(GzipFilter.java:310)
at org.mortbay.servlet.GzipFilter$GZIPResponseWrapper.sendError(GzipFilter.java:319)
at edu.mit.simile.butterfly.Butterfly.error(Butterfly.java:1020)
at edu.mit.simile.butterfly.Butterfly.service(Butterfly.java:528)
at com.google.refine.RefineServlet.service(RefineServlet.java:212)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:81)
at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:132)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

19:48:08.293 [ refine] POST /command/core/get-importing-configuration (315ms)
19:48:08.371 [ refine] GET /command/core/get-all-project-tags (78ms)
19:48:08.445 [ refine] GET /command/core/get-all-project-metadata (74ms)
19:48:08.739 [ refine] GET /command/core/get-csrf-token (294ms)
19:48:09.035 [ refine] GET /command/database/saved-connection (296ms)
19:48:09.245 [ refine] GET /command/core/get-languages (210ms)
19:48:09.425 [ refine] GET /command/core/get-version (180ms)
19:48:16.929 [ refine] POST /command/core/load-language (7504ms)
19:48:16.985 [ refine] GET /command/core/get-preference (56ms)
19:48:16.997 [ refine] GET /command/core/get-preference (12ms)
19:48:17.034 [ refine] POST /command/core/load-language (37ms)
19:48:17.065 [ refine] POST /command/core/load-language (31ms)
19:48:17.108 [ refine] POST /command/core/load-language (43ms)
19:48:17.130 [ refine] POST /command/core/load-language (22ms)
19:48:17.215 [ refine] GET /command/core/get-project-metadata (85ms)
19:48:19.201 [ project] Loaded project 2508413189435 from disk in 1 sec(s) (1986ms)
19:48:19.238 [ refine] GET /command/core/get-models (37ms)
19:48:19.558 [ refine] GET /command/core/get-history (320ms)
19:48:19.589 [ refine] POST /command/core/get-rows (31ms)
19:48:19.653 [ refine] GET /command/core/get-history (64ms)
19:48:20.967 [ refine] GET /command/core/get-csrf-token (1314ms)
19:48:21.268 [ refine] POST /command/wikidata/preview-wikibase-schema (301ms)
19:48:27.770 [ refine] GET /command/core/get-csrf-token (6502ms)
19:48:27.833 [ refine] POST /command/core/recon-judge-one-cell (63ms)
19:48:27.955 [ refine] GET /command/core/get-history (122ms)
19:48:28.272 [ refine] GET /command/core/get-project-metadata (317ms)
19:48:28.388 [ refine] GET /command/core/get-models (116ms)
19:48:37.951 [ refine] GET /command/core/get-csrf-token (9563ms)
19:48:38.099 [ refine] POST /command/core/recon-judge-similar-cells (148ms)
19:48:38.168 [ refine] GET /command/core/get-history (69ms)
19:48:38.453 [ refine] GET /command/core/get-project-metadata (285ms)
19:48:38.484 [ refine] GET /command/core/get-models (31ms)
19:48:38.522 [ refine] POST /command/core/get-rows (38ms)
19:48:38.925 [ refine] POST /command/core/compute-facets (403ms)
19:48:39.675 [ refine] GET /command/core/get-csrf-token (750ms)
19:48:39.797 [ refine] POST /command/wikidata/preview-wikibase-schema (122ms)
19:48:48.863 [ refine] POST /command/core/get-rows (9066ms)
19:49:28.609 [ refine] GET /command/wikidata/login (39746ms)
19:49:29.897 [ refine] GET /command/core/get-csrf-token (1288ms)
19:49:29.935 [ refine] POST /command/wikidata/preview-wikibase-schema (38ms)
19:49:37.377 [ refine] GET /command/core/get-csrf-token (7442ms)
19:49:37.439 [ refine] POST /command/wikidata/perform-wikibase-edits (62ms)
19:49:37.539 [..mWikibaseEditsOperation] Performing edits (100ms)
19:49:37.546 [..ting.EditBatchProcessor] Requesting documents (7ms)
19:49:37.546 [..ting.EditBatchProcessor] Retrying in 10000 ms (0ms)
19:49:47.548 [..ting.EditBatchProcessor] Retrying in 20000 ms (10002ms)
19:50:07.561 [..ting.EditBatchProcessor] Retrying in 40000 ms (20013ms)
19:50:47.576 [..ting.EditBatchProcessor] Retrying in 80000 ms (40015ms)
19:52:08.520 [..ting.EditBatchProcessor] MediaWiki error while editing [permissiondenied]: You do not have the permissions needed to carry out this action. (80944ms)
19:52:09.326 [..ting.EditBatchProcessor] MediaWiki error while editing [permissiondenied]: You do not have the permissions needed to carry out this action. (806ms)
Exception in thread "Thread-5" java.lang.IllegalArgumentException: Entity id "Nederlandse Antillen " is not supported.

at org.wikidata.wdtk.datamodel.implementation.EntityIdValueImpl.guessEntityTypeFromId(EntityIdValueImpl.java:169)
at org.wikidata.wdtk.datamodel.implementation.EntityIdValueImpl$JacksonInnerEntityId.<init>(EntityIdValueImpl.java:229)
at org.wikidata.wdtk.datamodel.implementation.EntityIdValueImpl.<init>(EntityIdValueImpl.java:95)
at org.wikidata.wdtk.datamodel.implementation.ItemIdValueImpl.<init>(ItemIdValueImpl.java:55)
at org.wikidata.wdtk.datamodel.implementation.DataObjectFactoryImpl.getItemIdValue(DataObjectFactoryImpl.java:43)
at org.wikidata.wdtk.datamodel.helpers.DatamodelConverter.copy(DatamodelConverter.java:252)
at org.openrefine.wikidata.editing.ReconEntityRewriter.copy(ReconEntityRewriter.java:94)
at org.wikidata.wdtk.datamodel.helpers.DatamodelConverter.visit(DatamodelConverter.java:582)
at org.wikidata.wdtk.datamodel.helpers.DatamodelConverter.visit(DatamodelConverter.java:47)
at org.openrefine.wikidata.schema.entityvalues.ReconEntityIdValue.accept(ReconEntityIdValue.java:151)
at org.wikidata.wdtk.datamodel.helpers.DatamodelConverter.copyValue(DatamodelConverter.java:576)
at org.wikidata.wdtk.datamodel.helpers.DatamodelConverter.copy(DatamodelConverter.java:393)
at org.wikidata.wdtk.datamodel.helpers.DatamodelConverter.copy(DatamodelConverter.java:374)
at org.wikidata.wdtk.datamodel.helpers.DatamodelConverter.copy(DatamodelConverter.java:468)
at org.openrefine.wikidata.editing.ReconEntityRewriter.lambda$rewrite$5(ReconEntityRewriter.java:120)
at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
at java.util.stream.AbstractPipeline.evaluate(Unknown Source)
at java.util.stream.ReferencePipeline.collect(Unknown Source)
at org.openrefine.wikidata.editing.ReconEntityRewriter.rewrite(ReconEntityRewriter.java:121)
at org.openrefine.wikidata.editing.EditBatchProcessor.performEdit(EditBatchProcessor.java:137)
at org.openrefine.wikidata.operations.PerformWikibaseEditsOperation$PerformEditsProcess.run(PerformWikibaseEditsOperation.java:208)
at java.lang.Thread.run(Unknown Source)

19:52:40.944 [ ProjectManager] Saving some modified projects ... (31618ms)
19:52:41.051 [ project_utilities] Saved project '2508413189435' (107ms)

Ciell added a comment.Mon, Aug 3, 6:25 PM

I'm looking at the BottPassword settings now, because of

19:52:08.520 [..ting.EditBatchProcessor] MediaWiki error while editing [permissiondenied]: You do not have the permissions needed to carry out this action. (80944ms)
19:52:09.326 [..ting.EditBatchProcessor] MediaWiki error while editing [permissiondenied]: You do not have the permissions needed to carry out this action. (806ms)

Ciell closed this task as Resolved.Mon, Aug 3, 6:53 PM
Ciell claimed this task.

That seemed to be causing the last errors.
First uploads are going the Wikidata now.