Moved the log to a paste for the sake of reducing the burden to scroll down to the discussion.

If I interpret the logs correctly, the token fetch is lines 253-267:

GET /w/api.php?type=edit&action=tokens&format=xml HTTP/1.1
Cookie: GeoIP=AU:censored:v4; WMF-Last-Access=16-Jul-2018; WMF-Last-Access-Global=16-Jul-2018; GeoIP=AU:::censored:v4; commonswikiSession=(...); commonswikiUserID=1713; commonswikiUserName=Misaochan; centralauth_User=Misaochan; centralauth_Token=(...); centralauth_Session=(...); loginnotify_prevlogins=(...)

the upload is lines 268-284:

POST /w/api.php HTTP/1.1
Cookie: GeoIP=AU:censored:v4; WMF-Last-Access-Global=16-Jul-2018; GeoIP=AU:::censored:v4; commonswikiUserName=Misaochan; loginnotify_prevlogins=(...); WMF-Last-Access=27-Jul-2018

so the session cookies are not sent on POST. (They do appear on later GET requests so they do not get lost, just not used on this request.)

So does it suggest that we are not sending the session cookies with our POST request or is there a possibility that they are just not getting printed? To add more context here's the network interceptor that I added to print these request logs:

https://github.com/commons-app/apps-android-commons/blob/2.8-release/app/src/main/java/fr/free/nrw/commons/mwapi/NetworkInterceptors.java

If the issue is session cookies not being sent at all, is there a way to explicitly set these cookies while making a POST request?

I'm not experienced with org.apache.http but it seems unlikely that it would print some cookies but not all.

@Tgr I figured out that the first call GET /w/api.php?type=edit&action=tokens is not returning a token(fails with the user not logged in error) so the subsequent upload call doesn't have the proper auth cookie set. This explains why the upload was failing.

To fix this issue, I implemented a custom MwApi layer(earlier we were using the library org.mediawiki:api:1.3) so that I can tweak some flows to handle forced logins. So, I started doing the following things,

• attempted to obtain a central auth token
• if the central auth token call fails, I do a re-login to fetch the centralAuthToken
  • get loginToken
  • do actual login
  • fetch the centralAuthToken again
• using this centralAuthToken, I obtain the csrf token.
• use this csrf token to perform the upload operation.

I get the following error:

<?xml version="1.0" encoding="UTF-8"?><api servedby="mw1231"><error code="badtoken" info="Invalid CSRF token." xml:space="preserve">See https://commons.wikimedia.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &amp;lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&amp;gt; for notice of API deprecations and breaking changes.</error></api>

My questions are:

• Am I doing anything wrong because of which badtoken error is coming? Specifically, I am not very sure if csrf token is to be used?
• When I am doing a re-login(behind the scenes), could the old auth cookies[even if they are incorrect] still cause an issue?
• Assuming that my steps are correct, what additional step would be required for a 2FA enabled account? Is it even possible to do it behind the scenes for such a user?

It seems I don't have permissions to create a paste with a custom policy. Have edited the same paste with the new logs. These logs are more descriptive than the ones I shared earlier. :)

{P7395}

[Note: For the benefit of the general audience, Android account manager securely stores the username and password for us so we can re-login the user without asking for explicit login]

@Tgr I added the centralAuthToken along with csrf token and the upload succeeded for me.

Can you validate the workflow and answer the question related to 2FA.

Just wanted to post that @maskaravivek 's solution works for me (for the most part). If no issues, we will be pushing a release with this implementation very shortly. :)

Just pinging the person who added the 2FA support to the Wikipedia app. Maybe he has a suggestion/idea? @Mholloway, could you help here? Sorry, if pinging wasn't the best thing I should have done.

In T200560#4477636, @maskaravivek wrote:

It seems I don't have permissions to create a paste with a custom policy.

I added you to Trusted-Contributors which should be sufficient. Feel free to add more people (it's an anti-spam/anti-vandalism limitation so pretty low bar).

To fix this issue, I implemented a custom MwApi layer(earlier we were using the library org.mediawiki:api:1.3) so that I can tweak some flows to handle forced logins. So, I started doing the following things,

• attempted to obtain a central auth token
• if the central auth token call fails, I do a re-login to fetch the centralAuthToken
  • get loginToken
  • do actual login
  • fetch the centralAuthToken again
• using this centralAuthToken, I obtain the csrf token.
• use this csrf token to perform the upload operation.

This seems super complicated (although not obviously wrong). The centralauthtoken API is meant for authorizing cross-wiki requests, and it's meant for web clients (which do not have control over what cookies to set). When using a mobile client, just set the centralauth_User and centralauth_Token cookies which are the same for all domains (note that the centralauth_Token cookie is a different thing from the token returned by the centralauthtoken API). If you don't do cross-wiki calls, you don't even need to do that - just do a normal login and rely on default session handling.

That would be something like

• fetch a login token
• send the username and password to the login API (this will fail if the user has 2FA and is using a real password, not bot password)
• fetch a csrf token
• do the upload

• Am I doing anything wrong because of which badtoken error is coming? Specifically, I am not very sure if csrf token is to be used?

Badtoken means the token you submitted does not match with the token stored server-side in your session. That could mean that the session was lost (e.g. you did not set session cookies properly on your request), or maybe actions authenticated with the centralauthtoken API and those authenticated with cookies do not share the same session - I would have to look that up but it sounds plausible.

• When I am doing a re-login(behind the scenes), could the old auth cookies[even if they are incorrect] still cause an issue?

Only if you are somehow mishandling them on your side. Normally, they get unset or set to a different value when you login.

• Assuming that my steps are correct, what additional step would be required for a 2FA enabled account? Is it even possible to do it behind the scenes for such a user?

Instead of the login API (which does not support 2FA) you'd have to use the clientlogin API, and depending on what response you get call it a second time with the 2FA response.
Clientlogin is an open-ended API that basically just replicates what the web login does, so after submitting the username + password it might ask you for something else (the 2FA response, but also e.g. a new password if your current password is too weak). So it's not trivial to implement. The original intent behind it is to provide the information clients need to build a login form, without having to understand what the elements in that form are for. You can try to only implement the steps that you care about, though - you can look at how the Wikipedia app does it.

you can look at how the Wikipedia app does it.

Some links. Obviously not exhaustive ;-)

• Gerrit: Add basic 2FA support
• GitHub: some commits related to 2FA

Instead of the login API (which does not support 2FA) you'd have to use the clientlogin API, and depending on what response you get call it a second time with the 2FA response.
Clientlogin is an open-ended API that basically just replicates what the web login does, so after submitting the username + password it might ask you for something else (the 2FA response, but also e.g. a new password if your current password is too weak). So it's not trivial to implement. The original intent behind it is to provide the information clients need to build a login form, without having to understand what the elements in that form are for. You can try to only implement the steps that you care about, though - you can look at how the Wikipedia app does it.

I wasn't able to frame my question correctly. The Commons app already uses the clientlogin API and it also has 2FA implemented. My question was regarding handing of re-login scenario.

@Tgr Thanks a lot for such a detailed reply. I will check your response in detail in probably a day or two and will get back with further questions. :)

@Kaartic Thanks for providing these links but as I mentioned, the app has 2FA already implemented and it was mostly inspired by Wikipedia app.

Re-login is not any different from login. It will clear the session (in general if the session times out there is no way to recover it), so you'll need new CSRF tokens.

In T200560#4479712, @maskaravivek wrote:

@Kaartic Thanks for providing these links but as I mentioned, the app has 2FA already implemented and it was mostly inspired by Wikipedia app.

I knew that. I provided the second link with hopes that you would possibly find a solution to your problem by seeing how the Wikipedia app handles that problem (whichs seems to be related to 2FA).

The following commit looks interesting.
https://github.com/wikimedia/apps-android-wikipedia/commit/be2fdb441321a0c0969283204afc30497eaa8b86