We're Hiring!

Mattermost, Inc.

Error retrieving the token

Hello,

Since installing the Mattermost 5.25.1 update and trying to log into Mattermost with Gitlab (we are using Gitlab to log in) I have an error message that pops up randomly: “Error retrieving the token”. Indeed, when we try to connect to Mattermost, this message appears but not all the time, and I cannot find the problem.

I started by updating Gitlab because I thought the problem came from there (Gitlab was in API v3 and Mattermost from a certain version no longer supports API v3 integrations from what I could read , so I made updates to Gitlab until I reached API v4) but that did not solve the problem.

I have been able to read about topics where people had a similar problem and maybe it is Let’s Encrypt certificates, we are using Let’s Encrypt for our certificates, could this be the problem?

Mattermost version: 5.25.1
Gitlab version: 12.9.2

Any solutions for this?

Hello, @Reapers59

Since you mentioned that the issue happens intermittently after the upgrade to Mattermost 5.25.1, may I ask the following questions?

  • Which version were you running on prior to upgrading the Mattermost server?
  • When you mentioned that the error pops up randomly, does it appear during the authentication process or after (Eg. when sending message, uploading attachments)?
  • Do you see anything in the mattermost.log when the issue happens so we can make sense of what’s going on in the backend?

With regards to the topics you came across, would you mind sharing the links here so I can read up on it as well before we consider the Let’sEncrypt being the root cause to it?

Keep me posted.

1 Like

Hi @ahmaddanial

Thanks you for your answer.

  • Looks to me like we were on version 5.22 but I’m not sure there is a way to check this?
  • The error appears when authenticating with Gitlab, once I click on “Sign in” to login. By randomly I mean that sometimes I click on “Sign in” and this error does not appear. So if we’re already connected, we don’t get the error until at least the next reconnection if we’re unlucky.
  • In mattermost.log I have errors like:

{“level”:“error”,“ts”:1599722273.8641682,“caller”:“web/oauth.go:278”,“msg”:“AuthorizeOAuthUser: Erreur lors de la récupération du jeton, response_body={“error”:“invalid_grant”,“error_description”:“The provided authorization grant is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client.”}, status_code=401”}
{“level”:“error”,“ts”:1599722384.425674,“caller”:“web/oauth.go:278”,“msg”:“AuthorizeOAuthUser: Erreur lors de la récupération du jeton, response_body={“error”:“invalid_grant”,“error_description”:“The provided authorization grant is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client.”}, status_code=401”}
{“level”:“error”,“ts”:1599726406.3801544,“caller”:“mlog/log.go:175”,“msg”:“Error getting access token from DB before deletion.”,“path”:"/api/v4/users/logout",“request_id”:“tq9455rn4trrzfufh3fi5q7ifo”,“ip_addr”:“192.168.6.55”,“user_id”:“d4rqtmbpx7gnjcui5imrecddye”,“method”:“POST”,“err_where”:“RevokeAccessToken”,“http_code”:400,“err_details”:""}
{“level”:“error”,“ts”:1599726413.112087,“caller”:“mlog/log.go:175”,“msg”:“Error getting access token from DB before deletion.”,“path”:"/api/v4/users/logout",“request_id”:“h8swnjd8nfbmupsm8y115nr7je”,“ip_addr”:“192.168.6.55”,“user_id”:“d4rqtmbpx7gnjcui5imrecddye”,“method”:“POST”,“err_where”:“RevokeAccessToken”,“http_code”:400,“err_details”:""}

I watched this one: https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2984 but that doesn’t mean anything to me, especially since I’m a novice in the field…

I also looked this one: [SOLVED] Mattermost Gitlab - Token request failed but I don’t have quite the same thing even though there it is looks like.

Hi, @Reapers59

You are welcome. Moving along with the updates you shared.

Point 1

You can check the version of Mattermost that you are running on if through the UI (About Mattermost) or from the backend using mattermost version CLI. For example:

ahmaddanial@mattermost:/opt/mattermost$ ./bin/mattermost version
...
...
Version: 5.26.0
Build Number: 5.26.1
Build Date: Tue Aug 25 22:21:22 UTC 2020
Build Hash: dd2f8185dbd9925b25e838cc4f6b12a92a074dec
Build Enterprise Ready: true
DB Version: 5.26.0

Point 2

In this case, it seems that the problem is not constantly reproducible. So, we need to find out under what circumstances does it occur specifically since “unlucky” might not be enough to pin point the root cause.

Point 3

Based on the log snippet you provided, we have 2 different errors that are highlighted:

  • The provided authorization grant is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client
  • Error getting access token from DB before deletion (I don’t see how this is directly related to the issue since it is calling the /api/v4/users/logout API but just putting it here in case we need to dig into it further)

After performing some research, the first error relates to a topic raised before in the forum - Mattermost Gitlab oauth problem - which in turn relates to the websockets that was switched off on the proxy level (in this example, NGINX).

Therefore, may I know if you have any proxy configured on your Mattermost and how the configuration is like? Did you configure according to the documentation if you are using NGINX?

Else, can you share which documentation you referred to when configuring it?

Another point that I would like to check is whether the issue continues to persist if you clear off every session logged in for the affected accounts. If you have the chance to log in without facing the error message on the desktop / web browser, navigate to Account Settings > Security > View and Log Out of Active Sessions and log out from all the sessions that was logged in before.

If you have access to the database, you should be able to confirm that all sessions are cleared by running the following command:

SELECT u.Email, s.Token, s.DeviceId, s.Props FROM Sessions AS s LEFT JOIN Users AS u ON s.UserId = u.Id WHERE u.Email = "<email>";

This command should return zero results if all sessions are cleared. Attempt to log in again after that to observe if the issue continues to persist.

1 Like
  • Point 1

  • Point 2

I tried a lot of manipulations (change browser, mobile application, mobile browser, clear the cache …), but I have the impression that the problem is really happening randomly. I also think that the problem appeared since the automatic renewal of Let’s Encrypt certificates (08/25/2020), before that date, we did not have the problem, or else nobody noticed it.

  • Point 3

We are not using nginx at all, we are using an apache reverse proxy.

Unfortunately, I don’t have the documentation that was used for the reverse proxy (there are two).

I followed the steps to disconnect all the sessions I had (there were quite a few) and the issue doesn’t seem to be happening at the moment, I’ll let you know if the issue comes back to me.

EDIT: I cleared the cache a few minutes later and the problem reappeared.

I don’t think I have access to the database to run the SQL query :confused:

The problem went away on its own, inexplicable…

Thank you all the same for your help :wink:

1 Like