Mattermost, Inc.

Mattermost blocks on pinging database

Summary

Mattermost do not go further than “Pinging SQL master database” on startup.

Steps to reproduce

Mattermost team 5.12.3 3.13.1
Ubuntu 18.04
Postgresql 10

config.json :
“SqlSettings”: {
“DriverName”: “postgres”,
“DataSource”: “postgres://mmuser:<my_password>@localhost:5432/mattermost?sslmode=disable&connect_timeout=10”,

I’ve tested db connection with :

psql --dbname=mattermost --username=mmuser --password

I’ve got access to db without any problem

Then I tried to start the server with :

sudo -u mattermost ./bin/mattermost

The last message is “Pinging SQL master database”

Here are the last logs for postgresql :

2019-07-23 09:27:06.537 CEST [17929] mmuser@mattermost ERROR: relation “idx_teams_description” does not exist
2019-07-23 09:27:06.537 CEST [17929] mmuser@mattermost STATEMENT: SELECT $1::regclass

2019-07-22 14:03:39.143 CEST [19023] mmuser@mattermost ERROR: duplicate key value violates unique constraint “users_username_key”
2019-07-22 14:03:39.143 CEST [19023] mmuser@mattermost DETAIL: Key (username)=(surveybot) already exists.
2019-07-22 14:03:39.143 CEST [19023] mmuser@mattermost STATEMENT:
insert into “users” (“id”,“createat”,“updateat”,“deleteat”,“username”,“password”,“authdata”,“authservice”,“email”,
“emailverified”,“nickname”,“firstname”,“lastname”,“position”,“roles”,“allowmarketing”,“props”,“notifyprops”,
“lastpasswordupdate”,“lastpictureupdate”,“failedattempts”,“locale”,“timezone”,“mfaactive”,“mfasecret”) values
($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25);

Expected behavior

The server should start or I should get an error.

Observed behavior

My server’s disk got full, so some sevices stopped working. I’ve clean my disk and restart all services but mattermost does not start anymore.

Update

I’ve upgraded to version 5.13.1 and the issue remains.

Hi @VoyagesDivins, as a first step, wondering if some of these earlier threads might help:

Hi, thanks for replying.

I don’t think it’s a firewall problem as db and service are on the same server.

Also I do not get messages of connection failed. For instance if I intentionnaly make a mistake, say in port number, I immediately get a message

"Failed to ping DB retrying in 10 seconds

When the connection is correct, I don’t get any message.

Another thing that I don’t if revelant : in the config.json the “&” character in postgresql config gets converted to \u0026

I don’t have any special characters in my password.

Hi @VoyagesDivins, a response from a community member: If the interruption (full disk) occurred during install, it might be worth a try to drop the mattermost database and try again.

The problem didn’t occure during install. Server was already up and running.

As you suggest, the problem came from the database.

Table “jobs” had more than 15000 rows.

I did a hard truncate on that table and I was able to start mattermost server again.

Thank you

The problem occured again after a server reboot. This time truncating “jobs” table didn’t help.