Mattermost, Inc.

Server stopped after upgrade (SOLVED)

FINAL Fix

Remove TLS from config.json

Summary

After running latest upgrade procedures that I have noticed it changed a bit from the previous versions…

Never had seen such a issue before this upgrade.

Other changes done were to replace platform with mattermost in mattermost.service

Steps to reproduce

Running on Centos 7, with MYSQL 5.7 and nginx Proxy 1.14.0
Using 5.1.0 and had previously version 5.0

Expected behavior

Mattermost must be accessible behind the proxy.

Observed behavior

[root@chat logs]# tail -f mattermost.log 
{"level":"error","ts":1531657420.5292244,"caller":"web/handlers.go:140","msg":"Invalid or missing file_id parameter in request URL","path":"/api/v4/files/undefined","request_id":"6ufrcbsrs3ygbf8ubi48a3hd7e","ip_addr":"89.103.72.238","user_id":"48bbj6nc97f55npw6br7gxo96h","method":"GET","err_where":"Context","http_code":400,"err_details":""}
{"level":"error","ts":1531657429.729779,"caller":"web/handlers.go:140","msg":"Invalid or missing file_id parameter in request URL","path":"/api/v4/files/undefined","request_id":"hshsepjb3prhjbww8pqsiai4xe","ip_addr":"89.102.15.184","user_id":"1zim4ugtd7rdmdsmjim4mixrow","method":"GET","err_where":"Context","http_code":400,"err_details":""}
{"level":"error","ts":1531657782.885846,"caller":"app/notification.go:871","msg":"Device push reported as error for UserId=48bbj6nc97f55npw6br7gxo96h SessionId=6u8k4cmo8fbyuk7x1xi3hqk3ra message=Failed because of missing device Id serverId=pa8mun94ebdgbx9ti46hkshkwr","user_id":"48bbj6nc97f55npw6br7gxo96h"}
{"level":"error","ts":1531657888.4722126,"caller":"app/notification.go:871","msg":"Device push reported as error for UserId=48bbj6nc97f55npw6br7gxo96h SessionId=6u8k4cmo8fbyuk7x1xi3hqk3ra message=Failed because of missing device Id serverId=pa8mun94ebdgbx9ti46hkshkwr","user_id":"48bbj6nc97f55npw6br7gxo96h"}
{"level":"error","ts":1531658125.0959096,"caller":"app/notification.go:871","msg":"Device push reported as error for UserId=48bbj6nc97f55npw6br7gxo96h SessionId=6u8k4cmo8fbyuk7x1xi3hqk3ra message=Failed because of missing device Id serverId=pa8mun94ebdgbx9ti46hkshkwr","user_id":"48bbj6nc97f55npw6br7gxo96h"}
{"level":"error","ts":1531774924.2835665,"caller":"app/notification.go:871","msg":"Device push reported as error for UserId=48bbj6nc97f55npw6br7gxo96h SessionId=6u8k4cmo8fbyuk7x1xi3hqk3ra message=Failed because of missing device Id serverId=pa8mun94ebdgbx9ti46hkshkwr","user_id":"48bbj6nc97f55npw6br7gxo96h"}
{"level":"error","ts":1532040361.9166467,"caller":"sqlstore/upgrade.go:96","msg":"Database schema version 5.1.0 is no longer supported. This Mattermost server supports automatic upgrades from schema version 3.0.0 through schema version 5.0.0. Downgrades are not supported. Please manually upgrade to at least version 3.0.0 before continuing"}
{"level":"error","ts":1532040594.0696962,"caller":"sqlstore/upgrade.go:96","msg":"Database schema version 5.1.0 is no longer supported. This Mattermost server supports automatic upgrades from schema version 3.0.0 through schema version 5.0.0. Downgrades are not supported. Please manually upgrade to at least version 3.0.0 before continuing"}
{"level":"error","ts":1532040628.8794994,"caller":"sqlstore/upgrade.go:96","msg":"Database schema version 5.1.0 is no longer supported. This Mattermost server supports automatic upgrades from schema version 3.0.0 through schema version 5.0.0. Downgrades are not supported. Please manually upgrade to at least version 3.0.0 before continuing"}
{"level":"error","ts":1532041740.980578,"caller":"jobs/jobs_watcher.go:70","msg":"Error occurred getting all pending statuses: SqlJobStore.GetAllByStatus: We couldn't get the jobs, Status=pending, invalid connection"}

Nginx proxy logs:

[root@nginx2017 nginx]# tail -f error.log
2018/07/20 01:03:52 [error] 4494#4494: *665 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 181.49.91.153, server: chat.uhlhosting.ch, request: "POST /api/v4/users/status/ids HTTP/1.1", upstream: "http://172.18.111.30:8065/api/v4/users/status/ids", host: "chat.uhlhosting.ch"
2018/07/20 01:04:12 [error] 4494#4494: *669 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 181.49.91.153, server: chat.uhlhosting.ch, request: "GET /api/v4/websocket HTTP/1.1", upstream: "http://172.18.111.30:8065/api/v4/websocket", host: "chat.uhlhosting.ch"
2018/07/20 01:04:52 [error] 4494#4494: *674 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 181.49.91.153, server: chat.uhlhosting.ch, request: "POST /api/v4/users/status/ids HTTP/1.1", upstream: "http://172.18.111.30:8065/api/v4/users/status/ids", host: "chat.uhlhosting.ch"
2018/07/20 01:05:52 [error] 4494#4494: *680 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 181.49.91.153, server: chat.uhlhosting.ch, request: "POST /api/v4/users/status/ids HTTP/1.1", upstream: "http://172.18.111.30:8065/api/v4/users/status/ids", host: "chat.uhlhosting.ch"
2018/07/20 01:06:52 [error] 4494#4494: *682 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 181.49.91.153, server: chat.uhlhosting.ch, request: "POST /api/v4/users/status/ids HTTP/1.1", upstream: "http://172.18.111.30:8065/api/v4/users/status/ids", host: "chat.uhlhosting.ch"
2018/07/20 01:07:52 [error] 4494#4494: *687 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 181.49.91.153, server: chat.uhlhosting.ch, request: "POST /api/v4/users/status/ids HTTP/1.1", upstream: "http://172.18.111.30:8065/api/v4/users/status/ids", host: "chat.uhlhosting.ch"
2018/07/20 01:08:52 [error] 4494#4494: *691 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 181.49.91.153, server: chat.uhlhosting.ch, request: "POST /api/v4/users/status/ids HTTP/1.1", upstream: "http://172.18.111.30:8065/api/v4/users/status/ids", host: "chat.uhlhosting.ch"
2018/07/20 01:09:52 [error] 4494#4494: *701 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 181.49.91.153, server: chat.uhlhosting.ch, request: "POST /api/v4/users/status/ids HTTP/1.1", upstream: "http://172.18.111.30:8065/api/v4/users/status/ids", host: "chat.uhlhosting.ch"
2018/07/20 01:10:52 [error] 4494#4494: *718 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 181.49.91.153, server: chat.uhlhosting.ch, request: "POST /api/v4/users/status/ids HTTP/1.1", upstream: "http://172.18.111.30:8065/api/v4/users/status/ids", host: "chat.uhlhosting.ch"
2018/07/20 01:11:13 [error] 4494#4494: *722 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 181.49.91.153, server: chat.uhlhosting.ch, request: "GET /api/v4/websocket HTTP/1.1", upstream: "http://172.18.111.30:8065/api/v4/websocket", host: "chat.uhlhosting.ch"
 2018/07/20 01:11:52 [error] 4494#4494: *726 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 181.49.91.153, server: chat.uhlhosting.ch, request: "POST /api/v4/users/status/ids HTTP/1.1", upstream: "http://172.18.111.30:8065/api/v4/users/status/ids", host: "chat.uhlhosting.ch"
2018/07/20 01:11:53 [error] 4494#4494: *734 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 181.49.91.153, server: chat.uhlhosting.ch, request: "POST /api/v4/users/status/ids HTTP/1.1", upstream: "http://172.18.111.30:8065/api/v4/users/status/ids", host: "chat.uhlhosting.ch"

Problem solved, since it was caused by the TLS in config.json .

To be honest this TLS/SSL stuff and not knowing what and how to do best config is confusing. Would like to see improvements here.