SOLVED - Nginx Config Ubuntu 16.04


#1

I will need to post this thread in meany small pieces because I get the error message " Sorry, new users can only put 2 links in a post. "
That restriction makes this very complicated.

Anyways, then I need to do it this way…

Summary

Ubuntu 16.04 server with nginx. It’s a rootserver in the internet (Hetzner).
It’s not a server in my local network.
It’s a first time install of mattermost.
Nginx generally runs successfully on the server for other apps.
I can’t get Nginx config to work for mattermost though.
I am trying to get a normal http:// connection first before swapping to ssl.
The server is available on IP-address:8065 and on chat.mydomain.com:8065


#2

Steps to reproduce

I use this nginx conf, generally copied from


upstream backend {
server 127 .0 .0 .1:8065;
}

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=mattermost_cache:10m max_size=3g inactive=120m use_temp_path=off;

server {
listen 80;
server_name chat. mydomain .com;

location ~ /api/v[0-9]+/(users/)?websocket$ {
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection “upgrade”;
client_max_body_size 50M;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Frame-Options SAMEORIGIN;
proxy_buffers 256 16k;
proxy_buffer_size 16k;
proxy_read_timeout 600s;
proxy_pass http://backend;
}

location / {
client_max_body_size 50M;
proxy_set_header Connection “”;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Frame-Options SAMEORIGIN;
proxy_buffers 256 16k;
proxy_buffer_size 16k;
proxy_read_timeout 600s;
proxy_cache mattermost_cache;
proxy_cache_revalidate on;
proxy_cache_min_uses 2;
proxy_cache_use_stale timeout;
proxy_cache_lock on;
proxy_pass http://backend;
}
}


#3

Do you see any errors in the nginx logs when starting up or attempting to connect to the Mattermost server?


#4

Thanks a lot for your answer and sorry that it took me a while to get back to you.

I don’t have an error message in /var/log/nginx access/log or error.log

Simply nothing is happening.


#5

Observed behavior

I get the error message

Safari Can’t Open the Page

Safari can’t open the page “chat. mydomain .com” because the server unexpectedly dropped
the connection. This sometimes occurs when the server is busy. Wait for a few minutes,
and then try again.


#6

The mattermost manual refers to this IP address:

upstream backend {
server 10.10.10.2:8065;
}

(https://docs.mattermost.com/install/config-proxy-nginx.html)

What IP address should I use if Nginx runs on the same machine as the mattermost server?

Or what IP address should I actually put here?

I am currently trying it with 127.0.0.1 but that might be wrong?


#7

Or does this line need any specific attention?

location ~ /api/v[0-9]+/(users/)?websocket$ {

The manual does not mention where I should add specific configuration details.


#8

If nothing is showing up your nginx server might be down. Can you verify that it’s running by posting the output of this command:

$ sudo service nginx status

It should show these two lines in the output:

   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2018-05-13 03:16:39 EDT; 1 day 7h ago

If it’s not running, check the nginx config by running this command:

$ sudo service nginx configtest

#9

Hi Paul

The Nginx server is running because it does serve other services, which work perfectly fine.

Am I looking at the wring log?

I check access.log and error.log in /var/log/nginx

But nothing in there for mattermost.


#10

Do you see any errors in the Mattermost log? If not, try setting the File Log Level in System Console > General > Logging to DEBUG and then reproduce the bug. If you can’t access the System Console, set the config value LogSettings > FileLevel to `DEBUG``. That will provide the most information.

If you don’t, try using the outside IP address instead of 127.0.0.1 for the Mattermost server in the nginx config


#11

Thanks so much for your support Paul.
It’s really very much appreciated.

I tried to put the external IP address instead of 127.0.0.1.
No difference in behavior though.

I am not getting through to mattermost. So there is nothing in the logs.

if I try ti access via browser

chat.mydomain.com

then nothing happens at all.

If I try

chat.mydomain.com:8065

then I can connect to the running Mattermost service directly and it works.

Something is wrong with the nginx config which leads all requests into the eternal emptyness…

I guess it has something to do with the websocket thing, which I do not understand yet.

Or could it have to do something with permissions and the mattermost user?

I am completely lost here.


#12

Ok,

The reason is the other Nginx Proxy.
I am not sure why.

the other server also listens on port 80. It’s a Seafile Server.

server {

listen 80 http2;
listen [::]:80 http2;
server_name cloud.mydomain.com;

rewrite ^ https://$http_host$request_uri? permanent;    # force redirect http to https

# Enables or disables emitting nginx version on error pages and in the "Server" response header$
server_tokens off;

}

Could this be a clue to what I need to change.

It’s a bit strange since on another of my servers I have several proxies configured on Nginx and they all work and are all; listening on 80 / 443…

Hmmm…


#13

Could be further upstream if other devices aren’t causing a similar problem. Have you verified your DNS settings?


#14

Hi Peter

It’s what I said above:

If I try
chat.mydomain.com:8065
then I can connect to the running Mattermost service directly and it works.

So I am sure that DNS is fine.

Can I ask what does

"Could be further upstream … "

mean? I don’t understand that part.


#15

Okay, if the DNS works then it’s not upstream. (Meaning earlier in the request stack.)

Can you turn on debug logging in Nginx and then tail the logs, and then run this shell command?

$ curl -v https://chat.mydomain.com/

That will give us a look at how Nginx is responding as well as the raw output.


#16

Hi Paul

I started a discussion in the nginx forum about my problem because it’s just nginx related and not really mattermost related.

What I did was adding https to the configuration and now it at least works with https://chat.mydomain.com

it does not properly redirect yet though so still does not work properly if both sites are on and someone accesses it with http://chat.mydomain.com

Thanks a lot for your help so far though. It’s really very much appreciated.

Should I find a solution, I will post it here.

I have another issue now with mattermost, which is that I have no system-admin anymore but I opened another thread for that.

I will close this thread once I found a solution and posted it here,

Chris


#17

Hi Paul

Actually I just tried something out and this did the trick: Delete the http2 in the seafile conf.

It seems that the way with the websocket does not work with http2 protocol?
I am not sure what the http2 does anyways.

Anyways, deleting the http2 seems to make all work as expected. The config of the not mattermost part now looks like this:

server {

listen 80;
listen [::]:80;
server_name cloud.mydomain.com;

rewrite ^ https://$http_host$request_uri? permanent; # force redirect http to https

 # Enables or disables emitting nginx version on error pages and in the "Server" response header$

server_tokens off;
}

Now I only need to get the actual server to work, which it doesn’t :wink:

See other thread ;-)))

greetings, Chris