We're Hiring!

Mattermost, Inc.

ProxyPass and RewriteRules for well-known not working (Docker, Apache2, ISPconfig) leads to no renewal Lets Encrypt

Hi everybody,

I am pretty glad to got Mattermost running, such a great advance for our team! Being still quiet new to this I am running in some issue. After searching and trying a lot, I thought it is worth to ask you guys. So thanks in advance for any help. Well the problem is that my case is a combination of “search strings” that brings up a lot of different articles but mostly not a matching one for my situation.

Main problem

Old story the Lets Encrypt Renewal fails in my installation.
But, I can narrow it down a bit.

Mostly the thing is:

https://sub.mydomain.de/.well-known/acme-challenge/

is not reachable, well it kind of is. When I try hitting it, I see a response but after that, MM sudently redirects internally to some url and answers with something like => “sorry, site can not be found”. The exact route would be:

https://sub.mydomain.de/login?redirect_to=%2F.well-known%2Facme-challenge

Consequence

This leads to the problem that Lets Encrypt won’t be able to to its renewal routines and this means, some day SSL will fail. I am really looking forward to fix that :wink:

Ideas

Well as far as I can understand it, there is something wrong with the well-known RewriteRules and ProxyPass Rules. But somehow (cause of the) redirect in the browser it seems like MM is doing something on it self. Maybe any hints on that?

Config

MM is running in a docker container V 5.22 on a Ubuntu 18 LTS Webserver.
I have many domains and subdomains running on my server, also some other docker containers.
I am using ISPconfig V 3.1.15 to organize domains, let Lets Encrypt do the work for certs and to link the docker stuff to my domains I usually use the Apache directive option field to set things up.
Well, in other cases this works also fine for Lets Encrypt and docker and ProxyPass Rules.

But in this case, as described above anyhow things get messed up.

Here is my Apache directive

ProxyPreserveHost On
ProxyPass /.well-known !

# Set web sockets
RewriteEngine On
RewriteCond %{REQUEST_URI} /api/v[0-9]+/(users/)?websocket [NC,OR]
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC,OR]
RewriteCond %{HTTP:CONNECTION} ^Upgrade$ [NC]
RewriteRule .* ws://127.0.0.1:8065%{REQUEST_URI} [P,QSA,L]

RewriteCond %{REQUEST_URI} !^/.well-known/$
RewriteCond %{HTTPS} !on
RewriteRule (.*) https://sub.mydomain.de$1 [R,L]

<Location />
      Require all granted
      ProxyPass /.well-known !
      ProxyPass http://127.0.0.1:8065/
      ProxyPassReverse http://127.0.0.1:8065/
      ProxyPassReverseCookieDomain 127.0.0.1 sub.mydomain.de
</Location>

So I really hope some as a more profound knowledge of ProxyPass than I do and could give me a hint, what I am doing wrong, or maybe where to have a closer look to.

Thankfully hoping to get some insights.
Have great day.
Veit