[SOLVED] Server Crashed - "runtime error: integer divide by zero"?

So my friends and I have been running our own Mattermost server for a couple weeks now without issue. Then out of nowhere, at 15:49 the server crashed/stopped. I’m running Mattermost 3.9.0 on Ubuntu 16.04, and I’ve included the only relevant log info I could find in syslog; as the mattermost log simply said “Stopping Mattermost”. As you can see, the whole thing seems to be correlated with the first log entry of “Starting Daily apt activities”. Any suggestions as to how to prevent this from occurring again? Thanks!

Jul 20 15:48:55 server1 systemd[1]: Starting Daily apt activities...
Jul 20 15:49:27 server1 systemd[1]: Reloading.
Jul 20 15:49:27 server1 systemd[1]: Started ACPI event daemon.
Jul 20 15:49:27 server1 systemd[1]: Stopping Mattermost...
Jul 20 15:49:27 server1 platform[8398]: [2017/07/20 15:49:27 CDT] [INFO] Stopping Server...
Jul 20 15:49:27 server1 platform[8398]: [2017/07/20 15:49:27 CDT] [INFO] Closing SqlStore
Jul 20 15:49:27 server1 platform[8398]: [2017/07/20 15:49:27 CDT] [INFO] stopping websocket hub connections
Jul 20 15:49:27 server1 platform[8398]: [2017/07/20 15:49:27 CDT] [INFO] Server stopped
Jul 20 15:49:27 server1 platform[8398]: 2017/07/20 15:49:27 http: response.WriteHeader on hijacked connection
Jul 20 15:49:27 server1 platform[8398]: 2017/07/20 15:49:27 runtime error: integer divide by zero
Jul 20 15:49:27 server1 systemd[1]: Stopped Mattermost.
Jul 20 15:49:27 server1 systemd[1]: Stopping MySQL Community Server...
Jul 20 15:49:29 server1 systemd[1]: Stopped MySQL Community Server.
Jul 20 15:49:29 server1 systemd[1]: Reloading.
Jul 20 15:49:29 server1 systemd[1]: Started ACPI event daemon.
Jul 20 15:49:29 server1 systemd[1]: Stopped MySQL Community Server.
Jul 20 15:49:30 server1 systemd[1]: Reloading.
Jul 20 15:49:30 server1 systemd[1]: Started ACPI event daemon.
Jul 20 15:49:32 server1 systemd[1]: Reloading.
Jul 20 15:49:32 server1 systemd[1]: Started ACPI event daemon.
Jul 20 15:49:33 server1 systemd[1]: Reloading.
Jul 20 15:49:33 server1 systemd[1]: Started ACPI event daemon.
Jul 20 15:49:33 server1 systemd[1]: Stopped MySQL Community Server.
Jul 20 15:49:33 server1 kernel: [3456696.879083] audit: type=1400 audit(1500583773.610:44): apparmor="STATUS" operation="profile_replace" profile="unconfin$
Jul 20 15:49:33 server1 kernel: [3456696.919040] audit: type=1400 audit(1500583773.650:45): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" n$
Jul 20 15:49:33 server1 kernel: [3456696.919086] audit: type=1400 audit(1500583773.650:46): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" n$
Jul 20 15:49:33 server1 kernel: [3456696.919157] audit: type=1400 audit(1500583773.650:47): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" n$
Jul 20 15:49:33 server1 kernel: [3456697.095348] audit: type=1400 audit(1500583773.826:48): apparmor="STATUS" operation="profile_replace" profile="unconfin$
Jul 20 15:49:34 server1 systemd[1]: Reloading.
Jul 20 15:49:34 server1 systemd[1]: Started ACPI event daemon.
Jul 20 15:49:34 server1 systemd[1]: Reloading.
Jul 20 15:49:34 server1 systemd[1]: Started ACPI event daemon.
Jul 20 15:49:34 server1 systemd[1]: Starting MySQL Community Server...
Jul 20 15:49:34 server1 kernel: [3456697.582936] audit: type=1400 audit(1500583774.314:49): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" n$
Jul 20 15:49:34 server1 kernel: [3456697.582983] audit: type=1400 audit(1500583774.314:50): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" n$
Jul 20 15:49:34 server1 kernel: [3456697.583051] audit: type=1400 audit(1500583774.314:51): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" n$
Jul 20 15:49:35 server1 systemd[1]: Started MySQL Community Server.
Jul 20 15:49:37 server1 systemd[1]: Reloading.
Jul 20 15:49:37 server1 systemd[1]: Started ACPI event daemon.
Jul 20 15:49:37 server1 systemd[1]: Stopping MySQL Community Server...
Jul 20 15:49:39 server1 systemd[1]: Stopped MySQL Community Server.
Jul 20 15:49:39 server1 systemd[1]: Starting MySQL Community Server...
Jul 20 15:49:39 server1 kernel: [3456702.723501] audit: type=1400 audit(1500583779.454:52): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" n$
Jul 20 15:49:39 server1 kernel: [3456702.723548] audit: type=1400 audit(1500583779.454:53): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" n$
Jul 20 15:49:39 server1 kernel: [3456702.723638] audit: type=1400 audit(1500583779.454:54): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" n$
Jul 20 15:49:40 server1 systemd[1]: Started MySQL Community Server.
Jul 20 15:49:47 server1 systemd[1]: Started Daily apt activities.
Jul 20 15:49:47 server1 systemd[1]: apt-daily.timer: Adding 3h 53min 14.035635s random time.
Jul 20 15:49:47 server1 systemd[1]: apt-daily.timer: Adding 9h 3min 44.540383s random time.

Hi @jibberish,

Thanks for your feedback,

v4.0.1 released this month and I wonder if upgrading to the latest version would resolve this issue for you? You can find the download links here

Let us know how it goes and whether your issue is resolved by upgrading.

Thanks!

Thank you, I did not know that there was a newer version of the server available. I guess I should subscribe to a mailing list for alerts or something.

Hi @jibberish,

You can sign up for our newsletter here :slight_smile:

We are also facing same issue .what caused this issue ? Is there any work around for this other than upgrade

Hi @Deepa,

Thanks for your question,

Difficult to say what the cause was but we recommend upgrading to the latest version as there have been numerous fixes and security updates.

I realize this is a very old post and I’m surprised there hasn’t been an answer to this issue yet. Mattermost is stopping because MySQL is being updated by the AppArmor Ubuntu module. Once the upgrade is complete, AppArmor starts MySQL, but leaves Mattermost hanging. To get around this issue, I created a script that checks to see if Mattermost is stopped and if it is, try to start it. Then I added execution of this script to the sudo cron job. Hope this helps anyone that has been running into this issue.

Hi @le3bl and welcome to the Mattermost forums!

So you’re saying that the daily automatic upgrades are causing issues for Mattermost when Ubuntu restarts the MySQL server? If so, I think this is expected, because Mattermost cannot work without the MySQL server, so if possible, you should either not restart it automatically or make sure that the Mattermost unit also restarts in case of an error (this can be achieved with a proper sytemd unit, f.ex.).
The systemd unit of the Omnibus installation f.ex. uses the Requires flag to denote that the database server is required for the operation, which means that Mattermost will automatically stop when the database server is bein restarted and systemd will also take care of the restart of the Mattermost application server then in the correct order.