[SOLVED] Runtime error: invalid memory address or nil pointer dereference

Troubleshooting
1

Summary

I saw the above error in the log. Just reporting it for dev feedback (didn’t see any specific issue at this time)

Steps to reproduce

Mattermost 3.10.0, local install on CentOS 7 - otherwise conditions unknown (nothing else in the log around that time)

Expected behavior

No runtime errors :slight_smile:

Observed behavior

Logs:

[2017/06/22 01:55:17 AEST] [EROR] Please check the std error output for the stack trace
[2017/06/22 01:55:17 AEST] [EROR] [runtime error: invalid memory address or nil pointer dereference]
[2017/06/22 01:55:35 AEST] [EROR] Please check the std error output for the stack trace
[2017/06/22 01:55:35 AEST] [EROR] [runtime error: invalid memory address or nil pointer dereference]

stderr:

Jun 22 01:55:17 redacted.server platform[4298]: goroutine 6497627 [running]:
Jun 22 01:55:17 redacted.server platform[4298]: runtime/debug.Stack(0x1796e20, 0xc425074440, 0x11c4e38)
Jun 22 01:55:17 redacted.server platform[4298]: /usr/local/go/src/runtime/debug/stack.go:24 +0x79
Jun 22 01:55:17 redacted.server platform[4298]: runtime/debug.PrintStack()
Jun 22 01:55:17 redacted.server platform[4298]: /usr/local/go/src/runtime/debug/stack.go:16 +0x22
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/vendor/github.com/gorilla/handlers.recoveryHandler.log(0x1796f20, 0xc4211d1910, 0x1796f60, 0x1805b60, 0x1, 0xc4250743e0, 0x1, 0x1)
Jun 22 01:55:17 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/vendor/github.com/gorilla/handlers/recovery.go:89 +0x70
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/vendor/github.com/gorilla/handlers.recoveryHandler.ServeHTTP.func1(0x17a0ee0, 0xc426333340, 0x1796f20, 0xc4211d1910, 0x1796f60, 0x1805b60, 0x1)
Jun 22 01:55:17 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/vendor/github.com/gorilla/handlers/recovery.go:74 +0xef
Jun 22 01:55:17 redacted.server platform[4298]: panic(0x105d700, 0x17cf3d0)
Jun 22 01:55:17 redacted.server platform[4298]: /usr/local/go/src/runtime/panic.go:489 +0x2cf
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/app.SendNotifications(0xc425a9e480, 0xc424a5ac60, 0xc424a5a580, 0xc4257bd7c0, 0xc423a16000, 0x0, 0xc4246da640, 0xc4241f7388)
Jun 22 01:55:17 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/app/notification.go:157 +0x956
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/app.handlePostEvents(0xc425a9e480, 0x0, 0x0, 0x1, 0xc421c8b8f1)
Jun 22 01:55:17 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/app/post.go:164 +0x2b9
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/app.CreatePost(0xc425a9e480, 0x0, 0x0, 0xc42219f801, 0xc4279cd9e0, 0x1a)
Jun 22 01:55:17 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/app/post.go:120 +0x3a9
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/app.CreatePostAsUser(0xc425a9e480, 0x1a, 0xc421855440)
Jun 22 01:55:17 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/app/post.go:37 +0x2bc
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/api.createPost(0xc425a9e240, 0x17a0ee0, 0xc426333340, 0xc426f23500)
Jun 22 01:55:17 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/api/post.go:63 +0x147
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/api.handler.ServeHTTP(0x11e3370, 0x1000001010001, 0x17a0ee0, 0xc426333340, 0xc426f23500)
Jun 22 01:55:17 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/api/context.go:234 +0xc46
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/api.(*handler).ServeHTTP(0xc421dc5e10, 0x17a0ee0, 0xc426333340, 0xc426f23500)
Jun 22 01:55:17 redacted.server platform[4298]: <autogenerated>:1 +0x6b
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc421d79130, 0x17a0ee0, 0xc426333340, 0xc426f23500)
Jun 22 01:55:17 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/vendor/github.com/gorilla/mux/mux.go:114 +0x10c
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/app.(*CorsWrapper).ServeHTTP(0xc4211d1910, 0x17a0ee0, 0xc426333340, 0xc426f23200)
Jun 22 01:55:17 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/app/server.go:76 +0x78
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/vendor/github.com/gorilla/handlers.recoveryHandler.ServeHTTP(0x1796f20, 0xc4211d1910, 0x1796f60, 0x1805b60, 0x1, 0x17a0ee0, 0xc426333340, 0xc426f23200)
Jun 22 01:55:17 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/vendor/github.com/gorilla/handlers/recovery.go:78 +0xb9
Jun 22 01:55:17 redacted.server platform[4298]: github.com/mattermost/platform/vendor/github.com/gorilla/handlers.(*recoveryHandler).ServeHTTP(0xc4224780f0, 0x17a0ee0, 0xc426333340, 0xc426f23200)
Jun 22 01:55:17 redacted.server platform[4298]: <autogenerated>:59 +0x87
Jun 22 01:55:17 redacted.server platform[4298]: net/http.serverHandler.ServeHTTP(0xc422628000, 0x17a0ee0, 0xc426333340, 0xc426f23200)
Jun 22 01:55:17 redacted.server platform[4298]: /usr/local/go/src/net/http/server.go:2568 +0x92
Jun 22 01:55:17 redacted.server platform[4298]: net/http.(*conn).serve(0xc422ff4140, 0x17a1ae0, 0xc4220a2240)
Jun 22 01:55:17 redacted.server platform[4298]: /usr/local/go/src/net/http/server.go:1825 +0x612
Jun 22 01:55:17 redacted.server platform[4298]: created by net/http.(*Server).Serve
Jun 22 01:55:17 redacted.server platform[4298]: /usr/local/go/src/net/http/server.go:2668 +0x2ce
Jun 22 01:55:35 redacted.server platform[4298]: goroutine 6498030 [running]:
Jun 22 01:55:35 redacted.server platform[4298]: runtime/debug.Stack(0x1796e20, 0xc423cd1c90, 0x11c4e38)
Jun 22 01:55:35 redacted.server platform[4298]: /usr/local/go/src/runtime/debug/stack.go:24 +0x79
Jun 22 01:55:35 redacted.server platform[4298]: runtime/debug.PrintStack()
Jun 22 01:55:35 redacted.server platform[4298]: /usr/local/go/src/runtime/debug/stack.go:16 +0x22
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/vendor/github.com/gorilla/handlers.recoveryHandler.log(0x1796f20, 0xc4211d1910, 0x1796f60, 0x1805b60, 0x1, 0xc423cd1c30, 0x1, 0x1)
Jun 22 01:55:35 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/vendor/github.com/gorilla/handlers/recovery.go:89 +0x70
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/vendor/github.com/gorilla/handlers.recoveryHandler.ServeHTTP.func1(0x17a0ee0, 0xc4230f6700, 0x1796f20, 0xc4211d1910, 0x1796f60, 0x1805b60, 0x1)
Jun 22 01:55:35 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/vendor/github.com/gorilla/handlers/recovery.go:74 +0xef
Jun 22 01:55:35 redacted.server platform[4298]: panic(0x105d700, 0x17cf3d0)
Jun 22 01:55:35 redacted.server platform[4298]: /usr/local/go/src/runtime/panic.go:489 +0x2cf
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/app.SendNotifications(0xc4242857a0, 0xc4263a74a0, 0xc4263a6fd0, 0xc424163400, 0xc426ffac00, 0x0, 0xc424677a90, 0xc4242c1388)
Jun 22 01:55:35 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/app/notification.go:157 +0x956
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/app.handlePostEvents(0xc4242857a0, 0x0, 0x0, 0x1, 0xc427199021)
Jun 22 01:55:35 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/app/post.go:164 +0x2b9
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/app.CreatePost(0xc4242857a0, 0x0, 0x0, 0xc42219f801, 0xc426ffa720, 0x1a)
Jun 22 01:55:35 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/app/post.go:120 +0x3a9
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/app.CreatePostAsUser(0xc4242857a0, 0x1a, 0xc421855440)
Jun 22 01:55:35 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/app/post.go:37 +0x2bc
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/api.createPost(0xc424285680, 0x17a0ee0, 0xc4230f6700, 0xc4260d4000)
Jun 22 01:55:35 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/api/post.go:63 +0x147
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/api.handler.ServeHTTP(0x11e3370, 0x1000001010001, 0x17a0ee0, 0xc4230f6700, 0xc4260d4000)
Jun 22 01:55:35 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/api/context.go:234 +0xc46
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/api.(*handler).ServeHTTP(0xc421dc5e10, 0x17a0ee0, 0xc4230f6700, 0xc4260d4000)
Jun 22 01:55:35 redacted.server platform[4298]: <autogenerated>:1 +0x6b
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc421d79130, 0x17a0ee0, 0xc4230f6700, 0xc4260d4000)
Jun 22 01:55:35 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/vendor/github.com/gorilla/mux/mux.go:114 +0x10c
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/app.(*CorsWrapper).ServeHTTP(0xc4211d1910, 0x17a0ee0, 0xc4230f6700, 0xc425193d00)
Jun 22 01:55:35 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/app/server.go:76 +0x78
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/vendor/github.com/gorilla/handlers.recoveryHandler.ServeHTTP(0x1796f20, 0xc4211d1910, 0x1796f60, 0x1805b60, 0x1, 0x17a0ee0, 0xc4230f6700, 0xc425193d00)
Jun 22 01:55:35 redacted.server platform[4298]: /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/platform/vendor/github.com/gorilla/handlers/recovery.go:78 +0xb9
Jun 22 01:55:35 redacted.server platform[4298]: github.com/mattermost/platform/vendor/github.com/gorilla/handlers.(*recoveryHandler).ServeHTTP(0xc4224780f0, 0x17a0ee0, 0xc4230f6700, 0xc425193d00)
Jun 22 01:55:35 redacted.server platform[4298]: <autogenerated>:59 +0x87
Jun 22 01:55:35 redacted.server platform[4298]: net/http.serverHandler.ServeHTTP(0xc422628000, 0x17a0ee0, 0xc4230f6700, 0xc425193d00)
Jun 22 01:55:35 redacted.server platform[4298]: /usr/local/go/src/net/http/server.go:2568 +0x92
Jun 22 01:55:35 redacted.server platform[4298]: net/http.(*conn).serve(0xc4241ea0a0, 0x17a1ae0, 0xc4249d5b80)
Jun 22 01:55:35 redacted.server platform[4298]: /usr/local/go/src/net/http/server.go:1825 +0x612
Jun 22 01:55:35 redacted.server platform[4298]: created by net/http.(*Server).Serve
Jun 22 01:55:35 redacted.server platform[4298]: /usr/local/go/src/net/http/server.go:2668 +0x2ce
2

Hi @gubbins,

Thanks for your feedback,

Does this information in our documentation help?

3

I don’t think so @lindy65 - we don’t do any manual deletions from the database…

4

Thanks @gubbins,

I’ve had a look at previous issues reporting this error message and none really match your environment so I’ll ask our devs for feedback

5

Hi @gubbins,
Thanks for sending these error logs. Could you please help answer the following:
- Is this a fresh install or an upgrade?
- Have you seen the error first time or it also logged in the past?
- If first time, do you have any recent change in settings (like in email/notifications)? If yes, what are those?

6

Hi,

  • it’s an upgrade (we’ve had this install a long time)
  • don’t recall seeing it before
  • no settings changes I’m aware of…

Sorry this isn’t very helpful!

7

SendNotifications (https://github.com/mattermost/platform/blob/v3.10.0/app/notification.go#L157) has not change for a while, especially if you’re upgrade came from v3.9.

The panic seemed to be related to email notification - to a certain user id (that no longer exists) or notification props that is not defined.

Honestly, I was not able to reproduce your case. I’m afraid we need further details from your end to further investigate the problem.

8

No worries, thanks for trying. I don’t think it’s happened again.

9

Thanks @gubbins, I’ll close this off for now… Please let us know if you experience problems again…