Push Notifications not working using TPNS


#1

Summary

Not able to get push notifications working using TPNS

Steps to reproduce

Server configured for TPNS using Google Play / iOS apps, users configured for push notifications. Send a user a direct message or mention, push notification never received. Check log and see push error i/o timeout

Expected behavior

User should receive push notification on mobile

Observed behavior

User never receives push notification. Log shows server attempted to send push notification, but error shows i/o timeout
{"level":"error","ts":1528326990.5882158,"caller":"app/notification.go:767","msg":"Device push reported as error for UserId=83h3ac3ump8btmx5tu3iew697y SessionId=ascgyw3hktf5mnhyrpq3dgtcsr message=Post http://push-test.mattermost.com/api/v1/send_push: dial tcp: i/o timeout","user_id":"83h3ac3ump8btmx5tu3iew697y"}

Other notes

I can curl http://push-test.mattermost.com from the server and receive back <html><body>Mattermost Push Proxy</body></html>, but when I ping push-test.mattermost.com it just times out

Edit

What OS and version is the Mattermost server installed?

Ubuntu 16.04

What is your Mattermost server version?

4.10.1

Are you experiencing the issues with the browser webapp, if so which one?

n/a

Are you experiencing the issues with the Mattermost Desktop App, if so what version and OS?

n/a

Are you experiencing the issues with the Mattermost Mobile App, if so what version and OS?

Android v1.8.0 (Build 101) as well as iOS

Can you send a snippet of the Mattermost server logs around the time of the incident?

{"level":"error","ts":1528377906.7466958,"caller":"app/notification.go:767","msg":"Device push reported as error for UserId=97cs51ui7pyx983drb68m7f8xw SessionId=f4iwoef5ajryxjxnho7frjreba message=Post http://push-test.mattermost.com/api/v1/send_push: dial tcp: i/o timeout","user_id":"97cs51ui7pyx983drb68m7f8xw"}
{"level":"error","ts":1528377988.0946994,"caller":"app/notification.go:767","msg":"Device push reported as error for UserId=97cs51ui7pyx983drb68m7f8xw SessionId=f4iwoef5ajryxjxnho7frjreba message=Post http://push-test.mattermost.com/api/v1/send_push: dial tcp: i/o timeout","user_id":"97cs51ui7pyx983drb68m7f8xw"}
{"level":"error","ts":1528378622.523168,"caller":"app/notification.go:767","msg":"Device push reported as error for UserId=66cetriqz7nidxdczawd8h4nse SessionId=un56qp7ftintprpwypgm1pxm6h message=Post http://push-test.mattermost.com/api/v1/send_push: dial tcp: i/o timeout","user_id":"66cetriqz7nidxdczawd8h4nse"}
{"level":"error","ts":1528378632.5719802,"caller":"app/notification.go:767","msg":"Device push reported as error for UserId=66cetriqz7nidxdczawd8h4nse SessionId=un56qp7ftintprpwypgm1pxm6h message=Post http://push-test.mattermost.com/api/v1/send_push: dial tcp: i/o timeout","user_id":"66cetriqz7nidxdczawd8h4nse"}
{"level":"error","ts":1528378641.2676554,"caller":"app/notification.go:767","msg":"Device push reported as error for UserId=nwnaeexaepy9dji7e5495xofbw SessionId=prndz4m55trbuyd73rgwjb87cr message=Post http://push-test.mattermost.com/api/v1/send_push: dial tcp: i/o timeout","user_id":"nwnaeexaepy9dji7e5495xofbw"}
{"level":"error","ts":1528378641.7068503,"caller":"app/notification.go:767","msg":"Device push reported as error for UserId=66cetriqz7nidxdczawd8h4nse SessionId=un56qp7ftintprpwypgm1pxm6h message=Post http://push-test.mattermost.com/api/v1/send_push: dial tcp: i/o timeout","user_id":"66cetriqz7nidxdczawd8h4nse"}

Are you running Mattermost in a container and/or using container orchestration?

No container

Link to debug log

https://drive.google.com/open?id=1vV9t5c4wL1hmTQL6jYLnCYjrBoR4o0RE


#2

Hi @sstoveld!

To give you the best help we can, can you help take a look at the following:

  1. Please provide any relevant additional information from this doc, including server version, etc.: https://docs.mattermost.com/process/support.html.

  2. Review this section on push notifications, along with steps to troubleshoot: https://docs.mattermost.com/administration/config-settings.html#push-notification-contents.

  3. Review these docs regarding TPNS:

Let me know if this helps!


#3

Hi Amy, thanks for the response. I’ve edited my post with the requested information. Unfortunately the links provided did not help. There does not seem to be any documentation that I can find that is relevant to the error I am getting using TPNS. I am using the open source Team Edition, so the HPNS documentation doesn’t seem to fit. It doesn’t seem like it’s a configuration issue, as it seems everything matches up to how it should in the documentation, but it’s just timing out for some reason.

Edit

Also forgot to mention. Push notifications previously worked when I first set up Mattermost. One day they just stopped working without any changes being made to the server. I just upgraded from Mattermost 4.3 to 4.10 last night in hopes that there may have been a fix for the push notifications

Edit 2

I just received one single push notification on my phone today. The first I have received in months. A co-worker also said the same, he received a single push notification today, first in months. Strange that one single notification made it through today. Any ideas?


#4

Hi @sstoveld! Thank you for the additional information!

I will be testing this today - if I can’t seem to reproduce this, I will ask an engineer to take a look at the logs that you have attached earlier.


#5

Hi @sstoveld - Giving you an update here that I have asked a colleague to help take a look at the logs that you sent over earlier.


#6

Hi Amy, thanks for the update. Let me know what you find out. Thanks!


#7

Hi @sstoveld! Just wanted to ask one more additional question:

  1. Can you share a screenshot of what you see when you go to Account Settings -> Security -> View and logout of active sessions? Does it have the apps listed?

#8

Hi Amy, yes I can see the apps listed


#9

Hi @sstoveld! Giving you an update that I have asked our team and other community members to take a look at this but this far nobody hasn’t been able to figure out what the problem might be. I am currently working on researching more information regarding issues around push notifications and TPNS + asking my colleagues on any other tips / questions to help troubleshoot this.