Mattermost, Inc.

New messages do not appear until refresh - two and more instances in k8s

Summary

Hello team. I deployed MM to my k8s cluster (RKE/Rancher - with 3 workers) base on the https://docs.mattermost.com/install/install-kubernetes.html (version 5.21 and 5.22) I used 1000users parameter and it created 2 instances of MM. I realised that not all new massages does not appear until I refresh web page or switch to other channel and back. I tested this also with Android and iOS app and it is same. If I scale MM instances to 3 than it appear only each 3rd new message.

Steps to reproduce

Deploy MM base on official document https://docs.mattermost.com/install/install-kubernetes.html
Use “1000users” or more to have more MM replicas
Send sequence of messages and between 2 users and observer if all messages appear in chat windows or no.

Expected behavior

All new messages are visible with out refreshing.

Observed behavior

It looks like it is caused by round-robin load balancer and some issue that MM instances does not communicate to each other or some other bug.
If I scale to only 1 MM replica all works just fine.
Could be the issue the missing license?

@kissi7, yes, our high-availability, multi-server functionality is only activated with a valid enterprise license. While all the posts are written back to the database such that nothing is lost, the communication among the servers is not yet enabled and would exactly match the behaviour you describe.