This morning we had an unexpected outage with Cloud POS, while conducting some maintenance on the Hyper-V hosts.

The outage was around from 6:50am until approx. 7:30am.

The maintenance was to fix some packet loss issues we have been experiencing and the fix causes the network cards in the hosts to briefly disconnect for a second or so.

Unfortunately, this caused one of the Load balancer nodes for Cloud POS to disconnect and attempt to failover to the other node.

The second node was in maintenance mode at the time and therefore the cluster went down preventing access to Cloud POS.

We rebooted the primary node which fixed the problem and also removed the 2nd node from maintenance mode.

Investigations will continue, to understand why this happened and implement preventative action in the future.