Saturday 5 August 2017;
Tuesday 8 August 2017.
Total outage time:
5th August 2017 10:09am – 11:05am AEST (56-mins)
8th August 2017 11:21am – 11:35am AEST (14-mins)
Normal service has been restored.
RedEye experienced a brief unplanned outage of the RedEyeDMS product, the outage affected all users. The root cause of the outage has been identified and subsequent mitigation plans have been implemented to prevent the issue from reoccurring.
There has been no impact to the integrity of information and data in the system.
We are sorry for any impact you experienced during the outage, and thank you for your understanding.
If you have any further queries or concerns please contact your Customer Success Manager.
RedEye DMS uses an in-memory cache service to optimise various parts of the website’s performance. Specific queries identified in the system were failing to set a TTL on the cache key. As such, they were not expiring from the cache after an elapsed period of time leaving it replete and unable to store new information.
Mitigation strategies to prevent this being a problem in the future include better monitoring of cache services and preventing a full cache from being able to cause an interruption in service.