Late this morning, our team was made aware of an issue with errors appearing on the Citrus dashboard. This was a symptom of the underlying issue of our message queue being full and unable to push and record user events over this time. This means the actions would sometimes be successful (depending on the action).
This directly affects outgoing emails, notifications, recorded activities and some other actions. Our logs show that the first failure began on Tuesday Jan 18 2022 at 15:27:26 (CST) and ended on Wednesday Jan 19 2022 at 12:19:28 (CST). Our team have released a hotfix and started to build a plan to implement better architecture to ensure this never happens again.
Please accept our dearest apology, we take incidents like this to heart and will strive to deliver better service and software in the future.
Sincerely, David Crossman Chief Technology Officer