Service Incident (April 21st, 2023)

What Happened?

At approximately 1:15pm CT on Friday, April 21st, 2023, RFMS received reports of clients being unable to use their Mobile Applications or anything that runs on the RfmsDataEndpoint service. (Formerly known as MeasureOrderEntryService) Services were failing to restart with the following error message:

"Error 1604: An Exception occurred in the service when handling the control request."

This issue affected a very small number of users. 

What Caused This?

Routine maintenance performed by Microsoft impacted a small number of Service Bus Queues used by RFMS. The cause is still unclear, but a ticket is open with Microsoft to ensure this will not happen again. 

What Do I Need to do?

The following areas have been impacted by this incident and will require adjustments from you:

For Mobile Work Order subscribers:

1. Please reconfigure permissions for installers & field supervisors by logging into admin.rfms.online > Mobile Work Order tile > Key icon in the top right.
2. Once complete, please perform a "sync now" in the RFMS Online tile.

For API subscribers:

1. Please generate new API keys by logging into admin.rfms.online > RFMS Online tile > API button in the top right.
2. Notice your store queue id has changed. Please send this along with your newly generated key to the development team in charge of the integration. 

For Mobile App Users: (includes Measure Mobile, RFMS Mobile, Warehouse Mobile, & CRM)                 **Does NOT include Mobile Work Order Installers

1. Log out of the app
2. Log back in