Regulator fines Singtel over fixed voice disruption

Thorough probe found cause of 'preventable' incident.

"The incident was within Singtel's control to prevent."

Those were the words of the Infocomm Media Development Authority (IMDA) when it announced the almost $800k (S$1m) fine it imposed on Singapore Telecommunications Limited (Singtel) over the 8 October 2024 fixed voice disruption that lasted for more than four hours.

Impacting approximately 500,000 residential and corporate Singtel clients, the incident also affected emergency call services and customer service lines. Singtel has since taken remediation measures to avoid a repeat of the significant disruption.

Sharing the findings of what was described as a "thorough" probe, the regulator noted: "IMDA's investigation found that Singtel hosted two separate virtualised firewalls on the same hardware – one for its fixed-line voice system (voice system) and another for the monitoring system for home broadband routers and Pay TV set-top boxes (monitoring system). This meant that both virtualised firewalls shared the same hardware memory resources.

"As the monitoring system's virtualised firewall did not have adequate filters installed to protect the hardware against high intensity traffic, the shared memory resources of the hardware were overwhelmed when the monitoring system faced increased traffic intensity on 8 October 2024. This caused the virtualised firewall of the voice system to also malfunction and operate intermittently."

IMDA pointed out that an automatic failover mechanism should have fully redirected voice traffic from the affected voice system to a separate one at an unaffected site.

"The failover did not happen seamlessly due to the intermittency of the affected voice system's virtualised firewall," it went on to explain. "Calls were dropped intermittently as voice traffic alternated between the affected and unaffected voice systems. The incident was resolved only after Singtel fully swung all voice traffic to the unaffected voice system."

Stressing the significance of the disruption, IMDA cited the impact on the public's access to emergency call services and customer service lines for entities like healthcare organisations, banks, and government agencies.

It was highlighted that the incident was not caused by a cyberattack.

Remediation measures taken by Singtel include the implementation of a separate hardware for the telco's voice system and monitoring system, as well as an intervention mechanism in the event of failovers.