The ASX has completed an analysis of the events that led to the temporary shut-down of its equities trading system on the morning of 19 September followed by an early market close.
The market has only closed on three previous occasions over the last decade, according to the ASX.
The outage was triggered by a hardware failure that affected the primary ASX Trade database.
The ASX report on the outage, made public today, said that hardware failures do not normally disrupt the exchange, with automatic failover typically ensuring that trading operations can continue.
In addition to maintaining a disaster recovery site (DRS) that replicates its primary data centre, there is redundancy for a number of systems within ASX’s primary DC.
The ASX described the hardware malfunction as “unprecedented”, with failover processes responding in an abnormal fashion.
From 6.57am on 19 September “there was a degradation, and ultimately failure, of the physical database that supports ASX Trade,” the ASX’s report states. “As designed, the backup database at the DRS successfully became the new primary database.”
“The sub-systems that comprise ASX Trade and that utilise the database are designed to automatically reconnect to the new primary database,” the report states.
“This reconnection process was only partially successful. A number of sub-systems incorrectly maintained connection to the failed database.”
Flow-on effects meant that some market participants could not log-on or connect trading operations.
“Similarly, ASX Trading Operations was not able to log-on and perform certain market functions, although alternative control mechanisms were available,” the report states.
During the afternoon “a reference data inconsistency emerged for a group of securities”. “As a result, the system stopped functioning properly for those securities,” the report states. After an investigation revealed the original hardware fault and partial failover was to blame, trading was halted.
The ASX said it is working with technology vendors to understand the underlying cause of the failover issues. The ASX has updated its procedures in an effort to prevent a reoccurrence and boosted hardware monitoring.
“ASX does not decide to close the market lightly. Understandably, concerns have been raised on various fronts,” said ASX managing director and CEO, Dominic Stevens.
“However, we make no apologies for following our long-established and well understood protocols nor for exercising our judgement to ensure the market was fair, orderly and transparent. That is ASX’s responsibility.
“I am determined that in understanding what happened on Monday we will emerge stronger and better able to manage the market’s myriad challenges. We are directing our efforts to ensure there is no recurrence and listening to what our customers say about improving the way we communicate with them. And of course, we are also cooperating closely with ASIC to help with its review.”
ASX reported growth in its operating expenses for FY16 of 6.5 per cent to $170.6 million, which the company attributed in part to investment in its technology transformation program.