Some workstations for Hong Kong’s glitch-prone air traffic management system (ATMS) have required additional reboots on about 20 occasions – most during peak hours – since April, according to sources. “Computer crashes’” were to blame, frontline air traffic controllers told FactWire.
Problems with the newly installed Raytheon Auto Trac III system – also known as AT3 – have reportedly intensified since FactWire revealed in May that some ATMS workstations had to be restarted fortnightly to prevent sluggish performance.
Meanwhile, the Air Traffic Controllers Guild of India has warned of a possible repeat of the 1996 Indian air crash that killed 349 people in an emphatic letter lambasting the AT3 system, which is also being used at the New Delhi airport.
The new ATMS workstations are supposed to provide integrated aviation, surveillance and communication data for air traffic controllers to manage flights in and out of the Hong Kong airport.
Since last December, the Civil Aviation Department (CAD) has started performing rotational restarts every fortnight on 53 workstations at the new air traffic control centre and the control tower, in an attempt to prevent sluggish performance and system crashes.
While some workstations even require weekly reboots, they are scheduled to take place from midnight to 7am, when the air traffic is relatively light.
CAD previously said that these regular reboots were part of its “routine maintenance procedure” to ensure safe and smooth operations.
However, according to frontline air traffic controllers, some workstations continue to suffer glitches days after their scheduled reboots and have already caused about 20 manual hardware or software restarts, with the majority of the incidents occurring during peak hours.
FactWire has learned of the details of 14 such manual restarts which happened between late April and August (see table below).
Several ATMS workstations have reportedly continued to experience performance issues that resulted in delayed flight data updates, frozen screens and unresponsiveness to commands.
A workstation has to be rebooted manually by support staff every time an incident occurs, while the controller working at that desk will need to be relocated to another functioning workstation.
The reboot process normally takes five to 15 minutes, causing major disruption to day-to-day operations.
“Only one controller position would have the jurisdiction [for] a particular group of aircraft on a particular route at one time,” an air traffic controller who asked to remain anonymous said.
“It would take time for the controller to realise the malfunction, report the problem, and it would also take time to allow the support staff to come in to confirm the problem, activate another workstation to take over the position and allow the aircraft to be monitored and controlled by that controller with a proper working workstation.”
He added that it was hard for air traffic controllers to notice immediately if the workstation starts becoming sluggish while performing their duties, so it would be “disastrous” if situations that required immediate attention or intervention arose.
“CAD management knew… even a schedule restart would impact operation and chose a timing to perform it to minimize the impact,” he said.
New Delhi airport
Meanwhile, air traffic control workstations at New Delhi airport, which also uses Raytheon’s AT3 system, have encountered similar technical problems.
In a strongly-worded letter dated 30 January – a copy of which was obtained by FactWire – India’s Air Traffic Controllers’ Guild (ATC Guild) complained to the Airports Authority of India and the Director General of Civil Aviation about the sluggishness of AT3 workstations, especially in adverse weather or heavy traffic conditions.
“Sluggishness is observed during heavy traffic situations either in bad weather conditions or just after an airspace closure ends,” the letter states, adding that the glitch has caused distress to air traffic controllers.
The trade union further warned in the letter – titled “Are We Waiting for a Replay of the Charkhi-Dadri Crash of 1996?” – that such sluggishness is “bound to lead to major incident” if the authority continues to ignore repeated requests for system improvements like it did in 1996. That year, insufficient radar facilities were a cause in the deadliest mid-air collision in history, killing 349 people on board.
Calling for immediate hardware and software upgrades and restrictions on air traffic, the group said the management, despite being aware of the problem, had “deliberately shut their eyes and ears.”
The letter also lists a number of instances of “poor and dangerous performance” of the workstations, including two which occurred on 15 July and 4 October last year when sluggish and erratic performance of the system kept controllers “on their toes.”
The letter also mentions that the performance issues intensified on 19 January following a scheduled airspace closure for a Republic Day parade rehearsal. “For a significant time, the system was not taking any input and the HMI (human machine interface) was almost frozen,” it reads.
The staff restarted the whole air traffic control system at New Delhi airport, only to encounter the same problems during heavy traffic the next day and on India’s Republic Day a week later.
The group lambasted Raytheon’s AT3 system for failing to perform at a time when the controllers were already hampered by adverse weather conditions and airspace closure and urged the management to take immediate actions or “wait for the inevitable to happen.”
Only four other airports besides Hong Kong – those in Dubai and the Indian cities of New Delhi, Mumbai and Chennai – are currently using the AT3 air traffic management system made by US defense contractor Raytheon.
CAD has formed an international users group for the AT3 system, which held its first meeting on September 19 and 20. The department said in a press release on September 27 that “users were in general satisfied with the overall performance of the ATMS.”
When contacted for comment, CAD said on Tuesday that computer crashes – when a radar screen freezes or becomes non-responsive – have not occurred since it launched the system last November, but admitted: “there were five instances when some workstations became sluggish while all flight details remained on the screens.”
It added that support staff did not need to restart the faulty workstations immediately and only performed the process during light air traffic.
However, CAD’s response is inconsistent with FactWire’s findings.
CAD also said India’s Airports Authority did not mention any comments about the AT3 system expressed by the ATC Guild during their international users group meeting.
The Airports Authority of India did not respond a request for comment before this story went to press.
New Delhi, Mumbai and Chennai airports started using the AT3 system from around July to September 2011, according to CAD.
Since then, Indian media have reported several incidents where the air traffic system at the New Delhi airport malfunctioned, including a reported failure to retrieve radar data at around 6pm on October 4 2016, which paralysed the airport.
The Indian government began construction of a new air traffic control centre at New Delhi airport in 2012 and the new centre is expected to be fully operational by October next year.
The new control centre will replace the current AT3 system with an air traffic management system developed by Spanish IT firm Indra, according to a press release from the company in December 2014.
