ATM application-oriented research for Artificial Intelligence (AI) for aviation

Topic description

ExpectedOutcome:

Project results are expected to contribute to the following expected outcomes.

• Environment: the proposed solutions are expected to demonstrate the positive impact of AI-based solutions on operational mitigation of aviation’s environmental impact.
• Capacity: the proposed solutions are expected to contribute to capacity by addressing AI-based human operator support tools to, for example, ensure the integration of new entrant aircraft types.
• Operational efficiency: the proposed solutions are expected to improve the operational efficiency by enabling better traffic predictions and forecasts, thus contributing to punctuality.
• Safety: the proposed solutions are expected to maintain at least the same level of safety as the current ATM system.
• Security: the proposed solutions are expected to maintain at least the same level of security as the current ATM system.

The challenge is to design and develop concrete innovative AI applications (that are already TRL1, achieved within SESAR programme or outside) that aim at:

• Enabling better use of data, leading to more accurate predictions and more sophisticated tools (e.g. new conflict detection, traffic advisory and resolution tools), increased productivity and enhancing the use of airspace and airport;
• Enriching aviation datasets with new types of datasets unlocking air/ground AI-based applications, fostering data-sharing and building up an inclusive AI aviation/ATM partnership to better support decision-makers, pilots, air traffic controllers and other stakeholders;
• Supporting (i.e. AI assistants) all ATM actors from planning to operations and across all airspace users;
• Enabling the virtualisation of infrastructure and air traffic service provision in all types of airspace, ranging from very low to high altitude operations. In doing so, AI will enable the system to become more modular and agile, while building resilience to disruption, traffic growth and greater airspace user diversity;
• Developing new ATM/U-space services.

The SESAR 3 JU has identified the following innovative research elements that could be used to achieve the expected outcomes. The list is not intended to be prescriptive; proposals for work on areas other than those listed below are welcome, provided they have already successfully achieved TRL1 (within SESAR programme or outside) and include adequate background and justification to ensure clear traceability with the R&I needs set out in the SRIA for the AI for aviation flagship.

• Innovative methodologies for quantifying the impact on safety and resilient performance of higher automation in ATM. This research aims at developing new methodologies for quantifying the impact on safety and resilient performance of higher automation in the ATM system applying data-driven techniques. These methodologies shall provide additional tools for the ATM stakeholders for evaluating and quantifying the impact of higher automation (e.g. AI/ML-based SESAR solutions) and could be potentially used for in-depth verification of safety criteria associated to a given SESAR solution in early R&I stages, but also to evaluate the state of the current system. Research shall built on the work performed by project FARO on en-route environment and automation levels 2/3, but it shall extend to other operating environments and to higher automation levels (4/5) (R&I need: trustworthy AI powered ATM environment).
• Safety filter for AI solution. Study safety filter concept for data prediction responsible for deciding on the usability of the ML model predicted data. Safety filter works as a safeguard, without human intervention, and qualifies the predicted sensor-data, as valid or invalid, by applying captured expertise rules. Research may address the application of dynamic risk assessment (DRA) and business impact assessment (BIA) techniques to distribute and enrich the exchange of information with federated learning architectures (R&I need: cyber-resilience). Research shall take into account the output of project SINAPSE.
• Artificial situational awareness: This research aims at developing artificial situational awareness solutions based on high-integrity information that are able to describe the traffic situation on a sector (en-route / TMA) or airport and, integrated with machine-learning (ML), enable the assessment of probabilistic events (e.g., trajectory prediction or conflict detection). By combining reasoning engine with machine learning, the proposed solutions shall be able to assess complex interactions between objects, draw conclusions, explain the reasoning behind those conclusions, and predict future system states. The objective is to develop AI-based systems that can assist ATCOs with monitoring tasks and contribute to the team situation awareness (SA): a situationally aware system would share the ATCOs’ SA since it would have access to the same data as an ATCO. Research may:
  • Assess how ML module predictions could use the optimization of their anticipation spans as they differ from ATCOs;
  • Analyse how automation and adaptation to novel system changes ATCOs’ learning success;
  • Investigate whether certain information could pertain to shift supervisors, Network Managers, ATSEP and FMPs;
  • Explore the possibility of flexible machine/machine coordination between sectors or units;
  • Develop an HMI that will allow the ATCOs to review information, inform ATCOs with the right timing, be able to recognise when certain information is necessary in order to avoid ATCOs attention dissipating on less important information and additional memory load, recognise when an ATCO SA is degraded and provide the necessary alerts at an appropriate time, etc.;
  • Develop scenario simulators so that, through probabilistic models, future scenarios can be predicted;
  • Validate benefits in terms of safety (e.g., by introducing an additional safety net performing tedious monitoring tasks with high reliability), interoperability between different systems (e.g., by enhancing data handling) and capacity (e.g., by automating some of the monitoring tasks and enabling the introduction of other automation systems).

Research shall take into account the output of project AISA (R&I need: human–AI collaboration: digital assistants).

• Explainable Artificial Intelligence (XAI): since the decisions provided by AI/ML algorithms are often opaque, non-intuitive and not understandable by human operators, this represents a limitation to their applicability in ATM. The objective of this research is to improve transparency of automated systems in the ATM domain investigating methods based on Explainable Artificial Intelligence (XAI) in operational use cases e.g., predicting air traffic conflict resolution and delay propagation, validating the robustness and transparency of the system, etc. Research shall take into account the output of project ARTIMATION (R&I need: human–AI collaboration: digital assistants).
• AI-powered co-piloting. Research aims at investigating how AI can support pilots in complex and critical situations, when workload may be high and/or the time to react very limited. For these situations, research should focus, for example, on how to exploit high levels of automation to perform non-critical tasks for pilots and how the HMI should work during such operations, so the pilot can focus on essential tasks e.g., during taxi-out, descend, approach and landing. These applications may play a significant role in the transition to single pilot operations. In addition, AI-powered applications could support the pilot in situations where workload is low e.g., engaging pilot’s attention and alert the pilot in case something unexpected happens. Research may address the development of algorithms based on reinforcement learning to help the pilot make decisions. The research results should demonstrate how the technology could support pilots in carrying out their tasks (e.g., demonstrate an increase in human capabilities during the execution of complex scenarios or a reduction in human workload in the execution of standard tasks), and assess the impact on the role of the human (R&I need: human–AI collaboration: digital assistants).
• AI for complex operations. This research is about developing AI-based human operator support tools to ensure the safe integration of new entrant aircraft types into an increasingly busy, heterogeneous and complex traffic mix (i.e. unmanned aircraft systems, supersonic aircraft, hybrid and fully electric aircraft). Algorithms for decision-making based on neural networks and classical optimization techniques could be addressed. Research may also consider the use of more advanced techniques such as reinforcement learning. Research should also cover the wider implications for other organisations and the impact on the network (R&I need: human–AI collaboration: digital assistants).
• User interface providing conflict resolution advisory transparency. This research focuses on visual elements that allow better understanding why a particular conflict resolution solution is recommended. The visual elements increase the transparency of advisories by providing the operator an insight into the deeper structure of the work domain as well as the inner workings of the AI agent. Research may address:
  • The potential benefits of advisory transparency on advisory acceptance and system trust in relation to ecological approaches, AI interpretability models, and the connection between the two. Previous research on this field suggests that transparency alone may not be suitable as a measure for increasing operators’ acceptance of advisories and trust in a system when that system performs different from the individual;
  • Transparency mechanisms for supporting the ATCOs in understand how the system works (e.g., the data processing, filtering, constraints etc. in the model), how it derived a specific advisory (relationships between input data and output), and why the proposed advisory is considered best (e.g., best match to the individual, group, or optimized according to reinforcement learning (RL) model).

Research shall take into account the output of project MAHALO (R&I need: human–AI collaboration: digital assistants).

• Guidelines for the design of future AI systems. This research relates to the application of EASA guidelines to the development of AI enabled systems in ATM. Research shall also contribute to the update of EASA guidelines, including feedback on the effects of conformance, transparency and complexity and other challenges associated to the design of future AI systems (e.g., trade-offs between privacy and transparency). Research may consider human-in-the-loop simulations considering controller trust, acceptance, workload and human/machine performance but also new approaches for validation, verification, and testing of AI applications, specifically for safety critical applications e.g., developing an agile validation methodology data centric security capabilities for AI systems to promote reliability, increase trust, and maintain a competitive edge in today's rapidly evolving technological landscape. Close coordination with EASA is expected, to ensure complementarity and consistency with EASA activities on the following areas:
  • Trustworthiness: capability to keep input and output privacy with relatively high cyber-security protection. Support the definition of the requirements and needs for input/output verification (related to trustworthiness in the framework of Structured Transparency) in the ATM context in support of the EASA certification process descriptions. Validate and further develop requirements and potential solutions with a co-joint analysis together with EASA and other operational experts. Clarify some of the challenges faced by EASA, e.g., to define the system requirements, processes, and tools that are needed to perform the validation and certification process.
  • Learning Assurance: including the consideration of realistic operational cases in realistic operational conditions and new ML techniques. Need to develop specific assurance methodologies to deal with learning processes;
  • AI explainability, which goes beyond the ML techniques to extract information from the models, and includes the interactions with other systems and with the human operators (human factors). Research may help to clarify which requirements and processes the target AI/ML system should comply with in order to be certifiable for operations;
  • AI Safety case: discussing with EASA and other safety experts about the needs and requirements of a concrete safety-case can help to clarify and support the development the EASA guidelines for certification.

The concept of safety critical levels need to be further developed for AI applications in ATM. Research covers the definition and analysis of safety-related use cases for different safety level assurances. These safety levels may imply either the adaptation of current SW verification methods or the development of new ones to guarantee the safe of operation of AI in ATM. Research shall take into account the output of projects MAHALO, AICHAIN (R&I need: human–AI collaboration: digital assistants).

• Support to the certification of novel ATM-related AI/ML-based airborne and ground systems that enable higher levels of automation. The objective of this research element is to address issues related to the certification of novel ATM-related airborne and ground systems that enable higher levels of automation. Research will address solutions, methods, etc. that could support and simplify the certification process of innovative systems based on machine learning or artificial intelligence techniques. It is expected that proposals define a holistic approach to address this challenge considering not only technical aspects of the certification but also legal and regulatory aspects including privacy. Research may explore and assess potential approaches that could be applied for the certification of automation and that allow to demonstrate the safety of automation during nominal and non-nominal conditions. Of particular interest is to show how safety can be ensured even if not all situations and variations of parameters can be anticipated during the design phase. Proposals may apply uncertainty quantification to address this issue. Research may also address the specific challenges of certification of automation that can adapt its behaviour to changes of the environment over time. Research activities shall take into account other initiatives developing safety of life systems that may have different approaches to certification and review their applicability to ATM (e.g., EGNOS) (R&I need: human–AI collaboration: digital assistants).
• Development of framework to achieve effective Human-AI Teaming. As AI is developed to provide more intelligent behaviours, it is argued that there will be an increased need for AI systems to function effectively as team members with humans. Just as human-only teams have many advantages over solo workers (e.g., to manage workload fluctuations, provision of a diverse set of skills and capabilities toward the completion of common goals), human-AI teams can have similar benefits over human-only teams. When considering an AI system as a part of a team, rather than simply a tool capable of limited actions, the need for a framework for improving the design of AI systems to enhance the overall success of human-AI teams becomes apparent. A failure to consider the needs of the many air traffic controllers, pilots, flight dispatchers, flow managers, etc. who are responsible for successful ATM operations will result in AI technologies that eventually fail to provide the necessary high levels of performance and may instead cause inefficiencies and safety problems. The design of AI systems for human-AI teams needs to incorporate several highly interrelated considerations. These include designing the AI system to support not only task work, but also teamwork. These interrelated considerations include considerations about human-AI team performance and processes, team trust, team biases, team situational awareness, team training needs, human-AI interaction methods, interface, transparency and explainability and Human-System Integration processes, measures, and testing (R&I need: human–AI collaboration: digital assistants).
• Extended reality (XR) in support of ATM operations. The term eXtended Reality (XR) is includes technologies that enhance or replace our view of the world: encompasses augmented reality (AR), virtual reality (VR), and mixed reality (MR). Virtual reality guides the observer out of his/her actual environment and into an artificial one. Research aims at using virtual reality to improve the efficiency of ATM operations. Augmented reality enhances certain objects through transparent lenses in the observers’ field of vision. MR sits somewhere between AR and VR, as it merges the real and virtual worlds. An evident area of interest is training be it training of maintenance personnel or ATCO/pilot training. ATCO virtual training goes beyond the execution of remote simulations or validations as the ATCO would be physically located anywhere since the simulator HMI is created with extended reality equipment such as glasses and/or haptic devices. The research would investigate the operational and technical feasibility of training individually or collectively with other ATCOs as if they were executing remote simulations or validations through connection with remote real simulators. Research includes how ATCO performance data could be collected in real time to monitor training progress. Artificial intelligence could support the monitoring and detect when the ATCO has acquired the corresponding competence, or the ATCO needs to emphasize training in particular aspects. Pseudo piloting as well as the representation of adjacent sectors could be also based on speech recognition making the pilots and adjacent controllers also virtual. However, the scope under research is not only limited to training, but could also address specific operational challenges in different environments e.g., airport, TMA, en-route supported by XR techniques:
  • Collaborative environment with multiple users that have the ability to interact with the rest;
  • Applications to access to all relevant information in real time for efficiently and safely managing operations;
  • Digital assistants to support decision-making, etc.);
  • Location of tactical elements of operation in the field of vision obtained through advanced communication protocols: pilots, ATCO, maintenance, etc.;
  • Display of indications, messages, status of own elements, alerts or information of interest;
  • Intelligent adaptation of the displayed content according to the operating environment;
  • Gestural or voice interaction (synthesizer and voice command);
  • Situational awareness of the state of operations through AI.

The proposal shall show a thorough knowledge of past SESAR activities on this field i.e. ER RETINA and solution PJ.05-W2-97.1 (R&I need: human–AI collaboration: digital assistants).

• AI based human-machine collaboration to anticipate and respond to human needs by understanding ATCO’s intent and goals. Research aims at developing potential AI based solutions able to understand the traffic situation and, in combination with ATCO attention tracking technologies, to infer ATCO’s intent. The proposed solutions will support the ATCO not only by performing tasks he/she already intends to perform, but also by autonomously performing a task that is outside of the current scope of ATCO’s attention. ATCOs could then set the desired level of supporting actions performed by the AI team member (adaptive automation) and still maintain situational awareness by performing their usual tasks. Research may evaluate the potential use of graph-based models and joint neural networks to help the ATCO better understanding the traffic situation. Research shall address the impact on the role of the human (R&I need: human–AI collaboration: digital assistants).
• Integrated platforms for the nowcasting and forecasting of multiple meteorological hazards. This research aims at developing of integrated platforms to incorporate predictions of atmospheric hazards (e.g., SO₂ contaminants, severe weather situations such as deep convection and extreme weather and climate hotspots potentially contributing to global warming, etc.). The focus is to enhance the situational awareness of all stakeholders in case of multiple hazard crisis by facilitating the transfer of required relevant information to end-users, presenting such information in a user-friendly manner to ATM stakeholders, ultimately anticipating severe hazards and fostering better decision-making. Research may address:
  • Extension of nowcasting models of SO₂ in 1D (values for a given location) to 2D (lat-long) and 3D and nowcasting products for dust, ash, volcanic aerosol and precursors and smoke;
  • The consideration of additional observations (e.g., radar, satellite, sensors on board the aircraft) to better characterize the weather extremes and enhance the quality of the extreme weather nowcasting;
  • The integration of space weather and climate change in the new MET services;
  • The application of artificial intelligence or deep learning models based on recurrent networks could be used to better predict weather phenomena;
  • Address potential air traffic controller decision support systems able to import and process the meteorological forecasts and to adapt tactical arrival and departure scheduling to changing extreme weather conditions;
  • Target airport, TMA and en-route operating environments and the potential use by different stakeholders (e.g., Network Manager, ANSPs (flow management positions and air traffic controllers), airports, airlines (dispatchers and pilots), etc.);
  • Address the assessment of potential benefits in terms of capacity, efficiency, safety, predictability and resilience.

Research shall take into account the output of project ALARM (R&I need: AI Improved datasets for better airborne operations).

• Standardised testbed platform for developing, testing, and benchmarking AI-applications in ATM. Research in artificial intelligence in ATM has been traditionally fragmented in the area due to the lack of standardised testbeds. The development of a testbed to be adopted as a common framework for future research in applied AI in ATM will enable reproducibility and considering open science practices. Research covers the definition and publication of a library of use cases, including input and output data associated to persistent identifiers (such as digital object identifier (DOI)). This will ensure findability and a wider adoption within the ATM research community (R&I need: AI Improved datasets for better airborne operations).