Shared databases and integrated systems for image recognition

Topic description

Objective:

Image recognition technologies become essential for defence applications. There is in particular an increasing need for forces to analyse their environment more efficiently in order to enhance decision-making, responsiveness and survivability while ensuring the observation function effectively. This need is reinforced by the emergence of new forms of threats such as hypersonic, swarming, miniaturised or stealth weapons, which require increased speed, sensitivity or accuracy of the recognition systems. This need applies to manned and unmanned platforms as well as to wide-area or long-lasting surveillance.

Besides, databases are essential for training, testing and certifying artificial intelligence (AI) systems such as image recognition systems. However, collecting data that is both representative of military operational scenarios and sharable for AI system development is a complex task. Furthermore, data annotation (e.g., definition of regions in images, labelling…) and curation need significant efforts that are often underestimated. The lack of specialised entities missioned to serve the community by actively creating representative and sharable databases further hinders the creation of such databases. These issues are often a bottleneck in system development. Frameworks should be developed that enable or facilitate cooperation and sharing of image databases for defence.

In addition, new high-resolution sensor technologies provide larger amounts of information that are difficult to transmit in their entirety in real time. Automatic processing located near the sensor is needed to reduce the information flow. This requires joint optimisation of software and hardware and can involve trade-offs between recognition performances and integration constraints.

Scope and types of activities

Scope

Proposals should address the development and objective testing of image recognition systems for defence, the creation of the needed new databases, and the integration of the developed image recognition technologies near the sensors and objective testing of these integrated systems. Any relevant types of images and sensors (visible, infrared, multiband, hyperspectral…) and any well-defined types of recognition tasks (detection and classification for well-defined classes of objects, detection and identification of known objects, tracking…) can be considered.

Types of activities

The following types of activities are eligible for this topic:

The proposals must address in particular the following:

• Creation and sharing of annotated image databases, and development of appropriate frameworks for that purpose
• Development of software image recognition systems
• Integration of such software systems on customised hardware near sensors (integrated technology demonstrators)
• Objective evaluation of the performances of the software and integrated systems

Functional requirements

The proposed solutions should fulfil the following requirements.

• A limited number of well-defined use cases should be addressed (possibly one).
• For each use case addressed:
  • The use case should address well-identified military operational needs and scenarios.
  • The use case should be defined by clear evaluation data, metrics and protocols described in the proposal. Evaluation raw data should be real images directly acquired through sensors. However, if real images do not yet exist (for instance for future threats) but hybrid or synthetic images can be expected to be representative of the anticipated threats, such images may be used.
  • Several approaches should be explored by different research teams (while being evaluated in a comparable way using the above-mentioned data, metrics and protocols). The proposed techniques should be presented in the proposal.
  • The state-of-the-art should be described in the proposal, relying as much as possible on past objective and quantitative evaluation results. The expected progress beyond the state-of-the-art should also be described, taking into account the foreseen amount of new training data and/or the ability to make a better use of existing data through innovative learning techniques, and a possible roadmap toward technological maturity beyond the project should be provided.
  • The needed image data should be collected, relying on dedicated trials and measurement campaigns as needed. Images should be annotated using documented annotation guidelines. The resulting databases should be shared at least with the project partners who need them in the framework of the project.
  • The possibility to share and reuse these databases beyond the project should be anticipated, including where they would be classified. The organisational and technical framework for data production and sharing should be described in the proposal. In particular, the entity in charge of curating and distributing the databases should be clearly identified and the conditions for sharing should be described. If hybrid or synthetic images are needed for system evaluation, the possibility to share the tools used to generate these images should be anticipated in the same conditions.
  • Setting up a framework for data production and sharing that can be reused beyond the project is encouraged. Synergies with similar efforts at the European level should be sought.
  • Training and evaluation data should be representative of the use case and cover the various conditions encountered in real-life scenarios (e.g., various climate, weather or lighting conditions, various types of background landscapes…).
  • If representative data that can be collected by users during operations is deemed needed to reach the expected system performance, machine-learning techniques to learn continuously from user supervision (user-driven adaptation) should be considered.
  • Software recognition systems should be optimised to offer the best possible recognition performance (e.g., high probability of correct detection and low false alarm rates, high area under the ROC ^[1] curve…).
  • Integrated recognition systems should maintain the recognition performance of software systems as much as possible while taking into account size, weight, power and cost constraints.
  • Both software and integrated systems should be benchmarked using the agreed-upon evaluation data, metrics and protocols.

The expected impacts are

• Shareable databases for image recognition
• Established frameworks easing the production and sharing of databases, creation or reinforcement of entities producing sharable databases
• Availability of new integrated image processing products
• Enhanced decision-making and responsiveness, reduction of cognitive load of soldiers during operations
• Enhanced situational awareness
• Enhanced safety, resilience and survivability
• Reduction of fratricides and collateral damages
• Enhanced unmanned system autonomy

[1]Receiver Operating Characteristic