Automation of security penetration tests

Topic description

Objective:

Cyber defence applications are in most cases relying on cybersecurity technologies. There are many actions in the civil domain on the automation of penetration test. However, due to the particular conditions of defence-related use-cases, civil technologies need to be adapted, further improved or combined with defence-specific technologies through additional R&D efforts to make them suitable for defence applications. This research topic aims to overcome defence-specific obstacles associated to the automation of penetration tests, and at least partially automate the process by developing a user-friendly software solution that performs network security penetration tests for cyber defence actors.

General objective

Vulnerability scanners and various policy audit tools are available to system administrators today. However, the use of such tools are not sufficient to protect computer networks against advanced threat actors and internal threats. To complement them, many organisations employ penetration testers who actively try to think as a threat agent and compromise computer networks. Penetration testers can be used for many different purposes. For example, they can be assigned the task of verifying a system administrators hypotheses concerning a vulnerability in the computer network, to identify vulnerabilities missed by scanners and administrators, or to act as a red team that test the security operations centre of an organisation.

Specific objective

To emulate the thinking and actions of a real threat agent is difficult, and is even more complex as the capabilities of likely threat agents increase. Consequently, competent penetration testers are scarce and to regularly run penetration tests is associated with considerable costs. A number of conceptual attempts have been made to automate this process, e.g. by modelling the process as hidden Markov model and train models on theoretical/artificial data. However, security audits and penetration tests involve many activities that are non-trivial to automate.

The proposals should address research that is based on use cases where the system owner administrates the penetration tests. Thus, non-cooperative computer networks are outside the scope of this topic. For instance, the use cases can include a) whitelisting of payloads in antivirus software, or b) release of initial information about the targeted network to the penetration testing system. The expected scope is to create a user-friendly software solution that performs network security penetration tests. Interference on deployed systems should be on a minimal/acceptable level, while simultaneously leaving a realistic imprint in the systems, as security logs, etc. Furthermore, the final outcome of the proposals should be suitable for the context of military security operation centres (SOC) and evidence of this should be provided in the proposals.

Types of activities

The following table lists the types of activities which are eligible for this topic, and whether they are mandatory or optional (see Article 10(3) EDF Regulation):

The proposals must cover at least the following tasks as part of the mandatory activities:

• Generating and integrating knowledge and studies:
  • automation of tasks performed by penetration testers. For example, network/vulnerability tests, security misconfiguration tests, identifications and authentication failure tests, broken access control tests, injection tests, web/contact scraping and credential harvesting, email validation, integration with online reconnaissance tools like Shodan, creation of password guessing lists based on per domain/organisation information for feeding password cracking tools;
  • bring about an artificial intelligence capable of making relevant decisions. For example, effective ways to perform a network scan without being blocked, choosing the most effective exploitation method on a vulnerability(ies), evaluate the outcome of the exploitation and in cases of exploitation failure decide whether the exploitation method was wrong or a secondary security control prevented the execution of the payload, etc., given the costs and benefits involved;
• • defining user interfaces for operators of the automated penetration testing solution, e.g. a GUI showing the progress of the test, showing future plans of the artificial intelligence, and making it possible to control these plans through constraints;
  • evaluation of the solutions capability and suitability for operational use, e.g. by comparing its behaviour and capability to penetration testers of different competence.
• Design:
  • producing the blueprints for a product capable of automating penetration tests based on the technical model, along with suitable use cases.

In addition, the proposals must include methods for the evaluation of the outcome of the automated penetration testing based on well-established standards such as the Common Vulnerability Scoring System (CVSS).

The proposals must also give due consideration to design principles and implement a specific ethics-focused approach during the development, deployment and/or use of AI-based solutions, e.g. by using the Assessment List for Trustworthy Artificial Intelligence (ALTAI) to develop procedures to detect and assess the level and address potential risks.

In order to avoid unnecessary duplications and to best complement R&D efforts already targeting civil applications, the research conducted must build on R&D results of projects funded by EU programmes targeting civil applications for efficient spinning-in of knowledge and innovative solutions to the defence sector.

Functional requirements

The proposals must benefit a future solution for the armed forces of the Member States and EDF associated countries (Norway).

The outcome should enable or be capable of:

• executing a number of tools and techniques typically used during penetration tests, e.g. tools available in platforms such as Kali Linux;
• assessing alternatives, predict the effect of actions, and/or plan for future actions, e.g. by evaluating which actions are the most valuable in the long run;
• performing actions so that they leave a footprint (e.g. host logs) that is representative of the actions taken;
• having a user interface which allows a human operator to specify its behaviour, e.g. by selecting profiles representing tests with different focus and aggressiveness;
• allowing a human operator to specify acceptable and non-acceptable actions, e.g. in terms of white-listed hosts and black-listed hosts;
• having a user interface which communicates plans, assessments, and previous actions to a human operator.

The outcome should contribute to:

• a stronger, more competitive and technologically independent European Defence Technological and Industrial Base (EDTIB) when it comes to solutions for security penetration test automatisation and capability to test the security posture of operational computer networks and emulate threat agents during training, exercises, and system tests;
• enhanced security for the EU, its Member States and EDF associated countries (Norway) and more capable and interoperable forces performing cyber defence operations;
• the spin-in of civil European R&D into the defence sector.