2.6.1 Definition
A Red Teaming (RT) exercise is an adversarial goal-centric security test, where red teamers focus on simulating a full-scope cyberattack without the blue team’s knowledge, to validate the effectiveness of an organisation’s security controls and security design principles, as well as demonstrating the potential impact of a real-world cyberattack.
2.6.2 Qualifications
The Supplier’s red teamers should minimally possess either of the following industry-recognised certifications.
- Offensive Security Certified Profession (OSCP)
- CREST Registered Tester (CRT)
In addition, they should minimally possess one of the certifications relevant to the various domains of the RT exercise, based on Clause 2.3.2. An RT exercise involving a particular domain should involve personnel who possess a matching certification for that domain, based on Clause 2.3.2. For example, if the scope of the RT exercise involves mobile applications, personnel with relevant mobile application PT certification should be included in the team.
In addition, red teamers should also possess the following certifications based on their roles in the RT exercise.
| Domain | Certification(s) |
|---|---|
| Adversarial Simulation | 1. CREST Certified Simulated Attack Specialist (CCSAS) 2. CREST Certified Simulated Attack Manager (CCSAM) |
| Exploit Development | 1. Offensive Security Certified Profession (OSED) 2. eLearnSecurity Certified eXploit Developer (eCXD) 3. GIAC Exploit Researcher and Advanced Penetration Tester (GXPN) 4. Offensive Security Exploitation Expert (OSEE) 5. Offensive Security MacOS Researcher (OSMR) |
| Social Engineering and Reconnaissance | 1. Social Engineering Pentesting Professional (SEPP) 2. GIAC Open Source Intelligence (GOSI) |
In the absence of the above certifications in the table above, the Supplier should furnish other evidence of the penetration tester’s proficiency in these domains, which may include a portfolio of work, relevant training courses attended, relevant CVEs, publications, contribution to industry guidelines, etc.
In addition, red teamers should possess the most recent “Mastering CVSS” certificate to demonstrate proficiency in assessing the severity of findings using the Common Vulnerability Scoring System (CVSS).
2.6.3 Lifecycle
2.6.3.1 Pre-engagement matters
2.6.3.1.1 Scoping
The Customer and Supplier should agree on a clearly defined scope that contains the following:
Objective of the RT exercise: This should usually include the Customer’s worst cyber incident scenario involving the Customer’s most critical/important assets (“crown jewels”), for example, any CII/SII systems, or systems with high security/data classification. The RT exercise typically covers the full end-to-end scope where attacks from both outside the perimeter (simulating an external threat actor) as well as within the perimeter (simulating a malicious insider) are conducted. However, in cases where specific outcomes are desired, attacks can be focused on specific threat/attack scenarios. 2 examples are provided below. Example 1 below shows a full-scope AS, while example 2 shows an “assume-breached” RT exercise that only simulates a malicious insider.
Example 1: Simulate a real-world adversary targeting Customer assets, devices and personnel to gain access, escalate privilege and exfiltrate information from and via system databases.
Example 2: Simulate an attacker who has breached the perimeter defences, where the objective is to assess the effectiveness of Customer’s:
Threat modelling: Based on the system assets and objective, to perform:
Threat intelligence gathering to identify probable real-world adversaries who will target the identified assets; and
Identify reasonable and realistic threat scenarios and associated Tactics, Techniques and Procedures (TTPs) that achieve the objectives. To facilitate comprehensive and accurate threat modelling, the Customer should share with the Suppliers all necessary information for the purpose of accurate threat modelling. This may include design documentations, network diagrams, past Table-Top Exercise (TTX)/Threat and Risk Assessment (TRA) reports, etc. 3 examples of threat scenarios are provided below:
Example 1: Risk of state-sponsored attackers successfully infiltrating Software as a Service (SaaS) solution and performing pivoting attack through a specific component to an on-premise server.
Example 2: Risk of solution provider working with a foreign government to abuse or embed technical backdoors in a SaaS solution.
Example 3: Risk of malicious external devices plugged into Customer devices to extract information.
In-scope assets: To set out the objectives of the RT based on the threat scenarios. The scope of the RT should include peripheral systems that connect/interact with the “crown jewels”, including assets and personnel that are not owned by the Customer (e.g. vendor-owned systems that interact with Customer-owned systems, and vendor personnel with direct access to Customer-owned systems).
Examples: list of URLs, IP addresses, applications, etc.
Testing environment: Non-production or production; on-premise, GCC 1.0/2.0 (including CSP), GDC, or GPC. Testing on Production is preferred due to the necessity for realism and covertness. If non-production environments are used, such environments should be an exact replica of the production environment for PT to be conducted there. If not, suppliers should assess if it is reasonable to perform PT in a non-production environment.
Methodology: Black box or grey box (with credentials/access/concessions for assume-breach scenarios).
Time and resources to be expended.
Rules of engagement including timeline, location, vulnerability proofs-of-concept, personal data handling, testing hours, cooperation with the Customer’s security team, and permission to test.
The Customer and Supplier agree to allocate adequate time and manpower for the adversarial simulation. If a system exceeds the size of the Large scope, the Customer should consider breaking it up into multiple smaller scopes. The following tables serve as a guideline, and the Supplier should work with the Customer to decide on a reasonable testing duration and manpower.
The size of the RT exercise may depend on the number of identified threat scenarios, as well as the complexity of simulating these threat scenarios.
| Threat Scenario | ||
|---|---|---|
| Complexity | Description | Multiplier |
| High | Deep-dive research/reverse engineering/preparation effort is required for the threat scenario. This includes threat scenarios with products/technologies that are obscure or mature. For example, to reverse engineer and find zero-day vulnerabilities in Microsoft products to achieve a threat scenario. If the threat scenario requires significant effort, it can also be classified as Complexity High. For example: a) Requires physical penetration to a high- security premise b) Requires cyber penetration to a limited number of/heavily secured entry points c) Involves targeted phishing of a large number of victims or high-value targets |
1 |
| Medium | Considerable research/reverse engineering/preparation effort is required for the threat scenario. This includes threat scenarios with products/technologies that are well-understood or less mature. For example, to find and gain a foothold into one of the agency’s Internet-facing web applications as initial access. | 0.75 |
| Low | Some non-significant research/reverse engineering/preparation effort is required for the threat scenario. This includes threat scenarios with products/technologies that have known vulnerabilities or publicly available exploits. For example, to test the effectiveness of device management controls when inserting malicious USB devices. | 0.5 |
Score = Sum of (threat scenarios * complexity per scenario)
| Score | Scope | Estimated man-months |
|---|---|---|
| <3.0 | Small | 1-4.5 |
| 3.0-5.9 | Medium | 4.5-9 |
| 6.0-9.0 | Large | 9-14 |
2.6.3.1.2 Information
The Customer should provide the Supplier with all relevant information required for the RT exercise. This includes the following:
- System/product name and description.
- Security Classification and Data Sensitivity.
- Source code of target application (for white-box testing).
- Past AS (purple and red), PT and VA reports (at least 1 each), including the name of the suppliers who conducted such PT and VA.
- Past Table Top Exercise (TTX) and Threat and Risk Assessment (TRA) reports
- Risk register, including risk acceptance with supporting evidence.
- Documentation of the various roles, privileges, and access control model.
- Network architecture diagram elaborating the following details:
-Identified network segments in scope for the test
- Servers
- Security controls/appliances
- Network/data flow
- Design documentations
- Development and infrastructure teams (whether in-house or from external contractors).
- Logging mechanism: Agency Security Operations team, and/or GCSOC.
2.6.3.1.3 Pre-engagement logistics
The Customer should support suppliers with the appropriate logistical requirements for the RT. This may include the following:
- Forming a “white team” that consists of key stakeholders within the Customer that are aware of the RT exercise. This may include personnel such as system owners, CISO/ACISO/DCISO, Exercise Coordinators, and independent advisors.
- Providing information on any scheduled downtime or changes to the testing environment, which should be taken into account during scoping.
- Providing appropriate clearance and onboarding of red teamers for their insertion as malicious insiders (including persona, staff pass, account/equipment provisioning, etc)
- Provisioning of the list of personnel with direct access to identified “crown jewels”, including vendors.
- For concessions/grey box testing:
- Provisioning of minimum 2 user accounts per role (including admin role).
- Providing of credentials (including MFA) to assume various roles and tenants (if applicable). The Customer is responsible for ensuring that credentials are working before the start of PT fieldwork, and to provide evidence that they are working.
- For remote penetration tests, provisioning of access for the Supplier to target nodes (e.g. via IP whitelisting).
- For on-site RT, the following should be performed where applicable:
- Clearance of the Supplier’s testing machines.
- Provisioning of testing machines of a mutually agreed upon technical specifications.
- Provisioning of a reasonable working area with desk and chair, functional power outlet and access to bathroom facilities.
- The Customer should perform a system backup prior to the scan.
2.6.3.2 Fieldwork
2.6.3.2.1 Methodology
RT exercises should adhere to relevant industry standards, including “Red Team: Adversarial Attack Simulation Exercises” (AASE) guidelines from The Association of Banks in Singapore (ABS) and the MITRE ATT&CK framework.
2.5.3.2.2 Conduct
The Supplier should perform the RT exercise in stages to simulate a real-world adversary attempting to reach their objectives. The following stages are common in a full-scope RT exercise:
Planning and preparation
- Determine scope and objective
- Establish rules of engagement (boundaries, a chain of escalation, etc)
- Identify threat scenarios and kill chains
- Preparation of attacker’s tools, payloads and infrastructure
- Research in technology
Information Gathering
- Attack surface exploration
- Network discovery
- Social engineering
Attack Evaluation
- Reconnaissance – active/passive information gathering to be used to support the exercise later, pertaining to the organisation and its personnel. These include websites, technologies, names, email addresses, phone numbers, usernames, etc.
- Initial access – the attacker gaining an initial foothold within the organisation’s network, usually allowing continued access (persistence) for further exploration.
- Privilege escalation – the process of gaining a different (usually higher) permission/privilege on the system/network to open up new access paths to achieve their objectives.
- Lateral movement – the attacker moving through the environment by discovering new devices, networks, and systems to reach their objectives
- Impact – the attacker taking action on their objectives
After-action analysis
- Root cause analysis
- Propose mitigation
- Advise the Customer’s security consultant
2.6.3.2.3 Industry tools and configurations
RT is performed with a mix of automated and manual testing, and the commonly used tools depend on the domain of testing. Please refer to the list provided in section 2.3.3.2.3.
In addition, the Supplier should use the following tools for RT-specific tasks:
| Tool category | Description | Examples |
|---|---|---|
| Cyberattack framework | An attack framework that simulates a real cyber threat, typically comprising of command-and-control (C2) server and a remote access implant deployed on the victim machine | Cobalt Strike, Empire, Covenant, Metasploit, custom built solutions |
| Open Source Intelligence (OSINT) and Threat Intelligent tools | Tools that aid attackers in discovering organization information, assets and personnel from the Open, Deep and Dark web | Maltego, Shodan, Recorded Future |
| Phishing framework | Frameworks that aid attackers in conducting phishing attacks | Gophish, evilginx |
2.6.3.2.4 Fieldwork logistics
The Supplier should provide regular (e.g. weekly) status updates via email or meetings for the duration of the fieldwork.
For discovery of High/Critical severity findings, the Supplier should inform the Customer within 24 hours of the finding.
The Customer should not make any changes to the systems/environment that may impact the fieldwork in terms of behaviour or functionality, unless necessary. This includes, but is not limited to, the following:
- Remediation of identified vulnerabilities
- Feature updates
- Changes in application logic
- Software version updatese and patches
If incremental features, additions or changes are made as part of agile development practices, the Customer should identify and log such changes for future testing.
2.6.3.3 Reporting
The Supplier should disclose any alterations made to the testing environment as part of the test for the Customer to remove such alterations.
The Supplier should provide a report to the Customer detailing the results of the RT exercise. The report should minimally contain the following components:
- Objective of the RT exercise
- Identified threat scenarios from threat modelling
- Sequential finding ID per scope
- Risk statement containing the vulnerability, business impact on the system component, and technical root cause of the vulnerability
- Brief description of the vulnerability
- Risk Severity of the vulnerability
- Reproduction steps with evidence (e.g. screenshots, HTTP request/response) and proof of concept (e.g. HTTP request, script)
- Corresponding location in the source code of the identified issue (for white-box PT)
- Recommendations for remediation and/or mitigation
Where possible, the Supplier should assign base metrics for reported vulnerabilities using the latest version of the Common Vulnerability Scoring System (CVSS) version 3 or newer or ICT RMM The Suppliers may also collaborate with the Customer to assign temporal and environment metrics to derive a more contextualised CVSS severity. If an alternative severity framework is used, the Supplier should describe it in adequate detail in the report. In cases where potential vulnerabilities are identified but direct evidence of exploitation is not available (e.g. modification of a file on the backend server that the red teamer has no access to), the Customer should cooperate with the Supplier to validate the vulnerability.
In cases where the Customer does not agree with the scoring metrics and results proposed by the Supplier (including CVSS severities), the Customer should discuss and clarify with the Supplier and provide relevant input that may moderate such metrics and results. The Supplier should consider this input when validating the findings, including their severities and risk levels.
The Customer should perform a risk assessment of the identified vulnerabilities (e.g. using ICT RMM) and seek risk acceptance for findings that cannot be remediated within the established timeframe.
If required, the Supplier should validate the remediation/mitigation and provide evidence of remediation/mitigation.
The Supplier should declare any extenuating circumstances that were encountered during the fieldwork which might negatively affect the results (e.g. Application was only partially functional during the fieldwork).
If there is insufficient time to complete the fieldwork, the Supplier should at their discretion, request for an extension of the testing duration to complete the scan.
2.6.3.3.1 Prioritisation for remediation (CVSS)
The Customer should prioritise remediation based on the assessed risk to the Customer’s operations, which may be based on CVSS severities of the identified vulnerabilities, residual risks as performed via ICT RMM risk assessment or such other metric used by the Supplier. The Customer should perform remediation according to the established timeframe. If the Customer is unable to complete remediation within the timeline, they should perform a risk assessment (e.g. via ICT RMM) and seek risk acceptance of residual risks at the appropriate forum.
2.6.3.4 Post-engagement matters
If requested by the Customer, the Supplier should secure erase and destroy any data (including confidential information) obtained in connection with the RT exercise.
The Customer should document and track individual findings internally (e.g. as a GitHub issue or GitLab vulnerability report.)
The Customer should document and track remediation efforts internally (e.g. screenshots, code changes, configuration changes, etc).
