2.7.1 Definition
A Purple Teaming (PpT) exercise is an adversarial goal-centric security test, where red teamers collaborate with the blue team to validate security controls and security design principles to improve overall detection and response capabilities through the emulation of real-world threat scenarios.
2.7.2 Qualifications
The Supplier’s red teamers should minimally possess either of the following industry-recognised certifications.
- Offensive Security Certified Profession (OSCP)
- CREST Registered Tester (CRT)
In addition, they should minimally possess one of the certifications relevant to the identified threat models of the PpT exercise, based on Clause 2.3.2. A PpT exercise involving a particular domain should involve personnel who possess a matching certification for that domain, based on Clause 2.3.2. For example, if the PpT exercise involves a cloud environment, personnel with relevant cloud certification should be included in the team.
In addition, red teamers should also possess at least one of the following certifications for them to fully demonstrate a fundamental understanding of threat scenario creation and PpT processes and workflow.
| Domain | Certification(s) |
|---|---|
| Adversarial Simulation | 1. MITRE ATT&CK Purple Teaming Methodology Certification |
| Exploit Development | 1. Offensive Security Certified Profession (OSED) 2. eLearnSecurity Certified eXploit Developer (eCXD) 3. GIAC Exploit Researcher and Advanced Penetration Tester (GXPN) 4. Offensive Security Exploitation Expert (OSEE) 5. Offensive Security MacOS Researcher (OSMR) |
In the absence of the above certifications in the table above, the Supplier should furnish other evidence of the personnel’s proficiency in these domains, which may include a portfolio of work, relevant training courses attended, relevant CVEs, publications, contribution to industry guidelines, etc.
In addition, red teamers should possess the most recent “Mastering CVSS” certificate to demonstrate proficiency in assessing the severity of findings using the Common Vulnerability Scoring System (CVSS).
2.7.3 Lifecycle
2.7.3.1 Pre-engagement matters
2.7.3.1.1 Scoping
The Customer and the Customer’s blue team should agree on a clearly defined scope with the supplier that contains the following:
Objective of the PpT exercise: The Customer, the Customer’s blue team and the Supplier should create an objective based on the overall architecture of the Customer’s system(s) through an emulation of TTPs from known real-world adversaries, e.g. Advanced Persistent Threat (APT) actors. This includes the identification of the Customer’s most critical/important assets (“crown jewels”), for example, any CII/SII systems, or systems with high security/data classification.
Threat modelling: Based on the system assets and objective, to perform:
Threat intelligence gathering to determine probable real-world adversaries who will target the identified assets; and
Identify reasonable and realistic threat scenarios and associated TTPs that achieve the objectives. The Customer should share with the Supplier all necessary information for accurate threat modelling. This may include design documentations, network diagrams, past Table-Top Exercise (TTX)/Threat and Risk Assessment (TRA) reports, etc. 3 examples are provided below:
Example 1: Risk of state-sponsored attackers successfully infiltrating Software as a Service (SaaS) solution and performing pivoting attack through a specific component to an on-premise server.
Example 2: Risk of solution provider working with a foreign government to abuse or embed technical backdoors in a SaaS solution.
Example 3: Risk of malicious external devices plugged in to Customer devices to extract information.
Crafting of Emulation Plan: Utilising the information from the threat modelling stage, the Supplier should generate a targeted emulation plan with customised procedures. For the Supplier to generate the most optimal procedures, the Customer’s blue team should provide the Supplier with their input on the current detection and response capabilities. This will allow the Supplier to consider these capabilities and adjust the complexity of the procedures as necessary. The threat emulation plan should attempt to cover as many stages within the MITRE ATT&CK attack chain as possible and the scenarios should vary in complexity and be based on actual TTPs from established APT groups relevant to the Customer as scoped in threat modelling.
Testing environment: Non-production or production; on-premise, GCC 1.0/2.0 (including CSP), GDC, or GPC. Testing on Production is preferred due to the necessity for realism. If custom non-production environments are provided, they should be set up with the current defence mechanisms which are present in the production environment, as dictated by the blue team (Logs piped to SIEM, EDR/XDR/EPP set up on the environment, custom defences, etc).
Methodology: Automated or Manual
Time and resources to be expended.
The Customer and Supplier agree to allocate adequate time and manpower for the adversarial simulation. The following tables serve as a guideline to size a Purple Teaming exercise scope, and the Supplier should work with the Customer to decide on a reasonable testing duration and manpower.
| Type of Emulation Execution | Approach | Sizing | Scope | Estimated man-months |
|---|---|---|---|---|
| Automated Execution | • Breach and Attack Simulation (BAS) agent-based emulation on Customer provided environments (Able to automate payload execution) • Could be used to launch both customised/vendor-provided payloads |
Sizing depends on the number of threat scenarios involved within the BAS emulation | Small | 4-man months |
| Medium | 5-man months | |||
| Large | 6-man months | |||
| Manual Execution | • Engineer executed emulation on Customer provided environments. • Will be more targeted on available tech stack for systems, red team engineers will conduct exploits development to emulate N-days or potential 0-days |
Sizing depends on the number and complexity of threat scenarios involved within the emulation, refer to table and algorithm below for complexity calculation | Refer to the tables below to derive scope | Refer to the tables below to derive estimated man-months |
| Threat Scenario | ||
|---|---|---|
| Complexity | Description | Multiplier |
| High | Deep-dive research/reverse engineering/preparation effort is required for the threat scenario. This includes TTPs that aren’t readily available, which requires time for an engineer to deep-dive and emulate. | 1 |
| Medium | Considerable research/reverse engineering/preparation effort is required for the threat scenario. This includes creating custom variants for known TTPs, which requires an engineer to spend considerable effort to rebuild and recreate the scenarios. | 0.75 |
| Low | Some non-significant research/reverse engineering/preparation effort is required for the threat scenario. This includes threat scenarios with products/technologies that have known vulnerabilities or publicly available exploits. | 0.5 |
Score = Sum of (threat scenarios * complexity per scenario)
| Score | Scope | Estimated man-months |
|---|---|---|
| <5.0 | Small | 2-3 |
| 5.0-9.0 | Medium | 4-6 |
| 9.1-12.0 | Large | 6-8 |
2.7.3.1.2 Information
The Customer should provide suppliers with all relevant information required for the PpT exercise. This includes the following:
- System/product name and description.
- Security Classification and Data Sensitivity.
- Source code of target application (for white-box testing).
- Past AS (purple and red), PT and VA reports (at least 1 each), including the name of the suppliers who conducted such PT and VA.
- Past Table-Top Exercise (TTX) and Threat and Risk Assessment (TRA) reports
- Risk register, including risk acceptance with supporting evidence.
- Documentation of the various roles, privileges, and access control model.
- Network architecture diagram elaborating the following details:
- Identified network segments in scope for the test
- Servers
- Security controls/appliances
- Network/data flow
- Design documentations
- Development and infrastructure teams (whether in-house or from external contractors).
- Logging mechanism: Agency Security Operations team, and/or GCSOC.
2.7.3.1.3 Pre-engagement logistics
The Customer should support the Supplier with the appropriate logistical requirements for the PpT exercise. This may include the following:
- Assigning a Point-of-Contact (POC) from the Customer to liaise with the supplier on all logistical issues.
- Assign a Point-of-Contact (POC) from the Customer’s “blue team”/Security Operations team to liaise with the supplier on all technical matters.
- Providing information on any scheduled downtime or changes to the testing environment, which should be taken into account during scoping.
- Providing credentials (including MFA) to assume various roles and tenants (if applicable). The Customer is responsible for ensuring that credentials are working before the start of PpT fieldwork, and to provide evidence that they are working.
- For remote PpT, provisioning of access for the Supplier to target nodes (e.g. via IP whitelisting), and/or support to install any software (e.g. tools, agents) on target nodes and ensure any necessary network routing is performed for the installed software to work.
- For on-site penetration tests, the following should be performed where applicable:
- Clearance of suppliers’ testing machines
- Provisioning of testing machines of mutually agreed upon technical specifications.
- Provisioning of a reasonable working area with desk and chair, functional power outlet and access to bathroom facilities
- The Customer should perform a system backup prior to the scan.
2.7.3.2 Fieldwork
2.7.3.2.1 Methodology
PpT exercises should adhere to relevant industry standards, including Scythe Purple Team Exercise Framework (PTEF) or Prepare, Execute, Identify and Remediate (PEIR) framework.
2.5.3.2.2 Conduct Supplier should perform the PpT exercise depending on which methodology is being used (see Clause 2.7.3.2.1). The following example is relevant for a PpT exercise conducted using the PEIR framework:
Prepare
- Project scoping
- Threat gathering
- Identify threat scenario and kill chains
- Development of TTPs
Execute
- Execution of attack emulation
- Conduct monitoring and analysis
Identity
- Identification of gaps in system and detection
Remediate
- Close identified gaps
- Rerun attack emulation
- Reporting
The red and blue teams should collaborate during the attack execution phase to identify and agree on the following:
- List of threat scenarios and TTPs
- List of security controls
- Expected detection and prevention results from the execution of each test case
- Actual detection and prevention results from the execution of each test case
2.7.3.2.3 Industry tools and configurations
Purple Teaming is performed with a mix of automated and manual execution. Automated testing should be performed using a reputable, industry-recognised (e.g. Leaders in Gartner Breach and Attack Simulation Tools or equivalent) Breach and Attack Simulation tool. The tool should have an existing threat library that supports the execution of a PpT exercise. A non-exhaustive list is provided below as a reference:
- AttackIQ
- Picus Breach and Attack Simulation Platform
- Cymulate Breach and Attack Simulation Platform
- MITRE CALDERA
2.7.3.2.4 Fieldwork logistics
The Supplier should provide regular (e.g. weekly) status updates via email or meetings for the duration of the fieldwork.
The Customer should not make any changes to the systems/environment that may impact the fieldwork in terms of behaviour or functionality, unless necessary. This includes, but is not limited to, the following:
- Remediation of identified vulnerabilities
- Feature updates
- Changes in application logic
- Software version updatese and patches
If incremental features, additions or changes are made as part of agile development practices, the Customer should identify and log such changes for future testing.
2.7.3.3 Reporting
The Supplier should disclose any alterations made to the testing environment as part of the test for the Customer to remove such alterations.
The Suppliers shall provide a formal report to the Customer detailing the results of the PpT exercise. The report should minimally contain the following components:
- Objective of the PpT exercise
- Identified threat scenarios from threat modelling
- Expected and actual detection and prevention results from the execution of each technique
- Detailed threat map of each technique’s execution status and analysis
- Recommendations for remediation and/or mitigation
The Customer should perform a risk assessment of the identified vulnerabilities (e.g. using ICT RMM) and seek risk acceptance for findings that cannot be remediated within the established timeframe.
If required, the Supplier should validate the remediation/mitigation and provide evidence of remediation/mitigation.
The Supplier should declare any extenuating circumstances that were encountered during the fieldwork which might negatively affect the results (e.g. Application was only partially functional during the fieldwork).
If there is insufficient time to complete the fieldwork, the Supplier should at their discretion, request for an extension of the testing duration to complete the scan.
2.7.3.3.1 Prioritisation for remediation (CVSS)
The Customer should prioritise remediation based on the assessed risk to the Customer’s operations, which may be based on residual risk as performed via ICT RMM risk assessment. Remediation should be performed according to the established timeframe. If the Customer is unable to complete remediation within the timeline, they should perform a risk assessment (e.g. via ICT RMM) and seek risk acceptance of residual risks at the appropriate forum.
2.7.3.4 Post-engagement matters
If requested by the Customer, the Supplier should secure erase and destroy any data (including confidential information) obtained in connection with the PpT exercise. The Customer should document and track individual findings internally (e.g. as a GitHub issue or GitLab vulnerability report.
The Customer should document and track remediation efforts internally (e.g. screenshots, code changes, configuration changes, etc).
