2.3.1 Definition
A Penetration Test is an asset-centric security test, where penetration testers focus on validating the security of the in-scope system components through automated and manual tests.
2.3.2 Qualifications
The Supplier’s penetration testers should minimally possess either of the following industry- recognised certifications:
- Offensive Security Certified Profession (OSCP)
- CREST Registered Tester (CRT)
In addition, penetration testers should possess certifications relevant to the domain of the penetration test. A non-exhaustive list is provided below as a reference, and the Supplier may seek validation from the Customer and GovTech CSG for certifications not in this list:
| Domain | Certification(s) |
|---|---|
| Web application PT | 1. Burp Suite Certified Practitioner 2. Offensive Security Web Expert (OSWE) 3. eLearnSecurity Web Application Penetration Tester (eWPT) 4. GIAC Web Application Penetration Tester (GWAPT) 5. CREST Certified Web Application Tester (CCT APP) |
| Network/Infrastructure PT | 1. Offensive Security Experienced Penetration Tester (OSEP) 2. GIAC Penetration Tester (GPEN) 3. PentesterAcademy Attacking and Defending Active Directory 4. eLearnSecurity Certified Profession Penetration Tester (eCPPT) 5. TCM Security Practical Network Penetration Tester 6. CREST Certified Infrastructure Tester (CCT INF) |
| Mobile application PT | 1. GIAC Mobile Device Security Analysis (GMOB) 2. eLearnSecurity Mobile Application Penetration Tester (eMAPT) |
| Cloud PT | 1. Certificate of Cloud Security Knowledge (CCSK) 2. Certified Cloud Security Professional (CCSP) 3. Certificate of Cloud Auditing Knowledge (CCAK) 4. CompTIA Cloud+ 5. GIAC Cloud Penetration Tester (GCPN) 6. Amazon AWS Certified Solutions Architect - Associate 7. Microsoft Azure Administrator Associate 8. Google Associate Cloud Engineer |
| Internet of Things (IoT) PT | 1. GIAC Assessing and Auditing Wireless Network (GAWN) 2. Offensive Security Wireless Professional (OSWP) |
| Thick client PT | 1. Offensive Security Certified Profession (OSED) 2. eLearnSecurity Certified eXploit Developer (eCXD) 3. GIAC Exploit Researcher and Advanced Penetration Tester (GXPN) |
In the absence of the above domain- specific certifications, the Supplier should furnish other evidence of the penetration tester’s proficiency in these domains, which may include a portfolio of work, relevant training courses attended, relevant CVEs, publications, contribution to industry guidelines, etc. The Customer may reject any penetration tester provided by the Supplier.
In addition, penetration testers should possess the most recent “Mastering CVSS” certificate to demonstrate proficiency in assessing the severity of findings using the Common Vulnerability Scoring System (CVSS).
2.3.3 Lifecycle
2.3.3.1 Pre-engagement matters
2.3.3.1.1 Scoping
The Customer and the Supplier should agree on the defined scope of the PT, which includes the following:
- The objective of the PT: Go-Live testing, testing of new features, testing of newly identified threats, routine PT, etc.
- Threat scenario: Based on the system assets and objective, identify reasonable and realistic threat scenarios.
- In-scope components (e.g. list of URLs, endpoints): to reasonably cover the objective of the PT based on the threat scenarios.
- Testing environment: Non-production or production; on-premise, GCC 1.0/2.0 (including CSP), GDC, or GPC. If non-production environments are used, such environments should be an exact replica of the production environment. If not, suppliers should assess if it is reasonable to perform PT in a non-production environment.
- Methodology: White box or grey box.
- Time and resources to be expended.
- Rules of engagement including timeline, location, vulnerability proofs-of-concept, personal data handling, testing hours, cooperation with the Customer’s security team, and permission to test.
The Customer and Supplier should allocate sufficient time and manpower for the penetration test based on the following suggested guide. If a system exceeds the size of the Large scope, the Customer should consider breaking it up into multiple smaller scopes. The following tables serve as a guideline, and the Supplier should work with the Customer to decide on a reasonable testing duration and manpower.
| Web app PT | |||
|---|---|---|---|
| Scope | Number of User Roles | Numbere of dynamic pages/API endpoints | Estimated man-days |
| Small | 1 | <=30 | 5-10 |
| Medium | 2-3 | 31-80 | 11-30 |
| Large | 4-10 | 81-250 | 31-80 |
| Network/infrastructure app PT | |||
|---|---|---|---|
| Scope | Number of User Roles | Numbere of IP addresses | Estimated man-days |
| Small | 1 | <=20 | 10-15 |
| Medium | 2-3 | 21-100 | 16-50 |
| Large | 4-10 | 101-200 | 51-90 |
| Cloud PT | |||
|---|---|---|---|
| Scope | Number of IAM policies (variations of policies, e.g identical except for resource targets or subsets of policies, are counted as 0.5) | Number of Cloud Services | Estimated man-days |
| Small | 1-15 | 1-5 | 5-10 |
| Medium | 15-30 | 6-10 | 11-15 |
| Large | 31-45 | 11-15 | 15-25 |
| Mobile app/Thick Client PT (per OS/platform) | ||
|---|---|---|
| Scope | Number of core functions | Estimated man-days |
| Small | 1-3 | 5-10 |
| Medium | 4-6 | 10-20 |
| Large | 7-9 | 20-30 |
Core functions are discrete, distinct operations facilitating the interaction between the mobile app/thick client (including the backend) and app users, that require measurable, non-trivial effort to test. They also include sufficiently important functions within the application, such as functions that access sensitive data (e.g. PII/PHI/PFI). Examples of core functions include user transactions (e.g. user/resource management, payment), security functions (e.g. authentication), user queries, push notifications (if they contain sensitive information), etc. They may also include new functions to be added to the application that Customers wish to test. Core functions may be combined or separated if appropriate (e.g. several similar small functions performing transactions of a certain category, or a single large function performing several smaller functions), and the Supplier should work with the Customer to decide on a reasonable testing duration and manpower.
If the Customer is unable to provide mobile app/thick client binaries with security protections disabled, additional testing duration should be catered for the Supplier to circumvent them, unless they can be circumvented using off-the-shelf scripts (e.g. SSL Kill Switch for SSL pinning, fridantiroot/MagiskHide for Android root bypass, shadow/A-Bypass for iOS jailbreak bypass. In other words, additional time should be catered if manual circumvention is required. The following table serves as a guideline, and the Supplier should work with the Customer to decide on a reasonable testing duration and manpower.
| Additional man-days for manual circumvention of security controls (per OS/platform) | |
|---|---|
| Number of security protection controls that cannot be disabled | Estimated additional man-days |
| 1-3 | 5 |
| 4-6 | 10 |
Security protection controls fall into the following categories:
| Control | Description |
|---|---|
| Root/jailbreak detection | The app detects and responds to the presence of a rooted or jailbroken device either by alerting the user or terminating the app. |
| Runtime tampering detection | The app detects and responds to, tampering of the application during runtime, including debugger and hooking/code injection frameworks. |
| SSL certificate pinning | The app either uses its own certificate store, or pins the endpoint certificate or public key, and subsequently does not establish connections with endpoints that offer a different certificate or key, even if signed by a trusted CA. |
| File integrity checks | |
| Obfuscation | Obfuscation is applied to programmatic defences, which in turn impede de-obfuscation via dynamic analysis. |
| Encryption | Executable files and libraries belonging to the app are either encrypted on the file level and/or important code and data segments inside the executable are encryption or packed. |
2.3.3.1.2 Information
The Customer should provide the Supplier with all relevant information required for the penetration test. This includes the following:
- System/product name and description.
- Security Classification and Data Sensitivity.
- List of in-scope application URLs (for web apps and APIs).
- Application binaries (for Mobile and Thick Client) – to provide 2 builds if applicable:
- one build with all typical protections (e.g. root/jailbreak detection, certificate pinning, anti-debugging) enabled
- one build with all protections either disabled or configured for information only.
- List of security controls (for Mobile) – e.g. Runtime Application Self Protection (RASP) or anti-reverse engineering libraries used
- Source code of target application (for white box testing).
- Past PT and VA reports (at least 1 each), including the name of the Supplier which performed such PT and VA.
- Risk register, including risk acceptance with supporting evidence.
- Documentation of the various roles, privileges, and access control model.
- Network architecture diagram elaborating the following details:
- Identified network segments in scope for the test
- Servers
- Security controls/appliances
- Network/data flow
- API documentation and test data (e.g. Open API documents, Postman collections).
- List of all expected services and ports exposed at the perimeter.
- Development and infrastructure teams: (whether in-house or from external contractors).
- Logging mechanism: Agency Security Operations team, and/or GCSOC
2.3.3.1.3 Pre-engagement logistics
The Customer agrees to support the Supplier with reasonable and appropriate logistical requirements for the penetration test. This may include the following:
- Assigning a Point-of-Contact (POC) from the Customer to liaise with the supplier on all logistical issues.
- Provisioning of the test environment. The Customer is responsible for ensuring that the test environment is stable, and the testing machines can access the target nodes successfully. The Customer should also provide system output to ascertain that the test environment and testing machines have been successfully configured prior to the test.
- Providing information on any scheduled downtime or changes to the testing environment, which should be taken into account during scoping.
- Provisioning of minimum 2 user accounts per role (including admin role).
- Providing credentials (including MFA) to assume various roles and tenants (if applicable). The Customer is also responsible for ensuring that credentials are working before the start of PT fieldwork, and to provide evidence that they are working.
- Providing appropriate clearance of penetration testers
- For remote penetration tests, provisioning of access of suppliers to target nodes (e.g. via IP whitelisting).
- For on-site penetration tests (where applicable):
- Clearance of suppliers’ testing machines
- Provisioning of testing machines of mutually agreed upon technical specifications.
- Provisioning of a reasonable working area with desk and chair, functional power outlet and access to bathroom facilities
- The Customer should perform a system backup prior to the scan.
- The Customer should provide an application walkthrough to the Supplier with an Application User Guide or UAT Report to support scoping.
2.3.3.2 Fieldwork
An effective penetration test goes beyond vulnerability scanning to provide explicit proof of mission risks and an indicator of the level of effort that an adversary would need to expend to cause harm to the organisation. As such, the fieldwork should go beyond automated scans. Rather, manual testing is required to validate detected vulnerabilities and minimally identify additional vulnerabilities that cannot be detected with automated scans. This is where a qualified penetration tester is required to perform manual testing.
2.3.3.2.1 Methodology
The Supplier should follow industry recognised frameworks for testing systems, e.g. OWASP Web Security Testing Guide (WSTG) for web app PT, and Mobile Application Security Testing Guide (MASTG) for mobile app PT. The Suppliers should also identify reasonable and realistic threat scenarios based on the system assets and objective and develop and execute test cases for these threat scenarios.
| Cloud PT | |||
|---|---|---|---|
| Methodology | Description | Pros | Cons |
| Black box | No information other than the target is provided to the penetration tester. Any knowledge of the target will need to be independently obtained by the penetration tester. | Realistic simulation of what a real-world attacker would face. | Testing is not comprehensive, and highly dependent on the duration and skillsets of the penetration tester. |
| Grey box | Some information of the target is provided to the penetration tester. This includes test accounts (to access various roles), and some level of system knowledge (e.g. design documentations, network diagrams). | Allows for a comprehensive coverage of the system under test as the penetration tester has access to all areas as well as the knowledge of how things should work. | Requires agencies to furnish the required information. Potentially takes more time and effort to maximise coverage. |
| White box | In addition to the above information, the penetration tester has full access to the source code and architecture documentation, etc. | Allows for the most comprehensive testing to uncover vulnerabilities that are deeply rooted within the system and not easily discovered otherwise. | Requires agencies to furnish the required information. Potentially takes more time and effort to analyse source code and maximise coverage. |
In general, PT should minimally be performed with a grey box approach, with white box being preferred to provide additional assurance for systems of higher importance and criticality. Black box approach should only be considered where realism is a key factor in the security test, e.g. red teaming exercise.
2.3.3.2.2 Conduct
The Supplier should perform and document manual testing that cannot be achieved through automated scans to identify and exploit vulnerabilities.
The Supplier should perform and document post-exploitation steps to achieve the maximum impact of a vulnerability within the scope (e.g. obtaining persistence, privilege escalation, lateral movement, and exfiltration).
The Supplier should provide verifiable and reproducible logs of all testing activities performed (e.g. Burp Suite project files, PCAP packet captures, etc) and working documentation that captures a list of test cases performed, including test cases that were detected and/or prevented.
Except as otherwise agreed with the Customer, the Supplier should not perform high-risk test cases that would cause an availability impact on production assets, but may selectively perform high-risk test cases against non-production assets. This includes:
| Test case | Scope |
|---|---|
| Denial of Service (DoS) attacks of any form | Out of scope |
| ARP spoofing | Non-production server for selective scenarios |
| SQL Injection payloads on Create/Update/Delete functions | Non-production server |
| Memory corruption-based exploit such as buffer or heap overflow | Non-production server |
| Brute forcing with dictionary containing more than 100k guesses using more than 5 threads | Non-production server |
As part of penetration testing, the penetration tester would need to access the system which may result in access to sensitive contents in the database or system. Penetration testers should access only test data where possible and should not extract or otherwise use any information obtained in connection with the penetration testing for any other purposes.
In cases where high-risk test cases need to be performed, the Customer should be consulted before they are performed.
2.3.3.2.3 Industry tools and configurations
PT is performed with a mix of automated and manual testing, and the following tools are commonly used for testing.
| Tool category | Description | Examples |
|---|---|---|
| Penetration Testing OS | An Operating System (OS) typically built on Linux, with a wide range of tools commonly used for penetration testing and digital forensics. | Kali Linux, Parrot OS |
| Web proxy/scanner | A web proxy tool used to intercept and modify HTTP/HTTPS traffic. | PortSwigger Burp Suite, OWASP Zap, Fiddler |
| Web Dynamic Application Security Testing (DAST) | A software tool that analyses a web application through the front end to identify vulnerabilities through simulated attacks. | PortSwigger Burp Suite, Microfocus WebInspect |
| Disassembler/ decompiler | A software tool that takes machine or intermediate code as input, and generates human readable code (i.e. assembly/source code), used for reverse engineering of application binaries. | Hex-Rays IDA Pro, Ghidra, Jadx, JEB, dnSpy |
| Instrumentation framework | A software tool that dynamically instruments applications at runtime to observe and modify their behaviour. | Frida, Theos, QDBI |
| Debuggers | A software tool that attaches to application processes to observe and modify their behaviour. | Windbg, OllyDbg, GDB, LLDB |
| Network packet analyser | A software tool used to intercept and modify low level network traffic (i.e. TCP/UDP). | Wireshark, tcpdump |
| Password cracking | A software tool used to perform offline/online password cracking to obtain plaintext passwords. | John the Ripper, hashcat, Hydra |
2.3.3.2.4 Fieldwork logistics
The Supplier should provide regular (e.g. weekly) status updates via email or meetings for the duration of the fieldwork.
For discovery of High/Critical severity findings, the Supplier should inform the Customer within 24 hours of the finding.
The Customer should not make changes to the testing environment once agreed with the Supplier that may impact the fieldwork in terms of behaviour or functionality, unless necessary. This includes, but is not limited to, the following:
- Remediation of identified vulnerabilities
- Feature updates
- Changes in application logic
- Software version updates and patches
If incremental features, additions or changes are made as part of agile development practices, the Customer should identify and log such changes for future testing.
2.3.3.3 Reporting
The Supplier should disclose any alterations made to the testing environment as part of the test for the Customer to remove such alterations.
The Supplier should provide a report to the Customer, detailing the results of the penetration test. The report should minimally contain the following components:
- Sequential finding ID per scope
- Risk statement containing the vulnerability, business impact on the system component, and technical root cause of the vulnerability
- Brief description of the vulnerability
- Risk Severity of the vulnerability
- Reproduction steps with evidence (e.g. screenshots, HTTP request/response) and proof of concept (e.g. HTTP request, script)
- Corresponding location in the source code of the identified issue (for white-box PT)
- Recommendations for remediation and/or mitigation
The Supplier should assign base metrics for reported vulnerabilities using the Common Vulnerability Scoring System (CVSS) version 3 or newer. The Supplier may also collaborate with the Customer to assign Temporal and Environment metrics to derive a more contextualised CVSS severity.
In cases where potential vulnerabilities are identified but direct evidence of exploitation is not available (e.g. modification of a file on the backend server that the penetration tester has no access to), the Customer should cooperate with the Supplier to validate the vulnerability.
Where the Customer does not agree with the CVSS severities proposed by the Supplier, the Customer should discuss and clarify with the Supplier and provide relevant input that may moderate the CVSS severities. This input should be technical and not people and process- oriented. The Supplier should consider this input when revising and validating the findings, including their severities.
The Customer should perform a risk assessment of the identified vulnerabilities (e.g. using ICT RMM) and seek risk acceptance for findings that cannot be remediated within the established timeframe.
If required, the Supplier should validate the remediation/mitigation and provide evidence of remediation/mitigation.
The Supplier should declare any extenuating circumstances that were encountered during the fieldwork that might negatively affect the scan results (e.g. Application was only partially functional during the fieldwork).
If there is insufficient time to complete the scan, the Supplier should at its discretion, request for an extension of testing duration to complete the scan.
2.3.3.3.1 Prioritisaion for remediation (CVSS)
The Customer should prioritise remediation based on the CVSS severities of the identified vulnerabilities in the scan report, according to the established timeframe. If the Customer is unable to complete remediation within the timeline, the Customer should perform a risk assessment (e.g. via ICT RMM) and seek risk acceptance of residual risks at the appropriate forum.
2.3.3.4 Post-engagement matters
If requested by the Customer, the Supplier should secure erase and destroy any data (including confidential information obtained in connection with the testing).
The Customer should document and track individual findings internally (e.g. as a GitHub issue or GitLab vulnerability report).
The Customer documents and tracks remediation efforts internally (e.g. screenshots, code changes, configuration changes, etc).
