Alternative Approaches to Penetration Testing

If your organisation has conducted penetration testing, it is likely that it has done so by taking a compliance-based approach, which is pen testing that is performed to meet an internal or external requirement.  You may be testing to remain compliant, for example, with an external standard or regulation such as the PCI DSS, or it may be your organisational policy, perhaps driven by contractual needs and requests of your client base, to perform penetration testing on a regular basis.  It can also include testing that is undertaken in response to major business changes, such as application updates, moving data centres, or transitioning to the new version of a standard.  This type of testing is usually repetitive, with little variation in scope and, therefore, often little variation in findings over time.  Your list of vulnerabilities will hopefully get shorter as you address any issues you’ve identified, or potentially lengthen if you introduce a new technology, but other than this, you’re unlikely to see much change.

Because compliance-based pen testing is repetitive, it requires limited consideration and is often very easy to conduct, rarely uncovering serious or surprising vulnerabilities which would require considerable effort to address.  It’s understandable that this straightforwardness can be appealing but taking the ‘easy route’ is not without downsides.  The most obvious metric you can use to measure your success when performing compliance-based penetration testing is the number of high severity issues you identify in comparison to the previous test.  This can lead to a simplistic approach to the security of your environment, i.e., if you have fewer high severity issues than last year, you may conclude that your security may have improved.   If you’re performing pen testing to remain compliant with a standard or regulation or at the request of a third party, it is also easy to fall into the trap of doing so with your primary focus being on producing a report rather than improving the overall security of your environment.

At this point, let us look at alternative approaches you can take in conducting pen tests,

Risk-Based Penetration Testing

When undertaking risk-based pen testing, the scope is informed by the outcomes of risk assessment, or, more specifically, defined by highlighted risks.  Here, you are likely to have identified risks within other governance, risk and compliance (GRC) work, and while you may have qualitative assessments, you may not have quantitative data.  Risk-based pen testing can provide this data, delivering greater insight into these risks and how likely they are to occur.  Unlike a compliance-based pen test, the scope of risk-based pen testing is not technologically bound.  When conducting a compliance pen test, you are often limited to testing applications, infrastructure, etc., whereas a risk-based pen test will broaden the scope, providing a much more realistic view of your technology.

Not limited to cyber security risks

The risk assessment you use to conduct risk-based penetration testing does not have to be specifically related to cyber security.  Any risk assessment can reveal risks that are relevant to technology or security.  As long as there is a risk that can be addressed by pen testing, it can be used as the basis of a test.  

The process of scoping a risk-based pen test should be a collaborative process between you and the tester.  If you have a particular risk you want to address, your tester can determine which technologies should be within the technical scope to give the best view of that risk.  Naturally, this will depend on the nature of your organisation and whether your risks are static or dynamic, and you are unlikely to test for every risk you have identified in your risk assessment.  In summary, a risk-based approach does allow you to tailor the scope to fit with your organisational needs..

Risk-based penetration testing requires additional services, such as risk consultancy, prior to the test taking place.  The technical findings themselves are presented identically to a compliance-based test.  The added value comes from the additional context which is provided in a risk-based approach.  The wider scope of a risk-based pen test can also expose findings which fall ‘in between’ the scopes of compliance-based tests.  

More novel and relevant pen test report

The report that is produced from a risk-based penetration test will be more novel and relevant to your organisation than a report that comes from compliance-based testing, inspiring a wider range of questions from its readership than can be expected from a compliance-based report.  If a risk is revealed to be less likely to occur than previously thought, this can lead to questions about whether that risk should subsequently be revalued.  Meanwhile, if the test reveals a risk is more likely, you may question how it can be mitigated.  This enables you to actively improve your security posture, rather than just ticking off vulnerabilities as you may do with a compliance-based pen test.

Business-Led Penetration Testing

This type of penetration testing is designed to address real-world concerns for organisations, allowing you to understand how easy or difficult it may be for those concerns to become a reality.  There may be concerns within your organisation that an attacker could access your sensitive data, e.g., what would happen if a user was phished or if ransomware was on the network, if you’re able to successfully mitigate incidents, etc. Business-led penetration testing can provide insight into these questions.  While a compliance-based test may come across as somewhat adversarial as you are presented with a list of things you’re doing wrong, a business-led test is much more collaborative as it is designed to guide you towards improving your security posture.

Highly flexible scope

Business-led pen testing has a highly flexible scope, and the scope of business-led testing scales almost infinitely.  The report that is produced from a business-led pen test will be direct and will respond to the questions and concerns you have outlined.  At a minimum, testers can contextualise their findings and tailor the way they report these findings to you.  

Possible expansion of a compliance-based test

Although you may need to perform compliance-based pen testing to meet certain requirements, you can always expand the scope of this testing in response to the questions and concerns you have, adding a ‘real-world’ dimension to an otherwise very generic form of testing.  Doing so does not necessarily have to be more expensive – if there are parts of your pen test that can be automated (which there will be in most cases), your tester can maximise value by working efficiently wherever possible.

As the scope is determined by your organisational concerns, you will likely find the report much more interesting and valuable than you would a compliance-based pen test report.  Like the risk-based approach, you are likely to discover findings which have fallen between the cracks of the specific technologies that are in scope for a compliance-based test.  The effort that has gone into defining the scope by your specific organisational concerns is also more likely to inspire buy-in to the outcomes detailed in the report, and positive action in response to the vulnerabilities identified.

Benefits and Limitations of Different Pen Testing Approaches

Compliance-based penetration testing

The scope of compliance-based pen testing is very easy to determine, as it will usually be very similar if not identical to the scope from your previous test.  This type of testing is also the most cost-effective approach, and the report will naturally meet your organisation’s compliance requirements.  The outcomes are generally predictable, and it is unlikely that they will spark major outcry among senior management if a major vulnerability is identified.

However, some of these advantages can also be interpreted as disadvantages, depending on your perspective.  As the scope is so easy to define and doesn’t require input from a range of individuals, it is unlikely to inspire significant interest or emotional investment from your organisation.  Meanwhile, the predictability of compliance-based reports can lead to complacency and incremental security improvements which are unlikely to provide the impetus necessary for tangible improvement of your security posture.

Risk-based penetration testing

The benefits and drawbacks of risk-based pen testing are almost completely oppositional to those of compliance-based testing.  Risk-based pen testing is not an ‘audit’ style assessment, so it’s unlikely to be useful for meeting external compliance requirements.  This also means that not every in-scope vulnerability identified by your tester will be listed on your report.  The tester will be working towards making a risk ‘real’ to your organisation, so will use the testing time for the achievement of this goal, rather than providing you with an exhaustive list of often inconsequential vulnerabilities.  

Greater financial investment

Risk-based testing also requires a greater financial investment from your organisation, as you will need to perform a risk assessment in order to conduct a risk-based test, but this does mean the scope is totally tailored to your organisation.  You will know ‘from the off’ that the areas you have tested are a realistic threat to your organisation and the report will directly address these threats.  Unlike a compliance-based report, the findings will be novel each time.  Each risk will be different, and the findings your tester uncovers and the ways these findings are explained will differ as they will be related to you and the potentially dynamic risks your organisation faces.  The findings of the report can also inform future consultancy.  For example, if you have risk registers evaluated and updated, the inputs from the pen test can be used to adjust how risks are weighted and treated internally.

Business-led penetration testing

Once again, the primary disadvantage of business-led penetration testing is that it is not an ‘audit’ style assessment, so won’t be as exhaustive as a compliance-based test.  But aside from this, business-led pen testing comes with a wealth of advantages. This approach to pen testing gives you ultimate control over the scope of your test.  When conducting compliance-based testing, your scope is dictated to you by compliance or internal requirements, and the scope of risk-based testing is defined by predetermined risks. However, with business-led testing, any concern your organisation has, whether it is ongoing or topical (e.g., your industry is being targeted by a spate of phishing attacks and you want to know if you could be vulnerable), can define the scope of your test.  

As with risk-based testing, the bespoke scope of a business-led test will inspire greater investment from those who have developed it, and therefore greater investment in actioning improvements.  It can also scale to extremes, with testers having the ability to test for specific vulnerabilities on a scale that may not be a possibility for your internal security team due to time constraints.  As well as this, the outcome of a business-led penetration test will directly address your organisation, rather than providing a generic list of vulnerabilities as is the case with compliance testing.  In URM’s experience, this type of testing can attract the greatest levels of business engagement.
‍

The diagram above highlights how different types of testing work. While technologically focused and compliance-based penetration testing (represented by boxes at the bottom of the diagram) go very deep, the breadth of the assessment is constrained.  Business and risk-based tests, on the other hand, don’t have the same depth, but have a much broader scope.  It can be argued that this more accurately reflects the approach of a real attacker, i.e., they won’t exploit a highly complex vulnerability if they can gain a quick return through phishing, for example.  Attackers won’t work harder than they need to, so it’s important to address those low-hanging fruit.  

Appropriate Scenarios for Risk-Based and Business-Led Penetration Testing

The most obvious scenario where risk-based pen testing would be a useful measure is following risk consultancy, as it can provide you with technical data about the risks you have identified.  If a security incident has occurred, this can also be a good time to conduct penetration testing. You can use the pen test to look at the same or a slightly modified scenario as the one that has recently occurred, and this can even move beyond penetration testing.  Your tester can conduct tabletop exercises and/or technical simulations to exercise your playbooks and runbooks, providing your team with the opportunity to respond again, discuss what they did last time and consider whether it could improve the efficacy of its approach to the incident.  

If you perform a compliance-based pen test you will ideally want to test your entire estate which, if your organisation is extremely large, will require significant time and financial resources.  Often due to the sheer size of the scope, this may result in organisations adopting a sample-based approach.  While this can work for some compliance-based tests, it isn’t always appropriate.  A business-based or risk-based test can work around these challenges and allow you to understand the environment and address areas of greater concern or sensitivity.  

If your organisation has an ongoing security project, a penetration test based on your project and business objectives can be extremely valuable. By having an assessment prior to and after the project, you will be able to see the improvements that have been made to your organisation’s security and evaluate their effectiveness.

Industry Examples of Business-led Engagements

If you’re a service provider and send appliances out to customers, you may question what your customers would be able to do within your network, but this is unlikely to ever be covered by a compliance-based pen test.  Customer edge devices are not on your network, so automatically fall out of scope for a compliance-based test.  However, a customer could potentially gain access to your network and perform actions on a device you have provided them with.  A business-led penetration test would allow you to address these concerns and uncover vulnerabilities which may not be identified with a compliance-based test.

Businesses with distribution centres may be concerned about the possibility of that distribution centre being interrupted from the enterprise network, particularly if stock is received on a ‘just in time’ basis and there aren’t major stockpiles to fall back on.  In these circumstances, a very straightforward attack which interrupts operations could potentially have a major impact on your ability to function, so these concerns make sense to be the scope or focus of a penetration test.  

Organisations may want to evaluate its response to ransomware following a spate of attacks among other providers in their sector.  In this setting, a tester would be unlikely to have ransomware live on the network, but they can conduct tabletop exercises, simulations, or perhaps even targeted testing on a single device.  A business-led approach allows a penetration tester to create a highly tailored scope around what’s available for testing, and what you as the client are comfortable with.

Although these examples are industry specific, the challenges they raise and the solutions business-led pen testing can provide are widely applicable.  

Incorporating Alternative Testing Approaches

When approaching your scheduled penetration testing, it is worth questioning why that testing is taking place, and whether there is a genuine need to provide anyone with the report produced from it.  If your customers have requested the test, have they told you what type of test you need to do?  Are you simply performing that test in order to maintain the status quo?  Do your current pen testing practices provide the best value for your organisation?  If you’re responsible for managing risks in your organisation you will have the policies and processes perfected. However, you may not have as clear an understanding of how those risks translate technically, and using an alternative approach to pen testing can provide you with this information to help you understand the practical impact of your organisation’s risks.

Raise internal profile

If you’re a security manager, your role can often be invisible until the event of a security incident. Business-led penetration testing can provide you with positive visibility outside of these circumstances and generate more organisational buy-in to security.  By proactively asking senior management to list their primary security concerns for the scope of a pen test, you can use the subsequent report to demonstrate that those concerns are unlikely, highlighting a job well done.  Alternatively, if the report shows that there are major weaknesses, the fact that you now have these weaknesses detailed in black and white means you are more likely to receive the necessary investment to mitigate your risks before an incident occurs.

This is not to say you should disregard compliance-based testing entirely – it fulfils a useful role and meets the requirements of clients and schemes such as the Payment Card Industry Data Security Standard .  But it is always worth considering whether you can do more than the bare minimum and extract greater value from your testing.

Summary

Alternative approaches to penetration testing can often yield better and more effective outcomes for your organisation.  It’s key to consider what you want to achieve through conducting penetration tests.  If you need a report to provide to a regulatory body, then compliance-based testing is achieving your objective.  However, if the reasoning for your pen testing is more arbitrary i.e., the testing is being done because your organisation wants it done, reconsidering your approach may prove to be beneficial.  

Your pen testing provider should be helping you to define your scope, and good providers want to see positive outcomes and improvement in your security.  Often, testing providers will simply send out a questionnaire to their clients, who then return a list of IP addresses which provides the basis for the test.  This approach gives the tester little to no information about your organisation, its business objectives, its existing GRC posture, etc.  Providing your tester with this information, as you would in a business-led or risk-based assessment, enables them to write their report in a way that is both relevant and valuable to you.

How URM Can Help

URM can offer your organisation a range of penetration testing services, the quality and reliability of which are verified by our CREST and CREST OVS accreditation.  

If you’re looking to explore alternative approaches to internal and external penetration testing, URM provides specialised, business-led pen testing services, using advanced technology-based methodologies to align with your organisation’s specific objectives and concerns.  Our testers can investigate a variety of issues, including assessing unauthorised access, evaluating your vulnerability in the event of a successful phishing attack, and scrutinising your IT administration privileges.  Rather than relying on generic, predefined scopes, our business-led pen testing services address the real risks and security challenges your organisation faces.  

However, we recognise that traditional penetration testing can still be incredibly valuable and can offer a range of more technologically bound testing options.  With its CREST penetration testing, URM is able to help you meet your compliance requirements and maintain the security of your organisation.  We offer infrastructure and network penetration testing services against all IP addresses associated with your organisation, location or service.  We can also conduct web and mobile application penetration testing, as well as cloud penetration testing, covering all types of deployment.  

Our team of testers will provide you with continuous support, and you will receive a free retest of any high or critical vulnerabilities we have identified within the first 30 days of the report being produced.  We also offer a range of GRC services to augment our cyber security penetration testing, including the development of policy, process and training solutions to address your security weaknesses.