1. Understand What You Are Buying
Depending on the provider, what you get when buying a penetration test can vary greatly. There is no unanimous standard for what a pentest is and how it is supposed to be conducted. It is therefore important for you to ask the provider about what methodology or standard they are following. If the answer is “my own”, there is reason to worry.
In order for you to maximize value of the test, the provider should follow one of the international standards for pentesting, such as the Penetration Testing Execution Standard (PTES) or OWASP for application testing. First of all this will ensure you a structured testing process. Second, you will have an idea of what you are buying.
2. Avoid the “Big Fat Button” Test
The pentest market is expanding, and so is the number of pentest providers. There are two types of providers, the ones that are passionate about what they do, and the ones that are in it for the money.
Some of them will probably offer you what is called “big fat button” test. This test consists of a security consultant directing an automatic tool against a network or application, and thereafter letting the tool do all the work. Unfortunately a tool can only scratch the surface and find obvious vulnerabilities. This approach will provide little value to organizations that have a minimum level of security.
Experience has shown, time and time again, that creativity and manual work will make a huge difference in the test results. Do not underestimate the importance of understanding context and business logic while finding and revealing critical vulnerabilities.
3. What to Expect From the Final Delivery: Reporting
A penetration test usually results in a report. The content of the report can vary greatly. As a minimum, you should make sure that the provider documents all the vulnerabilities and proves them. You should also make sure that a remediation plan with solutions is included.
Be aware that some remediation plans are made with the sole purpose of selling you more security solutions. Small changes in configurations or coding, can sometimes greatly improve your security at almost no cost. You should look for independent providers who can make this kind of recommendations in their remediation plans.
4. The Pentest is Only as Good as the Pentester
The result of the pentest is highly dependent on the person performing the job. When evaluating the providers’ pentesters, you should have a look at their CVs. How long have they been in the game? What formal education and certifications do they have?
Most importantly, you should look at security accomplishments. How many security advisories, papers or CVE’s have they published? Do they have their name on any “Security Hall of Fame”? Have they given talks on any security conferences? Which pentester will be allocated to your project?
5. Remember to Call References
The best way to evaluate a pentest provider is to ask for relevant reference customers. Make sure you ask the references about the value and quality of work that has been delivered. This will give you an honest perspective on what you can expect from the provider.
Guide – Pentesting: Scope & Frequency
We recommend that you read our guide for network security testing, where you can find tips related to recommended testing frequency, scope and methodology.