Stylized application in a science lab.

No security assessment can hope to find every vulnerability.  The time scoped by Meristem for each assessment is intended to find the majority of vulnerabilities, including those likely to be found by a skilled attacker.  Still, the more efficient the testers can be, the more attack surface they can cover, and ultimately, the more secure your application will be.  Meristem is always looking for ways to optimize our process, but there are steps that you can take to ensure that the majority of your assessment time is spent testing which will maximize your return on investment.

Choosing The Environment

The environment that the testing will be done in has a huge impact on the coverage of the test.  Ideally, a clone of a pre-production environment can be made, with the desired code version deployed and populated with a complete set of simulated data.  Alternatively, a pre-production environment could be dedicated to pentesting for the duration of the engagement, with a database backup taken just before testing begins so that the environment can be reset after the test.  If there are no other options, testing can be performed in a production environment, but no matter how careful the tester is, there is a risk that site performance could be degraded, or that side-effects from testing could be encountered by a legitimate user. Also, if the testers do find gaps in access control restrictions, they could gain access to sensitive information of real users which may be considered a data breach in some contexts.

An Ideal Test Environment Is:

  • Dedicated - Many tests involve storing attack payloads that may end up being triggered in other parts of the application. Having other users working in the same environment puts them at risk of falling victim to one of these attack payloads.
  • Disposable - Many automated tools submit a broad array of attack payloads and then test to see if any of them were successfully triggered. This often results in the creation of many business objects that may or may not be removable. The best environment is one where the database can be reset at the conclusion of the test.
  • Realistic - An environment that is populated with realistic values for all application workflows helps the tester understand how the application is actually used, and lets them focus on testing rather than figuring out what realistic data values might be.  Some features are triggered only by specific data values which might not be discovered if the tester is populating the application themselves.
  • Configured like Production - The findings included in a report must describe the environment that is actually tested. Even if a mitigating control exists in production, when a vulnerability is found in the test environment, the vulnerability must be included in the report. In some cases, it may be appropriate to test the mitigating control in production, and update the report accordingly, but this would need to be coordinated ahead of time.
  • Static - If the environment changes while testing is being performed, such as by code being deployed or a firewall being reconfigured, the testers are usually forced to go back and repeat tests, taking time away from other areas of the application. Of course, if a critical vulnerability is discovered, it may make sense to deploy and test a fix immediately, since eliminating that risk would be more important than identifying additional, typically lower severity findings.
  • Stable - An application that isn’t functioning correctly can’t be adequately tested. Freezing changes to the environment is one way to accomplish this. Also, Meristem will report any odd behavior, and appreciates being notified promptly of any outages that occur that impact the environment being tested.


Providing Test Users

In general, Meristem asks for two user accounts for each role that will be tested in order to properly assess both horizontal and vertical access controls. Horizontal access controls protect one user’s data from being accessed by another. If the application supports multiple tenants, it is important that the two users be configured in different tenants, since cross-tenant access is typically a worst case scenario.  If business rules dictate that data should be protected both from other users in the same tenant, and across tenants, three users from the same role may be required.

Vertical access controls prevent a user in one role from using features restricted to another role.  The “higher” privileged role will be used to understand how the features work, and then the tester will try to use that feature as the other lower privileged roles.  Even highly privileged roles that are not given to external-users should be included in the assessment, since these would carry the highest risk if compromised.

Applications that support dynamic roles, where custom roles can be created with specific permissions, are more complicated to test.  Here, it often makes sense to create a role with all allowed permissions and another role with a minimal amount of permissions.  Again, this allows the tester to understand the feature using the high-access role and test the limiting controls using the low-access user.  Of course, if there is a “standard” set of roles, it may also make sense to simply use these.


Access to Source Code

A source code assisted application security assessment is different from a full source code review. In an application security assessment the tester focuses on the user interface and network requests observed during use of the application. A source code review examines the code-level implementation of functions and security controls and uses automated tooling to scan the entirety of the code base if possible. While a source code review is likely to identify more vulnerabilities than application security assessment, it requires more time and effort to complete.

Having access to source code during an application security assessment is still very beneficial even if a full source code review is not being performed. Access to source code allows the tester to quickly look at the implementation of a specific control rather than deducing the implementation based on the results of multiple tests. For example, the tester can see exactly what rules are being applied during input validation, predict and test specific payloads that would bypass these rules, and provide recommendations on improving the specific implementation. Also if the tester identifies an injection vulnerability in one field they can examine the implementation of that feature and use code searching features to see if the same vulnerable construction is used elsewhere in the application. Access to source code may also enable the tester to identify specific line numbers where fixes should be made, reducing mitigation time.

Providing access to source code is not required and goes beyond most common definitions of an application security assessment, but will significantly increase the value received from the testing engagement without increasing the cost.



During an application security assessment the testers attempt to identify the vulnerabilities an opportunistic attacker or malicious user might exploit, with the exception that Meristem will not perform attacks that are likely to cause an outage or degrade the performance of the environment without specific permission. Attackers start with the advantage of unlimited time to gather information about the site and attempt attacks.  Since a finite time is allotted to an application assessment, limiting testers to the identical conditions an attacker would work under is counter productive. Providing additional information in the form of access and even source code, tilts the scale towards the testers.  An application pentest is not a contest to see how good the testers are, the goal is to find as many of the vulnerabilities that already exist as possible, so they can be eliminated, ensuring no attacker can find them...ever.