PCI DSS: When to Test Controls & Functions

By Jeff Man on January, 3 2019

Get latest articles directly in your inbox, stay up to date

Back to main Blog
Jeff Man

Respected Information Security advocate, advisor, evangelist, international speaker, keynoter, former host of Security & Compliance Weekly, co-host on Paul's Security Weekly, Tribe of Hackers, TOH Red Team, TOH Security Leaders, TOH Blue Team, and currently serving in a Consulting/Advisory role for Online Business Systems. Nearly 40 years of experience working in all aspects of computer, network, and information security, including cryptography, risk management, vulnerability analysis, compliance assessment, forensic analysis and penetration testing. Certified NSA Cryptanalyst. Previously held security research, management and product development roles with the National Security Agency, the DoD and private-sector enterprises and was part of the first penetration testing "red team" at NSA. For the past twenty-five years has been a pen tester, security architect, consultant, QSA, and PCI SME, providing consulting and advisory services to many of the nation's best known companies. Credentials: NSA Cryptanalyst, Hacker, QSA

Although 2019 promises a new version of the Payment Card Industry Data Security Standard (PCI DSS) the current version 3.2.1 is the de facto standard for measuring security programs for all merchants and service providers that participate in commerce using credit or debit cards.

There are twelve major requirements in the PCI DSS, and considering the complexity of the material we have chosen to dedicate individual blogs to the different requirements. The focus of these blogs will be to provide tips and pointers, help provide clarity for “what’s new” and to enhance understanding so that your organization can achieve a sustainable security posture that easily satisfies the requirements of the PCI DSS.


Since the release of PCI DSS Version 3.0, the PCI Security Standards Council (SSC) has included an informational section called “Guidance” within the requirements matrix. This section should always be referenced to determine the context of the requirement and to give you a glimpse of how the PCI SSC interprets the spirit or intent of the requirement.

Understanding the intent of the requirement is critical not only to assuring that you are meeting the requirement but even more so to determine viable alternatives or compensating controls to address the requirement.

To kick things off, we will look at Requirement 11, since it involves the most in-depth technical security controls and functions.

#11: Regularly Test Security Systems and Processes

11.1 –    Wireless Detection

This section has not changed much other than to explicitly allow multiple methods or technologies to achieve the intent of discovering both authorized and unauthorized wireless networks and systems.

11.2 –   Vulnerability Scanning

Companies have often complained about the difficulty of producing a “clean” or “passing” vulnerability scan as new vulnerabilities are discovered daily and subsequently the various scan engines update their checks on a frequent basis. As soon as one vulnerability gets remediated, and a new scan is performed to prove the remediation, there is very often new vulnerabilities discovered.

The PCI SSC acknowledged this difficulty and added a new “note” to 11.2 which gives companies the opportunity to submit multiple scan reports, that demonstrate that remediation processes are working on an ongoing basis, to satisfy the quarterly requirement for producing a passing result.

For example, if you scan on a monthly basis, the first scan might show 3 high-risk vulnerabilities that require remediation. The second would show that those 3 vulnerabilities had been addressed, but now there are 7 new high and 2 critical vulnerabilities that were published in the past month. Remediation occurs, and your third scan proves that these vulnerabilities have been addressed but there is still 1 more new high-risk vulnerability that prevents the “clean” result. What the PCI SSC is saying is that you can submit all three monthly scans to demonstrate that your company is effectively remediating vulnerabilities as they are discovered – even if you haven’t addressed the most recent vulnerability.


This allowance will make most company’s vulnerability management programs operate much more efficiently and should encourage more frequent scanning of their networks – which most security professionals already recommend. This allowance applies to both internal and external vulnerability scanning efforts so you should expect Approved Scanning Vendors (ASVs) to be updating their service offerings to accommodate this new feature.

11.3 – Penetration Testing

If you’re a QSA you’ve probably been confronted by clients that ask, “where does it say that?” and are firmly committed to only doing what the PCI DSS says it MUST do. This argument was often raised by my clients when it came to the penetration testing requirement which prior to PCI DSS v3 was quite vague. Well, those days are over as the requirement now stipulates that you must have a documented methodology for penetration testing that must be based on industry-accepted approaches and provide complete coverage – both internally and externally – of your cardholder data environment (CDE).

The Guidance section provides some much need context at this point stating that a penetration test attempts to “simulate a real-world attack situation with a goal of identifying how far an attacker would be able to penetrate into an environment.” The Guidance also stipulates that a vulnerability scan does not constitute a qualifying penetration test but must be part of a methodology that includes attempts at exploitation of known or discovered vulnerabilities. Ask any decent pen tester and they’ll tell you this requires manual exploitation attempts or at least a human behind the scenes that is making decisions based on available data as to what exploitation methods to attempt.

Another required element of the penetration testing methodology is to incorporate lessons learned from threats and vulnerabilities experienced in the prior twelve months (since the last pen test). What a great idea – but taking it a step further would be to incorporate threats and vulnerabilities experienced by the industry in the previous twelve months (think targeted malware attacks against POS systems).

The most important aspect of penetration testing though is to understand that it is intended to test the effectiveness of every aspect of your information security program – from system hardening to patch management to user access control to logging and monitoring. Closely related to this is the treatment or remediation of findings: if the penetration test indicates some success based on a missing patch (for example) the vulnerability is not the missing patch – it is the deficiency of the PROCESS that led to the patch not being applied to the target system. Maybe the penetration test was successful due to a misconfiguration or the presence of a default setting: the vulnerability then is the failure of the build PROCESS that put the vulnerable system into production in the first place. This point of view also applies to intrusion detection and incident response processes – if the penetration testing was not observed or detected there is a deficiency in the PROCESS of applying and monitoring any intrusion detection technologies you have deployed and/or the PROCESS of identifying and training qualified individuals with incident detection or reporting responsibilities.

Penetration testing exercises are “live fire” tests of your entire information or data security programs, and the results should be incorporated into your ongoing data security programs including annual risk assessment.


11.4 -    Intrusion Detection System / Intrusion Prevention Systems (IDS/IPS)

There’s not a lot new in this section. The basic premise of this requirement is to locate IDS/IPS sensors at strategic locations – both on the perimeter of your network (or your CDE) and at “critical points” within the network or CDE. The Guidance section says that you should monitor the output of these devices and investigate to determine the cause and/or extent of attempted intrusions. The more layers to your network – that is the more segmentation that has been implemented to isolate your CDE – the more “critical” locations you’ve created. Keep in mind that when you isolate your CDE from any other part of your internal network you are basically declaring that the other part is untrusted – which means you have a new “perimeter” to consider for intrusion detection.


11.5 -    Change Detection

The first and most obvious change is the language of the requirement shifting to “change detection mechanism” from the former requirement for “file-integrity monitoring tools.” The spirit of this requirement is the ability to detect changes to critical files such as operating system components, applications, libraries, or even existing security controls including log files. The most common technology to satisfy this requirement is white-listing solutions, which also help to address the anti-virus/anti-malware requirements in section. So now you have choices for addressing this requirement, and you may even get the additional benefit of covering more than one requirement with one technology solution.

11.6 -    Documented Policies and Procedures

Finally, because the old 12.1/12.1.1 requirement to “establish, publish, maintain, and disseminate a security policy that … addresses all PCI DSS requirements” apparently wasn’t clear enough, the PCI SSC added a new requirement to each section (starting with version 3.0) that says everything you are doing in this section must be written down in policies and procedures. So, 11.6 is a new requirement that states, “Ensure that security policies and operational procedures for security monitoring and testing are documented, in use, and known to all affected parties.”

Nobody likes to keep up with the documentation, but it really is a vital component to a viable information security program. Without policies and procedures, you cannot demonstrate that you know what [data] you need to protect, where it is located or transmitted (data-at-rest or data-in-flight), who requires access to the data (and who doesn’t), and you can’t effectively monitor access, attempted access, or administrative activities.

In short, without a written policy and documented procedures, you really don’t have a functional information security program.

Stay ahead of the game with our insightful blog posts on this topic, or contact someone from our security team with your questions!

Did you know that blog author Jeff Man is an original National Security Agency (NSA) Red Teamer and Department of Defense (DoD) cryptographer? If you have questions about this blog or a related topic:

Contact Jeff Man


Submit a Comment

Get latest articles directly in your inbox, stay up to date