Stress Testing Your Sanctions Screening Effectiveness

Independent testing can serve as a functional, scientific and transparent approach to bolster the effectiveness and efficiency of sanctions screening systems.

Amid growing complexity and fragmentation of global sanctions regimes, a key priority for banks in the year ahead will be to ensure their screening systems are performing effectively, reliably and predictably. Regulators today have likewise been increasingly expecting financial institutions to be able to demonstrate that their sanctions screening systems are configured correctly.

The HKMA (Hong Kong Monetary Authority) has demonstrated leadership in this area. In April 2018, it published findings from a thematic review of banks’ sanctions screening systems. At the time, it said the adequacy of banks’ sanctions screening systems and controls was a supervisory priority, particularly in the light of geopolitical developments, which have only grown more complex since.

Among the key recommendations from the review, the HKMA said banks should put in place “regular sanctions screening system testing that provides robust reporting and quality assurance to senior management that the regulatory expectations are being met.” Testing helps banks to ensure they understand their screening systems and processes, and are able to optimise performance, it added.

“As a good practice, [authorised institutions] should take steps to satisfy themselves the system is appropriate and operating as expected before relying on automated screening systems,” the HKMA said. Testing should be independent, and conducted prior to a system deployment or upgrade (and regularly thereafter), to validate that all system filters are working properly and sanctions list changes are captured.

Bulletproof analysis

Guy Sheppard, Head of Financial Crime for Asia-Pacific, SWIFT

Guy Sheppard, Head of Financial Crime for Asia-Pacific, SWIFT

According to Guy Sheppard, Head of Financial Crime for Asia-Pacific at SWIFT, several other Asia Pacific regulators have closely followed the outcomes of the HKMA review with a view to local replication, as they seek to raise industry standards around sanctions screening and its effectiveness. “This highlights a trend towards an outcomes-focused, rather than a risk-based approach, to financial crime compliance.”

The FATF (Financial Action Task Force), for one, has been increasingly focusing on effectiveness in its mutual evaluations, as opposed to mere ‘technical compliance’ with its recommendations. Through effective implementation, countries are able to make society safer by not only detecting illicit financial flows, but by preventing and stopping them before they enter the financial system and other sectors, it says.

According to Sheppard, sanctions screening testing not only helps banks to assess effectiveness and obtain assurance and visibility around their implementations; it helps compliance teams better understand the controls being used and how filters behave, so these can be configured with better alignment to a bank’s risk appetite, and ultimately improve outcomes.

“Being able to explain how your sanctions screening platform works and defend your risk policies to the board and senior management, as well as regulators, gives your stakeholders comfort that your bank is managing its compliance obligations and mitigating sanctions risk effectively,” he says. “Often the key concerns from internal and external stakeholders tend to be around performance, effectiveness, and high false positives in existing screening systems.”

The typical sanctions screening system at a bank is provided by a third-party vendor, and deployed with default settings which may not necessarily account for a bank’s risk profile. According to Sheppard, the high false positive rates common in these implementations present a significant operational burden to financial institutions, with implications on operating costs, licence fees, system maintenance costs, and data fees.

“With hard, irrefutable data, independent sanctions testing can demonstrate with a precise numeric score how accurate a bank’s sanctions screening programme is, including to benchmark against global banks as well as regional and local peers,” he says. “The level of detail provided in this externalised view helps firms identify specific weaknesses in their sanctions screening systems, which can then be rectified and help to reduce false positives, in some cases by over 60 percent.”

Some banks are also using independent sanctions testing as part of their procurement processes, ahead of making any significant investment to switch between vendors or upgrade existing systems. Such testing allows them to compare different upgrade options, which can often result in a decision against an upgrade, in favour of merely adjusting filter settings to achieve better results.

Describing a typical sanctions testing exercise conducted by SWIFT, Sheppard says the initial tests often lead to a “real leapfrog” in terms  of effectiveness and efficiency, simply by adjusting fuzzy matching thresholds from default levels, in line with a bank’s risk profile. The sample-based testing approach commonly used at banks is not enough; they need to be able to conduct a statistical analysis using full sanctions lists to confirm that they are being alerted to all true positives (hits), as well as false negatives (misses).

“We stress test your filters to give you an understanding of where your sanctions screening system is performing well and where it is not. And then you can gauge whether the system is actually fit for purpose and how to make it better,” Sheppard says. “This bulletproof piece of analysis can also be re-run as you incrementally make improvements and tweaks, one of the foundational elements of any financial crime prevention programme.”

Filter faults

Through its sanctions testing service, SWIFT has identified numerous instances of screening filter faults and poor practices at individual banks which required immediate remediation. At one institution, the filter had accumulated a list of 600 stop/ignore words over years to reduce false positives, excluding these words from screening entirely. The results from testing identified that the bank had failed to flag over 150 sanctioned names by excluding these words, examples of which included ‘Engineering Equipment Company’, ‘Foundation for Construction’, and ‘Real Estate Bank’.

In another example, a bank discovered fundamental flaws in its policy for declaring matches where it insisted on a heavily weighted name and country match before the system declared a hit, possibly due to previous challenges with high false positives. Consequently, the bank’s effectiveness scores were extremely low as it was not registering partial matches in line with its policy. “Not every testing process offers up a smooth result,” Sheppard says.

Several years ago, SWIFT also discovered that one vendor was inadvertently failing to flag every sanctioned individual with an Arabic surname containing a hyphen. In another instance, a data vendor had simply missed a sanctions list update. There have also been instances where sanctions names included in screening had already been removed from OFAC’s SDN list.

While these issues were ultimately fixed, as a result of SWIFT’s intervention (and collaboration), they highlight the importance of sanctions testing in protecting financial institutions against risk and potential penalties for noncompliance. In addition, they point to a need for banks to ensure that the lists they screen and test against are accurate and up to date, and the contents of third-party lists are classified correctly and names that are no longer relevant are cleansed.

Sheppard says banks need to be able to run sanctions testing in an environment that mirrors their production system, to ensure the test results are indicative of the outcomes that would occur during live customer screening. Some banks have been testing their systems using test environments that are shared for other purposes and therefore have configuration changes lumped in. In other cases, the test environment contains out-of-date sanctions lists, or resources or licence issues prevented testing, he says.

Scientific and transparent

Ultimately, banks need to be sure that their sanctions screening systems are effective, efficient and provide the necessary coverage. Regular testing and fine-tuning of these systems are key to providing this assurance, while also allowing financial institutions to demonstrate that their systems and processes are configured in line with regulatory obligations.

“Global sanctions regimes are becoming increasingly complex, and in some cases, openly contradictory; yet, they are incredibly nuanced,” Sheppard says. “In Asia Pacific, there is a realisation among regulators that some businesses are still helping to finance North Korean entities or evading other sanctions regulations. Authorities will be actively trying to ensure that this risk is being addressed.”

Still, some banks in Asia Pacific have been enhancing their screening processes with sanctions testing not to satisfy regulatory obligations, but to allay concerns from their US clearing partners. Others simply have too many sanctions lists to screen against, and use testing as a way to assess the operational impact of adding a new list as part of their decision-making process.

Whether prompted by a regulator, counterparty or internal considerations, independent testing can serve as a functional, scientific and transparent approach to bolster the effectiveness and efficiency of sanctions screening systems. Financial institutions that have not yet put in place regular sanctions screening system testing should look to do so in the year ahead.

“Testing is often the unsung hero of an AML or sanctions programme,” Sheppard says.

In December 2020, SWIFT was awarded ‘Best Solution in Sanctions Filter Testing‘ by an independent panel of industry experts serving as judges at the 3rd Regulation Asia Awards for Excellence 2020.

More information about SWIFT’s Sanctions Testing Service is available here.

To Top
Share via
Copy link
Powered by Social Snap