Every penny counts to a regulator—especially if those pennies are needed to fund data analytics projects and justify a new supervisory framework that imposes hefty fees on firms the regulator oversees.

In January, the European Securities and Markets Authority (Esma) was granted new supervisory powers over the European Union’s largest data reporting services providers (DRSPs), including businesses such as approved reporting mechanisms (Arms) and approved publication arrangements (APAs). These powers come in addition to Esma’s authority over credit rating agencies, trade and securitization repositories, and third-country central counterparties.

To fund Esma’s new mandate, the European Commission legislated for a significant increase in supervisory fees—leaving some DRSPs facing hikes of up to 4,000%.

Esma has a revenue-based fee process and all the DRSPs were smarting at the level of those fees, but at the end of the day, those fees are going to get passed on to the investment firms and potentially down to the consumer

Dario Crispini, Kaizen Reporting

“We have gone from paying €30,000 [$32,660] to our regulator to paying a million euros. We question the viability [of the business] because of the enormous increase in our cost base and we have not been taken seriously. They do expect us to swallow it,” said a regulatory expert at a regulated data provider back in April.

The following six months have not changed attitudes.

“The fees they’re raising are very significant for authorization,” Dario Crispini, founder and CEO of Kaizen Reporting, tells WatersTechnology. Crispini is also a former manager at the UK Financial Conduct Authority (FCA), which helped draft Mifid.

“Esma has a revenue-based fee process and all the DRSPs were smarting at the level of those fees, but at the end of the day, those fees are going to get passed on to the investment firms and potentially down to the consumer.”

Nikolay Arnaudov, a team leader in the data reporting unit at Esma, says the regulator is aware of the upset unleashed by the new fees. “Our intention is indeed to use and leverage new technologies as much as possible for our supervisory duties and policy initiatives,” he says, hence the need for the price hikes.

In July 2021, as WatersTechnology first reported, Esma penned a deal with Accenture and Microsoft Azure to provide it with a big data platform for analyzing trade and transaction reporting—starting with the DRSP dataset.

The tech leap

The big data platform is an off-the-shelf solution co-built by Accenture, Microsoft, and their joint venture, Avanade. The solution is hosted on Azure and uses capabilities from Databricks, an open-source data management platform. It includes self-service analytics, data integration services, a data dictionary, the ability to code custom features on top of the tech, and a variety of business-intelligence tools.

The cloud-based solution has been in production since January and can process, on average, 40 million Mifid II transactions a day, says Arnaudov—double what the regulator originally forecasted for DRSP volumes.

“Trying to do that on any other platform but cloud is not possible because you just can’t scale out to that kind of size,” says Paul Hussein, chief architect at Esma.

The qualitative step is when we further connect the dots and exploit it together with other datasets. With the exercise of connecting the dots and developing a bigger picture we will be able to do [more sophisticated] data analysis

Nikolay Arnaudov, Esma

Esma intends to use the new capabilities to scrutinize reported data faster and be able to react quickly to market risks or data quality problems. The regulator hopes it will be able to use the tech to cross-reference previous reporting errors, so it can feed that information back to the national competent authority (NCA) to prevent repeat offenses.

“We are able to almost automatically assess the quality of the data and then [offer] a feedback loop toward supervised entities—it’s much faster, which of course leads to positive spillover in terms of resolutions and remediation of those issues,” Arnaudov says.

Esma’s end goal is to have pan-European, real-time visibility of entities’ exposures to risks and to be able to quickly react to them. Until now, however, its legacy systems were slow and ill-equipped to process large volumes at speed, Arnaudov says.

Esma’s old Oracle relational database was straining to ingest the growing volumes of data being reported to it by financial firms following the rollout of expansive regimes such as the Markets in Financial Instruments Regulation (Mifir). The legacy system was not prepared to meet the urgent need for scalable computing power.

Previously, Mifid II DRSP data was supervised by each of the individual EU-27 NCAs. This is why, Hussein says, there is no benchmark to show how much better Mifid II data will be analyzed following the switchover. By instead looking at other regulations and the speed at which those are processed, he says this system will be quicker due to its scalable computing power, cutting the processing time from several hours to minutes—or potentially real time.

The system had one of its biggest tests in February when it faced a massive spike in trading and reporting volumes of up to 250 million transactions, just as Russia’s invasion of Ukraine began. The increase in activity was also a result of an onboarding frenzy with DRSPs and national authorities, which led to a high number of duplicated reports and misreporting.

“That’s not even orders of magnitude of difference in what we estimated the volume would be; it’s a massive difference—it’s 100 times what we thought it would be—and [the platform] was able to process it,” Hussein says.

Show me the money

Esma’s IT budget is planned three years in advance and must be approved by the European Commission. Cost was a big decider in who the regulator chose to develop or implement the system.

“The thing that made the difference is that we have an industrialized managed service provider (MSP) who’s giving us a service out of a service center, where they are already providing a similar one to three, four or five customers,” Hussein says.

The MSP model played a significant role in driving down the cost of implementing and maintaining the technology for Esma, Hussein contends, as opposed to just buying the solution outright and managing it in-house. He adds that the regulator’s tech and data team have learned to “optimize” the platform’s code based on their needs.

“They’re able to optimize the analytics in orders of magnitude. So, we have gone from something like a €40,000 ($39,400) spend per month to a €10,000 ($9,800). What we used to use is a reasonable amount of horsepower all day. But the processing is only required three or four hours a day. So what we learned afterward is to turn that down, and only have it on for those three hours, or on demand, and then switch off.”

But there are many unanswered questions as to what Esma’s new powers and tech capabilities mean for the EU markets. A big question on the minds of those sitting in regulated firms’ legal departments is how the reported data has been used until now, and why it took so long for it to implement better analytics, says Vinod Jain, strategic advisor at Boston-based consultant firm Aite-Novarica.

“Mifid II came in 2018, and after five years the regulator is still unable to provide feedback on what they are doing with the data.”

One possibility is for the regulator to turn the data into a valuable resource for market participants. Jain says that in later phases of Esma’s data-driven initiative, it could take the insights it gains from the reported data and feed that back to reported firms, as a form of return on investment.

But industry participants shouldn’t count on that happening. An Esma spokesperson says Mifid II data is used for market abuse monitoring and holds information that can’t be made public.

“These types of analyses, the techniques used, and the outcomes are extremely sensitive to the NCAs and thus they are not publicly available,” the say. “They are key for the protection of investors and the orderly functioning of the markets.”

Data quality woes

There are many reasons why Esma is upgrading its tech capabilities. A notable one is the pressure on the regulator to justify the cost of regulation on financial firms over the last decade and assure them that their reported data is not a wasted effort.

Since the introduction of sweeping rules like Mifid II and the European Market Infrastructure Regulation (Emir), regulated entities have carved out massive portions of their tech budgets to build reporting systems, outsource compliance requirements to vendors, and acquire legal teams to comply with the rules. As such, those regulated firms have asked whether Esma has the capacity to analyze the relentless stream of reporting information, whether the quality is even high enough to make sense of—or even identify—market risks, how they might use it, and whether machine-learning tools should be applied to poor quality data.

And while a big part of Esma’s data-led ambitions is about improving the quality of regulatory reporting, doing so by supervising the DRSPs under Mifid, or the trade repositories (TRs) under Emir, is the wrong approach entirely, Kaizen’s Crispini says.

For instance, he continues, Arms function like a “postbox” where firms merely aggregate their data and submit it to the regulators, but he maintains that data quality issues should be tackled much earlier in the transaction lifecycle—before it even reaches an Arm or TR.

I think having a data analytics platform doesn’t solve the problem; the problem is somewhere upstream in terms of how the information comes in. It’s where the information is aggregated before you even do the analytics

Vinod Jain, Aite-Novarica

“Arms, TRs—their role is to collate records, consolidate them, and get them to an endpoint where they can be investigated. They’re not involved in any of the nuances that pertain to the construction of those reports, and where the Arms do provide such services, they sit outside the regulated activity,” Crispini says.

Aite-Novarica’s Jain adds that the problem is not about analytics, but quality. “I think having a data analytics platform doesn’t solve the problem; the problem is somewhere upstream in terms of how the information comes in. It’s where the information is aggregated before you even do the analytics.”

There is a long way to go before the data quality issues are resolved. A standout example of this was the EC’s report on the quality of Emir reporting in 2021. That report stated that based on early 2021 data, around 7% of daily submissions were being reported late by counterparties, and up to 11 million of open derivatives did not receive daily valuation updates. Esma estimated that there tended to be between 3.2 and 3.7 million open non-reported derivatives on a given reference date during 2020. And, additionally, around 47% of open derivatives (totaling approximately 20 million open derivatives) are unpaired.

So the contention is that an analytics system can only be as good as the data flowing into the platform. It’s an age-old problem: Bad data in, bad data out.

Onward and upward

Nevertheless, Esma is pushing on and has big plans to expand well beyond DRSPs.

The platform will be applied to a whole host of other datasets by the end of the year, including Emir reporting, double volume caps, reference and transparency data, and data reported to the trade and securitization repository venues. Subject to approval by the European Commission, Hussein hopes Esma will be in a position to move the remaining datasets it retains to the new platform by end of 2023, including data about registered entities or prospectuses. The different types of datasets analyzed by Esma are listed on its website.

The regulator’s long-term ambition is to be able to house all the data in a single repository and identify patterns and potential risks across the different European markets.

“The qualitative step is when we further connect the dots and exploit it together with other datasets. With the exercise of connecting the dots and developing a bigger picture we will be able to do [more sophisticated] data analysis,” Arnaudov says.

To some, however, having visibility across only the EU’s derivatives markets doesn’t go far enough. Today’s markets are strongly interconnected, Crispini says, and to be able to predict global flash crashes or systematic risks, you need a worldwide view of derivatives markets.

“The problem with Emir data is you need to be able to see it on a global scale [in order] to see where all the connections end,” he says. “But today, the trail goes cold once you go outside of Europe. So the design of the system—on a theoretical level—is too restrictive.”