Unsurprisingly, the market data manager was angry. But not just “My boss wants me to control costs, and it’s very difficult” irritated. No—he was red-faced-furious.

Luckily this was not directed at myself or Substantive Research. We had just come to visit him to swap notes about possible benchmarking methodologies for market data product pricing, and this was one of many brainstorming meetings we held during 2018 and 2019. But why was he so angry?

The market data manager had just had to swallow a £100,000 annual increase from an already expensive provider, and his firm had almost no choice but to stick with this provider regardless of the new cost implications.

But even that is not really what was fueling his indignation. After hearing that his firm would have to absorb these increased costs, he had then accompanied the account manager out to the lobby of his building, who, as he left, added, “I’ll be out of contact next week. We’re having such a great year, the boss is flying us and our partners to Mauritius for a week!”.

This was perhaps not the most sensitive thing to have announced. As the market data manager walked back into the office, he anticipated the unpleasant discussions that would ensue about the constantly increasing data budget and his firm’s inability to control it.

At this time, we’d already been successfully benchmarking investment research costs for the majority of leading asset managers in Europe and the US. But we’d been reluctant to turn our price and supply transparency processes toward the market data industry unless we could confidently address three challenges:

1. “Apples-to-apples” comparisons looked impossible. Market data use cases seemed too complex and specific to each firm.
2. We didn’t know if pricing was consistent. If these providers were so powerful, then maybe they just charged everyone the same and increased prices for everyone all at the same time and at the same levels, making the benchmarking of pricing pointless.
3. What advantage can institutions gain in knowing they are paying too much when they have no alternatives and the providers have all the power?

The market is shifting

So what changed in the last two years that spurred us to begin building the database and attempt to help our disgruntled new friend?

First, many firms have succeeded in gaining an in-depth, comprehensive understanding of what they are using and how they are using it, so their use cases are more clearly defined and more understandable. This was the beginning of a journey they had taken to combat the perceived powerlessness of their existing situation.

Second, as we began to respond to increasing customer requests for clarity, our initial monitoring of the market showed clearly that pricing was not consistent, as we outline in detail below.

And finally, while the power dynamics of this market haven’t shifted fundamentally, market data consumers are now mobilizing their efforts to be able to un-handcuff themselves from certain providers in certain areas in the short term, while planning for greater flexibility in the long term.

Shining a light on opaque market data pricing

We began our efforts with the buy side to capture all possible and precise use cases—per product and per provider—and ensure accurate comparisons, a goal that has eluded industry participants until now. We could then overlay this with our proprietary pricing models that combined publicly available and user-specific data.

We can now look at the datasets’ first conclusions from an aggregated, market-wide perspective, using the index market—a core area of focus and cost for the buy side particularly—as an example. Our findings are stark:

• Pricing is not consistent: there is a large variance in pricing and little or no correlation to the size of the firm consuming the market data. Even accounting for specific “apples-to-apples” use cases, the range of inconsistencies applied to pricing post-negotiation is between 10% and 50%.
• Buy-side budgets vary from 0.55bps to 1.27bps of AUM. This is one of the largest costs of doing business for an asset manager and it is increasing materially year on year.
• Average potential savings per provider are $360,000, and with many institutions using a long list of providers, this figure scales up rapidly.

Our findings also highlight that the opacity of the index market enables large pricing disparities between buyers with similar use cases:

• The pricing they receive for the supply of a single index from the same provider differs by an average dispersion of 21%. The overall range from the lowest price to the highest price can be as high as 219% (some institutions are paying more than twice as much as peers).
• For reporting licenses, the average dispersion in pricing can be up to 37%. The range from lowest to highest can be as high as 472% (some institutions are paying almost five times more than their peer group).
• Average spend on index products per provider can be broken down into 44% spent on licenses and 56% spent on the underlying benchmarks—and licenses are where the greatest price variability occurs for similar use cases.

Note: The data was compiled using our Market Data Spend Analytics Service.

Even the regulators are taking notice

Transparency and competition aren’t just front-of-mind among the consumers of market data. Regulators, responding to concerns (and anger!) from these consumers are now getting involved. For example, Commissioner Hester Peirce of the Securities and Exchange Commission said recently that she was “open to exploring the need for and propriety” of the current framework specifically for index providers in the US.

In the UK, the Financial Conduct Authority has announced a number of market studies and a call for input on charges, competition and transparency in wholesale trading data, credit ratings, and index products.

In January, Sheldon Mills, who is executive director of consumers and competition at the FCA, said: ”Access to wholesale data is really important for those who want to make investment decisions. Without it, they lack the information they need to make properly informed choices. Our call for input and planned market studies are intended to ensure that competition is working well, that information is available to market participants that want it, and that innovation is keeping up with market developments.”

Specifically, in the index business, the FCA will look into concerns that complex contracts for benchmarks and index products prevent switching to cheaper, better quality or more innovative alternative providers.

We have reached a tipping point

These inconsistencies continue across rating products, pricing and reference data, research/analytics and environmental, social and governance data. From a budgeting perspective, the next piece in the puzzle is to track adoption rates per provider and per product across the buying market. Firms are now managing to switch in certain areas to cheaper, equally good or even better alternatives and we will track where certain providers are hitting tipping points in terms of adoption and market share.

The power dynamics of market data are entrenched and we suspect that without regulation, consumers will still struggle to control costs. We can’t change that, but it’s clear that this market is maturing and providers are changing their behavior gradually. Incumbent providers have ensured dominance in certain key areas, but they also acknowledge that transparency can only be a good thing for the industry.

What our data shows is that even with limited leverage and alternatives, the firms that negotiate well from a position of knowledge can significantly drive down cost and increase efficiency. With the right tools in place, our market data friend will perhaps be going on his own party weekends when the management sees what can be achieved with this newfound transparency!