Which is more important: ending dependency on fossil fuels or preventing energy poverty? There is no right answer to this question; the ways we think about social issues or saving the planet are inherently personal. It’s this idiosyncrasy that makes evaluating data for ESG investing a complex beast for asset managers—how do you estimate what’s good for the world and what’s good for an investor’s balance sheet? And how do you know you can trust the data you’re using, and avoid greenwashing, which involves misrepresenting a company’s environmental impact?

Asset managers buy ESG ratings from providers. These vendors use proprietary methodologies to measure the ESG scores of individual corporates, applying different weights to the environmental, social and governance factors. Because this type of analysis requires so much qualitative decision-making, investment firms want more transparency into how these vendors are sourcing their data and boiling that down to single ESG scores.

“Hundreds of billions of dollars are being allocated explicitly to these types of metrics, and it’s quite shocking to see how little these data providers give you in terms of transparency,” says George Mussalli, chief investment officer and head of equity investments research at PanAgora Asset Management. “They only give you a very broad sense, or top-level [information on] E, S or G, and maybe they’ll give you a couple of characteristics of the types of things they collect. But they don’t want to tell you how they rate these companies.”

Bloomberg Intelligence estimates that ESG assets are set to hit $53 trillion by 2025, a third of global assets under management. A huge portion of that flow relies on data informed by ESG ratings. A study by SquareWell, published in 2021, found that 38 out of 50 of the world’s top asset managers use two or more ESG research and data providers as part of their investment strategies.

“I would be very wary of allocating my clients’ assets according to these third-party choices, but I see it happen over and over. There needs to be more transparency and oversight of this,” Mussali adds.

Global industry bodies and regulators are starting to pay attention to the risks associated with this dependence on ESG rating firms. Standards-setter the International Organization of Securities Commissions (Iosco) published a report on ESG ratings and data products in November 2021 that expressed concerns about the potential risks of depending on the subjective views of a small pool of vendors.

As part of the European Commission’s Sustainable Finance Strategy, the lawmaker has also pledged to improve the reliability, comparability, and transparency of ESG ratings. The European Securities and Markets Authority (Esma) recently launched a call for information on the size and influence of different ESG rating providers.

Lifting the hood

Rating agencies aggregate data from disparate sources, mainly corporate disclosures of various kinds, and condense that information into a single subjective score.

Asset managers want to better understand the process by which these firms come up with that score to avoid risks like greenwashing. They want to know how that data is sourced, which sources are used and why; how they compare with other data points; which peer groups are used to contextualize the data; and how the data is benchmarked.

Nikita Singhal, co-head of sustainable investment and ESG at Lazard Asset Management, says portfolio managers also need to know where the underlying data gaps in the ESG score exist. One of the biggest problems with ESG investing is the lack of data consistency and standards associated with company disclosures.

To make up for these gaps, rating agencies impute what the answer is, basing their estimates on factors such as a company’s peer group or region. For instance, if there is no record of Company A’s carbon emissions, the rating agency might impute that data from Company B, a firm based in the same sector and of similar size. Singhal says ratings agencies and data vendors should be more transparent about where those imputations are made.

“That is where the errors or the discrepancies are created because the imputation of data can be based on certain assumptions that you’re making and if you’re carrying different assumptions you’re going to end up with very different results,” Singhal says.

Singhal says buy-side firms want to be able to drill down into the data to better understand the accurate level of data coverage. For instance, a ratings agency’s ESG dashboard might show that it has 97% coverage of a specific metric, whereas in reality, 50% of that 97% is imputed data.

A spokesperson for Morningstar-owned ESG ratings firm Sustainalytics says clients have access to the company’s methodology and can speak to their client advisors. They say Sustainalytics analysts follow “a structured analytical framework and explicit guidance on how they should evaluate companies.”

“Our goal is to create consistency so that any two analysts looking at the same situation should arrive at a matching outcome,” the spokesperson adds.

A spokesperson for research and analytics provider MSCI says the company regularly reviews its ratings methodology and models to incorporate new information, industry regulation, and technical enhancements.

“We adopt a formal, in-depth quality review process in our analysis, including automated and threshold-based quality checks of data,” the MSCI spokesperson says.

Sphere of influence

Asset managers say that at the heart of the transparency issue lies the fact that the ESG ratings space is effectively dominated by two players: MSCI and Sustainalytics. On a smaller scale, other providers like Bloomberg, ISS ESG and S&P-owned RobecoSAM are also vying for market share.

“It’s still pretty much an oligopoly in this space,” says Lazard’s Singhal.

The problem for asset managers like Singhal, she says, is that the ratings from MSCI and Sustainalytics have poor levels of correlations, as their methodologies and qualitative analysis vary greatly. She says their correlation ranges from 0.3 to 0.5. Contrast that to ratings of creditworthiness, where the two main players, Moody’s and S&P, agree at a range of 0.7 to 0.9.

This is important because investment firms are trying to find reliable and trustworthy signals to inform their ESG decision-making, but this can be perplexing when the two main ESG rating agencies have vastly different views on a company’s ESG output. This is why users want to know how these providers measure ESG and why they differ so much.

Consolidation in the ESG vendor market is not a new phenomenon. Over the last two years, there has been a flurry of merger and acquisition activity among major data vendors looking carve out a piece of the lucrative pie, which will only cement the dominance of incumbents, making it more difficult for new vendors to break into the industry, says Brunno Maradei, global head of responsible investment at Aegon Asset Management.

“The barriers to entry are very high. You have to build a massive database [of company coverage] just to have an introductory meeting with an asset manager. Even if you just cover one indicator, you’ve got to cover 50,000 companies on that one indicator,” Maradei says.

Sebastian Lancetti, head of portfolio strategy at PanAgora, says the rating industry would benefit from a more diverse group of vendors with different perspectives and methodologies to ultimately garner better ESG signals. He says that while rating agencies might not always agree in their final scores, and concedes that having to subscribe to multiple vendors creates additional work for the asset manager, a diversity of views helps generate alpha in the long run. In other words, the more data vendors to choose from, the higher chance of correlations in the data.

“We need to have a plurality of views going forward. And we don’t want to end up in a situation where ESG is defined by one or two major vendors, and everybody’s pushed toward that,” Lancetti says.

To regulate or not to regulate?

The consensus among the four asset managers interviewed for this article is that rating firms should provide more clarity on how their methodologies work. But the jury is out on whether regulating these firms is the answer to these data problems. Those in favor of regulation say the agencies should be subject to rules and oversight because of the influence they have over the market.

Others are less convinced that regulating these firms would work in practice. As each rating agency follows its own methodologies, it is less clear how regulators could regulate them.

“It’s difficult for a regulator to say that a rating is correct or incorrect. Maybe they could regulate the process of coming up with the rating, but that itself is challenging,” says Eric Nietsch, head of ESG Asia at Manulife Investment. “In credit ratings, they require certain processes to be followed and decisions are made by committees rather than just individual rating analysts.”

Another lightweight approach could involve regulators auditing or validating the experts tasked with analyzing the ESG data and making qualitative judgments. Maradei says regulators could enforce a governance framework where rating agencies must ensure that the analysts or vendors making the assessments have the right qualifications.

“You want to make sure that the person has the right ability to make that judgment for you because that’s what I’m paying for. That’s what I would like to see,” he says.

On the extreme end of the spectrum, regulating ratings agencies could introduce a whole host of problems. Over-standardization could discourage other data vendors from entering the market, or risk oversimplifying how ESG is defined.

“We are also wary of too much standardization, if done the wrong way by the regulators, where we would end up with one way of looking at ESG. ESG is a very complex field. It’s not just measuring the credit quality of a company or the ability of a company to pay back its debt—we are trying to capture so many different things,” Lancetti says.

For now, at least, many asset managers believe regulation is best placed to rectify underlying data problems. This includes implementing universal taxonomies for E, S, and G—such as the EU’s Taxonomy for environmentally sustainable activities—and developing international frameworks for disclosing ESG company information. Global organizations currently working on disclosure standards include the Global Reporting Initiative (GRI), the Sustainability Accounting Standards Board (SASB), and the newly formed International Sustainability Standards Board.

“Even more urgent than regulating data providers, we need more clarity from the standard setters and accounting bodies,” Singhal says.