Ever since the Michael Lewis book Flash Boys, high-frequency trading (HFT) has held a notorious place in the public imagination. But whether you believe that HFT is crucial for liquidity or a major threat to financial stability, it now represents just over half of all trading volume in the US equity markets, and 24% to 43% in Europe’s.

That’s why the range of practices and strategies caught by the definition of HFT are of increasing interest to academics, who look at HFT market microstructure—the complex interactions of price discovery, trading behavior, and venue structure—to help policymakers inform their thinking on its risks and benefits, and design more efficient markets.

One academic project led by the London School of Economics (LSE) has been focused on building a transatlantic network of academics and data to better analyze the impact of HFT trading activities. The LSE announced the project, which has come to be known as Atlantis, in 2017. It has since brought in academics from institutions in the UK, Germany, France, Finland, and the US, who over the last four years have pooled data sourced from exchanges and produced research papers.

The project has now closed, and the participating researchers are in the process of assembling the research outcomes from the past four years to make them available on the Atlantis website. Their code will also be available to anyone who would like to replicate their results or build on the findings.

Khaladdin Rzayev is a lecturer in finance at the University of Edinburgh and Koc University and research associate at the LSE. He joined the Atlantis project in 2019 as a post-doctoral researcher, and has since produced a number of papers on various HFT-related topics. He says the overarching aim of the initiative was to answer nothing less than the question: “What is the impact of HFT on the functioning and stability of the financial markets?

“We know that in the last 10 or 15 years, more than half of trades have come to be executed by these traders. They are super-fast. They have unbelievable information processing ability. They have unbelievable financial power. The natural question is, ‘Do they screw up markets or make them more efficient and liquid?’” he says.

Building limit order book

The findings are not cut and dried, but first it’s important to understand the process the researchers took to produce their findings.

Rzayev says an important outcome of the project has been the transaction-level and limit order book data assembled by the researchers, aided by assistants and data scientists.

“HFT is a critical and challenging topic. We have many questions, but it’s not easy to get data to test them. Financial markets are not always happy to share their data, and even when they are, it’s expensive. And we want to make this data available to everyone,” he says.

Atlantis leveraged its network of participating researchers and the various databases to which they had access to create this database. Most important to the project was Bedofih, an HFT transaction database run by an academic institution called Eurofidai, whose mission is to develop financial databases of stocks, mutual funds, and corporate events for use by academic researchers.

Bedofih is the only academic database of its kind in Europe, and covers data on stocks, exchange-traded funds, and other instruments from the London Stock Exchange, Bats and Chi-X, Deutsche Börse’s Xetra, Euronext, and Eurex. The data that goes into Bedofih is cleaned and normalized by Eurofidai, making it a high-quality dataset to work on, Rzayev says.

“We don’t have issues with limits or missing values in the data because it’s verified by Eurofidai. It’s unique data, clean, and very nice data. The main challenge is building the limit order book itself from this huge dataset,” he says.

Limit order books give an insight into the dynamics of trading venues, so they’re interesting for academics studying microstructure, as well as to exchanges and market participants. Building them up from Atlantis’ massive, transaction-level dataset required a lot of processing power, time and energy from research assistants and data scientists at the participating universities.

“Working with data requires extensive computing power. So we spent a significant proportion of our funding to get this,” Rzayev says. 

Besides Bedofih, Atlantis also received market data straight from various exchanges at a hefty discount. Nasdaq supplied Rzayev and his colleagues with data for free on the condition that they signed a non-disclosure agreement, since this data can contain sensitive information on counterparties and transactions.

Rzayev says, however, that in most of the data from exchanges, participants are not identifiable, and the researchers couldn’t link trading activity to specific firms. This raises the question: How do they know, then, that it’s HFTs making the transactions used to build up the limit order book? Rzayev says that first, the exchanges flag which transactions they understand to be those of high-frequency traders. After that, the researchers can use proxies to make this determination themselves.

One such method is to divide the number of messages sent by a market participant by the number of transactions it makes, Rzayev says. “For some markets, we have an HFT flag in the data. It allows us to link a specific trading activity to HFTs. Without the HFT flag, we need to use various proxies to measure HFT activity.” 

One widely accepted proxy is the ratio of messages to transactions. “HFTs continuously submit and cancel limit orders to monitor the markets. While they submit a huge number of messages, they make only a few transactions. So when there is more HFT trading activity, the message to transaction ratio increases. That is one of the proxies that we use in academia.”

Keeping it in the dark

Using this data, the researchers have tried to answer questions such as: What was the market impact of then-US President Donald Trump’s tweets? How is options market-making affected by HFT activity in equities? What is HFT’s role in flash crashes? 

Rzayev himself, along with a colleague at the University of Edinburgh, Professor Gbenga Ibikunle, produced a paper on dark pool trading dynamics in the wake of the massive volatility of March 2020. The pair had read a Financial Times article from April of last year, which reported that vast amounts of share trading normally done through banks or dark pools shifted back to lit exchanges, as the market turbulence caused by the Covid-19 outbreak tipped the scales on which traders balance anonymity and liquidity toward more transparent markets. They wanted to see if they could find some empirical basis for this phenomenon.

Ibikunle and Rzayev looked at samples of 110 stocks to examine how the volatility affected traders’ venue selection. They concluded that indeed, during that period last year, dark pools experienced a sharp loss of market share to lit exchanges, driven by lit market volatility and a search for immediacy by traders coming from dark markets. Their calculations show that in the most volatile trading period beginning February 24, 2020, the market share of dark pools declined from 2.5% to 2.1% by March 24, 2020.

Where traders choose to buy and sell is important to market quality considerations like price discovery, Rzayev says, and also for how regulators should think about their interventions in HFT and dark pools. The Markets in Financial Instruments Directive (Mifid) imposes a double volume cap of 8% on dark trading in Europe, for example; this kind of restriction—designed for normal trading conditions—becomes irrelevant when markets are impacted by major outside events like a global pandemic.

In another paper, Rzayev and two other co-authors used data from Nasdaq to find that “HFT activity in the stock market increases market-making costs in the options markets via two channels: the hedging channel and the arbitrage channel.” This is an unusual focus for HFT research, he says, which has tended not to analyze HFT outside of the markets it is directly trading in.

“The large and growing literature investigating the implications of HFT has predominantly focused on the within-market quality effects in equity markets,” Rzayev says. But in our paper under the Atlantis project, we showed that it matters for options market-making as well.”

Research also needs to focus on the cross-market impact of HFTs. “They don’t only have an impact on the market in which they are trading, they have an impact on all linked markets,” he says.

The Atlantis project is wrapping up, though it’s closure has been delayed by the hybrid teaching models to which the academics have had to adapt since March 2020. The question that Rzayev posed earlier still stands: “What is HFT’s impact?”

Rzayev says that on balance, HFT’s impact is complex. It’s necessary for liquidity and efficient pricing, but also for reasons that are considered less often: He says it’s not just capital markets participants that need that pricing efficiency, but also the directors of large corporations who want to make investment decisions or plan dividend payouts. However, he adds, it’s fair to say that HFT does present systemic risk implications, not least because of its possible pro-cyclical effects—amplifying volatility in unstable markets.

“We can’t simply say that HFT is beneficial or not: it has positive and negative sides, and we need to find the best level,” Rzayev says.