BMLL Now Has 5 Years of Order Book Data Available

BMLL Technologies now has five years of Level 3 order book data available for US customers and for European funds trading US stocks. The data engineering-as-a-service company ingests the data from trading venues, then harmonizes it to create datasets to help data scientists and quants derive insights for alpha generation.

“As of this month, we have five years of Level 3 data—that is the full order book with every individual order that has been sent to trading venues—on the US equity markets. Many of our clients have told us that five years of history is imperative for them to be able to do the backtesting they require, at a level of depth and granularity they require,” says BMLL CEO Paul Humphrey.

BMLL has a back-end system that ingests end-of-day files directly from multiple trading venues. The company then performs proprietary normalizations on the data, which is stored in its data lake that runs on Amazon Web Services. Quants can access the harmonized datasets via BMLL’s Data Lab platform, either in the platform’s Python environment, or by taking in the data through an API or FTP into their own research systems.

Because Level 3 is every single order on a limit order book (as opposed to Level 2, which is orders aggregated at a particular price level), a researcher can see not only the price level at what orders are being submitted, but also individual orders. Elliot Banks, chief product officer at BMLL, says this allows the user to trace orders and look at metrics such as the average resting time of an order or fill probability at a particular price level of the order book.

“Quants and data scientists want this data for backtesting, to look at historical performance, and to help them in decision-making. They feed it into smart order routers, or use it to derive different features and datasets to help in their alpha discovery process. But they need clean data. So we capture it, and we make it consistent,” Banks says.

These datasets are large and complex, and every venue records data in different ways, following different standards and formats. Many firms don’t have the computing infrastructure to derive meaningful analytics from the data, Banks says. BMLL aims to free up time for quants by taking on the technological lift of normalizing the data, so they can focus on their value-add activities, he adds.  

“We are taking Level 2 and 3 data, and turning it into datasets that quants can use in the alpha discovery process. They need enough history to be able to tally that with some of their other datasets,” Banks says.

Five years is a milestone in the collection of the Level 3 data because it’s necessary to have a sufficiently long time horizon to capture a wide spectrum of market scenarios, says BMLL COO William L’Heveder.

“When you are looking at microsecond granularity, in order to get a full cycle of trading events that are relevant to you, you probably don’t need that much history. When you start thinking about consolidating that microsecond granularity into intraday summaries or end-of-day summaries, you need to capture a longer time horizon in order to capture as many events cycles as possible,” L’Heveder says.