British mathematician Clive Humby once said that data is the new oil. Increasingly, the use of refined data, like refined petroleum products, drives the global economy and shapes business competition. And the past year, sophisticated techniques for applying data have become increasingly central to Trafigura’s business.

       The firm’s Data Science and Engineering Team (DSE), established in 2020, is the nerve centre for a concerted effort to find, sift, combine and analize a complex multiplicity of data sources in support of trading decisions across the company.

       This is achieved through data science – the systematic analysis of data within a scientific framework. Data science combines statistical analysis with elements of computer science (such as  machine learning), and domain -specific business expertise. As with crude oil or any raw material, data needs to be transformed into useful products. Data science allows us to do just that – extract actionable knowledge from large amounts of data to answer complex questions.

       Within Trafigura, the increased use of large data sets in gas trading, and planned entry into power trading were among the key drivers that gave rise to our data science. The gas and power trading desks require intense data analytics since prices move in response to a complex combination of factors including the weather, the seasons, the daily or weekly cycle of human activity, regulations, and the state of the economy. So, the data science team was first involved in producing machine learning models to forecast supply, demand, and price in these markets. Today in oil, metals, bulk minerals, natural gas we are analysing hundreds of aspects of the market and supply chains to build predictive analytics.

        This data analytics capability has since benefitted all other trading teams. As you will read below, sophisticated data analysis is helping traders across the business to spot potential market moving behaviour or unusual conditions that may affect prices.

        Data science is all about sharpening the firm’s existing competitive edge. This is increasingly important in a complex world where unpredictable phenomena such as climate change and evolving regulations are fundamentally changing market dynamics.

        The ability to forecast electricity prices as day or a week ahead, taking all the available drivers into account, is a task beyond the capacity of any human being. But it’s an ideal application for data science, in which a wide variety of data feeds are standardised and plugged into a centralised “data lake”, and machine learning – computer algorithms that can detect data patterns and use them to make predictions – products informed price forecasts.

        Machine learning is really powerful when you have lots of data sources which are too complex for humans to assess by conventional means such as using Excel spreadsheets, and have where outcomes have a certain predictability.  By centralising and integrating historical and current data and applying machine-learning models, we can predict electricity prices to a much higher degree of accuracy.

          Getting there involved assessing and assembling a bewildering array of databases and feeds and writing software to integrate the data into machine learning models. This is a new paradigm and it’s quite different from traditional software development. With traditional technology, the key inputs are data and business rules.

          Machine learning users historical examples to derive the underlying business rules directly from the data. Key to this is is having a data science and engineering team that can build its software, and work closely with traders and research analysts to give them an informational edge. 

          Another crucial ingredient of the team’s success so far is focus. Not every problem can be solved via machine learning. Those where data sources are unavailable or outcomes are fundamentally unpredictable are not worth bothering with. Equally, not every machine learning process creates value – company is only interested in those that can help the firm to improve its performance and make money. We have to innovate but we have to do it in a way that amplifies what the firm already does wll, and that depends on being close to the traders. 

        This is the case in the data science team’s work for non-ferrous metals traders. Firm understanding of metals supply and demand was already unrivalled, but data science is making it better still – for example by crunching a lot of granular industrial data from China (the world’s largest metals consumer), and using machine learning to find relevant market signals. Used well, machine learning can helpfully challenge and validate the assumptions of existing fundamental research.

         Another priority area is developing systematic trading strategies for derivatives traders, by adding measures of market sentiment to augment traditional quantitive strategies. The team is also using Natural Language Processing technology to monitor social media, using specific key words, to detect indicator of market shifts.

        Data science can also help provide answers in specific areas where unexpected events combine to produce outcomes that are hard to predict, such as the recent power shortages in China, which were a result of increased consumption, low rainfall impacting hydroelectric power production, and tight coal availability.  There too, machine learning from historical data of rainfall, river flow and power output could be combined to produce medium-range forecasts on the impact on a wide range of the commodities company trades, from coal, gas and refines oil products to metals.