Why Subtraction Might Be the Best Data Science Algorithm in Your Bag
This post is part of a series on how to build a data science team. The first was about forming a team with human skills, not technical skills. The second was about the importance of avoiding data science dancing bears.
In the 1990s, traders on Wall Street traded over the phone. (With cords.) By the early 2000s, computers could make trades. The press declared human traders would soon be extinct. I hated those headlines—as a developer I viewed computers as a tool, not a rival. Today, 80% of trades are algorithmic, and humans are more important than ever. (1)
Many firms on Main Street are just beginning to install data science teams. What lessons can they learn from Wall Street? One is: the tech might be simpler than you think.
* * *
At the dawn of automated trading, some data scientists discovered spread trading. It’s a trading strategy based on the insight that the price of related securities usually moves in lock-step: when IBM goes up, HP goes up. When IBM goes down, HP goes down. But when HP goes up, and IBM doesn't, it’s a sign something’s up.
Traders watched spreads with their eyes like this:
The best traders could track 6-8 pairs at once (indeed, banks measured this).
Visionary technologists set out to help traders. They did the math in real-time and showed a new kind of graph, like this. Normal pairs are on top; something might be going on with HP and IBM. The sudden change in the spread is a signal to act.
You learned the math required for spread trading in first grade. The algorithm is subtraction:
Spread = Price (HP) - Price (IBM)
Spread trading helped revolutionize Wall Street. Obviously, the math wasn’t magical. The breakthrough was that computers could calculate millions of spreads in real-time and show the strongest signals to traders. Sector analysts could monitor a whole industry sector. Portfolio managers could find needles in the haystack in real-time while the needles flew by.
The power of this algorithm was that it empowered human creativity, as one friend of mine said it:
Lots of banks missed spread trading. They overcooked their data science. I worked with one bank that went from 6th to 1st in FX trading thanks to a simple algorithm.
If you’re building a data science team, the lesson of spread trading is that the art of data science isn’t complex math. Innovation stemmed from human creativity, listening, and ignoring the ego. And avoiding data science dancing bears (brilliant but pointless work).
You might find that the simplest of tools, something like subtraction, might be the most valuable data science tool in your bag.
ENDNOTES
(1) Equities, or stocks, is the most heavily automated of the asset classes. Estimates of the percentage of automation vary by asset class from Tabb Group and Sang Lee at AITE Group. The percentages used in this article refer to equities trading.
For more, read these about data science culture:
