Seeing into the Singularity
Seeing into the Singularity
Predicting the future is simple.
The formula is: [pattern recognition x sequence modeling]
If you recognize the patterns playing out and you are capable of predicting the next 1, 2, 100+ patterns to appear in that sequence… you are seeing into the future.
Of course, we must remember another key truth: simple rarely equals easy.
Data scientists across the world are working to build models capable of recognizing novel patterns and then generating accurate predictions about the sequence.
I’ve just begun working on my entry for the ARC Price 2024 - a Nerd Olympics playing out with hundreds of teams, thousands of entrants and a massive goal: develop AI systems to efficiently learn new skills and solve open-ended problems, rather than depend exclusively on AI systems trained with extensive datasets.
They are crowd-sourcing solutions to help AI systems gain the ability to solve problems they haven’t encountered before.
We’re upgrading AI from a rememberer to a reasoner.
These are the details from the competition summary page:
Current AI systems can not generalize to new problems outside their training data, despite extensive training on large datasets. LLMs have brought AI to the mainstream for a large selection of known tasks. However, progress towards Artificial General Intelligence (AGI) has stalled. Improvements in AGI could enable AI systems that think and invent alongside humans.
The Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI) benchmark measures an AI system's ability to efficiently learn new skills. Humans easily score 85% in ARC, whereas the best AI systems only score 34%. The ARC Prize competition encourages researchers to explore ideas beyond LLMs, which depend heavily on large datasets and struggle with novel problems.
As we close that massive gap between the human score of 85% and the AI benchmark of 34%, our systems will gain texture and depth in their responses. They will be able to adapt to unpredictable situations with relative ease. This helps bring better AI into robotics, missile defense, logistical routing and a growing-to-near-infinite number of other industries, methodologies and practices.
This capability will also greatly enhance pattern recognition and sequence modeling. This won’t just make AI smarter, it will help it climb the capability ladder at rising rates.
How does this help us See into the Singularity?
Because predicting the future is already difficult, and as we accelerate into the Great Renaissance awaiting us inside the Singularity the forces that cloud our crystal ball will only grow.
These are the forces I mean:
Complexity: The world is incredibly complex, and many events have unpredictable elements.
Data Limitations: We don't always have access to all the relevant data, and the data we have may be noisy or incomplete.
Changing Dynamics: Patterns in the past may not always hold in the future, especially in rapidly changing systems.
The last is called non-stationarity and its a real bummer. Potentially the biggest of them all because machines are quite good with complexity while the data bottleneck is being addressed by synthetic data factories and other augmentation.
We’ll need to upgrade our crystal ball to see ANYTHING in the Singularity, much less a probable future.
But we’ll get to that.
Our soon-to-be-acquired reasoning capability will help our future AI systems to detect changes in these dynamics in real-time (actually it will have to predict them ahead of time) and feed those predictions into its model to gyroscopically correct for changes… everywhere.
Let’s start building by understanding the current crystal balls a bit better.
Building The Only Working Crystal Ball
At its heart, predicting the future is about identifying patterns in the past. These patterns can be simple (like the daily sunrise and sunset) or incredibly complex (like trends in the stock market or the spread of a disease).
Pattern recognition involves analyzing vast amounts of data to uncover repeating structures, correlations, and anomalies. In machine learning, this is often done using algorithms like neural networks that are designed to find hidden patterns.
Meteorologists use patterns in historical weather data, atmospheric conditions, and satellite imagery to predict future weather patterns.
Financial analysts look for patterns in market data to predict stock prices, interest rates, and economic trends.
Patterns are useless unless you can project them forward.
Many events in the world unfold in sequences. Sequence modeling is about understanding the order and dependencies within these sequences to predict what will happen next.
Sequence modeling techniques (like Recurrent Neural Networks or Transformers) analyze how events follow one another. They learn the underlying structure of sequences and use this knowledge to generate predictions.
Fusing Pattern Recognition and Sequence Modeling
The most powerful predictive models often combine both pattern recognition and sequence modeling. They identify underlying patterns in the data and then model the sequential relationships between events to make predictions that are more accurate and nuanced.
Here’s an open secret: people have been predicting the stock market with machines and making fortunes for decades.
Renaissance Technologies (RenTec) is an excellent example of a firm that has mastered the art of combining pattern recognition, sequence modeling, and computational power to achieve extraordinary results in the financial markets.
RenTec's Medallion Fund, which is exclusively for employees, has achieved legendary returns over the years. They earned a massive return in 2020, over 70%. In fact, in the span from 1989 through 2005 they only lost money once.
A single year of losses.
With decades of massive returns compounding around it.