I was doom-scrolling at like 11pm when I first saw it. The Strait of Hormuz effectively closed. Oil prices didn't gradually rise, they detonated. Brent crude went from $81 a barrel to nearly $128 in weeks. I started thinking about the models. Because somewhere out there, hundreds of forecasting models had just been handed a future they had never seen before. And most of them had absolutely no idea.
🔍 The Assumption Nobody Talks About
Every forecasting model makes one quiet assumption: The future will behave like the past. This is called stationarity. And it’s the invisible foundation that almost every time series model is built on.
On February 28th, that assumption stopped being true. We hit a structural break, the most dangerous thing that can happen to a model running in production.
.png)
The model stayed confident (flat line) while reality left it behind.
📊 What the Numbers Actually Showed
I pulled 578 trading days of real Brent crude data and trained an ARIMA model on everything up to the crisis. When I let it forecast forward, it flatlined around $70, very sure of itself. Reality went to $118.
The "Detective Work" Metrics:
I ran a Chow Test to ask if the data actually changed. The F-statistic came back at 1031. For context, 4 or 5 is usually significant. 1031 means the model was basically looking at a different planet.
🛠️ How to Catch a Silent Failure
A model that crashes is easy to fix. But a model that stays "green" on the dashboard while producing wrong predictions? That's a nightmare. Here’s how I catch it:
1. CUSUM Monitoring
I track the running total of forecast errors. When the cumulative error climbs away from zero, you get an alert. My first alert triggered just five trading days after the crisis started.
2. PELT Algorithm
I fed the data to the PELT algorithm with zero context. It found the break on February 27th. No prior knowledge, just pattern recognition.
3. Real-Time Adaptation
I retrained the model on just the 35 post-break days. The AIC dropped from 1821 to 224. Same code, just a different slice of reality.
🚀 The Final Takeaway
The original model wasn’t bad; it was just loyal to a world that no longer existed. Structural breaks happen everywhere, from shipping lanes to consumer trends. The models that survive aren't the ones with the best architecture. They're the ones built by people who assumed they’d eventually be wrong, and built systems to notice.