Are we living in a foveated reality?
Video games trick players by only rendering in high detail what's being observed. So do spatial computing headsets. Even our eyes and brain do it. Maybe the universe does as well ...
Does reality exist beyond what we can see and experience in the moment? Or do we inhabit a universe that is profoundly good at creating the illusion of reality that matches what we’re currently experiencing?
It’s a weird idea and, truth be told, just a little fringe. But it’s also one that’s deeply intriguing — especially as we’re entering holiday season where it’s always good to have something that makes people stop and think to throw into the conversation!
It’s also an idea that, bizarrely, loosely connects speculations that we may be living in a simulation with some of the more counterintuitive aspects of quantum physics.
The connection here is the concept of foveation — and more specifically, the question of whether we are living in a foveated reality.
Foveation is what our eyes and brains do to make us feel as if we experiencing reality to the full while conserving “brain compute.” They do this by only rendering what is directly in front of us in high definition, using a very small but highly sensitive part of the retina — the fovea. Because that small patch of “high definition reality” follows our visual focus of attention, we’re fooled into thinking we can see everything around us clearly.
The same technique is used in video games, where only the details that a game player is experiencing on their screen at any one time are rendered in high resolution. And it’s at the heart of how technologies like Apple’s spatial computing headset manage to be so immersive without you having a supercomputer strapped to your head — by only rendering in high definition what’s directly in front of your eyes.
In other words, we know that foveation is a technique that can very effectively fool us into feeling that we’re living in a richly detailed reality that extends far beyond our immediate perception. But practice, it’s only what’s right in front of us at any given time that’s actually being constructed — whether through the interplay of nerve signals from our fovea and our brain, the coordination between eye tracking sensors and processors in a headset, or the on-screen rendering of a vastly larger virtual game space.
And this raises something of a knotty question: if foveation works so well, how do we know that we are not, in fact, all characters in a massive computer simulation where what we think of as reality is, in fact, a foveated reality — one where the universe (or the mega-computer behind it) is generating the reality we perceive on the fly?
A crazy idea I know. And not one, if I’m honest, I buy into. But it is one that is a great catalyst for exploring interesting new ideas and possibilities … or simply throwing into conversations and seeing how people react!
It’s also an idea that I was reminded of quite delightfully a few days ago while in conversation with my good colleague and simulation theory expert Riz Virk on the latest episode of Modem Futura.
Riz has a new edition of his book The Simulation Hypothesis out (highly recommended by the way), and my co-host Sean Leahy and I used this as a chance to catch up with him. (Riz has also appeared on the Joe Rogan Experience and Jordan Harbinger Show, so we were pretty stoked to get some time with him).
As always, the conversation was wide ranging in best possible way (and is perfect holiday listening — you can find links at the end of this newsletter). But this concept of a foveated reality ended up being central to it.
Of course, the idea that we’re all just players in a massive cosmic video game feels deeply improbable. And one rather large problem the idea faces is the sheer compute power it would take to simulate everything in the known universe down to the smallest subatomic particle. From everything we know, this feels preposterous — impossible even — given the mind boggling complexity and vastness of the universe. I’ve even used this argument myself against the simulation hypothesis in my own writing.
But what if the mega supercomputer simulating the universe (and everything in it) didn’t need to simulate everything, everywhere, all at once? What if all it had to do was to construct what any one person was experiencing right in front of them at any given time?
What if, in fact, this universal simulation computer was doing exactly what our eyes and brain, video games, and spatial headsets, already do?
This possibility has the potential to transform the impossible problem of simulated reality into a merely improbable one — albeit one that is still mind-bendingly complex.
It still feels like something of a long shot to think that we’re all living in a massive computer simulation built on the back of foveated algorithmic optimizations. And yet, scientists are constantly chipping away at our conceptions of what reality actually is. And disturbing as this might seem, emerging ideas are bringing the possibility that our perceived reality is an illusion closer than we might think.
This hit home as I was reading a piece by another good colleague just after we spoke with Riz — the physicist and author Paul Davies.
Paul is an exceptionally well known science writer, communicator, and physicist, as well as being co-founder of the Beyond Center at ASU. He’s also someone who’s used to pushing back on established concepts of what we might think of as reality.
I’ve known Paul for some years now, and have always found his work and thinking wonderfully challenging and stimulating.
In this case, what grabbed my attention was an article by him for New Scientist titled “Why quantum mechanics says the past isn’t real.”
Paul was writing about the physicist John Wheeler's delayed choice experiment — a now-verified quantum phenomenon where the type of measurement you make today appears to determine what a particle did in the past, even billions of years ago.
It’s a possibility that has garnered renewed attention with recent breakthroughs, and one that raises intriguing questions about the nature of reality — especially if there is a connection between the act of observing the universe in the present, and how the past is constructed at that point to match the reality you experience.
Which sounds just a little bit like the idea of foveated reality …1
One explanation of Wheeler’s idea of delayed choice is that there are many pasts, and the act of observation ties your particular present to one particular past.
This begins to open up the brain-aching possibility of multiple universes, where everything that could happen has happened in some parallel version of the reality we’re experiencing.
But it also begs the question of whether, just as in video games and spatial reality headsets — and our own brains — the reality we experience is merely a construction that just feels like it’s part of something bigger and more coherent.
In other words, is the past that led up to the present you are experiencing here and now, and the context within which you are experiencing it, simply an on-the-fly simulation created by some cosmic mega computer that has learned the trick of fooling you into believing that this foveated slice of experience is part of a larger coherent reality?
Probably not, but it’s an intriguing idea; especially as it begins to tentatively connect the worlds of quantum science and the simulation hypothesis.
The writer Douglas Adams famously — although admittedly fatuously — wrote that “time is an illusion, lunchtime doubly so.” Maybe he was foreshadowing emerging thinking around time, quantum physics, and the nature of reality. Or maybe he realized that the only way to explain the irreconcilable weirdness of life, the universe, and everything, is to recognize that we’re all inside some massive simulation — and at any one point all we see is our own foveated illusion of reality.2
Either way, whether we’re living in a foveated reality, wrapped up in the illusion of one, or are merely stuck with the real reality as we know it (or think we do), what intrigues me about fanciful excursions like this is that, bizarre as they may seem, they do provide creative jolt to the imagination that helps see and think about things in different ways.
And, of course, they provide the perfect fodder for messing with people’s heads over a long, lazy, holiday lunch 😊
I’ll be taking a break from the Future of Being Human Substack newsletter over the new year break, but until next time, hope you have a restful and enjoyable end of the year, and see you in 2026!
And if you’re interested in listening to or watching the conversation with Riz — and biased as I am, I would highly recommend it for an entertaining and informative holiday listen over the break — you can catch it on Apple Podcasts, Spotify, or YouTube (below).
Happy listening!
I am, of course, pushing the alignment harder than is most likely warranted here, but the juxtaposition of the two ideas does raise some interesting possibilities.
Fans of Adams’ The Hitch Hiker’s Guide to the Galaxy will, of course, realize that this idea is further explored in Zarniwoop’s “universe in an office” and the story that unfolds around it. But that’s a story for another day …