Seeing is not understanding.
Why context is the missing layer in autonomous systems
Mohan Kumar
4 years at Spatia
Author's Perspective
One of the first engineering problems I worked on seemed straightforward establish raw socket communication with a Universal Robot. The challenge wasn't moving information from one place to another. The challenge was understanding exactly how that information should be interpreted. A small mistake in packing or unpacking data could change everything. The information was there. The understanding wasn't.
Observation captures moments. Understanding connects those moments together. The physical world isn't a collection of isolated frames it's continuous.
I've seen versions of this same problem throughout my career. Information by itself rarely tells the full story. Context does. That's true for people. It's true for software. And increasingly, it's true for autonomous systems. A camera can see a room the less obvious question is whether it understands what it's looking at.
Imagine walking into a warehouse. You immediately recognize pathways, equipment, people, and obstacles. You understand which areas are accessible, which routes are blocked, and which objects matter. You don't consciously calculate any of this it simply happens. Now imagine taking a photograph of that same warehouse. The image contains all the information. But the photograph doesn't understand any of it. That's the difference between seeing and understanding.
Divide & Conquer
Break large systems into smaller, verifiable problems. Composability is the foundation of reliable physical AI.
Iterate Continuously
Real systems evolve every day. Ship fast, observe reality, refine the model. The loop never ends.
Understand Reality
Build for the real world, not the benchmark. The physical environment is the test suite that matters.
“The future of autonomy won't be defined by systems that collect the most information. It will be defined by systems that understand the information they already have.”
Spatial intelligence deployed in real industrial environments.
Machines today are incredibly good at collecting information. Cameras capture images. Sensors capture measurements. Systems process enormous amounts of data every second. The challenge is rarely the lack of information. The challenge is making sense of it. What is this object? Why does it matter? How does it relate to everything around it? Has something changed? Should the machine respond differently because of that change? Those questions require context. And context is where things become difficult.
A detection might tell a machine that something exists. Understanding helps determine whether that thing matters. A camera might observe a person entering a space. Understanding helps determine how that changes the machine's behavior. Observation captures moments understanding connects those moments together. The physical world isn't a collection of isolated frames. It's continuous. Objects move. People move. Environments change. The machine has to maintain context across time, understand relationships, and understand what remains true even when the environment evolves.
I've always found that the interesting part begins when issues arise. That's usually when assumptions get challenged. That's usually when deeper understanding becomes necessary. The future of autonomy won't be defined by systems that collect the most information. It will be defined by systems that understand the information they already have. Because seeing is only the first step. Understanding comes next.
Share this article
Mohan Kumar
LinkedInPrincipal Software Engineer · 4 years at Spatia
Mohan Kumar is a Principal Software Engineer at Spatia, where he leads the development of the spatial reasoning and perception stack. He works at the intersection of computer vision, probabilistic inference, and real-time systems.
More engineering
perspectives.
Thoughtful essays from the people building intelligent machines. Delivered monthly, no fluff.
No spam. Unsubscribe any time.
Ready to build
Physical AI?
We're looking for engineers, researchers, and builders who want to work on some of the hardest problems in autonomous systems.
Explore Opportunities