How Do You Overengineer Outside Your Field? Lessons from Building a VR AI Office
For years, my VR headset was for escaping reality in Flight Simulator.
But as my work on the CerAImic Nerve Center expanded, reading text or even talking with my AI agents felt outdated. I wanted to walk up to my AI agents, talk face-to-face, and watch them collaborate in a virtual office. Eight months ago I picked up a Unity course on Udemy. I'm no game dev, and bridging local AI with 3D rendering was chaotic from day one.
To connect Unity to my backend, I refactored the Nerve Center with event hooks for voice, state, and spatial data. I don't have real avatars yet. Just ugly 3D blocks sliding across a basic floor.
I spent months fighting spatial cognition, making agents "see" me. I was overengineering 3D perception with depth fields and raycasts.
The breakthrough came when I realized the agents didn't need to know they were in a 3D environment to begin with.
Using a flat map of the office let them turn, follow, or face each other without the spatial overhead. I hope this fix doesn't come back to bite me later.
I still have a lot to learn. How do I turn those ugly blocks into human-like avatars? How do I make them sound and act realistic? Should I keep my current speech engine, Kokoro? I sometimes feel over my head, but the vision keeps me going.
The attached image is my vision for the final VR office. Every hurdle forces me to learn Unity deeper, building a foundation to accelerate progress. Who knows. Maybe I'll eventually package and sell it.
AI engineers and developers: have you ever worked on an integration project that felt this messy?