Self-driving golf cart
Rebuilding a 2018 GEM e4 into a self-driving vehicle from the control stack up. With FAU.
Robotics · Autonomy
ROS
Role
AI & Robotics Researcher
Timeline
Ongoing
team
With FAU
platform
GEM e4 (2018) · ROS · custom control stack
The Real Problem
A 2018 GEM e4 ships as a street-legal electric cart, not a robot. Turning it into a self-driving vehicle means owning the whole control stack — wiring, firmware, ROS nodes, and perception — down to the level where software can command steering and braking directly. There is no shortcut through simulation for that: a real vehicle only moves when every layer, from low-level actuation up to navigation, actually agrees.
And the hard part is not the perception demos everyone pictures. It is steering and braking integration — getting real hardware to obey a command reliably, every time. A stock cart has no clean interface for that; its mechanisms were built for a human, not a computer, so the first job is forcing physical actuators to behave before any autonomy is worth attempting.
What the hardware kept teaching me was blunt:
'A command that works on the bench does not always move the actual wheels.'
'Firmware and the ROS nodes have to agree, or the cart just sits there.'
'Nothing about steering and braking is solved until it is solved on this specific vehicle.'
The underlying issue was not any single component—it was that a self-driving vehicle only exists once every layer, from wiring up to navigation, is integrated into one working stack. You cannot buy that integration; you have to own it.
Finding the Fix
I started at the bottom of the stack—tracing how a command travels from a ROS node, through firmware, to the actuators that actually move the cart. Working up from the wiring, three problems stood out:
No clean actuation interface. The stock steering and braking were never built to take commands from a computer.
No safety net. On a real vehicle a bad command does not throw an error—it moves two thousand pounds of cart.
A gap between sim and reality. Everything that looked solved in simulation still had to be proven on the actual hardware.
The long-term goal is a cart that perceives and navigates real environments on its own. But that goal only means anything if the layers underneath it—steering, braking, the control stack—are reliable enough to trust with real motion.
So I focused on three fronts:
Getting steering and braking under direct, repeatable control—the unglamorous integration that everything else depends on.
Building the control stack in ROS so commands flow predictably from software down to hardware.
Adding perception only once the vehicle could be moved and stopped reliably.
All grounded in what the real hardware would actually allow, not what looked easy in simulation.
What Actually Happened
The plan met the hardware, and that is where I caught my mistakes. Getting real actuators to behave is nothing like commanding them in sim—the cart pushed back in ways no plan on paper anticipated.
For example: steering and braking looked like the same 'send a command, move a mechanism' problem, so I expected to solve them the same way. On the real cart they behaved completely differently—braking has to be safe by default, steering has to hold a precise angle—and treating them separately was the only thing that worked.
The firmware layer evolved the same way. My first pass assumed a command sent was a command executed. The hardware disagreed—timing and feedback mattered far more than I had planned for, and closing that loop between ROS and the actuators was the difference between a cart that twitched and one that moved predictably.
I brought it up one layer at a time—proving actuation on the bench, then on the cart, before trusting any of it with real motion. Every layer that held became the foundation for the next.
What Changed
The payoff is not a number yet—it is a stock cart becoming a platform I control end to end. Owning the wiring, firmware, ROS nodes, and perception means every problem is one I can actually reach and fix, instead of one hidden behind someone else's black box.
What owning the full stack actually gave me
'Every layer is now something I can open up and change.'
'Problems show up as real hardware behavior, not as mysteries.'
'The cart does exactly what the stack tells it to — no more, no less.'
The real measure is simple: nothing on this cart is a black box to me anymore.
What I Had to Work With
Real hardware, not a simulator. A physical 2018 cart drifts, wears, and misbehaves in ways no simulation reproduces. Everything has to be proven on the actual vehicle, with all the mess that implies. Sim is useful, but it never settles the question.
A vehicle built for a human, not a computer. The GEM e4's steering and braking were designed for hands and feet, so the control stack has to work around mechanisms that were never meant to take commands from software.
Sim-only autonomy, for now. The higher-level perception and navigation work happens in simulation, but none of it counts until it survives contact with the real cart.
These constraints shaped everything. Instead of chasing the flashy autonomy demo, I focused on the layers that everything else depends on—getting real steering and braking to behave first.
What I'd Do Differently
I would get onto the real hardware even earlier. Reasoning about the cart on paper and in sim was useful, but I missed things that only show up when actual actuators fight back. Even a little more early bench time would have sped up everything after it.
I would also instrument the control stack more rigorously from the start. I leaned on watching the cart's behavior to diagnose problems, which worked—but proper logging from the wiring up through ROS would have made every failure far faster to trace.
What I Learned
Owning the full stack beats abstracting it away. You learn things from wiring, firmware, ROS nodes, and perception—from owning every layer—that you simply cannot learn from simulation. The unglamorous integration work is where the real understanding lives.
Real hardware overrules the plan. What made sense in software—a command sent is a command done—rarely matched how the physical cart actually behaved. Validating every assumption against the real vehicle saved me from trusting things that were only ever true in sim.
The boring parts are the hard parts. Perception and navigation get the attention, but steering and braking integration—getting real hardware to behave reliably—is what actually makes an autonomous vehicle possible. Solve the unglamorous layer and everything above it finally has something to stand on.
You learn things from owning the full stack — wiring, firmware, ROS nodes, perception — that you can't learn from sim. Goal: a cart that autonomously perceives and navigates real environments.
