Traffic Sign Recognition From Scratch: Dataset to Deployed Model in One Weekend

Saturday morning optimism, Saturday night driver hell, and a Sunday afternoon demo that actually worked.

I'd been meaning to build a small traffic sign recognition demo for months — not for any project, just to prove to myself I could go from "no dataset" to "live webcam demo with bounding boxes" in a weekend, using nothing but a laptop, a cheap webcam, and whatever embedded board was already sitting in my parts drawer. This is the honest, unglamorous log of how that weekend actually went, bad GPU driver included.

Saturday, 9:40 AM — Deciding This Is Actually Happening

Coffee, a blank folder called tsr-demo, and a decision to keep the scope deliberately small: not "recognize every sign in the Vienna Convention," just five sign classes I could realistically photograph and print myself — stop, speed limit 30, yield, no entry, and pedestrian crossing. Small scope, finishable scope. I've started enough weekend projects that died from scope creep to know this decision matters more than almost anything else I'll do today.

Saturday, 11:15 AM — Building the World's Smallest Dataset

Printed small versions of each sign, taped them to cardboard, and spent about ninety minutes walking around my apartment and a nearby parking lot photographing them from different angles, distances, and lighting — indoors, outdoors, partial occlusion behind a plant, one deliberately blurry set for motion realism. Ended up with about 340 raw images across five classes, which is a laughably small dataset by any real production standard, and exactly enough for a weekend proof-of-concept.

Labeling took the rest of the morning using a free open-source annotation tool, drawing bounding boxes by hand. By 12:30 I had a labeled dataset in YOLO format and the specific kind of hand cramp that comes from clicking "next image" three hundred times.

Saturday, 1:30 PM — First Training Run, First Wall

Kicked off training on YOLOv8n (the smallest variant, deliberately chosen for the embedded deployment step later) using a pretrained checkpoint and my tiny custom dataset. Watched the loss curve for the first twenty epochs with the specific optimism of someone who hasn't yet been burned today.

By epoch 40, it hadn't been burned — it just wasn't learning. Loss was bouncing around without any clear downward trend, and validation mAP was sitting at something embarrassingly close to zero. My first assumption was "340 images is just too few," which is a reasonable assumption and also, it turned out, not actually the problem.

Saturday, 3:00 PM — The Actual Bug (Learning Rate, Not Dataset Size)

After a frustrating hour of second-guessing the dataset, I actually looked at my training config instead of my images, and found it: I'd copied a training config from an unrelated larger project, and it specified a learning rate tuned for a much larger dataset and batch size than what I was running. On a tiny 340-image dataset with a small batch size, that learning rate was too aggressive — the model was essentially thrashing rather than converging, taking steps too large to settle into a useful minimum.

Dropped the learning rate by roughly 10x, restarted training, and watched the loss curve actually behave like a loss curve is supposed to behave — a clean, if slightly noisy given the tiny dataset, downward trend. By 4:30 PM, training had converged to something genuinely usable: not state-of-the-art, but confidently distinguishing my five sign classes in a held-out validation split. Small, real win, and the exact kind of "oh, it was the hyperparameter, not the fundamental approach" moment that makes or breaks a weekend project's morale.

Saturday, 6:00 PM — Stepping Away Before I Break Something Out of Impatience

Trained model in hand, feeling good, I made the correct decision to stop for the day rather than immediately diving into embedded deployment while tired and slightly overconfident. Weekend project rule I've learned the hard way, more than once: the deployment step is exactly where impatience causes the most expensive mistakes.

Sunday, 10:00 AM — TensorRT Conversion, and the GPU Driver That Hated Me

Back at it, this time trying to convert the trained model to TensorRT for faster inference on my target embedded board. This is where the weekend's real fight happened.

Conversion failed immediately with a cryptic CUDA driver mismatch error — the embedded board's installed driver version didn't match what the conversion toolchain expected, a classic and thoroughly unglamorous embedded-AI problem that has nothing to do with machine learning and everything to do with dependency version hell. Spent close to two hours going in circles: updating the driver broke a different dependency, rolling back the driver brought back the original error, and at one point I genuinely considered abandoning the TensorRT step entirely and just running the un-optimized model instead, purely out of exhaustion with the toolchain.

Sunday, 12:30 PM — The Fix Was Boring, Which Is How These Always End

The actual fix, once I stopped trying random driver versions and instead read the toolchain's compatibility matrix carefully: I needed a specific intermediate driver version that supported both the board's other existing dependencies and the TensorRT conversion tool — not the newest driver, not the original one, a specific version in between that I'd skipped over twice already because I assumed "newer is safer." Installed that specific version, reran the conversion, and it completed without incident in about four minutes.

There's a genuinely useful lesson in how anticlimactic this was: the two hours of frustration weren't caused by anything deep or interesting. They were caused by not reading the compatibility documentation carefully the first time, and guessing instead. Embedded AI deployment has a lot of these moments, and the fix is almost always boring once you find it.

Sunday, 1:45 PM — Wiring Up the Live Demo

With a working TensorRT-optimized model on the board, the last piece was the actual demo: a small script pulling frames from a USB webcam, running inference each frame, and drawing bounding boxes with class labels and confidence scores directly onto the video feed in a simple display window. This part, refreshingly, went almost exactly as planned — the kind of straightforward integration work that feels like a reward after a morning of driver archaeology.

Sunday, 2:20 PM — It Actually Works

Held up my printed stop sign in front of the webcam. A green bounding box snapped onto it almost instantly, labeled "stop, 0.94," tracking smoothly as I tilted and moved the sign around. Tried the speed limit sign next — correctly identified, correctly distinguished from the visually similar "no entry" sign, which had specifically worried me during training given how few examples of each I had.

There's a very specific, disproportionate satisfaction in watching a green box lock onto a cardboard sign you taped together yourself less than 48 hours earlier, especially after a Sunday morning that had, for a while, felt like it might end in giving up on the embedded step entirely.

What This Weekend Actually Cost, Honestly

Total hands-on time: somewhere around 11-12 hours across the two days, unevenly split — roughly 4 hours on data collection and labeling, 3 hours on training (including the learning-rate detour), and a genuinely disproportionate 4-5 hours on embedded deployment and driver troubleshooting, which is almost always where a weekend AI project's time actually goes if you're honest about logging it, rather than the training step people usually assume will be the hard part.

If I were doing this again — and I probably will, with a slightly larger scope next time — I'd start the embedded toolchain compatibility check on Saturday morning, in parallel with data collection, rather than discovering the driver problem cold on Sunday with less slack time left to solve it. That's really the only thing I'd change. Everything else, including the frustrating parts, felt like the normal, honest cost of building something real in a weekend instead of just reading about how to build it.