A Week Teaching MentorPi to Follow a Line (And Actually Understanding Why It Works)

Monday to Friday, one road mat, one stubborn ceiling light, and the moment lane_detect.py finally made sense.

MentorPi ships with a working line-follow / lane-keeping demo out of the box — ros2 launch example self_driving.launch.py only_line_follow:=true, point it at a lane, and in theory it just works. This week was the process of finding out how much "in theory" was doing in that sentence, one weekday at a time.

Monday: The Mat, and a Fold Nobody Warned Me About

Rolled out the road mat across a clear patch of office floor for the first time — enough space for a full loop with a couple of curves, which felt appropriately ambitious for a Monday. Launched the demo, opened rqt to watch the camera feed live rather than just trusting the robot's behavior blindly, and set MentorPi down at the start of the yellow lane line.

First run: smooth for about two-thirds of the loop, then a sudden, confident swerve toward the edge of the mat for no reason I could immediately see on the physical track. Checked the rqt image view during a second run, watching frame by frame around where the swerve happened, and found it — a small fold in the mat, barely a few millimeters of raised material where it had been rolled up in shipping, catching the overhead light just enough to throw a thin, curved highlight across the yellow line at exactly the wrong angle. To the camera, for about four frames, that highlight apparently looked enough like a lane discontinuity to nudge the steering calculation off center. Weighted the mat down flat at the corners and the fold flattened out almost completely. Not a robot problem. A "the floor itself was lying about being flat" problem.

Tuesday: Fighting the Ceiling Lights, Back to Lesson 14

Ran the same loop Tuesday morning, same mat, same weighted-down corners — and got noticeably worse tracking than Monday afternoon's clean runs. Confusing at first, until I actually thought about what had changed: nothing about the mat, everything about the light. Monday afternoon had softer, more diffuse light from outside; Tuesday morning's overhead fluorescent lighting was hitting the mat's plain floor-colored background at an angle that apparently pushed its color values closer to the yellow line's range than I'd accounted for.

This sent me back to the kit's own "14. ROS+OpenCV Lesson" material to actually re-derive the color threshold properly instead of guessing at new numbers. Quick explanation for anyone not yet comfortable with this part: color-based line detection works by defining a range of acceptable color values — a lower bound and upper bound — and treating any pixel whose color falls inside that range as "probably part of the line." Get the range too wide, and background floor color starts getting misclassified as line, exactly what was happening Tuesday morning. Get it too narrow, and legitimate line pixels under slightly different lighting get excluded, causing dropouts. I sat with rqt open, nudged the threshold bounds in small steps, and rechecked the live segmented image after each change until the floor cleanly dropped out of the detected mask while the yellow line stayed solidly in it, specifically under this morning's lighting rather than yesterday's. Tedious, but exactly the kind of tedious that actually fixes the problem instead of papering over it.

Wednesday: Actually Reading lane_detect.py Instead of Just Tuning Numbers

After two days of tuning numbers somewhat blindly, I finally sat down and actually read through lane_detect.py properly rather than treating it as a black box with adjustable knobs. Worth walking through what's actually in there, for anyone else at this same stage.

The LaneDetector class doesn't process the whole camera frame as one blob — it splits the image into multiple horizontal and vertical ROIs (Regions of Interest), essentially cropped strips of the frame at different heights. This matters because the lane line looks different — different width, different apparent angle — near the bottom of the frame (close to the robot) versus higher up (further ahead), and processing several separate strips lets the detector estimate the line's position and angle at multiple depths rather than treating the whole frame as one flat problem.

Within each ROI, the frame gets converted from standard RGB into Lab color space rather than staying in RGB or even HSV — Lab separates lightness (the "L" channel) from the actual color information (the "a" and "b" channels) more cleanly than RGB does, which makes color-based thresholding meaningfully more robust to exactly the lighting variation that had been causing my Tuesday morning problem, since brightness changes affect the L channel much more than they affect the a/b color channels a yellow-line threshold actually depends on.

After thresholding each ROI into a binary mask, the code applies erode and dilate operations — morphological cleanup steps that shrink then regrow the detected regions, which has the practical effect of removing small, isolated noise blobs (a few stray misclassified pixels) while preserving the larger, genuinely-connected shape of the actual line. Finally, it finds all contours (connected boundary shapes) in the cleaned-up mask and takes the largest-area contour as the actual line — a deliberately simple, robust choice, since the real lane line is reliably the largest continuous colored region in a well-thresholded frame, and this approach naturally ignores small stray colored artifacts that survived thresholding but aren't big enough to be the real line.

From the largest contour's position across the multiple ROIs, the code computes both a lateral offset (how far the lane's center is from the frame's center, at each ROI height) and an angle (how the line's direction across those ROI heights compares to straight-ahead) — together giving the steering logic both "how far off center am I" and "which way is the line curving" as inputs, rather than just one flat offset number. Reading this end to end took most of the afternoon, and it was worth every minute — Tuesday's tuning-by-feel turned into Wednesday's tuning-with-actual-understanding of what each parameter was doing structurally.

Thursday: Deliberately Doing It Wrong, to See What "Wrong" Actually Looks Like

Documentation says center the robot on the lane at the start. So naturally, Thursday's experiment was deliberately not doing that — starting MentorPi noticeably off-center, closer to one edge of the lane than the middle, just to see what the correction behavior actually looked like under a condition the system wasn't specifically tuned to start from.

The result was genuinely instructive: instead of a smooth, gradual correction back toward center, the robot exhibited a visibly oscillating, slightly wobbly correction pattern — steering one direction, overcorrecting slightly, steering back, settling down only after several seconds of this back-and-forth rather than gliding back to center in one clean arc. Watching this alongside Wednesday's newly-understood ROI/angle logic made the mechanism click: starting significantly off-center presents the detector with a much larger initial lateral offset and a more extreme angle reading than it typically sees during normal operation (where the tuning assumes corrections starting from roughly-centered), and the steering response — reasonably well-tuned for small, typical deviations — was less well-matched to this much larger initial error, producing the oscillation as it repeatedly over-corrected and re-corrected trying to close a bigger gap than it was really tuned for. A good, concrete illustration of why the "start centered" recommendation in the documentation isn't just a suggestion for convenience — it's implicitly assuming the correction tuning's actual operating range.

Friday: One Clean Loop, Curves Included

Back to a proper centered start, Tuesday's re-tuned color threshold still holding up under Friday morning's lighting, Wednesday's understanding of the ROI/contour mechanism informing a couple of small final adjustments to how aggressively the steering responded to the angle reading specifically through the curved sections of the mat.

Ran the full loop, curves included, start to finish, without a single moment of the wobble from Thursday or the swerve from Monday. Watched it happen live in rqt this time not to debug anything, just to actually enjoy watching the segmented mask track the yellow line cleanly through the curve, lateral offset and angle numbers scrolling by looking exactly the way Wednesday's reading of lane_detect.py said they should.

Five days, one weighted-down mat fold, one re-derived color threshold, one properly-read source file, and one deliberately-broken test run later, and it's a genuinely different feeling watching MentorPi complete that loop on Friday than it would have been if it had just worked cleanly on Monday afternoon and I'd never had a reason to actually understand why.