La Mano Vision

La Mano is a family-owned tortillería in Los Angeles. Every day, workers at the prep table wrap tortilla stacks into plastic bags, then carry them to inventory. The owner — my father — has always tracked daily production by memory and rough observation. That means no reliable count, no trend data, and no way to know whether a busy day actually produced more bags or just felt busier.

The ask was simple: know how many bags leave the table each session. The constraint was equally simple: one existing Hikvision camera, no additional hardware budget, and the system had to run without a developer present to operate or debug it.

Standard off-the-shelf counting solutions either require dedicated hardware or cloud inference with ongoing per-frame costs. Neither fit a small family business. The system needed to be on-prem, fast enough to run on a consumer GPU, and honest about its limitations — if it gets the count wrong, the owner will notice. He’s in that room every day.

This shaped every technical decision: model size (nano, not large), inference method (sampled frames, not every frame), and tracking approach (custom logic I can debug and explain, not a black-box library).

I extracted training frames from NVR-exported footage using a custom script, then labeled 268 frames in Roboflow across two classes:

I fine-tuned YOLOv8n on these 268 frames in Google Colab over 150 epochs (T4 GPU, imgsz=640, batch=16), then built a custom tracking pipeline using IoU-based greedy matching across sampled frames — one frame every 0.25 seconds. A region-of-interest polygon constrains detections to the prep table surface so anything off-frame doesn’t generate false events.

A bag is counted when it satisfies both conditions: it received a PACKAGED event (model reclassified the track from unbagged to bagged), and its final classification at exit is still bagged_stack. Bags that sit on the table for 10+ seconds without a PACKAGED event also count — this handles pre-existing bags that were already wrapped when the session began.

Several approaches were tried and reverted. Each failure taught me something about where the real constraint actually lived.

The honest conclusion: the remaining accuracy gap (~2 bags per session) is a model consistency problem, not a tracking logic problem. More diverse training data — specifically footage from high-volume days where larger stacks and faster packaging are common — is the right fix. Tuning the tracker further risks overfitting to one day’s footage.

The system runs on a dedicated Windows gaming PC (RTX 5060 Ti, 16GB VRAM) permanently installed at the store. Inference runs at 2–5 ms per sampled frame — fast enough to run in real time on the live RTSP stream. On the April 18 baseline clip (15 bags, manually verified), the system counts correctly within ±2 bags.

The system is honest about what it doesn’t know. During busy packaging bursts, bounding box instability causes the tracker to occasionally split one physical bag into two tracks — both get counted. This is documented in the roadmap and will be addressed by retraining on April 19 footage (larger stacks, faster packaging), not by patching the tracker.