You can chase another point of accuracy for a month, or you can watch one operator use the thing for an hour. Usually the hour wins.
On the pallet counter, our validation numbers looked great long before the system was actually useful. The gap was never in the model — it was in the ten seconds of messy reality the model never saw in training.
What actually moved the needle:
- Sitting on the warehouse floor and watching how people really stack.
- Post-processing that assumed the world is noisy, not clean.
- Shipping something people trusted on a bad day, not just a good demo.