Test coverage against the real task
Ask whether the sample includes the objects, actions, phases, failure modes, and environmental variation the robot will see. A polished clip that misses the difficult transition is not representative.
Map each requested capability to evidence in the sample before discussing volume.
Inspect the package, not just the video
Open the metadata, annotation files, camera specifications, quality notes, consent references, and delivery manifest. The package should be usable by engineering and reviewable by legal.
Stable IDs and timestamps make it possible to trace a model example back to its source capture.
Make the pilot measurable
Choose a small set of checks before the pilot arrives: task completeness, label agreement, usable-frame rate, occlusion, privacy status, and export compatibility.
A measurable pilot turns a subjective vendor conversation into a decision your team can defend.