Choosing a capture rig is one of the first decisions in any video data project, and one of the hardest to undo. A single RGB camera is cheap, light and easy to deploy at scale. A stereo pair adds a second, calibrated camera and, with it, the ability to recover depth from geometry. Which one a project needs depends on what the model must understand about space — and on what it will see at deployment.
Mono-RGB: one camera, colour images
Mono-RGB capture records standard colour video from a single lens. It is the most common format in computer vision, and most pretrained models expect it. Mono rigs are small, inexpensive and simple to synchronise with other sensors, which makes them practical for large, distributed collection.
The limitation is depth. A single image cannot distinguish a small object close to the camera from a large object far away without additional cues. Monocular depth-estimation models infer depth from learned priors — familiar object sizes, perspective, shading — and can be surprisingly good, but their output is relative or approximate rather than measured.
Stereo: two cameras, depth from disparity
A stereo rig places two cameras a known distance apart (the baseline). Points in the scene appear at slightly different horizontal positions in the left and right images; that difference, the disparity, is inversely related to distance. With calibrated cameras, disparity converts to metric depth. This is the same principle human vision uses.
What stereo requires
- Calibration: intrinsic parameters for each camera and the precise relative pose between them, rechecked if the rig is knocked.
- Hardware synchronisation, so both images are captured at the same instant; otherwise motion creates false disparity.
- Texture in the scene. Plain white walls and reflective surfaces give matching algorithms little to work with.
- A baseline suited to the working distance: a wider baseline improves depth precision at range, a narrower one suits close work.
Applications in computer vision
For classification, detection and most activity recognition, mono-RGB is usually sufficient and far easier to scale. Stereo becomes valuable when the task depends on geometry: estimating object size, measuring distances, reconstructing scenes in 3D, or producing ground-truth depth to train and evaluate monocular depth models. A common pattern is to collect a smaller stereo dataset alongside a larger mono one, using the stereo data for supervision and evaluation.
Applications in robotics
Robots act in metric space. To grasp an object, a manipulator needs to know where it is in three dimensions, not just in the image. Stereo — or another depth source — provides that directly, which is why many robot heads and wrists carry stereo cameras.
For training data, the principle is to match the deployed sensor where possible. If the robot perceives through a stereo head, egocentric stereo recordings of humans performing the task give a model inputs with the same structure it will see at run time. If the robot uses a single wrist camera, mono-RGB egocentric data is the natural fit, and depth can be added from other sensors.
Hand–object interaction
Close-range manipulation is where stereo earns its cost. Whether fingers are in contact with an object or just in front of it is often ambiguous in a single image. Depth resolves it. For datasets focused on grasping and fine manipulation, stereo or depth data can substantially improve the reliability of contact labels.
Cost and scale trade-offs
- Hardware: stereo rigs cost more and are bulkier, which matters for wearable capture.
- Storage: two video streams roughly double raw storage.
- Processing: rectification and disparity computation add pipeline stages and QA checks.
- Annotation: depth can make some labels easier (contact, 3D boxes) but adds label types to manage.
- Coverage: a cheaper mono rig can reach more environments for the same budget.
Hybrid approaches
Many programmes combine the two: broad mono-RGB collection for diversity, plus a targeted stereo subset for depth supervision and evaluation. Others pair a mono egocentric camera with a fixed stereo or depth camera in the scene. The right mix depends on whether depth is needed at training time, evaluation time or both. Environmental breadth is discussed further in Why Environmental Diversity Matters in Robotics Training Datasets.
Conclusion
Mono-RGB and stereo capture serve different purposes. Mono scales; stereo measures. For most computer vision tasks, mono-RGB with good coverage is the efficient default. For robotics, where models act in metric space and contact matters, stereo or another depth source often pays for itself — particularly when it matches the robot’s own sensors.
Mvaak selects capture rigs per project for egocentric data collection. See Egocentric Video Data for the capture options, or read Egocentric Video vs Third-Person Video.
Written by Mvaak — AI Data Services.
