The data engine

Language models learned from text that already existed. Robot models need recordings of a body moving through the world, with every camera frame matched to the joint motion that followed, and almost none of that existed. So every serious robot company now runs a data business: headsets and handheld grippers, crowds filming their own hands, simulated robots, and fleets whose mistakes get fed back into the model. Raw collection is getting cheap fast. Data from robots doing paid work, and failing at it, stays scarce.

The data gap in numbers

Text models had about 300 trillion tokens on the web for free; GPT-4 alone trained on about 13 trillion estimate. Robots need something the web does not hold: synchronised joint angles, gripper forces, camera frames and the task being attempted, recorded while a body handles real objects. The largest open pool, Open X-Embodiment, is about 1 million trajectories from 22 robot types and 34 labs verified. Practitioners put the gap bluntly: "you're never going to teleoperate your way across."

The data pyramid

The industry answer is a pyramid, formalised by NVIDIA's GR00T program. Use huge cheap data at the bottom to teach the model what things are, and spend scarce robot data at the top to teach it how to move. In GR00T N1 the real robot layer was about 50,000 demonstrations against about 780,000 synthetic trajectories and about 3 million human video clips, so the action-labelled layer was about 1.7% of the base verified Mar 2025. The model borrows meaning from the base and learns motion from the top.

The data pyramid is how labs reason about mixing human video data, simulation data and real-robot data. Synthetic data from world models is the fastest-growing slice: GR00T-Dreams made 780,000 trajectories in 11 hours verified Jun 2025. Its limit is the same as simulation's: pictures and motion synthesise well, force and contact do not.

Teleoperation: puppeting the robot

The gold-standard source is a person driving the robot while every sensor records. Each recorded attempt at a task, from start to finish, is an episode, and a good one becomes a demonstration. The equipment is a teleoperation rig, and there are four families.

Throughput is the problem. A novice operator produces 8 to 12 usable episodes an hour, a trained one 25 to 40 estimate. Single-arm simple tasks pass buyer checks 90% of the time or more; two-arm ALOHA-style work passes 70 to 85% estimate. Real-robot teleoperation, fully loaded with rig and staff, runs about US$72 to US$145 an hour in China and US$90 to US$150 in the US unverified. The average price of teleop data on a target robot fell from about US$340 an hour in early 2024 to about US$136 by late 2025 unverified.

Human video: the 2026 pivot

In August and September 2026 the biggest labs demoted teleoperation from main fuel to fine-tuning and bet on cameras worn by people. Egocentric video, filmed from the worker's own eyes, shows hands approaching and handling objects. It has no robot actions, but hand pose tracking gives an approximate label, and it is cheap and diverse.

Does it work? The labs say yes, with their own numbers. Dyna-2's scores rose from 20% to 28% to 45% to 53% as its human video grew from 1,000 to 10,000 to 100,000 to 1 million hours across 14 tasks company figures. Figure says pretraining on Index lifted Helix 2.5 from 9% to 56% in 30 unseen homes, using half the task-specific robot data company figures. The skeptic's case, from robotics researcher Robert Griffin: video lacks the physical interaction signal that teleoperation captures unverified. Both can be true. Video teaches the approach; robot data still teaches the squeeze.

Prices are collapsing. Crowd phone video goes for about US$2 to US$5 an hour at the low end, dense-labelled wearable capture for US$25 to US$60 estimate. When Figure, Tesla, Build AI and Generalist run their own capture in over 100 countries, a new vendor selling raw human video competes with its own buyers.

China's training grounds

China turned data collection into public infrastructure. Local governments build a training ground, a hall of staged kitchens, factory lines and care rooms, buy robots from local makers and hire people to teleoperate them all day. Companies build their own data factory on the same model: AgiBot's 4,000 m² facility with 100 robots and 3,000+ staged objects produced the AgiBot World dataset of 1 million trajectories and 2,976 hours verified.

Output per robot is modest: about 4 hours of training data per robot per day, and 100 robots handle about 12,000 collection tasks a day at the Beijing site verified Apr 2026. China Mobile alone ordered RMB 124M (about US$17.6M) of robots from Unitree and AgiBot for this kind of work. Analysts told Rest of World there are "some potential bubbles" verified Jan 2026. A national standard for embodied-AI dataset quality, YD/T 6771-2026, takes effect on 1 Nov 2026 verified. The training grounds are also part of why China buys most of the world's humanoids: many are bought to generate data.

The data flywheel

The top of the pyramid fills itself once robots are working. Deploy a robot, let it run, have a person take over remotely when it gets stuck, log the takeover, retrain on it, ship the better model, repeat. That loop is the data flywheel, and each takeover is an intervention.

Who runs one today:

• 1X. When NEO's onboard model gets stuck, a remote 1X employee in the US drives it through VR, with the owner's approval per session. Every session becomes training data. Autonomy at launch is guided at 60 to 70% contested; the figure circulates with no primary source.
• Dyna Robotics. A reward model watches robot video, labels successes and errors automatically, targets error-recovery data and updates the model nightly verified.
• Apptronik. Apollo 2 fleets collect data at a ~90,000 sq ft "Robot Park" in Austin and at Robot Parks inside Google DeepMind, Mercedes-Benz and GXO, mixing teleop and autonomous runs verified Jun 2026.
• Skild AI. Bought Zebra's robotics unit, including Fetch mobile robots, in April 2026: a brain company buying a fleet because data needs real robots working verified.

Why failures are the most valuable data

A demonstration shows the robot one way to succeed. A failure data record shows it the situations where its current model goes wrong, which is exactly where the next improvement lives. π*0.6 trains on expert corrections during its own runs; π0.7 trains on autonomous data including failures, labelled with how well each attempt went verified.

Data operations

Collecting is half the job. The rest is turning raw recordings into something a training run can load.

• Episodes. One attempt, start to end, stored as time-aligned streams: each camera, joint positions, commanded actions, gripper state, the instruction text and a success flag.
• Annotation. Labelling segments ("grasp", "lift", "place"), marking success and failure, writing the instruction. Humans charge about US$0.25 to US$1.00 per episode; vision-language models assisting cut that to US$0.05 to US$0.10 estimate. A 5,000-hour annotated egocentric corpus costs about US$150,000 to US$200,000 estimate.
• Formats. Hugging Face's LeRobot format is the open default for small arms; its community had 487 datasets and 10M frames by June 2025 verified. Google's RLDS format underlies Open X-Embodiment unverified. Converting between formats is routine glue work.
• Storage. Video dominates. Build AI's 10,000 hours of 1080p head-camera video is 16.4 TB, about 1.6 GB per hour derived. A robot with three cameras produces several times that per hour of work.
• Quality control. Buyers reject episodes with dropped frames, unsynchronised streams, failed grasps marked as success or faces in view without consent. First-pass acceptance is the number that decides whether a collection business makes money.

Ownership, licences and privacy

Data rights are set by contracts and a few laws, and they vary by where the robot sits.

• Licences. Open sets range from Apache 2.0 (Build AI's Egocentric-10K, gated) to research-only licences (Ego4D) unverified. Read the licence before a commercial training run.
• Contributor consent. Figure's Index privacy policy lets it sell contributors' personal data to commercial buyers, with an opt-out verified Sep 2026. Expect regulators to look at crowd capture.
• The EU Data Act. Users of connected products can get their raw product data free and send it to a third party such as an independent repairer. Derived data is out of scope, and the recipient may not use it to build a competing product verified.
• Singapore. The Personal Data Protection Act (PDPA) governs personal data, which covers identifiable faces and voices in robot camera and microphone streams. No Singapore or ASEAN law gives anyone access to robot operating data unverified. In practice, access depends on the contract with the maker or distributor, and on whether logs sit on servers in China or Singapore.

Where Southeast Asia fits

Southeast Asia has cheap, skilled labour and time zones next to China, so it looks like a natural data base. Some of that is already taken. Build AI says its 100,000-hour set came from factory workers across Southeast Asia unverified. Roborax already markets egocentric capture and teleoperation pods in Manila and Cebu. The labs that buy data now collect their own at global scale.

Crypto shows up here too. PrismaX raised an US$11M seed led by a16z's crypto fund to sell data from browser teleoperation of robot arms, with no operator, hour or buyer numbers public; BitRobot pays tokens for robot missions; Reborn claims 5,000 motion capture suits shipped raises verified. The best-measured token project in the region, Auki, earned about US$109,288 of on-chain revenue in 12 months against about US$371,000 a year of token incentives verified Sep 2026. Stablecoin payouts to remote operators are a real cost saving in the Philippines, and crypto is illegal as a means of payment in Indonesia and Vietnam verified, so wages there go through licensed local converters.

Terms in this chapter