LeRobot Tutorial 2026 - Train Your First AI Robot Policy with SO100

This is the hands-on LeRobot tutorial we wanted when we first set up an SO100: minimal theory, current commands, and a straight path from teleoperation to a real learned policy.

If you are searching for a LeRobot tutorial, a LeRobot training guide, or the fastest way to train a robot policy with LeRobot on real hardware, this is the workflow to follow in 2026.

The goal is simple: collect demonstrations with a leader-follower SO100 setup, turn them into a clean Hugging Face dataset, train a first policy, and test that policy on the real robot arm. You do not need a full lab. You do need a robot arm that is already compatible with LeRobot, which is why the SO100 is such a strong fit.

If you have not done the full install yet, start with our LeRobot + SO100 setup tutorial. If you want the broader intuition first, read our imitation learning tutorial for robot arms. This article assumes you are ready to actually run the pipeline.

What Is LeRobot?

LeRobot is Hugging Face's open-source framework for robot learning. In practice, that means it gives you one stack for:

• teleoperation and data collection on real robots
• a standard dataset format that works with the Hugging Face Hub
• policy training for imitation learning and reinforcement learning
• evaluation, replay, and deployment on hardware

That standardization matters. Without a framework like LeRobot, most beginners lose time on glue code: logging joint states, syncing camera frames, formatting datasets, and writing evaluation scripts. LeRobot turns those chores into built-in tools so you can focus on the task itself.

For SO100 users, the fit is especially good because the arm is built around the same leader-follower workflow that LeRobot uses for imitation learning. You physically move the leader arm, the follower copies the motion, and LeRobot records synchronized observations and actions for training.

Prerequisites

Before you start this LeRobot SO100 tutorial 2026, make sure you have:

• an SO100 leader arm and follower arm
• a working Python environment on your laptop or desktop
• LeRobot installed
• a Hugging Face account for dataset and model storage
• one repeatable tabletop task, like moving a block into a tray
• ideally one front-facing camera

If you want the shortest possible setup, the SO100 complete kit is the easiest route because it gives you the leader-follower pair this workflow expects.

Here is the baseline software setup:

python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install lerobot
hf auth login

Practical advice before you touch the CLI:

• Pick one task with a tight workspace.
• Use the same lighting and camera angle for every demo.
• Secure the base of the follower arm so the workspace does not drift.
• Keep your reset state boring and consistent.

For a first policy, "pick up one colored block and drop it in one tray" beats "general tabletop manipulation" every time.

Step 1: Record demonstrations

This is the heart of the whole pipeline. Good data matters more than fancy model selection.

Start by teleoperating the robot and confirming both arms are connected correctly. Use stable IDs and keep them the same across teleoperation, recording, replay, and evaluation. LeRobot stores calibration data against those IDs, so consistency saves pain later.

lerobot-teleoperate   --robot.type=so100_follower   --robot.port=/dev/ttyACM1   --robot.id=so100_follower_lab   --teleop.type=so100_leader   --teleop.port=/dev/ttyACM0   --teleop.id=so100_leader_lab   --display_data=true

If LeRobot detects missing calibration for one of the devices, finish that first. Do not record data until the follower tracks the leader smoothly across the full workspace.

Once teleoperation looks clean, record your demonstrations:

HF_USER=$(NO_COLOR=1 hf auth whoami | awk -F': *' 'NR==1 {print $2}')

lerobot-record   --robot.type=so100_follower   --robot.port=/dev/ttyACM1   --robot.id=so100_follower_lab   --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}"   --teleop.type=so100_leader   --teleop.port=/dev/ttyACM0   --teleop.id=so100_leader_lab   --display_data=true   --dataset.repo_id=$HF_USER/so100_pick_place_first_policy   --dataset.num_episodes=50   --dataset.single_task="Pick up the red block and place it in the tray"   --dataset.streaming_encoding=true   --dataset.encoder_threads=2

That is enough to create a real dataset on the Hugging Face Hub and use it as the input for training.

Practical recording tips

• Move a little slower than feels natural. Jerky demonstrations teach jerky actions.
• Keep the object, tray, and camera framing consistent for the first run.
• Reset the arm to the same home pose before each episode.
• Throw away obvious bad takes instead of hoping the model will average them out.
• For a first ACT baseline, 30 to 50 clean episodes is a good target.

If you want a deeper theory pass on why this works, our imitation learning guide breaks down demonstrations, observations, actions, and policy learning in plain English.

Step 2: Process your dataset

Once recording finishes, LeRobot stores the dataset in its Hugging Face-native format. This is one of the biggest reasons to use LeRobot instead of rolling your own logger.

At a high level, your dataset contains:

Path	What it stores
meta/info.json	schema, feature definitions, fps, versioning, path templates
meta/stats.json	normalization statistics used during training
meta/tasks.jsonl	natural-language task labels
meta/episodes/	per-episode metadata and offsets
data/	parquet shards with frame-by-frame robot data
videos/	MP4 shards for each camera stream

Before training, inspect the data. Do this every time. One broken camera stream or one terrible episode can waste hours.

Visualize a recorded episode:

lerobot-dataset-viz   --repo-id $HF_USER/so100_pick_place_first_policy   --episode-index 0

Replay a recorded episode back on the real robot:

lerobot-replay   --robot.type=so100_follower   --robot.port=/dev/ttyACM1   --robot.id=so100_follower_lab   --dataset.repo_id=$HF_USER/so100_pick_place_first_policy   --dataset.episode=0

What you are checking for:

• camera frame matches the physical action you expected
• the arm never starts from wildly different poses
• the task description matches the actual task
• there are no episodes where the robot collides, stalls, or loses the object immediately

If you recorded junk, fix it now. Your first real gain in a train robot policy LeRobot workflow usually comes from dataset cleanup, not from changing hyperparameters.

Step 3: Train a policy

Now you choose a policy family. For most people following a LeRobot training guide on a real SO100, the right first move is ACT.

Here is the short version:

Policy	Best use case	Trade-off
ACT	first real-robot baseline	easiest starting point
Diffusion Policy	smoother, multimodal behavior	more compute and tuning
TDMPC	reward-driven or simulator-heavy work	not usually the first desk demo choice

If your goal is "I want the arm to copy a simple pick-and-place task this week," start with ACT. Once the baseline works, you can experiment with Diffusion Policy. TDMPC is powerful, but it fits better when you have a control-heavy or RL-style setup, not your first imitation-learning run.

Train an ACT policy like this:

lerobot-train   --dataset.repo_id=$HF_USER/so100_pick_place_first_policy   --policy.type=act   --output_dir=outputs/train/act_so100_pick_place   --job_name=act_so100_pick_place   --policy.device=cuda   --wandb.enable=true   --policy.repo_id=$HF_USER/act_so100_pick_place

A few notes:

• If you are on mps, swap the device from cuda to mps.
• If you are CPU-only, the run will still work, but it will be slower.
• Some LeRobot examples use the shorter --policy=act form. The important part is that you are selecting the ACT policy family.

To resume from the latest checkpoint:

lerobot-train   --config_path=outputs/train/act_so100_pick_place/checkpoints/last/pretrained_model/train_config.json   --resume=true

If your local machine is weak, train in the cloud. That is not cheating. It is normal. Record on the desk machine, train on a GPU box, then bring the checkpoint back for evaluation.

How to think about policy choice

For a first imitation learning tutorial robot arm project:

• Use ACT when you want the shortest path to a result.
• Try Diffusion Policy after ACT if your actions look hesitant or multimodal.
• Reach for TDMPC if your project starts looking more like RL or model-based control than pure demonstration learning.

Do not optimize policy selection before you have verified that:

1. your demos are clean
2. your resets are consistent
3. your camera stream is stable
4. your evaluation matches your training task

Step 4: Deploy and test

Once training finishes, run inference on the real SO100 and capture evaluation episodes. LeRobot uses the same recording stack for this, but now you pass a policy checkpoint.

lerobot-record   --robot.type=so100_follower   --robot.port=/dev/ttyACM1   --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}"   --robot.id=so100_follower_lab   --display_data=false   --dataset.repo_id=$HF_USER/eval_so100_pick_place   --dataset.num_episodes=10   --dataset.single_task="Pick up the red block and place it in the tray"   --dataset.streaming_encoding=true   --dataset.encoder_threads=2   --policy.path=$HF_USER/act_so100_pick_place

This is the moment that tells you whether your training loop actually worked.

Measure success in a boring way:

• How many of 10 evaluation episodes finished successfully?
• Does the arm hesitate at the grasp point?
• Does it miss only when the reset pose is off?
• Does it fail the same way every time or in random ways?

If the policy works on the easy version of the task, resist the urge to complicate it immediately. First get one reproducible success case. Then widen the tray position, change block colors, or add viewpoint variation.

Troubleshooting tips

Most failed LeRobot runs are not mysterious. They usually come from one of five things.

1. Teleoperation works badly

Symptoms:

• follower arm lags or jitters
• joints do not line up with the leader
• motion looks clipped or asymmetric

Fixes:

• re-run calibration before recording
• make sure the leader and follower IDs stay consistent
• confirm both ports are correct after reconnecting USB cables
• reduce speed and workspace size for your first task

2. The dataset looks fine, but the policy does nothing useful

Symptoms:

• arm moves, but not toward the object
• the gripper closes at random
• success rate stays near zero

Fixes:

• inspect the first few episodes with lerobot-dataset-viz
• delete low-quality demonstrations instead of keeping all 50
• reduce task complexity and tighten resets
• add more demonstrations from the exact failure mode you care about

3. Camera issues poison the run

Symptoms:

• camera frames freeze or stutter
• training looks unstable across similar episodes
• evaluation works only from one angle

Fixes:

• start with one front camera at 640x480 and 30 fps
• keep lighting stable and remove reflective clutter
• make the object color easy to segment visually
• only add more cameras after the one-camera baseline works

4. Training is too slow

Symptoms:

• hours of training for a tiny experiment
• laptop becomes unusable during the run

Fixes:

• use CUDA on NVIDIA or MPS on Apple silicon
• move training to Colab or another GPU machine
• keep your first dataset small and tightly scoped

5. Replay works, but inference does not

Symptoms:

• recorded episodes replay cleanly
• learned policy fails immediately

Fixes:

• compare the evaluation setup to the training setup frame by frame
• verify the object, tray, and camera are in the same place
• look for reset drift between episodes
• start with ACT before blaming the model family

If you are still stuck, our SO100 troubleshooting guide and setup tutorial cover the most common hardware and software problems in more detail.

Where to get the hardware

If you are ready to run this on a real arm, the hardware decision is straightforward: buy the setup that already matches the tutorial.

The SO100 complete kit is $199 and gives you the leader-follower workflow this article assumes. That matters because the fastest way to finish your first LeRobot project is to spend your time on recording, training, and evaluation - not on sourcing parts, printing frames, and debugging assembly.

Start here:

• Buy the SO100 robot arm kit
• See the LeRobot hardware page
• Compare sellers in our where-to-buy guide
• Read pre-assembled vs DIY before you decide

If your goal is to train your first AI robot policy this month, not build a kit for the next month, pre-assembled wins.

FAQ

How many demonstrations should I record for my first LeRobot tutorial?

For a narrow SO100 pick-and-place task, start with 30 to 50 clean episodes. If the task has more visual variation or more than one valid motion pattern, collect more.

Should I start with ACT, Diffusion Policy, or TDMPC?

Start with ACT. It is the most practical first baseline for real hardware. Move to Diffusion Policy once you have a clean ACT result and want to improve behavior. Save TDMPC for projects that are more reward-driven or simulator-heavy.

Can I train without a GPU?

Yes. Recording, replay, and evaluation are fine on an ordinary machine. Training is simply faster on CUDA, workable on MPS, and easy to offload to Colab if needed.

Do I need two robot arms for LeRobot imitation learning?

For the standard leader-follower teleoperation workflow, yes. You use one arm to demonstrate and the other arm to execute and record those demonstrations.

What is the fastest way to get the hardware?

The fastest route is the SO100 complete kit at $199. It matches this tutorial directly and gets you to the actual LeRobot work faster.

Final takeaway

The simplest successful LeRobot tutorial is not the one with the biggest model. It is the one with:

• one repeatable task
• one stable camera
• one clean dataset
• one baseline ACT run
• one honest real-world evaluation

Do that with an SO100 and you will learn the whole real-robot pipeline: demonstrations, datasets, training, deployment, and debugging. That is the foundation you need before scaling up to more cameras, more task diversity, or more advanced policies.

If you are ready to start today, get the SO100 complete kit, follow the setup tutorial, and use this guide as your first end-to-end LeRobot training run.