Robotics & Embodied AI
Vision-language-action models, action tokenisation, diffusion policies and sim-to-real.
5concepts
32flashcards
37minutes of reading
- 01 Action Tokenisation and Representation How continuous robot actions become discrete tokens an autoregressive transformer can emit, from per-dimension binning to frequency-space compression.
- 02 Imitation Learning and Diffusion Policies Why cloning a demonstrator's actions drifts into unseen states, and how generative action models such as diffusion policies and action chunking control the drift.
- 03 Sim-to-Real Transfer Why robot policies are trained in simulation, why they break on real hardware, and how domain randomisation closes the reality gap by making the real world look like one more random draw.
- 04 RT-2 and Web-Scale Robot Learning RT-2 co-trains one transformer on internet vision-language data and robot trajectories by encoding actions as text tokens, transferring semantic web knowledge into robotic control.
- 05 Vision-Language-Action Models Turning a pretrained vision-language model into a robot policy that maps camera images plus a language instruction to motor actions, so the robot inherits web-scale semantic knowledge it could never learn from robot data alone.