Robotics & Embodied AI
Vision-language-action models, action tokenisation, diffusion policies and sim-to-real.
5concepts
32flashcards
37minutes of reading
- 01 RT-2 and Web-Scale Robot Learning RT-2 co-trains one transformer on internet vision-language data and robot trajectories by encoding actions as text tokens, transferring semantic web knowledge into robotic control.
- 02 Vision-Language-Action Models Turning a pretrained vision-language model into a robot policy that maps camera images plus a language instruction to motor actions, so the robot inherits web-scale semantic knowledge it could never learn from robot data alone.