Santiago Valdarrama
@svpino
You can use this open project to connect a model to a simulated robot.
Then, you can measure how well the robot completes a task.
The basic loop is simple:
1. The model, SenseNova-RoboRSI-27B-Preview, receives a task and images from the robot's cameras.
2. It chooses a movement
3. The simulator executes that movement and returns new observations
4. The model uses that feedback to decide what to do next
Some of the model highlights:
• Inspects visual details and reasons about spatial relationships
• Adjusts actions based on execution feedback
• 26.13 average score and 20.83% task success on RoboDojo
• 96.1% success rate with direct control on LIBERO-Pro
This is pretty useful for comparing models and understanding why a robot failed.