LeHome 挑战：基于 VLA 的双臂衣物折叠LeHome Challenge: Bimanual Garment Folding with VLA Policies

Overview

This project targets the LeHome Challenge 2026: training a bimanual robot policy to fold garments in simulation. Garments are deformable objects with high-dimensional state, complex contact dynamics, and large shape variation across clothing categories, making the task difficult to solve with fixed trajectories or rule-based control.

Our goal was not to script a special case for one garment, but to train a policy that generalizes across categories. Our submission team, LaundryNauts, ranked 54th on the official evaluation with an official score of 40.00%. In our internal evaluation setup, the best model reached 55.2%.

Team

Task Setup

LeHome Challenge evaluates four garment categories:

• Long-sleeved tops
• Short-sleeved tops
• Long pants
• Shorts

For each category, the robot must start from unfolded, distorted, or shifted garment states and use bimanual manipulation to reach a folded target configuration. The evaluation measures not only final shape quality, but also generalization across initial states and garment geometry. Based on the task description, the challenge provides teleoperation demonstrations across the four garment classes for policy training.

Model Comparison

We evaluated five policy families:

We selected SmolVLA as the primary submission policy. It produced the most balanced performance across garment categories, with especially strong results on shorts.

Internal Evaluation

Shorts and long-sleeved tops were easier to stabilize into repeatable grasp-and-fold trajectories. Short-sleeved tops failed more often due to local cloth folds, sleeve orientation, and grasp-point drift. Long pants were in the middle, with the main bottleneck being pant-leg alignment and late-stage fold consistency.

Official Evaluation

On the official leaderboard, LaundryNauts achieved:

The gap between official and internal evaluation is likely due to differences in evaluation samples, randomized initial states, simulation rollout settings, and the submitted model checkpoint.

System Flow

1. Read multi-view simulation observations and task language.
2. Feed garment category, visual inputs, and robot state into the VLA policy.
3. Generate bimanual end-effector actions.
4. Execute the action sequence in simulation and score the fold using garment keypoints and target configurations.
5. Evaluate multiple models and checkpoints by category, then select the strongest overall submission.

Lessons Learned

• For deformable-object manipulation, training loss does not reliably predict final fold quality; rollout evaluation is mandatory.
• Per-category evaluation matters. A model can score well overall while systematically failing one garment class.
• Garment keypoints are useful for diagnosing failures such as sleeve misalignment, pant-leg mismatch, or grasp drift.
• Smaller models are not necessarily weaker. SmolVLA was the most stable and efficient option for rapid iteration under time constraints.

Acknowledgments

This project was completed by Md Saif Ahmad, Andnet DeBoer, Conor Hayes, Jyothi Swaroop Kasina, Chenyu Zhu, and Robert Zhu. Thanks to LeHome Challenge for providing the task, data, and evaluation environment.