Hacker News Boston Dynamics Spot Robot Watches Its Step [video]
https://www.youtube.com/watch?v=YD9EaS3VRbc
#ycombinator #boston_dynamics #boston_dynamics_spot #boston_dynamics_innovation #boston_dynamics_robot #artificial_intelligence #foundation_models #deep_learning #artificial_intelligence_robot #dynamic_robots #robots
Recent searches
Search options
Administered by:
#deep_learning
0 posts0 participants0 posts today
CAGE: Causal Attention Enables Data-Efficient Generalizable Robotic Manipulation
Authors: Shangning Xia, Hongjie Fang, Hao-Shu Fang, Cewu Lu
pre-print -> https://arxiv.org/abs/2410.14974
website -> https://cage-policy.github.io
arXiv.orgCAGE: Causal Attention Enables Data-Efficient Generalizable Robotic ManipulationGeneralization in robotic manipulation remains a critical challenge, particularly when scaling to new environments with limited demonstrations. This paper introduces CAGE, a novel robotic manipulation policy designed to overcome these generalization barriers by integrating a causal attention mechanism. CAGE utilizes the powerful feature extraction capabilities of the vision foundation model DINOv2, combined with LoRA fine-tuning for robust environment understanding. The policy further employs a causal Perceiver for effective token compression and a diffusion-based action prediction head with attention mechanisms to enhance task-specific fine-grained conditioning. With as few as 50 demonstrations from a single training environment, CAGE achieves robust generalization across diverse visual changes in objects, backgrounds, and viewpoints. Extensive experiments validate that CAGE significantly outperforms existing state-of-the-art RGB/RGB-D approaches in various manipulation tasks, especially under large distribution shifts. In similar environments, CAGE offers an average of 42% increase in task completion rate. While all baselines fail to execute the task in unseen environments, CAGE manages to obtain a 43% completion rate and a 51% success rate in average, making a huge step towards practical deployment of robots in real-world settings. Project website: cage-policy.github.io.
GARField: Addressing the visual Sim-to-Real gap in garment manipulation with mesh-attached radiance fields
Authors: Donatien Delehelle, Darwin G. Caldwell, Fei Chen
pre-print -> https://arxiv.org/abs/2410.05038
arXiv.orgGARField: Addressing the visual Sim-to-Real gap in garment manipulation with mesh-attached radiance fieldsWhile humans intuitively manipulate garments and other textile items swiftly and accurately, it is a significant challenge for robots. A factor crucial to human performance is the ability to imagine, a priori, the intended result of the manipulation intents and hence develop predictions on the garment pose. That ability allows us to plan from highly obstructed states, adapt our plans as we collect more information and react swiftly to unforeseen circumstances. Conversely, robots struggle to establish such intuitions and form tight links between plans and observations. We can partly attribute this to the high cost of obtaining densely labelled data for textile manipulation, both in quality and quantity. The problem of data collection is a long-standing issue in data-based approaches to garment manipulation. As of today, generating high-quality and labelled garment manipulation data is mainly attempted through advanced data capture procedures that create simplified state estimations from real-world observations. However, this work proposes a novel approach to the problem by generating real-world observations from object states. To achieve this, we present GARField (Garment Attached Radiance Field), the first differentiable rendering architecture, to our knowledge, for data generation from simulated states stored as triangle meshes. Code is available on https://ddonatien.github.io/garfield-website/
MoVEInt: Mixture of Variational Experts for Learning Human-Robot Interactions from Demonstrations
Authors: Vignesh Prasad, Alap Kshirsagar, Dorothea Koert, Ruth Stock-Homburg, Jan Peters, Georgia Chalvatzaki
pre-print -> https://arxiv.org/abs/2407.07636
website -> https://sites.google.com/view/moveint/home
arXiv.orgMoVEInt: Mixture of Variational Experts for Learning Human-Robot Interactions from DemonstrationsShared dynamics models are important for capturing the complexity and variability inherent in Human-Robot Interaction (HRI). Therefore, learning such shared dynamics models can enhance coordination and adaptability to enable successful reactive interactions with a human partner. In this work, we propose a novel approach for learning a shared latent space representation for HRIs from demonstrations in a Mixture of Experts fashion for reactively generating robot actions from human observations. We train a Variational Autoencoder (VAE) to learn robot motions regularized using an informative latent space prior that captures the multimodality of the human observations via a Mixture Density Network (MDN). We show how our formulation derives from a Gaussian Mixture Regression formulation that is typically used approaches for learning HRI from demonstrations such as using an HMM/GMM for learning a joint distribution over the actions of the human and the robot. We further incorporate an additional regularization to prevent "mode collapse", a common phenomenon when using latent space mixture models with VAEs. We find that our approach of using an informative MDN prior from human observations for a VAE generates more accurate robot motions compared to previous HMM-based or recurrent approaches of learning shared latent representations, which we validate on various HRI datasets involving interactions such as handshakes, fistbumps, waving, and handovers. Further experiments in a real-world human-to-robot handover scenario show the efficacy of our approach for generating successful interactions with four different human interaction partners.