Connect with us

Hi, what are you looking for?

Tweaks

RoboAgent Takes Charge: MT-ACT Integration is on the Horizon

Carnegie Mellon University
Image Source: Carnegie Mellon University

Researchers from Carnegie Mellon University have recently released a pre-print paper showcasing their latest system, MT-ACT: Multi-Task Action Chunking Transformer. The main purpose of this system is to train robots with minimal data, allowing them to break down movement tasks into 12 specific “skills” like pick, place, slide, and wipe. With the ability to combine these skills, the robots can perform complex movements in a variety of scenarios.

In their study, the researchers highlight how RoboAgent, after being trained on just 7500 trajectories, successfully executed 12 complex manipulation skills across 38 tasks. Furthermore, the system demonstrated the capability to generalize these skills to hundreds of new scenarios, even in situations with unfamiliar objects, tasks, and kitchens. Remarkably, RoboAgent can adapt and improve its capabilities through new experiences.

The researchers were particularly impressed by the system’s adaptability to minor variations in a kitchen scene, such as changes in object location, lighting, background texture, and the introduction of new objects. This adaptability is noteworthy as most vision-based systems struggle with similar variations. This limitation is why many robots are currently confined to repetitive tasks in secure spaces.

To aid RoboAgent in its tasks, the system utilizes a tool called SAM (Segment Anything Model), which was developed by Meta. This modular approach to generalizing tasks, combined with the implementation of SAM, has proven to be more successful than traditional methods of training systems for every potential variable.

For those interested in further exploration, the project’s GitHub site offers freely available datasets. Additionally, you can read the pre-print research paper in PDF format and watch the accompanying video below to gain more insights.

Image Source: Carnegie Mellon University

Advertisement. Scroll to continue reading.
AIAD

You May Also Like

Tweaks

Over the years, governments and politicians have attempted to implement internet censorship or surveillance measures in the name of protecting children. However, a proposed...

Tweaks

When it comes to defining a cyberdeck, there may be some debate. However, one thing is clear: a cyberdeck should have a battle-ready appearance....

Tweaks

We haven’t checked, but we’ll go out on a limb here and say this is the first project we’ve featured with a BOM that...

Tweaks

Guitarists often seek the perfect sound for their setup and are willing to introduce intentional distortion into the signal to achieve it. While this...