It's implied, and I'm hoping it's true, that this is a map-less navigation. Which is impressive. This kind of task is much easier if you have a pre-captured map of the environment, but if they are doing this without a map it's great. Historically you were always faced with "The Kidnapped Robot" problem where robots that didn't know where they were couldn't navigate even a little bit. Here the robot appears to be able to follow directions as long as they are interpretable from its current vision (or via dead reckoning).
iandanforth
What is the realistic path to getting to play with this? I would love to hook this up to OpenClaw for hobbyist exploration. My dream has been to embody OpenClaw into a farm robot (been looking at adapting one of those RC lawnmowers that is tracked and built for mowing steep hills) so that I can assign it various tasks around our acreage -- "Explore the fenceline take pictures of the plants. Find all of the poison ivy and invasive honeysuckle and spray it with your Roundup sprayer. Repeat this every week and report the species map after every pass. Come back to the barn and charge yourself whenever you get low."
It's not hard to put OpenClaw into a robot body (numerous YouTube videos showing people doing this sort of thing), but when you dig in and see what people have done, the actual movement portion is always the clunkiest part (and this matches my own experiments as-such as well). It feels like an 8B model like this would be perfect for solving pathing and navigation issues.
Anyone who may be more experienced with Mistral (or companies like them) -- are they interested in hobbyist builders who would be experimenting with things like this? Or are they primarily looking for commercial partners? I would be willing to pay a license fee to use the model in my experiments, but if I'm just one guy, I'm not sure they'd want to work with me unless I were building a business out of it (which I'm not).
HanClinto
This looks to not be an openly available model, but I think if it were, availability of an easy single-camera navigation setup could allow for a lot of cool hobbyist projects.
humanperhaps
This is very cool. Congratulations to the Mistral team. Map less navigation in the outside world has been around for quite a while. But map less navigation inside the buildings is relatively new. Some stanford researchers trained a vision model (PIGEON) which could tell the geo-location from any image. It was not released publicly due to privacy nightmarish (stalking!) possibilities but I am assuming similar type of tech has gone behind this robot. if someone knows more, feel free to correct.
Mistral seems to be going wide and niche. Could be a smart strategy going forward.
Gecko4072
I'm ready for my home helper robot that makes dinner and does the dishes and takes out the trash.
But I'm scared for when those home helpers get drafted to fight in wars, either for or against me...
montroser
I'm impressed, because it is so minimalistic.
On the other hand it reminds me of 2010ish robotic demo videos from academic researchers and willow garage.
Problem with robotics is usually that it's easy to produce a good-enough looking demo, but it's really hard to make somethin work in the general case. As exemplified by autonomous driving.
aktenlage
> achieves 76.6% on R2R-CE (Room-to-Room in Continuous Environments)
I would like to know what it did the other 23.4% of the time!
mil22
Nice. The robot can travel autonomously to a table tennis or chess tournament against other robots while humans are scrubbing the kitchen floor.
tracekl
>> State-of-the-art performance on R2R-CE
To be clear, R2R-CE is a benchmark consisting of simulated environments.
So what this means is that beating this benchmark is about as useful as getting a robot to play Minecraft, or some other video game. Great, but a robot must run in the real world, in physical reality: not in a digital environment.
Unfortunately it's extremely hard to evaluate the performance of robotic systems in physical reality. First of all because if they don't do well you need lots of spare robots to complete the evaluation.
The article above does include the obligatory video of a physical robot navigating a virtually empty, uncluttered "office" environment with nice, smooth surfaces, at 2x speed; virtually a trademark of the entire research field by now. Brownie points for having three guys shuffle carefully across the robot's path towards the end of the video (they're shuffling at 2x speed so they're going reeeaaaally carefully in real life, probably ready to jump out of the way if say a heavy metallic object hurtles towards them randomly).
But, this is like all the fanfare and hype about Aloha a couple of years ago: great stuff if you want your team to be bought by one of the large tech corps, or to get more funds to play with your cool tech (I mean who doesn't?). Not so great for anyone who's expecting this to be a step forward (or ahem a roll forward) on the way to having robot maids/butlers going 'round your house or office.
comments (10)
iandanforth
It's not hard to put OpenClaw into a robot body (numerous YouTube videos showing people doing this sort of thing), but when you dig in and see what people have done, the actual movement portion is always the clunkiest part (and this matches my own experiments as-such as well). It feels like an 8B model like this would be perfect for solving pathing and navigation issues.
Anyone who may be more experienced with Mistral (or companies like them) -- are they interested in hobbyist builders who would be experimenting with things like this? Or are they primarily looking for commercial partners? I would be willing to pay a license fee to use the model in my experiments, but if I'm just one guy, I'm not sure they'd want to work with me unless I were building a business out of it (which I'm not).
HanClinto
humanperhaps
here's the link to the PIGEON paper - https://lukashaas.github.io/PIGEON-CVPR24/
dwa3592
Gecko4072
But I'm scared for when those home helpers get drafted to fight in wars, either for or against me...
montroser
On the other hand it reminds me of 2010ish robotic demo videos from academic researchers and willow garage.
Problem with robotics is usually that it's easy to produce a good-enough looking demo, but it's really hard to make somethin work in the general case. As exemplified by autonomous driving.
aktenlage
I would like to know what it did the other 23.4% of the time!
mil22
tracekl
To be clear, R2R-CE is a benchmark consisting of simulated environments.
So what this means is that beating this benchmark is about as useful as getting a robot to play Minecraft, or some other video game. Great, but a robot must run in the real world, in physical reality: not in a digital environment.
Unfortunately it's extremely hard to evaluate the performance of robotic systems in physical reality. First of all because if they don't do well you need lots of spare robots to complete the evaluation.
The article above does include the obligatory video of a physical robot navigating a virtually empty, uncluttered "office" environment with nice, smooth surfaces, at 2x speed; virtually a trademark of the entire research field by now. Brownie points for having three guys shuffle carefully across the robot's path towards the end of the video (they're shuffling at 2x speed so they're going reeeaaaally carefully in real life, probably ready to jump out of the way if say a heavy metallic object hurtles towards them randomly).
But, this is like all the fanfare and hype about Aloha a couple of years ago: great stuff if you want your team to be bought by one of the large tech corps, or to get more funds to play with your cool tech (I mean who doesn't?). Not so great for anyone who's expecting this to be a step forward (or ahem a roll forward) on the way to having robot maids/butlers going 'round your house or office.
Sorry guys.
YeGoblynQueenne