RoBart - iPhone-controlled robot, first day of autonomy

[@AdilErt]

İt’s too hard to find a laptop from that angle. İt almost looks like a remote or something like that from that angle.

[@trzy]

Yeah. And also, the AI roams about in a scattershot manner without good long-term planning, so it misses a lot of things.

[@tochadenetherita8839]

do you use some api with integration with gpt?

[@trzy]

It supports Claude 3.5 Sonnet, GPT-4 Turbo, and GPT-4o. Claude is by far the best of the three. The interface to the robot is custom written, and I leverage ARKit features to try to do some very crude mapping of the environment (although I don't do proper pathfinding yet -- I have the code in the repo but there are some issues that prevent me from using it, so I restrict the robot to taking straight line paths that I test against the world mesh for obstructions).

[@AdilErt]

Can find anything?

[@trzy]

Anything the current state-of-the-art LLMs can identify. That is, it can *identify* those objects, but the LLMs aren't designed for operating robots, so this often fails. I have gotten it to navigate outside of the kitchen fully autonomously and find things quite far away from the initial starting point.