LLMs or Reinforcement Learning? Which is better for robot control?

[@mihir777]

I love how well he articulates complex ideas which intuitively make sense but hard to articulate

[@mihir777]

Corpus, lexicon, representation, semantic, etc.

[@lazycat2]

Interesting approach! Could it be possible to discard the language specific part from the ouput of the LLM and only use the tokens in a knowldege representation-like manner?

[@fernandofuentes7617]

The way of translating the world to machines is really changing right now

[@LV-426...]

Can LLMs be combined with RL for Digit? Or it's pointless to do so?

[@continuallearning8366]

Yes, LLM can be fine-tuned with RL on digit's embodiment

[@Andronyne]

I have a question about humanoid robots in general. The progress on them is quite astounding, yet they are nowhere close to human level movement. I saw a video on a paper on reinforcement learning for virtual humanoid figures, and by the end of it, they had achieved basically humanlike motion. Therefore, is robot movement constrained by the extra parameters put into the reinforcement learning, the physical capacity of the motors, or are we just making poorly balanced robots? Would this be do to odd weight distribution due to the constraints of the muscles (as it were) with these heavy highly concentrated servos and motors as opposed to human muscles which are more flat and stacked with better weight distribution? Is this a matter of speed? Human muscles can simply move quicker and harder pound for pound? Or just a lot of these factors? Another note, Digit appears to take many more steps than necessary. A human would be capable of spinning in one motion, whereas Digit takes many steps. How have you factored in the movement of the robot into its design? Again, it looks rather unstable with poor balance. Perhaps we might design electronic muscle fibers?

[@rasmasyean]

Animals have ball joint "motors". Robots have really... single axis Motors, no matter how you arrange them. We have to use what's available in technology to try to make it do something useful. Shadow has made balloon muscles, but it didn't turn out great. Real muscle fibers are molecular machines and we don't have anything close. Lol

[@thorwaldjohanson2526]

It is mostly a control problem with a incredibly large and often unknown number of variables. The human / animal nervous system is processing and controlling so many stimuli and muscles automatically. It is hard to model and train. Just think about standing still. You use pressure nerves in the skin of your feet, the inner ear, your vision to sense, and you have hundreds of muscles in your whole body that are finely actuated to keep the balance. Not just the legs, but arms, torso etc. As well.

[@rasmasyean]

@@thorwaldjohanson2526I think you give too much credit to biological "control". Just look at the Segway. There's no way any animal can control something that fast in "ONE DoF" even. Ultimately, robots will move a certain way, and humans another. That's the best for them anyway to accomplish their goals, unless your aim is to make a replicant to pass off as a human for whatever reason.

[@scottnewton9046]

Curious if some form of Asimov’s Three Laws of Robotics are being considered as part of the base firmware, at least the First Law.

[@Danuxsy]

The laws don't work, I think that is the entire point of Asimov's books.

[@heinrichwonders8861]

LLM... sounds like "Lilim". A name quite fitting, I think.

[@stanislavdidenko8436]

Nice talking skills, but no sigar