Why I Built This Robot

[@philippk736]

5:30 Love, how the cup is briefly categorized as a toilet...

[@bastianfrei5447]

and then as a dining table

[@DylanLisansky]

He was categorized as an umbrella and him and the chair together were categorized as a couch at around the same time.

[@Johanss-on]

It got drunk for a sec

[@CaliHime]

and here I'm wondering why James would put his printer on a suitcase...

[@hugodeandres1497]

He was also a dog 10:41

[@nielsencs]

"And although I'm not a great software developer, I feel I can copy and paste well enough to get robots to do what I want."! 😆👍

[@markmarney622]

A man after my own heart

[@BrandonPoulton]

Thank you! This sentence makes me feel more adequate.

[@devilduckietu]

Ah, yes. The most important software development skill.

[@youjustlostthegame2028]

I feel called out and I'm not even past college yet.

[@nielsencs]

Of course it is a bit more than that!

[@koosdeboer798]

It's really insane what this guy can make. Even if an intire company was making these robots I would be impressed. Hopefully you can inspire more young people like me to do more with robotics!

[@stevensteltenpool904]

@James Bruton , the Realsense library supports converting from one sensors pixel/space/point to another sensors pixel/space/point fia the sensors intrinsic/extrinsic. This way you can completely remove the parallax error between the rgb sensor and the depth sensor when a depth is found for that pixel (the first IR camera is the center of the depth sensors). The nice thing about these cameras is that they are per-calibrated and you can request these intrinsic/extrinsic for each sensor.

[@almondtech]

I just realized I've been following you since 2013-2014 or so, before I even had a YT channel to subscribe. lol Keep it going!

[@prakharjain2548]

He completely read out 'Thank to the 'DYNAMIXEL XC430-W240-T'' lol. Inspiring stuff :P .

[@alexisveynachter2699]

@james any follow up for this robot ? No manipulator planned ?

[@redd_rl]

Feels good to be so early to a video.

[@michael4576]

@@kevinbissinger it's a rush!

[@scottmudge2580]

The correct way to map RGB data onto the depth frame is to find the rectification transforms between the two optical systems (RGB and D). OpenCV (required by ROS) makes this incredibly easy, as it has very turn-key method/functions for this. You'll need to print out some kind of target (checkerboard grid or dot grid), and then invoke the required OpenCV methods. It will then spit out a few matrices (including those for lens distortion) that you can use to call a different OpenCV method to rectify the RGB image based on the depth camera's optical properties. The RGB data will then be mapped pixel-to-pixel (or very close to it) with the depth camera. Hacking static offsets without using any lens distortion coefficients (which an be significant in these Realsense Cameras) is going to cause a lot of problems for you later when you're trying to track and target objects.

[@mozkitolife5437]

James Bruton: fulfilling every man's dream of retrieving a beer from the fridge without leaving your recliner.

[@kevinmcaleer28]

I like how you're using the weekly KZbin videos to drive your knowledge of Robotics so that you make use what you've learned for future project - I do this too!

[@joshuadsuper101]

You should check out the MIT Kimera semantic SLAM stuff. I just got it running tonight and it’s pretty cool stuff. I would love to see you playing around with this powerful tool.

[@joshuadsuper101]

It’s a collection of programs which can do mapping, localization, in real time as well as label the map by what type of object it is ie floor, table, etc. There have since been improvements and more added functionality but I’m just learning about it now. I definitely think you would appreciate taking a look at it

[@jamesbruton]

Is there a ROS node/support?

[@joshuadsuper101]

@@jamesbruton Yup it’s all ros based!

[@Keneo1]

You could train your model to recognise the robot hand and then find the vector it needs to move in to touch the cup, then you can make it move relative to the cup and don’t bother with the Carthusian coordinates You’ll probably want to do this anyway when you have a grabbing hand and different sized cups to grab

[@SoundsLikeOdie]

Good idea. Maybe add some aruco markers on it. Then he will have location and orientation.

[@ziad_jkhan]

There is potential for eliminating the need for encoders on the motors with this method as well

[@evannibbe9375]

Vectors necessarily consist of Cartesian coordinates (as far as deriving one from the other goes).

[@Sustaita.handmade]

I may not know much about robots, coding, or engineering but I enjoy your videos and seeing how you solve problems. I hope that by watching your videos I can learn even just a little bit about robots over time.

[@makecodeandhardware1395]

Subscribed, our students ages 9-15 yr study these concepts and imagine an automated Starship factory; Mecanum wheels and various communication protocols are studied to better understand smart devices. Thank you for your practical research : mobility and automation.

[@ashaygoli3014]

Amazing!!... You can use moveit for the arm which can handle all the interpolation and obstacle avoidance.

[@MattJoyce01]

You're building a robot to fetch you a beer, we all know it, and frankly it's what robots are for!!!

[@jamespicking1140]

How do you know it's just to fetch beer? Have you ever seen that episode of Big Bang Theory with Howard and the robotic arm? 😭

[@haydonwolfe7536]

“And although Im not a grea software developer i feel i can program well enough to get robots to do what i want”

[@bigsteve6729]

That's not what he said

[@haydonwolfe7536]

@@bigsteve6729 Alright so the joke is I swapped the words "copy and paste" for "programming" because a lot of people say they program (myself included) but use copy and past (me also)

[@NeoFrontierTechnologies]

Fascinating. The right channel to watch when you like to tinker with robots or develop something of actual use. Totally made me subscribe.

[@richardlincoln886]

Any milage in going full human mode - i.e. visually detect the gripper and rather than trig to theoretically put it where the object is - Visually match the coords. Thus any error in the Kina model would null out.

[@Ithirahad]

Humans don't really 'visually match the coordinates' either; we run on both visual and kinesthetic/vestibular senses to error-correct one another constantly.

[@khalifdove7955]

Caught it 42 minutes after upload what a wonderful number

[@khalifdove7955]

Thank you very much for uploading!

[@richardbloemenkamp8532]

This may well become your most successful project ever. Some ideas: - Try to remote control your robot while you look at the video of the camera and using the remote control. If you can do useful things like that then in principle a good A.I. should be able to do similar things. - As you move the end-effector closer to the target, decrease the speed and detect both the target and the end-effector (you can put a special easy-detection marker on it) with the camera. This will allow you to control the last part of the movement much better. - It might be interesting to have a Pybullet or Gazebo or other model for your robot and environment. - Think about a battery-charging solution like the vacuum-cleaners it can return by itself to the charger bay. - It appears there is a small risk that your robot falls over. Don't make the top part too heavy and try to displace heavy elements to the base. - Think about real applications of your robot such as: help people that have difficulty to walk, have the robot walk around the house or company when you are on vacation, clean, water the plants, check/feed pets, get a beer from the fridge, play games, etc. Good luck

[@joannot6706]

2:55 Story of my life lmao!!!

[@louisepage716]

Wow! James, you are an inspiration. So humble, open and impressive. I’m very happy I found your channel. Awesome stuff!

[@sddiymakeitworthit7512]

Your skills is awesome 👏 👍

[@nonchip]

i like how at 10:50 the robot randomly decides that a cup plus your legs equals a dog... might be helpful to give it some kinda "short term memory" by having it bias the detected object "certainty" in favour of ones it saw in the previous few frames at the same position? like instead of each frame taking the "most likely result", try keeping a record of all the things it thought it might be over the last few frames and the percentage it calculated for its certainty, then average those out and sort for the average certainty to get rid of temporary glitches?

[@tanjiro3285]

hey James I came back to like and comment this Great informative video 12 hrs after downloading. Edit: I am really promoting ur Channel 🔥 Brother

[@rizkyp]

What my family think of me: Robot Genius Me: copy paste random codes from github.

[@RND_ADV_X]

I just realized... James is a young Rick Sanchez, and this is the butter passing robot 😉 (j/k) James, I have an idea for a project:. I'm a dj and music producer, and I love all the fun tech there is out there for electronic music. So, you know going from a traditional piano keyboard to a drum machine is stepping up from 1d to 2d, and adding knobs to a drum machine makes it 3d editing, so to speak? What if you used digital object recognition... To create a music device that uses dancers' body to define and edit procedurally generated music? So instead of an audience dancing to a song, the song creates ITSELF based on the people dancing? I dunno... It's weird but I think you might get a kick out of the idea... And I don't know ANYTHING about object recognition. So I'm gifting the idea to you if you want it! Also I'm getting kind of excited to see where your latest dog bot project is going!

[@michael4576]

Love this idea! I thought along the sane lines during a daydream many years ago... Back then, the technology didn't exist to realize the concept.

[@twobob]

Pretty sure this exists fella. Sorry.

[@RND_ADV_X]

@@twobob where? Even if it did, that doesn't mean someone couldn't try to do it BETTER. James didn't invent ROBOTS, and yet here he is enjoying building his own.

[@ThatGuy-fi9bm]

@@twobob ditto on where. That sounds cool, I want to look it up. You can’t leave us hanging like that.

[@twobob]

@@ThatGuy-fi9bm Well I have seen several projects along those lines. Some follow the dancers shapes with camera, some react to the various relatives movements of markers (or whatever fiducial ), all you are saying here is convert object recognition into the "fiducial". This is not even remotely hard. Other than lacking a UI for noobs this already seems like a very done thing. Frankly it's just the The Chemical Brothers - Star Guitar video but in reverse.... (Look it up). If you want a device that is probably "ready" to add a UI look at the studio version of Ableton. If you want specific links to some of these aforementioned projects I can look them up. All the research into midi data gloves /pretty much/ covers 90% of the code you need to get this up and running "as your vision". Couple of days project with a Nvidia Jetson

[@kristofferhellstrom]

Is James a genius or what? I'm blown away

[@jonathanmoothart8038]

Nothing but awesome videos from this channel! I love your stuff!

[@vaisakh_km]

If(probability (person) >96){ Kill(); } Oops! Syntax error, i ment to use ' Skill()'😱

[@parthd714]

Wait, I didn't subscribe?? How did I not? I thought I subscribed cause you are always in recommendation, whaever imma sub

[@Saxandviolins]

Crazy inspiring!

[@EvilSpyBoy]

Silly Question (because I haven't finished the video yet) - Have you considered adding an identifier of some sort on the robot arm so you can object rec on the 'hand' to give you some relative information to play with too? A QR code or something as a sticker on the joints for registration like Mo-Cap suits

[@jamesbruton]

Yes potentially - it would help a lot with the left-right detection once the arm is in view. I need to make a gripper first though

[@AsbjornOlling]

This is amazing The title really undersells the video.

[@knightdeluxegaming]

Can we get playlist of your ai robot videos. And how i can start with cheap machine vision microcobtroller

[@BlueScreenCorp]

I think its hilarious that the algorithm has a 60-70% confidence that a cup and a pair of jeans is a dog

[@alex.thedeadite]

The Purple walker you made looks like a half dismantled GONK power droid from star wars

[@charlesashurst1816]

I'm counting on you guys to have robots ready to take care of me in my declining years. Keep up the good work.

[@sammflynn6751]

You may want to look at *open3d* for pointcloud processing, it has builtin function which can segment objects from pointcould and find centroid from camera refence frame. Also , why not use ros TF info for co-ordinate transformation?

[@MuhammadDaudkhanTV100]

Amazing idea

[@oscar8005]

Are you going to put the Omni directional drive system on this project? Seems perfect for it

[@jamesbruton]

Not for this one, but it's an option for other robots

[@thomasre8073]

This and the robodog projects are my favorite.

[@interestedinstuff]

That was very groovy.

[@amirbahadormoineddini1639]

Love it, Amazing!

[@JulianMakes]

You are so awesome james!

[@faruknane]

Did you try to detect the arm of the robot with cam? Then you can just make a feedback mechanism for motors to move the arm to under the cup?

[@maxschik4684]

small tipp, real sense cameras should publish a topic called "aligned_depth" or smth which is aligned with the color image, so you don't have to do that yourself

[@Aubron]

Please never, ever stop using that B-roll of you and the Wolverine claws. Please. It should always be shown in its entirety, and with audio. Thank you.

[@extremeroom7u7]

10:44 robot is like dude that's a cup robot brain no is a dog well then is both

[@vainerfever8776]

Woah so cool

[@dineroparson6778]

This man needs to go to battlebots!!!! This intelligence and technology in a lifesize heavyweight would definently dominate it.

[@rdyer8764]

You're a Brainiac. In many quarters (including those I inhabit) that's among the greatest of compliments.

[@ayanvaidya2727]

Really really useful

[@thescienceysworkshop5368]

Sir can you please tell me which modules did you use in your omni wheel machine please? I didn't quite understand it

[@nilsdock]

why not use the reverse kinematic model to calibrate the position of the camera. then the arm could be fitted with a easily recognized pattern. then the arm could increase the precision of the movements by "hand eye coordination"

[@Metal-Possum]

Does it piss beer into a cup?

[@scottr3237]

He's coding the robodog to do that.

[@arnaudherault9168]

james you are the best

[@r.iyushofficial5318]

AWESOME 👌

[@woolliedev]

Wow which things do you use for it I want to make a kind of robot with a arduino it sound really cool to me

[@Sirenhound]

Robot... red cup... for a moment I forgot which channel I was looking at and thought the robot would be pissing beer into the cup.

[@enaudeni]

Hi James great videos, I am always amazed at what you get up too and build! I have a "Big Ask!" have you ever thought of doing a series of how to build an easier robot / hexapod / or something else for us technical wannabe's or for kids, yes the kids!!?!! Do it for the kids!!!!! Most people have 3D printers and a basic knowledge of Arduino, so if you had to do a series based for "kids" or even the kid at heart, a project broken up into small chunks so there is a better understanding of how it works and how it's programmed, building up to a meduim project when all put together. OK yes there is many tutorials on KZbin; on Arduino; on programming; on 3D printing; on designing; etc.... But there is only one of you, considering you started before KZbin and all the robots you have made you have the perfect experience and long term working knowledge. You could even sell a series like that! Just have a think about it, mention this to those who know you and see what they think. Anyway thanks for all the content and sharing your passion with us. Much appreciated.

[@Emile_v]

oh cool idea

[@tantamounted]

James, have you ever experimented with virtual biology, such as computer-simulated bodies, bio-inspired robotic functions, or other such things? I just have this hunch about creating instinct-driven AI in part through simulated bodily function.

[@MrSeanana]

Hi James, I was curious if you had any recommendations for adults with kids who want to get into robotics? I know my kids would love it but this is a request more for me instead of my kids. I want something that is pretty complicated but teaches the fundamentals really well. Any ideas? Thanks.

[@qvcybe]

like how for a second the cup was called a toilet XD and later the person turns in to a dog for a second man i love tech stuff like this

[@AMTunLimited]

James, you should look at how RC airplane pilots strap their remotes to themselves. I feel like the way you're doing it could lead to some strain

[@pebre79]

I want a robot that will cook me cuisine for breakfast, lunch, and dinner. Just biding my time and i cant wait! Haha

[@webjoeking]

Bring back the 914 PC Bot!

[@timschafer2536]

Would it be beneficial if you would use camera position of the arm in a kind of Feedback Loop to detect the delta of the two?

[@nmanbamboo1980]

Copy and Paste ... yessshhh.. you are not alone.

[@OZtwo]

Have you given any thought in looking into building a robot using RL? I'm still new to all this yet I really want to look into using RL as then the robot will seem to evolve more as it learns..I hope anyway.

[@jamesbruton]

Yes I have a project coming up like that

[@diegocarvajal233]

I'm also starting on ROS and isn't it supposed to be able to calculate the transform between the camera frame and the arm for the inverse kinematics?

[@TS-kg4lf]

Yes, if you published the end_effector

[@syedsulaiman8380]

Amazing

[@wilgarcia1]

well damn man. Its really moving along. Have you considered using binocular vision as opposed to these cameras? I have only seen one or two examples. But hat was years ago. I like it cause it's more like human vision and you just need a set of cheep webcams to make it work.

[@katherineespedido4978]

This is good

[@Genubath1]

Binocular vision!

[@manojsk6189]

Super

[@skuzlebut82]

Here I am in the US. I should be going to sleep but this video is more important.

[@AHHEUS]

got this in recommended cool video you gained a subscribe

[@jamesbruton]

thanks!

[@AHHEUS]

@@jamesbruton yw

[@VLSystems]

Hi James, i have robot with ROS. What should I do if i want to remote my robot with internet?

[@mcdevel1]

0:40 Gonk droid from star wars xd

[@giacomomilan8692]

Why the software doesn't remember the object, but only recognize it? For example the cup, sometimes il less than 50% shure that it's a cup even if 1 second before it was shure at 80%. If the object didn't leave the field of view it should be the same as before

[@jamesbruton]

That's interesting - I guess it's down the pre-trained model that detects any cup, I could retrain it for this specific cup and it would be a higher % certainty. Ultimately it's doing the detection on every loop from scratch so it will vary as I move the cup around.

[@sleepingneeds2588]

Question, could you make a program that is able to remember the layout of the house and can track where you are in the house to come to the area? or would it be too hard

[@darkener3210]

This man about to home brew a robot person

[@chiragscience9434]

Wooooo ✨✨

[@kabelloseskabel7029]

Hi, I want to build something similar, but with more objects that I want to manipulate one by one. Is this possible?

[@MEan0207]

Awesome

[@Car_Ram_Rod]

Ugh i want that nvidia jetson!

[@naiyahp]

I like how no one controls your robots. 🙂🙂🙂🙂🙂

[@oumarcoly4454]

Good job❤🇸🇳🇸🇳🇸🇳❤❤❤🇸🇳🇸🇳

[@jstro-hobbytech]

The former 400 dollar Xavier is now being msrp agonstic aka (enabled scalping) by nvidia for over 1000 and the jetson that used to be 150cad is now 4 to 800 depending where you buy it. I can't imagine how much the ampere/hopper ones will be given they've eol'd the Tegra x1 based on maxwell which the jetson b01 runs on. Nvidia is catering to higher end customers. It sucks trying to develop a program as a volunteer teacher

[@Toaster1]

swag

[@Jkauppa]

single reaction wheel multiple degree of freedom gyro balancing multi-wheel, contra rotation

[@deanallenjones]

I know those are words...

[@TheTurmanDreams]

Great !!!!!

[@joewow1229]

He made the GONK droid

[@bonvi2896]

Dude, i'm afraid the same way delight that someday i'll handle a robot in whoose hand there is cutting edge and on the back we'll se "JB Design"

[@sethmcbride8490]

10:45 when your legs are dog

[@EduardoPereira-bd7ub]

Kinect+robot+gun= Skynet.