Last year, I saw your augmented reality lecture and found out that you only make very useful videos! I'm currently a KZbinr who makes Android classes, and I'm still interested in your videos! Thank you for always making good videos.
@geri43674 жыл бұрын
Thanks for reminding me why I dropped ML and focused 100% on gamedev (:
@MatthewHallberg4 жыл бұрын
omg its the worst haha
@pecke863 жыл бұрын
sick man ! i was wondering about this process for a very long time. You just created the gate of the metaverse !
@magefront14853 жыл бұрын
You can use colab which has a GPU better than 2080Ti, just put all files in google drive. Colab comes with TensorFlow installed by default. Synthetic data on machine learning is definitely doable, there are some papers using blender to generate these training datasets. To address the poor results, by default all convolution-based networks tend to learn the texture of the image instead of the shape, maybe unity's texture is not that photo-realistic. It's very difficult to train a model from scratch, the common approach is to do a transfer learning, where you take a pre-trained model of a large dataset like imagenet, then unfreeze the top and bottom layer, tuning on your data. With 5000 images to train from scratch, you won't get good results unless it's a super simple classification, like 28x28 handwritten digits. Since it's synthetic data, might be better to just do a pixel-wise label instead of a bounding box label.
@UnofficialEngineering4 жыл бұрын
Fact: there is not another KZbinr out there as innovative as Hallberg.
@MatthewHallberg4 жыл бұрын
Love you bro
@marcojoao4 жыл бұрын
You can improve using enhance image technique, and transfer learning to improve accuracy. The EIT will just stretch, rotate, drift and flip the images, and the transfer learning will help you to go at 95% accuracy
@robosergTV2 жыл бұрын
EIT?
@Augmented_AI4 жыл бұрын
Well done bro! Great work!
@MatthewHallberg4 жыл бұрын
Thanks man!! It finally worked...somewhat lol.
@surajitsaikia16533 жыл бұрын
Very amazing work. You can also rig or animate the character to generate a larger dataset. For instance, while animating change the camera views and collect the data
@ilirvg4 жыл бұрын
Amazing!!! Never seen your tutorials before and I usually do not Subscribe on KZbin but you just got a new Subscribe (and a fan)
@minhdungdo45412 жыл бұрын
Amazing Amazing project. With outstanding informative video, I can't hardly thank you enough but still. Thank you very much for the helpful video. Would love to try this.
@richardbeare114 жыл бұрын
Lol I had this exact problem with the bounds! I used basically the exact* same approach you did with the mesh sweep. Also, I think that's sweet how you tuck your laptop in.
@bram_adams4 жыл бұрын
Your hard drive must be massive to hold all that data!
@MatthewHallberg4 жыл бұрын
Haha MASSIVE. And you know what they say about guys with big hard drives...
@jackcottonbrown2 жыл бұрын
That is commitment. A+ for effort.
@wanderstudi4 жыл бұрын
Idea: maybe try the 3d scan in a completely diffuse lighting setup. So you do not have hard shadows on the textures of your 3d scans. I think that could mess up the recognition of the shapes. Or did you try that already? Anyway , apart from the bulbosaur(?) (not a pokemon expert here) recognition seems quite good already.
@MatthewHallberg4 жыл бұрын
I did try that I just didn’t film that part I still did get some shadows though it might have helped if I had a box with the correct lighting setup and then I could spin the object inside.
@abudriaz96784 жыл бұрын
Always loved watching your videos❤
@MatthewHallberg4 жыл бұрын
thanks!!
@cheesiangleow47824 жыл бұрын
Great video ! What apps did you use to scan object to 3d models?!
@mrsbtheo.a.p26354 жыл бұрын
Impressed. Keep up the good work. Appreciate your insight.
@eessaaabrahams91243 жыл бұрын
should have used a bunch of different pikachu and bulbasaur images instead of the same ones, then your algorithm would learn that they come in different forms, shapes and sizes
@_wise_one4 жыл бұрын
You could have used Google colab, they give 12gb ram and GPU/TPU for free. You can keep it running even if you close the browser or shut down the computer
@iliesouldmenouer49764 жыл бұрын
" i have no idea what that is, And i don't even want to know " - MatthewHallberg 2:30 . . . . Much love and support for you bro , such a great content
@johariawang77134 жыл бұрын
i use vuforia app to scan the 3d object and use it to create augmented reality app just like this one... i think it provide better experience and faster workflow..
@MatthewHallberg4 жыл бұрын
It definitely will but my goal here was to create a generalized tracker that recognized more than one style of the same object... I just failed lol.
@DInfinity38 ай бұрын
Super!!
@hamzzashaffi4 жыл бұрын
Great video as usual! I'm also planning to start ML sooner :)
@MatthewHallberg4 жыл бұрын
thank you! Yeah do it so you can teach me lol.
@hamzzashaffi4 жыл бұрын
@@MatthewHallberg guess what? I haven't started it yet lol.
@MatthewHallberg4 жыл бұрын
Hamza Shafi haha me either
@hamzzashaffi4 жыл бұрын
@@MatthewHallberg ayy bro
@tharaniv62673 жыл бұрын
@@MatthewHallberg Hey what app u used for photogammetry(scanning 3d model)
@karandeepdps14 жыл бұрын
You need to remove the green bbox outline from images and then it will work.
@rogueyoshi4 жыл бұрын
you should join the Two Minute Papers Discord!
@jerinpwilson52884 жыл бұрын
@MathewHallberg Hey nice project.. what app did you use for photogrammetry? To make the model...please reply.
@tharaniv62673 жыл бұрын
I also have this doubt
@abhishekgoyaldev3 жыл бұрын
@@tharaniv6267 Have you even watched the full video? 11:16
@tharaniv62673 жыл бұрын
@@abhishekgoyaldev thanks
@jonalex4 жыл бұрын
Was recently asked to estimate the work need to make a mobile app that could detect specific (known) objects. I remembered watching this video a while back. Can some of this be combined with your previous ML video? Would more computer power be a partial solution to this? Would more lighting variations enrich your data set? Thanks so much for this video. You're my go-to guy for all those off the wall projects.
@Pathorian3 жыл бұрын
Wondering how hard this would be using 2D Data instead of 3D Data
@swannschilling4743 жыл бұрын
Awesome!!
@elidorvarosi9643 Жыл бұрын
Would reccomend to just use some cloud instance instead of a laptop GPU as it will usually start to throttle after a while. Also, most of those frameworks works better on Linux due to the fact that virtually all the server do run on whatever linux distro was chosen by the user(usually Ubuntu or Debian like distros, sometimes Red-Hat based etc)
@jeeteshsingh2094 жыл бұрын
Kudos to ur efforts man! 💯💯
@MatthewHallberg4 жыл бұрын
Thank you I tried lol, almost didn't make this video for sure.
@denzilstudios70723 жыл бұрын
that kickflip
@PlasmaSabre3 жыл бұрын
Have you heard of fractional factorial testing? Might allow you to change multiple variables at once and run far fewer tests.
@argmentum222 жыл бұрын
prototype with 2k pictures... make sure the item you want to identify is 100% in your frame in your training pics. Don't put two items in your picture - the AI will potentially try to think there's supposed to be two; like lenses in glasses. check the the type of tensor flow model is right for what your trying to do.. some are very quick but have a lower success rate.
@saifking75804 жыл бұрын
What is the name of mobile 3d images app
@utsavgupta46304 жыл бұрын
Literally loved this .....😍😍
@xXMaDGaMeR Жыл бұрын
very cool vid
@DrRehanZafar2 жыл бұрын
Great video
@rizzbod Жыл бұрын
sooo coool
@dudenarima25283 жыл бұрын
you should use for loop inside for loop inside for loop... instead of random :D idk anything about ml too
@manojmadushanka93564 жыл бұрын
hard fan from sri lanka you AI man
@evelynjunco45744 жыл бұрын
Hi Matthew, I have been following your tutorials for quite a while. I’m trying to do object detection of real objects with an iPhone. I know iPhones, Unity and window don’t always work well but still want to give it a try. I can use photogrammetry for real objects but I’m not sure on how to do machine training once I have these images. Any suggestions would be appreciated!
@samvidjhaveri6344 жыл бұрын
Just FYI, Google has a new API named AutoML. It is much easier than their old Vision API which is hard to use. But it cost $$$$.
@ashokkillo4 жыл бұрын
nice video with details.. thanks for showing.. whats ur laptop specs?
@Kaushik-eo4ll4 жыл бұрын
First like and first comment broo ❤️😘
@MatthewHallberg4 жыл бұрын
MY DUDE!
@ko-Daegu4 жыл бұрын
Am so confused when can I start learning about ML ... Do I need like probability theory and statistics class first after that intro to AI and read few books after that I can jump to ML What introductory courses would you recommend Side note : I’m coming from 2+ years with java and about few months with python and am learning flask now (just for the heck of it ) Do you think am ready to start doing stuff with tenser-flow or scikit maybe ??? And which one to start with ??
@MatthewHallberg4 жыл бұрын
I literally have no idea on that one I never took any courses just started playing with tensorflow and following tutorials which is probably why I have no idea what I am doing in this video.
@annabelgroenenberg94484 жыл бұрын
You have a very good machine learning course on youube by Andrew Ng. It's the ultimate starting point. You don't have to know a lot of probability stuff but know the basics like false positive, false negative etc...
@cintianakano53394 жыл бұрын
Good job!
@clifflin71494 жыл бұрын
great video
@maknien4 жыл бұрын
Did you have the green borders in all your rendered training images (7:57)?
@MatthewHallberg4 жыл бұрын
That was just for the video I didn’t actually train like that lol. I am dumb so that’s a valid question but not that dumb haha.
@maknien4 жыл бұрын
@@MatthewHallberg Good :D I thought for a while should I even ask... But for the geometry randomizing, you should definitely check Houdini. I've been using that for all kinds of synthetic ML data stuff.
@MatthewHallberg4 жыл бұрын
@@maknien INTERESTING, checking that out now thanks.
@hosammohamed71074 жыл бұрын
great stuff bro, i was searching to make something like wanna kicks app. Do you have any idea how to track the feet and put the shoes on legs like that? i know it's AR but i don't have any insights about how to do it :(
@robosergTV2 жыл бұрын
dude, use Unity Perception package..........
@ZiyueZhang09244 жыл бұрын
you are so great
@AllanPichardo3 жыл бұрын
Your model was overfit because you only had a few thousand images. Next time, print the summary from your model and take notice of the total number of parameters in the model. Then try to get 2 to 3 times that many image samples. You should get decent results then.
@brunoomardorivalgutierrez11414 жыл бұрын
Hi Mattehw, How can I do to enter a augmented reality house but first put the house in a specific place and then scale it?
@AnkitSingh-wq2rk4 жыл бұрын
Hey Matthew I had a question is there any possibility for hand detection in unity ? (without that paid plugin of OpenCV) I had a project in my mind where you could interact with models shown by vuforia's image target using bounding box coordinates and 3d model coordinates ... i have done some workaround by scanning the hand through openCv in an external script and then sending the coordinates as packets over UDP to unity every frame .... but the main problem is it is not feasible for mobile phones :(
@Jack-oq7rg3 жыл бұрын
is it possible to feed ReferenceImageLibrary remotely ? like download the images from remote server with the 3D prefabs to place
@DMTravelCinematography2 жыл бұрын
How do you draw the bounding box around the object it is detecting in-house object? I tried using renderer.bound but it does not work. Do you have samples?
@vasusraj4 жыл бұрын
How to place 3d model on detected object in unity3d with tensorflow.
@waleedough2 жыл бұрын
I am new to the package of perception from Unity and my experience it's just for game designing, please if anyone has experience in such thing like this, I need help
@samgrogan88154 жыл бұрын
I think you overfitted to your training data which may be why the first model didn't work. Cant be sure though.
@MatthewHallberg4 жыл бұрын
I was definitely getting that vibe but I don’t know how to tell for sure.
@annabelgroenenberg94484 жыл бұрын
@@MatthewHallberg Look at the loss function via tensorboard. If your training data loss curve jumps to a high accuracy while your validation curve is way behind, you're probably overfitting
@samgrogan88154 жыл бұрын
@@MatthewHallberg Really depends on your setup. But I think with tensorflow there is a callback in the keras library for early stopping and you just tell it to stop if its not seeing inprovement after so many steps and that can help avoid over fitting.
@djone76723 жыл бұрын
Can someone please explain what type of deep learning method this process uses? is it CNN?
@neverninetofive Жыл бұрын
Yes
@Caio-Mendez4 жыл бұрын
I need help with the coronavirus ar app
@aashutoshdabhade43253 жыл бұрын
one like just for ur Efforts! I can feel the pain.
@waleedough2 жыл бұрын
I wanna do keypoints annotations, anyone who could help me on that please???
@manaskumar25444 жыл бұрын
You don't neet 20000 epochs to train the model with such less data,.. it's over fits the model and the model can not predict all the labels with equally accuracy!!!, You have to train untill the loss is stable,.. thats it!! When your detecting every pichachu in the world,.. you need different types of pichachu with different background images,..... Don't worry about the huge data.., all you need is 1000 images each!!,.. and epochs not more than 500
@Unpopular_Facts3 жыл бұрын
someone knows of any tutorial
@tronpig4 жыл бұрын
Add jarvis to the play store!!!!!
@XRDeveloper-014 жыл бұрын
Why... 😑
@MatthewHallberg4 жыл бұрын
Not what you wanted??
@XRDeveloper-014 жыл бұрын
@@MatthewHallberg I hate machine learning 😅
@MatthewHallberg4 жыл бұрын
Hahah same
@tylersnard3 жыл бұрын
You have to use a GPU to speed it up. Try training on Google Colab. Also, tensorflow sucks, use PyTorch :)
@camdenparsons51143 жыл бұрын
two words: transfer learning :0
@MatthewHallberg3 жыл бұрын
Yeah this is transfer learning lol did you watch the video?
@camdenparsons51143 жыл бұрын
@@MatthewHallberg yeah haha I watched it. I was sugesting that you use a pretrain model and retrain the last or last few layers to save many hours on training. thats how the watson and google object detection APIs work
@stephanverbeeck4 жыл бұрын
Great vid, too bad you have no real computer :-)
@MatthewHallberg4 жыл бұрын
Haha yeah
@joemoulton18234 жыл бұрын
Lol "It was so hard to figure out how to convert bounding boxes from world space to screen space...". Really?
@MatthewHallberg4 жыл бұрын
Joseph Moulton I wish it was that simple lol Unity has a function for that.
@joemoulton18234 жыл бұрын
@@MatthewHallberg Instead why don't you manipulate the camera to look at the model and then window the frame? Then take the snapshot
@MatthewHallberg4 жыл бұрын
That’s interesting never thought of that, I guess the only problem would be getting multiple objects in the same image cause that helped the model a lot.
@haraldgundersen73034 жыл бұрын
Hope some loaded dude gives u a quantum computer... You might even find a way to prevent future AI to wipe out humanity...