Unity3d with AR Foundation - How To Instantiate A Game Object Per Tracked Image?

In this video about Image Tracking With Unity3d and AR Foundation I show you how to instantiate multiple game objects based on the reference image detected. We also go through how to setup the reference image library as well as how the new script works when handling multiple prefabs.
Source Code:
github.com/dilmerv/UnityARFou...
** Support me by Subscribing to the channel as it will help me in bringing a lot more content and also as a way to show your appreciation for the free source code provided above **
bit.ly/2NmnFMu
** Find the source code in Patreon and support me if you can even if it is with one of the lower tiers as it helps me in bringing you more content - THANK YOU **
/ dilmerv
** EQUIPMENT USED TO RECORD THIS VIDEO **
- Rode Wireless Go - amzn.to/2J7RCw6
- Rode smartLav+ Omnidirectional Lavalier Microphone for iPhone and Smartphones - amzn.to/2LBQra6
- Rode SC7 3.5mm TRS to TRRS Patch Cable - amzn.to/2XTaYxv
- Blue Snowball iCE Condenser Microphone - amzn.to/2Ija0lo
- JOBY GorillaPod 3K Kit. Compact Tripod 3K Stand - amzn.to/2Xh4Bni
- JOBY GorillaPod 5K Kit. Professional Tripod 5K Stand - amzn.to/31Ew0yW
- Photography Photo Portrait Studio 600W Day Light Umbrella - amzn.to/2WwI0Tc
- AmazonBasics 60-Inch Lightweight Tripod with Bag - amzn.to/2Wd8utF
- Canon EOS Rebel T6 Digital SLR Camera Kit - amzn.to/2WcnRlV
Get realtime indie development tips from me in Twitter
/ dilmerv
Subscribe to my indie development videos & game releases newsletter
www.dilmergames.com
Blog about Indie development and startup
www.dilmergames.com/blog

[@ajt3h]

Awesome video - thanks! I'm using this as a way to teach my kids programming. We're building some AR board games. To your "rotation word" comment. If it helps: Quaternion is pronounced like "KWA-turn-ee-on". Besides computer graphics, they're used a lot for spacecraft design + control.

[@endorphingames2957]

If anyone has a problem with multi image tracking you can comment out these lines in the UpdateARImage method: foreach(GameObject g in arObjects.Values) { if (g.name != image.referenceImage.name) g.SetActive(false); } The updated is called for all trackables regardless they are in view.

[@MikeMcCreadyTheVRGuy]

That seems to work for me. Thanks for the info. What's the point of those lines then?

[@straylight2008]

Kwa turn ee on ! You're welcome :-) I only know this because the place where William Rowan Hamilton, the mathematician who discovered them, wrote the equation on a bridge up the road from where i live! I have no feckin' clue what they are or how they work though! lol! I just know they are really important for rotations in 3D space and Unity! Thanks for your videos! They have been so helpful to me right now. Patreon Subs on their way!

[@dilmerv]

Thank you John I appreciate it very much and also for the feedback !

[@EliSpizzichino]

They are very important for animation in general, they are not that different from 3dimensional Eucledian rotations, think about it as 4d where the added dimension it's a variable that multiplies the value of the axis (think about it as a sort of a mixer with values that goes from 0 to 1 for the x,y,z axis). The advantage of using Quaternions Vs Eucledian rotations is that avoids the gimbal lock for certain rotation where during the tween from one postion to another the values happens to hit 0. en.wikipedia.org/wiki/Gimbal_lock en.wikipedia.org/wiki/Rotations_in_4-dimensional_Euclidean_space#Relation_to_quaternions

[@judoTRON]

DUUUUUUUUUUDDDE!!!! You are DA MAN! Thank you so much for this!!!

[@dilmerv]

You are very welcome ! Thanks for your feedback :)

[@hareshbabuveldurty8110]

Nice Video, I am trying this multiple image tracking and place diff prefabs using ARFoundation from couple of days and couldn't make it work. Thank you so much.

[@dilmerv]

You are more than welcome and thanks for the feedback !

[@skyeparker5822]

I have a large (house size) prefab we are instantiating with this code and it is working well except that we need the prefab to be rotated correctly with the image. Right now the prefab is aligned to whatever direction the camera was pointed to at startup.

[@satheshn2396]

Hey... Im tryiing the same as you said, But DO i create a seperate script for initialising the array as im not able to get it done with the tracking multile objects script

[@DIEGOGARCIREBOLLO]

hey been trying something similar to this code on android, normally it works, it just that sometimes when the image is recognized the state of the tracking image is none thus the transform position always remains in vector0 so the object instantiate on said position. do you have any idea why the tracking state would be none?

[@nickchang6753]

Is it possible to combine Plane Detection and Image Tracking? For example, have the prefab associated with the image spawned on a detected plane?

[@grenoffe]

Hey, this series of image tracking videos has been great! I've been having some problems translating this over the the object tracking side of things however. Any chance that you have any ideas on the differences in implementation? Thanks man!

[@dilmerv]

Not sure what your question is, what part of the image tracking do you need help with? Do you mean by moving an object after it has been instantiated by the image tracker?

[@grenoffe]

@@dilmerv I've been trying to do things similar to what you did here, just instead of using a reference image library I've been using reference object libraries. It actually did end up working, but the recognition of the objects is just much slower and overall less effective than recognizing an image. I think the real problem I'm running into is how to get really high quality scans of objects that wont stop working when in different lighting conditions etc. On top of this, moving a reference object doesn't seem to update the position of the model linked to that object like it works here with images. (but I suspect that may be directly related to the above issue more than anything else) Thanks for the reply!

[@ambihike6630]

Hey Dilmer, I wanted to ask, instead of tracking real word objects, can we use AR Face Manager to track face and put a gameobject over the face. (Reply based on ARCore Reference). Just to add, I am not looking into MARS implementation.

[@Jack-oq7rg]

is it possible to feed ReferenceImageLibrary remotely ? like download the images from remote server with the 3D prefabs to place

[@kawerishirke5568]

Thank you so much!!!!!!!!

[@dilmerv]

You're welcome!

[@matteo_sacco]

Hello Dilmer , thank you for your tutorials ... it seems . under IphoneX , that the prefabs remains all visibile on the camera before tracking teh images.... it's seems that i need to disableall the prefabs in the asset folder BEFORE run or build the app...it's correct?

[@dilmerv]

Correct you could disable all assets on the awake method and then as you find the trackers enable them.

[@margaridalopes5810]

hey! Great video, but i'm having some problems! When I spawn 1 of the prefabs it always stays and when I move the camera to the other reference image, the prefab doesn't spawn :( PLz help!

[@o.pedrobenetti]

Hi Dilmer, I would like to know how I make multiple prefabs appear simultaneously depending on the image traced (I'm a beginner) Thanks

[@EdgarasArt]

Yeah... ARFoundation should take some simplicity examples from Vuforia, so that nobody would need to code for achieving multiple image tracking.

[@dilmerv]

Thanks for the feedback :) and yes I am sure it will get better !

[@DungPham-xe3wz]

tks for your turtorial. i have a trouble . you make project with detect one image and show one prefab. i want make one image but multi prefab. i will have a button back and button next to change multi prefab. can you help me resolve this?

[@HaganeKen3d]

Thanks for the tutorial, pretty good stuff and easy to understand. I have a question, Do you know a way to use AR image tracking + Asset bundle to download prefabs? this is posible with AR foundation, thanks again.

[@dilmerv]

Look into this classes docs.unity3d.com/ScriptReference/Networking.UnityWebRequestAssetBundle.GetAssetBundle.html and yes it is possible to download and load this at runtime.

[@AngelSevillano]

Hola Dilmer! Fantástico video igual que todo tu canal. Funciona bien en una única escena pero cuando salgo del juego y vuelvo a entrar o cuando vuelvo a esa escena después de pasar por el menú no me funciona, no se instancian los prefabs en las imágenes. ¿Cómo puedo solucionarlo?

[@gravityhologramsolutions3624]

Hello sir, I am following all your videos what algorithm is used to track Image in AR Foundation in Unity ?

[@seanmichaelbuison1787]

ok very useful

[@dilmerv]

Thank you 🙏

[@unbelievablenoye7178]

I wanted to know regarding ARTrackedImagesChangedEventArgs.removed, even if I create a foreach loop it's like it doesn't work. The cycle works like this: foreach (ARTrackedImage trackedImage in eventArgs.removed) { _instantiatedPrefabs [trackedImage.referenceImage.name] .SetActive (false); } , is it an ARCore bug or am I not getting it very well. To summarize, did you also have problems with the .removed property, when using it with ARCore?

[@dilmerv]

I don’t believe I encountered this issue with Android but I will look into it, thanks for your feedback.

[@mejdoubselsebil9873]

Hi, how can I make the generated objects keep their positions in the real world when tracked (like extended Tracking in vuforia), I would like to display and track multiple objects simultaneously? My objects disappear once they are no longer in the camera's field of view thanks

[@dilmerv]

You could first instantiate them when the image target is detected, at that point create an anchor and set the instantiated object parent be the anchor transform :) This way the objects created will no follow the image target, instead they will stay with the anchor. Let me know if this is what you were thinking :)

[@carlas1720]

Hello Dilmer, thanks for your tutorials they help a lot in creating AR experiences. Do you have any idea how to create an alert in UI that gets active when the user did not allow camera access for an ARCamera in ARFoundation on iOS?

[@dilmerv]

Hello Carla, take a look at ARSessionState to make sure the AR Session is available, I would say for now log it to understand each state and look at this docs docs.unity3d.com/Packages/com.unity.xr.arfoundation@2.1/api/UnityEngine.XR.ARFoundation.ARSessionState.html - optionally you could look at subsystem descriptors as they also provide additional functionality to determine what's available.

[@yokj9081]

@@dilmerv excuse me Can I ask some AR questions?

[@anubisalves548]

Hey Dilmer! I'm having a hard time on this one. When my scene start, both my prefabs appear at the 0,0,0 and are not updated when the camera see one of the targets (not the text is updated) Any idea why?

[@dilmerv]

Are you updating the positions of the Prefabs by wiring to the image tracker callbacks ?

[@littlec9431]

@@dilmerv hi Dilmer, It seems I have same question, how could I updating the positions of the Prefabs by wiring to the image tracker callbacks?

[@graysonscottnyc]

I have the same issue. Been struggling with this for about two days straight. My app is completely done and once I figure this out I can launch. Any thoughts?

[@graysonscottnyc]

I should also clarify that I am using the prefab collection you provided (Thank you very much) and when the app starts all prefabs connected to images appear at 0,0,0 (in this case four different prefabs) but disappear once the first image is recognized and from that point everything is fine.

[@dinesht701]

Hey Dilmer i tried with different mesh scaling & position is fine but rotation needs to be updated can u help me with that ?

[@dilmerv]

Sure I will look into it for sure and do a new video thanks for the feedback :)

[@meggang9566]

Hi, how can I get the spawned object to stay visible, I would like to show and track multiple objects simultaneously? My objects disappear so fast

[@HeadPack]

If you download the ARFoundation examples Unity provides on GitHub, you find a scene covering multiple image tracking. It shows you how to track them all at once.

[@dilmerv]

Thank you 🙏

[@meggang9566]

I already solved this problem months ago but I appreciate the reply... 😃

[@dnightwalker]

Cool video. Just had a problem with the code. For some reason ARTrackedImageManager isn't removing the image when it's outside my phones camera so there was a big logic problem that it would always show the object of the las imaged that was scanned. Fixed this by only updating the image if it's "trackedImage.trackingState" was set to "Tracking"

[@muhammadtalhaali2541]

could you explain please ?

[@pabloruizponce1957]

Good video. Do you know if its possible to destroy a trackable?? Im trying to do: public ARTrackedImageManager ar; ... foreach (var tr in ac.trackables) { ARTrackedImage.DestroyImmediate(tr); } But this is not working for me. Any clue??

[@dilmerv]

I did something similar for Face Tracking check this code github.com/dilmerv/FaceTrackingDemo/blob/258d7cdd6a36ffc7556012099d006afad05ec581/Assets/ToggleFace.cs

[@djsonic_pr]

Hi I have a question . How can I fix this. Once I track the images they wont re spawn after first pass. Its like it doesnt refresh o destroys the prefab

[@dilmerv]

That’s the correct behavior, it is how ARKit and ARCore handle it, once it is created you are responsible for removing the instantiated prefab, trackables are not destroyed.

[@sharatbabukrishnan9312]

Hey Dilmer, a huge thank you for all your videos. Just wanted know whether multiple images could be tracked simultaneously? For e.g. could the stuff in above video track all the three image markers and show the RGB balls all in single camera frame.

[@dilmerv]

That’s a good question let me do some testing and find out !

[@makaruk_music]

@@dilmerv Hello - Thanks for this tutorial. I also have the same question - can a few images be tracked simultaneously in the same frame?

[@dilmerv]

Cool I haven’t had a chance to check it out but I will tonight :) thanks for your feedback and interest my friends.

[@livasadovska3052]

@@dilmerv Awesome video, thank you! Did you ever get to test tracking multiple images at the same time, same frame?

[@revotiva]

any answers?

[@davidgarzonmoleon8782]

Hi Dilmer, I'm trying this code to instantiate a canvas in my images but when the camera starts appear one white image (in AR) in the middle and I don't know why. Do you know what is my problem? Thank you so much!

[@dilmerv]

That may mean the canvas is too large or the canvas is not set to world space, could you double check ? Thanks

[@davidgarzonmoleon8782]

@@dilmerv I see that is the same canvas that I instantiate in my target. If I start with the canvas disabled everything goes right. But the instantiated canvas is rotated in Y and I do not know why. Any idea? Thank you so much!

[@dilmerv]

Is probably because the code is using the image target rotation, try to set it to zero on the method that updates the target prefab position.

[@seekuro6415]

Hi, I wanted to ask how can I set an instantiated obj to false when the camera can't see the reference image since the .removed list is always empty.

[@mythsheep]

I have the same question. Is it possible?

[@jorgitomuxamarcha]

hi Marco, you need to use trackedImage.trackingState to know when your card disappears from the camera. when this variable changes from tracking to limited means that the camera doesn´t see or recognizes your card, instead of using the event .removed

[@BogdanAndreiRO]

@@jorgitomuxamarcha can you please help me with the code? thanks!

[@blabberbytes9967]

How do you get the app on the phone? I'm trying to develop an ar experience for the public.

[@dilmerv]

For which type of phone ?

[@premsagar5289]

Invalid token 'void' in class, struct, or interface member declaration i am getting this error

[@pupsie7671]

hi! where can i find the script that you are using? please help

[@dilmerv]

Thanks for watching and for your question, you can find all the scripts shown here or other AR Foundation videos at github.com/dilmerv/UnityARFoundationEssentials

[@pupsie7671]

@@dilmerv thank you so much!

[@burakrucoglu6049]

Hi there Congratulations on a very good project. I implemented the project exactly, but it's not working properly for me either. When I return to the old image after reading a new image, it cannot remove its object. I've been dealing with this for over a week. I couldn't get it to work, can you help me please?

[@cohenjota]

I'm having the same issue... can someone help?

[@revotiva]

Buen vídeo Dilmer, porfa ayúdame con dos cosas. Los dos prefabs me cargan en escena en 0,0,0 position sin haber encontrado el marcador. Como hago que no aparezcan desde un inicio? y la otra es, como hago para que ambos objetos se vean al mismo tiempo? gracias man, y felicidades.

[@tavomontoya]

hola, me pasa lo mismo. pudiste resolverlo?? saludos!

[@revotiva]

@@tavomontoya no. Y no creo que nos vaya a responder Dilmer xD

[@PEDRO4075]

Hi! Can you help me on how to make different objects appear for each side of the Vuforia Multi-Image target? Imagine a multi-target cube, I want to place different objects for each face

[@dilmerv]

Thanks for your question and yes I plan to do new Vuforia videos covering this topic in the near future.

[@PEDRO4075]

@@dilmerv thank you

[@gerardonovelo8098]

Hello Dilmer! Your videos have been very educative. I'm having an issue here -- prefabs only load on images that have not been scanned. If I try and go back to a previously scanned image, it won't load its prefab and I have to restart the app. Do you have any idea what's happening?

[@marcoross4169]

me too....the software doesn't work well ( on android).

[@djsonic_pr]

Me too look like well need an iPhone for this

[@MarcelloNisiEngineer]

is it possible to display a different video on each different target? and in case you can explain me how to do or make a video on purpose to explain this thing, thanks

[@dilmerv]

Definitely possible, I will consider making a future video about this thanks for your suggestion.

[@MarcelloNisiEngineer]

@@dilmerv when do you plan to do it?

[@MarcelloNisiEngineer]

if instead of the prefab with the sphere I put a prefab that contains a video, does it work the same? that is, the video starts?

[@djsonic_pr]

I have a problem , once I track the prefabs when I go back to the same color it wont re spawn why is this?

[@dilmerv]

Take a look at this example as a reference which I created for my AR Course, I do have a routine to remove the older objects if tracking is lost, you could do something similar: github.com/dilmerv/ARFoundationCourse/blob/master/Assets/Scripts/Features/ARTrackedImageContentWithLimit.cs

[@ypedeboer2342]

Hey Dilmer, I am trying to get a video to play in the AR space after tapping the screen, but I can't get it (the code) to work. My end goal is to have users click any video that they like to play and pause it. Now all my videos just play from start when you instatiate the scene. Could you make a tutorial about that? You've helped me and a friend a lot already and it would be highly appreciated!!

[@dilmerv]

Oh yes I can help you with that, I will work on it and count on it to be available in the next few days, I would say mid next week :)

[@ceriousofficial6084]

@@dilmerv Hero!

[@kuntaldinda6557]

Hi, I keep getting an error Failed to generated ARCore reference image library 'ReferenceImageLibrary' Could you throw some light on that ?

[@mohabbadran271]

I have the same problem ... Did you fix it?

[@mohitgupta7633]

@@mohabbadran271 make sure u r using png as an image extension.

[@mohabbadran271]

@@mohitgupta7633 Thank you bro, I fixed it by changing to path with no spaces.

[@kawaljeetsingh8882]

Hello Dilmer i am trying to connect this tracking with server so that i can add more tracking images and Prefab from webserver .can you please help ???

[@dilmerv]

What part do you have questions on ? There are many parts in the process of making that work.

[@kawaljeetsingh8882]

@@dilmerv in this video you have uses a single image for track and placed sphere on image . I want to make it dynamic so that i can add image from web server and object like buildings or character on image from the webserver.

[@dilmerv]

Take a look at this video to load images in real-time to the image reference library, then you could pull the images from a web server by using a basic rest client kzbin.info/www/bejne/l5eQp4xoh9N8otU

[@kawaljeetsingh8882]

@@dilmerv thanx bro ...you are great

[@EdgarasArt]

By the way ARFoundation + ARCore from multiple image tracking perspective is buggy. Maybe got something for that? The issues appears not only in your case specifically, but in general. The way you show in video with red/green/blue spheres looks everything ok (iOS), but on Android device(s), tracking is poor or tracking even stops, content doesn't appear/disappear well, in short, works like s * i t.

[@dilmerv]

Definitely needs improvement and I will be doing new videos with AR Foundation 3.1.0 which has improvements. Thanks for your time and feedback !

[@EdgarasArt]

@@dilmerv Actually, 3.1.0 doesn't solve the issue.

[@luchy78ar1]

Videos en ESPAÑOL. Gracias

[@dilmerv]

Gracias si en algunas semanas planeo empezar !

[@MaverixGamesStudio]

yo sinceramente no creo que sean muy necesarios, mientras sigas avanzando con los tutoriales y compartiendo cosas no importa el idioma!!!

[@luchy78ar1]

@@MaverixGamesStudio Hola jose! No se en que te basas para saber si lo que pido es necesario o no, pero yo hablo Español y me gustaría oir en el mismo idioma por infinidad de motivos que justamente por su obviedad, tampoco seria necesario enmarcar

[@MaverixGamesStudio]

@@luchy78ar1 bien, bueno paso a comentarte primero que hay cursos pagos en Español que podes acceder en plataformas como Udemy, después te explico por que mi postura de que es innecesario los vídeos en Español.. básicamente es porque a cualquier programador del mundo en algún momento u otro llega el punto en que para aprender tiene que aprender Inglés... no importa de que lengua seas... y aparte porque como debe pasarle a este creador de contenido al hablar en Inglés tiene un rango de audiencia más grande...

[@MaverixGamesStudio]

@@luchy78ar1 entonces desde la más sincera parte de mi corazón te recomiendo acostumbrarte a aprender a ver vídeos en Inglés y a hablar en Inglés así vos mismo, para que tengas mejores oportunidades laborales... como así tmb hizo este creador de contenido... yo antes tenía mucha dificultad con el inglés pero con práctica pude aprender bastante... te recomiendo intentarlo

[@premsagar5289]

Where i can get the script for AR session orign

[@dilmerv]

You need to install AR Foundation from the package manager.

[@premsagar5289]

@@dilmerv Sir i have installed AR foundation,ARkit and ARcore too

[@premsagar5289]

please help me i have to submit the project tomorrow

[@premsagar5289]

@@dilmerv i am getting an error in the script that is my problem i found the AR session origin

[@marilupozos5862]

Hay alguna manera de que subas un vídeo pero en español, gracias de antemano

[@dilmerv]

En algunas semanas planeo empezar :) saludos

[@gafego]

@@dilmerv Seria muy bueno!!! Gracias por tu valioso aporte!!!

[@dilmerv]

Gracias a ti saludos

[@praveshsharma9194]

Is it also working with Android???

[@dilmerv]

I haven’t tried it but based on people’s comments I may say it doesn’t

[@akhil121ful]

How to hide an image tracker and just display objects on top of it

[@dilmerv]

Good question not really sure how to accomplish this but I will look into it

[@calvingaunce6193]

Not 100% sure but I think the only way to accomplish this would be to cover up the image tracker somehow, e.g. if your image tracker was on a white wall, you may need to animate a white gameobject over the image to make it appear as if it goes away. That could be tricky to make believable depending on the background behind the image tracker but I'm not sure how else you could accomplish it.

[@akhil121ful]

I think a quad object would do by superimposing.

[@c4lv1n_]

@@akhil121ful Yes exactly - I'm not sure how else it could be done. Your image tracker is a real-life object or image so you can't exactly remove it from the view. Superimposing seems to be the only way.

[@djsonic_pr]

I think the video is fake . Doesn't work correctly

[@dilmerv]

This video is not fake, you are probably using a newer version of Unity, I need to update this project since this was created a long time ago.

[@artyomarshakyan7619]

eventarg.removed is not doing that for me. im losing tracking of marker, but the list of removed is empty. also rescaning doesn't run Added event.

[@marcoross4169]

me too..the project ( android) doesn't work well