ARKit: What can it do? (visionOS)

Go to squarespace.com/seanallen to save 10% off your first purchase of a website or domain using code SEANALLEN.
Understanding ARKit is vital to building a great app for the Apple Vision Pro in the Swift Programming Language. If you've never built an app with ARKit and have no idea what it's even capable of, this is the video for you. In this video I highlight World Tracking, Scene Understanding, Face Tracking, Motion Capture, Location Anchors and visionOS.
My iOS Dev Courses:
seanallen.teachable.com/
My Source Code:
seanallen.teachable.com/p/sou...
X (Twitter):
Sean Allen - / seanallen_dev
Hired.com:
hired.com/x/1n01g
Book and learning recommendations that help out the channel if you decide to purchase (Affiliate Links):
Mark Moeyken’s SwiftUI Books:
www.bigmountainstudio.com/a/f...
Paul Hudson's Hacking With Swift:
gumroad.com/a/762098803
RocketSim - Enhance Your Xcode Simulator:
gumroad.com/a/51797971/ftvbh
Objc.io Books (Thinking in SwiftUI & Advanced Swift):
gumroad.com/a/656585843
Timestamps:
0:00 - What is ARKit
1:42 - World Tracking & Scene Understanding
2:29 - Face & Eye Tracking
3:20 - Motion Capture
3:34 - Real World Location Anchors
4:14 - ARKit in visionOS
#swift #softwaredeveloper #iosdeveloper

[@seanallen]

Learn more with my iOS developer courses (Swift, SwiftUI, UIKit) - seanallen.teachable.com

[@davidharraway8131]

I work with people with disabilities who have limited movements. We were so excited by ARKit when it first dropped and we got to see Hawkeye app in action ; but I guess over time the the usefulness of front device camera eye tracking hasn't really proven itself in the real world. Hawkeye and a few of the other ARKit enable solutions were always a bit difficult to use and subject to interference from lighting etc. Sure Vision Pro has great eye tracking sensors but it's my understanding that access to those for third party apps is still not a thing. I can see two possible ways forward - Apple build and iOS and MacOS Accessibility setting that allows full system wide eye mouse control (possibly via the current Assistive Touch menus which is what the iOS MFI eye trackers use) or they build next gen outdoor capable eye tracking sensors (the kind that are in a few Assistive Tech eye trackers) into iPhones, iPads, and Macs - for mine , this would be the preferred option as it would enable eye tracking for all thereby democratising it and making the setting available for game devs etc.

[@mortezafaraji479]

I’m going to need a vision series pleaseeeee. I’m developing on visionOS for my research this summer and you would make my lifeeee

[@seanallen]

I have a bunch of videos (and a full course) planned over the coming months.

[@mortezafaraji479]

@@seanallen AWESOME!!! You really are the king of iOS developing youtube. nobody matches your effort and willingness to share knowledge. 👑

[@UnshackledPursuit]

Great content. Keep it coming for VisionOS

[@seanallen]

Thanks! And that's the plan...

[@sorinmiroiu9404]

Great video Sean!! Can't wait for the one on RealityKit! Keep it up dude! :D

[@seanallen]

Thanks! Will do!

[@apbrandom]

I'm very excited to see next video :)

[@seanallen]

I'm editing it as we speak :)

[@apbrandom]

@@seanallen 🔥🔥🔥

[@bryceellis112]

Nicely done

[@seanallen]

Glad you liked it!

[@Leopugpiano]

Man I love your videos, really good!

[@seanallen]

Thanks! I appreciate that!

[@brycenrushing4412]

Just wanted to add since i ended up here while developing for the Vision Pro. Several ARKit features mentioned in this video are not possible on the Vision Pro currently. There is no face tracking or body tracking possibilities because of privacy concerns. This may change in the future but think those watching the video should be made aware that these things are not possible currently

[@davidharraway8131]

Nor can you use eye tracking in third party apps yet I believe

[@brycenrushing4412]

@@davidharraway8131 you can have stuff occur while hovering over an item but the actual eye tracking isn’t shared on the developers side. But you can tell if an item has been hovered over. And in that case it usually means it is being looked at. Kind of a weird thing though because the documentation is very clear that you don’t have the actual eye tracking data

[@CodeVideo]

fantastic!

[@seanallen]

Thanks!

[@nemesis0720]

hi, can you help me how I can manipulate blend shapes of a usdz file? I would appreciate any advice

[@josselincol]

I wonder why we can't use simple Plane Tracking (ARKit) in a bounded volume app, in the Shared Space. That would be so useful.

[@viperjay1]

Sean, AR just makes more sense on headsets. I know AR was on mobile devices but, I have a feeling it didn't take off because of the user needing to hold it for long lengths of time which probably got tiring. Thank you for these videos, I can't wait to see what can be done as of now that is on going question of which is Ok "so all I can do with the 3,500 device is watch videos?" I wonder is going to make the first killer app for AVP

[@davidharraway8131]

I tend to agree - we really need a whole new UI/UX language for AR/VR/MR. The current platforms only have limited Accessibility (Apple SAY Vision Pro has Accessiblity at it's core but that is still coming SFAIK). Sarah Herlinger is a very impressive person though so she may sway things in the direction they need to go.

[@allensu4222]

HYPEEEE

[@itsarifnaveed8459]

very informative video, @Sean I am struggling to find help for implementing clean architecture in swift ui, can you please make video on this

[@seanallen]

Architecture is a tricky topic. It can be very subjective and it depends a lot on the particular type of project.

[@1nspa]

Imagine creating a nice bowling game for visionOS. This can be very interesting.

[@seanallen]

There's a lot of interesting possibilities...

[@primarykey5507]

Hi, Sean. Does Face tracking and body tracking are avaliable in VisionOS now?

[@JonasViatte]

It’s funny that it’s called ARKit but then they tell devs to never use the word AR in their communication.

[@seanallen]

I know, right?

[@victorriurean]

[@asadchattha3470]

Waiting upcoming lectures...

[@seanallen]

Coming soon.

[@gideonocholi130]

001

[@JonatanEdOrtiz]

002

[@hardsayings2400]

The thought of seeing people wearing computers over their eyes everywhere I go makes me sick to my stomach.

[@shm5547]

It's ok. If you get a VisionPro headset, you can use AR to replace those people's goggled up eyes with a pair of cool shades, or whatever you like!

[@enterprisar863]

I think you are missing the big picture. The cell phone spent two decades from the 80' and 90's going from a big brick and luxury tool for business to more accessible tool in the late 90's and early 2000's to essential today. The same will happen now. I have had a list of independent techs who would have (and still will) pay 3x the cost of AVP of their own money for stuff i was working on back in 2018 with Hololens 2. But microsoft couldn't live up to the software side its and I was on Microsoft's AR research teams sharing independent data. There is MASSIVE opportunities on the business side for Pass Thru. Here is an early piece of work: kzbin.info/www/bejne/g6iQeJeljtqjkJo

[@shm5547]

@@enterprisar863 I took the comment to be less about the aesthetics, expense and usability of the hardware, and more about the potential societal impact this technology will have. With mobile phones, the technical development path was the easy (ish) bit, the societal change brought about by social media apps and cameras everywhere, was harder to predict. Still, cat's out the bag now, so best get on board and see what it can do.

[@hardsayings2400]

@@enterprisar863 I sincerely doubt you know what I'm talking about.

[@JasonMitchellAZ]