Design Spatial UI - Apple Vision Pro UX analysis

Apple annouced their Vision Pro mixed reality headset. Let's explore the user interface and user experience of designing for spatial interactions.
There's a lot to unpack here from glassmorphism, dark mode brightness levels on design layers all the way to the brand new input methods of combining eye, gesture and voice.
I believe learning mixed reality now, when the category is finally becoming established with some innovation is a great way for a designer to stand out.
I'm not an expert in it - obviously - but I plan to learn as much as I can.
At the moment there's only a handful experienced designers and developers in the space and this is a whole new category of jobs.
Well worth exploring, right?
You can find Apple's materials to learn from here:
developer.apple.com/videos/pl...
#visionpro #apple #design

[@Hamstimusprime]

Was waiting for your take on this. As always, Amazing content. Much love

[@MalewiczHype]

Much appreciated!

[@acslater017]

3:56 I just noticed, all the window adjustment interfaces animate when they change positions 😍 By default a stationary window has a handlebar (dot and horizontal line) beneath. When you gaze at the window’s bottom left corner to resize, the bottom handlebar kind of zooms off to the left and then appears in the corner. I love these little touches, looks like a joy to operate.

[@vineesque]

Glassomorphism is back ✨

[@MalewiczHype]

It never went anywhere ;)

[@VioletVolchok]

I'm starting a masters program in human-computer interaction this year and really excited for the Future Interfaces module... hopefully discussing the possibilities of devices (paradigms) like this. It's tricky though to find the balance between something really theoretical and academic and something entirely industry-focused, like "how do I become an AR UI designer now". I hope that now that Apple is throwing their weight behind the technology there will be much more conversation about it online and opportunities to develop new standards. Having one of these to play around with for experiments in uni would be perfect, shame there's no student pricing haha.

[@Riya-vz8tl]

I'm starting my HCI masters too this year! And thought of how students will be playing with MR!

[@thesidebrain]

Ultimately what you (and I) will have to do, is just get started. There isn't going to be this huge wealth of pre-existing information for us to draw from like there is with 2D UI. Pick a problem you want to solve, get into a 3D space and start testing. Look into how the brain processes sensory information, and take those learnings and apply them to solving the problem/s you choose. This is a brand-new, wild frontier that we're embarking into now. You might discover something no one ever thought of before. Don't worry about following a pre-defined path, and start carving your own. It will be worth it.

[@Riya-vz8tl]

@@thesidebrain True. Experiment-in one word. Would be great to collab/discuss if there's a way to connect?

[@Shlooomth]

I really love the glassmorphism, slightly flat but slightly rounded 3D edges

[@r3dp1ll]

That was quick ! Nice intro by the way

[@MalewiczHype]

Hey, thanks!

[@ifouzezi]

Thank you so much!

[@marzoval9551]

I do think a lot of the UI will be handled by the headset itself, such as lighting and shadow casting based on the user's surroundings, as well as rotating windows at different angles towards the user. But I can see how 3D modeling for UI will become more important if you want to maximize their presentation beyond 2 dimensions.

[@MalewiczHype]

Exactly! Those OS UI transformations (shadows, lighting) I talked about more to point them out as part of immersion. But that also means similar techniques and other of that caliber can be used in completely custom apps :)

[@AviBarel]

Can't wait to try it and design for it! Well, I've been waiting for Spatial Computing glasses from Apple since 2016, when the rumors started. Damn, I guess I'll have to write the 6th article in my "Design for XR" series, because I just published the 5th one last week... 😅

[@MalewiczHype]

never enough articles! ;)

[@badcatdesign]

Spatialism finally makes its comeback from the late 1940s

[@enterpriseux]

I did a similar video, but you have spotted indeed a lot of nice details :) Nice one :) Regarding what you've said about the future. I agree, it can be overwhelming if used all the time, but I think, this will not be a device you spend hours with and it will not replace the computer completely. At least not soon.

[@purnjayparmar]

Eagerly waiting for your master class on this area of user interface and experience 😀

[@MalewiczHype]

Not planning anything yet as I have zero experience with these things (yet). But I am buying one and will build some stuff for it. Once I do I may start sharing my findings :)

[@bjmgraphics617]

It’s freakiń awesome! I bet there will be glasses adaptor or the image can be magnified. I want to go on a run with it and play realistic VR games like in the movie Ready Player One.

[@MalewiczHype]

Who knows, maybe someone will make movement experiences for the outside - that can be cool.

[@Brian-oz8io]

@@MalewiczHype I have been thinking a lot about how useful mixed reality like this could actually be. You could certainly create outside movement based AR games, but also helpful apps for learning musical instruments like something that puts chord overlays onto your guitar and shows where your hands are supposed to go, improving your golf swing, fixing a car, drawing/painting, cooking, the possibilities seem endless. The ability to have an interface that measures eye and hand movements to give real time feedback while you don't need your hands to interact with it is so powerful for learning a skill or performing any manual task. Even just the ability to display real time information without having to get out your phone and stare at a tiny screen for 10 minutes could save so much time. I'm blown away by the implications of this to be honest. I could follow a recipe that is being demonstrated in 3D while video chatting with someone, reading the news, checking the stock market, texting, and browsing social media all at the same time! I almost want to become a developer and try to learn Unity/Xcode just to contribute because this is the future!

[@EvanBlack11]

Thanks so much for this video

[@MalewiczHype]

You are so welcome!

[@EvanBlack11]

@@MalewiczHype I would love to see a tutorial on how to design for vision pro

[@yashkalra6518]

now ux will even be a more important field for designers. i think we should also start learning 3d modeling tools like blender etc. it could be big in e-commerce

[@MalewiczHype]

Some 3d definitely won't hurt - spline is quite easy to learn.

[@yashkalra6518]

@@MalewiczHype i know i learned spline last year it was really fun and easy. 😀

[@dylanbanning]

I’m not that big of a fan of eye tracking so I don’t know if I’m going to get the Vision Pro however I do wonder are you able to use hand gestures and swiping to scroll through menus because I’d rather tap then just have my eyes do all the work

[@rosab6259]

Amazing analysis and video 😊 I think this headset is a thousand times better than the Oculus because of the input and ways to control with voice, eyes, and hand gestures...and also that you won't be completely "blindfolded" and can be aware of surroundings. I'd like to learn more about accesibility on it. Good case study topic!

[@MalewiczHype]

Yes, I am getting it as soon as it's available and I want to build some Mixed Reality apps definitely! Maybe a 3d virtual UI Challenge grading app with projects from the community floating in space :)

[@rosab6259]

@@MalewiczHype I'm looking forward to it 🙏🙏 This is the thing I was most excited about out of all the cool stuff Apple came out with. Are we expecting some XR design courses? Asking for a friend 😉

[@MalewiczHype]

I was planning to do just one or two more courses and retire, but now who knows. I will buy this device, build some apps for it and then will decide if I have anything to teach people ;)

[@rosab6259]

@@MalewiczHype I'm sure you will have lots to say about it and teach us but you also deserve a break (I'm low- key hoping you don't retire 😂). Thank you for the amazing work you do!

[@yashsalve7702]

How to you deal with scale? For 2D design we have fixed resolutions of screen but in mixed reality scale be can anything. how do you determine what should be the minimum font size, icon size, clickable area and so on…?

[@shreybgoel1568]

Hey kindly answer this question... I was learning web development ok and then my plan is to goo deeper in this only like learning 3 d web and after that blockchain stuff but yesterday I saw wwdc and saw vision pro and I am confused what to do?? 1. Should I switch? Or start focus on ar vr thing? Or should I focus on what I was doing... 2. Also tell like these days there are soo many tools through which a non coder or a normal person can create a website like legit cool and very functional website even with 3d designs sooo what you think is the future of web developers?? Is it still worth to learn or master this skill? 3. Kindly tell should I learn how to make websites that can be viewed or explore through VR headset... ???? If yes how???

[@brain_respect_and_freedom]

Thank you for this video. I saw this Apple's commercial, and I wonder: ✅ Great! Eye-control Empowers People with Disabilities but 🚩 Is it possible to wear glasses under this tool? /If you have a bigger vision defect, and you can't use contact lenses/

[@gouravjoshi953]

🔥🔥

[@MalewiczHype]

This is a different video than most people have done as they focus on the device itself. I want to explore purely the designing for it side :)

[@SLApple-hp9ed]

Except the price, it is really attractive.

[@MalewiczHype]

This is a first generation device aimed mostly at enthusiasts and developers - it will get cheaper, probably smaller and with a built in battery too :)

[@aashitarangnekar382]

How it will impact human eye🤔? It literally a revolution for designer 🤩

[@mirzarizwanbaig9449]

iOS needs this as overhaul It’s more than 10 years for same flat UI

[@MalewiczHype]

The problem is the design system is still mostly flat UI, just hanged in 3d space.

[@irfanadamm5819]

I sure could use a beautiful day

[@MalewiczHype]

Go make one for yourself then!

[@grafxdesigner93]

This is going to be some mindblowing experience if done right and proper. Again, this can become addictive somewhat. I can see that happening for all the wrong reasons. Awesome but frightening at the same time. We will know in due time. Definitely something to look forward to.

[@MalewiczHype]

People were addicted to tv's, then computers, then phones. Keeping a healthy approach is necessary no matter the device.

[@grafxdesigner93]

@@MalewiczHype I agree 100%.

[@Mithferion]

Wait for the new course con Spatial UI by Malewicz

[@MalewiczHype]

I won't do a course until I get the device myself and build some apps for it. And yes I am buying it as soon as it's available, but it will be a while ;)

[@Mithferion]

@@MalewiczHype Glad to know that you are interested in the matter (:

[@MalewiczHype]

I got bored with traditional UI after 20 years of doing it. Need a new challenge :)

[@md.toufiqueislam516]

Apple never disappoints, until you see the pricetag

[@MalewiczHype]

that's a first gen device mostly for developers though. it will be significantly cheaper in the future

[@TheMarouuu]

I was excited about this one. When Apple does something, they usually do it right so I was looking forward to trying the first good AR/VR headset ever. Buuuuut, I just watched a video on some of the patents they're using in their product and man... some scary stuff in there. Mind reading and whatnot... they actually measure a lot more than just your eye movement. I mean it sounds very sci-fi and conspiratorial but at the same time they do have a lot of similar tech already on their watches so it's not a really that big of a stretch to imagine them improving their tech even further. When you combine their sensors, and the fact that GPT is being used for similar stuff... It creeps me out a bit.

[@MalewiczHype]

can you share some of those patents? Would love to find out more

[@TheMarouuu]

For some reason I can't reply back to your comment. I saw the apple thing on the last video on The Friday Checkout channel.

[@MalewiczHype]

can you email me some details then? :) There should be an email somewhere in the about section of the channel. I'm genuinely very interested in this.

[@TheMarouuu]

@@MalewiczHype I just sent you an email, happy hunting! :)

[@jessybond]

Don't know why but it gives me 451 Fahrenheit vibes...

[@MalewiczHype]

Burning books?

[@jessybond]

@@MalewiczHype not really. Do you remember protagonist's wife? She was using kinda similar to the modern VR/AR technologies to watch soap operas, chat and basically escape the reality. If I'm not mistaken, she got mad and died at some point. I'm not saying Vision pro'll have such a fate cus there are tons of ideas to explore this device and create smth really meaningful. I'm just afraid that entertaining companies will monopolize the field and insist on creating artificial reality dragging people away from the real world and its real problems.

[@MalewiczHype]

That is a valid concern and I am aware of that push (Meta was very strong about it) towards a Ready Player One kind of scenario. It's scary, but when used right this device can be quite good. And people are addicted to smartphones right now, so technology addiction is a thing no matter the technology. They used to be addicted to TV's before ;)

[@ozeiasdsl]

I thought the product was incredible, but at the same time I thought that Apple was inconsistent in its speech. They showed some concern about myopia and, as a result of this concern, brought features to their devices related to identifying the distance between the users' eyes and their screens. Soon after, they present a screen that will be a few inches away from your eye. It didn't make sense.

[@MalewiczHype]

That is a very good point actually!

[@marzoval9551]

Have you tried VR before? The distance between the eyes to the screen (also called "eye relief") is short, yes, but it's the lenses between your eyes and the screen that eliminates the eyestrain you would otherwise experience looking at a screen that close. There's a whole engineering/physics explanation of it.

[@ozeiasdsl]

@@marzoval9551 Sure I have tried lol. Speaking of VR, I suggest you to search about eye strain on some of the most used VR devices. It's not difficult to see eyes strain being mentioned as one of the problems during use. *A short correction, Vision Pro is not about VR (Virtual Reality), it's about AR (Augmented Reality).

[@FriedChairs]

Your eyes focus in the distance when using the headset, unlike when holding a phone or watch close to your eyes.

[@g6ter1]

Eye strain in headsets is more caused by the slight latency/headtracking which the M2/R1 chip is claming to have solved.

[@loumachugh9641]

as a scottish person i am utterly terrified at the voice input options. it seems this will become the most popular options - but unless you speak with a 'proper' accent, this will simply not work. voice inputs are horribly exclusionary, and apple are not going to change that at all, any time soon - there isn't any financial incentive to do so.

[@MalewiczHype]

I think Ai is getting better at recognizing all voices and accents. Are you having problems with Google voice or Siri when speaking with a scottish accent? It's an interesting thing actually - I wasn't aware of this and may be something worth exploring in a video.

[@MalewiczHype]

however - there's a virtual keyboard (as an option) available, they just didn't showcase if as a prominent feature - but it is there

[@1a1o]

Why everyone acting like this is new? Meta and Microsoft been doing this for years 😂

[@dylanbanning]

If glassmorphism is the future of UI design then is Apple implementing glass glassmorphism in iOS 18 because as far as all the betas versions go, everything still looks as flat and boring as ever

[@m4r_art]

Let's actually for once hope this technology fails so that people remain connected to the physical world instead of jumping into layers of screens which will only feed the ego and reduce creative thought.

[@MalewiczHype]

I was thinking about that (how most of the people in the presentation were alone in a room reliving memories of loved ones) but in reality people were addicted to stuff for a long time now. Tv's, computers, phones - you name it. The awareness is growing and if a large chunk of population wants to disconnect it's their choice. I like the real world.

[@m4r_art]

@@MalewiczHype hi5

[@NN-sp9tu]

Too late

[@lynxrbeam8732]

Yes let’s hope this tech fails so you can have your reality. Or, we can hope it succeeds to it can evolve into a more useful tech like a sunglasses format where you can avidly use it and not look weird. But yes let’s hope everything fails because we don’t like sitting that’s fucking mature.

[@m4r_art]

@@lynxrbeam8732 Not everyone on the internet is cancel culture. Some things that big tech companies offer are not the right move forward and are just dopamine/ego feeding trinkets that disconnect you from others, build fake happiness, while pushing you away from the real world. Escapism is okay, but layering it on people's eyes is simply dystopian. It's a screen for the self not a tool for togetherness.