Steam VR's Prism Changes THINGS!

Steam VR Beta 1.25.1 released earlier this week. It's time to showcase all the strings we found relating to the Valve Deckard and future Steam VR features
bradsmells.com/patreon
0:00 Steam VR 1.25.1 is Here
0:50 Prism has Haunted us
1:50 Disclaimers
2:14 Display Flags Being Added to Drivers
3:00 Display Multi Framerate
3:30 Variable Refresh Rates for VR?
5:40 Updating Subpixel Shifting
7:07 Prism Correction
7:33 Device JSON Explanation
8:38 Prism Overrides Device JSONs
9:00 Panel Canting, Clocking, Pitching
11:40 Valve's Counterrotation/Canting Patent
12:26 User Canting/Pitching Adjust?
14:49 Valve Leaks some of HTCs future things
15:52 Shader/Dashboard Updates
17:01 Overlay Control Bars
17:46 Allow SkyDome Feature
18:16 Spatial Encode Mode
18:26 Unused OpenXR Inputs in Lighthouse Drivers
19:01 Thanks for your Support!
#ValveDeckard #SteamVR #valve

[@SadlyItsBradley]

One more thing I forgot to mention - Valve is currently working on porting the new Big Picture Mode/Steam Deck UI to Steam VR: twitter.com/SadlyItsBradley/status/1589765954824593411?s=20&t=3AxsJu6oe57i6RmpXzDtfA

[@Donvermicelli]

This alone by itself is such a great thing when they finish. I love SteamVR but the UI is such a frankenstein patchwork that never got the love it deserved.

[@CraigMansfield]

I'm sure their intention is to have the Steam Deck and VR working together.

[@BunglowBill]

I low key hate the big pic mode UI.

[@vblaas246]

As long Steam Link Big Picture mode can still be minimized for streaming other stuff without cast functionality (dvd, disney+) I'm happy.

[@oliver-nation4377]

Great video, but it could need some musuic, since it gets kinda stale to watch.

[@ekointothefuture6743]

Multrate kind of reminds me of Virtual Desktop using Spacewarp. The dev's are trying to find new ways to push VR's hardware to the limits.

[@Onihikage]

I'm all but certain the canting and pitch adjustments in software and hardware are just to make it easier for Valve's engineers to iterate on placements, and find the best default positions. Could be wrong, though! Letting the end user change the angle o' the dangle and weeble-wobble the displays all caddywumpus to get them exactly where it works best for their particular set of eyeballs _might_ be a compelling option. They might also find that for 99% of users, one specific config is more than good enough, and therefore it's not worth the manufacturing cost of including the option. I'm expecting the latter.

[@8octorine8]

With face tracking and some motors they could wibble wobble everything for you automatically.

[@Scootz_]

@@8octorine8 increasing complexity and cost significantly!

[@kuromiLayfe]

@@8octorine8 exactly what I was thinking in a way… they might make it so that the displays don’t have to increase in size for a bigger fov but the displays physically shift and angle for the maximum possible fov that the eyes actually have to see , together with foveated rendering for the best fov and clarity even with smaller displays

[@JonathonBarton]

1:43 _Shine On, You Crazy Diamond!_ 💍💍💍💍💍💍💍💍 7:39 - EDIT: Brad - as a software developer and platform architect...and one of your biggest supporters and cheerleaders...please take this in the spirit in which it's offered. - A JSON is (and always has been) called a "Jay-sawn" (or "Jace On"), and _never ever_ a "Jay Ess Oh En".

[@KrisTiasMusic]

Pitching the upper edge of the display towards us results in the lower edge being farther away, thus a higher (sub) pixel per degree. We often look down when reading text. Maybe this improves readability? My guess at least.

[@JorgetePanete]

Same assumption as progressive lenses for glasses, I'd prefer no assumptions

[@mrshaunkim]

Possibly change display orientation real-time based on eye tracking? That'll be insane.

[@SadlyItsBradley]

That’s what I was thinking. Again. Would be a crazy engineering task.

[@GT-tj1qg]

Crazy engineering task or straight up impossible?

[@mrshaunkim]

@@GT-tj1qg Possible, I would say, but QoL would be horrendous. How will you handle noise from the motors constantly turning, for example. Also, all that flexibility will probably mean more chunk to the device. They can make them small and quiet with the latest precision engineering and material, but then that would increase cost like crazy. Only time will solve this problem and we all know Valve knows how to take time lol.

[@pali122]

@@mrshaunkim what if instead of that, they used this electric lenses that can change focus to give the illusion of a changed screen or something like that

[@Dudeinator]

@@pali122 ie pivoting the lens instead of display?

[@Lordmau5]

It's a good day when there's a new Brad upload

[@atlastanker]

YES lol I practically jump for joy when I see that he uploaded 😁

[@mw-cf5dr]

But 1 is never enough!

[@llealloo]

Especially so when the topic at hand is related to anything that could dethrone the Index

[@realtimestatic]

can't believe how Brad is pronouncing JSON

[@tijoro]

Does anyone actually say Jason? I always call it Jayson…

[@realtimestatic]

@@tijoro he calls it je es o an

[@JBrinx18]

We just call JSON files J SAUWNs

[@michaeloconnor9739]

Being able to adjust the pitch along with eye tracking could give more bottom or top fov depending on where the user is looking. Assuming they could get the latency low.

[@irahughes8931]

I feel like the eye moves to fast for us to not see something move mechanically like that. Idk though

[@BlakeRosss]

Do you think this could be hinting at Valve experimenting with adding varifocal capabilities to their HMD? Like maybe through Prism they can bend the light from the screens in such a way that you can focus on things that are near/far and the optics would adjust accordingly, and the HMD can move its lenses in real time to allow for this? That combined with HDR and VRR would blow my mind.

[@thesral96]

Nice video. Btw, JSON files are usually pronounced Jason.

[@rfc5718]

Valve call it "the JSON"

[@BunglowBill]

Thanks for taking the time to share this with us. You got my sub.

[@araghon007]

I thought that subpixel shifting is either the thing some lower resolution projectors use to produce a 4K-like image, or what OLEDs do where they shift the image ever so slightly on each frame so less burn-in occurs. I'm pretty sure what you've described is subpixel rendering

[@UNSCPILOT]

Day by day we move closer to Deckard, getting the Steamdeck (256gig) has only made me more excited, that power and versatility of that handheld makes me think that Deckard will be a true gamechanger. Really gotta start saving ahead of time...

[@elderith3244]

thanks for digging into the code diveing for all of us

[@o0shad0oo]

A "prism" is, among other things, a chunk of glass added to a pair of eyeglasses to adjust if someone's eyes don't point in identical directions. I'm wondering if they've found that small adjustments result in less eyestrain for people using VR for an extended period of time.

[@arir43]

I hope, if this is the case, there’s a way to adjust it manually for people (like myself) who do have convergence insufficiency.

[@egerlachca]

I have minor binocular vision issues (I don't require glasses but they reduce strain). I lose BV diverter frequently in VR and it can take me a few seconds to get it back again. Would love a manual adjust for it.

[@o0shad0oo]

@@egerlachca Might be tough to tweak it manually, but if they're putting eye tracking in new VR headsets they might conceivably be able to calibrate it automatically as you play.

[@BHBalast]

Maybe tilt would allow more space for eye tracking or other hardware? This also seems possible.

[@karstux]

The Prism factoids are interesting and all, but you've had me hooked when you teased the HTC leak video! That can't come soon enough. 😆

[@mikeb3172]

Spatial Encoding on materials is SteamAudio related. IE. 3d audio reverb/occlusion depending on material type.. wood, concrete etc.

[@-nomi.-]

If the panels rotated with our eyelid movements, that could be cool, raising vertical fov when we widen them

[@llealloo]

Heck yeah, go Valve go! Thanks for reporting on this Brad! Sooo hype about any new hardware from that company

[@mockmoodylove]

GladlyitsBradley! Love your depth and perception!!

[@blaster1185]

Could dynamically changing the screen position be related to the idea of shaking the screen to get a higher perceived resolution? Or perhaps it could be paired with eye tracking to have the display tilt towards where you are looking, giving you a higher ppi in the center of your vision? Probably not, that would be a lot of moving parts for a small gain in visual fidelity. But maybe the screen shake could be relevant.

[@Pikminiman]

The thing about adjusting physical screen pitch in software is wild.

[@Alexus00712]

I was just about to write that the displays could be dynamically adjusted based on eye tracking and where you're looking and what you're focusing on.. And then you mentioned exactly that idea in the video.. That would be really cool..

[@Musicman247]

Imagine the screen physically moves to give you the best view of wherever you're eyes are looking. Foveated shifting.

[@SadlyItsBradley]

thatd be pog AF haha

[@GT-tj1qg]

Would be pretty weird if you look slowly to one side and then suddenly to the other. It probably wouldn't keep up and you'd just be staring into the void.

[@Musicman247]

@@GT-tj1qg what about this, then? There was something a few months ago about shaking or vibrating the screen or mirrors or something to simulate higher resolution. What if the screens moved in a half spherical pattern constantly to give you a larger fov all the time O_O Probably not practical and would be noisy as hell, but fun to think about

[@richardburnett4991]

orientation adjustments on the fly would suit eye tracking and autosetting the images for each user to get them zeroed in on the eye. if that could be automated then you're looking at being able to throw on a headset between users without having to spend 15mins tweaking all the settings/straps to get it to fit properly - press a button, headeset grumbles for a minute, perfect image. Another way of looking at it, I have a HUD (google glasses style) in my motorcycle helmet for maps, if you nudge it you then can only see a portion of the image, then usually stop to take gloves off, tweak until clear, put gloves on - sneeze or adjust lid, stop take gloves off... - if that device could self level that would be a game changer.

[@LK-gk3ry]

Dropping this here just for the algorithm.

[@SadlyItsBradley]

thx bro

[@jasonvidal4655]

lol, my dog started barking at you when you put up the image of you with the foil hat!

[@WindFireAllThatKindOfThing]

Watching Meta try to code: Like watching 80's Yugoslavians design a car Watching Valve code: Like watching DaVinci create a masterpiece painting then stop halfway to go get a sandwich and never finish

[@epicvillain8308]

VectorMatrixes refer to points in ‘space’. Basically xyz values or positions. Hence spatial encoding. It’s obvious isn’t it? MXR mode…

[@jathen7275]

Prism is a term used for glasses, it could be that it's adjusting software and hard based off prescription aswell

[@shinzanmono6376]

Thx Always for the digging and leaking of all those informations to us.. I share the opinion of other comment posted before mine. I could not imagine how 'blowing mind' it could be to associate this kind of tilt function you described with eyes tracking , what a huge fov expansion and more imersion sensation improvment it could become.(with just few mm tilt, even only one, if we take in consideration the scale and distance from the eye) Look at the corner of the eye to have a natural automatic fov expansion which could lead in an interesting way. Beware of the chills with immersion in a suspensfull game moment or pursuits, in horror game like for example.

[@MrMycelium]

There's no way this new headset can compete with Pimax's new Steamdeck/Switch/ROG Phone/ Nintendo Wii/ Xbox Kinect/Google DayDream/Quest Pro/Arcade cabinet! Get real Gaben! your time has come! There's a new king of vydia gayms and his name is Mr. Pimax!

[@evolicious]

Pimax irl: **farts**, **dies**

[@dtz1000]

Valve had their chance but have been far too slow to deliver. If Pimax delivers what they promise then I won't care any more about a Valve headset.

[@contrabardus]

Panel pitch could be useful for eye tracking. Pitch a bit when the user looks up or down, so the screen stays more "in view" providing greater FOV when following up and down eye movements.

[@junglemotion8]

Panel pitch reminds me of the angle of the old arcade cabinets. Could this make an arcade oriented view say for a Virtual arcade setting? very interesting stuff.

[@A7md]

10:00 do you think it might work with eye tracking? basically if you move your eyes up the display will rotate the bottom towards you and if you look to the left it rotates it there aswell vive versa all of this mechanically

[@yoanperez172]

The moving of the lens is more likely to focus on closer or far away objects like it was presented in a Meta prototype.

[@marcopolo8584]

Ah sweet a good valve news channel, I've been looking for one of these.

[@wasdplayer]

Love your research and theories! I will definitely buy what's Valve cooking next.

[@cgromo]

When I heard multrate, I thought of Varjo's multiple displays per eye and how they could maybe have different framerates. In that case, you could use a higher framerate for one display per eye, probably where you're focused. But if that's the case, why not set the framerate once per display, rather than add something called multrate?

[@moncimoov]

panel pitch is brilliant idea!

[@jonathanvoshell7914]

Adjustable Pitch may allow them greater control over manufacturing issues with display placement or display placement to lense placement. Pancake lenses may just require more precise positioning. Or other physical issues.

[@kiwnk]

i'm guessing the pitch would be more for the edges of your view rather than centre if i had to guess

[@MotoCat91]

Considering how differently shaped everyone's head/face is - I love the idea of being able to adjust the position and angle of displays Though maybe that kind of movement should be more difficult than a simple IPD adjuster since the displays and their connectors are more delicate than lenses - like an inset allen key adjustment I already have enough trouble with a 57mm IPD and giant roman shaped nose (thanks Mum for those genetics), so any extra adjustments are appreciated.

[@MotoCat91]

Maybe I should just get a nose job and cite "VR Gaming" as my reason.. Even taking the facial interface completely off the Quest2 (BoboM2Pro halo strap) so that it hovers in front of my face, the bridge of my nose still catches the housing as I turn my head That said, the additional airflow is very welcomed and somehow the fact I can see the outside world on all sides in my peripherals doesn't bother me

[@synystersocks4275]

looks to be a leading to a variable function that can calculate, corrrect and offset for the transform of the screen/lenses and type, makes me think of an inverse prism, concave like a contacts lens with sharper angles.

[@EndersSaga]

What i took from the prism stuff is valve wants to put multiple screens per eye. You want tilt if you have displays above and below your center display. You may also want it if you have a beam splitting prism that combines a high res display over a low res display.

[@EndersSaga]

Sometimes people refer to a discoball like configuration as prismatic.

[@EndersSaga]

You would also want multiple display resolutions in a disco ball configuration to do "foveated" rendering to do high resolution at each facet of a disco ball you are looking at/between

[@psybunk]

yeah could be something like how varjo did foveated rendering

[@thaphreak]

Has anyone mentioned that Prism correction is used in eyeglasses for some people with diplopia, or double vision. This is when someone sees two separate images of one object. The prism helps align the two images, so that only one image is seen. edit: the rotation of the displays can be all about the axis.. it looks like Valve is exploring vision correction IN the headset

[@cjlooklin1914]

Man, I don't know what sort of lighting this is, but dude is damn near glowing, I feel like screaming "my eyes!!!" Like that fish from SpongeBob XD

[@zwenkwiel816]

Forget about Fresnel or pancake lenses, Valves going straight for prisms, VR will be like a fucking kaleidoscope XD

[@chrisc3825]

Super cool. I'm really confused about how panel orientation, lenses, and actual resolution affect FOV. PiMax headsets, for example, give a super-wide FOV but part of that is just stretching the picture with the lens and not actually adding any more data on the edges. With monitors, of course, you add more resolution which allows for more data or things to be drawn. Of course, if you do this at a standard 16:9 aspect ratio, a lot if not most games won't do that rather they just zoom you back into the same point regardless of resolution. You can potentially counter this by either manually increasing the FOV if the game allows for this, or by using a wider aspect ratio to get a wider horizontal FOV, again if the game supports that. The point here is that FOV is generally determined by resolution in one or both directions with software support to draw a larger portion of the picture into the additional pixels. How does VR handle this? Simply making lenses that skew the picture around you isn't all that great if you're not really seeing more of the world, that is unless the software is somehow adjusting for this despite the resolution staying the same. How does lens rotation fit into this? Thanks!

[@rjhyou2821]

I wonder if the rotation is to get rid of horizontal and vertical edge aliasing. So much of the stuff we look at is a perfect horizontal or vertical edge (like menus and windows etc). You notice the aliasing along these edges as your view angle changes slightly. With the pixels essentially rotated, and each eye being rotated in opposite directions, these edges would disappear giving us a better result.

[@zeekjones1]

Keep in mind that the Index was built to be modded, so their next headset I'd bet would be too. These could just be groundworks to be able to adapt software to any possible headset hardware; big business product or homebrew.

[@BrianScotland]

What if the panels change pitch mechanically as you look up or down. We know they may move left right to auto adjust to , why not on a curve with direct correlation to hmd pitch

[@ApertureAce]

I'm willing to bet prism is what they call their solid state varifocal lens system (or atleast the drivers that operate it). Meaning that when you look at things close to your face in VR the headset tracks your eyes and adjusts focus to match what your eyes are focusing on. No more eye straining when reading text in front of you!

[@iambakuhatsu]

you're killing me haha. in the industry, we pronounce JSON. jay-sahn

[@MixedRealitySimRacing]

Would it make sense if they want to change the pitch and so on automatically depending on eye tracking?

[@ekurisona663]

is there any chance that deckard could be a different kind of hybrid where we could swap the SOC out between the VR headset and our steam deck?

[@MrCornerMan]

has pitch in all directions, and overides the json, its gonna be dome shaped screens with some knarly lenses

[@owlatom]

With all the adjustments, I would think it was a dev headset, so they could test out which ways they panels fit best for the most immersive experience? Also JSON is pronounced like the name Jayson 😃

[@ultimatebusinesslover]

We love you Brad!

[@DirectorX59]

Regarding the display pitch, if you look straight ahead and move your hand up and down, you'll notice you have a longer range of vision going down than up. Might be a solution to facilitate a field of view more natural to us. Also, it would make for a tighter pixel density when looking up in the distance.

[@CraigMansfield]

I'm just happy that Valve are working on VR, and that one day we might get the next great addition. In an ideal world, I'd support Valve over Meta

[@yousafkhan6921]

In an ideal world, Valve's headset doesn't cost $1000.

[@grayscale420]

@@yousafkhan6921 Enthusiast hardware for enthusiast costs.

[@ninz_0]

@@yousafkhan6921 tell that to Facebooks 1.5k headset that gives me nausea when the index doesn't... not to mention the weird controllers you get with the Quest Pro.

[@weignerg]

10:40 or tilt the screen when your eyes look up or down to raise the fov for both.

[@LhunVideo]

Sounds like vision correction perhaps. Prism and tilt could be related to astigmatism.

[@nicholasreynolds6609]

Excited! G-sync is amazing.

[@WoodysAR]

Brad, didn't it say somewhere that the new HTC could be used without base stations but that there's a thing you can clip over the front of it so that you can use base stations,,?

[@SunnyOst]

13:30 Is this certain to go into production? Maybe they are just playing around, seeing what changes, how it looks etc. Much easier to experiment in real time, having a bunch of variables you can change on the fly. Plus, they have those static headsets that are fixed in place, wouldn't even require too much engineering to set up a testing rig.

[@axansaga]

Great video

[@BHBalast]

Today is a magic day, it's a special day for every VR gamer

[@directrix1]

I mean the prism functions are debug functions so maybe it's just there to help with rapid iteration and refinement for new device creation?

[@SadlyItsBradley]

Your guess is as good as mine, for sure ^^7

[@jameskirk4692]

Nice. Thank you!

[@BHBalast]

It would be cool if spatial encoding is associated with encoding sth like lightfield content

[@r0levrai]

I don't understand variable refresh rate for VR : if you're reprojecting, the displayed image is already decoupled from the game render loop, so why synchronise with the render loop?

[@Netsuko]

Now we just need to figure out how to get John Carmack to work for Valve.

[@dachopper1]

Brad.......... Do you see any similarities between the steamdeck // PC role in the Deckard setup verses the recently announced Pimax Portal and the Pimax .... well processing PC or whatever it is meant to be?

[@mattcy6591]

Rotated and canted sounds like a nifty solution to expand FOV. Though will it be at the expense of binocular overlap in the middle? If so wouldn't that reduce the quality of viewing items up close to the face?

[@mattcy6591]

Then again maybe they calibrated it to match the normal occlusion of the nasal bridge

[@tituss_]

Don‘t really know what to say since I followed all of it in the discord as well, but I‘ll give you a comment for the algorithm!

[@darkl3ad3r]

Anyone noticing tons of pink line stuttering in VR on this beta update? I had to roll back to 1.24 to fix it. Nvidia + Index setup.

[@Shubhammjha]

this bro has cranked up the exposure so much he's looks like sun light itself 🤣🤣

[@pipiriripuriruri9804]

Nice shirt Brad.

[@SEXCOPTER_RUL]

I'm at 2 min in...I'm guessing it's internal lighthouse sensors for their new hmd. If this is the case then mabye hmd based tracking might not suck as bad?!

[@elMuele]

Thanks!

[@yoplait3256]

Hey Brad, have you seen the new Pi-Max headset? It has X and Y buttons on it and I think it might be the headset being mentioned in those XR strings you mentioned. Hope this helps you out in your research on new VR news.

[@SadlyItsBradley]

If it used lighthouse drivers, I’d agree! But it will not be.

[@yoplait3256]

@@SadlyItsBradley ah, well back to the drawing board on figuring this puzzle out

[@videogamesart8057]

Prism = optical geometry optimizer?

[@BOLL7708]

Imagine if the whole display package pivots freely as two separate eyecups, popping onto your face 😛 Panasonic's glasses but with pivots, lololol. Regarding pitch specifically, I've noticed that my eyebrows makes me see less upwards in my real life field of view, so for me it would make sense to see more downwards.

[@MachineCode0]

Wouldn't work for many people, always gotta think of the glasses wearers, and no "just get contacts" is not a reasonable response. Companies need to either work to provide third party partnerships (or provide first party) solid, quality, affordable inserts, or lens covers, or swappable lenses or some sort of hardware to account for this. Or they have to design their headsets so that they work as intended while the user is wearing glasses. Or something else. But this is not an issue VR hardware produces can afford to ignore (any more than they already have).

[@TheKaarnageShow]

Can the pitch change automatically to let you get more up and down fov…

[@PjotrSve]

I think they override the driver json now so they can iterate faster. I don't think it will end up in the consumer version.

[@hiasausmwoid1247]

Danke!

[@Alexander-yp5mw]

time to watch this

[@trademarker]

thanks for rescuing me from the pimax presentation

[@denisborisevich1221]

Bradley's the vr goat

[@mayplemoon]

thanks drab.

[@KCM25NJL]

Perhaps the prism vernacular is more literal than we imagine. Kinda makes more sense to me that they'd include an actual motor mounted "prism" in a pancake stack to optically induce cant and pitch without physically moving the displays. Not too sure of the technicalities of achieving such a thing, but certainly doesn't seem too far fetched.

[@nikolas7615]

Imagine if they use eye tracking with prism to dynamically move the displays around your eyes.

[@RobertDodge]

I just want a one off Brad on Pimax x Especially after last nights laughter fest that was the Pimax Portal

[@yoplait3256]

I think that was the headset he mentioned in the XR strings that uses an X and Y button lol. We probably are getting a vid from him about it eventually.

[@vert2552]

We like things changing!

[@josephwimerYoYoYo]

Most of the changes here would benefit AR most of all. With regards to automatic eye tracking. I would love an AR HMD from Valve.

[@muddhopkins]

Brad I clicked on your video because we have the same wall hanging. Have a good day