Does Avoiding SteamVR Actually Help Performance? | Benchmarking the Pimax Crystal and RTX 4090

[@Omniwhatever]

Very important thing I forgot to note, slipped by me since I had to redo all my benchmarks and script and only remembered after I finished the video. In the frametime plots you may see the graphs end at slightly different spots. That's because, given the lack of FPSVR and ability to see my runtime in the headset, I have to kind of count myself when it comes to starting/stopping the benchmark run. I still ensured there was no more than about 3-4 seconds difference between runs, and reran benchmarks until then in case I couldn't get my testing runs in that time frame. Most comparative runs here ended within about a second or less of each other. Just a heads up if the graphs make it look a bit worse than it is! At about 20:20 I also misspoke and meant to say DX12, not DX11.

[@SenorSchnitz]

Great video! And YES: QUEST 3 version PLEEEEEEASE

[@frankzappa2274]

I completely went back to SteamVR a while ago ... Thank you for confirming my thoughts on this actually and they were just thoughts and not tested. The games just felt better in steam VR less hiccups. Your work is greatly appreciated thank you!!!

[@Simmymind360]

What headset are y using?

[@Ghostie.]

I'd like to see this video done with a Quest 3, If only so I know which I should be using.

[@mikeserds2333]

Openxr ALL THE TIME

[@Omniwhatever]

Quest 3's on the to do list at some point! But, at least based on some anecdotes and less indepth comparisons I've seen, you probably wanna stick with OpenXR(NOT SteamVR's) whenever you can. How much you get is something I'll look at someday!

[@Upgrayedddd]

Find a game like cyberpunk and run the benchmark using oculus, steam client, VD with openxr and steamvr so you'll know what runs best for your build

[@farmertrueVR]

Excellent in depth info in a side of the VR world that few people go down. It’s very time consuming, not mainstream and at times aggravating. But that’s the life of being an enthusiast! Props to you for doing this and sharing so others can learn. I’ve done my own tests using these programs with various headsets over the years and I’ve always ended up staying with SteamVR. As a VR only live streamer/creator, the tools on SteamVR make life easier. But the performance difference never seemed to make a big difference. It’s great to see your results! Thanks again for doing this and sharing! Had looked forward to this one since you had mentioned you were working on it. Cheers Omni! PS, mbucchia is the dude!

[@JaapGrolleman]

Thank you Omni for doing these benchmarks! Nothing else like it for VR.

[@angrybeaver6667]

Ok, subbed because this is a lot of work and you nailed it.

[@EndersVRcade]

The chart from Mbucha (sorry I don’t have the spelling handy) was something I’ve wanted to see for quite a while, and your explanation of why Open XR is something I hear people asking for all the the time makes sense - in theory. It was great to see your testing numbers to see that one vs the other isn’t always the best answer. Thank you for your work and the new lighting is cool.

[@arneolsen7299]

It looks like the Pimax Crystal is set at 120 Hz refresh during the testing. What happens if you set the refresh to the 1% lows, such as 60 Hz or 72 Hz? Will the variance stabilize? Also, in IL-2 there is a significant change in the appearance of the game after making changes in XR toolkit. Thanks for the effort to make these comparison tests.

[@Omniwhatever]

That's actually a good question! Something I haven't tested. I set it to 120hz because that's the highest refresh rate my HMD supports and thus gives me the best chance to avoid becoming framerate limited. Unlike flatscreen, with a VR headset you can't tell the game "Oh, just render as many frames as you can, monitor refresh rate be damned". Even if I had a GPU that could output 1000 frames, it'd still be limited to 120hz here. So finding differences in performance is easier. In theory it could help though, but it may not be as simple as that. Because in VR, you can't pre-render and buffer a bunch of frames due to latency, it's gotta be very real time. Which means that if a huge, sudden intense load shows up that'd cause an FPS spike, your GPU has to be quite powerful to avoid dipping since it can't "prepare" for that increase as well with flatscreen. Generally though, yeah meeting HMD's refresh rate impotant. I might take a look at this someday! Would be a good idea for a video.

[@GYGOtv]

Very interesting... I play DCS and have felt the performance was about the same either way. I prefer OpenVR as I can run overlays and also the Motion Compensation for OpenVR for my Motion Platform seems to work better than the one made for OpenXR. I plan to test this myself down the road but your video seems to have answered the questions I had for now. Thanks for doing all the work on this, I really appreciate it.

[@stianronningo5469]

Thank you so much for this! Finally confirmed my suspicions. Great work, you are needed on the VR scene.

[@shApYT]

I didn't know Steve GN had a child. Immaculate conception! Praise be!

[@Omniwhatever]

Hah, thanks! GN's one of my favorite channels on youtube and was a major inspiration! Them and Hardware Unboxed, mainly.

[@jasonvidal4655]

Love how you drill down and actualy do the testing, salute sir o7

[@Omniwhatever]

Thank you! I don't see many people going super indepth on VR benchmarks like this and feels like something really lacking from the space.

[@Iionmask]

How to use OpenXR SteamVR instead of OpenXR PimaxXR?

[@Omniwhatever]

Just launch the game like you would normally with SteamVR as your OpenXR runtime. If the game supports OpenXR, it'll launch with SteamVR's OXR. If the game supports both OpenVR AND OpenXR, then you usually need to use a launch command "-hmd=OpenXR" to do so.

[@ivana6141]

Will psvr2 work with open composite and perform better?

[@Omniwhatever]

No, it'll do worse. Because, from what I can see, the PSVR2 acts like a "native" SteamVR headset on PC and thus it can only use SteamVR as a runtime. You don't wanna add extra layers to that.

[@ivana6141]

@Omniwhatever that's a shame. Alot of plugins for the unreal injector mod use openxr

[@ryanpalmer8180]

The work that has gone into this video is incredible, thank you.

[@crentist5723]

I love this!!! There definitely needs to be more videos on this so we can save fps anywhere we can even with a 4090 so we can hit a fairly consistent 90fps in all games

[@garrettferrell6821]

The Pimax crystal looks crazy,I’m sure it is amazing through

[@JesseDeya]

Yes would very much love this done for Quest headsets

[@Omniwhatever]

I've got it on the to do list! I may be a little before I can get to it since I'm gonna be too busy to do much of any work on videos for at least a week and a halfish. I also got another important video or two I wanna get to first.

[@timmyl0]

I am not doubting the compatibility issues with Quest headsets. That being said I've only encountered issues on one single game not running properly in steam VR which was TWD:SS

[@Omniwhatever]

That IS fixable if you edit some configuration files. I think people using Virtual Desktop had to do that for a while. I wish I had the link on hand for instructions, but I don't sadly. The Pimax Crystal controllers emulate CV1, so S&S thinks it's an Oculus runtime and because of some goddamn reason decides that it means you must run Oculus or you can't play, unless you edit config files to force SteamVR.

[@regwatson2017]

Disconcerting to be given information by a human fly....

[@bigbangerz5856]

Some really good info here, thanks for doing that and sharing

[@centurionstrengthandfitnes3694]

Quest 3 version, please.

[@Omniwhatever]

I'm planning to get to it eventually! May be a little bit before I do, since quite busy right now and have a backlog of things to do.

[@_R4nd0m.]

Well, I don't care about 5% -/+ performance gains. What I know is that some games I run through SteamVR will crash (themselves or SteamVR itself), suddenly pixelize or tear like my headset had been smashed against a wall when they do fine through AirLink (although I hate the UI).

[@Upgrayedddd]

You can use Virtual Desktop without Steam VR. It benchmarked well in cyberpunk vr despite some of the claims.

[@mi-vanyan9092]

I would LOVE to see a video comparing Virtual desktop vs Steam Link performance on quest 3! Pretty please!

[@markm75]

Msfs with steamvr and the crystal with motion smoothing runs at about 38 to 40 fps. With native open XR I get a solid locked 45 with client 1.14. Steamvr was still a no go for me at least

[@montagyuu5163]

Once Monado's support for Pimax hmd stabilizes I'd be curious how it compares to PimaxXR. I try to use my Vive with Monado as much as possible because it's reprojection is far more reliable than with the Linux builds of SteamVR.

[@Cellinsky-2302]

I use Steamvr with my Pimax Headset exclusively because of exactly the reasons you show here. Thank you for making a Point. There is a lot of confusion around when G2 or Q2 users change to a Pimax Headset. A lot of outdated info too. Well done!

[@yeyu999]

Thank you, you black shadow adviser . That is an excellent piece of information. The only thing I wish to have in the content is the advice for DCS VR and IL2VR since this is the reason for doing VR games.

[@brockoala2994]

Your annoying voice and the great benchmark earned a sub from me.

[@MooshPaw]

A couple tips if you do decide to do the oculus test Other than the PC framerates, I'd advise checking on the transmission framerates, oculus loves to take about 60-100% of the encoder just because it can (for contrast VD medium and steam link take about 30% on a 4070), making it impossible to get 120fps on lower end cards Besides runtimes which im not sure how they work, i can confirm that on a 1060 you could play beat saber at a resolution of 1920 120fps on VD, but using oculus you had to drop to 1400ish to barely achieve 120fps I was also using oculus killer which honestly didnt feel like it did much, but i also found a fork of the project that has received many updates and i think the performance is better(?) There are a lot of variables, but in summary i advise to test VD Steam link Oculus direct Oculus steam link Oculus killer (the updated version from the fork) Also important to check on the transmission framerate

[@Omniwhatever]

Yeah, I'm aware that the overall Quest comparison is kind of more complicated. Because of the latency when it comes to streaming, a method with noticeably lower latency but little lower FPS would be overall more responsive and arguably the better option for somebody chasing the highest FPS given one of the two main benefits to higher frames is lower latency. The thing about that is that while I know every method has some stuff you can look at when it comes to overlays or programs, they're all not the same in format or even how you view them. in theory it should be measuring the same thing and as long as the right metric is being looked at it... PROBABLY shouldn't vary much, but for the sake of being absolutely sure I generally like to try and use the same program for all my comparisons. And that's a little more difficult on something like this. But concerning the runtimes, uh there's only 1 Steam Link? Least for VR gaming. There's steam link for playing flat games but that's not the same. And Oculus Killer is no longer being maintained, I saw two github repos and one of them was updated last near the very end of 2023 and flatout said it was not being maintained, suggesting Steam Link instead. So, I don't think it's worth testing.

[@jackr.749]

To the man in black (man, that suit looks itchy, LOL) thank you. I'm supposed to get my PCL late tomorrow, but with my G2, I use SteamVR (OpenVR) 99.5% of the time, only title I have which gets the OpenXR treatment is MSFS and the rare times I play iRacing. As for open composite, I can't say I'm a fan of this approach, the way I look at it, either a title has native OpenXR support or it doesn't in which case I use SteamVR. Open Composite just looks janky and a pain in the butt! Again, thank you for the video, a bit of a eye opener (pardon the pun 🤪)

[@Omniwhatever]

It's not itchy, surprisingly, but it gets HOT AS HEEEEEEEEEEEEELL. Especially with all the lights on me. I'll tell you that much. Hope you enjoy the Crystal Light!

[@Simmymind360]

I hope that thete can be a better setup on Steam VR for mist heafsets..eg..where it can detect yi headset and automatic set it for user😢😮

[@nigaudlotovrbenchmarkandvi313]

Hi, any extra to install for running a modded skyrim ? Mine with open composite-open xr and pimax xr won't start Thanks.

[@Omniwhatever]

No. I did nothing unusual. I just used Mad God's Overhaul and it works just fine with or without OpenComposite.

[@djp1234]

Do you have to use PimaxXR in order to get dynamic foveated rendering to work?

[@Omniwhatever]

No. I never used PimaxXR except for testing or benchmarking purposes. If the game is OpenXR, then you have to use the OpenXR Toolkit. For OpenVR that's what Pimax natively works with, but it's not universal.

[@turbohenk702]

​@@OmniwhateverThe best performance is always with pimaxxr and openxr toolkits tweaks to the max. Without it I get massive performance drops because I use the oval ring at 180%(advanced settings) and custom resolutions in all titles that I play and that could only be done with OpenXR toolkit. You underestimate the performance boost that the toolkit gives you. So it's not a question of openvr/openxr, it's a question of openxr runtime of SteamVR or opencomposite, because without one of the two you cannot use the toolkit !

[@kaikios]

Quest 2 / 3 / Pro with steamlink is pretty much the same as a steamvr native headset like a Valve index or Vive pro no?

[@Omniwhatever]

On paper it should be since you can control a lot of functions with SteamVR then, like refresh rate and the like, but in practice could depend on how everything was integrated. I would assume it's not a big issue, but don't think anyone else has done a big deep dive into the performance so I can't be 100% confident in saying it's the best right now. Stuff like latency with streaming since no DP connection complicates comparisons a bit though. I tend to hear, at least anecdotally, that people feel like Steam Link has lower latency. So, if that were true, then even if the actual FPS was a little lower it could still FEEL more responsive than the technically higher FPS option.

[@JP-yv7zo]

Great info! Thanks.

[@MrZoops17]

But lets get our semantics straight. Pimax does not even have their own native OpenXR, you are usaing PimaxXR which is a tool built by MBuccia and runs without steam but not technically native.

[@Omniwhatever]

Yes? I said that? It still functions like a non-Steam OpenXR runtime would.

[@gwizdekf1liga]

Agreed. You compare community made runtime + community made tanslation layer to vendor runtime. VR was left by the vendors in a Tower of Babel state. OpenXR is the way to go (Valve admitted it themselfs), yet they develop SteamVR stuff to this day making consolidation/unification even harder.

[@davidl2153]

Yes would like to see quest 3 version of this.

[@Vindaloooooo]

Fab. Noticed something was off with Pimax XR and OpenComposite too but MSFS always made me question myself cause it was noticeably better perf with Pimax XR. Nice to have some numbers to see these differences across titles. Will keep Pimax XR for MSFS and Steam for everything else. Bit of a pain switching between everything... c'est la vie.

[@jamieh8240]

This is exactly what I do as well for IL-2.

[@StekuVideo]

Switching between PimaxXR OpenXR runtime and SteamVR OpenXR runtime is literally one click in PimaxXR companion app. You don't need OpenComposite for MSFS2020, DCS or any other native OpenXR app.

[@Vindaloooooo]

@@StekuVideo Yeah I know it's minor just a bit of an annoyance to me. Anyhow trying out DX12 with MSFS and using SteamVR to see how things go for me seeing Omni found that DX12 had similar perf between the runtimes. DX12 used to be a bit unstable in MSFS but it seems good to go now.

[@Iionmask]

Have you tried it with UEVR?

[@Omniwhatever]

This kind of test specifically? No, but the same general logic should apply given this is a compositor thing every game uses when using a Pimax HMD.

[@Iionmask]

I think I get better results using Pimax XR than Steam VR for UEVR

[@Trigger911]

I would love to see a video like this for the meta quest 3 headsets vs using steamlink\airlink or virtual desktop this is pretty interesting.

[@raffytaffy3480]

A Quest 3 version of this video would be awesome. Personally I use VD and I'd be very interested to see how that stacks up

[@justjack8989]

Wondermelon

[@patrlim]

Please do a quest comparison!

[@obamabarrack4450]

cool and awesome. an overview on the options on oculus would be very interesting too

[@jamieh8240]

I always use Opencomposite with my Pimax and SteamVR with my Quest 3. I only use the Pimax for Flight Sims and Opencomposite gives me a far better frame rate than SteamVR can when the graphics settings are high. I do this because the Pimax has far better clarity for viewing long distance. I use the Quest for anything else, mainly FPS type games, and Steam VR gives a good performance with this type of gaming.

[@StekuVideo]

Open Composite is absolutely not needed for MSFS2020, as it is the OpenVR to OpenXR translator, MSFS2020 is native OpenXR app, doesn't need such translation.

[@user-wq9mw2xz3j]

Im so confused about VR 😂

[@Omniwhatever]

Unfortunately, it do be kind of like that.

[@FierceFire14]

Curious if theres a relation between (mostly) GPU and CPU bound games, what if we have a NVIDIA drivers situation, where the drivers use more cpu resources thus reducing its performance?

[@Omniwhatever]

I made sure everything I tested here was GPU bound by a large degree. Nvidia improved the driver over head issue a while back too IIRC, though don't believe they completely got rid of it.

[@fullstomache]

Very good video, appreciate the work you put in here! I would've liked to see VRChat performance though. Single user instance in the UDON Benchmark world would probably be consistent enough to post. So if you do a Quest 3 video, perhaps you can put that in there.

[@Omniwhatever]

VRChat is not applicable to this specific kind of test because it can only be run via OpenVR and OpenComposite does not work. You ALWAYS have to use SteamVR and OpenVR.

[@fullstomache]

@@Omniwhatever Ah, ok then, good to know!

[@Endemiq2]

Doesn't foveated rendering and culling outer mask used with openxr flips the table completely?

[@Omniwhatever]

You can use foveated rendering on OpenVR or with SteamVR's OpenXR. Unless you're on Pimax though, OpenVR is limited to FFR and doesn't have DFR. By cull the outer mask do you mean the Hidden Area Mask? Every headset has one of those by default.

[@Endemiq2]

@@Omniwhatever I would love to have reliable OVR solution for FFR but I'm not aware of one. There is vrperfkit but it has very limited success rate in my experience.

[@RidgeXR_2]

What happened to your mic omni? it sounds like it's buried in sand at the bottom of a pringles can

[@Omniwhatever]

If you mean the CRYSTAL, I'd been doing a little testing with a Crystal Light I got sent for review and seems like that boosted the gain higher than I had it originally up to what was set on the Light for... Some reason, Pimax I guess, I dunno. If you mean the OTHER mic, I think what might've happened is I use some filters in OBS that were specifically tuned for the Crystal and while going back and forth testing some audio on the Light and some filters to try improving, I think I got some wires crossed and changed the wrong source by accident since I typically need to reassign audio sources because I have so many USB ports and devices sometimes I gotta unplug a thing or two and that includes the Yeti. And since they're carefully set for each mic being applied to the wrong one didn't end well. I did kind of notice it in the first half, think I managed to fix it up a bit in the second half. Was going to try rerecording things but I'm so incredibly busy this week that if I didn't get the video out yesterday it'd probably have been pushed out at least a week and a half or so and I thought the audio was "okay" enough to not warrant that.

[@RidgeXR_2]

@@Omniwhatever Fair enough tbh, would appreciate if you'd set the mic settings back though next time you record xD I had to watch this one on mute with CC enabled, the plosives really hurt my ears

[@ricepony33]

Is there a way to actually test VR performance without having the headset on your head?

[@Omniwhatever]

Soooort of? As long as you have a headset to run the game in, of course. You can just set it on a stable platform and give it some occasional shakes to ensure it doesn't go to sleep, then do your run. Though, if the headset has a sensor to detect your face, it may need to be taped over or something to avoid it going to sleep.

[@ricepony33]

@@Omniwhatever thanks

[@MrAminmohamed]

What help is supersampling and removing all AA

[@Omniwhatever]

I'm not sure what you mean? Generally not the best idea, but the Crystal is physically a much higher PPD display than, say, a Quest 3. So that means things are sharper and it needs less AA. I did a video looking at AA in high vs lower PPD headsets a while back.

[@MrAminmohamed]

@Omniwhatever I mean the AA methods the games use blur the images and lower the framerate I prefer using the processing power for supersampling. On a samsung odyssey + it does a big difference, I am sure it would improve the clarity of the picture in a lot of headset and also the sweetspot. Also the framerate is the key to immersion, I played recently Half life 2 VR and with supersampling and high framerate the experience was amazing. The low res textures was not even a problem.

[@Upgrayedddd]

The main reason I try to avoid steam is the bloat and stacked launchers. Some games don't play at all.

[@Despiser25]

Or it is as simple as Pimax software has not fully matured making SteamVR by comparison seem good?

[@Omniwhatever]

No? PimaxXR isn't made by Pimax and was made by somebody who was very prolific in making OpenXR apps, from the OpenXR Toolkit to the OpenXR runtime for Virtual Desktop, among other apps that provide some pretty impressive functionality to OpenXR games which a lot of people have a lot of praise for. They've also worked officially on VR headsets before. Pimax may have some issues on their software, but when it comes to performance overhead for not being a fully native SteamVR headset, they're doing pretty good on that front.

[@Despiser25]

@@Omniwhatever I wasn't aware of this. Thanks for the clarification. Seems a pretty big deal the guy that does OpenXR (VDXR) makers a headset, lol. Virtual Desktop is magic.

[@turbohenk702]

Great gains with bypassing SteamVR, the difference between 85fps/90fps is a difference between playable and not playable. So I don't agree that its an performance difference to ignore. If you want max. performance the way to move forward is OpenXR toolkit with maximum tweaks with SteamVR bypassed. You need the toolkit for max. performance, so its not a choice of openvr or openxr, its SteamVRs OpenXR runtime or opencomposite and then the difference is significant! You forget that most important fact in your video. SteamVR is and stays as terrible as it ever was.

[@Omniwhatever]

I mean, did you see that it tended to perform better more often than not and had much tighter frame pacing, so in particular it tended to be better in the lows? What you can do with the OpenXR Toolkit is quite similar to the VRPerfkit, save only having FFR vs DFR but at least with Pimax you can DFR in a chunk of games with OpenXR. So, no, I didn't ignore that fact at all. Frame rate consistency is just as important as meeting your target hz rate,