Does Benchmark Duration Matter? One Year of Testing

[@GamersNexus]

Article here -- For Honor actually proved a bit more interesting than we thought (see article). The data was not finalized at time of filming, but is in the written piece! www.gamersnexus.net/guides/3219-bench-theory-does-benchmark-duration-matter-one-year-of-testing We are working on FFXV right now! First test is here: www.gamersnexus.net/game-bench/3220-final-fantasy-xv-benchmark-variance-run-to-run // Check the website for updates as we post! Articles go up first. You might also like our latest hardware news video: kzbin.info/www/bejne/faebiH2ql9mKoas

[@__aceofspades]

Steve have you seen the Battle(non)sense channel? The guy there does a lot of network testing and input lag testing, hes a very small youtuber that has grown a lot in a short period of time, both you and him love to explore data instead of pizza warming machines, maybe you and him could work together on some content? I just think you guys both cover very similar but different content that each channels viewers would appreciate.

[@RitaMaSTeR]

You are always talking in "us" please make a video behind the scenes! with all your crew!

[@Delcolux]

Game request for benchmarking - Obduction. It features HDR, Ansel, and VR support, so it could be something neat to test with, especially if you get into HDR monitor reviews or something. Full disclosure, I am a hardcore Cyan Worlds fanboy, so to some extent I am just trying to drum up more exposure for their game. But I legitimately think it would be interesting for you to tear into the tech of the game. I would like to see a Digital Foundry video comparing the PC, PS4, and Mac versions of the game, but with a lot more insight into the graphics tech. I don't even know what I am rambling about at this point.

[@OhGeeWhy]

This is why I subbed to GN, not only you provide excellent videos but get on point. 5 Seconds in and we have the answer!

[@bothellkenmore]

Really appreciate how you don't talk down to your audience. Those of us that can't keep up will figure it out via our own initiative or through context over time.

[@randyyyyyyy6977]

This is phenomenal. I absolutely love the work you're doing here. You guys are maturing into true leaders in this category, and it seems to be on the back of just a shitload of hard work. Great work!

[@dragon2knight]

Jeez, I knew you guys tested thoroughly...but this is simply amazing. And the fact you're not resting on your laurels, well, what more can be said. This is the reason I jump on your vids as soon as uploaded. Great work guys, and thank you!!

[@astilen5647]

But he didnt even mention crysis

[@saladwithsalad]

The game is shit and is irrelevant.

[@ChiefFalque]

The game is awesome, but yes, irrelevant.

[@Sqtgdog]

In this video, Steve was rendered using nVidia HairWorks.

[@tee_es_bee]

As a long time SW tester, I applaud this piece. It is great to see that you pay attention not only to your testing methodology, but also that your audience understands how it is utilized. Keep up the good work!

[@boltanater]

Good guy nexus, has a click bait title and actually answers the question right away. instead of waiting till the end to announce it was inconclusive. This channel is top tier

[@GamersNexus]

How is "does benchmark duration matter?" a "clickbait title?"

[@boltanater]

Gamers Nexus I really had trouble conveying what I was trying to say. I just really like the style of the channel. I guess I misused the term click bait. Honoured to get a reply from you guys. I enjoy every video this channel produces. Never meant to offend by saying click bait. Edit: The comment you liked from Ogy better conveys my thoughts.

[@james2042]

You should contact linus and teach him and his 18 man crew at LTT how to properly benchmark stuff

[@Biga101011]

My video froze after 10 seconds. Thank God you led with the answer.

[@devon-crain]

Thank you all for consistently providing excellent data! You guys set and continuously raise the bar for this kind of stuff.

[@vapenation7061]

praise the lord, tech jesus himself

[@CattyRayheart]

I like that you share your methodology. It's interesting to hear about.

[@Robohac01]

Holy shit i didn't expect an answer to the headline in the first 7 seconds :P Great vide though and good explanation.

[@JustinHEMI05]

Excellent insights and I appreciate a deeper look into just how much effort you put into this for us. Best computer channel on TY, hands down.

[@Deltarious]

I had not even considered the necessity of nuancing your testing methodology as much as you do to achieve valid relative performance over a given test duration and the variance that some hardware might exhibit. That's an extraordinary level of dedication and only seeks to increase my confidence and support in GN and all of the benchmarks you put out. Thanks for all the hard work, and I look forward to all your future content!

[@GamersNexus]

Thanks for your support!

[@iwantmypot]

Good stuff. It's impressive the amount of work you guys dedicate to getting your testing done as fairly as possible to accurately represent the hardware's performance. Pretty much why I trust hardware reviews here over other channels. ;)

[@kostasbezaitis2695]

You are saints for this industry.

[@danceFuchs]

this intro was the best opening to a question-based video topic I have ever seen and saved me a lot of video skipping. thanks to the thought- and respectful creation of that video. please reproduce significantly to turn this planet into a better place.

[@zeniax4841]

This is the best tech channel though, 5 seconds in and you have the answer. Straight into the info we care about, gonna watch it anyway

[@parkerxgps]

Awesome subject, testing and data is cool. Title good too. Though I can't remember seeing bad titles. Top notch stuff. I Hope you blow away all your goals.

[@factofrealitytv4294]

i wish you all would also test the 970 and 980ti. those cards are still very relevant and within fair budget range

[@GamersNexus]

They are, but they are also irrelevant to the information in this content piece.

[@factofrealitytv4294]

i agree. just merely making a comment because i would like to see more tests on those cards.

[@GamersNexus]

Understood! Will try to include these more often.

[@crissto8591]

It's nice to see the new stuff ,vit ï sometimes also want to know what my hardware can do. The 970 does very well today, but it,s still interesting to see how it does before getting the games

[@HarryWizard]

ouch I think LinusTechTips just took a punch from Gamers Nexus (1 Year benchmark *cough cough*)

[@radar._.]

rip

[@katerwhall1865]

That's unfair. GN's background is that of a tester. who better to drill down and get to the point. Linus's background is that of an unboxer. polar opposites. 1 deep, 1 shallow.

[@Liatin1]

It’s just a theory... a BENCH theory!

[@betawolfhd]

I really appreciate the short answer at the beginning. Made me actually stay and watch

[@MichaelReznoR]

That's a very nice sum up of some of the intricacies related to benchmarking and how should PC users treat average data. I'm also very curious how do you determine/choose the benchmark spot/map/location in one particular game. These can be quite hard to choose, because there is never a "one wins all" solution. Some locations/maps/spots are more CPU-bound than others, and some more GPU-bound than everything else through the entire playthrough of one game.

[@hunkysdorys]

I never like videos but for the chancler of the ministry of technology and bomb ass Bolton hair I always drop a like......fucking stupidly underrated channel!

[@unchained_0177]

GUY'S you do very well and are my "go to" for 90 percent of my buys Keep up the good work !!

[@eccejas80]

An industry reference. Good work

[@408janus]

Benchmark idea I would love to see you guys do. Take a 2 year old, well used PC that has not been maintained (no reformatting, lots of bloat from use, no real cleaning, etc) and benchmark it. Then clean, reformat, maybe redo thermal paste, etc and benchmark it again to see if there are gains.

[@MrKZdemos]

Adoredtv is probably looking at his 600th GN article to find a misspelled word to called you biased and flawed.

[@Vaisero]

GODDAMN! THANK YOU FOR ANSWERING THE TITLE QUESTION RIGHT AT THE START, DAMN, THATS HAVING BALLS OF STEEL. CHEERS.

[@Pokemonfan4ever]

Have you guys ever tried Gears of War 4's benchmark? It takes 2 areas out of its campaign for the benchmark. It's not necessarily the best for extrapolating multiplayer performance, but it's still a good starting point. Plus, if you're performing well in the benchmark, your multiplayer experience will probably be smoother

[@jacobrzeszewski6527]

I've noticed certain settings that can affect certain hardware configs differently than others; for example, I had a FX 4300 paired with a RX 470 and could game consistantly @ a smooth 75 FPS when I optimized for minimal draw calls. Texture, Geometry, lighting/shader, and most reflections settings had little effect on frame rate; though, in Dirt Rally, adding even low amounts of foliage killed frame rate and introduced stutter, despite the GPU not being a bottleneck. But these settings effect games differently. I would like to see a video on game settings optimization to help people understand how the CPU and GPU work together to make a game run, because honestly, I'm a bit confused myself.

[@OdinAlgeron]

a fx 4300 with a rx 470 is very unbalanced, plenty of gpu power but seriously lacking on the cpu

[@jacobrzeszewski6527]

Sorry it took so long to respond, but the seed I'm trying to plant isn't mixing a weak CPU and powerful GPU, but instead what different combinations thereof are capable of, and how to take advantage of them. AKA a video dedicated to what certain game settings do, and how they affect both the CPU and GPU in proportion to one another.

[@Mr1Tanker]

Very interesting stuff, and it seems quite logical!! This is the excellent content that set's you apart from almost all other KZbin tech channels! WRT to testing games in the future: What about racing games/sims? I find Dirt4 is very hard to nail down...with a GTX 1060 6GB/ R1700X@ ~3.95GHz and 32GB of DDR4 @3200 C14.... with the same settings (mix of High/Ultra), and using RTSS (Shadowplay), FPS can swing between 40 and 140 over the course of a 4 minute race. The problem is going from open fields into forest tracks, and back. So, i lower tree quality to bring up the lows, and now the whole spectrum changes to ie. 60-170. Tearing and stuttering are side effects, as i only have a 24" 1080p 60 monitor. I'm now so desperate, that i'm using V-Sync, and it's smoothed things right up, i don't have to OC my GPU, and temps( and obviously fan noise) are lower.

[@Thermophobe]

appreciate how meticulous you guys are about performance testing. i would really appreciate if you guys test CSGO and even make a performance guide to the likes of overwatch. this game is very mature now, is easier to benchmark than other online multiplayer. This can help a lot of players who want to ooze out as much frames as possible!

[@andrewskaterrr]

adb27 there’s not a reason to do it for CSGO, the game is old and there’s tons of fps reviews on it. It’s not hard to run unless you have a low powered cpu

[@dnaofgamers9595]

Amazing work that you doing guys. Keep at it, it's why i stay subscribed and look at each video even if sometime i have tough time to follow the flow of data you trow a us as english is not my primary language ! :)

[@Dskumm3312]

i would like to see you use wolfenstein or some other vulkan based title.

[@madfinntech]

You are the one with 580 who disliked the video?

[@Dskumm3312]

MadFinnTech no that does not sound like me

[@SeriusSim]

Nice upload! Here's a question for the next Ask GN, if you happen to see it by any chance : Prior to the Vega reveal, there were rumors of a dual polaris gpu card (possibly two 480s on the same board). Is there any chance for either an RX 590 or a dual 580 card to get released eventually?

[@clandestinerobot14]

Can you say mind blown@! Sometimes I watch these videos just to get an idea of how much I do NOT understand. The amount of meticulous data mining (no pun intended) over the span of a year.... That's just next level guys!

[@justinadson3412]

I appreciate your work guys . This stuff is great.

[@raulsaavedra709]

About games to use for testing: Hellblade's Senua's Sacrifice

[@NotQuiteAsianOG]

Before I have watched the video and this is my opinion as I work at a calibration lab for an aviation test solutions manufacturing company- benchmarking should always be done in climate controlled area to be able to record equipment ambient temperatures getting base line when not at 100% usage and then monitoring them again once the equipment has met its optimal performance Temps then benching it on that temp to ensure it can hold that performance temp over a period of time based off repeatability and reliability over load not off one test run. In a cal lab we use the term traceability to know that when checking one item with another item that it'll have credible 4:1 accuracy ratios over the DUT (device under test) and that item that is used to cal the DUT is cald by something of 4:1 ratio of that which everything at the end should be traced all the way back to NIST (but since you're using internal components to measure your overall scope will vary, but let's just say the accuracy of the program used to capture data and the components used to measure (on board diagnostic chips) 100% accurate with no uncertainties). For a tighter tolerance in measurement we at least do 10 benchmarks (time consuming for sure also I feel like I went on a tangent). Anyways keep up the great work your videos are always good to watch.

[@kayos1203]

Amazing video as usual :D Can you add starcraft 2 in your gaming benchmarks?

[@LarsBars17]

Awesome science going on here. Would love to see World of Tanks or World of Warships get benchmarked.

[@Qweasdzxc-q1i]

Man I hope you become the best review channel!

[@Qyngali]

You should plug all of the data into a database and put it on the website and add some auto graphing. Let people select say a 580 and a 1060, select game, resolution and settings, (show which driver was used ++), maybe switch CPU to see how the performance changes. Etc etc. But you probably do something similar behind the scenes already. :)

[@HarlesBently]

Qyngali gpu.userbenchmark.com/Compare/Nvidia-GTX-980-Ti-vs-Nvidia-GTX Like this website? I use it a lot when buying used gpus and such. Gives very nice stats due to taking multiple sources from more than one gpu and by users. Love gamers nexus but they cant have this much data just because time constraints.

[@crissto8591]

It would be nice to have accès to their data tho. We know it's proper and reliable benchmarks

[@technologu]

would you use Kryonaut UNDER the IHS? since you recommend the product

[@ouroesa]

I loved that intro!

[@2000cobraguy]

I would be happy to purchase with GN links but I try not to buy from Amazon, as I much prefer Newegg. How about Newegg links? Thanks!

[@kuuzer]

Idea. Use of game speedruns for "realistic" benchmarks. They would be relatively consistent and representative of real gameplay to an extent.

[@matthewkinney5474]

What software do you use for recording framerates and frametimes?

[@JETWTF]

I thought you had a system in the back of the shop that's been running a benchmark for a year when I read the title. The results would be very accurate if it's average FPS.

[@bobohbuboh8604]

Thank you PC Jesus and all the staff for your work, very interesting and thanks for sharing your data. Kids take note: that's what happens when you know ur shit

[@gijoeljamesmaldonado]

Tech Jesus, my savior.

[@sudd3660]

looks like 30 sec of quality benchmarks shows enough data for frame performance , i think of longer test valuable for testing clock stability since temps can become a factor.

[@astilen5647]

Thank you gamers nexus ( . Y . )

[@XFourty7]

22:56 Can you test Commander Keen? I only get 16 colors and like 10 fps!... Lol jk obviously... Thanks as always Steve.

[@Visentinel]

Is there a way to monitor ddr as a bottleneck while playing games, because it just normally shows as cpu usage while the cpu just sits at 100 percent waiting on memory.

[@CST-ft9dv]

Great info...TY!

[@rustyheckler8766]

Do "Star Citizen", so you can revisit it as it moves along it's alpha and beta testing to see how well optimization is coming along and also best practices and settings to run the game.

[@rustyheckler8766]

Rodri Labarre Its not that bad, 16gigs does the trick though 32 will probably end up being ideal.

[@kin0025]

Main problem is that performance isn't easy to keep stable - the number of players, and time the server has been running affect performance significantly, compared to different GPUs

[@ntomnia585]

AdoredTV shield activated? LoL

[@dodoroa]

Thank you.

[@paulgo1146]

Could you still test GTA V in 2018? I know you dropped it recently, but could you follow up why you will or won't be providing this game.

[@regnittoekim46123]

Would like to see a cpu benchmark for Cities skylines

[@dcphillips1991]

Damn I thought we were going to a conclusion in the first sentence of the video xD

[@LumenateTV]

star citizen testing would be nice.

[@WakenWanangi]

GN is like the only channel that I would base my purchasing decisions on. But on the topic of playlists, you guys are really lacking.

[@jeremybeezy]

This is why you the MVP Gn.

[@sumithbhat]

Great videos but shorter ones would have been better

[@josephmcgolrick3920]

Holy mother of data analysis...

[@gudenau]

Can't you calculate the FPS based on time between frames? That's how I would do it in an engine at least.

[@scorpizy]

Ok GN. Are you good and what we live in is the matrix that you created and the big bang is the boot up?????

[@CamelBiden]

Id recommend the Final Fantasy PC benchmark

[@RambUz]

No dx11 games that came out before 2016. Then i would like to see 1-2 games that are optimized from both sides red/green. Vulkan and dx12 games. FP16 should be in the list to like Farcry 5.

[@analkarldervierte1759]

And then there's Linus Tech Tips, who just put a "FPS per Dollar" chart at the end of their graphics card reviews.

[@raleighcockerill]

I'd love to see some rainbow six siege tests if the game is a good bench. If not don't worry about it

[@VileLasagna]

Even though the video is not ABOUT the 580, recently with Vega being completely unavailable here in NZ and me still needing a GPU, it ended up being pretty good information to absolutely not get a 580 as I'd end up disappointed. Feels bad man (AMD fanboy here so trying real hard to stay away from Team Green).

[@benbai5808]

Vile Lasagna at this moment in time if you are lucky enough to find a good deal on ANY gpu you should snag it asap regardless of its green or red.

[@VileLasagna]

I think I might be able to get a second hand Team Green 9-series in the near future, so might end up having to make do with that. Still disappointed but I can 100% see your point. At least it'll be good for working as I need to deal with CUDA From regular vendors, tho.... Oof.... Vega when listed is even more expensive than it was at launch, about 50% over the best prices I've seen (shame I couldn't fit it in the budget then), 580s are AT that best V56 price now. And even in the lower end... remember getting a LP 1050Ti for my HTPC for some 260NZD last year, it's another 100 on top of that now.... Sad = very yes

[@SupremeRuleroftheWorld]

great information, but for god sake, buy a better table. the moving table is REALLY obvious and annoying.

[@thetriplea9928]

Could you try The Division as well? This game is still an impressive title visually, buy it has the worse in game benchmark I've ever seen. This game uses like 20% average during benchmark with my 1950x. But the game is brutally CPU intensive, I sometimes get above 70% CPU usage during normal gameplay. The optimization is sometimes outstanding, and wired at times. I think this title is a very good example of crap in game benchmark.

[@Plague_GG]

Interesting, not as “black and white” as I had thought previously.

[@OlettaLiano]

Interesting stats, but I can't afford either or these card sat their current prices.

[@sandflesh]

so how much did you make after one year of mining?.. id consider that a stress test.

[@cowpker4life333]

is someone on the GN team a stats major or something?

[@maxfacts1]

GN is my go to channel/site for real world BS busting benchmarks. It's good to see GN growing its SUB base, 1,000,000 + coming this year. Nvidia's massive performance per watt advantage makes buying a factory overclocked Pascal 1070 the obvious logical choice for smart gamers who should avoid being a brainwashed AMD fanatic wasting money on a watt sucking, lower performing, global warming Polaris 580.

[@wakesake]

Benchmark duration matter when reviewing anything with liquid cooling

[@GamersNexus]

Yeah, but these aren't liquid coolers.

[@saberlo8296]

Benchmark Bloodborne please! It's an "old" game but i think it's still graphically intensive. jk, love your vedioes, steve

[@madfinntech]

One person bought 580 instead of 1070.

[@Fals3Agent]

please please please PLEASE test rainbow 6!!!

[@ayushdabral]

Mind blown

[@AshleyandLancelotShow]

nice vid

[@scott6634]

SOMETIMES 0:04

[@Tempest12322]

id like to see some testing with Esacape from tarkov, it might not be easy and it might not be uses for a long time, as it is early access, but i like your benchmark numbers and testing methodology than alot of other random youtubers

[@MrBriman27]

Rainbow six siege gtx 1070!

[@SerjStar]

I think right now with the shortage bang for the buck is the Rx 560 I found it actually cheeper than the gtx 1050 even so that concludes that Rx 560 is a good delay right now if u can find it snag it . It on Amazon right now close to MSRP price

[@thatkoalabear]

is this the tech Jesus we all been waiting for?

[@tylerhinman505]

You bench your ass it does...

[@JONNYGEEKMANG]

What about out of the box benchmark duration?

[@SerjStar]

I now switched over from 240 master liquid cooler master to the Corsair h110i 280 and man is that thing loud as fuuuuuck and I thought coolermaster was loud my gosh but the CPU is maintain 5ghz overclock under the max no thermal throttle and most of the time it's more consistent other than the cooler master 240 I feel like the cooler master pump isn't so good cuz I had push pull on the cooler master AIO so I know that fan isn't the key the pump and liquid inside and the surface of the radiator does more than 240 mm

[@ImHughJassol]

Steve, how much for a lock of your hair? It's so mystical.