Star Citizen's first server meshing test, was it a success?

Recently Star Citizen had its first server meshing test on the Tech Preview channel. The test was only open to Evocati. The elite cadre of testers that play early builds of SC before it gets pushed to the wider PTU and eventual LIVE.
Thanks to CIG promoting more transparency, we Evocati now are allowed to talk more openly about the behind closed door tests like this one as well as the patch notes. Join me in today's video as I talk about this first static server mesh testing. How it went and what it means for star citizen's future.
Check out the static server meshing patch notes:
robertsspaceindustries.com/sp...
Learn more about server meshing in Star Citizen:
robertsspaceindustries.com/co...
TIMESTAMPS:
0:00 Intro
1:23 What is server meshing?
5:23 My experience with the test
10:03 What is next?
14:19 Outro
~~~
Join the Discord!
/ discord
Looking for a disciplined, teamwork-oriented org to play SC with?
Apply to join The Forgotten Heralds today!
forgottenheralds.org
Interested in creating an account to play Star Citizen? Hold on there buddy! Be sure to use my referral code when you sign up and you will be given an additional 5000 credits which you can then use to buy extra gear for your character or ships!
Code: STAR-4L6V-VLWM
~~~
System specs:
GPU: RTX 4090
Mem: 64gb DDR5
CPU: i9-13900k
Storage: Gen5 m.2
#starcitizen #gameplay

[@HybridVAudio]

I'm looking forward to seeing how server meshing progresses and if it can help with performance. Again, I think it was the reduced cap that helped with performance, it may be that server FPS may still not be the greatest even if server meshing releases in the near future. More servers need to be spooled up to assist with the load bearing of all the players and entities that need to be tracked. We'll see how things pan out. Hopefully, CIG will talk more in-depth about it in future ISCs and SCLs. Did you get a chance to try out the test? Let me know! See you all in the next one. o7

[@thedoctor5478]

Let's get real about the NDA. There are ppl who upload evo content on youtube. Once that happens, you can cover the uploaded content. I'm pretty sure that's how that works.

[@tropicthndr]

Salty Mike said it was a flop, servers crashed and rebooted in three minutes, still garbage.

[@HybridVAudio]

@@tropicthndr I agree with Mike that 3 minutes is not good. It is a lot better than before. But still, it completely throws off the tempo of the action. If they can continue to improve recovery times then I'll be satisfied. However, I don't see this as a flop since it worked just fine and even when it didn't, it recovered which is already leagues ahead of what we have now.

[@ErgonomicChair]

... star citizen is not the "largest scale" of server meshing, not even close. That would be Second Life, and a few other projects before it. Dynamic Server Meshing, Server Meshing, etc does not fix shitty architecture btw. It means making shitty server architecture, which CIG has and SC has, and making it live communicate with more shit live instances. HDoing handshakes, interpolation,e tc. Server Meshing is a waste of time and a boondoggle, game devs discovered this fucking decades ago. It won't help performance btw. It will worsen it, dynamic will worsen it even more. The hilarioust hing is CIG hired Network Programmers (not engineers btw!) who were out of the industry for 20 years+ each, and they aren't up-to-date enough to understand that this technology is not practical with our internet OR our hardware.

[@NozomuYume]

Dev here who's worked on similar server meshing issues -- it's basically an information distribution and authority issue. This has been solved for high latency applications -- think about how you can interact with everyone on Facebook even though Facebook is running on tens of thousands of servers, no one server is controlling the whole shebang. What you do on one server can be expressed via another server that another user is experiencing, e.g. you post a comment using one server and it enters a distributed database, and a notice of the change is sent to parties that are subscribed to this information. I worked on a RTS that implemented something like this, and we got the latency good enough (tenths of a second) so a vast number of players could share a single world. An RTS can get away with this. The groundbreaking unique challenge for CIG is they're trying to do this in an action game where latency needs to be measured in milliseconds.

[@ErgonomicChair]

... nothing is groundbreaking with what SC is doing. There are live enviornments running what they are trying to do, they just control at times, with DIlation etc. This makes me think you are talking out of your ass. The server still 30k'ing is hilarious because running multiple parallel servers that can't even handle more than 30 players together, now needing to do interpolation, handshake, physics and communicating liveas people get near the border will KILL it so fast lol.

[@anthonyrodriguez96]

just a correction because i dont think people understand the monumental achievement server meshing is, other games, such as sea of thieves, do not have anything similar to server meshing technology. they have server transitioning. server meshing serves an entirely separate purpose and once completed will be sought out by every triple a and indi dev making a mmo game to completely change the nature of what we know as a multiplayer online experience.

[@andrijagrgic2357]

I still can't see the achievement by your comment. Can you elaborate?

[@NotAFoe]

​​@@andrijagrgic2357one thing many people say is: you shoot a bullet and it transitions to a different physical server with no delay and hits a target. Or having a star system with 1000 player cap due to there being 10 100 player servers.

[@andrijagrgic2357]

@@NotAFoe so like a huge single shard only on multiple small server? The way i interpreted is, it's just automated server hopping. If i jumped into pyro and my.mate a few moments later. I land in one server but it's full and he on the next server. Would we technically still be together in the same "mesh". Or would we seperate at that point?

[@Jeremy_Walker]

I’m lost as to how anything was meshed. They had two separate servers running two separate zones. No passthrough…

[@anthonyrodriguez96]

@@Jeremy_Walker data was passed through. Both servers were communicating with each other and they tested how crashes would effect the replication layer

[@tlove21]

This was to test if the 2 servers can connect to the Replication Layer. Now that it is proven in the Live environment, then they can do handoff tests. So yes it is a major milestone, for this test shows that they can have a server per planet instead system. So only the handoff test needs to work then we can be sure it works at scale.

[@Deathbyhampster]

Thanks for the update on this! Everyone is excited about this feature and it was great news to hear the test went well!

[@Kevlar-78]

Thanks for the update / recap homie ! Cheers 🍻

[@Anonnymouse53]

A long way to go yet, but it's encouraging to finally have proof of life for this part of the project after so many years. If they can make it scale far enough & get the latency down where it needs to be, it will be a pretty big milestone that reaches beyond this project and might propel CIG longer-term

[@KaryudoDS]

The test seems like it was intended to test removing a crashed server from a shard and adding a different server to that shard, from what people have been describing. Which is still very important if not slightly less exciting that the transversal between servers (okay maybe way less exciting but I guess we saw that in theory already once). Only thing I'd add to the static vs dynamic server meshing idea is that with static you aren't necessarily running a server for areas without players usually until that condition changes although you could. The big gain from CIG's dynamic idea seemed like just being able to adjust server boarders while the shard is live and that seems like it would save some money, but I'd think a granular static model should also work just fine for what Star Citizen is doing. At this point the idea of dynamic meshing sounds nice, but the servers can barely run an entire star system at once as it is so I'm not sure how much they'd really be able to squeeze out of that.

[@frooxiegamer]

Thats why static server meshing can also work as one system spreaded on 2 or 3 servers which will evenly distribute load. And thats CIG next goal after they manage to connect 2 systems on 2 different servers trough replication layer. One server is already struggling with Stanton, Pyro is double of size so if they want to work in future they need split Pyro to minimum 2 servers. Thats why in tech preview for now we have access only to the center of the Pyro system in almost same size as Stanton.

[@MalmqvistM]

Did NPCs seem more responsive, smarter with the higher server fps?

[@Nine-Signs]

They already have their moments even with a normal boggy server. I had an NPC bounty that was on the planet floor, I was coming up on it from above and was real close when it took off from the ground instantly firing a missile at me while the NPC yelled "eat this". I was quite impressed as I died.

[@stormw777]

The best explanation I have heard of what server meshing is.

[@Nemethon]

Nice to hear. I'm looking forward to hearing how they test the star system portals. :)

[@thx1137]

Good video. The inability to jump between systems was simply a bug. They could have cancelled the test but decided to do it anyway. So, the next test will definitely allow that. IMHO! Pyro had more bugs than Stanton, but the experience was similar to that described here. Everything seemed very responsive even though the replication layer was servicing 100 players (2 x 50) so I found that to be really interesting given how the RL can have problems with 100 players in live currently. The test certainly made me optimistic, but I do expect any real issues to happen when transitioning shards. I think the fact that a bug would cause everyone to disconnect shows that is the next tricky bit.

[@hawkzulu5671]

The next server meshing test is Friday. They solved the bug - so this Friday test will include players using the jump gates to travel between Stanton and Pyro. Cant wait to see and hear the results.

[@j.d.4697]

EVE has static server meshing. I am pretty sure every game that has instance on separate servers uses server meshing.

[@ThreatCultureMedia]

Jesus, those tanks in the beginning were eating those FS9 bullets like pebbles lol.

[@51ghardy]

Good stuff. Thanks

[@festersmith8352]

Everyone nitpicking every word and detail. What they tested worked out really well. Thats what matters.

[@Skyrat12]

I was in a server of 46 people in Stanton and was getting between 15-20 Server FPS. Until they figure out the networking, they should limit Live to 50 players like before. Just allowing MORE people into a game, doesn't an MMO make. But we know it's not about the player right now.

[@anthonyrodriguez96]

they cant, the point is to get every player in existence on a single replication layer. reverting back to a lower player count does nothing for their data

[@Skyrat12]

@@anthonyrodriguez96 yeah I know, but it’s still frustrating after 9 years.

[@anthonyrodriguez96]

@@Skyrat12 trust me bro, i know. but for whats being projected, im more than willing to wait for polish

[@Skyrat12]

@@anthonyrodriguez96 I hope so! Guess only time will tell. Cheers.

[@d34d10ck]

Don't confuse replication with simulation. The purpose of server meshing is to split up the simulation amongst multiple servers and not so much the entities that are replicated to them. That's what the replication layer is for.

[@matt-88]

There was no meshing, it was just a server selection screen! 😂

[@bobstark4201]

I think CIG increased server cap from 50 to 100 to save money by not having to spin up as many AWS servers. That is partially why performance is so bad at times in the PU. I have serious doubts on server meshing actually benefitting SC backers. They would have to spin up quite a bit of servers to make it work. That's going to cost more money than they're likely wanting to spend. It seems player experience will always be a tug of war between needing more servers and CIG wanting/needing to save money. Server meshing won't change that.

[@Jeremy_Walker]

Two separate and segmented servers… not sure what was replicated or meshed.

[@Billy-bc8pk]

Data between a single replication layer. Players who were in parties who joined Stanton and Pyro separately were able to voice chat with one another.

[@SC4RFAC327]

How can CIG improve server FPS? Is server meshing the fix for that

[@festersmith8352]

Yes that is the plan.

[@theempireofthepeople]

Wtf?!...20 f*cking seconds ago?!!!

[@grasthube]

Is this vaporware still going on?

[@festersmith8352]

Heck yeah! And we love it.

[@littlefoot2869]

look if only test small groups they will never work on it work one minn and shit out the next ,this keeps happing over and over ,if they put it to live i can almost bet on live it will end up like all the other events works for a few then no longer works just like when trams crap out been the same issues going 5 year i been around for this .

[@DracoEX]

Heres a major problem i see with server meshing. Network lag, processing lag. If server A is sending data that needs to be close to real time, fps combat, any data has to travel through the replication layer (lag oppertunity) to server B (second lag opportunity)

[@festersmith8352]

Its been discussed heavily, and you can't change laws of the universe. Lag is in every game.

[@DracoEX]

@@festersmith8352 but you miss my point, im sorry that the education system has failed you that i have to spell it out for you. NORMAL communication between a game like overwatch, final fantasy, world of warcraft carries out communication between You the player and the (single) game server that you are on. When you increase the amount of servers your ping has to bounce between that will cause you to have more lag than a regular game. Primarily like a FPS like call of duty where this lag migitation is key to the games success. Star Citizen is going to lag more than a normal game.

[@festersmith8352]

@@DracoEX And as I said its been discussed, and made me more aware of the issue. Its something no one seems to have a solution for, because you can't change what you just described. Though I know game called Mechwarrior who created their own tech to counter the issues caused by lag. And I have no doubt that CIG will also try to as well. Do you offer any suggestions that may help the situation. Or do you like to point out the obvious negative, while trying to cut others down. People way more educated than both of us are aware. Some of those people I listened to, educated me on it. Take the time to research and educate yourself.

[@paulcotton1992]

wish they make the ships heavy not like paper airplanes.

[@Billy-bc8pk]

That's coming with flight control surfaces and Maelstrom.

[@MikePhilbin1966]

So, ship Physics collisions are still as ugly as they've always been then. Is there even Physics in this game, or is it all control curves?

[@liolio9282]

2 players at a time maybe no crash 😂😂😂😂

[@theheromeat9367]

Bunker missions are about to get difficult 😅 they All john wicks down their 😂

[@nahbro3240]

Cant wait for the "finaly server meshing is implemented omfg biggest milestone ever in star citizen!!" Only to find out is server meshing part 1 out of 10 more added updates to it over the next 3 years.

[@Nine-Signs]

"it's early days" 😬

[@osuave7811]

Eather way, it'll be epic.

[@Timbodacious]

there was zero server meshing in this test.

[@Jeremy_Walker]

Facts

[@raptorkysen177]

2 servers connected to 1 replication layer , so it's definitly server meshing , and players not allowed to move between servers does not change anything regarding this statement People have to distinguish the back end tech , and the player/user frond end those are 2 different thinks regarding software what a user can see and experience is only on the top of iceberg of the software content

[@Jeremy_Walker]

You people

[@fizzywhizzbanger5610]

Are you sure you understand what server meshing is?

[@Jeremy_Walker]

No. Seems like some sort of logic layer that ties independent servers together into some sort of cluster.

[@thedoctor5478]

They didn't even activate the jumpgate though. So idk what this accomplished. Assuming it went well though, I imagine they intend to activate the jumpgate in evo super soon.

[@Meister_Dwight]

Both servers were connected to the same Replication Layer, plus server crash recovery was much faster, like 1-2 minutes instead of 10+ like before. It's a very early test for data gathering.

[@OGDMaco]

From what I understand the jump gates were able to be used but there was a bug attached to them at the moment which would break everybodys game. Super looking forward to using the gates soon though but don't plan much to go to pyro

[@thedoctor5478]

yeah, it's all good to me. My biggest concern is that server meshing might not improve server framerate by a significant amount. I hope it does though. I'm also worried about desync during hand-offs. I'm a network & software engineer of 25 years btw.@@Meister_Dwight

[@Nathanity]

@@thedoctor5478 Static server meshing will have little impact on performance. Dynamic server meshing is what will save the games abysmal performance.

[@Meister_Dwight]

@@thedoctor5478 Yeah I don't believe this initial "static" implementation will, we probably will have to wait for "dynamic" where they can have servers running much smaller containers like planets, LZ's, ships etc. before we see a notable improvement.

[@Suirady]

It's just impressive to me how little progress has been made on this.. a lot of other games have server meshing.. and for CIG to work on this since 2017 just to have so little to show and the community thinking that having 2 servers with 50 players each, with no interaction between them is a milestone.... it's insane, delusion level of insane

[@nahbro3240]

😂 true.

[@AlpacaZ]

Literally no other game has server meshing, or at least the server meshing CIG is trying to accomplish, not even close.

[@Billy-bc8pk]

There isn't one game that allows entity transfer from one server to the next in real time via a replication entity layer. No game engine exists that enables a game to perform this feat, which is why CIG had to R&D the tech to do so.

[@paulh2343]

Just because you're unable to understand it, doesn't make everyone else delusional.

[@jakobfriedrich5117]

Do you know what the replication layer is? What it is used for? It's the way for those two servers to interact with each other and replicate what is happening on the other servers. What you were writing is just plain wrong

[@-_Nuke_-]

Let's take a look at the "Server Meshing" testing... Abstract: _No meshing has been achieved yet_ The test featured two servers. One for Pyro. One for Stanton. *[1]* The Server you played on could be selected ONLY at launch. *[2]* There was NO WAY to transition between servers during play. *[3]* Global chat did NOT span across servers. *[4]* Nor was any information shared across servers. *[5]* From *[1][2][3][4][5]* we see that this was not Meshing. *[6]* Any use of the word meshing here is a lie.

[@raze4789]

I like when people don't: watch the video (1) listen to what is being said (2) know anything about what they are talking about (3) (1)(2)(3) tells me OP is a clown.

[@-_Nuke_-]

@@raze4789 That is irrelevant to the video. What I stated was just the tldr truth

[@raptorkysen177]

@@-_Nuke_-People in pyro were able to see stanton missions People in party could do voip accross the 2 systems People in party accross the 2 systems inside one shard could see that they had the same shard ID People could experience on server crash which did not affect the rest of the shard So you're wrong

[@jakobfriedrich5117]

You don't understand. The servers don't communicate directly with each other, but with the replication layer. This was the first time actual game servers have been connected to the replication layer. So yes, while the players didn't notice any communication, the servers were sending their info to the RL and we're able to pull it from there which is the technical prerequisite for that

[@-_Nuke_-]

@@jakobfriedrich5117 I agree with your statement! But what you described is not meshing. Its something else. Real meshing will come down the line - this was not a meshing test, there was no server "meshing" happend yet.