Serialize and Deserialize a struct in C++ - Stream-Based I/O part 8 of n- Modern Cpp Series Ep. 198

[@scullyy]

Overloading, specifically the ostream operators, was the first thing I learned about c++. std::cout

[@jameshansen801]

I've felt the same. I still have yet to figure out how and when to manipulate the spaceship operator when it comes to operator overloading.

[@nahweh5938]

Nice intro, Mike. This is just the tip of the iceberg. If your project is long-lived, it and it's data will evolve. You'll be dealing with serialization issues with each new version. How will you handle deleting a field? Where will you place new fields? That's just the beginning. At that point you'll likely look at designing something far more generic probably with a fair amount of meta-data to assist in handling prior versions. Ugh; if you have external clients, you'll have to maintain the entire version hierarchy because you can't rely on the clients being up to date with your software. There's a playlist for you. Good start, Mike. I do like it.

[@MikeShah]

Agreed -- It's a deep topic. Some folks will literally write things out to json (or some other structured format) and version the data with versioning -- though that's going to vary widely if that's a feasible solution (i.e. file sizes will grow large!). Plenty to talk about, and it's an interesting problem for sure!

[@pranavlohar8731]

Hey mike, i am pranav and i just started leaning c and c++ with your videos n it's going very good thanks to you . there is suggestion of mine is that, Sorting the playlist by topics or sequence will enhance learning and ensure a smoother experience. Like : 1)Basics of c++ 2)Array & Pointer 3)OOP 4)STL Thanks for your time

[@MikeShah]

Cheers and welcome! Yes to some degree this playlist can be followed linearly. courses.mshah.io roughly organizes various topics into modules as well.

[@jameshansen801]

In your last example after you de-serialize; how does the Gameobject know about the data the ifstream just read? In earlier examples, for example when you would use the getline method, another variable was needed so you could pass along what was in the stream, then pass that variable to std::cout so that you could verify what was read. You didn't have to do that here, and yet it worked. Why is that?

[@MikeShah]

Each call to is.read in the Deserialize function is effectively 'reading' another chunk of data. ifstream's keep track of the position of the buffer object internally (you can move it with things like 'seek' and so on -- see a few earlier videos in the series on streams for some visualizations :) ).

[@jameshansen801]

@@MikeShah That part I get. What I'm asking is that are you able to directly pass the results of the input stream into the output without needing to create a variable to handle the middle part of the task? Also, if I understand correctly your intention was to overwrite the contents of the 2nd GameObject, not just read the data from the file?

[@MikeShah]

@@jameshansen801 Sure, could use fstream which allows for input and output operations. And yup, the goal was to write over the contents(field1,field2,field3) when deserializing.

[@robertstrickland9722]

I always get tripped up with these streams when it comes to implementing custom streams and what the CharT and Traits template arguments do for the std::basic_ streams. For instance, if I wanted to create a stream that worked strictly with uint8_t and didn't have to constantly cast it to uint8_t when using regular char streams.

[@VoidloniXaarii]

🎉🎉🎉🎉

[@josiascabrera8404]

Hello everyone! Can you help me please, I am wondering what is the best approach to create your database in c++, is it through binary files? I would like to know the best way if am planning to extend my project in the future. Thanks for your support!

[@androth1502]

wouldn't it be easier just to (de)serialize the entire struct at once by just reading and writing the size of the struct? processing individual fields might be necessary only if the field members are pointers that have allocated objects that need to be dealt with separately.

[@MikeShah]

I think you answered the question in your second half as it sort of depends -- if you have variable length data (e.g. a graph, a string, etc.) then the size of the struct is not quite what we want. Serializing a float is also another interesting challenge itself.

[@androth1502]

@@MikeShah i guess that's the allure of formats like json. they provide an easy way to serialize variable length data.

[@revealingfacts4all]

I see this as way over complicated. Why not just FILE *fp; fp = fopen(); fwrite(&go, sizeof(GameObject), fp); and be done with it. This way if I update or change my GameObject, I never need to go and find/update the serialization/deserialization logic.

[@amber1862]

Portability: fwrite isn't the same on every machine. Efficiency: you don't always want to serialize everything and you can compress specific data in different ways. Debugging: you can make parts human-readable to see if things are being serialized correctly in the binary file.

[@MikeShah]

@@amber1862 Yeah, the trick is, if we just have a POD with primitive types we can indeed probably get away with something trivial like fwrite. But as Amber has mentioned, once you have variable sized things (data structures, strings, floating points) then there's more work to be done.

[@revealingfacts4all]

@@amber1862 portability? I'm my travels fwrite/fread works on ARM, and X86 hardware with clang. Gcc, and embedded compilers like green hills multi and vxworks version of gcc. So I beg to differ. The other argument about not wanting to sterilize everything is irrelevant because that's a design issue. Further, std::ostream isn't even available in some of the embedded c++ compilers I have to deal with yet read, write, fread, and fwrite are. I still think this video way over complicates things.

[@ensuretime]

@@revealingfacts4all Take a look at zppbits, disregarding the amount of templates and metaprogramming, this library does magic with reflection in c++20.

[@rasitsimsek9400]

Please don't use platform dependent types like integer, long and short etc. These types can be different on different platforms / OS. You write in Windows and want read on Linux this can be sucks. C++ has inherited the the uncertainty of primitive types from C. You can't trust the size of these types.

[@MikeShah]

Agreed -- forget if I mentioned it in this or the previous videos -- this is exactly where the fixed-width types help! Then we're just left to deal with endianness, but ideally from a spec we can figure out the types and data sizes reasonably.