Why Linux Has This Syscall?!

Source Code: github.com/tso...
Here is another lecture style video similar to the previous one. I'm not sure if these kinda videos are watchable, but they liberate me from the chore of video editing which I can't stand anymore, I really apologize for that! The need of editing the video was probably one of the main reasons that demotivated me from uploading here too often. I'm gonna keep doing these types of videos and see how it goes.
The videos are recorded live at my Twitch channel: / tsoding

[@greob]

Thanks for sharing these stream recordings! I can never join in time for the live streams so I'm very grateful for these uploads. Very interesting and you make things easy to understand. Much respect! :)

[@leong4847]

I love this type of video! Definitely don't mind the lack of editing, it's great to see the full process

[@4aasdaw]

Edit: The current title "Why Linux Has This Syscall?!" shows how misleading English can be. In the current form, this title is a subordinate clause -- a sentence which needs a leading sentence to make sense: - "This is the reason *why Linux has this syscall*" With an addition of a 'do' this subodinate clause turns into an independent question: - "Why *does* Linux *have* this syscall?" In Russian, the present form of a verb is the same form that is used in questions (particularly if we disregard prefective aspects). However, English isn't such a language. It differentiates verb form, sentence ordering and a lot more, even between types of questions (i.e. where/why/what). In the case of the 'why' question: the third person singular simple present form of 'to have' is 'has': - "Linux has this syscall" But, in questions, the active verb is wrapped in a do form ('to do' in respective tense + infinitive verb): - "He goes to school" / "Why *does* he *go* to school?" - "I ate an apple" / "Why *did* I *eat* an apple?" - "Linux has this syscall" / "Why *does* Linux *have* this syscall?" When used to slavic linguistic patterns, where the verb form in statements and questions is not distinguished, it is a bit unusual to place unnecessary constructs like 'do'. Furthermore, this 'do' rule does not apply for all tenses; For example, most passive verb forms in questions do not use do: - "A crime has been investigated" / "Why has a crime been investigated" This is because (oversimplified a bit) the verb has already been wrapped in a supporting construct: the 'has been' passive form. English is full of these redundancies and obscure rules with which you can easily guess the speaker's native tounge, provided you know it. Most of the time, a word-by-word translation to their mother tongue will get you the meaning behind an otherwise unintelligble statement. On a more interesting note: languages, as they age and get more fine-tuned, tend to move information from a single word into more complex structures. For example, in Russian, you can easily omit the subject, and with just the verb form you can tell the perfectiveness (has the action been finished), the subject, the transitivity (existance of an object), etc. In English, there is a lot of legwork getting all this information through as it usually involves many overlapping constructions. A clear example is the use of prepositions in favour of grammatical noun cases like those that Latin has. At first, it is a bit counterintuitive why it'd be easier to split words like this. The real complexity comes not from the amount of words necessary to convey a full meaning, but the mechanisms of forming those words that the speaker uses. If you think about all the productivity noun declensions, verb tenses, position and even intonation have, and the necessary premeditation to put together a meaningful utterance, it becomes clear. I think the same applies for programming languages: reusing short 'idiomatic' code to achieve greatly varying results ends up creating unnecessary illegibility and confusion when reading the code. btw, this type of video is fine by me

[@smuecke]

I wouldn't say that languages "get more fine-tuned" as they age; I'd argue they're always just as expressive and nuanced as their speakers, but the strategies to express these nuances just tend to shift and change over time.

[@4aasdaw]

​@@smuecke That's more or less what I meant; they lose in complexity and sometimes even bits of productivity in certain areas, but retain the ability to express the same meaning as before. They also adapt more to the speakers' needs (though you could argue it is the other way round), where some things simply are removed altogether. This would be features such as middle and mediopassive voice or subjunctive moods. Of course, you can still convey a near identical meaning without them, but the strategy you'd use to get there didn't develop from reductions to the archaic forms.

[@jit_rs]

why did you spend your time on this comment?

[@sadcat520]

Nobody asked

[@aBamieh]

Yay a 2 hours topic discussion! Thanks for uploading, Cheers

[@ishdx9374]

do you have 20 kilograms of

[@joedavid8662]

0:7:43 Allocating Memory 0:25:41 Reading/Writing Files 0:56:21 IPC ( 0:58:52 ) 01:23:56 precise control over memory protection

[@lordmushroom723]

Thanks Joe

[@AdmiralMaur]

".. but here you have do everything yourself. Like an adult" oof- I fealt that

[@tornikebuchukuri4114]

im doing some os stuff rn and was struggling with mmap, this stream really helped

[@menumenu287]

great stuff, high quality, so much knowledge transfer... Thank you; hope you make more videos

[@michaelutech4786]

"single purpose [..] go away" - That's a bit unfair. System calls are not application programming interfaces, they are the interface between the kernel and processes. The guiding principle for this kind of interface is to minimize the number of interactions between kernel and user space, because that interaction is expensive. Another guiding principle for this interface is to be stable, don't change, be as general as possible. You're supposed to use malloc, because that's designed to be an api and that's actually observing the single purpose principle (malloc(size) -> memory).

[@alrshdn]

Good take. I definitely agree that for less system calls, you would need functions that do many things. But I would still argue that single purpose functions are more stable and require less change in the future, thus complying to the standard would be easier for the OSes. So I would agree with the first guiding principle, but it would definitely make the goal of second principle be harder to achieve. This is because it is harder to not have to change a huge function in the future.

[@michaelutech4786]

@@alrshdn I see the point regarding ease of change, but again, this is not something that is very relevant in this environment, because kernel interfaces (syscalls) rarely if ever change. System calls look a lot like any other API, but they are in fact not. They are designed to be used by system libraries (as in libc). That does not mean that you can't use them, it only means that usability in their design is not tuned to the ease of use but instead focusses on performance and stability of kernel and user space (as opposed to library/developer or system/user). A system call change is a catastrophic event. Otherwise there would not be things like "cgroup2".

[@michaelutech4786]

"This is probably why you shouldn't use mmap directly" - No. Mmap actually works on most if not all Unix-like systems and Windows has something close enough. The reason why you should not use mmap for memory allocation is because it the process of allocating memory from the kernel is probably more expensive than the task you are doing with that memory. Malloc works by allocating a big chunk of memory and then providing what you ask for. So f.e. you ask for 80 bytes, malloc allocates 1M and has enough memory to answer the next 50-1000 requests efficiently. Free is not necessarily returning memory to the system, it just marks it as usable for future malloc calls. You want to use mmap in favor of using sbrk, because using sbrk will mess with the stdc-c library (or any runtime you might use). You want to use it in favor of using malloc if you have a long running task in need of memory that you either need to control (f.e. exec code) or in some other fringe use case where malloc doesn't work or is inefficient (which next to never happens, unless you're writing a jit compiler). The problem with sbrk is that it increases the linear address space of the process. That was the only way how a process could get more memory in ancient UNIX. It's similar to what happens if you dynamically increase the memory of a virtual machine. Mmap solved this, by allowing a process to obtain access to memory outside of the linear address space that is assigned to a process at creation time. Sbrk is basically a DOS way of allocating memory ("extended" or "expanded" memory?).

[@ILoveSoImAlive]

memory map is kind of a table, where system memory areas are "maped" into memory address range of a process. process doesnt have direct access to all the ram. only to the parts of the ram, which are mapped into his memory. the memory adress range of a process is virtual so to speak. sitting inside the sanbox. so you can map any range of system memory into process memory. for example file buffer. so you can read the file inside your process. or you can map a memory range to 2 different processes and use this part of system memory as shared memory between 2 processes. or you can map range of system memory, which ist reserved for hardware buffer into process memory, then your process will be able to access this hardware directly. btw, how do you think it was possible to have 2MB of system memory on a 8bit pc? btw2, most of the system functions are writen based on basic input output system functions. its the first thing you learn, if you do something with assembler.

[@NinjaScorpio]

This is a very smart BBoomer i learn a lot in his streams! I love his "Unity" streams! Keep being awesome KKomrade

[@dandymcgee]

Thanks for sharing the VOD!

[@vysakhanh7878]

thank you, learned a lot!

[@Turalcar]

mmap is versatile in the same way read and write are, i.e. perfectly fine

[@DanDart]

11:18 best moment

[@errelin1326]

Probably few universities even teach these. This is AWESOME 🎉❤

[@spandanroy8380]

This is one the best videos I ever saw on KZbin

[@andrewlankford9634]

Software evolves to work with new hardware. Hardware evolves to make better software. Some hardware problems problems provoked software people to find workarounds for slow hardware.

[@dubbeltumme]

big thanks for this. Really appreciate those bitesize Makefile tips, had no idea about that wildcards thing. Also best tabbed layout in i3wm.

[@mariuszdziem8203]

Amazing shait ;) Great lecture professor!

[@coffee-is-power]

tsoding is my favorite teacher!

[@rodsmade]

Hi, I'm writing my own implementation of malloc/realloc/free and this vod really helped getting into it. Thank you so much! Take care

[@ynchen]

I absolutely enjoy this video!!

[@marcosconceicao3223]

Man, what a good content! Thank you!!!

[@alexisfrjp]

mmap is used to map physical memory to virtual memory. more in embedded system.

[@Heater-v1.0.0]

What? Most embedded systems I have worked on don't have virtual memory support in their processors and hence no mmap.

[@johnnyphoney5669]

This video was very useful for me, thank you Mr.!

[@jokerpb4778]

Amazing video thank you so much!

[@bradocksolo]

Really good video! Thanks

[@invokesus]

wow amazing stuff! Learned a lot.

[@ErikNorlander-go1zv]

Great video!

[@gscacco]

Great work !

[@xaro6132]

I was thinking when i saw the video, damn tscoding got rich and fat uwu. But i saw the video is 4 years old. Sorry for your loss

[@rahantr1]

mmap is a posix system call. It's not supposed to be user friendly. It's supposed to be an interface all posix compliant OSes provide as standard. Old and low level code can't be judged by SOLID principles or design patterns. Those are luxuries that system engineers don't have or need.

[@prayer4675]

Actually this syscall is not Linux only and originally it came from the BSD. So actually it will work at any today's BSD as well. At least I see it in the FreeBSD.

[@yuriykhodak6299]

you are good, man

[@qerdcv]

Hello! And welcome to another Tsoding session. How about that?

[@jacobscrackers98]

1:08:50 Why use assert here()? If N == 1 then the for loop will be skipped because i < 1 => 1 < 1 is false, so only the first printf (without the space) will be executed. assert doesn't fix anything here, in fact it breaks something.

[@frederickwinter1823]

Is there a Linux syscall for allocating disk space?

[@uglyoldbob]

open?

[@SekharPariga]

does it possible to do the last example storing stripped hello world code using MALLOC?

[@user-it8yh8tu7d]

I think its cute when I forget that just as I am weak on graphics programming graphics programmers are weak on kernel programming.❤

[@bitshift264]

Great stream! Thank you!

[@nigmar957]

Спасибо за видео)

[@alexandermyasoedov4564]

Keep it up man! Awesome video!

[@jacobscrackers98]

1:53:50 What is up with the top line of the hex editor? That address is way too big to fit in the 51-byte file. What gives?

[@DantevanGemert]

If you're still wondering after 4 months (probably not but who knows who else might be confused): it's the header, showing what the values correspond to (87654321 and 0123456789abcdef). They're not actual values.

[@jony1710]

Pretty much nothing using morecore (a.k.a the sbrk syscall) on linux. I think plan9 still uses that approach. It's all mmap as far as I know these days.

[@davidprock904]

I'm working on my own OS on my own architecture, just realized what you said there is no difference about allocation of memory and a file. That is true with my system, also my system is IMPOSSIBLE to have a buffer overflow, and has a Physical switch, if its off, then no assembly / machine instructions can be changed or added... or removed, they can still run thow...but everything else can be modified... the perfect level of protection! Bring it on all you worms out there lol,, those couldn't run on my system anyway

[@josedejesuslopezdiaz]

Your content is amazing! you have a new follower.

[@AlguienMas555]

Awesome

[@oliverscott6421]

How does your manpage search work?

[@tatterdemalion898]

from byte -> kb -> mb -> gb etc. you multiply with 1024 not 1000 :D

[@luigidabro]

Na, you do multiply with 1000. What you are talking about is kib, mib and gib

[@DanDart]

16:09 is that your professional opinion?

[@luigidabro]

Why is he no longer so rotund?

[@sadhlife]

started exercising :D

[@teenspirit1]

wow fat tsoding existed??

[@MsDuketown]

lol, yeah the world behind the compiler...

[@sharpfang]

I'm baffled at this question. I'm not going to spend 2h to satisfy my curiosity why the author is surprised by mmap, but honestly the standard ways to access file look more suitable for some sort of tape memory where all the lseek calls rewind or fast forward the tape and read acts as play button. Mmap is a much more natural way to access files on a random access medium, you get the pointer to the first byte of the file and you can read, modify, move whatever you want wherever you want in any order just like you were operating on a mmalloc'd chunk of memory. Then you commit the changes and the file is what you wanted it to be. You can even mmap /proc/kmap as root and have unrestricted access to the entire RAM. Mmap is so easy, so convenient, so straightforward and obvious I'm completely stumped why anyone would ask 'why'.

[@2003vito]

has*

[@andrewdunbar828]

Why questions are asked funny?

[@totlajos587]

What make great programmer is it IQ or something else?

[@yunusozd]

practice

[@cobbcoding]

The 3 P's of programming: practice, patience, and programming socks.

[@PLAYGAME-wj9bw]

​@@cobbcodinghahaha

[@sadcat520]

This was a really good and informative videos but all the mouth sounds were so annoying...

[@ac130kz]

!ban UB lmao

[@darkshadow5891]

is this live stream is real. i dont thing so its real elon expretion not match with the voice.