Creating and Linking Shared Libraries on Linux with gcc

[@kjell-e]

A short comment on ld-linux-x86-64.so.2. This is the dynamic linker/loader. It is responsible to find and load the shared libraries needed by a program, prepare the program to run, and then run it. And by the way, it was a very good video.

[@binwei2829]

Great and practical video for embedded software engineers!

[@nitakachhadiya4987]

So easy to understand this complicated topic.... Thank you so much sir.👍

[@cabc74]

Excelent series of videos.

[@embeddedarmdev]

Thank you. I've been busy lately but starting next week I'll have more time and I have a lot more videos planned.

[@joemuthui8682]

Such a good resource. Makes it easy to understand the topic. Better audio quality would be very helpful.

[@adityagujaria2598]

Thanks for the video!!! Would there be any video on PLT and GOT tables for shared library/dynamic linking?

[@khadidjam1148]

Thank you so much

[@martingoodrich9438]

Brilliant. Thanks!

[@embeddedarmdev]

Thank you.

[@t21go60]

Thank you.

[@김익환-g9g]

Thank you so much!!!

[@et6926]

Hi, thanks for the great tutorial series. I have a question about the linker libmath.so which is removed on 24:47. Why does that work later on in the video while earlier you mentioned that the gcc -lmath command needs to search for that linker ? Does that mean that libmath.so is not needed in the compilation step in the first place ?

[@embeddedarmdev]

Hello, thanks for the question. The short answer is that nothing I do in the rest of the video looks for libmath.so. Note that I did not recompile doMath. I only recompiled the shared library libmath and gave it the name libmath.so.1.3.0. When I compiled this shared library, it was not looking for libmath.so anywhere. Also, when I execute doMath again, it is still looking for libmath.so.1 (not libmath.so) which I had updated to point to libmath.so.1.3.0. So when doMath asks for libmath.so.1 it gets directed to the newer updated version 1.3.0. However, you are correct to point it out because if I had tried compiling doMath again linking to libmath, it would report an error because the linker would look for libmath.so and would not find it. So, I should have restored libmath.so at the end of the video and pointed that out to be more clear. I hope this answers your question.

[@Rudransh.Rohit1401]

@embeddedarmdev Hello sir, i have a query... Why we do not see libmath dynamic linked library when we grep for Shared in the executable using readelf.

[@igorpupkinable]

Great videos for a newbie. Thank you so much! Does GCC always link .a (archives) statically and .so (shared libraries) dynamically? I feel I am missing something because doMath compile commands seems to be the same in both videos except the order of -lmath parameter.

[@embeddedarmdev]

Yes, if you compile with the static flag, it will look for the .a file. Otherwise it looks for the .so If you try compiling for static and don't have the .a library, I believe it throws an error.

[@treyquattro]

good info but could you make the font large in future videos, thanks.

[@takenonya1672]

THX

[@martingeorgiev999]

Great video but I have a question, why do you separate the assembling and linking stages?

[@eugenps]

Hi, great video! I was wondering, what do you typically do when the includes change from one version to the next one? Shouldn't the contents of the include directory be versioned, as the library itself?

[@embeddedarmdev]

Eugen, Thanks for your question. The short answer: Use git. The long answer: So, the first thing I would make clear is that the include files are only needed at build/compile time and so would not need to be on the target system and there should be no need to worry about versioning in that regard. As far as maintaining the include files for development purposes, I would say that it depends on how you are using the include files; if you are an end user of the library or if you are the developer/maintainer. Are you going to be the one modifying the code in the library and the corresponding header files, or will you just be using a library that someone else developed? If you are the developer/maintainer of the library then you should definitely use some form of version control software such as git. When the project is ready to be called a release, you would give it a version tag for that release. Using git commands, you can checkout any version of the project using the given tag. You could then create a release distribution, which could be just the source code or could just be the compiled binaries and header files. If you did not develop the library and you are just using functionality provided by the library, then you are the 'end user'. As an end user, you basically have two options. One is to acquire a release distribution and the other is to download the project git repository if it is available. If you choose to acquire a release distribution, it normally comes as a tarball (tar.gz file). It may have pre-compiled binaries in it or it may only be source code and you have to build it yourself. If you are doing embedded development, specifically cross-compiling, you'll want the source code so you can build it yourself. In either case, the include files will be specific to that version of the library. Once you have the distribution, you extract the contents to a directory in your development environment. The name of the directory would normally indicate the version being used. Then you just specify the directory where the include files are using the -I option at build time. If for some reason your project required the use of two different versions of the same library, then you may need to have two separate directories. The other option is to use git. You can acquire the git repository and place it in your development environment. If the project uses version tags, then you can simply checkout a specific version using the version tag and then build your project with that specific version of the library. One downside to doing this is git repositories can be very large and take up more disk space than they are worth. In general, you only need to acquire the git repo if you are going to be doing development for that library which is normally not the case as an end user. One final thought; I prefer git over any other version control software I have used. Also, here is a link to a rather popular git branching model that I personally think is good. nvie.com/posts/a-successful-git-branching-model/ Some people say it is overly complicated. For small projects, that may be true, but for really large projects with lots of developers I think it works well. I hope this answers your question, and feel free to ask again if you have more questions.

[@multivalfran]

Great video. I was wondering if I could do all of this in a makefile, which i accomplished until the part where you need to change te enviroment variables and you cant use cd in that makefile. How can i solve this?

[@embeddedarmdev]

Are you talking about the LD_LIBRARY_PATH?

[@multivalfran]

@@embeddedarmdev exactly! I would need to CD into lib, change the environment variable, export and then come back from last cd but that seem to be not possible

[@embeddedarmdev]

If you are talking about modifying the LD_LIBRARY_PATH, then you probably don't really want to do it that way. I should have been more clear about it in the video, but I was just doing it to demonstrate that the loader needs to know where the shared library is and will complain if it can't find it. So, I appended the directory that contained the newly-built library to the LD_LIBRARY_PATH. But, if you really wanted to do this the right way, you would do it as part of the 'install' target in a make file and you would just copy the shared library to one of the directories that are already in the LD_LIBRARY_PATH. I hope that answers your question.

[@multivalfran]

@@embeddedarmdev yes it does! Thank you very much for answering.

[@bubanalefamily874]

I am struggling with shared library linking for my ARM board. I have eclipse on Ubuntu configured as cros-gcc connect with beaglebone over ssh. But shared library is unable to load on beaglrbone when deployed. Are there specific env changes needed on Ubuntu host to link arm .so before deploying on ARM board ?

[@embeddedarmdev]

When you say 'shared library is unable to load', what exactly are you doing and what error does it give you? What directory are you putting the shared library in? I'm not sure what your exact setup is, but I'll give you some ideas of things to look for. More than likely, the loader is unable to find the .so when the executable program asks for it. Make sure that your shared library is in a location where the loader will look for it. I'm not sure what the default directories are on Beaglebone, but usually there is at least /lib and /usr/lib. Also look at the directory /etc/ld.so.conf.d/. There will be a number of .conf files that contains list of other directories that the loader will search. Your shared library needs to be in one of these directories. Note: If the directory /etc/ld.so.conf.d/ doesn't exist, there should at least be a file /etc/ld.so.conf there that contains a list of additional directories. If you want to have it in a directory other than those I described above. You should be able to create a new .conf file in /etc/ld.so.conf.d/ with the new directory listed in the file (or add the new directory to the file /etc/ld.so.conf if that directory doesn't exist). Then you run the command ldconfig and it should update the ld cache. The other thing I would check is the soname that the executable program is looking for. For the program that requires the shared library, use readelf to determine the soname that it is looking for. readelf -d program_name | grep Shared You should see something like: Shared library: [libmath.so.1] If your shared library is not named exactly this, then you will need to have a symbolic link that points one to the other. For example, if your program is looking for libmath.so.1 but your library is called libmath.so.1.2.3, then you need to have a symbolic link pointing libmath.so.1 to libmath.so.1.2.3. I hope this helps.

[@bubanalefamily874]

@@embeddedarmdev thanks a lot , i will add my comments later after more understanding of ur reply

[@tanja84dk1]

Just a question is there a way to automatic tell the application to look for the shared library in a predefined path ( like a predefined sub folder or root ) lets say in the situation where the project is distributed in a tar ball and you don't have control of the end users path variables. Like I'm always worried that I'm going to conflict with other libraries installed on end users system EDIT: I think I found my solution there works for me except one small flaw that I have to fixed. I found out the rpath in linking do allow for relative path so I got is to work with making a folder in the target/build folder called Libraries and then gave the path ./Libraries to the rpath when linked to the shared library in the compiling Thank you very much for the great video

[@embeddedarmdev]

I'm sorry for the late reply. Yes, I think rpath is what you want. You need to invoke it with gcc using the -Wl option: gcc -Wl,-rpath="path/to/lib" You can also try setting the environment variable LD_RUN_PATH with the path you want the binary to search. linux.die.net/man/1/ld

[@anuprakashsubramaniam632]

​@@embeddedarmdev Then What is the use of -L , we are also giving the path here to search library right?

[@embeddedarmdev]

​@@anuprakashsubramaniam632 Good question. One is a compile-time path and the other is a runtime path. First, let's make a distinction between these two gcc options -l (lowercase L) tells us the name of the library to link to. -L provides a pathname to search for these libraries. The -L is an option passed to gcc that tells the compiler where to look for shared libraries at compile time. These libraries would need to exist on the machine where the compiling is happening. This is for compile-time linking. The rpath is different. The linker compiles the rpath into the binary and when the binary gets loaded on the target machine the runtime linker will also look in the rpath for any libraries it might need at runtime. You would use this if the library of interest is not in a standard location on the target machine. Note that the rpath is a linker option. We pass options to the linker by using the syntax "-Wl, [options]" with the gcc command. see man gcc for -Wl man7.org/linux/man-pages/man1/gcc.1.html See the man ld for rpath linux.die.net/man/1/ld

[@SteaM992]

What should I do if I try to link one shared library to another? The 2 libs(inv.so and swl.so)are in separate directories and when i try to build the inv.so that depends on the swl.so i get that the compiler can t find the swl.so needed by inv.so...i specified with - L and rpath where the shared object is but no succes

[@embeddedarmdev]

I can try to help, but I'll need more info. Can you give me the following info: - Exact command used to compile swl.so - The actual name of the compiled swl shared library. Is it swl.so or swl.so.1.2.3 ? - The exact command you are using to compile inv.so - The error it gives you when you try to compile inv.so Double-check the path you give after -L. It may be as simple as a missing /. If you are using a relative path, try using an absolute path to see if it fixes it. When you compile inv.so and use the -l option, (as in -lswl) it will look for "libswl.so" in the search paths. So, if your library has any other name (like libswl.so.1.2.3), then it won't find it. You'll need to add a symbolic link that points libswl.so to the actual name of it. Other than that, I'm not sure what else you could look for.

[@SteaM992]

@@embeddedarmdev hello, thank you for the fast response. The warning it gives me is libSwl_Log.so needed by swl_inv.so not found... I also tried to give the absolute path to - L option but no luck. I do not use versions in my shared objects, so the soname was the same as the library. The command i tryed to compile the inventory.so was: cc swl_inv.o $(CFLAGS) -I./include - I. /.. /swl_log - L. /.. /swl_log - Bsymbolic - shared - rdynamic - fPIC - Wl, soname, $@,-rpath,./../swl_log - lSwl_Log.so - o $@ For $@ i specify the name of the library, meening libswl_inv.so

[@SteaM992]

To compile Swl_log.so i just use - gcc shared - o libSwl_Log.so swl_log.o, and of course the .o was compiled as - fPIC

[@embeddedarmdev]

At the very end, instead of: - lSwl_Log.so Try: - lSwl_Log If you do "- lSwl_Log.so", I think it will look for "libSwl_Log.so.so" With "- lSwl_Log" it will look for "libSwl_Log.so" Also, it may be case-sensitive. I see in some places you have swl_log and in others Swl_Log. If you don't have a strong reason for mixing the cases, you may consider keeping everything lower-case, just to simplify things. Also, I'd like to point out that the rpath is used at runtime. This path is compiled into the binary. It is an additional path that the loader will search for shared libraries when loading the binary. You use this if you think the shared library will be in a non-standard directory but you don't want to have to modify the LD search paths on the target system. Let me know if this works or not.

[@SteaM992]

@@embeddedarmdev hello, it was a typo from me, my bad. I use rpath because the library is not in a standard path and i can not modify de ld.conf file. The question is should it work linking the 2 libraries from different folders the way i tryed? I checked numerous times for typo's

[@giangbuitam4868]

How about linking shared lib in window ?

[@ssskk12]

How to user's shared lib call shared lib Ex: lib1.so==>func1() Call lib2.so==>func2()

[@embeddedarmdev]

In the source code for the user's shared library (lib1.so), you would need to have include statements to include the header files for the second library lib2.so. Then whenever you reference func2() (from lib2.so) it is able to resolve that function at compile time. These header files essentially define an API for the library you are calling lib2.so. You just have to make sure you link in the shared library lib2.so when you compile the user's library. The compiler/linker need to know the location of the header files and the .so file.

[@ssskk12]

@@embeddedarmdev How to user's shared lib call shared lib Ex: lib1.so==>func1() Call lib2.so==>func2()， lib2.so.func3() call lib1.so.func1(); Since gcc build lib1.so will reference lib2.so, but lib2.so dot not exist. How to ....