Something I've never quite understood is why can't you statically link against an so file? What specific information was lost during the linking phase to create the shared object that presents that machine code from being placed into a PIE executable?
At a fundamental level I don't understand why we have two separate file types for static and dynamic libraries. It seems primarily for historical reasons?
The author proposes introducing a new kind of file that solves some of the problems with .a filed - but we already have a perfectly good compiled library format for shared libraries! So why can't we make gcc sufficiently smart to allow linking against those statically and drop this distinction?
Oh, it's even better on the Windows side of things, at least how the MSVC toolchain does it. You can only link a statically-linked .lib library, period. So if you want to statically link against a dynamic library (what a phrase!), you need to have a special version of that .lib library that essentially is just a collection of thunks (in MSVC-specific format) that basically say "oh, you actually want to add symbol Bar@8 from LIBFOO.DLL to your import section" [0]. So yeah, you'd see three binaries distributed as a result of building a library: libfoo_static.lib (statically-linked library), libfoo.dll (dynamic library), libfoo.lib (the shim library to link against when you want to link to libfoo.dll).
Amusingly, other (even MSVC-compatible) toolchains never had such problem; e.g. Delphi could straight up link against a DLL you tell it to use.
"So if you want to statically link against a dynamic library (what a phrase!)"
Yes but like an artificially created remarkableness. "dynamic library" should just be "library", and then it's not remarkable at all.
It does seem obvious, and your Delphi example and the other comment wcc example shows, that if an executable can be assembled from .so at run time, then the same thing can also be done at any other time. All the pieces are just sitting there wondering why we're not using them.
Because with the current compilation model shared libraries (.so/.dll) are the output of the linker, but static libraries are input for the linker. It is historical baggage, but as it currently stands they're fairly different beasts.
you could say historical reasons, in that dynamic libraries are generated using relocatable position independent code (-pic), which incurs some performance penalty vs code where the linker fills in all the relocations. my guess is thats somewhere around 10%? historical in the sense that that used to be enough to matter? idk that it still is
personally I think leaving the binding of libraries to runtime opens up alot of room for problems, and maybe the savings of having a single copy of a library loaded into memory vs N specialized copies isn't important anymore either.
I think the question isn’t why statically link but rather why bother with .a files and instead use the shared libraries all the time (even if only to build a statically linked executable).
Yeah, I misunderstood the question. Although if you could statically link .so/.dlls and have it work reliably, it would still be a great convenience, as some libraries are really hard to build statically without rewriting half their build system.
Then just use an rpath with `${ORIGIN}` and ship your .so files along with your binary.
The only time where you shouldn't do this is if your executable requires setuid or similar permission bits, but those generally can only reasonably be shipped through distro repos.
Sorry what do you mean by "symbol interpolation" and "interposition" in this context?
Natively I would assume you can just take the sections out of the shared object and slap them into the executable. They're both position independent so what's the issue?
If PIE allows greater assumptions to be made by the compiler/linker than PIC that sounds great for performance, but doesn't imply PIC code won't work in a PIE context.
PIC code would work where PIE does, but likely perform worse. In shared libraries, calls to any non-static function can't be assumed to be the same function at runtime, since another library linked or using LD_PRELOAD may also define the symbol. Thus all calls to non-static functions must go through the shared library lookup machinery. This prevents inlining opportunites as well. Functions in an executable can't be overwridden in this manner, and override the symbols in shared libraries. Thus PIE code can have cheaper function calls and freely inline functions.
Its not that you couldn't use the PIC code, but it would be better to just recompile with PIE.
