Originally published 23 April 2012, updated 10 September 2019 (add link to troubleshooting tool), updated 18 October 2019 (moved to Jekyll), updated 3 April 2023 with links to other tools
Development of large systems using many shared (dynamically) loaded libraries can sometimes lead to some frustrating bugs that are difficult to diagnose. These bugs often arise because there a few different versions of libraries on the system and the “wrong” version gets loaded instead of the one the developer used wanted.
Note You may also find of use my online tool for diagnosing linker problems on Linux implemented as an expert system. It is being expanded at the time of writing this but probably useful already.
I used to often debug these problems reasonably efficiently using the
strace command (man strace)
and checking which libraries are getting accessed. However, there is a
much more efficient but perhaps not very well known way of debugging
shared library loading problems: the LD_DEBUG environment
variable.
If the LD_DEBUG variable is set then the Linux dynamic linker will
dump debug information which can be used to resolve most loading
problems very quickly. To see the available options just run any
program with the variable set to help, i.e.:
LD_DEBUG=help cat
Valid options for the LD_DEBUG environment variable are:
libs display library search paths
reloc display relocation processing
files display progress for input file
symbols display symbol table processing
bindings display information about symbol binding
versions display version dependencies
all all previous options combined
statistics display relocation statistics
unused determined unused DSOs
help display this help message and exit
To direct the debugging output into a file instead of standard output
a filename can be specified using the LD_DEBUG_OUTPUT environment
variable.
Note There are many other useful tools to deal with linking issues:
- The
straceprogram provides insight into all system calls, including searching, opening dynamic libraries- The
lddprogram resolves dynamic library dependencies- The
objdump -x YOURFILE | grep NEEDEDcommand lists the records in the program or library showing which other libraries are needed- The
patchelfprogram makes it easy to change therpathof an ELF executable, changing the built-in search order- The
LD_PRELOADenvironment allows easy substitution of dynamic librariesNeed further advice? With over 20 years of experience we are uniquely placed to help – contact us at webs@bnikolic.co.uk
Note This variable is available Linux only. On MS Windows similar information can be obtained by enabling “Show Loader Snaps” (https://abitofscotland.wordpress.com/2019/12/20/dll-load-failure/) using the
gflags.exeprogram to enable this and then usingwindbgto view the log. For example:
- Install Windows SDK for
gflagsandwindbgwinget install Microsoft.WindowsSDK.10.0.19041(or download from Microsoft website )- Enable ShowLoaderSnaps for an executable. Example: to enable for
notepad.exefor example do in Powershell&"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\gflags.exe" /i notepad.exe +sls- Execute
notepad.exeunder WinDbg app. Detailed DLL loading log will be displayed
And here is an example run on my laptop:
LD_DEBUG=all cat
28504:
28504: file=libc.so.6 [0]; needed by cat [0]
28504: find library=libc.so.6 [0]; searching
28504: search path=/home/bnikolic/s/lib/tls/x86_64:/home/bnikolic/s/lib/tls:/home/bnikolic/s/lib/x86_64:/home/bnikolic/s/lib:tls/x86_64:tls:x86_64::/home/bnikolic/d/p/R-2.14.1bin/lib/R/lib/tls/x86_64:/home/bnikolic/d/p/R-2.14.1bin/lib/R/lib/tls:/home/bnikolic/d/p/R-2.14.1bin/lib/R/lib/x86_64:/home/bnikolic/d/p/R-2.14.1bin/lib/R/lib (LD_LIBRARY_PATH)
28504: trying file=/home/bnikolic/s/lib/tls/x86_64/libc.so.6
28504: trying file=/home/bnikolic/s/lib/tls/libc.so.6
28504: trying file=/home/bnikolic/s/lib/x86_64/libc.so.6
28504: trying file=/home/bnikolic/s/lib/libc.so.6
28504: trying file=tls/x86_64/libc.so.6
28504: trying file=tls/libc.so.6
28504: trying file=x86_64/libc.so.6
28504: trying file=libc.so.6
28504: trying file=/home/bnikolic/d/p/R-2.14.1bin/lib/R/lib/tls/x86_64/libc.so.6
28504: trying file=/home/bnikolic/d/p/R-2.14.1bin/lib/R/lib/tls/libc.so.6
28504: trying file=/home/bnikolic/d/p/R-2.14.1bin/lib/R/lib/x86_64/libc.so.6
28504: trying file=/home/bnikolic/d/p/R-2.14.1bin/lib/R/lib/libc.so.6
28504: search cache=/etc/ld.so.cache
28504: trying file=/lib/x86_64-linux-gnu/libc.so.6
28504:
28504: file=libc.so.6 [0]; generating link map
28504: dynamic: 0x00007fc38e525b40 base: 0x00007fc38e18c000 size: 0x00000000003a0368
28504: entry: 0x00007fc38e1ad420 phdr: 0x00007fc38e18c040 phnum: 10
28504:
28504: checking for version `GLIBC_2.4' in file /lib/x86_64-linux-gnu/libc.so.6 [0] required by file cat [0]
28504: checking for version `GLIBC_2.3' in file /lib/x86_64-linux-gnu/libc.so.6 [0] required by file cat [0]
28504: checking for version `GLIBC_2.3.4' in file /lib/x86_64-linux-gnu/libc.so.6 [0] required by file cat [0]
28504: checking for version `GLIBC_2.2.5' in file /lib/x86_64-linux-gnu/libc.so.6 [0] required by file cat [0]
28504: checking for version `GLIBC_2.3' in file /lib64/ld-linux-x86-64.so.2 [0] required by file /lib/x86_64-linux-gnu/libc.so.6 [0]
28504: checking for version `GLIBC_PRIVATE' in file /lib64/ld-linux-x86-64.so.2 [0] required by file /lib/x86_64-linux-gnu/libc.so.6 [0]
28504:
28504: relocation processing: /lib/x86_64-linux-gnu/libc.so.6 (lazy)
28504: symbol=_res; lookup in file=cat [0]
28504: symbol=_res; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `_res' [GLIBC_2.2.5]
28504: symbol=_IO_file_close; lookup in file=cat [0]
28504: symbol=_IO_file_close; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `_IO_file_close' [GLIBC_2.2.5]
28504: symbol=stderr; lookup in file=cat [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to cat [0]: normal symbol `stderr' [GLIBC_2.2.5]
28504: symbol=error_one_per_line; lookup in file=cat [0]
28504: symbol=error_one_per_line; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `error_one_per_line' [GLIBC_2.2.5]
28504: symbol=__malloc_initialize_hook; lookup in file=cat [0]
28504: symbol=__malloc_initialize_hook; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__malloc_initialize_hook' [GLIBC_2.2.5]
28504: symbol=__morecore; lookup in file=cat [0]
28504: symbol=__morecore; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__morecore' [GLIBC_2.2.5]
28504: symbol=__key_encryptsession_pk_LOCAL; lookup in file=cat [0]
28504: symbol=__key_encryptsession_pk_LOCAL; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__key_encryptsession_pk_LOCAL' [GLIBC_2.2.5]
28504: symbol=__progname_full; lookup in file=cat [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to cat [0]: normal symbol `__progname_full' [GLIBC_2.2.5]
28504: symbol=__ctype32_tolower; lookup in file=cat [0]
28504: symbol=__ctype32_tolower; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__ctype32_tolower' [GLIBC_2.2.5]
28504: symbol=__key_gendes_LOCAL; lookup in file=cat [0]
28504: symbol=__key_gendes_LOCAL; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__key_gendes_LOCAL' [GLIBC_2.2.5]
28504: symbol=_environ; lookup in file=cat [0]
28504: symbol=_environ; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `_environ' [GLIBC_2.2.5]
28504: symbol=_rtld_global; lookup in file=cat [0]
28504: symbol=_rtld_global; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: symbol=_rtld_global; lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib64/ld-linux-x86-64.so.2 [0]: normal symbol `_rtld_global' [GLIBC_PRIVATE]
28504: symbol=__progname; lookup in file=cat [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to cat [0]: normal symbol `__progname' [GLIBC_2.2.5]
28504: symbol=argp_err_exit_status; lookup in file=cat [0]
28504: symbol=argp_err_exit_status; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `argp_err_exit_status' [GLIBC_2.2.5]
28504: symbol=mallwatch; lookup in file=cat [0]
28504: symbol=mallwatch; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `mallwatch' [GLIBC_2.2.5]
28504: symbol=__rcmd_errstr; lookup in file=cat [0]
28504: symbol=__rcmd_errstr; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__rcmd_errstr' [GLIBC_2.2.5]
28504: symbol=svcauthdes_stats; lookup in file=cat [0]
28504: symbol=svcauthdes_stats; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `svcauthdes_stats' [GLIBC_2.2.5]
28504: symbol=__libc_enable_secure; lookup in file=cat [0]
28504: symbol=__libc_enable_secure; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: symbol=__libc_enable_secure; lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib64/ld-linux-x86-64.so.2 [0]: normal symbol `__libc_enable_secure' [GLIBC_PRIVATE]
28504: symbol=_res_hconf; lookup in file=cat [0]
28504: symbol=_res_hconf; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `_res_hconf' [GLIBC_2.2.5]
28504: symbol=malloc; lookup in file=cat [0]
28504: symbol=malloc; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `malloc' [GLIBC_2.2.5]
28504: symbol=getdate_err; lookup in file=cat [0]
28504: symbol=getdate_err; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `getdate_err' [GLIBC_2.2.5]
28504: symbol=__tzname; lookup in file=cat [0]
28504: symbol=__tzname; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__tzname' [GLIBC_2.2.5]
28504: symbol=__timezone; lookup in file=cat [0]
28504: symbol=__timezone; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__timezone' [GLIBC_2.2.5]
28504: symbol=_rtld_global_ro; lookup in file=cat [0]
28504: symbol=_rtld_global_ro; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: symbol=_rtld_global_ro; lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib64/ld-linux-x86-64.so.2 [0]: normal symbol `_rtld_global_ro' [GLIBC_PRIVATE]
28504: symbol=optarg; lookup in file=cat [0]
28504: symbol=optarg; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `optarg' [GLIBC_2.2.5]
28504: symbol=__ctype_tolower; lookup in file=cat [0]
28504: symbol=__ctype_tolower; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__ctype_tolower' [GLIBC_2.2.5]
28504: symbol=__ctype_toupper; lookup in file=cat [0]
28504: symbol=__ctype_toupper; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__ctype_toupper' [GLIBC_2.2.5]
28504: symbol=svc_max_pollfd; lookup in file=cat [0]
28504: symbol=svc_max_pollfd; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `svc_max_pollfd' [GLIBC_2.2.5]
28504: symbol=__ctype_b; lookup in file=cat [0]
28504: symbol=__ctype_b; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__ctype_b' [GLIBC_2.2.5]
28504: symbol=argp_program_version_hook; lookup in file=cat [0]
28504: symbol=argp_program_version_hook; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `argp_program_version_hook' [GLIBC_2.2.5]
28504: symbol=__after_morecore_hook; lookup in file=cat [0]
28504: symbol=__after_morecore_hook; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__after_morecore_hook' [GLIBC_2.2.5]
28504: symbol=__environ; lookup in file=cat [0]
28504: symbol=__environ; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__environ' [GLIBC_2.2.5]
28504: symbol=__ctype32_b; lookup in file=cat [0]
28504: symbol=__ctype32_b; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__ctype32_b' [GLIBC_2.2.5]
28504: symbol=__curbrk; lookup in file=cat [0]
28504: symbol=__curbrk; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__curbrk' [GLIBC_2.2.5]
28504: symbol=argp_program_version; lookup in file=cat [0]
28504: symbol=argp_program_version; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `argp_program_version' [GLIBC_2.2.5]
28504: symbol=__daylight; lookup in file=cat [0]
28504: symbol=__daylight; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__daylight' [GLIBC_2.2.5]
28504: symbol=__memalign_hook; lookup in file=cat [0]
28504: symbol=__memalign_hook; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__memalign_hook' [GLIBC_2.2.5]
28504: symbol=__malloc_hook; lookup in file=cat [0]
28504: symbol=__malloc_hook; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__malloc_hook' [GLIBC_2.2.5]
28504: symbol=__free_hook; lookup in file=cat [0]
28504: symbol=__free_hook; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__free_hook' [GLIBC_2.2.5]
28504: symbol=svc_pollfd; lookup in file=cat [0]
28504: symbol=svc_pollfd; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `svc_pollfd' [GLIBC_2.2.5]
28504: symbol=_nl_domain_bindings; lookup in file=cat [0]
28504: symbol=_nl_domain_bindings; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `_nl_domain_bindings' [GLIBC_2.2.5]
28504: symbol=_nl_msg_cat_cntr; lookup in file=cat [0]
28504: symbol=_nl_msg_cat_cntr; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `_nl_msg_cat_cntr' [GLIBC_2.2.5]
28504: symbol=argp_program_bug_address; lookup in file=cat [0]
28504: symbol=argp_program_bug_address; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `argp_program_bug_address' [GLIBC_2.2.5]
28504: symbol=__key_decryptsession_pk_LOCAL; lookup in file=cat [0]
28504: symbol=__key_decryptsession_pk_LOCAL; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__key_decryptsession_pk_LOCAL' [GLIBC_2.2.5]
28504: symbol=h_errlist; lookup in file=cat [0]
28504: symbol=h_errlist; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `h_errlist' [GLIBC_2.2.5]
28504: symbol=program_invocation_short_name; lookup in file=cat [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to cat [0]: normal symbol `program_invocation_short_name' [GLIBC_2.2.5]
28504: symbol=optind; lookup in file=cat [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to cat [0]: normal symbol `optind' [GLIBC_2.2.5]
28504: symbol=_dl_starting_up; lookup in file=cat [0]
28504: symbol=_dl_starting_up; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: symbol=_dl_starting_up; lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
28504: symbol=stdout; lookup in file=cat [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to cat [0]: normal symbol `stdout' [GLIBC_2.2.5]
28504: symbol=obstack_alloc_failed_handler; lookup in file=cat [0]
28504: symbol=obstack_alloc_failed_handler; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `obstack_alloc_failed_handler' [GLIBC_2.2.5]
28504: symbol=error_print_progname; lookup in file=cat [0]
28504: symbol=error_print_progname; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `error_print_progname' [GLIBC_2.2.5]
28504: symbol=optopt; lookup in file=cat [0]
28504: symbol=optopt; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `optopt' [GLIBC_2.2.5]
28504: symbol=_IO_funlockfile; lookup in file=cat [0]
28504: symbol=_IO_funlockfile; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `_IO_funlockfile' [GLIBC_2.2.5]
28504: symbol=obstack_exit_failure; lookup in file=cat [0]
28504: symbol=obstack_exit_failure; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `obstack_exit_failure' [GLIBC_2.2.5]
28504: symbol=error_message_count; lookup in file=cat [0]
28504: symbol=error_message_count; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `error_message_count' [GLIBC_2.2.5]
28504: symbol=svc_fdset; lookup in file=cat [0]
28504: symbol=svc_fdset; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `svc_fdset' [GLIBC_2.2.5]
28504: symbol=program_invocation_name; lookup in file=cat [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to cat [0]: normal symbol `program_invocation_name' [GLIBC_2.2.5]
28504: symbol=loc1; lookup in file=cat [0]
28504: symbol=loc1; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `loc1' [GLIBC_2.2.5]
28504: symbol=free; lookup in file=cat [0]
28504: symbol=free; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `free' [GLIBC_2.2.5]
28504: symbol=loc2; lookup in file=cat [0]
28504: symbol=loc2; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `loc2' [GLIBC_2.2.5]
28504: symbol=__fpu_control; lookup in file=cat [0]
28504: symbol=__fpu_control; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__fpu_control' [GLIBC_2.2.5]
28504: symbol=re_syntax_options; lookup in file=cat [0]
28504: symbol=re_syntax_options; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `re_syntax_options' [GLIBC_2.2.5]
28504: symbol=stdin; lookup in file=cat [0]
28504: symbol=stdin; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `stdin' [GLIBC_2.2.5]
28504: symbol=__check_rhosts_file; lookup in file=cat [0]
28504: symbol=__check_rhosts_file; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__check_rhosts_file' [GLIBC_2.2.5]
28504: symbol=opterr; lookup in file=cat [0]
28504: symbol=opterr; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `opterr' [GLIBC_2.2.5]
28504: symbol=__ctype32_toupper; lookup in file=cat [0]
28504: symbol=__ctype32_toupper; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__ctype32_toupper' [GLIBC_2.2.5]
28504: symbol=__realloc_hook; lookup in file=cat [0]
28504: symbol=__realloc_hook; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__realloc_hook' [GLIBC_2.2.5]
28504: symbol=_dl_argv; lookup in file=cat [0]
28504: symbol=_dl_argv; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: symbol=_dl_argv; lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib64/ld-linux-x86-64.so.2 [0]: normal symbol `_dl_argv' [GLIBC_PRIVATE]
28504: symbol=rpc_createerr; lookup in file=cat [0]
28504: symbol=rpc_createerr; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `rpc_createerr' [GLIBC_2.2.5]
28504: symbol=_IO_2_1_stderr_; lookup in file=cat [0]
28504: symbol=_IO_2_1_stderr_; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `_IO_2_1_stderr_' [GLIBC_2.2.5]
28504: symbol=_IO_2_1_stdout_; lookup in file=cat [0]
28504: symbol=_IO_2_1_stdout_; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `_IO_2_1_stdout_' [GLIBC_2.2.5]
28504: symbol=_IO_2_1_stdin_; lookup in file=cat [0]
28504: symbol=_IO_2_1_stdin_; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `_IO_2_1_stdin_' [GLIBC_2.2.5]
28504:
28504: relocation processing: cat (lazy)
28504: symbol=__gmon_start__; lookup in file=cat [0]
28504: symbol=__gmon_start__; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: symbol=__gmon_start__; lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
28504: symbol=__progname; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__progname' [GLIBC_2.2.5]
28504: symbol=__progname_full; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__progname_full' [GLIBC_2.2.5]
28504: symbol=optind; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `optind' [GLIBC_2.2.5]
28504: symbol=stderr; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `stderr' [GLIBC_2.2.5]
28504: symbol=stdout; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `stdout' [GLIBC_2.2.5]
28504:
28504: relocation processing: /lib64/ld-linux-x86-64.so.2
28504: symbol=_r_debug; lookup in file=cat [0]
28504: symbol=_r_debug; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: symbol=_r_debug; lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
28504: binding file /lib64/ld-linux-x86-64.so.2 [0] to /lib64/ld-linux-x86-64.so.2 [0]: normal symbol `_r_debug' [GLIBC_2.2.5]
28504: symbol=free; lookup in file=cat [0]
28504: symbol=free; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib64/ld-linux-x86-64.so.2 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `free' [GLIBC_2.2.5]
28504: symbol=__libc_memalign; lookup in file=cat [0]
28504: symbol=__libc_memalign; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib64/ld-linux-x86-64.so.2 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__libc_memalign' [GLIBC_2.2.5]
28504: symbol=malloc; lookup in file=cat [0]
28504: symbol=malloc; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib64/ld-linux-x86-64.so.2 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `malloc' [GLIBC_2.2.5]
28504: symbol=__tls_get_addr; lookup in file=cat [0]
28504: symbol=__tls_get_addr; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: symbol=__tls_get_addr; lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
28504: binding file /lib64/ld-linux-x86-64.so.2 [0] to /lib64/ld-linux-x86-64.so.2 [0]: normal symbol `__tls_get_addr' [GLIBC_2.3]
28504: symbol=calloc; lookup in file=cat [0]
28504: symbol=calloc; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib64/ld-linux-x86-64.so.2 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `calloc' [GLIBC_2.2.5]
28504: symbol=realloc; lookup in file=cat [0]
28504: symbol=realloc; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib64/ld-linux-x86-64.so.2 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `realloc' [GLIBC_2.2.5]
28504: symbol=free; lookup in file=cat [0]
28504: symbol=free; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib64/ld-linux-x86-64.so.2 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `free' [GLIBC_2.2.5]
28504:
28504: calling init: /lib64/ld-linux-x86-64.so.2
28504:
28504:
28504: calling init: /lib/x86_64-linux-gnu/libc.so.6
28504:
28504: symbol=gettimeofday; lookup in file=cat [0]
28504: binding file cat [0] to cat [0]: normal symbol `gettimeofday' [LINUX_2.6]
28504: symbol=clock_gettime; lookup in file=cat [0]
28504: binding file cat [0] to cat [0]: normal symbol `clock_gettime' [LINUX_2.6]
28504: symbol=__libc_start_main; lookup in file=cat [0]
28504: symbol=__libc_start_main; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__libc_start_main' [GLIBC_2.2.5]
28504:
28504: initialize program: cat
28504:
28504:
28504: transferring control: cat
28504:
28504: symbol=getpagesize; lookup in file=cat [0]
28504: symbol=getpagesize; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `getpagesize' [GLIBC_2.2.5]
28504: symbol=strrchr; lookup in file=cat [0]
28504: symbol=strrchr; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `strrchr' [GLIBC_2.2.5]
28504: symbol=setlocale; lookup in file=cat [0]
28504: symbol=setlocale; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `setlocale' [GLIBC_2.2.5]
28504: symbol=malloc; lookup in file=cat [0]
28504: symbol=malloc; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `malloc' [GLIBC_2.2.5]
28504: symbol=free; lookup in file=cat [0]
28504: symbol=free; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file /lib/x86_64-linux-gnu/libc.so.6 [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `free' [GLIBC_2.2.5]
28504: symbol=bindtextdomain; lookup in file=cat [0]
28504: symbol=bindtextdomain; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `bindtextdomain' [GLIBC_2.2.5]
28504: symbol=textdomain; lookup in file=cat [0]
28504: symbol=textdomain; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `textdomain' [GLIBC_2.2.5]
28504: symbol=__cxa_atexit; lookup in file=cat [0]
28504: symbol=__cxa_atexit; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__cxa_atexit' [GLIBC_2.2.5]
28504: symbol=getopt_long; lookup in file=cat [0]
28504: symbol=getopt_long; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `getopt_long' [GLIBC_2.2.5]
28504: symbol=__fxstat; lookup in file=cat [0]
28504: symbol=__fxstat; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `__fxstat' [GLIBC_2.2.5]
28504: symbol=malloc; lookup in file=cat [0]
28504: symbol=malloc; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `malloc' [GLIBC_2.2.5]
28504: symbol=read; lookup in file=cat [0]
28504: symbol=read; lookup in file=/lib/x86_64-linux-gnu/libc.so.6 [0]
28504: binding file cat [0] to /lib/x86_64-linux-gnu/libc.so.6 [0]: normal symbol `read' [GLIBC_2.2.5]
The output of the command is also very useful for tracking performance problems for excessive dynamic loading etc. So it is a very simple but extremely useful tool for debugging linux ld problems!