Static linking on linux

Apr 17, 2014 at 7:58 AM
Hi,

I made this changes to try to build a static version of casablanca :
diff --git a/Release/CMakeLists.txt b/Release/CMakeLists.txt
index 779d470..62e1491 100644
--- a/Release/CMakeLists.txt
+++ b/Release/CMakeLists.txt
@@ -69,6 +69,14 @@ elseif("${CMAKE_CXX_COMPILER_ID}" MATCHES "GNU")
 
   set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -fno-strict-aliasing")
   set(STRICT_CXX_FLAGS ${WARNINGS} "-Werror -pedantic")
+
+  if(NOT BUILD_SHARED_LIBS)
+    add_definitions(
+      -D_NO_ASYNCRTIMP
+      -D_NO_PPLXIMP
+    )
+    message("-- Static compilation.")
+  endif()
 else()
   message("-- Unknown compiler, success is doubtful.")
 endif()
But when I try to compile my service against casablanca I have this link error :
/usr/local/lib/casablanca/libcasablanca.a(asyncrt_utils.cpp.o): In function `utility::conversions::usascii_to_utf16(std::string const&)':
asyncrt_utils.cpp:(.text+0xd70): undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> > boost::locale::conv::to_utf<char>(char const*, char const*, std::string const&, boost::locale::conv::method_type)'

/usr/local/lib/casablanca/libcasablanca.a(asyncrt_utils.cpp.o): In function `utility::conversions::latin1_to_utf16(std::string const&)':
asyncrt_utils.cpp:(.text+0xe60): undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> > boost::locale::conv::to_utf<char>(char const*, char const*, std::string const&, boost::locale::conv::method_type)'

/usr/local/lib/casablanca/libcasablanca.a(asyncrt_utils.cpp.o): In function `std::basic_string<char, std::char_traits<char>, std::allocator<char> > boost::locale::conv::to_utf<char>(char const*, char const*, std::locale const&, boost::locale::conv::method_type)':
asyncrt_utils.cpp:(.text._ZN5boost6locale4conv6to_utfIcEESbIT_St11char_traitsIS3_ESaIS3_EEPKcS9_RKSt6localeNS1_11method_typeE[_ZN5boost6locale4conv6to_utfIcEESbIT_St11char_traitsIS3_ESaIS3_EEPKcS9_RKSt6localeNS1_11method_typeE]+0x18): undefined reference to `boost::locale::info::id'
asyncrt_utils.cpp:(.text._ZN5boost6locale4conv6to_utfIcEESbIT_St11char_traitsIS3_ESaIS3_EEPKcS9_RKSt6localeNS1_11method_typeE[_ZN5boost6locale4conv6to_utfIcEESbIT_St11char_traitsIS3_ESaIS3_EEPKcS9_RKSt6localeNS1_11method_typeE]+0x76): undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> > boost::locale::conv::to_utf<char>(char const*, char const*, std::string const&, boost::locale::conv::method_type)'

/usr/local/lib/casablanca/libcasablanca.a(pplxlinux.cpp.o): In function `pplx::details::platform::YieldExecution()':
pplxlinux.cpp:(.text+0x21): undefined reference to `boost::this_thread::yield()'
Do I am missing to link against extra libraries ?

FYI : my service use Boost also internally and I also add definitions _NO_ASYNCRTIMP and _NO_PPLXIMP to my project

Thanks

PS: This thread is a clone of https://casablanca.codeplex.com/workitem/102 which can be deleted
Coordinator
Apr 17, 2014 at 9:31 AM
Hi lpouget,

Thanks for posting this diff!

It looks like you aren't linking against boost locale. Boost is actually a collection of several libraries, so it's possible that the boost lib you're linking against does not have the locale component.

As an aside: what issues were you receiving that required you to define _NO_ASYNCRTIMP and _NO_PPLXIMP?

Thanks,
roschuma
Apr 17, 2014 at 3:20 PM
Edited Apr 17, 2014 at 3:37 PM
Hi roschuma,
As an aside: what issues were you receiving that required you to define _NO_ASYNCRTIMP and _NO_PPLXIMP?
In my project or when I build casablanca ?


I modified the cmake of my project :
  • I removed _NO_ASYNCRTIMP and _NO_PPLXIMP definitions
  • Here is libraries which I link against :
    • mysqlclient
    • z
    • dl
    • boost (with ${Boost_LIBRARIES})
      • /usr/lib/x86_64-linux-gnu/libboost_system.a
      • /usr/lib/x86_64-linux-gnu/libboost_thread.a
      • /usr/lib/x86_64-linux-gnu/libboost_program_options.a
      • /usr/lib/x86_64-linux-gnu/libboost_regex.a
      • /usr/lib/x86_64-linux-gnu/libboost_locale.a
      • /usr/lib/x86_64-linux-gnu/libboost_filesystem.a
      • /usr/lib/x86_64-linux-gnu/libpthread.so
    • -l:/usr/local/lib/casablanca/libcasablanca.a
And I had exactly the same error.

Thanks,

Laurent

Edit : Boost locale would need deps with ICU ?
Edit 2 : I add ICU link but no more success (same error)
Coordinator
Apr 17, 2014 at 4:08 PM
Edited Apr 17, 2014 at 4:13 PM
You're completely right that newer versions of boost require ICU (and not linking against it would produce issues sooner or later).

Now, on to the main problem. I think that your issue is your link order. I suspect you have a link line that looks like
gcc main.o foo.o bar.o -ldl -lz -lboost -lcasablanca
Unfortunately, due to the particulars of how linking works, this will fail to properly link all the symbols from Casablanca against boost. Long story short, you fix this by putting -lcasablanca as the first link in the list:
gcc main.o foo.o bar.o -lcasablanca -ldl -lz -lboost
Long story long: When linking occurs, the linker works by building a table of "unresolved symbols needed thus far". So first, it goes through your primary objects (main, foo, and bar) and records all the symbols they need. Then it looks at (in the first case) dl.a and says "What parts of this library do I need? dlsym and dlopen," which it then adds to final binary. Then, it goes to libz and does the same. Then we get to boost. At this point, the linker says "Hmm, you don't seem to be using any boost symbols. Oh well, just toss the library." and doesn't include any of the symbols. Then we get to Casablanca and it says "Ok, you're using function utility::conversions::usascii_to_utf16 which relies on boost::locale::conv::to_utf" and adds boost::locale::conf::to_utf to the list of symbols needed.

The crucial point here, is that the linker will not backtrack. Since it has already passed boost, it will not go back to look again and will therefore not locate the symbol.

If this doesn't fix your issue, could you please post the full link line (you can reveal it by doing make VERBOSE=1)?
In my project or when I build casablanca ?
I am curious about your steps for building Casablanca -- specifically, I would like to understand why you needed to patch Release/CMakeLists.txt to include these two definitions.

Edit: After re-reading your initial post, I realize that you (were?) using those macros in your actual project, thus needed to add them to the CMake file when compiling Casablanca.

Sincerely,
roschuma
Marked as answer by roschuma on 4/18/2014 at 5:05 PM
Apr 18, 2014 at 8:29 AM
Thanks a lot for your precisions !

You're right the link order was not right and your tips solved it.
I didn't have too much time to explain the full process but I will make a feedback about static compilation.

Have a nice day :)

Laurent
Apr 26, 2014 at 2:44 PM
Bonjour Laurent,

We will have to statically link soon. Any help would be appreciated.

Merci, Robert
May 26, 2014 at 8:16 AM
Bonjour Robert ;)

Sorry for the delay...
To statically link on linux, you just need to ensure that libicu (dev) is available. It's needed by Boost Locale.
Then, ensure you have the right import order like roschuma states it.

This can help you if you use CMake to build your project :
  • https://github.com/julp/FindICU.cmake
  • set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -static -static-libgcc -static-libstdc++ -std=c++11 -fno-strict-aliasing")
  • set(LIBS "-lcasablanca" ${Boost_LIBRARIES} ${ICU_LIBRARIES} "-lz -lm -ldl -lc")
If all goes well, you should have a fully statically executable.

De rien,

Laurent
May 26, 2014 at 10:28 AM
Merci Laurent for the timely info. Timely indeed. By coincidence, today is the day for linking my project together on Linux. Yesterday I finally got clean compiled source code on both Windows and Linux so today is linker day. -- Robert - Metro Villiers.
Sep 11, 2014 at 7:58 PM
I've been trying to follow these steps with v2.2.0 and while Casablanca compiles fine, when I try to link my application I get the error:
/usr/bin/c++       CMakeFiles/dcodr_service.dir/dcodr_service.cc.o CMakeFiles/dcodr_service.dir/viterbi.cc.o CMakeFiles/dcodr_service.dir/utils.cc.o CMakeFiles/dcodr_service.dir/hmm.cc.o CMakeFiles/dcodr_service.dir/grammar.cc.o  -o dcodr_service  -L/home/user/casablanca/Release/Binaries -rdynamic -lfftw3f -lcpprest -lcommon_utilities -lboost_program_options -lboost_regex -lboost_system -lboost_filesystem -lboost_iostreams /opt/OpenBLAS/lib/libopenblas.a -Wl,-rpath,/home/user/casablanca/Release/Binaries 
CMakeFiles/dcodr_service.dir/dcodr_service.cc.o: In function `web::http::http_request::extract_json(bool) const::{lambda(unsigned long)#1}::operator()(unsigned long) const':
/home/user/casablanca/Release/include/cpprest/http_msg.h:829: undefined reference to `web::http::details::http_msg_base::_extract_json(bool)'
collect2: error: ld returned 1 exit status
make[2]: *** [dcodr_service] Error 1
This worked fine when linking against the shared library version. Thanks.