Build errors for dynamic module on Ubuntu 20.04 on Raspeberry Pi 4 ARM.

[Danrancan]

I am trying to build the Brotli modules on Ubuntu 20.04 on a Raspberry Pi 4. I tried building with the master and the RC1, but em encountering the same errors. Below is the error I am encountering:

sudo make modules
make -f objs/Makefile modules
make[1]: Entering directory '/usr/local/src/nginx-1.25.4'
cc -c -fPIC -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g  -Wno-deprecated-declarations -I src/core -I src/event -I src/event/modules -I src/event/quic -I src/os/unix -I /usr/local/src/ngx_brotli/deps/brotli/c/include -I objs -I src/http -I src/http/modules \
	-o objs/addon/filter/ngx_http_brotli_filter_module.o \
	/usr/local/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c
cc -c -fPIC -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g  -Wno-deprecated-declarations -I src/core -I src/event -I src/event/modules -I src/event/quic -I src/os/unix -I /usr/local/src/ngx_brotli/deps/brotli/c/include -I objs -I src/http -I src/http/modules \
	-o objs/ngx_http_brotli_filter_module_modules.o \
	objs/ngx_http_brotli_filter_module_modules.c
cc -o objs/ngx_http_brotli_filter_module.so \
objs/addon/filter/ngx_http_brotli_filter_module.o \
objs/ngx_http_brotli_filter_module_modules.o \
-L/usr/local/src/ngx_brotli/deps/brotli/c/../out -lbrotlienc -lbrotlicommon -lm \
-shared
/usr/bin/ld: cannot find -lbrotlienc
/usr/bin/ld: cannot find -lbrotlicommon
collect2: error: ld returned 1 exit status
make[1]: *** [objs/Makefile:1219: objs/ngx_http_brotli_filter_module.so] Error 1
make[1]: Leaving directory '/usr/local/src/nginx-1.25.4'
make: *** [Makefile:16: modules] Error 2

What can I do to make this work on a raspberry pi 4 running ubuntu 20.04 server?

[hieuhtr]

Document's a little bit confused. You need to build dependencies first.

git clone --recurse-submodules -j8 https://github.com/google/ngx_brotli
cd ngx_brotli/deps/brotli
mkdir out && cd out
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_C_FLAGS="-Ofast -m64 -march=native -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_CXX_FLAGS="-Ofast -m64 -march=native -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_INSTALL_PREFIX=./installed ..
cmake --build . --config Release --target brotlienc
cd ../../../..

after that, you will have brotlienc & brotlicommon

Then continue to compile statically or dynamically based on your need

Static

$ cd nginx-1.x.x
$ export CFLAGS="-m64 -march=native -mtune=native -Ofast -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections"
$ export LDFLAGS="-m64 -Wl,-s -Wl,-Bsymbolic -Wl,--gc-sections"
$ ./configure --add-module=/path/to/ngx_brotli
$ make && make install

Dynamic load

$ cd nginx-1.x.x
$ ./configure --with-compat --add-dynamic-module=/path/to/ngx_brotli
$ make modules

[Danrancan]

I'm getting compile errors. Looks like -m64 is an invalid option. Any advice?

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_C_FLAGS="-Ofast -m64 -march=native -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_CXX_FLAGS="-Ofast -m64 -march=native -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_INSTALL_PREFIX=./installed ..

-- The C compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- broken
CMake Error at /usr/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake:60 (message):
  The C compiler

    "/usr/bin/cc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /usr/local/src/ngx_brotli/deps/brotli/out/CMakeFiles/CMakeTmp

    Run Build Command(s):/usr/bin/make cmTC_1bb78/fast && /usr/bin/make -f CMakeFiles/cmTC_1bb78.dir/build.make CMakeFiles/cmTC_1bb78.dir/build
    make[1]: Entering directory '/usr/local/src/ngx_brotli/deps/brotli/out/CMakeFiles/CMakeTmp'
    Building C object CMakeFiles/cmTC_1bb78.dir/testCCompiler.c.o
    /usr/bin/cc   -Ofast -m64 -march=native -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections    -o CMakeFiles/cmTC_1bb78.dir/testCCompiler.c.o   -c /usr/local/src/ngx_brotli/deps/brotli/out/CMakeFiles/CMakeTmp/testCCompiler.c
    cc: error: unrecognized command line option ‘-m64’
    make[1]: *** [CMakeFiles/cmTC_1bb78.dir/build.make:66: CMakeFiles/cmTC_1bb78.dir/testCCompiler.c.o] Error 1
    make[1]: Leaving directory '/usr/local/src/ngx_brotli/deps/brotli/out/CMakeFiles/CMakeTmp'
    make: *** [Makefile:121: cmTC_1bb78/fast] Error 2

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:11 (project)

-- Configuring incomplete, errors occurred!
See also "/usr/local/src/ngx_brotli/deps/brotli/out/CMakeFiles/CMakeOutput.log".
See also "/usr/local/src/ngx_brotli/deps/brotli/out/CMakeFiles/CMakeError.log".

Okay, I tried now with the -march=armv8-a option, and deleted the -m64 options since I believe cmake automatically includes the -my 64 option since I'm running an arm64 OS. Here is the output, which still gives a warning:

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_C_FLAGS="-Ofast -march=armv8-a -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_CXX_FLAGS="-Ofast -march=armv8-a -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_INSTALL_PREFIX=./installed ..
-- The C compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc

-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Build type is 'Release'
-- Performing Test BROTLI_EMSCRIPTEN
-- Performing Test BROTLI_EMSCRIPTEN - Failed
-- Compiler is not EMSCRIPTEN
-- Looking for log2
-- Looking for log2 - not found
-- Looking for log2
-- Looking for log2 - found
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_CXX_FLAGS

Looks like the command works, but should I be worried about this warning? Do I need the manually specified variables? What can I do here?

NEXT: I tried with the following command:

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_C_FLAGS="-Ofast -march=armv8-a -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_CXX_FLAGS="-Ofast -march=armv8-a -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_INSTALL_PREFIX=./installed ..
-- The C compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc

-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Build type is 'Release'
-- Performing Test BROTLI_EMSCRIPTEN
-- Performing Test BROTLI_EMSCRIPTEN - Failed
-- Compiler is not EMSCRIPTEN
-- Looking for log2
-- Looking for log2 - not found
-- Looking for log2
-- Looking for log2 - found
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_CXX_FLAGS

-- Build files have been written to: /usr/local/src/ngx_brotli/deps/brotli/out

This shows two errors/warnings:

1. -- Performing Test BROTLI_EMSCRIPTEN - Failed
   Q: How do I get BROTLI EMSCRIPTEN to work?
2. Manually-specified variables were not used by the project: CMAKE_CXX_FLAGS

NEXT I tried the following command (notice I changed -mtune to -mtune=armv8-a):

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_C_FLAGS="-Ofast -march=armv8-a -mtune=armv8-a -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_CXX_FLAGS="-Ofast -march=armv8-a -mtune=armv8-a -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_INSTALL_PREFIX=./installed ..
-- Build type is 'Release'
-- Compiler is not EMSCRIPTEN
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_CXX_FLAGS

-- Build files have been written to: /usr/local/src/ngx_brotli/deps/brotli/out

Which DID NOT show EMSCRIPTION errors, but DID show "Manually-specified variables were not used by the project: CMAKE_CXX_FLAGS".

NEXT, I tried the same command again, to yield different results:

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_C_FLAGS="-Ofast -march=armv8-a -mtune=armv8-a -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_CXX_FLAGS="-Ofast -march=armv8-a -mtune=armv8-a -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_INSTALL_PREFIX=./installed ..
-- The C compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- broken
CMake Error at /usr/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake:60 (message):
  The C compiler

    "/usr/bin/cc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /usr/local/src/ngx_brotli/deps/brotli/out/CMakeFiles/CMakeTmp

    Run Build Command(s):/usr/bin/make cmTC_e8cf3/fast && /usr/bin/make -f CMakeFiles/cmTC_e8cf3.dir/build.make CMakeFiles/cmTC_e8cf3.dir/build
    make[1]: Entering directory '/usr/local/src/ngx_brotli/deps/brotli/out/CMakeFiles/CMakeTmp'
    Building C object CMakeFiles/cmTC_e8cf3.dir/testCCompiler.c.o
    /usr/bin/cc   -Ofast -march=armv8-a -mtune=armv8-a -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections    -o CMakeFiles/cmTC_e8cf3.dir/testCCompiler.c.o   -c /usr/local/src/ngx_brotli/deps/brotli/out/CMakeFiles/CMakeTmp/testCCompiler.c
    cc1: error: unknown value ‘armv8-a’ for ‘-mtune’
    cc1: note: valid arguments are: cortex-a35 cortex-a53 cortex-a57 cortex-a72 cortex-a73 thunderx thunderxt88p1 thunderxt88 octeontx octeontx81 octeontx83 thunderxt81 thunderxt83 emag xgene1 falkor qdf24xx exynos-m1 phecda thunderx2t99p1 vulcan thunderx2t99 cortex-a55 cortex-a75 cortex-a76 ares neoverse-n1 neoverse-e1 a64fx tsv110 zeus neoverse-v1 saphira neoverse-n2 cortex-a57.cortex-a53 cortex-a72.cortex-a53 cortex-a73.cortex-a35 cortex-a73.cortex-a53 cortex-a75.cortex-a55 cortex-a76.cortex-a55 generic
    make[1]: *** [CMakeFiles/cmTC_e8cf3.dir/build.make:66: CMakeFiles/cmTC_e8cf3.dir/testCCompiler.c.o] Error 1
    make[1]: Leaving directory '/usr/local/src/ngx_brotli/deps/brotli/out/CMakeFiles/CMakeTmp'
    make: *** [Makefile:121: cmTC_e8cf3/fast] Error 2

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:11 (project)

-- Configuring incomplete, errors occurred!
See also "/usr/local/src/ngx_brotli/deps/brotli/out/CMakeFiles/CMakeOutput.log".
See also "/usr/local/src/ngx_brotli/deps/brotli/out/CMakeFiles/CMakeError.log".

NEXT I MADE changes to -mtune (-mtune=cortex-a72) and tried the following command:

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_C_FLAGS="-Ofast -march=armv8-a -mtune=cortex-a72 -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_CXX_FLAGS="-Ofast -march=armv8-a -mtune=cortex-a72 -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_INSTALL_PREFIX=./installed ..
-- The C compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Build type is 'Release'
-- Performing Test BROTLI_EMSCRIPTEN
-- Performing Test BROTLI_EMSCRIPTEN - Failed
-- Compiler is not EMSCRIPTEN
-- Looking for log2
-- Looking for log2 - not found
-- Looking for log2
-- Looking for log2 - found
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_CXX_FLAGS

-- Build files have been written to: /usr/local/src/ngx_brotli/deps/brotli/out

THEN, I tried the exact same command a second time which yielded different results:

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_C_FLAGS="-Ofast -march=armv8-a -mtune=cortex-a72 -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_CXX_FLAGS="-Ofast -march=armv8-a -mtune=cortex-a72 -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_INSTALL_PREFIX=./installed ..
-- Build type is 'Release'
-- Compiler is not EMSCRIPTEN
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/local/src/ngx_brotli/deps/brotli/out

As you can see, running the cmake command twice with the -mtune=coretx-a72 and -march=armv8-a options, yields results without any problems except that "Compiler is not EMSCRIPTEN". So I am going with using this last command twice and proceeding hoping that this is the right way of doing things.

I must ask:

1. Can you verify that this last command is correct?
2. And why must I issue the last command twice for it to pick up and read the manually specified variables?
3. How can I get the compiler to be "EMSCRIPTEN", and what are the benefits of doing so?
   Is that right or...?

Finally:

after that, you will have brotlienc & brotlicommon
The only thing I see in ngx_brotli/deps/ folder is the brotli folder i.e. ngx_brotli/deps/brotli. I do not see a ngx_brotli/deps/brotlicommon folder.

4. Where should brotlicommon be located?

Thanks!

[Danrancan]

Any word from the dev's on if I am doing this correctly or not? Let me know.

[ibarchenkov]

Any word from the dev's on if I am doing this correctly or not? Let me know.

I'm not qualified to answer your question. However, I encountered the exact same issue with Nginx 1.26.0 running on Ubuntu 20.04 with an ARM processor (Ampere). Following your route, I decided to compile Brotli with fewer flags as specified in the official repo README. After doing so, I was able to successfully compile Nginx with dynamic modules. I tested the setup, and everything worked just fine, so I called it a day. I guess/hope that if you're compiling for the same architecture and OS, the compiler is smart enough to come up with a decent configuration on its own.