[OpenWrt-Devel] Problem building modules out-of-tree (kbuild) for x86

Alexandru Ardelean ardeleanalex at gmail.com
Mon Dec 19 03:25:10 EST 2016 

On Mon, Dec 19, 2016 at 1:41 AM, Philip Prindeville
<philipp_subx at redfish-solutions.com> wrote:
> Hi.
>
> I’m trying to add the Intel e1000e, igb, and ixgbe drivers that are on Source Forge.
>
> A snapshot of my work is here:
>
> https://github.com/pprindeville/packages/tree/sf-ether
>
> When I build this as “make -j1 V=s” using a mostly unmodified x86_64 generic configuration, I get:
>
>
>
> make[3]: Entering directory `/home/philip/bertram/bmu-philip.sanity/openwrt/feeds/packages/kernel/e1000e'
> . /home/philip/bertram/bmu-philip.sanity/openwrt/include/shell.sh; gzip -dc /home/philip/bertram/bmu-philip.sanity/openwrt/dl/e1000e-3.3.5.3.tar.gz | tar -C /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/.. -xf -
> touch /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/.prepared_b318088b5b0515ca987313e94558d492
> rm -f /home/philip/bertram/bmu-philip.sanity/openwrt/staging_dir/target-x86_64_musl-1.1.15_powercode-bmu/stamp/.e1000e_installed
> (cd /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/./; if [ -x ./configure ]; then find /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/ -name config.guess | xargs -r chmod u+w; find /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/ -name config.guess | xargs -r -n1 cp --remove-destination /home/philip/bertram/bmu-philip.sanity/openwrt/scripts/config.guess; find /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/ -name config.sub | xargs -r chmod u+w; find /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/ -name config.sub | xargs -r -n1 cp --remove-destination /home/philip/bertram/bmu-philip.sanity/openwrt/scripts/config.sub; AR="x86_64-openwrt-linux-musl-gcc-ar" AS="x86_64-openwrt-linux-musl-gcc -c -Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -iremap /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3:e1000e-3.3.5.3 -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro" LD=x86_64-openwrt-linux-musl-ld NM="x86_64-openwrt-linux-musl-gcc-nm" CC="x86_64-openwrt-linux-musl-gcc" GCC="x86_64-openwrt-linux-musl-gcc" CXX="x86_64-openwrt-linux-musl-g++" RANLIB="x86_64-openwrt-linux-musl-gcc-ranlib" STRIP=x86_64-openwrt-linux-musl-strip OBJCOPY=x86_64-openwrt-linux-musl-objcopy OBJDUMP=x86_64-openwrt-linux-musl-objdump SIZE=x86_64-openwrt-linux-musl-size CFLAGS="-Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -iremap /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3:e1000e-3.3.5.3 -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro " CXXFLAGS="-Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -iremap /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3:e1000e-3.3.5.3 -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro " CPPFLAGS="-I/home/philip/bertram/bmu-philip.sanity/openwrt/staging_dir/target-x86_64_musl-1.1.15_powercode-bmu/usr/include -I/home/philip/bertram/bmu-philip.sanity/openwrt/staging_dir/target-x86_64_musl-1.1.15_powercode-bmu/include -I/home/philip/bertram/bmu-philip.sanity/openwrt/staging_dir/toolchain-x86_64_gcc-5.3.0_musl-1.1.15/usr/include -I/home/philip/bertram/bmu-philip.sanity/openwrt/staging_dir/toolchain-x86_64_gcc-5.3.0_musl-1.1.15/include/fortify -I/home/philip/bertram/bmu-philip.sanity/openwrt/staging_dir/toolchain-x86_64_gcc-5.3.0_musl-1.1.15/include " LDFLAGS="-L/home/philip/bertram/bmu-philip.sanity/openwrt/staging_dir/target-x86_64_musl-1.1.15_powercode-bmu/usr/lib -L/home/philip/bertram/bmu-philip.sanity/openwrt/staging_dir/target-x86_64_musl-1.1.15_powercode-bmu/lib -L/home/philip/bertram/bmu-philip.sanity/openwrt/staging_dir/toolchain-x86_64_gcc-5.3.0_musl-1.1.15/usr/lib -L/home/philip/bertram/bmu-philip.sanity/openwrt/staging_dir/toolchain-x86_64_gcc-5.3.0_musl-1.1.15/lib -znow -zrelro "   ./configure --target=x86_64-openwrt-linux --host=x86_64-openwrt-linux --build=x86_64-linux-gnu --program-prefix="" --program-suffix="" --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/lib --sysconfdir=/etc --datadir=/usr/share --localstatedir=/var --mandir=/usr/man --infodir=/usr/info --disable-nls   ; fi; )
> rm -f /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/.configured_*
> touch /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/.configured_yyyyyyyyy
> make -C "/home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/linux-4.4.14" ARCH="x86" CROSS_COMPILE="x86_64-openwrt-linux-musl-" M="/home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/src" modules
> make[4]: Entering directory `/home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/linux-4.4.14'
>   CC [M]  /home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/src/netdev.o
> gcc: error: unrecognized command line option '-mskip-rax-setup'
> make[5]: *** [/home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/src/netdev.o] Error 1
> make[4]: *** [_module_/home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/src] Error 2
> make[4]: Leaving directory `/home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/linux-4.4.14'
> make[3]: *** [/home/philip/bertram/bmu-philip.sanity/openwrt/build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/e1000e-3.3.5.3/.built] Error 2
> make[3]: Leaving directory `/home/philip/bertram/bmu-philip.sanity/openwrt/feeds/packages/kernel/e1000e'
> make[2]: *** [package/feeds/packages/e1000e/compile] Error 2
> make[2]: Leaving directory `/home/philip/bertram/bmu-philip.sanity/openwrt'
> make[1]: *** [/home/philip/bertram/bmu-philip.sanity/openwrt/staging_dir/target-x86_64_musl-1.1.15_powercode-bmu/stamp/.package_compile] Error 2
> make[1]: Leaving directory `/home/philip/bertram/bmu-philip.sanity/openwrt'
> make: *** [world] Error 2
>
>
> but looking through the logs, I don’t see anywhere else that -mskip-rax-setup gets passed in as a compiler flag.  What am I missing?
>
> The option is coming from build_dir/target-x86_64_musl-1.1.15_powercode-bmu/linux-x86_64/linux-4.4.14/arch/x86/Makefile:
>
>         # Use -mskip-rax-setup if supported.
>         KBUILD_CFLAGS += $(call cc-option,-mskip-rax-setup)
>
> It seems to have been introduced with:
>
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/arch/x86/Makefile?id=d9ee948d82203811a545ba26b0172fce4970d1dc
>
>
> and the functionality in gcc here:
>
> ------------------------------------------------------------------------
> r218870 | hjl | 2014-12-18 10:35:45 -0700 (Thu, 18 Dec 2014) | 31 lines
>
> X86-64: Add -mskip-rax-setup
>
> The Linux kernel never passes floating point arguments around, vararg
> functions or not. Hence no vector registers are ever used when calling a
> vararg function.  But gcc still dutifully emits an "xor %eax,%eax" before
> each and every call of a vararg function.  Since no callee use that for
> anything, these instructions are redundant.
>
> This patch adds the -mskip-rax-setup option to skip setting up RAX
> register when SSE is disabled and there are no variable arguments passed
> in vector registers.  Since RAX register is used to avoid unnecessarily
> saving vector registers on stack when passing variable arguments, the
> impacts of this option are callees may waste some stack space, misbehave
> or jump to a random location.  GCC 4.4 or newer don't those issues,
> regardless the RAX register value since they don't check the RAX register
> value when SSE is disabled.
>
> gcc/
>
>         * config/i386/i386.c (ix86_expand_call): Skip setting up RAX
>         register for -mskip-rax-setup when there are no parameters
>         passed in vector registers.
>         * config/i386/i386.opt (mskip-rax-setup): New option.
>         * doc/invoke.texi: Document -mskip-rax-setup.
>
> gcc/testsuite/
>
>         * gcc.target/i386/amd64-abi-7.c: New tests.
>         * gcc.target/i386/amd64-abi-8.c: Likwise.
>         * gcc.target/i386/amd64-abi-9.c: Likwise.
>
> ————————————————————————————————————
>
> Why would the option -mskip-rax-setup be present when doing module kbuilds out-of-tree, but not be present when building the kernel image itself or building modules in-tree?
>
> Any help is appreciated.
>
> Thanks,
>
> -Philip
> _______________________________________________
> openwrt-devel mailing list
> openwrt-devel at lists.openwrt.org
> https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel

Maybe your host gcc is being picked by `make` ?
In that case, it probably detects that option as valid/available.

I don't know how to better address this ; I'd assume, maybe by
ensuring that the CC variable points to the right GCC.
Tho, curiously,  CROSS_COMPILE="x86_64-openwrt-linux-musl-"   should
that be x86_64-openwrt-linux-musl-gcc ?

Maybe, also look for missing env vars that point to the right GCC.
_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel

More information about the openwrt-devel mailing list