[RFC PATCH] ext-toolchain: add C wrapper for external toolchain
Ye Holmes
yeholmes at outlook.com
Mon Feb 22 07:59:55 EST 2021
Building GNU toolchains on a server shared by a team of developers
can be expensive, the fact that some developers tend to re-build
openwrt now and then makes the server on high load. So an external
GNU toolchain is mandatory to keep the building processing fast.
Glibc needs to be used instead of uclibc/musl for some packages, such
as opencv/ffmpeg (opencv package has been removed from packages.git),
however, building openwrt with an external toolchain (with glibc)
has produced many errors, which leads me to the conclusion that
openwrt has been rarely built with external toolchains (and with glibc).
Some errors are due to glibc, for example:
1. ubox-2020-10-25-9ef88681/log/logd.c:263:3: error:
ignoring return value of 'setuid' declared with attribute
'warn_unused_result' [-Werror=unused-result]
2. /path/to/ext-toolchain/bin/ld: openwrt/tmp/pppd.p7KbAi.ltrans2.ltrans.o:
in function `DesSetkey': <artificial>:(.text.DesSetkey+0xc0):
undefined reference to `setkey'
(pppd fails to link, function setkey has been removed from
recent versions of glibc)
Another error is due to external toolchain's inability to handle object
files generated with LTO enabled:
/path/to/bin/aarch64-none-linux-gnu-ar: bn_cutoffs.o:
plugin needed to handle lto object
aarch64-none-linux-gnu-ranlib libtommath.a
/path/to/bin/aarch64-none-linux-gnu-ranlib: libtommath.a(bn_cutoffs.o):
plugin needed to handle lto object
/path/to/aarch64-none-linux-gnu/bin/ld:
openwrt/tmp/dropbearmulti.RYKalz.ltrans0.ltrans.o:
in function `dropbear_chachapoly_start':
<artificial>:(.text+0x4c0): undefined reference to `chacha_setup'
CROSS_PREFIX-ar and CROSS_PREFIX-ranlib from the external toolchain
cannot process static archives correctly, in this scenario,
CROSS_PREFIX-gcc-ar and CROSS_PREFIX-gcc-ranlib can be used to
avoid the problem. So, this patch changes shell script
*aarch64-none-linux-gnu-ar* in $(TOOLCHAIN_DIR)/bin directory from
> #!/bin/sh
> exec "/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-ar" "$@"
to
> #!/bin/sh
> exec "/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc-ar" "$@"
Besides dropbear, there are many other packages have the problem.
The change is optional, via a configurable option:
-> Advanced configuration options (for developers) --->
--> Use external toolchain --->
---> [*] Toolchain LTO archive handling
Also, the CROSS_PREFIX-gcc-ar and CROSS_PREFIX-gcc-ranlib cannot be
used due to incorrect command-line arguments:
CC ../src/ap/acs.c
CC main.o
/path/to/ext-toolchain/bin/ar: two different operation options specified
make[3]: *** [Makefile:1332: hostapd_multi.a] Error 1
The contents of CROSS_PREFIX-gcc-ar is:
> openwrt/staging_dir/toolchain-aarch64-linux-gnu/bin$ cat aarch64-none-linux-gnu-gcc-ar
> #!/bin/sh
> for arg in "$@"; do
> case "$arg" in -l*|-L*|-shared|-static)
> exec "/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc-ar" -Os -pipe -mcpu=cortex-a53 -mglibc -I/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/usr/include -I/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/include -L/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/usr/lib -L/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/lib ${STAGING_DIR:+-idirafter "$STAGING_DIR/usr/include" -L "$STAGING_DIR/usr/lib" -Wl,-rpath-link,"$STAGING_DIR/usr/lib"} "$@" ;;
> esac
> done
> exec "/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc-ar" -Os -pipe -mcpu=cortex-a53 -mglibc -I/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/usr/include -I/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/include -L/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/usr/lib -L/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/lib ${STAGING_DIR:+-idirafter "$STAGING_DIR/usr/include"} "$@"
The patch will also fixes this problem, simplify CROSS_PREFIX-gcc-ar/gcc-nm/gcc-ranlib:
> #!/bin/sh
> exec "/opt/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc-ar" "$@"
The enable faster wrapper invocation of external toolchain, a C
implementation of wrapper utility has been written to wrap around
CROSS_PREFIX-cpp/gcc/g++ executables. The benefits also includes
more complex command-line arguments processing impossible in shell
scripts, precise detection of certain compilation/linking options,
and more flexible command-line alterations. I've tested the new
toolchain wrapper with a raspberry Pi 3b+ device, it works well.
However, I think the change is heavy and too much to be merged in
the master shortly, especially from someone new to openwrt such
as me, I would like to hear some comments about the patch, and
I would be very happy and grateful if someday the patch could
be merged.
Last but not least, the patch is intended to supersede two other patches:
http://patchwork.ozlabs.org/project/openwrt/patch/ME1PR01MB081749A42D9D5923842870AAD0D20@ME1PR01MB0817.ausprd01.prod.outlook.com/
http://patchwork.ozlabs.org/project/openwrt/patch/SYXPR01MB083243A82C6D345B61FE4DD5D0D50@SYXPR01MB0832.ausprd01.prod.outlook.com/
Best,
Ye Holmes
---
scripts/ext-toolchain.sh | 82 +--
toolchain/Config.in | 12 +
toolchain/wrapper/Makefile | 9 +-
toolchain/wrapper/src/toolchain-wrapper.c | 644 ++++++++++++++++++++++
4 files changed, 715 insertions(+), 32 deletions(-)
create mode 100644 toolchain/wrapper/src/toolchain-wrapper.c
diff --git a/scripts/ext-toolchain.sh b/scripts/ext-toolchain.sh
index ee7d9532f5..912f0391e9 100755
--- a/scripts/ext-toolchain.sh
+++ b/scripts/ext-toolchain.sh
@@ -27,7 +27,7 @@ CFLAGS=""
TOOLCHAIN="."
LIBC_TYPE=""
-
+LTO_AR_RANLIB=""
# Library specs
LIB_SPECS="
@@ -201,43 +201,49 @@ find_bins() {
wrap_bin_cc() {
local out="$1"
- local bin="$2"
-
- echo '#!/bin/sh' > "$out"
- echo 'for arg in "$@"; do' >> "$out"
- echo ' case "$arg" in -l*|-L*|-shared|-static)' >> "$out"
- echo -n ' exec "'"$bin"'" '"$CFLAGS"' ${STAGING_DIR:+' >> "$out"
- echo -n '-idirafter "$STAGING_DIR/usr/include" ' >> "$out"
- echo -n '-L "$STAGING_DIR/usr/lib" ' >> "$out"
- echo '-Wl,-rpath-link,"$STAGING_DIR/usr/lib"} "$@" ;;' >> "$out"
- echo ' esac' >> "$out"
- echo 'done' >> "$out"
- echo -n 'exec "'"$bin"'" '"$CFLAGS"' ${STAGING_DIR:+' >> "$out"
- echo '-idirafter "$STAGING_DIR/usr/include"} "$@"' >> "$out"
- chmod +x "$out"
+ # remove the target if already exists
+ [ -e "$out" ] && rm -rf "$out"
+ ln -sv toolchain-wrapper "$out" || exit 2
+ return 0
}
-wrap_bin_ld() {
+wrap_bin_other() {
local out="$1"
local bin="$2"
echo '#!/bin/sh' > "$out"
- echo -n 'exec "'"$bin"'" ${STAGING_DIR:+' >> "$out"
- echo -n '-L "$STAGING_DIR/usr/lib" ' >> "$out"
- echo '-rpath-link "$STAGING_DIR/usr/lib"} "$@"' >> "$out"
+ echo 'exec "'"$bin"'" "$@"' >> "$out"
chmod +x "$out"
}
-wrap_bin_other() {
+wrap_bin_lto() {
local out="$1"
local bin="$2"
- echo '#!/bin/sh' > "$out"
- echo 'exec "'"$bin"'" "$@"' >> "$out"
+ if [ -z "$LTO_AR_RANLIB" ] ; then
+ echo '#!/bin/sh' > "$out"
+ echo "exec \"$bin\" \$@" >> "$out"
+ chmod +x "$out"
+ return 0
+ fi
- chmod +x "$out"
+ # the script is executed by GNU bash,
+ # regular expressions are possible via [[ ]]
+ if [[ "$bin" =~ (^.+)-([^-]+)$ ]] ; then
+ bin="${BASH_REMATCH[1]}-gcc-${BASH_REMATCH[2]}"
+ if [ ! -x "$bin" ] ; then
+ echo "Error, toolchain not found: \"$bin\"." 1>&2
+ exit 1
+ fi
+ echo '#!/bin/sh' > "$out"
+ echo "exec \"$bin\" \$@" >> "$out"
+ chmod +x "$out"
+ return 0
+ fi
+ echo "Error, invalid toolchain binary: \"$bin\"" 1>&2
+ exit 1
}
wrap_bins() {
@@ -250,18 +256,28 @@ wrap_bins() {
local out="$1/${cmd##*/}"
local bin="$cmd"
- if [ -x "$out" ] && ! grep -q STAGING_DIR "$out"; then
- mv "$out" "$out.bin"
- bin='$(dirname "$0")/'"${out##*/}"'.bin'
- fi
-
case "${cmd##*/}" in
+ *-gcc-ar)
+ wrap_bin_other "$out" "$bin"
+ ;;
+ *-gcc-nm)
+ wrap_bin_other "$out" "$bin"
+ ;;
+ *-gcc-ranlib)
+ wrap_bin_other "$out" "$bin"
+ ;;
+ *-ar)
+ wrap_bin_lto "$out" "$bin"
+ ;;
+ *-nm)
+ wrap_bin_lto "$out" "$bin"
+ ;;
+ *-ranlib)
+ wrap_bin_lto "$out" "$bin"
+ ;;
*-*cc|*-*cc-*|*-*++|*-*++-*|*-cpp)
wrap_bin_cc "$out" "$bin"
;;
- *-ld)
- wrap_bin_ld "$out" "$bin"
- ;;
*)
wrap_bin_other "$out" "$bin"
;;
@@ -573,6 +589,10 @@ while [ -n "$1" ]; do
exit 1
;;
+ --lto-ar-ranlib)
+ LTO_AR_RANLIB=yes
+ ;;
+
*)
echo "Unknown argument '$arg'" >&2
exec $0 --help
diff --git a/toolchain/Config.in b/toolchain/Config.in
index 6dda9af92d..a1f64ed298 100644
--- a/toolchain/Config.in
+++ b/toolchain/Config.in
@@ -96,6 +96,18 @@ menuconfig EXTERNAL_TOOLCHAIN
default "/opt/cross/powerpc-unknown-linux-gnu" if powerpc
default "/opt/cross/x86_64-unknown-linux-gnu" if x86_64
+ config TOOLCHAIN_LTO_AR_RANLIB
+ bool
+ prompt "Toolchain LTO archive handling"
+ depends on EXTERNAL_TOOLCHAIN && !NATIVE_TOOLCHAIN
+ default n
+ help
+ Specify the object files compiled with LTO enabled should be
+ handled by CROSS_PREFIX-gcc-ar/CROSS_PREFIX-gcc-ranlib durning
+ static archives processing, instead of by CROSS_PREFIX-ar/ranlib.
+ This can be especially helpful for some external toolchains,
+ such as arm/arm64 toolchains provided by ARM.
+
choice TOOLCHAIN_LIBC_TYPE
prompt "Toolchain libc" if DEVEL
depends on EXTERNAL_TOOLCHAIN && !NATIVE_TOOLCHAIN
diff --git a/toolchain/wrapper/Makefile b/toolchain/wrapper/Makefile
index 3398e407c9..1c713ca273 100644
--- a/toolchain/wrapper/Makefile
+++ b/toolchain/wrapper/Makefile
@@ -20,7 +20,7 @@ $(strip $(SCRIPT_DIR)/ext-toolchain.sh --toolchain $(CONFIG_TOOLCHAIN_ROOT) \
--cflags "$(if $(call qstrip,$(CONFIG_TOOLCHAIN_LIBC)),-m$(call qstrip,$(CONFIG_TOOLCHAIN_LIBC))) $(if $(CONFIG_SOFT_FLOAT),-msoft-float)" \
--cflags "$(patsubst ./%,-I$(TOOLCHAIN_ROOT_DIR)/%,$(call qstrip,$(CONFIG_TOOLCHAIN_INC_PATH)))" \
--cflags "$(patsubst ./%,-L$(TOOLCHAIN_ROOT_DIR)/%,$(call qstrip,$(CONFIG_TOOLCHAIN_LIB_PATH)))" \
- $(1))
+ $(if $(CONFIG_TOOLCHAIN_LTO_AR_RANLIB),--lto-ar-ranlib) $(1))
endef
# 1: config symbol
@@ -49,9 +49,16 @@ define Host/Configure
endef
define Host/Compile
+ $(HOSTCC) -D_GNU_SOURCE $(HOST_CFLAGS) \
+ -DTCW_ROOT=\"$(CONFIG_TOOLCHAIN_ROOT)\" \
+ -DTCW_PREFIX=\"$(CONFIG_TOOLCHAIN_PREFIX)\" \
+ $(if $(CONFIG_TARGET_OPTIMIZATION),-DTCW_CFLAGS=\"$(CONFIG_TARGET_OPTIMIZATION)\") \
+ -o $(HOST_BUILD_DIR)/toolchain-wrapper src/toolchain-wrapper.c
endef
define Host/Install
+ [ ! -e "$(TOOLCHAIN_DIR)/bin" ] && mkdir -p "$(TOOLCHAIN_DIR)/bin"; exit 0
+ $(INSTALL_BIN) $(HOST_BUILD_DIR)/toolchain-wrapper "$(TOOLCHAIN_DIR)/bin/"
$(call toolchain_util,--wrap "$(TOOLCHAIN_DIR)/bin")
endef
diff --git a/toolchain/wrapper/src/toolchain-wrapper.c b/toolchain/wrapper/src/toolchain-wrapper.c
new file mode 100644
index 0000000000..837c9507fd
--- /dev/null
+++ b/toolchain/wrapper/src/toolchain-wrapper.c
@@ -0,0 +1,644 @@
+/*
+ * toolchain-wrapper.c - external GNU toolchain wrapper
+ *
+ * Copyright (C) 2021 Ye Holmes <yeholmes at outlook.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ */
+
+#include <errno.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <sys/types.h>
+#include <fcntl.h>
+#include <unistd.h>
+
+/*
+ * toolchain-wrapper aims to wrapper
+ * following GNU toolchain executables:
+ *
+ * CROSS_PREFIX-gcc
+ * CROSS_PREFIX-gcc-X.Y.Z
+ * CROSS_PREFIX-c++
+ * CROSS_PREFIX-g++
+ * CROSS_PREFIX-gfortran
+ *
+ * If the wrapper thinks it necessay to add compilation or
+ * linking flags, it will insert the arguments after argv[0]:
+ *
+ * ARGV[0] { LIST OF INSERTED COMPILATION ARGS } ARGV[1] ARGV[2] ...
+ *
+ * No arguments are modified or removed from the original argument array.
+ */
+
+/* specific toolchain type */
+enum tcw_type {
+ TCW_TYPE_UNKNOWN = 0,
+ TCW_TYPE_CPP, /* CROSS_PREFIX-cpp */
+ TCW_TYPE_GCC, /* CROSS_PREFIX-gcc */
+ TCW_TYPE_GCC_XYZ, /* CROSS_PREFIX-gcc-x.y.z, example: aarch64-linux-gnu-gcc-7.5.0 */
+ TCW_TYPE_GPP, /* CROSS_PREFIX-g++, or CROSS_PREFIX-c++ */
+ TCW_TYPE_GFORTRAN, /* CROSS_PREFIX-gfortran */
+};
+
+/*
+ * wrapper command-line argument information
+ */
+struct wrapper_info {
+ enum tcw_type type; /* type of executable */
+ int orig_argc; /* argc from main(...) */
+ char * * orig_argv; /* argv from main(...) */
+ char * orig_name; /* invoked application base name of argv[0] */
+ char * real_path; /* absolute path of real executable */
+ const char * staging_dir; /* $ENV{STAGING_DIR} */
+ const char * output; /* argument given by -o, points to the output file name */
+ int total_argc; /* total number of new arguments avaiable from `new_argv */
+ int new_argc; /* number of new arguments added in the argument array */
+ char * * new_argv; /* new arguments added in the front of argument array */
+
+ /* command-line arguments traits */
+ unsigned int found_l : 1; /* -l* found in the argument array */
+ unsigned int found_L : 1; /* -L* found in the argument array */
+ unsigned int f_nostdinc : 1; /* -nostdinc found in the argument array */
+ unsigned int f_nostdincpp : 1; /* -nostdinc++ found in the argument array */
+ unsigned int f_nostdlib : 1; /* -Wl,-nostdlib found in the argument array */
+ unsigned int d_kernel : 1; /* -D__KERNEL__ found */
+ unsigned int d_uboot : 1; /* -D__UBOOT__ found */
+};
+
+/* check for the absolute path of external toolchain */
+#ifndef TCW_ROOT
+#error TCW_ROOT not defined
+#endif
+
+/* check for cross compiler prefix, such as arm-linux-gnueabihf- */
+#ifndef TCW_PREFIX
+#error TCW_PREFIX not defined
+#endif
+
+/* sub-directory containing all the GNU toolchain binaries */
+#ifndef TCW_BINDIR
+#define TCW_BINDIR "bin"
+#endif
+
+/* private definitions for toolchain-wrapper */
+#define TCW_DEBUG_NEW 1
+#define TCW_DEBUG_ALL 2
+
+/* determine the toolchain specific type */
+static enum tcw_type wrapper_info_type(const char * name);
+
+/* initialize wrapper runtime information */
+static int wrapper_info_init(struct wrapper_info * pwi,
+ int argc, char * argv[]);
+
+/* process wrapper runtime information */
+static void wrapper_info_process(struct wrapper_info * pwi);
+
+/* merge command-line arguments */
+static int wrapper_info_merge(struct wrapper_info * pwi);
+
+/* debug wrapper information */
+static void wrapper_info_debug(struct wrapper_info * pwi);
+
+/* destroy wrapper runtime information */
+static void wrapper_info_destroy(struct wrapper_info * pwi);
+
+int main(int argc, char *argv[])
+{
+ struct wrapper_info wi;
+
+ memset(&wi, 0, sizeof(wi));
+ if (wrapper_info_init(&wi, argc, argv) < 0) {
+ wrapper_info_destroy(&wi);
+ return 1;
+ }
+
+ if (wi.type != TCW_TYPE_GFORTRAN)
+ wrapper_info_process(&wi);
+
+ if (wrapper_info_merge(&wi) < 0) {
+ wrapper_info_destroy(&wi);
+ return 3;
+ }
+
+ wrapper_info_debug(&wi);
+
+ /* invoke the real GNU toolchain executable */
+ execv(wi.real_path, wi.new_argv);
+ fprintf(stderr, "Error, failed to invoke %s: %s\n",
+ wi.orig_name, strerror(errno));
+ fflush(stderr);
+
+ wrapper_info_destroy(&wi);
+ return 4;
+}
+
+/* dump the command-line argument array */
+static void wrapper_dump_argv(int argc, char * * argv)
+{
+ int i;
+ size_t tlen;
+ const char * arg;
+
+ tlen = 0;
+ for (i = 0; i < argc; ++i) {
+ size_t arglen;
+ arg = argv[i];
+
+ if (arg == NULL)
+ break;
+
+ if (arg[0] == '\0') {
+ /* zero-length argument */
+ arglen = 2;
+ arg = "\"\"";
+ } else
+ arglen = strlen(arg);
+
+ arglen++; /* for the trailing space */
+ if ((tlen + arglen) >= 80) {
+ tlen = arglen = 0;
+ fprintf(stderr, "%s\n%s", arg,
+ (i + 1) >= argc ? "" : "\t");
+ } else {
+ fputs(arg, stderr);
+ fputc(' ', stderr);
+ tlen += arglen;
+ }
+ }
+ if (tlen != 0)
+ fputc('\n', stderr);
+ fflush(stderr);
+}
+
+void wrapper_info_debug(struct wrapper_info * pwi)
+{
+ unsigned long opt;
+ char * next, * semi;
+ const char * debug, * old;
+
+ opt = 0;
+ debug = getenv("TCW_DEBUG");
+ if (debug == NULL)
+ return;
+
+ old = debug; /* backup debug string pointer */
+ semi = strchr(debug, ';');
+ if (semi == debug)
+ goto err0;
+
+ if (semi != NULL) {
+ size_t outlen;
+ const char * bn;
+
+ /* we're tracing specific output file */
+ if (pwi->output == NULL)
+ return;
+
+ /*
+ * fetch basename for output file,
+ * do not use `basename(char * path) because, from man-page:
+ * Both dirname() and basename() may modify the contents of path...
+ */
+ bn = strrchr(pwi->output, '/');
+ if (bn == NULL)
+ bn = pwi->output;
+ else
+ bn++;
+
+ outlen = semi - debug;
+ /* basename compare of output file */
+ if (strncmp(semi, bn, outlen) != 0)
+ return;
+
+ /* check again */
+ if (bn[outlen] != '\0')
+ return;
+
+ semi++;
+ debug = semi;
+ }
+
+ errno = 0;
+ next = NULL;
+ /* pase the debug integer option */
+ opt = strtoul(debug, &next, 0);
+ if (errno != 0 || next == debug)
+ goto err0;
+
+ /* dump the command-line arguments */
+ if (opt & TCW_DEBUG_ALL)
+ wrapper_dump_argv(pwi->orig_argc + pwi->new_argc, pwi->new_argv);
+ else if (opt & TCW_DEBUG_NEW) {
+ if (pwi->new_argc == 0) {
+ fprintf(stderr, "No new arguments added: %s\n", pwi->orig_name);
+ } else
+ wrapper_dump_argv(pwi->new_argc + 1, pwi->new_argv);
+ }
+ fprintf(stderr, "PATH=%s\n", getenv("PATH") ? : "nil");
+ fflush(stderr);
+ return;
+
+err0:
+ fprintf(stderr, "Warning, invalid TCW_DEBUG: %s\n", old);
+ fflush(stderr);
+}
+
+/* function to finally merge the argument array */
+static int wrapper_info_merge_(struct wrapper_info * pwi)
+{
+ int i, argc, argt;
+
+ argc = pwi->new_argc;
+ argt = pwi->total_argc;
+ if ((argt - argc) <= pwi->orig_argc) {
+ int args;
+ char * * new_argv;
+
+ /* allocate new argument array */
+ args = argc + pwi->orig_argc + 3;
+ new_argv = (char * *) realloc((void *) pwi->new_argv,
+ (size_t) (args * sizeof(char *)));
+ if (new_argv == NULL) {
+ fputs("Error, system out of memory!\n", stderr);
+ return -1;
+ }
+ pwi->total_argc = args;
+ pwi->new_argv = new_argv;
+ }
+
+ pwi->new_argv[0] = pwi->real_path;
+ /* copy the original command-line arguments */
+ for (i = 1; i < pwi->orig_argc; ++i)
+ pwi->new_argv[argc + i] = pwi->orig_argv[i];
+
+ /* Do not update pwi->new_argc, for debug purpose */
+ /* pwi->new_argc = argc + i; */
+ pwi->new_argv[argc + i] = NULL;
+ return 0;
+}
+
+/* function to add a compilation argument */
+static void wrapper_info_addarg(struct wrapper_info * pwi,
+ char * new_arg, int alloc)
+{
+ int argc, argt;
+
+ argc = pwi->new_argc;
+ argt = pwi->total_argc;
+
+ /* need to allocate new memory to hold new argument ? */
+ if ((argc + 1) >= argt) {
+ int args;
+ char * * new_argv;
+
+ args = (argt < 0x3) ? 0x3 : (argt << 1);
+ new_argv = (char * *) realloc((void *) pwi->new_argv,
+ (size_t) (args * sizeof(char *)));
+ if (new_argv == NULL) {
+ fputs("Error, system out of memory!\n", stderr);
+ exit(5); /* just terminate wrapper */
+ }
+
+ /* the new_argv is indexed from 1, not 0 */
+ pwi->total_argc = args;
+ pwi->new_argv = new_argv;
+ }
+
+ /* store a copy of the new argument */
+ if (alloc == 0) {
+ pwi->new_argv[argc + 1] = new_arg;
+ } else {
+ pwi->new_argv[argc + 1] = strdup(new_arg);
+ if (pwi->new_argv[argc + 1] == NULL) {
+ fprintf(stderr, "Error, strdup(%s) has failed!\n", new_arg);
+ exit(6);
+ }
+ }
+ pwi->new_argc = argc + 1;
+}
+
+#ifdef TCW_CFLAGS
+/*
+ * split a string such as "-Os -pipe -mcpu=cortex-a53",
+ * add the compliation arguments one by one.
+ */
+static int wrapper_add_cflags(struct wrapper_info * pwi, const char * flags_)
+{
+ char * flags, * p0, * p1;
+
+ /* the string duplication is needed becase we need to modify it */
+ flags = strdup(flags_);
+ if (flags == NULL) {
+ fprintf(stderr, "Error, strdup(%s) has failed!\n", flags_);
+ return -1;
+ }
+
+ p0 = flags;
+ /* skip spaces or tabs */
+ while (*p0 == ' ' || *p0 == '\t')
+ p0++;
+
+ /*
+ * naïve way to split a string of compilation arguments.
+ * TODO: find a better way to split string
+ */
+ while (*p0 != '\0') {
+ char cha;
+
+ p1 = p0;
+ cha = *p1;
+ while (cha != ' ' && cha != '\t' && cha != '\0') {
+ p1++;
+ cha = *p1;
+ }
+
+ /* p0 != p1 */
+ if (cha == '\0') {
+ wrapper_info_addarg(pwi, p0, 1);
+ break;
+ }
+
+ *p1++ = '\0'; /* terminate the compilation flag */
+ wrapper_info_addarg(pwi, p0, 1);
+
+ cha = *p1;
+ /* again, skip spaces or tabs */
+ while (cha == ' ' || cha == '\t') {
+ p1++;
+ cha = *p1;
+ }
+ p0 = p1;
+ }
+
+ free(flags);
+ return 0;
+}
+#endif
+
+int wrapper_info_merge(struct wrapper_info * pwi)
+{
+ /* for kernel and u-boot compilation, do not add any arguments */
+ if (pwi->d_kernel || pwi->d_uboot)
+ return wrapper_info_merge_(pwi);
+
+ /*
+ * openwrt _rarely_ invokes gfortran compiler; in fact
+ * gfortran compiler is never used by openwrt (as far as I known).
+ */
+ if (pwi->type == TCW_TYPE_GFORTRAN)
+ return wrapper_info_merge_(pwi);
+
+ /* no standard C/C++ header inclusion */
+ if (pwi->f_nostdinc || pwi->f_nostdincpp)
+ goto link_dirs;
+
+#ifdef TCW_CFLAGS
+ wrapper_add_cflags(pwi, TCW_CFLAGS);
+#endif
+
+ if (pwi->staging_dir != NULL) {
+ size_t dirlen;
+ char idir[64], * incdir;
+
+ /* Add compilation flag: -I$(STAGING_DIR)/usr/include */
+ dirlen = strlen(pwi->staging_dir) + 64;
+ incdir = (char *) malloc(dirlen);
+ if (incdir == NULL) {
+ fprintf(stderr, "Error, system out of memory: %zu\n", dirlen);
+ return -1;
+ }
+
+ strcpy(idir, "-idirafter");
+ wrapper_info_addarg(pwi, idir, 1);
+
+ /* use asprintf(...) instead ? */
+ snprintf(incdir, dirlen, "%s/usr/include", pwi->staging_dir);
+ wrapper_info_addarg(pwi, incdir, 0);
+ }
+
+link_dirs:
+ if (pwi->type == TCW_TYPE_CPP)
+ /* preprocessor needs no linker options */
+ return wrapper_info_merge_(pwi);
+
+ if (pwi->found_l == 0 && pwi->found_L == 0) {
+ /*
+ * no extra library needed,
+ * nether -l* nor -L* found in the command-line arguments.
+ */
+ return wrapper_info_merge_(pwi);
+ }
+
+ if (pwi->f_nostdlib == 0 && pwi->staging_dir != NULL) {
+ size_t dirlen;
+ char * libdir;
+
+ dirlen = strlen(pwi->staging_dir) + 64;
+ libdir = (char *) malloc(dirlen);
+ if (libdir == NULL) {
+ fprintf(stderr, "Error, system out of memory: %zu\n", dirlen);
+ return -1;
+ }
+
+ /* Add linking flag: -L$(STAGING_DIR)/usr/lib */
+ snprintf(libdir, dirlen, "-L%s/usr/lib", pwi->staging_dir);
+ wrapper_info_addarg(pwi, libdir, 1);
+
+ /* also for the linker library searching */
+ snprintf(libdir, dirlen, "-Wl,-rpath-link=%s/usr/lib", pwi->staging_dir);
+ wrapper_info_addarg(pwi, libdir, 0);
+ }
+
+ return wrapper_info_merge_(pwi);
+}
+
+void wrapper_info_process(struct wrapper_info * pwi)
+{
+ int idx, argc;
+ char * * argv;
+
+ argc = pwi->orig_argc;
+ argv = pwi->orig_argv;
+ for (idx = 1; idx < argc; ++idx) {
+ const char * arg;
+
+ arg = argv[idx];
+ if (arg[0] != '-')
+ continue;
+
+ switch (arg[1]) {
+ case 'l':
+ pwi->found_l = 1;
+ break;
+
+ case 'L':
+ pwi->found_L = 1;
+ break;
+
+ case 'n':
+ if (strcmp(arg, "-nostdinc") == 0)
+ pwi->f_nostdinc = 1;
+ else if (strcmp(arg, "-nostdinc++") == 0)
+ pwi->f_nostdincpp = 1;
+ else if (strcmp(arg, "-nostdlib") == 0) /* linker option */
+ pwi->f_nostdlib = 1;
+ break;
+
+ case 'D':
+ /* check only for -D__KERNEL__ or -D__UBOOT__ */
+ if (arg[2] == '_' && arg[3] == '_') {
+ if (strcmp(arg, "-D__KERNEL__") == 0)
+ pwi->d_kernel = 1;
+ else if (strcmp(arg, "-D__UBOOT__") == 0)
+ pwi->d_uboot = 1;
+ }
+ break;
+
+ case 'o':
+ if (arg[2] == '\0' && (idx + 1) < argc) {
+ idx++;
+ /* store output file */
+ pwi->output = argv[idx];
+ }
+ break;
+
+ case 'W':
+ if (arg[2] == 'l' &&
+ arg[3] == ',' &&
+ strcmp(arg, "-Wl,-nostdlib") == 0)
+ pwi->f_nostdlib = 1;
+ break;
+
+ case 'X':
+ if (arg[2] == 'l' &&
+ strcmp(arg, "-Xlinker") == 0 &&
+ (idx + 1) < argc &&
+ strcmp(argv[idx + 1], "-nostdlib") == 0) {
+ idx++;
+ pwi->f_nostdlib = 1;
+ }
+ break;
+
+ default:
+ break;
+ }
+ }
+}
+
+void wrapper_info_destroy(struct wrapper_info * pwi)
+{
+ if (pwi->real_path != NULL) {
+ free(pwi->real_path);
+ pwi->real_path = NULL;
+ }
+
+ if (pwi->new_argc > 0) {
+ int idx;
+
+ for (idx = 1; idx <= pwi->new_argc; ++idx) {
+ free(pwi->new_argv[idx]);
+ pwi->new_argv[idx] = NULL;
+ }
+
+ pwi->new_argc = 0;
+ pwi->total_argc = 0;
+ free((void *) pwi->new_argv);
+ pwi->new_argv = NULL;
+ }
+}
+
+enum tcw_type wrapper_info_type(const char * name)
+{
+ size_t plen;
+ const char * prefix;
+ enum tcw_type type = TCW_TYPE_UNKNOWN;
+
+ prefix = TCW_PREFIX;
+ plen = strlen(prefix);
+ if (strncmp(prefix, name, plen) != 0)
+ return type;
+
+ name += plen;
+ if (strcmp(name, "cpp") == 0)
+ type = TCW_TYPE_CPP;
+ else if (strcmp(name, "gcc") == 0)
+ type = TCW_TYPE_GCC;
+ else if (strcmp(name, "g++") == 0)
+ type = TCW_TYPE_GPP;
+ else if (strcmp(name, "c++") == 0)
+ type = TCW_TYPE_GPP;
+ else if (strcmp(name, "gfortran") == 0)
+ type = TCW_TYPE_GFORTRAN;
+ else {
+ if (name[0] == 'g' && name[1] == 'c' &&
+ name[2] == 'c' && name[3] == '-') {
+ /* TCW_PREFIX-gcc- */
+ type = TCW_TYPE_GCC_XYZ;
+ }
+ }
+ return type;
+}
+
+int wrapper_info_init(struct wrapper_info * pwi,
+ int argc, char * argv[])
+{
+ char * arg0;
+ size_t arglen;
+ const char * tc_prefix;
+
+ pwi->orig_argc = argc;
+ pwi->orig_argv = (char * *) argv;
+ /* use basename(...) instead ? */
+ arg0 = strrchr(argv[0], '/');
+ if (arg0 != NULL)
+ arg0++;
+ else
+ arg0 = argv[0];
+
+ pwi->orig_name = arg0;
+ pwi->type = wrapper_info_type(arg0);
+ if (pwi->type == TCW_TYPE_UNKNOWN) {
+ fprintf(stderr, "Warning, unknown TCW type: %s\n", arg0);
+ return -1;
+ }
+
+ tc_prefix = TCW_ROOT "/" TCW_BINDIR;
+ arglen = strlen(tc_prefix) + strlen(arg0) + 16;
+ /* construct the absolute path of real executable */
+ pwi->real_path = (char *) malloc(arglen);
+ if (pwi->real_path == NULL) {
+ fputs("Error, system out of memory!\n", stderr);
+ return -1;
+ }
+ snprintf(pwi->real_path, arglen, "%s/%s", tc_prefix, arg0);
+
+ /* check whether the real executable exists */
+ if (access(pwi->real_path, X_OK) != 0) {
+ fprintf(stderr, "Error executable not found: %s\n", pwi->real_path);
+ return -1;
+ }
+
+ /* find staging_dir via environment variable */
+ pwi->staging_dir = getenv("STAGING_DIR");
+ if (pwi->staging_dir == NULL) {
+ fputs("Warning, STAGING_DIR not defined.\n", stderr);
+ fflush(stderr);
+ }
+ return 0;
+}
--
2.27.0
More information about the openwrt-devel
mailing list