perl/host/compile: fails on macOS 11 Big Sur, workaround

[Georgi Valkov]

Dear Rosen,

Thanks to your help I discovered the feeds/packages repository, however I was not able to resolve the race condition during build and discover why Errno.pm is not built when multiple workers are used e.g.: make package/feeds/packages/perl/host/{clean,compile} -j 12

The good news is that if we move to perl 5.30.0 or newer, the race condition is resolved. We also need to add a few flags to files/base.config:
d_attribute_always_inline=‘define'
d_setlocale_accepts_any_locale_name=‘undef'
d_sockaddr_storage=‘undef’
d_towlower=‘undef'
d_towupper=‘undef'
d_wcrtomb=‘undef'
i_wctype=‘define’

Then we can compile perl/host 5.30.0 up to 5.32.0. However building perl for the target fails the install phase:

error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: input file: /Volumes/test/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/perl/perl-5.30.0/ipkg-install/usr/bin/perl5.30.0 is not a Mach-O file
Cannot update /Volumes/test/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/perl/perl-5.30.0/ipkg-install/usr/bin/perl5.30.0 dependency paths
make[3]: *** [makefile:445: install-all] Error 1
make[3]: Leaving directory '/Volumes/test/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/perl/perl-5.30.0'
make[2]: *** [Makefile:152: /Volumes/test/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/perl/perl-5.30.0/.built] Error 2

Another symptom I see, is that with 5.30.0 build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/perl/perl-5.30.0/ipkg-install/usr/bin contains only perl5.30.0, while with the original 5.28.1 it contains many executables.

Can you please update perl to 5.30.0 or 5.32.0? Just make sure it compiles correctly on Linux or whatever environment you are using, and I will test it on macOS 11. I seem to be missing something, and it might take me a very long time to learn my way through, because I lack experience.


Thank you again for any help!
Georgi Valkov


> On 2020-11-18, at 10:32 AM, Georgi Valkov <gvalkov at abv.bg> wrote:
> 
> 
> 
>> On 2020-11-18, at 4:24 AM, Rosen Penev <rosenp at gmail.com> wrote:
>> 
>> On Tue, Nov 17, 2020 at 3:16 PM Georgi Valkov <gvalkov at abv.bg> wrote:
>>> 
>>> Dear OpenWRT team,
>>> 
>>> I just updated my laptop to macOS 11 Big Sur. Presently only the perl hostpkg package fails during the build. Once I get past perl, everything else compiles correctly. Below is a patch, workaround, and details from my research. My target is Marvell EBU Armada, Linksys WRT3200ACM.
>>> 
>>> Before applying the patch, make package/feeds/packages/perl/host/compile V=sc fails with the following error:
>>> Which of these apply, if any? [darwin]
>>> 
>>> *** Unexpected product version 11.0.
>>> ***
>>> *** Try running sw_vers and see what its ProductVersion says.
>>> 
>>> After the patch, make package/feeds/packages/perl/host/compile succeeds, however when multiple workers are used: make package/feeds/packages/perl/host/compile -j 12 V=sc, the build fails:
>>> Writing Makefile for CPAN
>>> make[4]: Entering directory '/Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/Archive-Tar'
>>> make[4]: Entering directory '/Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/CPAN'
>>> Generating a Unix-style Makefile
>>> Writing Makefile for Errno
>>> make[4]: Entering directory '/Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/ext/Errno'
>>> "../../miniperl" "-I../../lib" Errno_pm.PL Errno.pm
>>> Can't locate Errno.pm in @INC (you may need to install the Errno module) (@INC contains: /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/AutoLoader/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/Carp/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/PathTools /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/PathTools/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/ExtUtils-Install/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/ExtUtils-MakeMaker/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/ExtUtils-Manifest/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/File-Path/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/ext/re /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/Term-ReadLine/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/Exporter/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/ext/File-Find/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/Text-Tabs/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/constant/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/version/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/lib .) at /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/PathTools/Cwd.pm line 444.
>>> Unsuccessful Makefile.PL(cpan/ExtUtils-MakeMaker): code=512 at make_ext.pl line 518.
>>> make[3]: *** [GNUmakefile:599: cpan/ExtUtils-MakeMaker/pm_to_blib] Error 2
>>> 
>>> I have also seen errors where the build is trying to use /miniperl instead of ./miniperl, as well as /build_dir/hostpkg/perl/perl-5.28.1/perl instead of /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/perl. The will occur if we try to build perl without host/clean and clean. Seems like a race condition due to dependency misconfiguration.
>>> 
>>> Here is the list of all commands in order to compile OpenWRT starting from a clean build root:
>>> make -j 12
>>> make package/feeds/packages/perl/host/clean
>>> make package/feeds/packages/perl/clean
>>> make package/feeds/packages/perl/compile
>>> make -j 12
>>> 
>>> The first make fails, then we perform host-clean and clean for perl, next compile will also do host-compile - we have to use a single worker here. Finally we can complete the rest of the build process. I should note that Languages, Perl, perl is not selected in menuconfig. I can understand if the hostpkg/perl is required during the build process, however perl/host{clean,compile} is not enough, I also need to perform perl/{clean,compile}.
>>> 
>>> Here is the quick patch I made in order to verify if there are any other broken packages: no. Only perl fails to compile. I am also not sure how to handle patches and pull-requests for feeds, so I thought I should rather share my research here and ask for help.
>>> 
>>> feeds/packages/lang/perl/patches/302-macos_11_support.patch
>>> Index: perl-5.28.1/hints/darwin.sh
>>> ===================================================================
>>> --- perl-5.28.1.orig/hints/darwin.sh
>>> +++ perl-5.28.1/hints/darwin.sh
>>> @@ -327,7 +327,7 @@ EOM
>>>    # "ProductVersion:    10.11"     "10.11"
>>>        prodvers=`sw_vers|awk '/^ProductVersion:/{print $2}'|awk -F. '{print $1"."$2}'`
>>>    case "$prodvers" in
>>> -    10.*)
>>> +    10.*|11.*)
>>>      add_macosx_version_min ccflags $prodvers
>>>      add_macosx_version_min ldflags $prodvers
>>>      ;;
>>> 
>> This patch is for https://github.com/openwrt/packages
>> 
>> Please submit a PR there.
>> 
>> Speaking of perl, it could probably use an update.
> 
> I would only make a pull-request when I’m certain I have corrected all issues and tested thoroughly.
> Indeed the perl package must be updated. I’m not sure if I am allowed to create patches for /feeds, because
> /feeds and /package/feeds are in .gitignore, so I shouldn't force changes inside these directories.
> At least I’m not aware of a proper way to do so.
> 
>>> 
>>> Thank you for any help!
>>> Georgi Valkov
>>> 
>>> https://bugs.openwrt.org/index.php?do=details&task_id=3458
>>> 
>>> 
>>> _______________________________________________
>>> openwrt-devel mailing list
>>> openwrt-devel at lists.openwrt.org
>>> https://lists.openwrt.org/mailman/listinfo/openwrt-devel