[OpenWrt-Devel] [PATCH 2/3] IP Virtual Server - Load Balancer - ipvsadm tool

Yousong Zhou yszhou4tech at gmail.com
Wed Feb 24 20:56:31 EST 2016


Hi, Mauro


On 25 February 2016 at 01:00, Mauro Mozzarelli <mauro at ezplanet.net> wrote:
> Yousong,
>
> Thank you for your suggestions.
> Please see my notes below after your comments
>
> Mauro
>
> On Wed, February 24, 2016 13:31, Yousong Zhou wrote:
>>                 yousong
>>
>>
>> On 24 February 2016 at 03:57, Mauro Mozzarelli <mauro at ezplanet.net> wrote:
>>> All,
>>>
>>> Part 2/3 includes the ipvsadm tool only.
>>> It depends on the ipvs kernel modules patch 1/3
>>>
>>> Mauro
>>>
>>> Author: Mauro Mozzarelli <mauro at ezplanet.net>
>>> Date:   Tue Feb 23 19:22:51 2016 +0000
>>>
>>>     This ipvsadm package adds Linux IP Virtual Server capability to OpenWRT
>>>     IPVS (IP Virtual Server) implements transport-layer load balancing inside the Linux kernel, so called Layer-4 switching.
>>> IPVS running on a host acts as a load balancer at the front of a cluster of real servers, it can direct requests for
>>> TCP/UDP based services to the real servers, and makes services of the real servers to appear as a virtual service on a
>>> single IP address.
>>>
>>>     More information at: http://www.linuxvirtualserver.org/software/
>>>
>>>     This package contains:
>>>     - ipvsadm utility to configure IP VS tables, virtual and real servers
>>>
>>>     Dependencies:
>>>     - ipvs kernel modules
>>>
>>>     Signed-off-by: Mauro Mozzarelli <mauro at ezplanet.net>
>>>
>>> diff --git a/package/network/utils/ipvsadm/Makefile b/package/network/utils/ipvsadm/Makefile
>>> new file mode 100644
>>> index 0000000..8abde77
>>> --- /dev/null
>>> +++ b/package/network/utils/ipvsadm/Makefile
>>> @@ -0,0 +1,59 @@
>>> +#
>>> +# Copyright (C) 2016 OpenWRT, Mauro Mozzarelli
>>
>> Not quire sure about this, but it's OpenWrt instead of OpenWRT :)
>
> Amended, just in case
>
>>
>>> +#
>>> +# This is free software, licensed under the GNU General Public License v2.
>>> +# See /LICENSE for more information.
>>> +#
>>> +# AUTHOR: Mauro Mozzarelli
>>> +#
>>> +include $(TOPDIR)/rules.mk
>>> +
>>> +PKG_NAME:=ipvsadm
>>> +PKG_VERSION:=1.26
>>> +PKG_MAINTAINER:=Mauro Mozzarelli <mauro at ezplanet.net>
>>> +PKG_LICENSE:=GPL-1.0
>>> +PKG_RELEASE:=1
>>> +
>>> +PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
>>> +PKG_SOURCE_URL:=http://www.linuxvirtualserver.org/software/kernel-2.6/
>>> +PKG_MD5SUM:=eac3ba3f62cd4dea2da353aeddd353a8
>>> +
>>> +PKG_BUILD_PARALLEL:=1
>>> +PKG_INSTALL:=1
>>> +
>>> +ifeq ($(CONFIG_DEVEL),y)
>>> +       export QUILT=1
>>> +endif
>>
>> What's the above 3 lines for?
>
> Left over from previous testing. Now removed. Thank you for spotting.
>
>>> +
>>> +include $(INCLUDE_DIR)/package.mk
>>> +
>>> +define Package/$(PKG_NAME)
>>
>> use ipvsadm instead of $(PKG_NAME)
>>
>>> +  SECTION:=net
>>> +  CATEGORY:=Network
>>> +  TITLE:=IP Virtual Server Configuration Manager
>>> +  URL:=http://www.linuxvirtualserver.org
>>> +  DEPENDS:= +kmod-ipvs +libnl-tiny +libpopt
>>> +endef
>>> +
>>> +#      -I$(STAGING_DIR)/usr/include/libnl/linux -fPIC -DLIBIPVS_USE_NL
>>
>> This commented out code should be removed
>
> I removed it, however the above works for Chaos Calmer.
> Trunk requires -D_GNU_SOURCE or it would not compile
>
>>> +TARGET_CFLAGS += \
>>> +       -I$(STAGING_DIR)/usr/include/libnl-tiny -fPIC -DLIBIPVS_USE_NL -D_GNU_SOURCE
>>
>>  - `make HAVE_NL=1` should be preferred over -DLIBIPVS_USE_NL
>>  - -fPIC already is already set in libipvs/Makefile
>>  - Is that -D_GNU_SOURCE required?
>>
>
> Yes, see above
>
>>> +
>>> +MAKE_FLAGS += \
>>> +       CC="$(TOOLCHAIN_DIR)/bin/$(TARGET_CC)" \
>>> +       CXX="$(TOOLCHAIN_DIR)/bin/$(TARGET_CXX)" \
>>> +       CFLAGS="$(TARGET_CFLAGS)" \
>>> +       LIBS="-lnl-tiny -lpopt"
>>> +
>>
>>  - These should be assigned to MAKE_VARS and `make POPT_LIB=-lpopt`
>> should be preferred over `LIBS=-lopt`
>>  - I have not run-tested it, but those assignment to CC, CXX, CFLAGS
>> can be omitted here as they are already handled by the OpenWrt build
>> system
>>
>
> I removed CC and CXX.
> CFLAGS and LIBS are necessary to compile
>
>>> +define Build/Compile
>>> +       $(call Build/Compile/Default)
>>> +endef
>>
>> This can also be removed as it is the default behaviour of build system.
>>
>
> Removed, thank you for spotting.
>
>>> +
>>> +define Package/ipvsadm/install
>>> +       $(INSTALL_DIR) $(1)/usr/sbin
>>> +       $(INSTALL_BIN) $(PKG_BUILD_DIR)/ipvsadm $(1)/usr/sbin/
>>> +       $(INSTALL_BIN) $(PKG_BUILD_DIR)/ipvsadm-save $(1)/usr/sbin/
>>> +       $(INSTALL_BIN) $(PKG_BUILD_DIR)/ipvsadm-restore $(1)/usr/sbin/
>>> +endef
>>> +
>>> +$(eval $(call BuildPackage,$(PKG_NAME)))
>>
>> Just use ipvsadm instead of $(PKG_NAME)
>
> Where possible I prefer to use variables instead of hard coded names.
> This helps when renaming packages or re-using code for other packages.

Yes, it makes sense if all packages' Makefiles are written in this
way.  But almost all if not all of them just write the name directly.
I think these are just the convention which is not that bad against
code reuse/refactor anyway.

Other arguments are

 - <name> in Package/<name> is not strictly related to PKG_NAME
 - The patch here has Package/ipvsadm/install

>
>>> diff --git a/package/network/utils/ipvsadm/patches/001-ipvsadm.patch
>>> b/package/network/utils/ipvsadm/patches/001-ipvsadm.patch
>>> new file mode 100644
>>> index 0000000..643d5ae
>>> --- /dev/null
>>> +++ b/package/network/utils/ipvsadm/patches/001-ipvsadm.patch
>>> @@ -0,0 +1,199 @@
>>> +diff -Naur ipvsadm-1.26/ipvsadm.spec ipvsadm-1.26.owrt/ipvsadm.spec
>>> +--- ipvsadm-1.26/ipvsadm.spec  2011-02-08 00:25:36.000000000 +0000
>>> ++++ ipvsadm-1.26.owrt/ipvsadm.spec     1970-01-01 01:00:00.000000000 +0100
>>
>> This part is irrelevant and should be removed...
>
> Thank you for spotting. It is now removed.
>
>>
>>> +@@ -1,100 +0,0 @@
>>> +-%define prefix   /usr
>>> +-
>>> +-Summary: Utility to administer the Linux Virtual Server
>>> +-Name: ipvsadm
>>> +-Version: 1.26
>>> +-Release: 1
>>> +-License: GPL
>>> +-URL: http://www.LinuxVirtualServer.org/
>>> +-Group: Applications/System
>>> +-Source0: http://www.LinuxVirtualServer.org/software/ipvsadm-%{version}.tar.gz
>>> +-BuildRoot: /var/tmp/%name-%{PACKAGE_VERSION}-root
>>> +-Provides: %{name}-%{version}
>>> +-Conflicts: piranha <= 0.4.14
>>> +-
>>> +-%description
>>> +-ipvsadm is a utility to administer the IP Virtual Server services
>>> +-offered by the latest Linux kernel 2.6.x.
>>> +-
>>> +-
>>> +-%prep
>>> +-%setup -n %{name}-%{version}
>>> +-
>>> +-
>>> +-%build
>>> +-CFLAGS="${RPM_OPT_FLAGS}" make
>>> +-
>>> +-
>>> +-%install
>>> +-rm -rf $RPM_BUILD_ROOT
>>> +-mkdir -p ${RPM_BUILD_ROOT}/{sbin,%{_mandir}/man8,etc/rc.d/init.d}
>>> +-make install BUILD_ROOT=${RPM_BUILD_ROOT} MANDIR=%{_mandir}
>>> +-
>>> +-
>>> +-%files
>>> +-%defattr(-,root,root)
>>> +-%doc README
>>> +-%config /etc/rc.d/init.d/ipvsadm
>>> +-/sbin/ipvsadm*
>>> +-%{_mandir}/man8/ipvsadm*
>>> +-
>>> +-%post
>>> +-/sbin/chkconfig --add ipvsadm
>>> +-
>>> +-%preun
>>> +-/sbin/chkconfig --del ipvsadm
>>> +-
>>> +-
>>> +-%clean
>>> +-rm -rf $RPM_BUILD_DIR/%{name}
>>> +-rm -rf $RPM_BUILD_ROOT
>>> +-
>>> +-
>>> +-%changelog
>>> +-* Thu Jun 23 2005 Steve Nielsen <snielsen at comscore.com>
>>> +-- Respect rpmmacros that might be set (by using rpm --eval)
>>> +-
>>> +-* Sat Dec 20 2003 Wensong Zhang <wensong at linux-vs.org>
>>> +-- tidy up the description
>>> +-
>>> +-* Sat Apr  5 2003 Wensong Zhang <wensong at linux-vs.org>
>>> +-- Removed the unnecessary Docdir setting.
>>> +-
>>> +-* Thu Dec 16 2001 Wensong Zhang <wensong at linuxvirtualserver.org>
>>> +-- Changed to install ipvsadm man pages according to the %{_mandir}
>>> +-
>>> +-* Thu Dec 30 2000 Wensong Zhang <wensong at linuxvirtualserver.org>
>>> +-- update the %file section
>>> +-
>>> +-* Thu Dec 17 2000 Wensong Zhang <wensong at linuxvirtualserver.org>
>>> +-- Added a if-condition to keep both new or old rpm utility building
>>> +-  the package happily.
>>> +-
>>> +-* Tue Dec 12 2000 P.Copeland <bryce at redhat.com>
>>> +-- Small modifications to make the compiler happy in RH7 and the Alpha
>>> +-- Fixed the documentation file that got missed off in building
>>> +-  the rpm
>>> +-- Made a number of -pedantic mods though popt will not compile with
>>> +-  -pedantic
>>> +-
>>> +-* Wed Aug 9 2000 Horms <horms at vergenet.net>
>>> +-- Removed Obseletes tag as ipvsadm is back in /sbin where it belongs
>>> +-  as it is more or less analogous to both route and ipchains both of
>>> +-  which reside in /sbin.
>>> +-- Create directory to install init script into. Init scripts won't install
>>> +-  into build directory unless this is done
>>> +-
>>> +-* Thu Jul  6 2000 Wensong Zhang <wensong at linuxvirtualserver.org>
>>> +-- Changed to build rpms on the ipvsadm tar ball directly
>>> +-
>>> +-* Wed Jun 21 2000 P.Copeland <copeland at redhat.com>
>>> +-- fixed silly install permission settings
>>> +-
>>> +-* Mon Jun 19 2000 P.Copeland <copeland at redhat.com>
>>> +-- Added 'dist' and 'rpms' to the Makefile
>>> +-- Added Obsoletes tag since there were early versions
>>> +-  of ipvsadm-*.rpm that installed in /sbin
>>> +-- Obsolete tag was a bit vicious re: piranha
>>> +-
>>> +-* Mon Apr 10 2000 Horms <horms at vergenet.net>
>>> +-- created for version 1.9
>>> +diff -Naur ipvsadm-1.26/libipvs/ip_vs.h ipvsadm-1.26.owrt/libipvs/ip_vs.h
>>> +--- ipvsadm-1.26/libipvs/ip_vs.h       2011-02-07 02:38:57.000000000 +0000
>>> ++++ ipvsadm-1.26.owrt/libipvs/ip_vs.h  2016-01-24 19:59:27.304631247 +0000
>>> +@@ -10,6 +10,7 @@
>>> + #include <sys/socket.h>
>>> + #include <arpa/inet.h>
>>> + #include <linux/types.h>      /* For __beXX types in userland */
>>> ++#include <sys/types.h>
>>> +
>>> + #ifdef LIBIPVS_USE_NL
>>> + #include <netlink/netlink.h>
>>> +diff -Naur ipvsadm-1.26/libipvs/libipvs.c ipvsadm-1.26.owrt/libipvs/libipvs.c
>>> +--- ipvsadm-1.26/libipvs/libipvs.c     2011-02-07 02:38:57.000000000 +0000
>>> ++++ ipvsadm-1.26.owrt/libipvs/libipvs.c        2016-01-24 20:02:49.840430677 +0000
>>> +@@ -32,7 +32,7 @@
>>> + struct ip_vs_getinfo ipvs_info;
>>> +
>>> + #ifdef LIBIPVS_USE_NL
>>> +-static struct nl_handle *sock = NULL;
>>> ++static struct nl_sock *sock = NULL;
>>
>> Am I right that this is for converting from the use of libnl1 to
>> libnl3 style api?
>
> I tried first with libnl but for some odd reasons it gave AF_INET errors and
> thus it could not parse IP addresses correctly.

Okay, I guess only libnl1 is available at the time of ipvsadm-1.26

>
>>
>>> + static int family, try_nl = 1;
>>> + #endif
>>> +
>>> +@@ -73,7 +73,7 @@
>>> + {
>>> +       int err = EINVAL;
>>> +
>>> +-      sock = nl_handle_alloc();
>>> ++      sock = nl_socket_alloc();
>>> +       if (!sock) {
>>> +               nlmsg_free(msg);
>>> +               return -1;
>>> +@@ -88,7 +88,7 @@
>>> +
>>> +       /* To test connections and set the family */
>>> +       if (msg == NULL) {
>>> +-              nl_handle_destroy(sock);
>>> ++              nl_socket_free(sock);
>>> +               sock = NULL;
>>> +               return 0;
>>> +       }
>>> +@@ -104,12 +104,12 @@
>>> +
>>> +       nlmsg_free(msg);
>>> +
>>> +-      nl_handle_destroy(sock);
>>> ++      nl_socket_free(sock);
>>> +
>>> +       return 0;
>>> +
>>> + fail_genl:
>>> +-      nl_handle_destroy(sock);
>>> ++      nl_socket_free(sock);
>>> +       sock = NULL;
>>> +       nlmsg_free(msg);
>>> +       errno = err;
>>> +diff -Naur ipvsadm-1.26/Makefile ipvsadm-1.26.owrt/Makefile
>>> +--- ipvsadm-1.26/Makefile      2011-02-08 00:24:23.000000000 +0000
>>> ++++ ipvsadm-1.26.owrt/Makefile 2016-01-24 12:40:39.427307725 +0000
>>> +@@ -37,18 +37,17 @@
>>> +
>>> + CC            = gcc
>>> + INCLUDE               =
>>> +-SBIN          = $(BUILD_ROOT)/sbin
>>> ++SBIN          = $(DESTDIR)/usr/sbin
>>> + MANDIR                = usr/man
>>> +-MAN           = $(BUILD_ROOT)/$(MANDIR)/man8
>>> +-INIT          = $(BUILD_ROOT)/etc/rc.d/init.d
>>> ++INIT          = $(DESTDIR)/etc/rc.d/init.d
>>> + MKDIR         = mkdir
>>> + INSTALL               = install
>>> + STATIC_LIBS   = libipvs/libipvs.a
>>> +
>>> + ifeq "${ARCH}" "sparc64"
>>> +-    CFLAGS = -Wall -Wunused -Wstrict-prototypes -g -m64 -pipe -mcpu=ultrasparc -mcmodel=medlow
>>> ++    CFLAGS += -Wall -Wunused -Wstrict-prototypes -g -m64 -pipe -mcpu=ultrasparc -mcmodel=medlow
>>> + else
>>> +-    CFLAGS = -Wall -Wunused -Wstrict-prototypes -g
>>> ++    CFLAGS += -Wall -Wunused -Wstrict-prototypes -g
>>> + endif
>>> +
>>> +
>>> +@@ -104,12 +103,6 @@
>>> +               $(INSTALL) -m 0755 ipvsadm $(SBIN)
>>> +               $(INSTALL) -m 0755 ipvsadm-save $(SBIN)
>>> +               $(INSTALL) -m 0755 ipvsadm-restore $(SBIN)
>>> +-              [ -d $(MAN) ] || $(MKDIR) -p $(MAN)
>>> +-              $(INSTALL) -m 0644 ipvsadm.8 $(MAN)
>>> +-              $(INSTALL) -m 0644 ipvsadm-save.8 $(MAN)
>>> +-              $(INSTALL) -m 0644 ipvsadm-restore.8 $(MAN)
>>> +-              [ -d $(INIT) ] || $(MKDIR) -p $(INIT)
>>> +-              $(INSTALL) -m 0755 ipvsadm.sh $(INIT)/ipvsadm
>>> +
>>> + clean:
>>> +               rm -f ipvsadm $(NAME).spec $(NAME)-$(VERSION).tar.gz
>>
>> Another patch can be applied to ipvsadm Makefile to let target
>> `ipvsadm` depend on `$(STATIC_LIBS)`.  This is required for correct
>> description of dependency for parallel build.  You may want to check
>> https://github.com/yousong/build-scripts/blob/master/build-ipvsadm.sh#L35
>>
>>                 yousong
>
> I prefer to keep the modifications of the original Makefile to a minimum.
> It works fine as it is.

It works fine only randomly on `make -j <BIGNUMBER>`.  The problem is
that ipvsadm depends on libipvs.a' being built first, yet the original
Makefile does not specify that dependency.

                yousong

>
> --
> Mauro Mozzarelli
> Phone: +44 7941 727378
> eMail: mauro at ezplanet.net
>
_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel



More information about the openwrt-devel mailing list