download: regression with USE_SOURCE_DIR and non-tarball packages?
luz at plan44.ch
Mon Jun 14 05:11:27 PDT 2021
a while ago (after switching projects from 19.07.3 to 19.07.6) I experienced unexpected rebuilds of packages. I asked about this in the forum  with no echo, so last week I dug deeper and think I found your commit from last november "4e19cbc - download: handle possibly invalid local tarballs"  causing what I see.
I'm a build system newbie, so please apologize if I misunderstand things here. But it seems to me that this patch only works for packages that pull their upstream source from a tarball via download.pl, because only there a hash check prevents redownloads (causing rebuilds) in the last possible moment by exiting the script before the download would happen.
For packages with PKG_SOURCE_PROTO=git the new line in package.mk `$(DL_DIR)/$(FILE): FORCE` forces re-cloning the upstream repo and thus rebuilding the entire package every time. Also, the USE_SOURCE_DIR mechanism does not work any more.
Both git based packages and USE_SOURCE_DIR are central to my work (which is building automation devices based on OpenWrt, so most work is happening *in* my own packages, which are in git or, during hot development phases, via USE_SOURCE_DIR even directly linked dirs). I realize this is not a common situation for OpenWrt platform developers, but only for developers basing other stuff on OpenWrt.
As said, I don't understand the details of the build system enough to actually propose a proper patch right into the heart of it. But below is my current workaround to re-enable USE_SOURCE_DIR and git-based packages, hopefully not affecting anything else I can't see. So far, works for me, but definitely needs a thorough look by someone who actually understands the whole thing...
@@ -177,6 +177,12 @@ define Build/Exports/Default
+ifneq ($(wildcard $(PKG_BUILD_DIR)/.source_dir),)
+ FORCE_DL=$(if $(findstring git,$(PKG_SOURCE_PROTO)),,$(DL_DIR)/$(FILE): FORCE)
@@ -185,7 +191,7 @@ define Build/CoreTargets
- $(DL_DIR)/$(FILE): FORCE
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 488 bytes
Desc: Message signed with OpenPGP
More information about the openwrt-devel