[FS#3494] piped dd > 4096 will occasionally truncate on multi-core systems. Upstream suggests iflag=fullblock

[OpenWrt Bugs]

THIS IS AN AUTOMATED MESSAGE, DO NOT REPLY.

A new Flyspray task has been opened.  Details are below. 

User who did this - John (johnth) 

Attached to Project - OpenWrt/LEDE Project
Summary - piped dd > 4096 will occasionally truncate on multi-core systems. Upstream suggests iflag=fullblock
Task Type - Bug Report
Category - Base system
Status - Unconfirmed
Assigned To - 
Operating System - All
Severity - Medium
Priority - Very Low
Reported Version - Trunk
Due in Version - Undecided
Due Date - Undecided
Details - Found on mikrotik hap ac2 (currently an open PR), where radio calibration data will occasionally be truncated, showing as an incorrect wifi phy macaddr
https://github.com/openwrt/openwrt/pull/3037#issuecomment-734246786

Piped dd for this device introduced with 7557e7f267e845db5a403139c49f2637f9021992 for caldata_from_file, caldata_sysfsload_from_file

Upstream busybox suggests using the iflag=fullblock https://git.busybox.net/busybox/commit/coreutils/dd.c?id=77a6678c427db5ea15d6d0005a579f441277df06
http://lists.busybox.net/pipermail/busybox/2020-November/088320.html
http://lists.busybox.net/pipermail/busybox/2020-November/088326.html

This worked as expected in a test of 1000 tries:

dd if=/sys/firmware/mikrotik/hard_config/wlan_data/data_0 iflag=skip_bytes,fullblock bs=$((0x2f20)) skip=0 count=1 2>/dev/null | sha256sum


Reproduce:


test_caldata_extract() {
	local counter
	local count_good
	local count_bad
	local result
	local result_latest
	local dd_bs

	dd_bs=$((0x2f20))

	dd if=/dev/urandom of=/tmp/wlan_data_0 bs=1M count=1
	result=$(cat /tmp/wlan_data_0 | \
	dd iflag=skip_bytes bs=$dd_bs skip=0 count=1 2>/dev/null | wc -c)
	while [ ! $result -eq $dd_bs ]; do
	echo "bad initial result"
	result=$(cat /tmp/wlan_data_0 | \
	dd iflag=skip_bytes bs=$dd_bs skip=0 count=1 2>/dev/null | wc -c)
	done

	counter=0
	count_good=0
	count_bad=0
	echo "initial run $counter $result"

	while true; do
	counter=$(( counter + 1 ))
	result_latest=$(cat /tmp/wlan_data_0 | \
	dd iflag=skip_bytes bs=$((0x2f20)) skip=0 count=1 2>/dev/null | wc -c)
	if [ "$result_latest" = "$result" ]; then
	count_good=$(( count_good + 1 ))
	#echo "run $counter okay"
	else
	count_bad=$(( count_bad + 1 ))
	echo "run $counter bad $result_latest ($count_bad / $counter)"
	#break
	fi
	done
}

test_caldata_extract



More information can be found at the following URL:
https://bugs.openwrt.org/index.php?do=details&task_id=3494

You are receiving this message because you have requested it from the Flyspray bugtracking system.  If you did not expect this message or don't want to receive mails in future, you can change your notification settings at the URL shown above.