#openwrt-devel on 2023-06-13 — irc logs at oftc.irclog.whitequark.org

2023-05-02 04:42 ChanServ changed the topic of #openwrt-devel to: Forum: https://forum.openwrt.org/ | Wiki: http://openwrt.org/ | Release: https://downloads.openwrt.org/releases/22.03.5 | Notes: https://openwrt.org/releases/22.03/notes-22.03.5 | Logs: https://oftc.irclog.whitequark.org/openwrt-devel

00:13 <owrt-2203-builds> Build [#258](https://buildbot.openwrt.org/openwrt-22.03/images/#builders/40/builds/258) of `mvebu/cortexa53` failed.

01:23 danitool has quit [Ping timeout: 480 seconds]

02:00 tSYS has quit [Quit: *squeak*]

02:00 tSYS has joined #openwrt-devel

02:13 grid has joined #openwrt-devel

03:04 dangole has quit [Ping timeout: 480 seconds]

05:17 <KGB-0> https://tests.reproducible-builds.org/openwrt/openwrt_ath79.html has been updated. (98.6% images and 100.0% packages reproducible in our current test framework.)

06:22 <nick[m]1234> Ansuel: I saw that you already tried to fix the gihub_archive download. For me this is broken. If I use in on my host machine and than on a x86 openwrt docker, the hash is just different. I already tried setting different modes. Any suggestions?

06:27 rua has quit [Quit: Leaving.]

06:40 goliath has joined #openwrt-devel

06:40 shoragan has joined #openwrt-devel

06:52 bluew has quit [Ping timeout: 480 seconds]

07:47 ukleinek has joined #openwrt-devel

07:55 ukleinek has quit [Quit: Reconnecting]

07:56 ukleinek has joined #openwrt-devel

08:17 <Ansuel> nick the hash should not be related to the host

08:18 <Ansuel> might be that some mirror have an old tar

08:18 <Ansuel> or github changed the tar settings

08:19 <\x> Ansuel: you now can update ipq60xx boardfile right?

08:19 <\x> regdb update*

08:23 <nick[m]1234> Ansuel: yes. I also thought that. However, when trying to bump the olsrd version (or even just select another pkg_source_date), the download fails. You can even see than in the CI, that download is sucessfull for some targets and for some not.

08:33 <Ansuel> \x maybe

08:33 <Ansuel> nick do you have some link for CI so i can quickly repro it?

08:38 <\x> Ansuel: i can try later tonight, ill send you my MR7350 boardfile, but i want regdb master since a friend updated the proper ones for our country recently.

08:38 <nick[m]1234> Ansuel: I don't have the old ci values. Here is the PR I'm currently working on: https://github.com/openwrt/routing/pull/977

08:39 <Ansuel> \x saddly i won't do regdb mod and i will stick to what qcom provide

08:39 <\x> https://litter.catbox.moe/5graqf.7z

08:40 <\x> oh, i thought its like you use linux regdb

08:40 <nick[m]1234> Ansuel: can you tell me if the PKG_MIRROR_HASH changes when running "make package/olsrd/{clean,download,check} -j1 V=s FIXUP=1" ?

08:40 <\x> https://git.kernel.org/pub/scm/linux/kernel/git/sforshee/wireless-regdb.git/

08:40 <Ansuel> nick are you sure it's a target thing and not just luck ?

08:40 <Ansuel> luck in the sense that a different mirror is selected and that have a different version?

08:41 <Ansuel> i see you also bump the release and the version so it should not be a mirror thing

08:42 <nick[m]1234> you mean a different github mirror? I thought it is now downloaded via archive api from github

08:42 <nick[m]1234> Ansuel: I'm not sure what's the cause of this behavior

08:43 <Ansuel> nick do you have one target that fail and one that works?

08:43 <Ansuel> so i can test this on my local buildroot?

08:43 <nick[m]1234> Ansuel: I thought maybe it has some time race condition whatever. So I looked at the mtime again.

08:46 nixuser has joined #openwrt-devel

08:47 <Ansuel> from CI i can see every test failed with mismatched hash

08:48 <nick[m]1234> Ansuel: yes please test it. I'm very interested if it works at your setup.

08:48 <Ansuel> in my local buildroot it works correctly

08:49 <Ansuel> repacked git produce the same hash

08:49 <nick[m]1234> and FIXUP cmd?

08:49 <Ansuel> check FIXUP=1 ?

08:50 <nick[m]1234> yes

08:50 <nick[m]1234> sha256sum dl/olsrd-2023-06-12-a9b3f1ac.tar.xz d8c80228b4a79fed12d6f967008da1e54ef893e6b252a394597907b2796ef211 dl/olsrd-2023-06-12-a9b3f1ac.tar.xz

08:52 <Ansuel> nope same

08:53 <Ansuel> wtf???

08:56 <Ansuel> b7e1cd4b43a4d8607d6761267f23d28fb09c577d746c22ca60d2ed2fc19b1ea8 is the hash reported by CI

08:56 <Ansuel> and match mine

08:56 <dwfreed> note that github does not produce reproducible tarballs

08:57 <Ansuel> we are not using github api

08:57 <Ansuel> the script just clone the repository

08:57 <Ansuel> strip some file

08:57 <Ansuel> and make a tar out of it

08:57 <Ansuel> and the mirror hash is the tar from it

08:58 <Ansuel> maybe nick have some special option set in git config (global?) that are affecting repository cloned?

09:03 xback has joined #openwrt-devel

09:14 schwicht has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]

09:16 schwicht has joined #openwrt-devel

09:19 <nick[m]1234> I thought we are using the github API to download it via commit archie like : https://github.com/OLSR/olsrd/archive/a9b3f1ac6e73a39b5bd97d1e66b1e039998314f5.tar.gz

09:19 <nick[m]1234> Ansuel: my github conifg looks clean

09:25 PJin[m] has joined #openwrt-devel

09:26 PJin[m] has quit [Quit: Reconnecting]

09:28 PJin[m] has joined #openwrt-devel

09:34 <Ansuel> nick maybe for some reason download.py is failing

09:34 <Ansuel> and you are fallbacking to git-raw

09:34 <Ansuel> ?

09:34 <Ansuel> and maybe that produce different hash?

09:36 schwicht has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]

09:37 <Ansuel> nope even git-raw produce same hash

09:47 <jow> did you compare the tarballs yet?

09:47 <jow> diff or vbindiff -u over the unzipped tar's

09:50 <ukleinek> Ansuel, jow, nick[m]1234: that reminds me of https://github.blog/changelog/2023-01-30-git-archive-checksums-may-change/

09:51 <nick[m]1234> jow: where is the tarball located?

09:53 <jow> nick[m]1234: dl/

09:54 robimarko has joined #openwrt-devel

09:56 <Habbie> if i build something that depends on openssl, i can't avoid building openssl first, right?

09:56 <nick[m]1234> jow: yes I already did a diff between local build and docker build. and I did not find any difference.

09:56 <jow> nick[m]1234: well if the checksums don't match then there must be a diff ;)

09:57 <nick[m]1234> jow: I think the checksums can also be because of metadata of the tar.xz format? Like ownership stuff and packaing time?

09:57 <jow> so you say the uncompressed tars have identical checksums but the compressed ones not?

09:59 <robimarko> Hm, we had a issue with reproducible checksums that the script produced but that was fixed

10:02 <nick[m]1234> jow: https://user-images.githubusercontent.com/10708466/245437844-4becea6a-e501-44a0-b7e6-31ecfca8e589.png

10:02 <nick[m]1234> maybe my diff is wrong

10:03 <jow> nick[m]1234: yes, it is

10:04 <jow> xzcat olsrd-2023-06-12-a9b3f1ac.tar.xz > /tmp/file1.tar

10:04 <jow> xzcat dockerbla/olsrd-2023-06-12-a9b3f1ac.tar.xz > /tmp/file2.tar

10:04 xback has quit [Ping timeout: 480 seconds]

10:04 <jow> sha256sum /tmp/file*.tar

10:04 <jow> diff -u /tmp/file1.tar /tmp/file2.tar

10:04 <jow> if those are identical, then the issue is the .xz compression

10:04 <jow> if not, the issue is with tar packing (file permissions, sorting)

10:05 <nick[m]1234> jow: thanks. they differ https://gist.github.com/PolynomialDivision/d2914419dff83c914f51ad34ef08afb8

10:05 <jow> also tar -tf /tmp/file1.tar > /tmp/file1.list and tar -tf /tmp/file2.tar > /tmp/file2.list

10:05 <jow> diff -u /tmp/file1.list /tmp/file2.list

10:05 <ynezz> Ansuel:

10:05 <ynezz> GitHub Actions has encountered an internal error when running your job.

10:05 <ynezz> have you seen that yet?

10:06 <ynezz> https://github.com/openwrt/openwrt/actions/runs/5248102683

10:06 <jow> nick[m]1234: use diff -a -u ... to force text comparison

10:07 <jow> this should display actual diffs

10:07 <nick[m]1234> the diff doesnot show anything

10:07 <jow> of the list files?

10:08 <nick[m]1234> yes

10:08 <jow> then the issue is in the tar headers, not ordering or exlusions

10:08 <jow> use diff -a -u /tmp/file1.tar /tmp/file2.tar

10:08 <jow> or better, if you can easily install it, use `vbindiff`

10:08 <nick[m]1234> https://gist.github.com/PolynomialDivision/b61518751693eb874e53036f75285c2e

10:09 <jow> nick[m]1234: not the .list files, the .tar files

10:09 <jow> empty diff for list files is good, means both archives contain the same files at least

10:09 <jow> in the same order

10:10 <jow> so the diff must be either in tar headers or even in packaged file contents themselves (but you ruled that out with your recursive diff over the unpacked source trees)

10:10 <jow> so it must be tar headers

10:10 <jow> my bet is on GNU extended header fields

10:10 <nick[m]1234> jow: I'm seeing now a diff

10:10 <jow> or symlink permissions

10:11 <jow> nick[m]1234: can you upload those two .tar files somewhere?

10:12 danitool has joined #openwrt-devel

10:13 <nick[m]1234> jow: https://file.io/oorqKwE957Zc

10:14 <Ansuel> ynezz just lovely github https://www.githubstatus.com/history

10:14 <nick[m]1234> jow: sorry, I have to go afk for a short time

10:14 <Ansuel> nothing we can do about it aside from restarting the job, not our fault

10:15 <ynezz> Ansuel: well, I've tried that already, so maybe should do that harder or rebase/force push...

10:16 <jow> nick[m]1234: "The transfer you requested has been deleted."

10:16 <ukleinek> nick[m]1234, jow: numeric vs. named owners I think

10:17 <ukleinek> jow: huh, my fault, sorry, "Files will be deleted after download"

10:17 <jow> ukleinek: we should pass --numeric-owner to our tar packing routines

10:17 dangole has joined #openwrt-devel

10:17 <jow> ukleinek: can you reupload them, then? :)

10:17 <jow> *already pass

10:18 <robimarko> jow: We already are passing --numeric-owner

10:18 <robimarko> https://github.com/openwrt/openwrt/commit/5f1758ef14575df4e86896526b1c2035c231899e

10:18 <ukleinek> jow: https://paste.debian.net/hidden/ab1cacba/

10:18 <jow> robimarko: I know, bad formulation

10:18 <jow> ukleinek: ahh, sticky bit :)

10:18 <jow> erm brainfart, symlink permissions

10:19 <jow> so one of both repos was packed with a wrong .tar

10:19 <jow> *archives

10:19 <Habbie> any idea why 'make package/h2o/compile' is causing 'package/kernel/linux compile'?

10:19 schwicht has joined #openwrt-devel

10:19 <jow> OpenWrt buildroot uses a patched GNU tar which forces symlink permissions to 0777

10:19 <Habbie> (using the SDK)

10:20 <ukleinek> jow: https://www.kleine-koenig.org/tmp/file[12].tar

10:20 <jow> GNU tar running on systems which support symlink permissions (macOS) will embed the specific permissions like 0644 instead of the expected 0777

10:21 <ukleinek> jow: there is also "root" missing, look at 003c9500

10:22 <jow> also belonging to the symlink header

10:22 <ukleinek> jow: ack

10:22 <ukleinek> so it's permission bits + owner + group I think

10:23 <jow> https://git.openwrt.org/?p=openwrt/openwrt.git;a=blob;f=tools/tar/patches/100-symlink-force-root-name.patch

10:23 <jow> https://git.openwrt.org/?p=openwrt/openwrt.git;a=blob;f=tools/tar/patches/110-symlink-force-permissions.patch

10:24 <jow> I guess nick[m]1234 uses macOS on the host and the archive creator invoked the wrong tar

10:24 <nick[m]1234> Im using arch Linux

10:24 <ukleinek> note that the permission bits are 1160 vs 0777, so "|= 0777" isn't enough

10:25 <ukleinek> nick[m]1234: which file system?

10:25 <owrt-images-builds> Build [#36](https://buildbot.staging.openwrt.org/images/#/builders/30/builds/36) of `master_ath79/generic` failed.

10:27 <jow> 01160 is a weird permission to begin with

10:28 <jow> what is that? sticky + owner executable + group rw + others denied?

10:29 <Ansuel> strange that we have this only now o.o we have many macos user

10:30 <Ansuel> ynezz yes i guess the entire state of the workflow got corrupted so i guess force push is the way

10:32 <jow> ukleinek: actually the difference appears to be the checksum field, not the mode (permission) one

10:32 <Ansuel> ynezz i retriggered the entire job and not only the failed one should be the same

10:38 <ynezz> Ansuel: ok, thanks, lets see

10:41 tlj has joined #openwrt-devel

10:51 wvdakker has joined #openwrt-devel

10:53 torv has quit [Ping timeout: 480 seconds]

11:04 schwicht has quit [Read error: Connection reset by peer]

11:06 torv has joined #openwrt-devel

11:14 <nick[m]1234> ukleinek: ext4

11:15 <nick[m]1234> maybe it builds again my host tar?

11:20 wvdakker has quit [Quit: ZNC 1.8.2+deb3.1 - https://znc.in]

11:31 schwicht has joined #openwrt-devel

11:47 stintel has joined #openwrt-devel

11:48 <stintel> 12|23:35:25 [OFTC] -!- #openwrt-devel Cannot join channel (Need to be identified and verified to join this channel, '/msg NickServ help' to learn how to register and verify.)

11:48 <stintel> how is this possible if I use certfp auth

11:48 <stintel> sigh

12:15 TianlingShen[m] has joined #openwrt-devel

12:27 mrkiko has quit [Remote host closed the connection]

12:44 tidalf has quit [Ping timeout: 480 seconds]

13:08 schwicht has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]

13:24 <stintel> is anyone aware of a tool to generate Kconfig files from e.g. a JSON file?

13:26 schwicht has joined #openwrt-devel

13:43 <f00b4r0> stintel: Perl

13:43 * f00b4r0 runs

13:47 <stintel> welpo

13:48 <nick[m]1234> ukleinek: jow what are you using as buld system?

13:50 enyc has joined #openwrt-devel

13:53 <nick[m]1234> I renamed host my tar binary, so that I'm sure build process is not accessing it. However, still wrong hash... :/

13:58 schwicht has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]

14:17 <Habbie> 10:19Z <Habbie> any idea why 'make package/h2o/compile' is causing 'package/kernel/linux compile'?

14:18 <Habbie> because h2o depends on openssl and OPENSSL_ENGINE is on by default

14:20 schwicht has joined #openwrt-devel

14:22 tidalf has joined #openwrt-devel

14:30 tidalf has quit [Ping timeout: 480 seconds]

14:31 schwicht has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]

14:47 tidalf has joined #openwrt-devel

14:51 <owrt-images-builds> Build [#38](https://buildbot.staging.openwrt.org/images/#/builders/64/builds/38) of `master_bcm47xx/mips74k` completed successfully.

14:53 goliath has quit [Quit: SIGSEGV]

15:10 <Slimey> i got the Adtran Bluesocket BSAP-1930 working using the ws-ap3715i dts as base, what is this simpleboot- stuff?

15:22 <Slimey> heh and sysupgrade tries to write a ubi part when it shouldnt be

15:32 schwicht has joined #openwrt-devel

15:52 goliath has joined #openwrt-devel

16:28 <Slimey> my board is the only one with "mpc85xx,p1010" after the device for this target, is this ok?

16:58 rua has joined #openwrt-devel

17:10 schwicht has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]

17:12 schwicht has joined #openwrt-devel

17:18 Borromini has joined #openwrt-devel

17:22 schwicht has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]

17:23 schwicht has joined #openwrt-devel

17:37 goliath has quit [Quit: SIGSEGV]

17:57 Borromini has quit [Quit: Lost terminal]

18:14 tidalf has quit [Remote host closed the connection]

18:16 tidalf has joined #openwrt-devel

19:08 dangole has quit [Quit: Leaving]

19:54 <schmars[m]> robimarko i think the fix for https://github.com/openwrt/asu/issues/515 needs to be backported to 23.05 - i still have 23.05 imagebuilders failing with libubox/mtd dep error

19:55 <robimarko> schmars[m]: I dont have commit rights

19:56 <schmars[m]> belkin e8450

19:56 <schmars[m]> ah i always though you did :) thanks for the fix though! (cc Ansuel )

19:56 <robimarko> schmars[m]: You need to jog my memory, I only vaguely remember this issue

19:57 <schmars[m]> i don't actually understand the underlying issue, i'm afraid

19:57 <robimarko> I mean, how was I involved in it?

19:58 <schmars[m]> ah yes you authored the fix :) https://github.com/openwrt/openwrt/commit/4747b96287aa3a0f1df5047fc0b2cc1835ff9cc8

19:58 <schmars[m]> no clue how that commit fixes it, but fwiu mtd simply fails to build and is then not available as a dependency of libubox

19:58 <robimarko> schmars[m]: That commit has no relation

19:58 <robimarko> It was probably that buildbots make a new build after that commit

20:00 <schmars[m]> right, that asu issue got me all confused

20:02 <schmars[m]> yeah, issue is that 23.05 still has libubox20220927_2022-09-27-ea560134-1_aarch64_cortex-a53.ipk

20:03 <schmars[m]> and mtd wants libubox20230523

20:10 <schmars[m]> b6e0a24c492537e5bbfa015e2a3638ccc53c164b is what needs backporting to 23.05, i think

20:20 <schmars[m]> feels like dejavu?!

20:22 <Ansuel> i'm not following this?

20:23 <schmars[m]> yeah i just got all confused about someone being wrong about a fix :) sorry about the ping

20:23 <schmars[m]> last 4 lines about libubox still stand though :)

20:31 _lore_ has joined #openwrt-devel

20:38 schwicht has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]

20:41 Tapper has joined #openwrt-devel

20:48 robimarko has quit [Quit: Leaving]

21:23 <Znevna> is anyone looking into this? https://github.com/openwrt/openwrt/issues/11650

21:26 bluew has joined #openwrt-devel

21:27 <Ansuel> chance it's caused by the bridge offload hack patch

21:27 <Ansuel> that is sched to be dropped

21:30 tidalf has quit [Ping timeout: 480 seconds]

21:36 <schmars[m]> jfc i'm so blind :) fix for the libubox thing was committed yesterday, just need to be patient for the builds :)

21:39 tidalf has joined #openwrt-devel

21:50 lucenera has quit [Quit: The Lounge - https://thelounge.chat]

21:51 lucenera has joined #openwrt-devel

21:52 rua has quit [Quit: Leaving.]

22:01 tidalf has quit [Ping timeout: 480 seconds]

22:23 schwicht has joined #openwrt-devel

22:32 <owrt-images-builds> Build [#37](https://buildbot.staging.openwrt.org/images/#/builders/30/builds/37) of `master_ath79/generic` completed successfully.

22:40 Tapper has quit [Ping timeout: 480 seconds]

22:52 schwicht has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]

23:25 Tapper has joined #openwrt-devel

23:43 tidalf has joined #openwrt-devel