mcbridematt has quit [Read error: Connection reset by peer]
ynezz has quit [Ping timeout: 480 seconds]
ynezz has joined #openwrt-devel
robimarko has joined #openwrt-devel
<mrkiko>
rmilecki: not sure as well there is an easy way out (sysupgrae issue); you may define a list of mountpoints and bind them inside the overlay itself. Not the most elegant solution but doable.Of course you shouldbe sure no process keeps any file open there, otherwise things get messy really fast
jeff___m has joined #openwrt-devel
jeff___m has quit [Ping timeout: 480 seconds]
<rmilecki>
mrkiko: finding all mounts, redoing them, that will be some extra shell code
<rmilecki>
(that answer isn't 100% clear to me, mabe because of language)
<rmilecki>
(but it sounds like we can't get it work the way we need)
<mrkiko>
rmilecki: I do read it that way as well. And I don't like the bind solution either.
<mrkiko>
rmilecki: but have no other ideas at the moment
<rmilecki>
mrkiko: what's wrong with bind mounts?
<rmilecki>
i'm not familiar with that except of understanding basic concept
goetz has joined #openwrt-devel
<rmilecki>
what i'm afraid of is that won't result in overlaying anymore
<rmilecki>
if I mount /tmp/ to /tmp/sysupgrade.ABCDE/backup/tmp/ then writing anything to /tmp/sysupgrade.ABCDE/backup/tmp/ would affect real tmpfs content
goetz has quit [Quit: goetz]
goetz has joined #openwrt-devel
goetz is now known as Guest532
goetz has joined #openwrt-devel
Guest532 has quit [Quit: Guest532]
xback has quit [Ping timeout: 480 seconds]
xback has joined #openwrt-devel
Piraty has quit []
<mrkiko>
rmilecki: infact
<mrkiko>
or - create an overlay for each of these mounts? Really crazy :D
goetz has quit [Quit: goetz]
goetz has joined #openwrt-devel
<mrkiko>
rmilecki: and then - what kind of processes to we expect to run inside that virtual root? Do we really need fullaccess to all mounts? Or may selected ones be enough?
Borromini has joined #openwrt-devel
Jimbot_ has joined #openwrt-devel
<rmilecki>
mrkiko: well, it's only for the backup purpose
<rmilecki>
including /var/ files in backup is likely
<rmilecki>
well, it's proven
<rmilecki>
i'm not sure if any other mount points are likely to be used, i really don't expect anyone to be putting /procd/ or /dev/ content in backup
<mrkiko>
rmilecki: and implmenting this in general terms is going to become too complex, there are crazy setups around :D Maybe just emit it as warning in sysupgrade - you can't backup things coming from mounts. end of the story
<rmilecki>
mrkiko: but people seem to backup content from /var/ for some reason :/
<rmilecki>
e9hack wanted to backup /var/dhcp.leases + /var/lib/logrotate.status + /var/log/logrotate.log
<mrkiko>
rmilecki: seen that in ML. My impression is - if you choose to support this use-case, you go very higher in complexity, because even the "create an overlay for each mount" solution seems pretty complex to me. And assuming this all works well with more "exotic" setups like - involving fuse
<rmilecki>
i agree totally with all those mounts
<rmilecki>
that will become a nightmare for shell script
<mrkiko>
well, I think it would be hard to implement this in a robust fashion even without the shell scripting :)
<rmilecki>
tbh I think i'd be simpler to implement tar --append and archive from multiple directories
<mrkiko>
yes, but I am guessing there is a good reason you didn't do it that way
<rmilecki>
busybox's tar doesn't support --amend
<rmilecki>
i didn't feel need to understand tar format and implement --apend support in busybo
<mrkiko>
rmilecki: I would just try to detect if a given file is in a mountpoint and error out at the beginning
<mrkiko>
sorry, I know it's not a nice anaswer
<mrkiko>
I see it as the acceptabletradeof at the moment
<mrkiko>
this could also be a stopgap measure until something better comes out
<rmilecki>
mrkiko: so disallow backup including /boot/ and/or /var/ content/
<rmilecki>
?
<stintel>
what's wrong with backing up content from /var?
<mrkiko>
rmilecki: disallowing backup containing files located in different filesystems (on a mountpoint). but I was forgetting about extroot for example
<rmilecki>
stintel: i think it's a case we need to support, that's why I asked mrkiko to clarify
<mrkiko>
stintel: I agree with rmilecki but in my opinion it gets too complex too fast, so was thinking to now allow that for the tim being and maybe think about it
<stintel>
we should probably consider implementing /run as tmpfs like every other distro nuke the /var symlink to /tmp/var (or what was it), and move all default locations to /run
<stintel>
people can then decide they want something to survive reboot by putting it in /var
<stintel>
I tried this many years ago
<stintel>
but all the code in procd just made my head explode and I gave up
<rmilecki>
it's hard to believe a simple backup creation is so complex ;)
<stintel>
ahh well I just build with CONFIG_TARGET_ROOTFS_PERSIST_VAR=y
Mangix has quit [Read error: Connection reset by peer]
<Borromini>
stintel: pong :^)
goetz is now known as Guest541
goetz has joined #openwrt-devel
Guest541 has quit [Ping timeout: 480 seconds]
Tapper has joined #openwrt-devel
* russell--
is trying to completely disable IPV6 for a particular application, but make defconfig is thwarting him
<russell-->
modifying config/Config-build.in seems to do it for me
Borromini has quit [Ping timeout: 480 seconds]
goetz has quit [Quit: goetz]
goetz has joined #openwrt-devel
<jow>
rmilecki: that's why all those seemingly simple things aren't done
<jow>
rmilecki: if they'd be featureful enough to be simple we'd be looking at a full Distro
<rmilecki>
jow: any idea how to solve that backup problem?
<rmilecki>
i can't think of anything
<jow>
rmilecki: tbh I don't even understand the problem to begin with
<jow>
didn't pay attention to waht was changed and why, and how it broke things
<rmilecki>
jow: do you care for a short explanation?
<jow>
sure
<rmilecki>
jow: we need to include extra files in backup that don't exist in filesystem
goetz has quit [Remote host closed the connection]
<rmilecki>
i decided to handle that by creating a "virtual" temp root filesystem using overlay (lower: / and upper: /tmp/sth/upper)
<rmilecki>
i mounted that to /tmp/sth/backup
<rmilecki>
that way I:
<rmilecki>
1. have a "copy" of real filesystem in /tmp/sth/backup
<rmilecki>
2. i can create extra files in /tmp/sth/backup that are not written to flash
<rmilecki>
it seemed almost perfect
<rmilecki>
i could treat /tmp/sth/backup as a playground for creating backups, I could see real filesystem content and create custom files for backup needs only
<rmilecki>
the problem is mounts don't "propagate"
<rmilecki>
there is nothing in /tmp/sth/backup/tmp
<rmilecki>
there is nothing in /tmp/sth/backup/var/ etc.
<rmilecki>
sorry, i've to run now, get to family for a moment, if that was enough to be some problem explanation, any ideas are welcom
<rmilecki>
ah, just to be clear: i need /tmp/sth/backup/var/ to work because some people want to backup /var/dhcp.leases + /var/lib/logrotate.status + whatever
<jow>
rmilecki: the easiest solution is to roll back your changes and minipulate the tar archive directly
<jow>
*manipulate
<jow>
GNU tar has an --append mode but busybox tar is too limited
<jow>
a plain .tar archive has a very simple format though, the last 1024 byte are all zero
<jow>
and an archive member is just a fixed size header block followed by 512 byte blocksize padded file data
<jow>
so I would propose to
<jow>
1) generate a plain tar consisting of actual files as it was done in the past (means revertzing latest changes)
<jow>
2) generate the .tar on the fly and pipe it to stdout to avoid wasting disk space
<jow>
3) read the stdout tar stream using a loop in chunks of 512 byte
<jow>
4) always buffer two blocks, means when you read the 3rd block, write the 1st to output, when reading the 4th, write the 2nd etc.
<jow>
5) on eof, statically print a tar member header and the desired file data, pad to 512 byte
<jow>
6) print 1024 * '\0'
<jow>
that read-print loop in turn can be piped to gzip and the gzip output piped to the final file
<jow>
now you've got a compressed .tar file starting with static backup data and dynamic data appended
<jow>
actually, since .tar has no file wide header but directly starts with the first member header it's even easier
sa7mfo has quit [Read error: Connection reset by peer]
<Borromini>
stintel: okay, was worth a shot.
<rmilecki>
nbd: could you take my patch for the next merge request, please? [PATCH] dt-bindings: net: wireless: mt76: allow all 4 interrupts for MT7981