<Ansuel>
stintel just to make sure tools were also built with AUTOREMOVE right? also probably you are missing some directory to update the timestamp.. wasted too much time on that thing so i'm really sure what i wrong if tools are rebuilt when moved on a different buildroot
<stintel>
Ansuel: ahhh
<stintel>
Ansuel: right, didn't build tools with AUTOREMOVE
srslypascal has quit [Remote host closed the connection]
srslypascal has joined #openwrt-devel
mcbridematt has quit [Remote host closed the connection]
srslypascal has quit [Remote host closed the connection]
srslypascal has joined #openwrt-devel
mcbridematt has joined #openwrt-devel
SlimeyX_ has joined #openwrt-devel
SlimeyX__ has joined #openwrt-devel
robimarko has joined #openwrt-devel
SlimeyX has quit [Ping timeout: 480 seconds]
SlimeyX__ is now known as SlimeyX
SlimeyX_ has quit [Ping timeout: 480 seconds]
srslypascal has quit [Remote host closed the connection]
srslypascal has joined #openwrt-devel
goliath has joined #openwrt-devel
<rmilecki>
anyone familiar with tplink-safeloader.c ?
<rmilecki>
i'm trying to understand "firmware" partitions
<rmilecki>
do we (for OpenWrt) merge kernel with rootfs and store them in just one "firmware" partition?
<rmilecki>
then OpenWrt is expeced to magically split that into kernel and rootfs without any clear info about correct offsets?!?!
MaxSoniX has joined #openwrt-devel
csrf has joined #openwrt-devel
<robimarko>
Isnt that used only by the bootloader?
<robimarko>
And then OpenWrt uses the SquashFS splitter to dynamically find the kernel and rootfs and create those partitions on the fly?
rua has quit [Quit: Leaving.]
rua has joined #openwrt-devel
<rmilecki>
robimarko: uh, it's kind of stupid
<rmilecki>
robimarko: we know exact offsets when creating firmware image
<rmilecki>
and then we hide offsets
<rmilecki>
and tell openwrt to find kernel and rootfs
<robimarko>
Yeah, that is what we have been doing for a long time
danitool has joined #openwrt-devel
Atomicly- has joined #openwrt-devel
AtomiclyCursed has quit [Ping timeout: 480 seconds]
Atomicly- is now known as AtomiclyCursed
<rmilecki>
robimarko: thanks for confirming my thoughts anyway
<rmilecki>
why is it i keep finding OpenWrt ugly hacks every time I want to fix something ;)
<robimarko>
Well, there are years and years of hacks
<robimarko>
Especially those for earlier targets
<rmilecki>
hm, i'm looking again at tplink-safeloader.c and it seems to actualy replace "firmware" with "os-imag" and to insert "file-system" right after it
<rmilecki>
so... it looks sane after all I guess?
<rmilecki>
but then how OpenWrt mtd split is supposed to work
<rmilecki>
i don't get it
<f00b4r0>
rmilecki: I'm not familiar with this particular platform but depending on the image format the splitters don't necessarily need to walk the flash to find rootfs
<f00b4r0>
in fact looking at mtdsplit_tplink.c, it doesn't, except as a failover
<rmilecki>
ok, I booted my old TP-Link Archer C59 v1 and I understand it now
<f00b4r0>
because the kernel size and the rootfs size may change from flash to flash
<f00b4r0>
and it's a lookup that is most of the time simply a header load
<f00b4r0>
takes no time at all, happens once during boot
<f00b4r0>
in exchange for total flexibility for flashing whatever kernel/rootfs sizes we want
<rmilecki>
i don't agree ;/
<f00b4r0>
I'm not quite sure where you're getting at, we moved away from the nightmare of fixed splits eons ago
<rmilecki>
there is no gain in that
<f00b4r0>
of course there is
<f00b4r0>
don't you remember every time a device broke because the kernel was too big to fit in the designated "kernel" partition?
<rmilecki>
you're stuck in a nightmare of fixed partitions
<f00b4r0>
I mean no disrespect but you sound like 5 years ago :)
<f00b4r0>
well isn't that exactly what you suggest we go back to?
<f00b4r0>
or am i misunderstanding?
<rmilecki>
i want to totally drop fixed partitions
<f00b4r0>
what do you suggest instead?
<rmilecki>
i did that for bcm53xx so we can bring back TP-Link devices support
<mrkiko>
f00b4r0: rmilecki: I would suggest taking a pause,and maybe rmilecki can explain things from his perspective completely, then we can go on?
<f00b4r0>
i'm listening :)
<rmilecki>
there is partition table
<f00b4r0>
(or well, reading :)
<rmilecki>
it DOES contain entries os-image and file-system
<mrkiko>
f00b4r0: :D
<rmilecki>
(i thought it doesn't initially)
<rmilecki>
i wrote a parsers for SafeLoader
<rmilecki>
*parser
<mrkiko>
rmilecki: so, you're saying it makes no sense to re-do the work if the tp-link partition table gives us the information we need?
<f00b4r0>
partition table == image header or something else?
<rmilecki>
no, actual partitions table
<rmilecki>
give me a sec
<rmilecki>
i'll be back in 3-5 minutes with a patch
<rmilecki>
hm, the problem is OpenWrt may require "firmware" partition for sysupgrade
<rmilecki>
i didn't think about that
<f00b4r0>
that's exactly the point :)
<mrkiko>
rmilecki: and, I am under the impression in some cases OpenWrt choose a slightly different arrangement than the one provided in the TP-Linkptn structure, if I am understanding you correctly.
<rmilecki>
sorry, this situation was changing for me from minute to minute and maybe my all ideas got too messy
<rmilecki>
starting point: i didn't know tplink-safeloader.c can generate "os-image" and "file-system" dynamically
<rmilecki>
i didn't think about such solution
<rmilecki>
so I wrote a kernel parser to read actual partitions table
<rmilecki>
so I can adjust os-image and file-system sizes in tplink-safeloader.c without breaking kernel
<rmilecki>
and I think it's still better than fixed-partitions
<rmilecki>
the only problem I see is that "partition-table" doesn't define "firmware" partition which may be very useful for sysupgrade
<rmilecki>
and that seems to break my whole idea
<rmilecki>
i'm wondering if we could define "firmware" partition in tplink-safeloader.c so OpenWrt can sysupgrade easily
<f00b4r0>
rmilecki: what I don't get is the difference between the kernel reading a partition table and the kernel reading a tplink image header that provides the same information?
<f00b4r0>
(as mtdspilt_tplink.c apparently currently does, if I read the code correctly)
<rmilecki>
I thought reading partitions table is cleaner (and just in theory more reliable)
<f00b4r0>
oh
<f00b4r0>
it's that kind of discussion we're having :D
<rmilecki>
you could add/remove some extra partitions
<f00b4r0>
are these flash devices we're talking about,
<rmilecki>
other partitions could get reorganized
<f00b4r0>
?
<f00b4r0>
because the whole concept of partitions on flash devices seems rather abstract to me
<rmilecki>
"are these flash devices we're talking about" - not sure if I understand
<f00b4r0>
well, spi nor, spi nand?
<f00b4r0>
16-256MB
<rmilecki>
er, I still don't understand ;)
<f00b4r0>
ok
<f00b4r0>
nevermind then :)
<rmilecki>
"partition-table" content should always match current flash device
<rmilecki>
i'm quite sure it deos
<rmilecki>
*does
<mrkiko>
rmilecki: on a tangential note - I looked at the partition table back in the days with the C60. Not clear to me actually how things work here in the tp-link world. Who does manage the partition table in the original firmware? I guess the software updater modifies it based on the image headers information? And - am I wrong in suspecting that in the end the idea of the layout in the partition table
<mrkiko>
doesn't match the one in openwrt at least in some cases? For some cases it's not relevant like for the parts describing the first-stage u-boot and the second-stage one in e.g.: an Archer C60. But in some other areas?
<rmilecki>
mrkiko: i don't really know how TP-Link handles that in their software
<russell-->
well, this is weird: "This feature depends on another which has been configured as a module. As a result, this feature will be built as a module." for wpad-basic-wolfssl, but .config doesn't have anything =m afaict
<mrkiko>
rmilecki: ok, but once openwrt is installed, where does that partition table get used? By which components?
<jow>
sysupgrade for example
<jow>
it wants a continuous "firmware" partition spanning kernel + rootfs to write an image onto
<mrkiko>
jow: but there are partitions we define in our DTS. I was wondering what uses the stored-in-flash partition table from tp-link itself. Or am I misunderstanding your answer?
<rmilecki>
mrkiko: actual partitions table is read by OpenWrt
<jow>
I probably missed context
<rmilecki>
I wanted to make it do so
<rmilecki>
jow: "firmware" partitions comes from DT file (it's harcoded there), not from "partitions-table" flash partiton (it's absent there)
<f00b4r0>
rmilecki: if it ain't broke...
<mrkiko>
rmilecki: ok, but as things are now, openwrt ignores it I guess. I was curious to know what parts of the system use it. U-Boot surely does if my memory serves me well
<rmilecki>
mrkiko: U-Boot should use it to know where kernel is
<rmilecki>
mrkiko: it's a good question how U-Boot handles it
<rmilecki>
i should check that in some free moment
<mrkiko>
thanks to all of you guys for the always instructive and nice chat
<rmilecki>
i probably should work on better describing problems on IRC
<rmilecki>
i usually take a good care of that in my patche
<rmilecki>
here i was rather chaotic as I was just learning things ;)
<f00b4r0>
;)
<rmilecki>
ok, this exaplains my idea a bit better I hope:
<mrkiko>
rmilecki: well, my opinion is that IRC should be more comunicative, helpful and overall calm. And having well-formed and finalized ideas shouldn't be mandatory, since communication is a process we build togeter. And we are IRC. :) This of course reflects only my own opinion and defects.
<robimarko>
Agreed, whole point of IRC is discussion
<robimarko>
And bouncing some ideas
<rmilecki>
thanks
<rmilecki>
does ath79 actually have full sysupgrade for SafeLoader?
<rmilecki>
i believe during sysupgrade we need to extract partitions from image
<rmilecki>
and flash them in right flash spaces
<rmilecki>
so for example if flash image with different os-image and file-system, that updated partition-table gets written to the flash
xes_ has quit []
bluew has quit [Ping timeout: 480 seconds]
<russell-->
rmilecki: on this ath79 tplink wdr3600 i'm hacking on right now, the firmware is written in one continuous chunk, no extraction of parts during sysupgrade afaik
<rmilecki>
it extracts os-image & file-system and flashes them
xes has joined #openwrt-devel
srslypascal is now known as Guest1072
srslypascal has joined #openwrt-devel
Guest1072 has quit [Ping timeout: 480 seconds]
gladiac has joined #openwrt-devel
<f00b4r0>
rmilecki: one downside I suspect from your scheme is that it forces you to have rootfs EB-aligned, whereas there is presumably no such requirement when flashing a continuous "firmware"
Tapper has joined #openwrt-devel
<linusw>
rmilecki: hey you backported the new device trees to OpenWrt in no time! :D thanks!
<linusw>
rmilecki: I have some patches to generate DIR-890L and DWL-8610AP images but need to test them first.
<linusw>
the seama signature makes me nervous because it seems like something the CFE could be checking to be correct... I have a signature that DD-WRT was using though so going with that.
danitool has quit [Quit: Cubum autem in duos cubos, aut quadratoquadratum in duos quadratoquadratos]
Ansuel has joined #openwrt-devel
<Ansuel>
stintel did you check my email ?
<stintel>
Ansuel: I saw it, didn't get to it yet
<stintel>
Ansuel: I should probably first understand fully how this mechanism works
<Ansuel>
my idea at times was that COPY doesn't preserve some stuff and that makes all the makefiles angry but WAIT let me check something...
<Ansuel>
OHHHHHHHHHHHHHHHHH RIGHT I'M STUPID
<Ansuel>
stiintel the dl dir!!!!!!!!
<Ansuel>
openwrt recompile stuff if the package in dl is not present
<stintel>
arf
Tapper has quit [Ping timeout: 480 seconds]
<Ansuel>
we shoudl consider adding a skip for that if we really want to not bloat the container
<stintel>
yeah
<Ansuel>
anyway while at it i would suggest to include in the COPY command the dl dir to test your idea
<\x>
Ansuel: what does ipq806x do with sqm now, like nss-codel, codel and cake?
<\x>
performance wise
<stintel>
Ansuel: will do, thanks again!
<soxrok2212>
Ansuel: i dumped the fw for that tp-link
<Ansuel>
fixing that later is very easy as you will have to just delete dl dir from the dockerfile
<soxrok2212>
their support channel basically said f-off about firmware and gpl
<Ansuel>
\x nss-codel is not supported as that require all the nss infra so only codel or cake
<stintel>
Ansuel: yeah, for now adding the dl/ dir is a good test
<Ansuel>
soxrok2212 it's strange having tp link not following gpl rules still waiting for half treatning gpl mail with noticing tham that it's againt gpl rules and eu law to not comply with them EHEHEH
<\x>
Ansuel: ah, well theres something in the forum about that, just forget it then. How about sqm codel and cake?
torv has joined #openwrt-devel
<Ansuel>
\x afaik codel can handle more traffic and there is even an improved version but cake better handle bufferbloat (but can handle lower speed)
<Ansuel>
but my idea is that they are all blocked by an uboot env special variable
<\x>
i forgot what you call it, the one where you send a file over the serial uart
<\x>
Ansuel: well yeah, what I meant is that, can you give me numbers on performance?
<Ansuel>
soxrok2212 did you test if failsafe works ? LOL
<\x>
i remember mt7621 has this sbl mode where you send it another uboot
<\x>
>inb4 the only way to install something custom on this is to trigger that and replace uboot
<Ansuel>
oh wow /bin is full of bloat from other oem
<Ansuel>
there is even a readme in chinese ??????
<soxrok2212>
failsafe, nope
<soxrok2212>
\x: probably ATF you're thinking of
<\x>
yeah whatever that is
<soxrok2212>
arm trusted firmware
<soxrok2212>
Ansuel: failsafe is blocked
<\x>
different thing then, theres that thing Borromini has done some time a go to recover a device that had uboot fail, he sent another uboot via minicom
<\x>
then he booted from that and rewrote the uboot region
<Ansuel>
then yes from the dts info we can totally extract SBL2
<soxrok2212>
dts is on the forum post
<Ansuel>
partition@0 {
<Ansuel>
reg = <0x00 0x100000>;
<Ansuel>
label = "BL2";
<Ansuel>
read-only;
<Ansuel>
};
<soxrok2212>
does the dts contain the offset from the start of UBI or from the 0x0 on the chip
<Ansuel>
chip
<soxrok2212>
ok so first 0x1000000 is u-boot
<soxrok2212>
0x100000*
lynxis has quit [Remote host closed the connection]
lynxis has joined #openwrt-devel
<soxrok2212>
oh no wouldnt u-boot be in fip
guidosarducci_ has quit [Remote host closed the connection]
guidosarducci has joined #openwrt-devel
<Ansuel>
well yep that nand dump doesn't seems to have uboot...
<soxrok2212>
O.o
<soxrok2212>
ill dump it again. i had to hold the clip on with my hand, maybe it came loose
<Ansuel>
or it's at an offset and I'm missing that
<soxrok2212>
ive been looking for it too and cant find it
<stintel>
Ansuel: there appear to be some broken symlinks pointing to things in staging_dir/target-..., which is to be expected as I only copy {build,staging}_dir/host
<stintel>
Ansuel: but pretty sure you will have the same problem with a tarball
<Ansuel>
but why host tools have links to targets ?
<stintel>
yeah good question
<stintel>
it doesn't seem right
<Ansuel>
(they are probably regenerated on target install IMHO)
<Ansuel>
stintel the migration pahse will be nightmare as everything will fail but whatever
<aiyion>
I've got the bootloader, serial console, lan and wan, 2.4GHz wifi, reset button as well as the gpios for LEDs working. But I cannot find the correct settings for the LTE modem.
soxrok2212 has quit [Read error: Connection reset by peer]
<aiyion>
The stockfirmware is some mediatek dirt, but comes with a flattened device tree. Is it feasible to extract it, or even useful?
soxrok2212 has joined #openwrt-devel
<aiyion>
I think I extracted some dtbs a while ago
<aiyion>
but no fdts yet
<vulpes2[m]>
can anyone give me a reference for the `0xDEADC0DE` padding at the end of a typical openwrt image?
<vulpes2[m]>
which part of the build system is responsible for generating the metadata in that padding?
Borromini has quit [Ping timeout: 480 seconds]
Borromini has joined #openwrt-devel
<Ansuel>
stintel very sad github ci blacklist some char in path so i have to package the prebuilt tools anyway to pass them to the upload job
<Ansuel>
still need to test if the container would work tho
lmore377 has quit [Ping timeout: 480 seconds]
<aiyion>
the answer to my question above: you can inspect a firmware dump using binwalk to find the offsets and sizes of the flattened device tree and can axtract it using dd. if successful you gain a binary, which ile`can identify as dts-blob.
lmore377 has joined #openwrt-devel
<soxrok2212>
^ i did this about 6 hours ago
<aiyion>
calling dtc -O dts <dtsblob> outputs the dts, I was lookig for.
<aiyion>
In my case that thing appears to be completely useless and utterly generic.
<aiyion>
it describes three more buttons than this device has...