<Begasus_32>
Re-using download '/boot/system/packages/administrative/state_2022-01-02_18:28:18/poppler_data-0.4.11-1-any.hpkg' from previous transaction (partial)
<Begasus_32>
*** Failed to download package poppler_data: Interrupted system call
<Begasus>
This is getting annoying :P
<humdinger>
gotta run
* humdinger
waves
humdinger has quit [Quit: Vision[]: Oi with the poodles already!!]
DKnoto has quit [Quit: Leaving]
jmairboeck has joined #haiku
DKnoto has joined #haiku
Gues__________________________ has joined #haiku
Gues__________________________ is now known as anuragpd4
<anuragpd4>
Hello. I'm trying to build haiku on an M1 mac and I'm facing some issues. Can anyone help me?
gouchi has joined #haiku
AlaskanEmily has quit [Remote host closed the connection]
<PulkoMandy>
hi, what issues do you get exactly? also, are you running MacOS? It may not be the best supported setup yet, but let's see if we can fix problems :)
gouchi has quit [Remote host closed the connection]
gouchi has joined #haiku
<anuragpd4>
PulkoMandy Yes, I'm currently running MacOS. I've created a case-sensitive volume and cloned haiku and buildtools in it. But I'm unable to build x86 compiler toolset. It is failing with some errors.
<anuragpd4>
I've two versions of 'less' in my path and I'm not sure which of them is the GNU version. I doubt this is happening because of either 'bison' or 'less'. I'm not sure if I've configured them properly.
n0toose has joined #haiku
vdamewood has joined #haiku
<PulkoMandy>
can you share a copy of the output of the buildprocess? also, are you trying to build the ARM or the x86 version of Haiku? The ARM version is quite experimental at this point
<PulkoMandy>
x512: yes, userlandfs exposes a filesystem like any other filesystem (mounted anywhere you want), and then you can open files in that filesystem and all the calls will be forwarded to your code running in userland in userlandfs_server
<PulkoMandy>
n0toose: which repository do you mean? haikuports is never frozen, it just switches its builders to the latest release usually a short while before it becomes available
<n0toose>
i mean, say, beta1 alpha1
<n0toose>
or, rather, alpha4
<n0toose>
i can only download software that were available for alpha4 up to that point before the switch, right?
<x512[m]>
Does it require handlers to be loaded in userlandfs_server? Is it possible to create anonymous FD?
tqh has joined #haiku
<PulkoMandy>
x512: I think oyu do whatever you want in your open() call. So you have to start with a path passed to open(), and allocate an inode number to it, probably) but then you do whatever you want with it. So if another call to open() with the same path returns a different fd, essentially you have anonymous FS
<PulkoMandy>
FDs*
<PulkoMandy>
n0toose: yes. Previous alphas and betas don't remain supported after a new one is released. I think this should be different, maybe, for R1 vs R2, but, we are not there yet
<PulkoMandy>
(and I think it probably will stay the same for R1 vs R1.1, R1.2, etc. Just have people run the latest release when it becomes available, and support for the previous one stops)
<x512[m]>
What is the purpose of userlandfs_server? Can't arbitrary process be a userlandfs protocol server?
n0toose has quit [Remote host closed the connection]
<PulkoMandy>
maybe, I think the main reason it's there is to establish the initial connection. The mount command knows how to start it with appropriate arguments, and this keeps the command line args you have to pass to mount to set this up reasonably short
<PulkoMandy>
if you work on something not meant to be manually mounted by users, you can possibly have a more complicated setup that you handle yourself?
<x512[m]>
I want to expose device and dma_buf FDs from RadeonGfx GPU userland driver. It do no need to be manually mounted.
<PulkoMandy>
anuragpd4: you are trying to build the 32bit version of Haiku which uses gcc2. I don't expect this to work on M1 and possibly not even on macOS at all. Do you really need the 32bit version or would 64bit be OK? In which case you can try instead: ./configure --cross-tools-source ../buildtools --build-cross-tools x86_64
<PulkoMandy>
it will probably save you at least some of the problems with getting things to build
<anuragpd4_>
PulkoMandy: I've tried building the 64bit version too but that too is failing. The build process output for 64bit version is very verbose. It is filled with compiler warnings and errors. Is there something basic that I could be missing? The output file is huge(33MB): https://drive.google.com/file/d/1dwaCVqrUmvTi4hEAwWpEGFWSosYJarhE/view?usp=sharing
mmu_man has joined #haiku
<PulkoMandy>
I think it's just that this configuration does not get enough testing. It seems gcc does not want to be compiled on macOS? I'll see if I can find something in gcc sources
<anuragpd4_>
Thanks a lot for your time. Is there any other way that I can build haiku? I'm a student looking to apply for GSoC this year through haiku.
<PulkoMandy>
you can try building from inside Haiku running in a virtual machine, or from Linux
HaikuUser has joined #haiku
<PulkoMandy>
I am looking at gcc code now, it seems there is a missing include in a "darwin" file (this is how they name macOS)
HaikuUser has quit []
<PulkoMandy>
you can try to edit buildtools/gcc/gcc/config/host-darwin.c and add a #include "hosthooks.h" and see if the build goes further
<PulkoMandy>
it looks like this could be a problem in how we imported sources from gcc in our git repository... I will investigate further...
<PulkoMandy>
if it still doesn't work with hosthooks.h, try to also add #include "hosthooks-def.h"
<PulkoMandy>
and if that makes it work, you can submit the change to our code review system in review.haiku-os.org :) let us know if you need help with setting that up
<anuragpd4_>
I'm trying to apply that and I'll let you know if it works
tbcr has quit [Ping timeout: 480 seconds]
nhtello has joined #haiku
nhtello has quit [Quit: Vision[]: i've been blurred!]
HaikuUser has joined #haiku
HaikuUser has quit []
Jupp_S has quit [Remote host closed the connection]
<anuragpd4_>
PulkoMandy: Including both of the header files did the work. binutils and gcc are successfully built. I'll go through the guidelines for submitting patches and submit the changes.
<PulkoMandy>
great, thanks :)
Jupp_S has joined #haiku
<anuragpd4_>
I tried building haiku anyboot images and it failed. Please check it out: https://pastebin.com/kxFWiQw6
<PulkoMandy>
for the first error you need to install development headers for the zstd library
<PulkoMandy>
for the second error it looks like the configure script didn't detect that you are running macOS? can you check the output of the "uname" command (it should say "Darwin")
<PulkoMandy>
and also check the variable HOST_PLATFORM in generated/build/BuildConfig (generated/ being the directory where you ran the configure script), it should be "darwin"
<nekobot>
[haiku/haiku] 071a83af4f2e - fix macOS hosted build after hrev55802
<PulkoMandy>
this commit should fix the problem ^
<anuragpd4_>
I'll try it and update here.
tqh has quit [Remote host closed the connection]
tqh has joined #haiku
piku has quit []
<anuragpd4_>
I'm unable to fix the first error. I've followed https://www.haiku-os.org/guides/building/pre-reqs and installed zstd via brew. Still it says 'zstd.h' file not found. How can I fix this?
<PulkoMandy>
do you know where zstd.h is installed? in the include path passed on the command line I see -I /opt/local/include -I /usr/local/include, is it there?
<anuragpd4_>
no it is not there
<anuragpd4_>
i tried manually copying zstd.h to /usr/local/include. Now I'm getting linker command failed error https://pastebin.com/xf2NMu2M
<tqh>
you need to have zstd on your build machine as well IIRC.
<PulkoMandy>
well it looks like zstd was not installed, or not installed in a place where your compiler knows how to find it, at least
<tqh>
Maybe we should just remove zstd? It's causing more problem than needed..
<PulkoMandy>
what would you replace it with? go back to zlib and force all haiku users to have larger and slower hpkg files?
<tqh>
arn't we using lz4 already?
<PulkoMandy>
we only support zlib and zstd for hpkg files currently
<tqh>
If we use cloudflare's optimized zlib we should be on par with zstd.
<PulkoMandy>
cloudflare zlib seems to be about using even more SIMD and other modern instructions that may not be available everywhere
<PulkoMandy>
it looks like in the case of porting to new architectures, this is only going to complicate things
<PulkoMandy>
"The improvements seen in zlib-cloudflare are primarily due to the usage of Arm NEON, x86 SSE intrinsics, and loads wider than 1 byte at a time. "
<PulkoMandy>
zstd is probably in a similar situation
<tqh>
yes, seems we get same the problems no matter what we do. At least the lib we use is supported by facebook :/
<PulkoMandy>
well, I think changing libraries is probably not going to help much, the two problems we have is, for the arm and other ports, that the setup with haikuports cross-compiling is not great, and we don't control the way our dependencies will be built. And for regular builds, that we just expect to reuse dependencies from the host OS, where maybe we should try to be more self-contained and maybe build things
<PulkoMandy>
from source?
<tqh>
The lz4 was only a todo in the pkg management doc, it is much faster a decompression so I think it would still be a great fit compared to zstd.
<PulkoMandy>
which may be easier if we have a compression/decompression lib that would fit in a single C or C++ file
<PulkoMandy>
yes, lz4 is a lot faster but it compresses less. Which means a bit longer download times and a bit more space used on disk (that is probably OK in our cases, the baseline is other OS who don't compress anything on disk at all)
togermano has joined #haiku
<PulkoMandy>
would be nice to have some proper benchmarks to know how often we decompress things, and also, how well lz4 behaves with our small-ish compression blocks (we need to do random access to the package content, so we compress it in relatively small blocks and not as a single big file)
<tqh>
I think that is ok, but in long run, you'd want to download diff of files in packeges I think.
<togermano>
haiku kernel is very advanced
<PulkoMandy>
not sure about diffs. it would sure save a lot of space, but maybe not so much on downloads unless you do a protocol where you tell the server which version you have and the server can generate a diff on the fly? Otherwise you would need to download a lot of incremental diffs?
<PulkoMandy>
anyway; I have a quite fast and unlimited internet access here, so indeed if you ask me I would also put my preference on decompression speed
<tqh>
You'd probably save a lot on downloads if you can look inside pkgs, all media assets that don't change between versions would not be downloaded.
<tqh>
haiku.pkg would probably have libs that don't change between versions.
<tqh>
Either that or really fine-grained packages.
<PulkoMandy>
if we had reproducible builds, maybe. But I don't think we have that currently (at lesat gcc2 puts a timestamp in every compiled file...)
<PulkoMandy>
finegrained packages would work if we didn't change their name with each hrev (like happens to the makefile_engine, it is never updated but it get rebuilt in every nightly build with a new hrev number :( )
<PulkoMandy>
looking at https://lz4.github.io/lz4/ the lz4-hc version seems quite interesting. higher compression time (but still not as bad as zlib on max settings), but smaller file and much faster decompression time
<tqh>
We should do a BeGeistert thing around packages...
<tqh>
And would love to have reproducable builds. Can that be done with gcc?
<anuragpd4_>
I tried installing zstd via macports instead of brew to see if it would fix things. Now I'm stuck with a new error https://pastebin.com/wZCxyvTi
<tqh>
I think MacOS requires tricks to use gcc for compiling. If you havn't done those you probably use the wrong compiler.
HaikuUser has joined #haiku
HaikuUser has quit []
<PulkoMandy>
?
<PulkoMandy>
clearly the correct compiler is used, we can see it in the command line
<PulkoMandy>
that is our gcc, which was built from whatever compiler is default on macOS (probably llvm/clang). With gcc11 this works fine now, I think
<anuragpd4_>
should I try building from linux in a VM instead ?
ClaudioM has joined #haiku
<PulkoMandy>
but it is building with -nostdinc so it wouldn't find system headers, I think?
<PulkoMandy>
building from Linux is probably going to work better, as it is done more often
<tqh>
I don't know anyone that compiles from Mac, so probably not well tested to build there.
<PulkoMandy>
seems a good idea to start that way, and maybe retry this if you are interested in debugging the build on macOS
<PulkoMandy>
I think John Scipione does, but possibly not on M1 hardware
<tqh>
linux should work fine as long as the correct libs are installded.
<PulkoMandy>
we accepted some patches for M1 support already so some people tried it somewhat recently (but before we switched from gcc8 to gcc11)
togermano has quit [Quit: Vision[]: i've been blurred!]
<anuragpd4_>
i'll try building from linux for now.
<tqh>
btw lz4 has built-in blocks that can be independent, so it supports random access if compressed correctly. Blocks being 64KB by default.
<tqh>
Maybe if you use zstd for 64KB blocks it won't compress much better than lz4
anuragpd4 has joined #haiku
B2IA has quit [Quit: Vision[]: i've been blurred!]
B2IA has joined #haiku
anuragpd4_ has quit [Ping timeout: 480 seconds]
<PulkoMandy>
well, zstd uses lz4 as one of the low level elements, I think. And in https://github.com/lz4/lz4/issues/187 someone says a good way to build a dictionary for such a thing is... to use zstd tools to do it
anuragpd4 has quit [Ping timeout: 480 seconds]
<PulkoMandy>
(but we currently don't use zstd in that mode, yet)
<tqh>
No I think since we do our own block thing, we probably need to benchmark that, and compare to use formats own seekable streams.
HaikuUser has joined #haiku
HaikuUser has quit []
gouchi has joined #haiku
HaikuUser has joined #haiku
HaikuUser has quit []
HaikuUser has joined #haiku
HaikuUser has quit []
x10z has joined #haiku
ClaudioM has quit [Quit: leaving]
HaikuUser has joined #haiku
HaikuUser has quit []
_Dario_ has quit [Quit: Vision[]: i've been blurred!]
<waddlesplash>
yes, we do our own "block mode"
<waddlesplash>
it would indeed be VERY cool if we used zstd block mode
<waddlesplash>
however, this would require a bunch of rework of the package kit :/
<waddlesplash>
it's overengineered in all the wrong ways
BrunoSpr has joined #haiku
<Begasus>
heading down here
<Begasus>
waddlesplash, no luck on fixing texstudio so far (had a patch but doesn't seem to do it's work on the latest release)
<Begasus>
will look into it further (when time permits)
<Begasus>
cu peeps!
Begasus has left #haiku [Ik ga weg]
HaikuUser has joined #haiku
andreasdr_ has quit [Remote host closed the connection]
HaikuUser has quit [Quit: Vision[0.10.3]: i've been blurred!]
HaikuUser has joined #haiku
HaikuUser has quit []
unimatrix has joined #haiku
yeti has quit [Ping timeout: 480 seconds]
Skipp_OSX has joined #haiku
AlienSoldier has joined #haiku
B2IA has quit [Quit: Vision[]: i've been blurred!]
B2IA has joined #haiku
BrunoSpr has quit [Quit: Vision[]: Ich wurde verwaschen!]
BrunoSpr has joined #haiku
Gaspare has joined #haiku
BrunoSpr has quit [Quit: Vision[]: Ich wurde verwaschen!]
x10z has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Gaspare has quit [Ping timeout: 480 seconds]
BrunoSpr has joined #haiku
B2IA has quit [Quit: Vision[]: i've been blurred!]
B2IA has joined #haiku
Begasus_32 has quit [Quit: Vision[]: Gone to the dogs!]
Skipp_OSX has quit [Quit: My Mac Pro has gone to sleep. ZZZzzz…]
anuragpd4 has joined #haiku
anuragpd4 has quit [Ping timeout: 480 seconds]
x10z has joined #haiku
jmairboeck has quit [Quit: Konversation terminated!]
BrunoSpr has quit [Quit: Vision[]: Ich wurde verwaschen!]
gouchi has quit [Remote host closed the connection]
bitigchi has quit [Ping timeout: 480 seconds]
x10z has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]