ChanServ changed the topic of #cedrus to: Cedrus development channel (http://linux-sunxi.org/Cedrus) | *only registered users can talk* | Logs at https://oftc.irclog.whitequark.org/cedrus/
<jernej> linkmauve: header parsing in kernel won't be accepted for mainline kernel
<jernej> all data must come from controls
Daanct12 has joined #cedrus
Daaanct12 has joined #cedrus
Daanct12 has quit [Ping timeout: 480 seconds]
<linkmauve> jernej, the kernel already has include/media/v4l2-jpeg.h for JPEG header parsing.
<linkmauve> Used by the coda and imx platforms.
<linkmauve> But it makes less sense to do that in the request API I think.
Daaanct12 is now known as Daanct12
<linkmauve> Ugh, my phone just rebooted once I ^C’d this command: % gst-launch-1.0 filesrc location=Videos/vp8.webm ! matroskademux ! v4l2slvp8dec ! glimagesink
<linkmauve> It otherwise decodes it almost correctly, just there is a green line at the bottom, probably some incorrectly-negociated buffer size somewhere.
<linkmauve> And I can reproduce the reboot.
<linkmauve> Is Gstreamer the easiest userland to work on that, or is there something a bit more self-contained?
<linkmauve> I have a crate for dealing with the media and request APIs, but so far no video API support so I can’t really use it to test decoding.
<linkmauve> Ok, not just if I ^C, also if the video finishes properly.
<linkmauve> And besides the reboot and the green line, there is also a weird chroma drift from the top-left towards the bottom-right, which gets reinitialised on keyframes or strong movement.
<linkmauve> And it also happens with H.264.
<linkmauve> (I’m on a master kernel built earlier, I could check on 5.16 if it’s a regression.)
<linkmauve> With H.264 I also get this warning: imported bo size is smaller than expected: 462848 (BO) < 471040 (expected)
<linkmauve> No, same behaviour on 5.16.13…
<linkmauve> Also, when using waylandsink instead of glimagesink, I get this: (gst-launch-1.0:2118):
<linkmauve> ERROR: from element /GstPipeline:pipeline0/v4l2slvp8dec:v4l2slvp8dec0: Unsupported pixel format
<linkmauve> GStreamer-Video-CRITICAL **: 11:12:59.871: gst_video_format_to_string: assertion 'format != GST_VIDEO_FORMAT_UNKNOWN' failed
<linkmauve> And obviously it doesn’t play.
<linkmauve> Ah, waylandsink doesn’t support NV12.
<linkmauve> With videoconvert in the middle it does play, but with a very green tint!
<linkmauve> Of course, waylandsink would have had to support NV12_32L32, which is probably specific to this SoC.
Daanct12 has quit [Quit: Quit]
<jernej> linkmauve: afaik coda and imx platforms use stateful driver, where parsing is done by coprocessor, not kernel driver
<jernej> it was agreed a while back that request api drivers won't do any kernel parsing
<jernej> why don't you use normal NV12?
<jernej> last time I tested gstreamer h264 support on H6, it rendered correctly directly on screen, but that was without any desktop environment
<linkmauve> jernej, not sure, it was automatically negociated by Gstreamer, I’m just looking at the pipeline SVG it produced (before crashing my phone…).
<linkmauve> Now I’m testing on a SBC, which doesn’t crash but has an A20 instead of an A64.
<linkmauve> From what I read in some cedrus file, the A20 only supports 32L32.
<linkmauve> (I assume this is what CEDRUS_CAPABILITY_UNTILED means.)
<jernej> true
<jernej> I thought you're using something newer
<jernej> if you're running gstreamer without desktop environment and using kmssink, it should work and you don't even need videoconvert
<linkmauve> My second screen is busted, so I’m relying on waypipe for now. :(
<jernej> at the end, you can always decode to images and check that
<linkmauve> Do you have any sample code for that?
<linkmauve> I’m still in the middle of adding video API support to my crate, but I won’t know whether this userland code isn’t working or if it’s the kernel driver, or even the hardware. :D
aedancullen has quit []
aedancullen has joined #cedrus
aedancullen has quit []
aedancullen has joined #cedrus
<jernej> no, I'm more of a ffmpeg user
<jernej> but there is nothing in upstream, just collection of patches
<jernej> nothing for JPEG request api though
<linkmauve> What is holding up upstreaming these patches?
<jernej> time and motivation
<jernej> 2 rounds were already done, but original author got busy IRL
<jernej> now that RPi also uses request API for HEVC, there are additional improvements, which nobody make sure yet that works for all
<jernej> you can volunteer :D
<linkmauve> :/
<linkmauve> I don’t have any Raspberry device to test against though, only a bunch of Allwinner, one Amlogic, and one Nintendo.
<linkmauve> And a Qualcomm which doesn’t start any more, I’d like to figure out why someday.
<jernej> it's relevant only for RPi4
<jernej> older use SW HEVC decoding
<linkmauve> I could try to shepherd this series to upstream, this wouldn’t be my first time. :p
<linkmauve> Btw, thanks you for making me figure out that kernel dev isn’t harder than userland dev!
<linkmauve> I’ve been steadily upstreaming the Wii U port this past year, writing drivers as it goes. :)
<jernej> sure, I'm just not sure how I helped you to come to this conclusion :)
<linkmauve> It was about the VP8 decoder I helped fix for 5.11. ^^
<jernej> ah
<jernej> VP8 isn't best representation of request api driver
<jernej> I would go with simplest - MPEG2
<linkmauve> That’s what I did so far, here is my current attempt (untested, before reworking it to receive the tables into ctrls): https://gitlab.com/linkmauve/linux-wiiu/-/commits/cedrus-jpeg
<linkmauve> It’s in a state of “at least it boots”, and completely based off the MPEG2 code.
<linkmauve> Would it be ok to basically require userland to pass a struct v4l2_jpeg_header as the ctrl, btw?
<jernej> looking at current linux code, there is no jpeg request api decoder
<linkmauve> Yes, that’s what I’m trying to add.
<jernej> so you have to invent some things first, like format
<jernej> V4L2_PIX_FMT_JPEG is full image, you should add something like V4L2_PIX_FMT_JPEG_PARSED
<jernej> although probably more sensible
<jernej> but yeah, if aforementioned structure contains everything you need, it can be used as control
<jernej> uh, no
<jernej> it contains pointers, that a no-go
<ndufresne> "linkmauve> I have a crate for dealing with the media and request APIs, " ineresting, I have a colleague working on VA decoder for cross-vm, virtio-gpu, and V4L2 Stateless CODEC support (I hate calling is request), is coming next
<ndufresne> He's adding crate for whatever is missing or not good enough for parsing, but also for state manangement
<ndufresne> jernej: oh, and yes, there is exception for jpeg stateless HW in few places, cause it was "simple and safe enough"
<ndufresne> they don't call it stateless, since that make no sense, JPEG encoder are always stateless
<ndufresne> they call it raw encoder, or raw decoder (without headers)
<jernej> ok, then linkmauve, your approach should work
<jernej> just make sure you properly set start of bitstream
<jernej> which should be right after the header
<jernej> ndufresne: is there any userspace app which knows how to use v4l2 jpeg decoder?
<ndufresne> jernej: chromium, gstreamer, not sure why ffmpeg stateful v4l2 could not support that
<ndufresne> it all statefull, so you collect an entire jpeg in 1 buffer, and give it to the driver
<ndufresne> as its stateless, if you picked the buffer size wrong, you just reset the thing and realloc
<ndufresne> live is so easy with still pictures
<ndufresne> * life
<ndufresne> of course, it not ideal if you HW support progressive load, but its not a trendy these days
<ndufresne> these are mostly use to handle mjpeg, normal jpeg but produce by camera to workaround USB2 bandwidth
<linkmauve> jernej, start of bitstream, do you mean the offset I’ve (so far) left set to 0?
<linkmauve> ndufresne, would you prefer me to publish what I already have, or to add it to their existing crate?
<linkmauve> would they* prefer ^^
<linkmauve> So far it’s just a repository on my SSD.
<ndufresne> linkmauve: perhaps ping dwlsalmeida on #gstreamer, he's the colleague I'm referering to
<linkmauve> Ack. :)
<jernej> yes, I mean offset
<ndufresne> not sure were his stuff goes tbf, but he said that he got VP8/VP9/AV1 ready, and will start H.264
<jernej> but you also didn't select jpeg mode
<ndufresne> offset to the raw jpeg start, skipping the header right ?
<jernej> yes
<ndufresne> and I wouldn't be surprise this is what pH5 helper for coda does
<jernej> btw, what is your source for register documentation? old CedarC source?
<linkmauve> jernej, jpeg mode where? In the cedrus registers?
<jernej> yes
<linkmauve> cedrus_engine_enable(ctx, CEDRUS_CODEC_JPEG); does that I think.
<jernej> that's engine
<jernej> which can decode mpeg1/2/4, vp6.2 and jpeg
<jernej> so you have to tell which codec you have in mind exactly
<linkmauve> In the proof of concept code I used as an example, set_format() was a thing which wrote to one byte of the trigger register, so far I hoped writing to all four bytes as an uint32_t would do the same.
<linkmauve> They do writeb(0x0e, ve_regs + VE_MPEG_TRIGGER);
<linkmauve> Oh, I’m wrong, I still had VE_DEC_MPEG_TRIGGER_MPEG2 from the MPEG2 code.
<linkmauve> Will change that asap, thanks. :)
<linkmauve> “22:13:45 ndufresne> jernej: chromium, gstreamer, not sure why ffmpeg stateful v4l2 could not support that”, how do you do that with Gstreamer? I couldn’t find a v4l2sljpegdec element anywhere.
<jernej> you set VE_DEC_MPEG_TRIGGER_MPEG2 instead of VE_DEC_MPEG_TRIGGER_JPEG
<linkmauve> That’s fixed now, thanks. :)
<linkmauve> The offset as well.
<jernej> btw, don't use static variables inside functions
<linkmauve> I went for a static buffer in order to avoid blowing up the stack, the compiler was configured to warn for any function using more than 1024 bytes of stack apparently.
<linkmauve> Should I make it a global instead?
<jernej> no, static buffer is same as global
<jernej> and neither is acceptable for kernel
<jernej> allocate it via kmalloc and store pointer in jpeg context
<linkmauve> Ok.
<jernej> and maybe it would be possible to avoid it with some code reorganization
<linkmauve> I could do that, yeah.
<jernej> ndufresne: which helper do you have in mind?
<linkmauve> I don’t understand the data we have to put in said buffer at all.
<linkmauve> Like, the sum *= 2; is black magic to me.
<linkmauve> Or why we never have to put the last number of the huffman tables.
<ndufresne> jernej: ./drivers/media/v4l2-core/v4l2-jpeg.c (but I never looked at the one), I mostly play with the h264 and vp9 one, which are not about parsing
<jernej> I have no idea about actual register values
<linkmauve> I’ll just leave the values as they are, and try to avoid the buffer altogether.
<linkmauve> I think I can reduce the memory usage from 2048 to 64 bytes.