ChanServ changed the topic of #wayland to: https://wayland.freedesktop.org | Discussion about the Wayland protocol and its implementations, plus libinput
<mriesch>
hey all! i have got a v4l2 device that delivers nv12 and a gstreamer pipeline with v4l2src ! queue ! waylandsink
<mriesch>
if i use the contiguous variant (v4l2 nv12), everything is fine
<mriesch>
however, if i use the non-contiguous variant (v4l2 nm12, one buf for luma, one for chroma), the colors are all wrong and there are halos
kts has joined #wayland
<mriesch>
halos = the chroma plane (half the size in nv12) is somehow stretched over the complete output
<mriesch>
any idea what could be going wrong here?
JakeSays has quit [Read error: Connection reset by peer]
<soreau>
with weston?
<mriesch>
yep
<soreau>
weston git?
<soreau>
have you tried any other compositor by chance?
<mriesch>
weston 13.0.3
<mriesch>
uh no
<soreau>
I'd recommend trying weston git if you haven't, and maybe another compositor to see what happens there
<soreau>
just to gain a better scope on the issue
<soreau>
if it doesn't work with weston git though, you could file an issue
<mriesch>
ok, will do
<daniels>
I wouldn't expect anything's changed in git
<daniels>
but I would be very interested to see what the offset param is when the dmabuf gets imported
<daniels>
it sounds to me like the luma plane is being sampled again as the chroma plane, and yeah, nm12 is going to be funny because v4l2 mplane doesn't bother telling you what the byte offset is going to be
<mriesch>
what is the meaning of "byte offset" in this scope?
JakeSays has joined #wayland
<daniels>
the offset into the dmabuf at which the buffer data begins, e.g. for the chroma plane in NM12, the byte at which the first UV pair can be found
<mriesch>
so.. should be zero for both buffers in NM12 case, right?
<soreau>
basically the offset would be the size of the Y buffer data, IIUC
<mriesch>
soreau: in the nv12 case, surely
<mriesch>
however, in nm12 you have two separate buffers, right?
<soreau>
ah, I see
<soreau>
so U would be the second plane @ 0 and the V would be second plane @ size of U
rasterman has joined #wayland
<mriesch>
UV are interleaved, but yes, second plane @ 0
<mriesch>
daniels: ok, two params, both have offset 1 and stride 3840 (for my 3840x2160 video input)
<mriesch>
(offset 1 seems strange to me, but this is due to some gstreamer "skip" magic that i don't get)
<soreau>
hm..
<soreau>
mriesch: what if you pipe to file instead of waylandsink? or try glimagesink
<soreau>
maybe it's gstreamer being funny
<mriesch>
soreau: good point, forgot to mention that. i tried ! glupload ! glcolorconvert ! glimagesink and everything works find, even in the nm12 case
<soreau>
ah
<soreau>
so this smeels kinda like a waylandsink gstreamer bug
<mriesch>
wouldn't want to waste too much of your time here, but could we talk briefly about what is happening here at the big picture level?
<soreau>
mriesch: @ 3840x2160, you certainly have a big picture ;)
<mriesch>
soreau: nice one ;-)
kts has quit [Quit: Leaving]
<mriesch>
how are the two buffers combined to a single picture in weston?
<daniels>
if the underlying GL or KMS driver supports importing as a single image then we do that, if not then we fall back to one R8 and one GR88 buffer and do the colourspace conversion manually
<daniels>
what hardware & driver are you using?
<mriesch>
this is on the rockchip rk3568 with mainline drivers
<mriesch>
i.e., vop2 drm + panfrost gpu
<wlb>
wayland-protocols Merge request !369 closed (xdg-shell: Add request to make a toplevel surface always on top)
glennk has joined #wayland
<daniels>
mriesch: hmm yeah, that certainly should work
<daniels>
but it seems like it's definitely an issue in either GStreamer itself or waylandsink I'm afraid, because both NV12 and NM12 on the V4L2/GSt side resolve to just NV12 on the GPU/display side
<mriesch>
daniels: possibly, yes. i guess this is quite a corner case with some barely tested code paths involved
<soreau>
maybe try gstreamer git if you're not already and file an issue if it's still a problem
<mriesch>
where does this check happen whether direct import is possible?
<soreau>
np, and of course there is also #gstreamer on this network too
<daniels>
mriesch: yeah, NV12 in Weston is exhaustively tested, but NM12 in GSt probably less so
<mriesch>
daniels: but what about nm12 in weston?
<mriesch>
from an outsider's point of view nm12 does make a difference, even if at some point nv12 and nm12 are handled the same way
<daniels>
weston never sees 'nm12'
<daniels>
weston only ever sees DRM_FORMAT_NV12, with the Y and UV plane defined separately
<daniels>
so if V4L2/GSt NV12 works but V4L2/GSt NM12 doesn't, then either the V4L2 device is producing erroneous NM12, or GSt is erroneously translating it
<soreau>
garbage in.. garbage out
kts has joined #wayland
<mriesch>
ah ok
<mriesch>
just for the sake of completeness, could the drm driver be the culprit?
<soreau>
if it works with glupload/glimagesink, it sounds more like gst to me
<linkmauve>
soreau, there are two DRM drivers here, panthor will serve the GL parts, and rockchip-drm will take either a RGBX8888 dmabuf from panthor, or a NV12 buffer from V4L2 (after some mangling in Gstreamer).
<soreau>
mriesch: I'm just kinda curious why you'd want waylandsink over glimagesink
<linkmauve>
But I believe I have already tested decoding on the rk3568, although IIRC WebP decoding was broken on that SoC.
<mriesch>
soreau: in my experience, better performance since the gpu is not involved
<soreau>
huh
<linkmauve>
soreau, it is more efficient, as it can use the specialized hardware present in the SoC to scale the buffer and convert it from Rec.709 YUV to RGB.
<linkmauve>
Instead of doing it all in shaders.
<soreau>
I see..
<linkmauve>
At 4K resolutions it can make a huge difference, but even at lower resolutions.
<soreau>
so for my case of live streaming rtsp streams to wl windows, I should use waylandsink instead of glimagesink for better/best performance?
<linkmauve>
On my Intel laptop it can be the difference between watching a series in three hours in a train, or watching it for eight hours, something like that IIRC.
<linkmauve>
Before the battery runs out.
<linkmauve>
soreau, if your hardware supports it and you don’t need anything else from OpenGL, probably.
<soreau>
linkmauve: how would I check if the hw supports it?
kts has quit [Remote host closed the connection]
kts has joined #wayland
<linkmauve>
% v4l2-ctl --list-formats-out -d0
<linkmauve>
That’s for /dev/video0, increase the -d parameter for /dev/video1 etc.
<soreau>
no, this is rtsp streams
<linkmauve>
And on desktop, you most likely want % vainfo instead.
<soreau>
well I am using vaapi..
<linkmauve>
soreau, which you demux on the CPU, and then pass on to the hardware for decoding/presentation.
<linkmauve>
soreau, if it’s already output by vaapi, then your hardware certainly supports it.
<linkmauve>
Try s/glimagesink/waylandsink/ and see for yourself. :)
<linkmauve>
You can use # intel_gpu_top to monitor the usage of the 3D parts.
<soreau>
it's radeon, but thanks
<soreau>
I'll give it a go
<linkmauve>
# radeontop then, and possibly some other tools to check the power usage of the whole computer.
<soreau>
sure
* soreau
types `top`
<linkmauve>
That likely won’t give you anything but the CPU’s relative usage.
* soreau
colors linkmauve informative
<linkmauve>
soreau, # powertop is what I use on my laptop.
<soreau>
linkmauve: ok, thanks
<soreau>
I usually just judge cpu usage by listening to the fan xD
kts has quit [Ping timeout: 480 seconds]
kts has joined #wayland
Moprius has joined #wayland
kts has quit [Remote host closed the connection]
kts has joined #wayland
iomari891 has quit [Ping timeout: 480 seconds]
iomari891 has joined #wayland
Brainium has joined #wayland
kts has quit [Remote host closed the connection]
kts has joined #wayland
feaneron has joined #wayland
<linkmauve>
soreau, in neither case should there be much CPU usage, as you are delegating to either the GPU (in the OpenGL case) or to various dedicated bits of hardware (in the Wayland + DRM planes case, if your compositor supports that).
rgallaispou has quit [Read error: Connection reset by peer]
abris has joined #wayland
Moprius has quit [Quit: bye]
kts has quit [Remote host closed the connection]
kts has joined #wayland
fmuellner has joined #wayland
kasper93_ has joined #wayland
rgallaispou has joined #wayland
kasper93 has quit [Ping timeout: 480 seconds]
abris has quit [Ping timeout: 480 seconds]
kts has quit [Ping timeout: 480 seconds]
abris has joined #wayland
kts has joined #wayland
kts has quit [Ping timeout: 480 seconds]
leon-anavi has quit [Remote host closed the connection]
<soreau>
linkmauve: using wlroots without libliftoff if it matters.. I'm not sure if it's placebo effect because I haven't properly measured, but it seems waylandsink uses slightly less resources in this scheme