ChanServ changed the topic of #wayland to: https://wayland.freedesktop.org | Discussion about the Wayland protocol and its implementations, plus libinput
<JoshuaAshton>
riteo: You should have no problems if your frame scheduling comes from swapchain backpressure or WaitForPresentKHR. There is no need to do anything with the frame callback yourself.
DemiMarie has left #wayland [#wayland]
rasterman has quit [Quit: Gettin' stinky!]
<JoshuaAshton>
For games, being blocked from AcquireNextImageKHR or WaitForPresent (eg. if you want to limit latent images) is definitely the right way to go.
<JoshuaAshton>
I guess WaitForPresent is similar to the frame callback (it's presentation_time presented) -- but it really needs to be linked to a specific image so you can get the right depth going to avoid starvation
<JoshuaAshton>
Really what you'd want is some WaitForPresent2 picking LATCH there (rather than TTL), for some depth into the queue, but that isn't a thing yet. This is the behaviour Gamescope will give you with regular WaitForPresent fwiw.
<JoshuaAshton>
I hope we can get there someday =(
<JoshuaAshton>
Really native Wayland gaming is not viable at all until commit_queue is merged
<JoshuaAshton>
Any GPU intensive app that requires 2+ swapchain depth to avoid starvation stutters very badly because FIFO is broken without it
Brainium has quit [Quit: Konversation terminated!]
glennk has quit [Ping timeout: 480 seconds]
i509vcb has joined #wayland
Brainium has joined #wayland
nerdopolis has joined #wayland
shoragan has quit [Quit: quit]
shoragan has joined #wayland
co1umbarius has joined #wayland
Leopold_ has quit [Remote host closed the connection]
Leopold_ has joined #wayland
columbarius has quit [Ping timeout: 480 seconds]
<riteo>
JoshuaAshton: well, until the necessary infrastructure is there we have to handle them ourselves, otherwise it all stalls on surface suspension
* riteo
shrugs
<JoshuaAshton>
riteo: commit_queue + commit_timing should fix that as well, fwiw. Unfortunately there just isn't a good mechanism to solve this right now. Native Wayland for games is not really feasible without those.
<JoshuaAshton>
`frame` doesn't help you when you don't get `frame` after all, and only rendering when you recieve frame doesn't work for games, because the time you are spending waiting for frame is time you should be preparing your next frame
<LaserEyess>
has any other compositor been interested in implementing that though?
<LaserEyess>
lol @ 2 wlroots impls though
<JoshuaAshton>
I believe Derek Foreman is working on a Mutter implementation?
<JoshuaAshton>
We are shipping commit_queue (albeit vendored) right now in Gamescope/SteamOS. There's a Vulkan layer, Gamescope WSI, that wraps X11/XCB -> Wayland surfaces and tells Gamescope to override X11 windows with a Wayland surface. This is how we bypass XServer for better latency, HDR, etc. So the commit queue thing was something we had to solve there to avoid stutters/jank from swapchain starvation
Leopold_ has quit [Remote host closed the connection]
<LaserEyess>
hopefully that counts as a client then
<JoshuaAshton>
Mesa is the client
Leopold_ has joined #wayland
<LaserEyess>
oh, duh
<LaserEyess>
and I saw those patches
<JoshuaAshton>
I was tempted to implement the commit_queue stuff in the layer, but I saw Derek already did it for Mesa, so might as well get it tested :-)
<LaserEyess>
speaking of
<JoshuaAshton>
Regardless, stalling on QueuePresent etc like now when occluded violates Vulkan forward progress guarantees, so it is really not something an application should have to deal with/work around. It's non-compliant behaviour
<LaserEyess>
does that work with "regular" gamescope?
suntory_ has joined #wayland
<LaserEyess>
i.e. not on the steam deck
<JoshuaAshton>
Pragmatically, I would like to workaround it -- the reason we haven't is that every workaround reeeally sucks and doesn't fully solve the problem :/
<JoshuaAshton>
LaserEyess: The layer, HDR, all of that stuff, yes, commit_queue does require Derek's Mesa commits though.
<JoshuaAshton>
The Gamescope package from your distro should come with Gamescope + the associated Vulkan layer
<LaserEyess>
yes, it does
<LaserEyess>
but I should have specified the wayland backend
<JoshuaAshton>
The Wayland backend of what sorry?
<LaserEyess>
gamescope
<JoshuaAshton>
Gamescope just uses SDL and regular Vulkan WSI to present when nested
<LaserEyess>
ah
<JoshuaAshton>
I've been wanting to potentially have another method of using subsurfaces and forwarding commits immediately to the host compositor and such, but have not had the time
<JoshuaAshton>
Like a "proxy compositor" mode
<LaserEyess>
the other features are cooler anyway
<JoshuaAshton>
Which? :D
cool110 has joined #wayland
<LaserEyess>
all of the ones you've currently implemented
<JoshuaAshton>
:frog: ty
cool110 is now known as Guest9907
<LaserEyess>
even without the other improvements gamescope nested on wayland is still great
Guest9679 has quit [Ping timeout: 480 seconds]
suntory has quit [Ping timeout: 480 seconds]
privacy has quit [Quit: Leaving]
tlwoerner__ has quit [Read error: Connection reset by peer]
tlwoerner_ has joined #wayland
GentooPhysicist3935426 has quit [Quit: Ping timeout (120 seconds)]
GentooPhysicist3935426 has joined #wayland
fmuellner has quit [Ping timeout: 480 seconds]
privacy has joined #wayland
nerdopolis has quit [Ping timeout: 480 seconds]
<riteo>
JoshuaAshton, regarding commit_queue + commit_timing: thanks, great to know. Can't wait for this stuff to get agreed upon so that we can have a way better solution than handling frame events oursleves or refactoring everything :D
<riteo>
You've said though that for GPU intensive applications blocking on ANI or WFP is the right thing. Why?
<riteo>
Wouldn't the frame event be good too, given that the use case allows for stalling while suspended?
<riteo>
A poll-only approach would be very pretty IMO if possible, although I don't know if any game engine works like that either
Brainium has quit [Quit: Konversation terminated!]
garnacho has quit [Ping timeout: 480 seconds]
Guest9907 has quit [Remote host closed the connection]
cool110 has joined #wayland
cool110 is now known as Guest9915
kts has joined #wayland
kts has quit [Ping timeout: 480 seconds]
kts has joined #wayland
kts has quit [Quit: Konversation terminated!]
kts has joined #wayland
ecloud has joined #wayland
ecloud_ has quit [Ping timeout: 480 seconds]
kts has quit [Ping timeout: 480 seconds]
ecloud has quit [Remote host closed the connection]
ecloud has joined #wayland
bindu has quit [Remote host closed the connection]
bindu has joined #wayland
sima has joined #wayland
floof58 has quit [Quit: floof58]
floof58 has joined #wayland
Company has quit [Quit: Leaving]
manuel_ has quit [Ping timeout: 480 seconds]
privacy has quit [Quit: Leaving]
sevz has quit [Quit: WeeChat 4.1.1]
nnm has quit []
nnm has joined #wayland
rgallaispou has joined #wayland
luna has joined #wayland
glennk has joined #wayland
luna has left #wayland [#wayland]
manuel_ has joined #wayland
rv1sr has joined #wayland
leon-anavi has joined #wayland
flokli has quit [Quit: WeeChat 4.0.5]
flokli has joined #wayland
garnacho has joined #wayland
Leopold_ has quit [Remote host closed the connection]
Leopold has joined #wayland
tlwoerner_ has quit [Read error: Connection reset by peer]
tlwoerner_ has joined #wayland
rasterman has joined #wayland
i509vcb has quit [Quit: Connection closed for inactivity]
neniagh_ has quit [Read error: Connection reset by peer]
neniagh has joined #wayland
<pq>
riteo, you're welcome. I just hope I don't come out as too ignorant of your particular restrictions. I can only describe how Wayland is intended to work, and you'll need to adapt that what you can actually do.
<pq>
riteo, re: timer; if your requirement is "render at 30 Hz or bust" (like Dark Souls I think?), then a timer gives you that while frame callbacks do not. Even if you had access to hardware vsync events, they might not be (a multiple of) 30 Hz. And if you need to render regardless of what's happening on the display, then that's by definition a timer. Yes, the drift will happen even if rates match.
<pq>
riteo, re: when to render a scheduled frame; usually when you have no more incoming events to process i.e. you would be blocked in (the single threaded model) the event loop.
<pq>
riteo, since you have different threads, maybe the main event loop thread (serving Wayland and timers) should send a wake-up to the rendering thread when it's about to block again?
<pq>
riteo, the main idea would be to be able to spin the main event loop without rendering on every cycle.
glennk has quit [Ping timeout: 480 seconds]
<pq>
riteo, I see that demimarie and kennylevinsen already discussed this nicely.
vyivel has quit [Remote host closed the connection]
vyivel has joined #wayland
DodoGTA is now known as Guest9946
Guest9946 has quit [Read error: Connection reset by peer]
cmichael has quit [Remote host closed the connection]
cmichael has joined #wayland
cmichael has quit [Remote host closed the connection]
<pq>
JoshuaAshton, were you talking about games that have a single-threaded main loop design of the naive kind: do { handle_events; update_simulations; render_and_present; } while (running);? The gist being that for each rendering cycle, you handle input and run simulation exactly once, even if spread to multiple threads. And then you might pipeline those across multiple threads to run render of frame N,
<pq>
simulation N+1, and input N+2 in parallel, but still keeping a strict 1:1:1 cadence?
<pq>
IMO, blocking on anything but epoll/poll/WaitForMultipleObjects/etc fundamental OS event loop functions is a non-optimal design. Unfortunately, some standard APIs have been designed to not integrate with that at all, so you get to play with threads to work around it.
cmichael has joined #wayland
paulk-bis has quit []
paulk has joined #wayland
Ariadne has quit [Read error: Connection reset by peer]
panzeroceania has quit [Read error: Connection reset by peer]
jimjams has quit [Read error: Connection reset by peer]
sudocurse__ has quit [Read error: Connection reset by peer]
bwidawsk has quit [Write error: connection closed]
sudocurse__ has joined #wayland
jimjams has joined #wayland
panzeroceania has joined #wayland
bwidawsk has joined #wayland
Ariadne has joined #wayland
leon has joined #wayland
leon-anavi has quit [Ping timeout: 480 seconds]
fmuellner has joined #wayland
nerdopolis has joined #wayland
pq has quit [Quit: brb]
pq has joined #wayland
floof58 has quit [Quit: floof58]
floof58 has joined #wayland
Leopold has quit [Remote host closed the connection]
kts has joined #wayland
Leopold has joined #wayland
qaqland is now known as Guest9954
tlwoerner_ has quit [Read error: Connection reset by peer]
Russ[m] has quit [Quit: Client limit exceeded: 20000]
kts has quit [Quit: Konversation terminated!]
<Company>
does ajax still exist on irc?
Brainium has joined #wayland
rasterman has quit [Quit: Gettin' stinky!]
bindu_ has joined #wayland
<JoshuaAshton>
pq: Games typically only want to block until they can render a frame -- which is when AcquireNextImageKHR is complete and the fence from AcquireNextImageKHR is signalled. They have nothing to block on epoll etc like a regular event loop. Games are integration based and not event driven.
bindu has quit [Ping timeout: 480 seconds]
<JoshuaAshton>
You still simulate without user input, etc -- you merely integrate done events per-frame, not with a typical event loop. They are discrete simulations after all :p
<JoshuaAshton>
pq: And yes, that is basically the design, but place AcquireNextImageKHR after present
<JoshuaAshton>
You always want to acquire before doing a new frame otherwise you end up with an inversed frame cadence
<JoshuaAshton>
I wouldn't call it naiive, it's entirely the right solution for discrete simulations such as games
<JoshuaAshton>
It's pretty over simplified, but explains a lot.
<emersion>
JoshuaAshton: ideally the game processes input events as they arrive, runs the game state update without having it tied to rendering, and renders asynchronously in a different thread
<emersion>
close to vsync
<JoshuaAshton>
Well, not ideally, no. That's not possible. As I mentioned games are discrete simulations and as such things happen in discrete steps. They are tick-based -- some are fixed tick rate simulations and some tick every frame.
<soreau>
of course not all games are created equally
<JoshuaAshton>
Unless you are making a text-based adventure game, they really are.
<JoshuaAshton>
Physics engines are also entirely discrete systems that work on ticks.
<emersion>
the tick doesn't need to be vsync
<emersion>
and the rendering doesn't need to be tied to the tick
<emersion>
why should i get better input latency (regarding the game state) if my screen has a higher refresh rate?
<JoshuaAshton>
What you are describing would not help input latency
<emersion>
why not?
Moprius has quit [Quit: bye]
<JoshuaAshton>
(One sec writing up)
<emersion>
no rush
<emersion>
this is pretty surprising to me, it's the first time i see a gamedev saying that the basic game loop s the right thing to do
<emersion>
is*
<JoshuaAshton>
Oh I'm not saying it should be *that* simple, just that any simulation basically boils down to that.
<emersion>
any simulation has a update_state() function which needs to be run periodically, sure
i509vcb has joined #wayland
<JoshuaAshton>
Really what you are describing is what a lot of games already do -- they split out the engine into a client (running at refresh rate) and a server (running fixed tick), it does not improve input latency.
<JoshuaAshton>
Source 1 works like this -- the engine always runs at 64/66.7tps and the client runs at your native refresh rate (lets say 144Hz). Even in singleplayer this model holds true.
<JoshuaAshton>
This can either be done by only simulating a server tick until a certain amount of time has accumulated, or by being in another thread/process/machine (eg. multiplayer). You can see an example of this in that game loop article I sent above.
<JoshuaAshton>
The client when rendering is simulating -- it predicts actions that are going to happen, or interpolates things that have, which it missed in the past, across a small timestep in order to look smooth (eg. things other player inputs).
<JoshuaAshton>
This split doesn't help you reduce latency -- the client's still waking up after AcquireNextImageKHR is complete to poll input, send that to the server/accumulate to future send and prepare it's frame.
<JoshuaAshton>
The way you *can* reduce input latency is basically doing what Gamescope had in the past, where we measured how long it took for a client to prepare it's buffer and the work to complete on the GPU from when we released a buffer to it -- timing between AcquireNextImageKHR and GPU work done. We then withheld the buffer from the client for an amount of time in order to make it so it was released and had it's work complete just in time for
<JoshuaAshton>
latching (with some redzone).
<JoshuaAshton>
I do need to bring back that Gamescope mode, the problem it has is that it's very prone to getting into GPU clock feedback loops, but now we can set min clocks and clocks in general without hangs, it's probably a good thing to explore again.
<kennylevinsen>
Yeah frame scheduling is great, just a bit tricky to get right
<JoshuaAshton>
There's some more to this as well with some recent games doing things like having another thread waiting for inputs and storing off the time the key/button was pressed/released such that they can incorporate that into both the server ticks and client prediction.
<JoshuaAshton>
Like that's something where some part of having event based stuff makes sense I guess -- but that's not part of the main loop of the game, that's stuff that's going to be on another thread -- then fed back into the main loop for either client/server/both doing a little poll every step
rgallaispou has quit [Quit: Leaving.]
<emersion>
ty for replyin, i'll read that up tomorrow, need to get off the computer now
<kennylevinsen>
For more complex games approximate busy loops, you don't have much perf to gain from being event driven. Same for e.g. super high perf networking where busy looping is normal...
<JoshuaAshton>
And to reply more to this:
<JoshuaAshton>
> <emersion> JoshuaAshton: ideally the game processes input events as they arrive, runs the game state update without having it tied to rendering, and renders asynchronously in a different thread
<JoshuaAshton>
The game state update is discrete in nature -- eg if you press a key to move forward, that's not something that can be processed instantly -- that needs to be integrated to move you forward by a certain amount after a certain amount of time
<ManMower>
is deferring buffers a universally correct way to get games to have lower latency? I feel like someone out there has probably written code that allocates a new buffer it there are none free when a frame callback arrives.
<JoshuaAshton>
That's not universal no, that's just one way of doing it
<JoshuaAshton>
Some games integrate with NVIDIA Reflex and such which is slightly more advanced, but in a different place
<JoshuaAshton>
In essence, it's solving the same problem though
<JoshuaAshton>
I think NV had something called NVIDIA Ultra Low Latency which would be basically the same concept as what I described above (blocking *after* Present in D3D though, which is the same as blocking in Acquire for Vulkan semantically)
<kennylevinsen>
But if you don't block on acquiring the next buffer, you would free up the thread for event handling and could run the ticks off an event loop, at least for less intense games. It also solves the problems when people start writing regular applications in the game toolkits (sdl2, Godot, ...), ...
<JoshuaAshton>
kinda
<kennylevinsen>
Being a simulation that can't get much better input latency doesn't rule out an event driven design
<ManMower>
ooi, is nvidia's reflex something that works on linux/with wayland compositors in general/with gamescope in particular?
pac85 has joined #wayland
<pac85>
I see that the topic is games, I want to add that a renderer in the context of a modern game is not stateless. It's easy to imagine a game as a Moore machine where the renderer is the output function but in reality renderer really means "code that touches the gpu" and it might involve things like a particle physics simulation. Things get even more complex for things like virtual texturing where the renderer might be feeding back
<pac85>
some resource streaming code. Those systems are all written with the assumption that they can incrementally update their state or incrementally keep some state in a gpu buffer in sync with something on the host so the idea of stopping rendering when the window is occluded is simply not feasible in many cases. In short this means that things must be very tightly coupled and must run with a strict order, something an event driven
<pac85>
design is not a good fit for. Also the rate at which the simulation state is required is exactly the framerate and "right before beginning rendering" is exactly when it's best to do it for latency. For this reason the trivial loop is not necessarily a bad idea and also why prediciton runs right before (or as close to) rendering in games that have a fixed tickrate.
<kennylevinsen>
pac85: an event loop still allows you to e.g. batch inputs while simultaneously servicing a fixed tick rate for the actual state update, which in turn is sampled at render time at some third (irregular, yay VRR) rate
<kennylevinsen>
hence event loop not being mutually exclusive with fixed rate simulations
rv1sr has quit []
<kennylevinsen>
It's just that when you tick a simulation at a fixed rate, you're kind of limited in what benefits you get...
<kennylevinsen>
... but there's an issue when such fixed tick rate leaks into a toolkit also used for regular apps :)
<pac85>
kennylevinsen: keep in mind that fixed rate doesn't necessarily mean the code actually runs at a fixed rate, it can also mean that the simulation will be advanced in fixed steps until it's time is > current time (or a step before) so that the last two states can be interpolated (or predicted). This can be done at the beggining of the main loop for example
<kennylevinsen>
Sure, I am the furthest from an expert in gamedev, just suggesting that the requirement could be met by an event loop. Using a frame event in an event loop rather than blocking *until* frame event would also never have had the eglSwapInterval stall issue, as other tricks or events would continue even when frame events are absent
<ManMower>
pac85: are you saying that a frame event that goes away on occlusion can never be sufficient, or that it's broken because existing games are expecting something else?
Brainium has quit [Quit: Konversation terminated!]
<kennylevinsen>
This of course won't change what games already do, it's just input to what e.g. Godot could consider doing to have best of both worlds, allowing both normal apps and games to effectively coexist
<kennylevinsen>
ManMower: a lot of games do e.g. eglSwapBuffer with a non-zero swapinterval, which in Mesa block until previous frame callback completes. Such games never anticipated that this could block their main loop indefinitely if the window is not visible.
<kennylevinsen>
If the game was event driven instead of busy looping and blocking it would not be an issue, but that's not how many games are written...
<pac85>
ManMower: I'm saying that when occluded games often need to render anyway. Many games will limit their maximum framerate to save some resources (with some letting the user configure such rate). A frame event that goes away is not very helpfull to implement this and ideally games that are not "occlusion aware" shouldn't break. Also games generally use whatever the API provides which is good for portability
<pac85>
Why would a game busy loop? You usually wait on a fence in the case of vulkan
<kennylevinsen>
JoshuaAshton: hmm, curious question: how common is it to have simulation/main game tick separately from rendering? Do such games mind when render thread blocks indefinitely?
<ManMower>
I'm just curious as to whether there's some bigger problem in play that I don't understand
pac85 has joined #wayland
<ManMower>
pac85: why does a game have to render when it's not visible?
<riteo>
ManMower: that's actually a good question. Actually, I think that it does most "rendering" stuff without actually calling the GPU. For example, godot keeps calling `_process` (the render notification) even when a window is hidden
<riteo>
because most scripts depend as that as an "unreliable" loop method
<riteo>
instead of `_physics_process` which is fixed rate
<pac85>
ManMower: I sent a long message above, let me know id you've nissed it and I need to send it again. The TL;DR is that "rendering" might involve advancing state (eg. A particle system that does physics simulation on the gpu)
<riteo>
pac85: godot doesn't actually render when occluded, it just ticks some "rendering" methods in the scripts but otherwise it just normally loops
<ManMower>
pac85: ah, I missed the "particle system that does physics simulation on the gpu" - I guess that would be purely for aesthetics, so it's ok to cap that to the frame rate even if some other game loop is simulating at a different rate?
<ManMower>
I wasn't aware that a game using gpu for physics would tie such physics simulation to the frame rate. do I get better physics if I have a faster refreshing monitor?
<riteo>
I'm pretty sure that if it were framerate independent it could simply work in function of the time and such it could simply tick forwards once visible
<ManMower>
that's what I want to believe. :)
<riteo>
mh. I find this whole situation a bit underwhelming tbh
<pac85>
ManMower: depends what you mean. Like riteo see the game code is often exposed to a fixed rate update callback and a variable rate one that matches the framerate. As the game code in the variable update interacts with the renderer those two might be tied to together and might have to run at the same rate, but it really depends
<soreau>
I want to hover over the window-list button in the panel and have a miniature live window preview
<riteo>
this problem for some reason feels trivial yet we're still at the "just hack around it + lotta of protocol MRs" phase
<riteo>
(it isn't trivial)
<riteo>
I'm doing whatever I can but yet "ticking" at 2000 Hz looks like the simplest and most effective solution still, which is just dumb
<riteo>
I'm trying polling with time outs and I'm not even sure that all the setup to do that is actually less taxing than just polling at that rate
<riteo>
I have no idea what to do to improve the situation. I really want to help at unblocking this up, but the only thing I got last time was a lot of flaming and a closed MR IIRC :(
<ManMower>
can you point me at that MR? I'd like to see what was presented and why it was flamed.
<riteo>
yeah it was this one. I just asked how it was going and the thing imploded
<soreau>
kennylevinsen: just read code, not user comments :)
<riteo>
Correct me if I'm wrong, but I'm pretty darn sure that it was indeed technically wrong. It was the flaming that contributed to this underwhelming
<riteo>
feeling that I had
<pac85>
Also regarding the idea of an event driven loop in a game, a game will spend most of it's time in it's simulate and render phases meaning the event queue will fill up. This makes the whole event loop just an event pump that drains the queue in disguise. In order to begin simulation the game must have processed all the events that have occurred until that point. For example if a game is running lower than the refresh rate it might
<ManMower>
ah ok. I can see where that would have been a contentious MR.
<riteo>
indeed
<soreau>
pac85: your message cut off after 'it might'
<ManMower>
^
<pac85>
it might receive more than one frame event (at least in my understanding of wayland, correct me if I'm wrong) so starting rendering when a frame event is received would not work,
<pac85>
the whole queue must be drained necessarily to make sure there aren't more inputs following that frame event and other frame events. This makes it functionally equivalent to the classic loop but just harder to follow (I know before I've actually implemented sometjing similar to this)
<pac85>
I apologize
<pac85>
Thanks
<ManMower>
you won't receive "other frame events" unless you asked for them, so that's not really a problem
<riteo>
re: event pump equivalent, that's exactly why we did the wayland thread
<riteo>
most of the times the engine would stall while loading and make the fixed length "native" wayland event queue overlfow
<kennylevinsen>
pac85: just as I mentioned earlier, if the simulation is very intense there is no real difference between event driven and not as indeed everything will occur in the same order regardless. But that's not all games, or when e.g. Godot is used to write non-game apps like the game editor, or when people write text editors in SDL2
<riteo>
(I say native to differentiate from the "custom" one done in the wayland thread)
<kennylevinsen>
you can't get more than one frame event on Wayland (you subscribe to 1 callback on commit), but for others irs trivially handled with e.g. a flag and a idle callback
<pac85>
kennylevinsen: what would be the functional difference in other cases?
<riteo>
continuing the discussion above: what I can do to fix this? I want to help as much as possible to unblock this issue if possible
<kennylevinsen>
pac85: For the non-game cases, a well-behaved app that correctly uses 0 cycles when idle. For simple games, the simplicity of an off the shelf event loop that lets you be nicely responsive to I/O on a single thread.
<pac85>
ManMower: I see, though you might still have input events in the queue after the frame event so you still want to drain the whole queue then render if (have_seen_frame_event || if the player wants to run the game uncapped)
<ManMower>
kennylevinsen: "you can't get more than one frame event...", is that true? I thought you could invoke wl_surface_frame() as many times as you want before your commit, and you'll get a callback for each at the same time.
<kennylevinsen>
I'm mostly concerned with the intersection where these toolkits are used for productivity applications, with SDL2 in a text editor being why I'm pre-bugged by the issue...
<ManMower>
pac85: yeah... I'm not sure I'm qualified to have an opinion on how to handle that. you really do want to get them out of the queue before the compositor kills you, but which frame of the simulation you assign them to is not something I have a strong opinion on.
<kennylevinsen>
ManMower: you can but then you're literally asking for it so it's not an issue you need to design around. :)
<ManMower>
kennylevinsen: yup yup :)
<ManMower>
I can see doing that if for some reason you want a frame callback and mesa also wants a frame callback? shrug. :)
<kennylevinsen>
heh sure, but each listener sees 1 event then
<riteo>
kennylevinsen: re GUI apps, I suppose that an alternative might be to have a variable frame time event
<kennylevinsen>
Just more than one wl_callback listener firing
<riteo>
like, window is occluded, now the rendering driver syncs to 5 FPS
<riteo>
maybe
<ManMower>
I really think frame's semantics are good
<riteo>
no wait, I think that a suspended flag would be more flexible since we could control the ticking, a bit like we're doing in godot
<ManMower>
they're just not good for what some people would like to do with them
<ManMower>
so another mechanism makes sense
<riteo>
ManMower: indeed, I really like frame's semantics too
<kennylevinsen>
To clarify, I consider a productivity app having any tick rate above zero when idle (I.e., no input and no changing output) to be strictly a bug
<riteo>
kennylevinsen: I find that a bit hard, even for "native2
<riteo>
oops
<riteo>
"native" productivity apps
<riteo>
like, there's still networking poll and all of that
<ManMower>
I think that's "input" in this context?
<riteo>
And I doubt that godot will have _that_ level of optimization. GUI apps must be supported because the editor, but they're not "first class". If something is more beneficial to games than UI apps, it will be done to favour the former AFAIK
<kennylevinsen>
indeed, event loop takes care of that. It won't wake until something happens - if you get incoming traffic, it's of course fine to process.
<ManMower>
riteo: do you forsee writing something like an email client with godot? something I'll want sleeping on my laptop all day and not eating my battery life?
<kennylevinsen>
It's absolutely doable in a complex GUI app - event loops even make this simple, because you register interest and only wake if needed - no wake to *see* if it's needed
<riteo>
ManMower: yup
<ManMower>
then you'll have to do better than constant 2000hz ticking :)
<kennylevinsen>
the main blocker is gui's that always animate, e.g. pulsing animations
<riteo>
ManMower: absolutely, that's the underwhelming part :(
<kennylevinsen>
riteo: but trivial to make GUI apps that have 0 tick rate and spend 0 cycles != trivial to retrofit existing framework
<kennylevinsen>
not suggesting your work is going to be easy :P
<riteo>
I'm (and have been) trying a shitload of solutions, but I'm starting to give up from finding a local solution, as JoshuaAshton suggested
<kennylevinsen>
at times, a working solution > a good solution
<ManMower>
I'm not sure how much of that solution will be part of the wayland integration?
<riteo>
I'm starting to feel like it's impossible to have "render-driven" logic rn and I really want to help whenever I can
<kennylevinsen>
also need to consider your own voluntary time and remaining sanity
<riteo>
kennylevinsen: that's the thing though, no solution truly works
<riteo>
the 2000Hz is actually the simplest and most robust. I tried also the timeout stuff and found negligible differences in a debug build
<riteo>
there's _no_ reliable way to get proper gpu frame timing information from what I can tell
rtjure_ has quit [Ping timeout: 480 seconds]
<riteo>
there's also presentation time but it's not enough for that AFAICT
<kennylevinsen>
depends on what timing you're looking for. You can observe the interval, but it might wobble if e.g. VRR is a thing. You don't currently know the precise *latch* time, only the present time though.
<ManMower>
what I'm trying to do is a combination of commit-queue and presentation time, so I can have a presented event if I'm visible, but I'll also get a discarded event if I'm not. but this would still be pinging off the compositor at <refresh rate>fps, which I think is ok for games, but less so for a productivity app
<pac85>
kennylevinsen: for a simple game you migjt have a loop containing another loop that drains the queue, runs simulation/rendering then waits on either the aquire fence or uses a semaphore for presentation and waits on some other fence. (which really means running an event loop again, this time in wsi) this does not necessarily involve more than one thread (look at vkgears for a pratical and minimal implementation of this that works
<riteo>
ManMower: uhhh that's interesting
<riteo>
so a mix of frame semantics and a "fallback" tick embedded in the graphics stack?
<riteo>
(or anything implementing eglSwapBuffers, VK ANI or whatever)
<ManMower>
yeah, fallback tick seems like a reasonable way to think of it
<riteo>
that's really interesting. I wonder, who will decide the final solution? There are _lots_ of proposals
<riteo>
like, will there be a poll or something? I can help with testing implementations and perhaps even make ones themselves if I can manage to and it's needed.
<ManMower>
there's a betting pool somewhere I'm sure
<kennylevinsen>
pac85: indeed, with that block on a fence causing the infamous engine blocking, and stealing available CPU time from the main thread. I feel like an event loop that handles I/O, has a timer for any fixed rate engine advance if needed, and a frame callback for rendeirng is simpler and cleaner tbh
<kennylevinsen>
riteo: fallback tick sounds like something we discussed yesterday... :P
<riteo>
kennylevinsen: the issue is that there's no integrated way to do it at a native, "polled" rate
<riteo>
to bring back yesterday's terminology
<ManMower>
just throw a timerfd in your event loop
<ManMower>
based on the refresh rate in a pres feedback event
<riteo>
uh
<ManMower>
what could possibly go wrong? :)
<riteo>
I wondered if something like that existed but I didn't think that it would
mvlad has quit [Remote host closed the connection]
<riteo>
Perhaps that could work, mh
<riteo>
But I suppose that it would still just be a workaround, wouldn't it?
<riteo>
like, the issue I'm noticing is that we have no reliable way to get the correct timing info "out of the gpu"
<riteo>
and still, it should be the ANI/swapbuffer call that does this thing I think
<riteo>
I suppose we're talking about latch timehere
<ManMower>
there's a proposal somewhere to get latch time, but it's not landed. :(
<pac85>
kennylevinsen: if I read this right you are basically saying that you could save on the time spent pumping events by doing that as the events arrive, is that correct? However you still need to somehow wait for the gpu to finish it's work so starting the frame in response to the frame callback doesn't seem like an option to me. Waiting after the frame callback doesn't work because it would mean missing events that arrjve while wait
<kennylevinsen>
that's also only needed for advanced frame scheduling - if a simple frame event is sufficient, that is meant as a "start now" event
<kennylevinsen>
riteo: note that you do *not* need accurate timing when the fallback is in effect
<riteo>
tbh I'm not completely sure if that's the right solution. Like, otherwise we wouldn't have a problem if we could use the presentation time protocol?
<kennylevinsen>
the use of the fallback implies *not* being visible, so latch timing have no meaning - nothing is going to latch after all
<riteo>
kennylevinsen: wait a sec. The issue is that fetching the frame event, sending and waiting it to be "realized" by the main loop is too unreliable
<kennylevinsen>
pac85: for a simple game you don't wait for the gpu to finish, you just render on frame event.
<riteo>
So, I think that the solution would be to fix the "wait" parts of the graphics stack. no?
<kennylevinsen>
riteo: the "wait" part of the graphics stack is a frame event
<kennylevinsen>
the graphics stack calls wl_surface_frame, registers a listener and dispatches the wayland socket until it arrives
<pac85>
kennylevisen: wgat do you mean? You need to acquire a swapchain image which means eaitinf on a fence (or on the gou to finish and waut on the aquire semaphore in the submission)
<riteo>
yeah, but it does that _there_, instead of another thread
<pac85>
Sorry I tagged the wronf perdon above
<riteo>
perhaps I could have a separate queue somehow and update it in the main thread, right before rendering?
<pac85>
Not Man Power but riteo
<riteo>
pac85: oh that's pretty
<ManMower>
timerfd is pretty nice
<riteo>
I just did a dumbity dumb "deltaTime / repeat time" thingy
<Company>
the question what you should be waiting on is mostly about what you're bottlenecked by, isn't it?
<kennylevinsen>
pac85: I am not well-versed in vulkan (yet), but the only reason for a swapchain to wait is if it's out of buffers which shouldn't happen if you're rendering throttled to frame events for a simple game...
<kennylevinsen>
(and if it was out of buffers you'd wait on a buffer release, which an event loop can do for you :P)
<riteo>
Company: good point
<Company>
kennylevinsen: the biggest difference with swapchains is that the application controls the number of images
<pac85>
Company: I think it gets more complex when we talk about the various present modes, for fifo I think the condition is "the refresh cycle after the aquire fence is ginaled has occurred"
<Company>
kennylevinsen: while GL can just allocate new buffers
<kennylevinsen>
yeah, that's also what we do (manually) in wlroots - a fixed swapchain with a max len of 4
<riteo>
so, to recap, rn since we have to poll a separate thread there is synchronization and everything else between event handling and drawing running pretty darn constantly (e.g. physics if necessary, vr logic, process logic and whatnot)
<kennylevinsen>
(4 because of the direct scanout corner-case)
<riteo>
by waiting right for an image on, well, ANI, we can time most of our logic at the most convenient moment
<pac85>
kennylevinsen: the vulkan spec requires to wait somehow, I suppose you could yolo the synchronization but that seems like a bad thing even for a simple game
<ManMower>
riteo: ooi, does godot have some way to tell you "there's nothing exciting in the physical simulation, so feel free to sleep until you've got new stimulus for me"?
<riteo>
ManMower: not sure. Might be some internal cleverness
<Company>
pac85: I was mostly thinking about GTK - and depending on the app that's running, we can run into GPU load, CPU load (in the toolkit and in the application) and even compositor load being the thing we want to optimize for
<kennylevinsen>
pac85: doing a call that is defined to allow blocking, but never will because of your timing in response to frame events is fine
<riteo>
we just do something like "main loop stuff -> update physics -> other main loop stuff"
<riteo>
lemme check
<ManMower>
just wondering about that productivity app use case, where things should really settle down for external stimulus instead of ticking at all
<ManMower>
(or my laptop will set my backpack on fire)
<riteo>
ManMower: it just does "for each physic step that we have to do update physics"
<riteo>
I suppose that it would do nothing on a productivity app outside of calling `_physics_update` on any node that wants to
<Company>
kennylevinsen: one fun thing I was thinking about with the 4 number was VMs - the VM might want to have multiple buffers outstanding in the host and if you want to direct-scanout through that whole stack, you may need more buffers
<riteo>
which is the fixed loop stuff
<pac85>
Company: Uh I see, then I guess it sounds reasonable but I'm not knowledgeable at all about GUI apps
<kennylevinsen>
Company: hmmm, but that would imply some KMS changes. There's only one committed buffer per plane at at ime after all, and no "buffer release" events.
<kennylevinsen>
Just "page flip completed", which imply that the last configuration took effect
<kennylevinsen>
so client -> compositor -> KMS should, for now, be the whole stack to consider
<Company>
kennylevinsen: so basically KMS is a swapchain with a hard limit of 2 buffers
<Company>
?
sima has quit [Ping timeout: 480 seconds]
<pac85>
I don't think there is such an hard limit, afaik you can alloxate and use as mamu buffers as you want
<pac85>
I'm pretty sure you can implement mailbox on top of it
<Company>
kennylevinsen: asking because I'm wondering what the right approach for the VM application should be when it gets the buffer from qemu - sounds like it should immediately do a copy to free up the buffer for KMS again?
* Company
no clue about KMS
andreasbackx has joined #wayland
<kennylevinsen>
Sorta - you have the active configuration, and the pending/in-flight configuration. It's not a swapchain because the next buffer could be anything from anywhere, as it doesn't concern itself with rendering.
<Company>
because now that we work on passthrough everywhere, we could technically achieve full scanout passthrough for a maximized app inside a maximized vm
<ManMower>
what's "passthrough" in this context?
<Company>
ManMower: no copies
<ManMower>
ah ok, when I hear "passthrough" in a vm/graphics context, I think we're talking about handing a pci device to a vm, where none of this is a problem
<Company>
yeah, that term is overloaded
<ManMower>
indeed :)
<Company>
basically, it's about the fullscreen app inside the vm allocating the buffer that's ultimately used for direct scanout on the host
<kennylevinsen>
The direct-scanout 4-buffer edge-case is: 1 buffer being scanned out, 1 buffer committed to replace it, 1 buffer just sent to the compositor after the atomic commit was done (atomic commits cannot be ammended), and 1 buffer for the client to render to.
<Company>
right, but with VMs, "the compositor" becomes "the compositors"
<ManMower>
with nested VMs it's compositors all the way down?
<kennylevinsen>
If the host does direct-scanout with the VM's buffer, I think it just prolongs the timing for the first two - it doesn't add another stage
<kennylevinsen>
not sure what qemu-virtio does, but I imagine it would need to delay the completion of the atomic commits or something
<kennylevinsen>
or maybe they just take the short-cut of copying the buffers, idk
<kennylevinsen>
on virgl/virtio-gpu stuff? no idea :S
<riteo>
you know what, I think indeed as Company pointed out that we should identify the actual problem
<kennylevinsen>
+1 to that
<Company>
the cirtio/virgl hackers seem to be a mystery to anyone
<riteo>
why do we need a separate protocol, why can't we just unlock ANI after a default timeout? Is there some other reason against that? Maybe the problem is more related to what the program knows
<riteo>
Let's take a step back
<ManMower>
what does "unlock ANI" mean to a wayland compositor?
<riteo>
frame
<riteo>
but that's the important thing. We need this check to be as close as possible to ANI I think, because we want to have as much time as possible to process a guaranteed rendering frame
<riteo>
now, we either tell the client "uh, you won't get frame events anymore, you're on your own", or we add a default timeout to e.g. ANI for vk
nerdopolis has quit [Ping timeout: 480 seconds]
<riteo>
(ftr, ANI is AcquireNextImage)
<ManMower>
oh, I didn't realize that's what you meant by that.
<riteo>
sorry
<riteo>
should've made that clearer
<ManMower>
thought it was some shorthand for animation that I was unfamiliar with
<riteo>
not sure where I heard that acronym but I'm pretty sure that it's used in this context somehow
<riteo>
again, I'm not _that_ educated on this stuff
<riteo>
I think that the issue is twofold
<riteo>
Despite Wayland's philosophy, we _want_ to know if we won't get frame events, as we might want to force our own tick rate independent of the "gl stack waiting stuff" (ANI/eglSwapBuffers)
<riteo>
_But_, and that's important, we also want to get our framerate manipulated by the compositor, for power saving
<riteo>
I think that explains the logic behind !199
<ManMower>
I think for power saving the compositor will want to go to sleep and not tell you to do anything at all
<riteo>
ManMower: re MR, that would make the framerate "consistent" even when occluded?
<ManMower>
yes
<ManMower>
you don't get throttled by the compositor in any way really
<riteo>
and how are "frame" dependent applications, like UI productivity ones, affected?
<ManMower>
"fine, you want to draw? draw. I don't care."
<ManMower>
I haven't paid much attention to that use case, tbqh
<ManMower>
they're get pulled along for the ride
<ManMower>
*they'll
<ManMower>
but I'd think productivity apps shouldn't be blocking in ANI 60x per second
<ManMower>
you shouldn't need a compositor to tell you there's no new email to display
<riteo>
fair
<riteo>
not even godot does that in "low power mode"
<kennylevinsen>
you know you've done something wrong if you check for new emails on frame callback...
<ManMower>
:D
<ManMower>
I'm on a lot of lists, I get a lot of email, polling at 60fps actually reduces overhead. ;)
<kennylevinsen>
"help, my network broke after I got my new 500Hz monitor"
<ManMower>
haha
<riteo>
the annoying thing is that there's this design decision behind wayland's graphic stack of blocking on frame which, despite making a lot of sense for most apps (especially embedded ones), breaks gpu-intensive apps
<kennylevinsen>
the blocking on frame is optional, and for application that actively ask for it. The problem is that it doesn't quite match the spec.
<kennylevinsen>
but you can always just *not* block
<riteo>
ManMower: I picked the reference :P just to be clear, it's not a capacity thing, instead the "latency" issue where we're doing a shitload of stuff before checking if we need to draw and so on, missing lots of deadlines
<riteo>
kennylevinsen: yeah, but we're just turning off a mechanism without any fallback solution
<riteo>
not even the graphic stack has a valid solution either, hence the black-or-white situation
<riteo>
it's not like turning it off we can do much different outside of hacks
<kennylevinsen>
riteo: I wouldn't say "not even" - not blocking/having fallbacks is trivial to do *outside* the graphics stack/WSI. It's only really an issue *because* it's in the WSI...
<kennylevinsen>
(well, knowing in advance that frame callbacks will no longer arrive would be nice in general though)
<ManMower>
we just need some AI to tell us when they'll start arriving again. ;)
<riteo>
lmao
<kennylevinsen>
predicting when the user will unminimize a window, hmmmm
<riteo>
kennylevinsen: re "I woulnd't say not even..." not sure what you mean by that
<kennylevinsen>
riteo: an application can have whatever logic for frame events it would like - waiting for the event, having a timeout, changing to background mode if the window is suspended even if frame event has not arrived. The WSI needs to stuff it into e.g. the pre-exisitng eglSwapBuffers API that just has to block until done.
nerdopolis has joined #wayland
<kennylevinsen>
(non-WSI land is entirely asynchronous)
<riteo>
mh
<riteo>
then perhaps I shouhld just handle the frame event differently right before drawing and block there?
<riteo>
would that make sense?
<kennylevinsen>
That's basically what eglSwapBuffers does on the inside - if the previous frame callback has not arrived yet, wait for it
<kennylevinsen>
but when you do it on your own, you get to have whatever control you want
tlwoerner__ has quit [Read error: Connection reset by peer]
tlwoerner__ has joined #wayland
<ManMower>
still using egl, you could set swapinterval to 0, and do your own frame callback shenanigans. I think SDL does this?
<riteo>
yeah we do too actually
<riteo>
that's the annoying thing though, how and when to do that
<riteo>
I suppose I could make a "main-threaded" frame-only queue
<riteo>
would that work? How would I commit the surface? This whole multiqueue shenaningan stuff is not really documented
<kennylevinsen>
the thread safety is mainly about where you register listenres. Unless you use multiple queues, all listeners will fire from whereever wl_display_dispatch is called.
<riteo>
I'm still genuinely confused on whether this is an architectural issue (thus, impossible without mesa patches) or a Godot thing in my case
<riteo>
JoshuaAshton seems to point towards the former, while y'all to the latter
<kennylevinsen>
but writing is synchronized, so you can e.g. wl_surface_commit(); wl_display_flush(); from wherever.
<riteo>
yeah but the WL Thread is, well, a thread
<riteo>
and the rendering is on the main thread
<riteo>
if we want to handle frame events like the GL Driver to avoid latency and maximize time for processing, I suppose we should make a main-thread queue, no?
<kennylevinsen>
When you use a queue, it means that wl_display_dispatch fills the other queues but does not dispatch them. Then you need to call dispatch on those queues afterwards on other threads
<riteo>
mh all right
<kennylevinsen>
but if you need to signal for the main thread to do something, you might as well just signal the frame event directly instead of "dispatch my queue"
<Company>
the more evil thing is that the GL thread can call wl_surface_commit() while the main thread is partially through setting up something else
<riteo>
so we're back to square one
<riteo>
My brain is melting a bit rn
<kennylevinsen>
I wonder why you need the wayland thread if the main thread can tick as fast as previously mentioned
<kennylevinsen>
if that solves your event issues too :/
<ManMower>
Company: that seems like user error to me. I don't think you should be doing surface state changes from one thread while another is doing anything (swapbuffers/presentimage) that will lead to a commit. that's a mutex mistake?
<kennylevinsen>
(you can still render on another thread)
<riteo>
kennylevinsen: because the main thread can stall. If we had also wl logic there it'd crash because of the fixed size event queue
<riteo>
also responsiveness
<Company>
ManMower: I't think that's something that can easily happen when you have a "main thread" and a "rendering thread"
<ManMower>
Company: being an easy bug to write doesn't make it not a bug?
<Company>
sure
<riteo>
I'm seriously confused. What's the problem space? I can't wrap my head about the nature of this issue for the life of me
<ManMower>
really feels EDONTDOTHAT to me. is that a case where it's completely unavoidable?
<Company>
it's an ugly design choice
<riteo>
sounds so damn trivial yet I've wasted a considerable amount of time trying to work around it
<Company>
forcing the whole wl_surface state into the rendering thread
<kennylevinsen>
riteo: hmm, maybe you could have the wl thread just read into queues but not dispatch
ebassi has quit [Ping timeout: 480 seconds]
<kennylevinsen>
then the main thread does all dispatching but the wl thread made sure you didn't die
<kennylevinsen>
it's a very random idea late at night, but it's a thought
<riteo>
I think it'd still overflow
<ManMower>
that's still a small ring buffer, isn't it? it's just not the *tiny* socket buffer
<riteo>
yup
<riteo>
that's the real annoying thing
<riteo>
but still, I'd argue that it wouldn't fix much
<riteo>
you know what. Lemme try turning again the whole egl swap interval thing on and see what happens when it comes to frame times
<riteo>
like, let's see what mesa does
<ManMower>
well, swapinterval > 1 is just 1. I think swapinterval == 0 will get you mailbox without tearing.
<kennylevinsen>
It's not the connection ring buffer if you call read_events, it's a linked list of pending events
<ManMower>
so really, other than swapinterval == 1, nothing seems to really be in spec there?
<ManMower>
kennylevinsen: oh, nice. I should've remembered that but didn't.
<riteo>
I'm using the VK layer for getting a frame time graph
<ManMower>
ah
<ManMower>
I'm not really sure what to expect from that
<riteo>
I'd have expected a nice flat graph for an empty scene on vsync
<riteo>
maybe it's really just the fact that it's a debug build?
<riteo>
shouldn't matter that much, should it?
<kennylevinsen>
for eglSwapInterval >= 1, only if you render immediately as the swap returns
<riteo>
now I'm even more confused
<riteo>
I'll have to test an optimize build because I refuse to believe that we can't get solid vsync on an empty scene
<riteo>
If it works fine on XWayland but not Wayland then something is really wrong
andreasbackx has quit [Remote host closed the connection]
<riteo>
oh dear lord I think I'm missing something. I'm really sorry.
nerdopolis has quit [Ping timeout: 480 seconds]
<riteo>
I swore I saw different frame times with the "plain" vsync but it's been a while. I'll investigate properly to gather further data.
Moprius has joined #wayland
Moprius has quit [Quit: bye]
nerdopolis has joined #wayland
ebassi has joined #wayland
<riteo>
all right. I've got some slightly clearer ideas
<riteo>
I'm really sorry for the newbie question but I think that it's relevant about this discussion: I'm getting a "jitter" of about 1-2 ms compared to the vsync time (16.666) I believe
<riteo>
I've built an optimized copy of vkcube and through the mesa vulkan layer I can get this behaviour even at low resolutions
<riteo>
I swear that I would get a flat graph with vsync on X11. Am I just ignorant or is this normal behaviour?
<riteo>
Either Mesa's overlay is jittery or the frame event is not very accurate for GPU intensive situations indeed as JoshuaAshton said