Canvas/context inversion of control

[Korijn]

I know this branch is hideous and broken, but I wanted to put it up in draft anyway to communicate something that's bothered me about wgpu-py's API from the start. It's the hidden coupling between wgpu-py and the GUI layer. This self-referential class object thing just feels wrong. Below is an AI generated summary that I thought helps to illustrate my thoughts on this subject.

You can check out this branch and run the gui_direct.py example to see that it works (including resizes).

My main intention of opening this draft PR is to initiate a discussion with you @almarklein :)

In my opinion it's really not worth it to litter the codebase with class to get_context via a canvas weakref via a wrapper object, just to be able to call get_physical_size, and require all users of wgpu-py to work with this awkward class interface.

I strongly believe that implementing this decoupling API change will make wgpu-py much easier to apply in a wide variety of applications using libs such as glfw, sdl and so on.

Summary

Replace the implicit self-referential canvas/context wrapper with a plain, caller-owned GPUCanvasContext. The application is now responsible for window/canvas lifetime and must explicitly supply canvas-related state (for example, physical framebuffer size) to the context.

What changed

GPUCanvasContext no longer depends on being wrapped by a canvas object that calls back into itself.
Applications may construct a GPUCanvasContext directly and must call context.set_physical_size((w, h)) and update it on resizes.

Example: gui_direct.py creates GPUCanvasContext directly and calls set_physical_size() from GLFW frame-buffer queries.

Rationale

• Inverts control: the application manages window/canvas state instead of the context pulling it implicitly.
• Simplifies interop with GUI toolkits where the app already owns the window.
• Makes responsibilities explicit and reduces hidden coupling between GUI and wgpu-py.

Example (conceptual)

• Before: canvas -> canvas.get_context("wgpu") -> context pulled size callbacks
• After: app owns window; app creates context; app calls context.set_physical_size(...) on resize.

[almarklein]

Thanks @Korijn!

Executive summary: I agree!

I should start by saying that I share your feeling that the context feels somewhat awkward. Although for me the confusion lies more in how it ties rendercanvas and wgpu-py together, in a way that does not feel quite right yet ...

Have you seen pygfx/rendercanvas#124? In short, it proposes that rendercanvas implements its own context class, and (only if necessary) wraps a wgpu.GPUCanvasContext. It fits quite well with your idea, in the sense that rendercanvas and wgpu-py are more independent, and can each decide on their own API for the the context (or even drop the concept of a context?).

The main reason why our GPUCanvasContext is the way it is, is because it follows the JS/WebGPU GPUCanvasContext API. But as I am composing this reaction, I've come to realize 💡 that this API has quite some historic influence, and is geared to the situation in the browser. And we can certainly ask ourselves whether its worth to strictly follow that API. Spoiler alert: I now think its probably not.

To go in a bit more detail ... in JS, both the CanvasRenderingContext2D and the WebGLRenderingContext are objects that have a lot of methods, which are used by user-code to perform the rendering. These objects get passed around a lot, and it's very convenient for such code to then also be able to access the canvas. The GPUCanvasContext was likely given a .canvas method mostly for compatibility with these other contexts; it has only a few methods which are used in a very specific places. Further, the only canvas in a browser is a <canvas> which is owned by the browser and a rather opaque object for the user. In contrast, in Python the "canvas" can be many different things, and is managed outside of wgpu-py.

With that said, I think we can justify adopting a different API for wgpu-py. One that makes wgpu-py oblivious of any canvas, and that makes code outside of wgpu-py responsible for keeping the context up-to-date. That makes cases like gui_direct.py simpler, and rendercanvas can just abstract such boilerplate away for the user. 🚀

A few more practical comments:

• I think we can drop the get_physical_size() since the code that handles the context knows the physical size, or calls get_current_texture() and checks its size. The context needs to know the physical size internally because it's needed to (re) configure the context.
• Let rendercanvas provide its own context objects rendercanvas#124 means that the wgpu.GPUCanvasContext will only be used for presenting to screen, so the present_methods dict will be replaced with a dict that contains a smaller dict that conains the platform and winid (which we use to get the surface id).
• The one other thing that the context gets from the canvas is the _vsync prop. We can add that to the aforementioned present-dict.
• I'll think about the best order in which to apply these changes 🤔

[Korijn]

Executive summary: I agree!

👍 Sweet!

Have you seen pygfx/rendercanvas#124? In short, it proposes that rendercanvas implements its own context class, and (only if necessary) wraps a wgpu.GPUCanvasContext. It fits quite well with your idea, in the sense that rendercanvas and wgpu-py are more independent, and can each decide on their own API for the the context (or even drop the concept of a context?).

It seems okay but it depends on the implementation details I think. RenderCanvas should be free to choose any API it wants to, because the wgpu-py API allows for it, without imposing requirements on the implementation details. Ideally wgpu-py just provides some public functions and a context object with public members, and you can work with that in any way you like.

The main reason why our GPUCanvasContext is the way it is, is because it follows the JS/WebGPU GPUCanvasContext API. But as I am composing this reaction, I've come to realize 💡 that this API has quite some historic influence, and is geared to the situation in the browser. And we can certainly ask ourselves whether its worth to strictly follow that API. Spoiler alert: I now think its probably not.

To go in a bit more detail ... in JS, both the CanvasRenderingContext2D and the WebGLRenderingContext are objects that have a lot of methods, which are used by user-code to perform the rendering. These objects get passed around a lot, and it's very convenient for such code to then also be able to access the canvas. The GPUCanvasContext was likely given a .canvas method mostly for compatibility with these other contexts; it has only a few methods which are used in a very specific places. Further, the only canvas in a browser is a <canvas> which is owned by the browser and a rather opaque object for the user. In contrast, in Python the "canvas" can be many different things, and is managed outside of wgpu-py.

I get that! 💯 In the case of libs like glfw, sdl2, imgui, there's no concept of a "widget" or "window" or "canvas" tired directly to an OOP model like there is in the browser or in PySide. I think we need to realize that the latter is a higher level abstraction than the former. We need to make sure our API supports the lower level style first (primary priority), so that the higher level style can be built on top (secondary priority).

With that said, I think we can justify adopting a different API for wgpu-py. One that makes wgpu-py oblivious of any canvas, and that makes code outside of wgpu-py responsible for keeping the context up-to-date. That makes cases like gui_direct.py simpler, and rendercanvas can just abstract such boilerplate away for the user. 🚀

Music to my ears. :)

A few more practical comments:

• I think we can drop the get_physical_size() since the code that handles the context knows the physical size, or calls get_current_texture() and checks its size. The context needs to know the physical size internally because it's needed to (re) configure the context.

I tried to implement this in this branch, but I think I broke the bitmap present method in the process. In any case it seems to work just fine for the screen present method! The gui_direct example still runs just fine, including resizing. 🎉

• Let rendercanvas provide its own context objects rendercanvas#124 means that the wgpu.GPUCanvasContext will only be used for presenting to screen, so the present_methods dict will be replaced with a dict that contains a smaller dict that conains the platform and winid (which we use to get the surface id).

I guess this is where I got things mixed up while attempting to implement this. :) It needs some more untangling then I guess.

• The one other thing that the context gets from the canvas is the _vsync prop. We can add that to the aforementioned present-dict.

I implemented this as well! Is it correct that vsync cannot be changed during a reconfigure?

• I'll think about the best order in which to apply these changes 🤔

That would be appreciated!

---

A thought I had while working on this; there's only a handful of spots left where canvas is passed into wgpu-py's API. And it's always just to make a call to canvas.get_context. Should we just change the argument to let users provide a context object instead of a canvas? It would make wgpu-py 100% unaware of canvases.

[almarklein]

Is it correct that vsync cannot be changed during a reconfigure?

Actually, we could, but the current API to set vsync on canvas instantiation together with update_mode and max_fps probably makes more sense.

I'll think about the best order in which to apply these changes 🤔

That would be appreciated!

Proposal:

• In Implement own context classes rendercanvas#127 I implement context classes in rendercanvas. It will replace the whole bitmap-present logic in wgpu-py, and only wrap the wgpu.GPUCanvasContext for present-method 'screen'. I'll make it such that it can work with the current API as well as the changes in this PR.
• Release rendercanvas
• Then in this PR we add a dependency on rendercanvas with that release. The dependency is temporary to make sure ppl use the right version while we transition to this new system.
• In a few months we can probably remove the hard dependency on rendercanvas.

[almarklein]

Should we just change the argument to let users provide a context object instead of a canvas? It would make wgpu-py 100% unaware of canvases.

For request_adapter() I think it definitely makes sense. We can still support "an object that has get_context()" for backwards compat. Are there any other cases?

[Korijn]

If your hands are itchy feel free to take over this branch...

[Korijn]

Semantically speaking, why is the class called GPUCanvasContext? Why isn't it just GPUContext? Is it in reference to a "canvas on the GPU"?

[almarklein]

Semantically speaking, why is the class called GPUCanvasContext? Why isn't it just GPUContext? Is it in reference to a "canvas on the GPU"?

That's just what the WebGPU spec calls it. In JS it makes perfect sense. For wgpu-py we try to ban the word 'canvas' from the codebase 😉, and we change its API somewhat, but it's still the same concept that we're exposing, so I'd prefer to keep the name the same as in the spec.

[almarklein]

Locally confirmed to work with pygfx/rendercanvas#127. The plan:

• Merge Implement own context classes rendercanvas#127
• New release of rendercanvas
• Merge this
• New release of wgpu.

[Korijn]

[Vipitis]

I think this might allow wgpu to run with pyodide even without rendercanvas (minus the events). I could give that a try on the PR branch.
A recent wgpu update added WGPUSurfaceSourceSwapChainPanel and from my understanding of the linked docs, this should allow for a more direct integration on windows apps. Might be an option for a new example.

[almarklein]

I think this might allow wgpu to run with pyodide even without rendercanvas (minus the events). I could give that a try on the PR branch.

Definitely! Actually, I should probably just document what should be in the struct already.

[Korijn]

I can't approve this cause it's my own PR :) so feel free to approve and merge when you are ready for it