Memory Management of VideoFrame image buffers

[FirefoxMetzger]

This issue started from a comment in a PR: #788 (comment)

I'm wondering about the memory ownership/management model in PyAV. For example, the following works without any complaints:

import av
import numpy as np
frame = av.VideoFrame(16, 16, "rgb24")
no_copy_array = np.frombuffer(frame.planes[0], dtype=np.uint8).reshape((16, 16, 3))
del frame  # will this be dangerous?
no_copy_array[:5, :5] = 42

However, I wonder if this is actually undefined behavior because FFmpeg/libav doesn't know about no_copy_array. Hence, when I call del frame (which internally calls av_freep) then the buffer should be freed and no_copy_array should point into unallocated memory which is undefined behavior. I suspect a similar situation would happen if the buffer came from container.decode() instead of me creating it manually (in which case the default would be that the buffer is internally ref-counted and freed by FFmpeg).

So my question is if PyAV does any explicit memory management here, and if not if it relies on python's gc or ffmpeg's gc (or a mix of the two).

Edit: I just played around with this on something more sizable (5GB worth of frames). and it correctly allocates and frees memory. This isn't a definite answer, but perhaps a first step towards it.

>>> import av
>>> import numpy as np
>>> shape = (int(1e3)*16, int(1e3)*16)     
>>> frames = [av.VideoFrame(*shape, "rgb24") for _ in range(10)]
>>> arrays = [np.frombuffer(frame.planes[0], dtype=np.uint8).reshape((*shape, 3)) for frame in frames]
>>> for array in arrays:
...     array[...] = 4
... 
>>> np.frombuffer(frames[0].planes[0], dtype=np.uint8)[:10]  # proof that buffers are shared  
array([4, 4, 4, 4, 4, 4, 4, 4, 4, 4], dtype=uint8)
>>> del frames  # buffers are always freed after the second del
>>> del arrays  # order doesn't matter here