Support for transparent video

[jam-le]

When filing a feature request:

Fill out the sections below, leaving the headers but replacing the content. If
you're unable to provide certain information, please explain why in the relevant
section. We may close issues if they do not include sufficient information.

Playing Transparent Videos (with the transparency intact)

iPhones allow videos with transparent backgrounds to play. I would like ExoPlayer (on Android devices) to support videos which have an alpha channel. Currently, it is impossible to play high quality transparent videos that retain their transparent properties (i.e. their alpha channel properties).

Proposed solution

None, unfortunately.

Alternatives considered

Chromakey is one alternative solution, but tends to create grainy resolution around edges where transparency meets solids, along with other subtle issues.

[AquilesCanta]

I think for this to work you'll need to use a decoder that supports alpha channel, plus a texture view.

I am not sure whether any of the platform decoders supports alpha channel though, but it's possible to know this through CodecCapabilities: https://developer.android.com/reference/android/media/MediaCodecInfo.CodecCapabilities#colorFormats. If you cannot find a video codecs with the desired color format, you can bundle your own software decoder in your app. ExoPlayer already supports VP9 and AV1 and the ffmpeg video extension is in progress.

With a texture view and a alpha-channel-supporting decoder, this should work. Note however, that this is not something we'll be able to include in our roadmap to support out of the box any time soon, soo you'll need to do some research on your own.

@andrewlewis do you have any input on this?

[AquilesCanta]

Aside, a sample video would be welcome. It will help whoever tries to explore this line of work.

[andrewlewis]

To add to @AquilesCanta's suggestion about decoding via extensions, #2509 discusses rendering videos with an alpha channel using the vp9 extension. I think it's still the case that Android platform decoders don't support this.

[md84419]

+1 for chromakey support please

[md84419]

Aside, a sample video would be welcome. It will help whoever tries to explore this line of work.

See http://test.bsl.nrw/obv-client-demo/boy-test.html

The webpage uses the seriously.js JavaScript library from https://github.com/brianchirls/Seriously.js/ which in turn uses WebGL. The code is distributed under the MIT Open Source license. The chromakey implementation is at https://github.com/brianchirls/Seriously.js/blob/master/effects/seriously.chroma.js

Both videos are Copyright 3rd parties. The URL for the video is http://test.bsl.nrw/videos/signer.mp4

[lilinxiong]

@jam-le Hi～ I encountered the same problem as you, have you solved it now? In other words, is there a solution?

[md84419]

No solution yet. This issue is assigned to @andrewlewis.

[ColtonIdle]

Here is a video with transparency to use as a sample. https://static.videezy.com/system/resources/previews/000/048/733/original/200515-AlphaBluePlanet.mp4

[mirswith]

That alphaBluePlanet.mp4 video does not seem to have transparency (tested on iOS). This sample provided by Apple however does include a properly encoded video with alpha: https://developer.apple.com/documentation/avfoundation/media_playback_and_selection/using_hevc_video_with_alpha , included in the download is the file puppets_with_alpha_hevc.mov, you can convert this to an mp4 while retaining alpha with ffmpeg -i puppets_with_alpha_hevc.mov -codec copy puppets_with_alpha_hevc.mp4. I have tested this and confirmed the alpha is retained when playing on iOS.