I got the hardware video decoder working on my firefly/rk3288. I can blit the yuv/nv21 output frame to rgb frame memory using fireflys dedicated 2d acceleration hardware. All I need right now is a memory pointer, pointing directly to the texture memory, in order to use the decoded video frame in shaders. In android I would create a GraphicBuffer object and give it to eglCreateImageKHR() and use glEGLImageTargetTexture2DOES. Right now I use glTexSubImage2D to copy and I get about 12 fps for 1920x1080 frames, too slow for showing a movie.
What is the best way to do this for firefly linux 3.10 with fbdev mali t760 driver? How can I simply get a pointer sharing CPU/GPU texture memory? And is there maybe some sample code available?
Right now I stuck and waiting for X11 mali drivers that support XCreatePixmap that could hopefully bring me further. My goal is XBMC, mplayer, ... running on linux on firefly.
Any help would really be appreciated! TNX!
Mac
Background, see:
accelerated video / video processor (VPU) running on linux on RK3288 / firefly - FreakTab.com
Hi Anthony,
Thanks for your answer, very helpful!
rk3288 video decoder specs state that decoders output format is YCbCr 4:2:0 semi-planar; that is the same as NV21.
Right now I use other dedicated blitter hardware on the rk3288 for this 2D hw accelerated conversion to GL_RGBA/RGBA888.
I dont know if eglImage functions for fbdev support NV21 to RGBA conversion internally that are efficient, deducing from your code it seems it does.
In my blitter conversion code I use RK_FORMAT_YCbCr_420_SP as source formart, and in xbmc's code on android I see that HAL_PIXEL_FORMAT_YCrCb_NV12 and HAL_PIXEL_FORMAT_YCrCb_NV12_10 are stated (the latter probably for dedicated HEVC decoder).
As destination format I use RK_FORMAT_RGBA_8888.
Currently my textures have GL_RGBA format.
(As a background thought: I also assume using an additional framebuffer /dev/fb1 formatted NV21 as overlay with rk3288's hardware composer might possibly work too and speed things up, especially for 4k/60fps.)
So if possible, could you give me the magics for both formats:
1) NV21 = YCbCr 4:2:0 semi-planar
2) RGBA_8888 = GL_RGBA
In your sample code I also see dma_img struct and especially dma_img.dma_fd(). I am not familiar with this dma-img/buf stuff, so maybe you can tell/show me how to create this struct and how to get the texture memory address pointer? Do you have some sample code or links to this?
Hi,
In that case that's probably the format you want:
#define NV12_BT709_WIDE_FORMAT 0x0000000C01028A88
When you sample your texture in GLES you obtain RGB.
The 10 bit variant is not supported by T760 I'm afraid.
Regarding dma_buf: please ignore the method calls, they are just local methods from the piece of code I created the example from. (I'll edit my previous comment to remove them).
Sorry, forgot to reply to your other questions:
Here is the RGBA_8888:
#define RGBA8888_FORMAT: 0x00000000010BB053
Regarding dma_buf: all you need is the file descriptor (int) for your dma_buf allocation.
Unfortunately there is no standard kernel interface to allocate dma_buf memory, so you will have to look into your platform's kernel sources.
Sometimes you've got a special IOCTL to get a handle to the framebuffer for example (There is one on Firefly for example).
I'm confused by what you're asking: are you on Linux FBDev or Android ? The example code above is for fbdev only, if you're running on Android, you will have to use GraphicBuffer and EGL_NATIVE_BUFFER_ANDROID.
Things are clear to me now, this will help me for sure.
I am on Linux FBDev for this question, so your example is exactly what gives me the right direction! Confusion comes from me because I also use libhybris in linux/ubuntu, making android libs (also your android libmali) available under linux, and actually that way i have access tp the video encoder/decoder hardware en hwcomposer.
You mention firefly has a special ioctl to get a handle to the framebuffer: can you give me more detail/ or code for this? You would help me out in saving a bit of kernel searching here.
Anyways, thanks for your answer again!
For the Firefly ioctl, it's in the 3.10 kernel and requires ION to be turned on:
https://bitbucket.org/T-Firefly/firefly-rk3288-kernel/src/f4d954d67f9e5689f1ddbbbd3050a8041424dd48/drivers/video/rockchip/rk_fb.c?at=master#cl-2469
T-Firefly / Firefly-RK3288-kernel / source / include / linux / rk_fb.h — Bitbucket
However the issue is that the fbdev Mali-T76x user-side binary driver available from malideveloper.arm.com is not specific to this kernel, or not even to Firefly but to any system with a Mali-T76x GPU. So it doesn't know about this ioctl. It would need to call it to get a DMA-BUF file descriptor from the display driver and then pass it to the kernel-side driver.
We're planning to fix support for DMA-BUF in fbdev in future drivers with a portable kernel patch, but unfortunately I can't yet tell when this will be available as we rely on other things to be released first.
Then to import a DMA-BUF file descriptor from the video decoder into the GPU, there is an extension for that:
https://www.khronos.org/registry/egl/extensions/EXT/EGL_EXT_image_dma_buf_import.txt
It is not yet enabled in our r5p0 driver but should be in a future version. I'll post another message when this is released.
Thanks Guillaume! Your explanation is clear and helps me in digging further into dma-buf concept and usage.
For the rk3288 video decoder I can obtain the filedescriptor by using rockchips VPUMemGetFD() provided by their vpu memory allocator code: T-Firefly / Firefly-RK3288 / source / frameworks / av / media / libstagefright / libvpu / common / vpu_mem_dmabuf.c —… Using libhybris I can access this android library - libvpu.
As dma-buf import is not yet supported in r5p0 driver I guess, for now, will have to use your android mali driver with libhybris - far from ideal.
I am of course very interested in your future release!
PS: for those interested, also found some interesting links related to dma_buf / ion stuff:
DMA buffer sharing in 3.3 [LWN.net]
The Android ION memory allocator [LWN.net]
Daeseok's Blog :: [ION] Sharing a buffer between kernel-space and user-space
ION Buffer sharing mechanism | :(){ :|:& };: