How to access(zero copy) android AHardWareBuffer and ANativeWindow_Buffer, with mali opencl?
please note, ANativeWindow_Buffer is not ANativeWindowBuffer.
The short answer is that there is no clear, easy and guaranteed-to-work way of importing one of these without a copy.
We are currently working on adding support to our memory import extension  for importing an AHardwareBuffer directly. This will exist in upcoming versions of the driver. You can contact our customer support to learn more about this.
Until the extension is available, there are a number of workarounds that you can try:
With help from the platform vendor you should be able to get hold of the underlying dma_buf file descriptor. You can then import it using cl_arm_import_memory .
If you have access to VNDK functions or can get some help from the platform vendor (or maybe by reading the Android source code), you can:
get a native handle from the AHardwareBuffer:
const struct native_handle *native_hnd = AHardwareBuffer_getNativeHandle(m_ahb);
cast that to a buffer handle:
buffer_handle_t buffer_hnd = static_cast<buffer_handle_t>(native_hnd);
from which you can extract a dma_buf file descriptor that you can import with help from your platform vendor.
I should emphasize that it is definitely not a clean or portable solution but it would allow you to get things working before the extension is available.
You can *try* to import ANativeWindow_Buffer::bits as a host pointer with clImportMemoryARM selecting CL_IMPORT_TYPE_HOST_ARM. This is however unlikely to work as on most platforms bits is a mapping of an underlying dma_buf allocation that clImportMemoryARM won’t be able to import.
You can pass ANativeWindow_Buffer::bits as host_ptr when creating a CL buffer with CL_MEM_USE_HOST_PTR. This should work but the OpenCL runtime will currently make a copy in most scenarios.
Lastly, and I know that’s ultimately what you’re trying to avoid, you can always create a buffer and copy data in and out.
Hope this helps and sorry there is no good solution currently.
1, About AHardwareBuffer.
Can i use pass that buffer to eglCreateImageKHR to create an EGLImage resource type, then use clCreateFromEGLImageKHR to get a cl_mem?
there is some info below, about AHardwareBuffer, on https://developer.android.google.cn/ndk/guides/stable_apis#hardware_buffer_apis,
"You can allocate an AHardwareBuffer and use it to obtain an EGLClientBuffer resource type via the eglGetNativeClientBufferANDROID extension. You can pass that buffer to eglCreateImageKHR to create an EGLImage resource type, which may then be bound to a texture via glEGLImageTargetTexture2DOES on supported devices. This can be useful for creating textures that may be shared cross-process"
2, About ANativeWindow_Buffer.
Is there any other way to render the preview, with OPENCL? For real time preview, i wonder how to directly write the display buffer, with openCL.
there is a opencv sample , opencl can precess GL texture, but the preview is rendered via OpenGL textures other than OPENCL. docs.opencv.org/.../tutorial_android_ocl_intro.html
I started to think about this after posting my reply yesterday. This looks like it would be a plausible route but unfortunately the Mali driver doesn't support EGL_ANDROID_get_native_client_buffer.
If rendering a texture with OpenGL is an acceptable solution, a possible route to explore would be:
- create an EGL image from a dma_buf allocation (via ION most likely)
- import it in CL using clCreateFromEGLImageKHR
- create a GL external texture from it
- write to it using CL
- render it using GL
This is definitely something that we support and validate.
Also, If I understand correctly, you're using OpenCV. You may want to have a look our compute library  (built on top of OpenCL) that has a number of algorithms optimised for Mali GPUs.
Lastly, I am going to be away for a couple of weeks and won't be able to reply until I get back but don't hesitate to keep posting your findings and questions. I'll pick up the thread when I'm back (and other might be able to help as well).
I have queried the system with huawei P30 pro, with eglQueryString, it looks like support the EGL_ANDROID_get_native_client_buffer.
EGL_EXTENSIONS:EGL_KHR_get_all_proc_addresses EGL_ANDROID_presentation_time EGL_KHR_swap_buffers_with_damage EGL_ANDROID_get_native_client_buffer EGL_ANDROID_front_buffer_auto_refresh EGL_ANDROID_get_frame_timestamps EGL_EXT_surface_SMPTE2086_metadata EGL_EXT_surface_CTA861_3_metadata EGL_KHR_image EGL_KHR_image_base EGL_EXT_image_gl_colorspace EGL_KHR_gl_colorspace EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image EGL_KHR_fence_sync EGL_KHR_create_context EGL_KHR_config_attribs EGL_KHR_surfaceless_context EGL_EXT_create_context_robustness EGL_ANDROID_image_native_buffer EGL_KHR_wait_sync EGL_ANDROID_recordable EGL_KHR_partial_update EGL_EXT_pixel_format_float EGL_KHR_mutable_render_buffer EGL_EXT_protected_content EGL_IMG_context_priority EGL_KHR_no_config_context
Yes, while we don't support this extension directly, it is indeed implemented in Android's EGL layer so you can rely on it. Have you been able to make the EGL route work?
Thanks for the update. Good luck and let us know how you get on.
The Mali OpenCL driver now supports Importing AHardwareBuffer: www.khronos.org/.../cl_arm_import_memory.txt
yes, I have read the develope guide, and can AHARDWAREBUFFER_FORMAT_Y8Cb8Cr8_420 be Imported with the extention for R/W?
and, another question is, Can ION buffer imported with cl_arm_import_memory_dma_buf?
Yes, you can share the ION buffer to obtain a dma_buf file descriptor that can be imported with cl_arm_import_memory_dma_buf. You can look into ion_share or ION_IOC_SHARE.
If the buffer uses a single plane, it should be supported. The format's documentation implies that the buffer could either be mutiplanar (only the first plane would be imported) or semiplanar.
EDIT: we will be adding full support for this.
It look like that, AHARDWAREBUFFER_FORMAT_Y8Cb8Cr8_420 can't be import by the extention.
As ION can be imported, Can I import a NV21(YVU420P) image with graphic buffer, in such way:
get file handle of the ION from gb(graphic buf), the import the ION buffer with fd:
int fd =( (private_handle_t*) gb->handle)->fd;
AHARDWAREBUFFER_FORMAT_Y8Cb8Cr8_420 is very important because android camera provide the image data in this format. For Other format such as RGBA8888, It's not difficult and not nessesary to use the cl_arm_import_memory, opencl can acess it with openglTEXTURE2D.
Thanks for the feedback. What behaviour are you getting?
This should work but is not guaranteed to be portable.
View all questions in Graphics and Gaming forum