I have a question about how to make Ethos-U NPU work on a ARM Cortex-A + Cortex-M processor. First, I found ethos-u-linux-driver-stack and ethos-u-core-software on https://git.mlplatform.org/.
1. I know ethos-u-linux-driver-stack is Ethos-U kernel driver. Should it be integrated into the Linux OS running on Cortex-A or be integrated into the Linux OS running on Cortex-M? I am nor clear about which core it need to perform on.
2. For ethos-u-core-software, how to run it? I did't find the detail steps to run it. Does it run on NPU or any core?
3. Except the above two repos, is there any other repo necessory to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?
Thanks for your suggestion in advance.
Hi
1. There is no other firmware needed to drive the Arm Ethos-U NPU. The entry point for running an subgraph is ethosu_invoke_v2(), which takes a pointer to driver actions and an array of base addresses.
The Arm Ethos-U supports power save mode, which may result in loss of power between inferences. Because of this the driver will always do a complete setup of all necessary registers before every inference.
Running an inference requires the TFLu framework. The inference process shows how to setup the model and the arena before invoking the framework. The connection between TLFu and the driver is implemented in tensorflow/lite/micro/kernels/ethos-u/ethosu.cc.
inference process -> TLFu framework -> ethosu.cc -> ethosu_driver.c
TLFu operators that are not supported by the Arm Ethos-U NPU will be executed on the Arm Cortex-M. For an inference this might mean that execution ping-pongs between running operators on the NPU and on the CPU. For an ideal case the complete inference would run on the NPU.
2. We had some issues with patches not making it into TLFu in time for the release, so I suspect that there might be minor compilation issues with the 20.08 tag. I did however just try this and it worked,.
$ git clone "https://review.mlplatform.org/ml/ethos-u/ethos-u" $ cd ethos-u $ ./fetch_externals.py fetch $ mkdir core_software/build $ cd core_software/build $ cmake .. -DCMAKE_TOOLCHAIN_FILE=../cmake/toolchain/arm-none-eabi-gcc.cmake -DCMAKE_SYSTEM_PROCESSOR=cortex-m33 && make -j8
The commands for building TFLu can be found in core_software/tensorflow.cmake. With the current setup libethosu_core_driver.a is built separately and passed as ETHOSU_DRIVER_LIBS to the TFLu build system. The driver will consequently not be linked into libtensorflow-microlite.a, but is provided as a separate library. In other words you need to pass both libtensorflow-microlite.a and libethosu_core_driver.a to the linker when you build your firmware binary.
Hi, Kristofer,
Thanks for your reply.
For item 2, yes, I met some compilation issues with the 20.08 tag too. Then I did some changes and made it compile successfully. If it is not necessary to update it to the latest one, I will keep working on 20.08 tag, ok?
For libtensorflow-microlite.a and libethosu_core_driver.a, I have integrated both of them in FreeRTOS. The linker links both of them. But I am not sure whether I need to do other configuration, such as add “ETHOSU_DRIVER_LIBS=” you have mentioned to make the process executes as TLFu framework -> ethosu.cc -> ethosu_driver.c. Do I need to do other configuration? If so, please tell me how to do it in linker.
The commands we use to build libtensorflow-microlite.a are defined in core_software/tensorflow.cmake. You can find them in the build log if you call 'make VERBOSE=1' when building core software. I guess all arguments are important, but the argument triggering Arm Ethos-U to be enabled is TAGS="ethos-u cmsis-nn".
The Tensorflow lite micro build system is spread out across a number of Makefiles. The most important ones for us are tensorflow/lite/micro/tools/make/Makefile and tensorflow/lite/micro/tools/make/ext_libs/ethosu.inc.
If you take a look at ethosu.inc you will see that ETHOSU_DRIVER_LIBS controls if the driver sources are built directly into libtensorflow-microlite.a, or if they are excluded and you need build libethosu_core_driver.a yourself and provide the library to the linker of your binary.
Running a tflite model on the Arm Ethos-U of course requires the tflite file to be optimized by Vela. Else it will run on Cortex-M only.
Hi, Kristofer, thanks a lot for your guild.
1. I checked the build log for generating libtensorflow-microlite.a and libethosu_core_driver.a. The part log is as below.
CMSIS_PATH=xxx/ethos-u_20.08_build/core_software/cmsis ETHOSU_DRIVER_PATH=xxx/ethos-u_20.08_build/core_software/core_driver ETHOSU_DRIVER_LIBS=xxx/ethos-u_20.08_build/core_software/build/core_driver/libethosu_core_driver.a ETHOSU_FAST_MEMORY_SIZE=0 TAGS="cmsis-nn ethos-u" BUILD_TYPE=release
According to the log, Arm Ethos-U is enabled by "TAGS="ethos-u cmsis-nn" argument. The driver source is built as libethosu_core_driver.a. Anyway, I have linked both libtensorflow-microlite.a and libethosu_core_driver.a in the binary. I guess the process is TLFu framework -> ethosu.cc -> ethosu_driver.c.
2. I have another question. I saw the entry point ethosu_invoke which is called in ethosu.cc. In ethosu_driver.c, there are other necessary driver functions, such as ethosu_init, ethosu_irq_handler.. But I didn't find where these driver functions are called. Shouldn't they be executed?
3. I think the ethos-u register base address and irq number need to be used in the driver. For example, register base address is needed for ethous_init and ethosu_invoke. What is the preferred way to transfer these values to these functions?
Actually, I don't know how much work I should do for this core_driver. Please give me some guild.
Up until recently there were no Arm Ethos-U compatible platforms available in the public domain. Because of this we have only published platform generic software components like applications, frameworks and drivers.
Now that the Corstone-300 + Ethos-U has been published we will be able to upstream target specific code as well. This code will demonstrate how to setup the interrupt vector, initialize drivers and how to link the software into a binary. We have created ethos-u-core-platform that in the next month or two will be populated with examples.
https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps
2.Code for setting up the platform is not present in core software. Examples of this will later on be published in ethos-u-core-platform. This code will for example show how to setup the interrupt vector and how to call ethosu_init_...().
3. There are many options how the base address could be set. The address could be hard coded in the code; a build system could set a define; a build system could generate a header header file with a variable or a define; etc. I can't say which would be the preferred way, only that there are several options that would solve the problem.