<?xml version="1.0" encoding="UTF-8" ?>
<?xml-stylesheet type="text/xsl" href="https://community.arm.com/utility/feedstylesheets/rss.xsl" media="screen"?><rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:slash="http://purl.org/rss/1.0/modules/slash/" xmlns:wfw="http://wellformedweb.org/CommentAPI/"><channel><title>How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/developer/tools-software/oss-platforms/f/machine-learning-forum/47593/how-to-make-ethos-u-npu-work-on-an-arm-cortex-a-cortex-m-processor</link><description> I have a question about how to make Ethos-U NPU work on a ARM Cortex-A + Cortex-M processor. First, I found ethos-u-linux-driver-stack and ethos-u-core-software on https://git.mlplatform.org/ . 
 1. I know ethos-u-linux-driver-stack is Ethos-U kernel</description><dc:language>en-US</dc:language><generator>Telligent Community 10</generator><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168897?ContentTypeID=1</link><pubDate>Tue, 01 Dec 2020 06:45:04 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:34861459-2749-4236-bc8a-294a1b745fae</guid><dc:creator>alisonw</dc:creator><description>&lt;p&gt;&lt;span&gt;Hi, Kristofer&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;Thanks for your reply. I have fixed the issues. Now the process inference process -&amp;gt; TLFu framework -&amp;gt; ethosu.cc -&amp;gt; ethosu_driver.c works.&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;As now there is not real hardware to verify ethous-driver, I want to use the original model (not optimized by vela too) to run on m-core. &lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;1. For the original model, it will go through CMSIS-NN, not ethos-u, right?&amp;nbsp;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;2. I remember you have said there is one small patch that has not yet reach upstream, that adjusts the build flags and a few paths to CMSIS-NN. How could I get it?&amp;nbsp;&lt;/span&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168883?ContentTypeID=1</link><pubDate>Mon, 30 Nov 2020 15:03:00 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:b123b3f8-cb5f-4d60-b684-f817ed3e8625</guid><dc:creator>Kristofer Jonsson</dc:creator><description>&lt;p&gt;Hi&lt;/p&gt;
&lt;p&gt;In case you are interested we have just uploaded code to &lt;em&gt;Core Platform&lt;/em&gt;. It demonstrates how the Arm Ethos-U driver stack including FreeRTOS can be built for Corstone-300.&lt;/p&gt;
&lt;p&gt;Please use &lt;em&gt;fetch_externals.py&lt;/em&gt; to download all repositories and follow the instructions in &lt;em&gt;core_software/README.md&lt;/em&gt; how to build with either ArmClang or Gcc.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://git.mlplatform.org/ml/ethos-u/ethos-u.git"&gt;https://git.mlplatform.org/ml/ethos-u/ethos-u.git&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;1. You should in theory be able to build &lt;em&gt;Core Software&lt;/em&gt; for any Arm Cortex-M. However, all variants are not built and tested because they are expected to be too weak for running ML workloads, so I assume that the smaller cores would need some minor adjustments to build.&lt;br /&gt;&lt;br /&gt;The driver stack is tested with a wide range of network models. I don&amp;#39;t know for sure where they originate from.&lt;/p&gt;
&lt;p&gt;2. Hard to tell for sure, but my guess is that the Tensor Arena might be too small. The required Tensor Arena size varies a lot from network to network.&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168770?ContentTypeID=1</link><pubDate>Tue, 24 Nov 2020 01:46:10 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:a50131ab-b275-43dd-a304-eb62cc2b3ade</guid><dc:creator>alisonw</dc:creator><description>&lt;p&gt;Hi, Kristofer, I have two questions about running the core software on Cortex-M.&lt;/p&gt;
&lt;p&gt;1. Does the core software (wrapper application + tflite micro + ethos-u driver) be verified on any Cortex-M core? Do you use the model on&amp;nbsp;&lt;a href="https://www.tensorflow.org/lite/guide/hosted_models#automl_mobile_models"&gt;https://www.tensorflow.org/lite/guide/hosted_models#automl_mobile_models&lt;/a&gt;&amp;nbsp;to verify?&lt;/p&gt;
&lt;p&gt;2. I have tried the core software on Cortex-M7. When calling&amp;nbsp;interpreter.AllocateTensors() in&amp;nbsp;applications/inference_process/src/inference_process.cc, it returns&amp;nbsp;kTfLiteError. What&amp;#39;s your suggestion about it?&lt;/p&gt;
&lt;p&gt;Thanks.&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168762?ContentTypeID=1</link><pubDate>Mon, 23 Nov 2020 10:05:42 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:f2a86c86-69fd-498c-b41b-fd911a27c07d</guid><dc:creator>Kristofer Jonsson</dc:creator><description>&lt;p&gt;Up until recently there were no Arm Ethos-U compatible platforms available in the public domain. Because of this we have only published platform generic software components like applications, frameworks and drivers.&lt;/p&gt;
&lt;p&gt;Now that the &lt;em&gt;Corstone-300 + Ethos-U&lt;/em&gt; has been published we will be able to upstream target specific code as well. This code will demonstrate how to setup the interrupt vector, initialize drivers and how to link the software into a binary. We have created &lt;em&gt;ethos-u-core-platform&lt;/em&gt; that in the next month or two will be populated with examples.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps"&gt;https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;2.Code for setting up the platform is not present in &lt;em&gt;core software&lt;/em&gt;. Examples of this will later on be published in &lt;em&gt;ethos-u-core-platform&lt;/em&gt;. This code will for example show how to setup the &lt;em&gt;interrupt vector&lt;/em&gt; and how to call &lt;em&gt;ethosu_init_...()&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;3. There are many options how the base address could be set. The address could be hard coded in the code; a build system could set a define; a build system could generate a header header file with a variable or a define; etc. I can&amp;#39;t say which would be the preferred way, only that there are several options that would solve the problem.&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168726?ContentTypeID=1</link><pubDate>Fri, 20 Nov 2020 05:58:30 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:8ec8d90d-40aa-4924-b88b-734bc7e824eb</guid><dc:creator>alisonw</dc:creator><description>&lt;p&gt;&lt;span&gt;Hi, Kristofer, thanks a lot for your guild. &lt;/span&gt;&lt;/p&gt;
&lt;p&gt;1. I checked the build log for generating&amp;nbsp;libtensorflow-microlite.a and&amp;nbsp;libethosu_core_driver.a. The part log is as below.&lt;/p&gt;
&lt;p&gt;&lt;span&gt;CMSIS_PATH=xxx/ethos-u_20.08_build/core_software/cmsis ETHOSU_DRIVER_PATH=xxx/ethos-u_20.08_build/core_software/core_driver ETHOSU_DRIVER_LIBS=xxx/ethos-&lt;/span&gt;&lt;span&gt;u_20.08_build/core_software/build/core_driver/libethosu_core_driver.a ETHOSU_FAST_MEMORY_SIZE=0 TAGS=&amp;quot;cmsis-nn ethos-u&amp;quot; BUILD_TYPE=release&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;According to the log, &amp;nbsp;Arm Ethos-U is enabled&amp;nbsp; by &amp;quot;TAGS=&amp;quot;ethos-u cmsis-nn&amp;quot;&amp;nbsp;argument. The driver source is built as&amp;nbsp;libethosu_core_driver.a. Anyway, I have linked both libtensorflow-microlite.a and&amp;nbsp;libethosu_core_driver.a in the binary. I guess the process is&amp;nbsp;TLFu framework -&amp;gt; ethosu.cc -&amp;gt; ethosu_driver.c.&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;2. I have another question. I saw the entry point ethosu_invoke which is called in ethosu.cc. In ethosu_driver.c, there are other necessary&amp;nbsp;driver functions, such as ethosu_init, ethosu_irq_handler.. But I didn&amp;#39;t find where these driver functions are called. Shouldn&amp;#39;t they be executed?&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;3. I think the ethos-u register base address and irq number need to be used in the driver. For example, register base address is needed for ethous_init and ethosu_invoke. What is the preferred way to transfer these values to these functions?&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;Actually, I don&amp;#39;t know how much work I should do for this core_driver. Please give me some guild.&lt;/span&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168684?ContentTypeID=1</link><pubDate>Tue, 17 Nov 2020 14:37:44 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:84c98ac1-2099-40f0-b191-c0fff136a290</guid><dc:creator>Kristofer Jonsson</dc:creator><description>&lt;p&gt;The commands we use to build libtensorflow-microlite.a are defined in core_software/tensorflow.cmake. You can find them in the build log if you call &amp;#39;make VERBOSE=1&amp;#39; when building &lt;em&gt;core software&lt;/em&gt;. I guess all arguments are important, but the argument triggering Arm Ethos-U to be enabled is TAGS=&amp;quot;ethos-u cmsis-nn&amp;quot;.&lt;/p&gt;
&lt;p&gt;The Tensorflow lite micro build system is spread out across a number of Makefiles. The most important ones for us are tensorflow/lite/micro/tools/make/Makefile and tensorflow/lite/micro/tools/make/ext_libs/ethosu.inc.&lt;/p&gt;
&lt;p&gt;If you take a look at ethosu.inc you will see that ETHOSU_DRIVER_LIBS controls if the driver sources are built directly into libtensorflow-microlite.a, or if they are excluded and you need build libethosu_core_driver.a yourself and provide the library to the linker of your binary.&lt;/p&gt;
&lt;p&gt;Running a tflite model on the Arm Ethos-U of course requires the tflite file to be optimized by Vela. Else it will run on Cortex-M only.&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168624?ContentTypeID=1</link><pubDate>Fri, 13 Nov 2020 00:24:18 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:be08278a-fad1-462a-a1af-c250c89bca3a</guid><dc:creator>alisonw</dc:creator><description>&lt;p&gt;&lt;span&gt;Hi, Kristofer,&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;Thanks for your reply.&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;For item 2, yes, I met some compilation issues with the 20.08 tag too. Then I did some changes and made it compile successfully. If it is not necessary to update it to the latest one, I will keep working on 20.08 tag, ok?&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;For libtensorflow-microlite.a and libethosu_core_driver.a, I have integrated both of them in FreeRTOS. The linker links both of them. But I am not sure whether I need to do other configuration, such as add &amp;ldquo;ETHOSU_DRIVER_LIBS=&amp;rdquo; you have mentioned to make the process executes as TLFu framework -&amp;gt; ethosu.cc -&amp;gt; ethosu_driver.c. Do I need to do other configuration? If so, please tell me how to do it in linker.&lt;/span&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168609?ContentTypeID=1</link><pubDate>Thu, 12 Nov 2020 08:01:08 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:ad76d0a4-8721-4a6f-87f3-ae46746970f5</guid><dc:creator>Kristofer Jonsson</dc:creator><description>&lt;p&gt;Hi&lt;/p&gt;
&lt;p&gt;1. There is no other firmware needed to drive the Arm Ethos-U NPU. The entry point for running an subgraph is &lt;em&gt;ethosu_invoke_v2()&lt;/em&gt;, which takes a pointer to &lt;em&gt;driver actions&lt;/em&gt; and an array of base addresses.&lt;/p&gt;
&lt;p&gt;The Arm Ethos-U supports power save mode, which may result in loss of power between inferences. Because of this the driver will always do a complete setup of all necessary registers before every inference.&lt;/p&gt;
&lt;p&gt;Running an inference requires the TFLu framework. The &lt;em&gt;inference process&lt;/em&gt; shows how to setup the &lt;em&gt;model&lt;/em&gt; and the &lt;em&gt;arena&lt;/em&gt; before invoking the framework. The connection between TLFu and the driver is implemented in &lt;em&gt;tensorflow/lite/micro/kernels/ethos-u/ethosu.cc&lt;/em&gt;.&lt;em&gt;&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;inference process -&amp;gt; TLFu framework -&amp;gt; ethosu.cc -&amp;gt; ethosu_driver.c&lt;/p&gt;
&lt;p&gt;TLFu operators that are not supported by the Arm Ethos-U NPU will be executed on the Arm Cortex-M. For an inference this might mean that execution ping-pongs between running operators on the NPU and on the CPU. For an ideal case the complete inference would run on the NPU.&lt;/p&gt;
&lt;p&gt;2. We had some issues with patches not making it into TLFu in time for the release, so I suspect that there might be minor compilation issues with the 20.08 tag. I did however just try this and it worked,.&lt;/p&gt;
&lt;p&gt;&lt;pre class="ui-code" data-mode="text"&gt;$ git clone &amp;quot;https://review.mlplatform.org/ml/ethos-u/ethos-u&amp;quot;
$ cd ethos-u
$ ./fetch_externals.py fetch
$ mkdir core_software/build
$ cd core_software/build
$ cmake .. -DCMAKE_TOOLCHAIN_FILE=../cmake/toolchain/arm-none-eabi-gcc.cmake -DCMAKE_SYSTEM_PROCESSOR=cortex-m33 &amp;amp;&amp;amp; make -j8&lt;/pre&gt;&lt;/p&gt;
&lt;p&gt;The commands for building TFLu can be found in &lt;em&gt;core_software/tensorflow.cmake&lt;/em&gt;. With the current setup &lt;em&gt;libethosu_core_driver.a&lt;/em&gt; is built separately and passed as &lt;em&gt;ETHOSU_DRIVER_LIBS&lt;/em&gt; to the TFLu build system. The driver will consequently not be linked into &lt;em&gt;libtensorflow-microlite.a&lt;/em&gt;, but is provided as a separate library. In other words you need to pass both &lt;em&gt;libtensorflow-microlite.a&lt;/em&gt; and &lt;em&gt;libethosu_core_driver.a&lt;/em&gt; to the linker when you build your firmware binary.&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168607?ContentTypeID=1</link><pubDate>Thu, 12 Nov 2020 03:17:33 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:eacd2c28-11d3-4eee-a2d4-deb67270cf4e</guid><dc:creator>alisonw</dc:creator><description>&lt;p&gt;Hi, Kristofer, I have two more questions. Please give me some guide.&lt;/p&gt;
&lt;p&gt;1. According to my understanding, ethos-u/core_software/core_driver/ is the ethos-u hardware driver and accomplishes the registers&amp;#39; configuration. I want to know whether it is sufficient for ethos-u&amp;#39;s initialization and working. For example, is there any firmware needed to run on ethos-u?&lt;/p&gt;
&lt;p&gt;2. I am using ethos-u/ tag 20.08 as base. For core_software, my building command is as below.&lt;/p&gt;
&lt;p&gt;cmake .. -DCMAKE_TOOLCHAIN_FILE=../cmake/toolchain/arm-none-eabi-gcc.cmake -DCMAKE_SYSTEM_PROCESSOR=cortex-m33&lt;br /&gt;make&lt;/p&gt;
&lt;p&gt;I can get all the static libraries. I want to know whether ethos-u is used instead of Cortex-M in the generated libtensorflow-microlite.a using the above command and whether libethosu_core_driver.a is called.&lt;/p&gt;
&lt;p&gt;Thanks.&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168434?ContentTypeID=1</link><pubDate>Tue, 03 Nov 2020 17:40:58 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:9e3f0028-ea1e-4ac6-9d50-d67e90c83c42</guid><dc:creator>Kristofer Jonsson</dc:creator><description>&lt;p&gt;We have tried to mimic what Linux does and abstract the Mailbox driver behind a Mailbox API. The APIs can be found in &amp;lt;ethos-u-core-software&amp;gt;/drivers/mailbox/include/mailbox.hpp and should allow developers to use any mailbox IP they prefer.&lt;/p&gt;
&lt;p&gt;We have a driver implementation for the Arm MHU v2, but it has not yet reach upstream.&lt;/p&gt;
&lt;p&gt;For a bare metal application instantiation could for example look like this:&lt;/p&gt;
&lt;p&gt;// Use section attributes to allow the linker script to place the queues at given addresses in memory&lt;br /&gt;__attribute__((section(&amp;quot;ethosu_core_in_queue&amp;quot;))) MessageProcess::Queue&amp;lt;1000&amp;gt; inQueue;&lt;br /&gt;__attribute__((section(&amp;quot;ethosu_core_out_queue&amp;quot;))) MessageProcess::Queue&amp;lt;1000&amp;gt; outQueue;&lt;br /&gt;Mailbox::YourCustomMailboxDriver mailbox;&lt;br /&gt;InferenceProcess::InferenceProcess inferenceProcess;&lt;br /&gt;MessageProcess::MessageProcess messageProcess(*inQueue.toQueue(), *outQueue.toQueue(), mailbox, inferenceProcess);&lt;br /&gt;&lt;br /&gt;// You need to implement this class for your custom mailbox driver&lt;br /&gt;class YourCustomMailboxDriver : public Mailbox {&lt;br /&gt;public:&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; YourCustomMailboxDriver();&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; virtual ~YourCustomMailboxDriver();&lt;br /&gt;&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; // Trigger an IRQ on the remote CPU&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; virtual bool sendMessage() final;&lt;br /&gt;&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; // This function should be called from the IRQ routine&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; // It should clear the IRQ and call notify() to inform registered clients about the received message&lt;br /&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; virtual void handleMessage() final;&lt;br /&gt;};&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168429?ContentTypeID=1</link><pubDate>Tue, 03 Nov 2020 09:31:05 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:dad1419e-f3c6-464b-8fd5-e3ce919b048f</guid><dc:creator>alisonw</dc:creator><description>&lt;p&gt;Kristofer, I have a question about message handling on Cortex-M. As we have hardware Message Unit IP on our silicon, and the driver supports mbox_send_message API in Linux kernel, so the message handling on Cortex-A is Ok. But I didn&amp;#39;t find the detailed mailbox code in core_software/applications/message_process/src/message_process.cc. How should the mailbox work in&amp;nbsp;&lt;span&gt;message_process.cc?&amp;nbsp;&lt;/span&gt;Could you give some guide?&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168230?ContentTypeID=1</link><pubDate>Thu, 22 Oct 2020 15:31:02 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:c89f7f2d-3ea6-4456-a0b3-e008231eafcd</guid><dc:creator>Kristofer Jonsson</dc:creator><description>&lt;p&gt;The Linux Driver Stack for Arm Ethos-U is provided as an example how a rich operating system like Linux can dispatch inferences to an Arm Ethos-U subsystem. The driver stack currently produces following binaries:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;inference_runner - An example user space application that dispatches inferences to the Arm Ethos-U subsystem. Takes a TFLite file as input.&lt;/li&gt;
&lt;li&gt;ethosu.a - A driver library that presents a C++ interface for the Arm Ethos-U kernel driver.&lt;/li&gt;
&lt;li&gt;ethosu.ko - Kernel driver that handles the communication with the Arm Ethos-U subsystem.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Ideally you pass a TFLite file optimized by Vela to the inference_runner. The inference will be executed on the Arm Ethos-U subsystem and accelerated by the Arm Ethos-U NPU.&lt;/p&gt;
&lt;p&gt;Would you however pass a TFLite file not optimized by Vela to the inference_runner, then the inference will be executed on the Arm Cortex-M only. You will still get the correct result, but the inference will not be accelerated by the Arm Ethos-U.&lt;/p&gt;
&lt;p&gt;ArmNN does currently not support Arm Ethos-U.&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168227?ContentTypeID=1</link><pubDate>Thu, 22 Oct 2020 14:03:50 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:aaec6cff-d27c-4694-bc37-ff3bafd705ab</guid><dc:creator>devinwxie</dc:creator><description>&lt;p&gt;What is the input source of ethos-u-linux-driver? Is it a TFLite mode which convert by Vela tools? Or ArmNN stack ?&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168226?ContentTypeID=1</link><pubDate>Thu, 22 Oct 2020 14:03:20 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:5da5ba3b-7576-49d9-8a9c-db173b9a9f16</guid><dc:creator>devinwxie</dc:creator><description>&lt;p&gt;&lt;strong&gt;What is the input source of ethos-u-linux-driver? Is it a TFLite mode which convert by Vela tools? Or is it ArmNN stack ?&lt;/strong&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168188?ContentTypeID=1</link><pubDate>Wed, 21 Oct 2020 07:01:20 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:93608d40-18d8-4283-9872-b5a4eea4adc4</guid><dc:creator>Kristofer Jonsson</dc:creator><description>&lt;p&gt;User facing APIs would be part of the OS and should be generic enough to support multiple frameworks (TFLu, TVM, etc). They should allow multiple applications to share NPU resources and ideally provide hardware abstraction (the application is unaware of which hardware that accelerates the network). An application would do an OS call to run an inference, instead of directly calling interpreter.Invoke().&lt;/p&gt;
&lt;p&gt;These APIs don&amp;#39;t exist today and we do not yet have a clear picture of what they would look like, or if this even is the right way to go. Hardware abstraction might also be difficult to achieve, because networks might have been optimized for a specific hardware.&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168160?ContentTypeID=1</link><pubDate>Tue, 20 Oct 2020 01:59:35 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:05294df3-c9b4-4fb2-a135-5d8a9de5cc17</guid><dc:creator>alisonw</dc:creator><description>&lt;p&gt;Kristofer, according to my understanding about the current code, Tensorflow Lite APIs (for example, interpreter.Invoke) are used by the application and call the ethosu driver in micro/. Are Tensorflow Lite APIs one kind of the user facing OS APIs you mentioned? I am not clear about the user facing OS APIs which is not (yet) supported. Please correct and guide me.&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168149?ContentTypeID=1</link><pubDate>Mon, 19 Oct 2020 07:57:21 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:68123d7b-adf3-4d20-beea-f11834a1b014</guid><dc:creator>Kristofer Jonsson</dc:creator><description>&lt;p&gt;I would like to divide this answer into two parts.&lt;/p&gt;
&lt;p&gt;The Arm Ethos-U NPU driver is OS agnostic and does not use any OS specific primitives (like mutex, queues, etc). It consequently can be paired up with any RTOS. You spawn a thread that drive the TFLu runtime from within the thread. I would even go one step further and say that it is recommended to use a RTOS. So yes, you can use FreeRTOS or any other RTOS you prefer.&lt;/p&gt;
&lt;p&gt;What is not (yet) supported are user facing OS APIs. The APIs would be used by applications to schedule inferneces, and implemented by drivers to run inferences on the physical device. The APIs would provide hardware abstraction (you do not know which hardware that accelerates your network) and scheduling (multiple applications can share multiple NPUs).&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168146?ContentTypeID=1</link><pubDate>Mon, 19 Oct 2020 01:03:27 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:7aa881c8-45b6-4dc6-a3c7-dd9a097978cc</guid><dc:creator>alisonw</dc:creator><description>&lt;p&gt;Kristofer, thanks for your reply, I got it. Another question, what is your suggestion about the OS or Non-OS running on Cortex-M? How about running FreeOS on Cortex-M?&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168131?ContentTypeID=1</link><pubDate>Fri, 16 Oct 2020 06:31:08 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:acfb1b6a-4f9d-425b-bb1d-1fe624c363b3</guid><dc:creator>Kristofer Jonsson</dc:creator><description>&lt;p&gt;Hi&lt;/p&gt;
&lt;p&gt;The code published on MLPlatform is sufficient for Linux to dispatch inferences to the Arm Cortex-M.&lt;/p&gt;
&lt;p&gt;The reason for moving towards virtio/rpmsg (OpenAMP on the Arm Cortex-M side) is that those are Linux native APIs. They provide standard, well designed and well tested communication channels. With that said, what we have published so far is fully functional.&lt;/p&gt;
&lt;p&gt;For reference you could follow the call chain from ethosu_inference_create() to see how the kernel driver creates an inference and dispatches it to the Arm Ethos-U subsystem. On the Core side the message is received and handled in MessageProcess::handleMessage().&lt;/p&gt;
&lt;p&gt;Best regards&lt;br /&gt;Kristofer&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/168126?ContentTypeID=1</link><pubDate>Fri, 16 Oct 2020 02:04:04 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:8ee7c7a6-d23c-4dd3-bba5-4047e8cdf414</guid><dc:creator>alisonw</dc:creator><description>&lt;p&gt;Hi,&amp;nbsp;&lt;span&gt;Kristofer, I am still confused about the communication between Cortex-A and Cortex-M. As you mentioned, the current communication is using the Linux kernel mailbox APIs and virtio/rpmsg are not used. I want to know whether the current codes are sufficient to accomplish&amp;nbsp;the communication between Cortex-A and Cortext-M. Do virtio and rpmsg need to be used?&lt;/span&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/167492?ContentTypeID=1</link><pubDate>Wed, 16 Sep 2020 06:24:18 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:3be1cca9-c020-4b85-8a2a-fd24d703acf8</guid><dc:creator>alisonw</dc:creator><description>&lt;p&gt;Thanks a lot for your reply.&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/167482?ContentTypeID=1</link><pubDate>Tue, 15 Sep 2020 06:53:35 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:69a6d24d-df1e-428f-a381-da4cd52fe792</guid><dc:creator>Kristofer Jonsson</dc:creator><description>&lt;p&gt;1. Yes it is the Tensorflow framework from GitHub. The build system under &lt;em&gt;tensorflow/lite/micro/tools/make/&lt;/em&gt; is used to produce a static library, including CMSIS-NN and the Ethos-U driver. There is to my knowledge one small patch that has not yet reach upstream, that adjusts the build flags and a few paths to CMSIS-NN.&lt;/p&gt;
&lt;p&gt;2. OpenAMP is in the plan, but for the moment the communication is defined in &lt;em&gt;linux_driver_stack/kernel/ethosu_core_interface.h&lt;/em&gt; and does not make use of &lt;em&gt;virtio&lt;/em&gt; and &lt;em&gt;rpmsg&lt;/em&gt;. It does however use the Linux kernel maibox APIs, to abstract which hardware block that is used to trigger IRQs on the remote CPU.&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/167461?ContentTypeID=1</link><pubDate>Mon, 14 Sep 2020 03:43:52 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:6aea022c-675d-4471-b7cc-cc51d6f9930b</guid><dc:creator>alisonw</dc:creator><description>&lt;p&gt;Kristofer, thank you very much for your detailed reply. It&amp;#39;s very useful for me. I have two more questions.&lt;/p&gt;
&lt;p&gt;1. The&amp;nbsp;&lt;span&gt;TensorflowLite&lt;/span&gt;&lt;span&gt;&amp;nbsp;microcontroller framework you mentioned is the common source&amp;nbsp;&lt;/span&gt;&lt;a href="https://github.com/tensorflow/tensorflow"&gt;https://github.com/tensorflow/tensorflow&lt;/a&gt;, right? I checked the tensorflow directory in ethos-u/core_software, it seems the common one, no other external patches.&lt;/p&gt;
&lt;p&gt;2. Is OpenAMP used for IPC between Cortex-A and Cortex-M? According to the&amp;nbsp;&lt;span&gt;AMP communication you mentioned, it seems the current codes are&amp;nbsp;sufficient.&lt;/span&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item><item><title>RE: How to make Ethos-U NPU work on an ARM Cortex-A + Cortex-M processor?</title><link>https://community.arm.com/thread/167445?ContentTypeID=1</link><pubDate>Fri, 11 Sep 2020 09:32:51 GMT</pubDate><guid isPermaLink="false">dd9e70c8-6d3c-4c71-b136-2456382a7b5c:8a3707d9-2809-47be-b004-68ec878cf792</guid><dc:creator>Kristofer Jonsson</dc:creator><description>&lt;p&gt;&lt;span&gt;1.&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;The &lt;/span&gt;&lt;i&gt;&lt;span&gt;Linux driver stack&lt;/span&gt;&lt;/i&gt;&lt;i&gt;&lt;span&gt; for Arm Ethos-U&lt;/span&gt;&lt;/i&gt; &lt;span&gt;is provided &lt;/span&gt;&lt;span&gt;as an example&lt;/span&gt;&lt;span&gt; of&lt;/span&gt; &lt;span&gt;how an Arm Cortex-A running Linux can dispatch inferences to an Arm Ethos-U subsystem&lt;/span&gt;&lt;span&gt; (Arm Cortex-M, Arm Ethos-U, SRAM, &amp;hellip;)&lt;/span&gt;&lt;span&gt;. &lt;/span&gt;&lt;span&gt;This is a &lt;/span&gt;&lt;span&gt;so-called&lt;/span&gt;&lt;span&gt; Asymmetric &lt;/span&gt;&lt;span&gt;MultiProcessing&lt;/span&gt; &lt;span&gt;(AMP) &lt;/span&gt;&lt;span&gt;system&lt;/span&gt;&lt;span&gt;,&lt;/span&gt; &lt;span&gt;which &lt;/span&gt;&lt;span&gt;requires a small &lt;/span&gt;&lt;span&gt;amount of &lt;/span&gt;&lt;span&gt;shared &lt;/span&gt;&lt;span&gt;memory &lt;/span&gt;&lt;span&gt;and &lt;/span&gt;&lt;span&gt;an &lt;/span&gt;&lt;span&gt;external hardware &lt;/span&gt;&lt;span&gt;block&lt;/span&gt;&lt;span&gt; (e.g. the Arm MHU) to trigger IRQs on the remote CPU.&lt;/span&gt;&lt;span&gt;&amp;nbsp;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;The Linux driver stack currently contains a user space application, a user space driver library, and a kernel driver. &lt;/span&gt;&lt;span&gt;Important to notice is that the kernel driver will not drive the Arm Ethos-U &lt;/span&gt;&lt;span&gt;NPU &lt;/span&gt;&lt;span&gt;directly, but &lt;/span&gt;&lt;span&gt;instead &lt;/span&gt;&lt;span&gt;send&lt;/span&gt;&lt;span&gt;s&lt;/span&gt;&lt;span&gt; a message to &lt;/span&gt;&lt;span&gt;an Arm Cortex-M in &lt;/span&gt;&lt;span&gt;the &lt;/span&gt;&lt;span&gt;Arm Ethos-U subsystem &lt;/span&gt;&lt;span&gt;that &lt;/span&gt;&lt;span&gt;drives the NPU&lt;/span&gt;&lt;span&gt;.&lt;/span&gt;&lt;span&gt;&amp;nbsp;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;The setup of the AMP communication is platform dependent and is done in the DTB file.&lt;/span&gt;&lt;span&gt;&lt;br /&gt;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;2.&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;All software running on the Arm Cortex-M is referred to as &lt;/span&gt;&lt;i&gt;&lt;span&gt;core&lt;/span&gt;&lt;/i&gt;&lt;span&gt;.&lt;/span&gt;&lt;span&gt; The code that runs on the NPU is referred to as &lt;/span&gt;&lt;i&gt;&lt;span&gt;command stream.&lt;/span&gt;&lt;/i&gt;&lt;span&gt;&amp;nbsp;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;The Arm Ethos-U &lt;/span&gt;&lt;span&gt;(&lt;/span&gt;&lt;span&gt;sub&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;system &lt;/span&gt;&lt;span&gt;requires an &lt;/span&gt;&lt;span&gt;Arm Ethos-U, some SRAM and an &lt;/span&gt;&lt;span&gt;Arm Cortex-M&lt;/span&gt;&lt;span&gt; to drive the NPU.&lt;/span&gt; &lt;span&gt;The &lt;/span&gt;&lt;span&gt;(&lt;/span&gt;&lt;span&gt;sub&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;system is highly customizable&lt;/span&gt;&lt;span&gt;. A &lt;/span&gt;&lt;span&gt;customer may choose which &lt;/span&gt;&lt;span&gt;Arm C&lt;/span&gt;&lt;span&gt;ortex-M to use, the amount of SRAM, which peripheral to attach, which software to run &lt;/span&gt;&lt;span&gt;etc.&lt;/span&gt;&lt;span&gt;&amp;nbsp;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;Because of th&lt;/span&gt;&lt;span&gt;e&lt;/span&gt;&lt;span&gt; high &lt;/span&gt;&lt;span&gt;degree&lt;/span&gt;&lt;span&gt;s&lt;/span&gt; &lt;span&gt;of f&lt;/span&gt;&lt;span&gt;lexibility &lt;/span&gt;&lt;span&gt;Arm can&amp;rsquo;t provide a&lt;/span&gt; &lt;span&gt;ready &lt;/span&gt;&lt;span&gt;packaged software&lt;/span&gt;&lt;span&gt; to boot on the Arm Cortex-M&lt;/span&gt;&lt;span&gt;.&lt;/span&gt; &lt;span&gt;Core software &lt;/span&gt;&lt;span&gt;only &lt;/span&gt;&lt;span&gt;contains the necessary software components needed to run an inference using the &lt;/span&gt;&lt;span&gt;TensorflowLite&lt;/span&gt;&lt;span&gt; microcontroller framework.&lt;/span&gt;&lt;span&gt;&amp;nbsp;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;You would need to &lt;/span&gt;&lt;span&gt;write your own &lt;/span&gt;&lt;span&gt;main(&lt;/span&gt;&lt;span&gt;) function&lt;/span&gt;&lt;span&gt; t&lt;/span&gt;&lt;span&gt;o&lt;/span&gt;&lt;span&gt; initialize the &lt;/span&gt;&lt;span&gt;platform. &lt;/span&gt;&lt;span&gt;You will also need to &lt;/span&gt;&lt;span&gt;define a scatter file (Arm Clang) or &lt;/span&gt;&lt;span&gt;a &lt;/span&gt;&lt;span&gt;linker script (GCC) that describes the memo&lt;/span&gt;&lt;span&gt;ry layout&lt;/span&gt;&lt;span&gt;.&lt;/span&gt;&lt;span&gt; &lt;br /&gt;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;3.&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;All software publicly available for Arm Ethos-U can be downloaded with fetch_externals.&lt;/span&gt;&lt;span&gt;py&lt;/span&gt;&lt;span&gt; from the Arm Ethos-U repository.&lt;/span&gt;&lt;span&gt;&amp;nbsp;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://git.mlplatform.org/ml/ethos-u/ethos-u.git"&gt;&lt;span&gt;https://git.mlplatform.org/ml/ethos-u/ethos-u.git&lt;/span&gt;&lt;/a&gt;&lt;span&gt;&amp;nbsp;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;Other repos worth mentioning is Vela, which takes a &lt;/span&gt;&lt;span&gt;tflite&lt;/span&gt;&lt;span&gt; file as input and produces an&lt;/span&gt;&lt;span&gt;other&lt;/span&gt;&lt;span&gt; optimized &lt;/span&gt;&lt;span&gt;tflite&lt;/span&gt; &lt;span&gt;as output&lt;/span&gt;&lt;span&gt;. The optimized &lt;/span&gt;&lt;span&gt;tflite&lt;/span&gt;&lt;span&gt; file contains &lt;/span&gt;&lt;i&gt;&lt;span&gt;custom operators&lt;/span&gt;&lt;/i&gt;&lt;span&gt; that are executed on the Arm Ethos-U.&lt;/span&gt;&lt;span&gt;&amp;nbsp;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;The Arm Ethos-U IP has just recently been released, so it will still take some time before you can buy an Arm Ethos-U capable platform&lt;/span&gt;&lt;span&gt; to test on&lt;/span&gt;&lt;span&gt;. There are currently no virtual platforms available for download &lt;/span&gt;&lt;span&gt;on &lt;/span&gt;&lt;span&gt;the &lt;/span&gt;&lt;span&gt;Arm&lt;/span&gt; &lt;span&gt;website &lt;/span&gt;&lt;span&gt;that include the &lt;/span&gt;&lt;span&gt;Arm Ethos-U, but hopefully that will change in a not too distant future.&lt;/span&gt;&lt;span&gt;&amp;nbsp;&lt;/span&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;</description></item></channel></rss>