We are running a survey to help us improve the experience for all of our members. If you see the survey appear, please take the time to tell us about your experience if you can.
In my ARMNN network initialization routine I am creating an ARMNN runtime like this:
armnn::IRuntimePtr runtime = armnn::IRuntime::Create(armnn::IRuntime::CreationOptions());
armnn::IOptimizedNetworkPtr optNet = Optimize(*network, backends, runtime->GetDeviceSpec(), armnn::OptimizerOptions());
runtime->LoadNetwork(networkId, std::move(optNet), errorMessage);
I need to run the network in another C++ function. How can I pass the runtime pointer to that other function? Or, is there a way for the other function to retrieve the pointer to the runtime?
Hi,
The ArmNN team are very responsive to issues on their github: https://github.com/ARM-software/armnn/issues so questions for ArmNN are usually best there as first port of call for fastest response from those who know it best.
On the github site is also the documentation for ArmNN: https://github.com/ARM-software/armnn/wiki/Documentation where you can get more on all the classes for the different releases. Presuming you're looking at latest, 21.05, I looked up IRuntime and IRuntimePtr is just a std::unique_ptr of IRuntime, so it'll need to be passed into functions in the same way as unique_ptrs require. Unique pointers being unique, there can only be one of them and they can't be passed by value. The "good" way to do that is with a std::move(ptr) when calling the function (as you've got for optNet there in LoadNetwork()). The "bad" way is to pass it as a reference.
Unless you've already tried that and I'm missing something?
Cheers,
Ben
Hi Ben,
Thanks for your reply.
std::move(runtime) would work if I wanted to call the network execution function directly from the network initialization routine. But what I would like to do is to have the initialization routine *save* the pointer to the runitme, and then having the execution function use that pointer to run the network. I can't use a static variable to save the pointer to the runtime in the initialization routine. And I can't seem to find a way to retrieve the pointer to the already created runtime. That is what I am struggling with...
Any idea how to do it?
I'm a little confused, but it does sound like you're fighting the concept of unique pointers - you definitely can't save them for later.
There is armnn::IRuntime::CreateRaw to give you a Runtime* - would that work for you?
That's exactly what I was looking for. Thank you, Ben, for your very prompt assistance!
It depends if you are training the model, say using Tensorflow, you can do that entirely with x-86 if you want to.Then that model can be used by the ArmNN on your board to run inferences.
If you need to link the arm64 Arm NN libraries to an executable or library then that executable/library needs to be compiling for arm64 as well. So you would have to use the cross compiler. I had a similar issue while working on chatbot development services for my company, and if you visit that page you will get to know how handled it.
Profiling is enabled using the ExternalProfilingOptions inside IRuntime::CreationOptions, which are passed in when Runtime is created.
Like this for example:
IRuntime::CreationOptions options;options.m_ProfilingOptions.m_EnableProfiling = true;options.m_ProfilingOptions.m_TimelineEnabled = true;IRuntimePtr runtime(IRuntime::Create(options));
Some executables calling Arm NN, like ExecuteNetwork, may also have a command line option to do this.With ExecuteNetwork it is -a for instance.