This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Issues while running benchmarks for Arm RAN Acceleration Library version 23.04

Hi,

I have installed ARM RAL version,23.04 on my Linux machine and successfully built static libraries for Release  mode with Arm GNU Toolchain 12.2.Rel1 (arm-gnu-toolchain-12.2.rel1-x86_64-aarch64-none-linux-gnu.tar.xz) for targets NEON and SVE.I enabled build options ,-DBUILD_TESTING=On` and `-DBUILD_EXAMPLES=On. Python version installed on my machine is 3.8.10.and Linux, Perf tool version is perf version 5.15.87,required for running benchmarks.

After build,I installed ARMRAL libraries for 2 targets in 2 different installation folders.

Afterwords,I tried running benchmarks with command, make bench for 2 targets separately one by one but for each one I am getting errors like following for all benchmarks.....

If I direct o/p to file,JSON objects printed are like below  indicating that benchmarks not ran at all.:--

{"name": "demodulation_qam256_8", "error": true, "ts": 304990550, "dur": 192588}
{"name": "demodulation_qam256_31", "error": true, "ts": 305183343, "dur": 192250}
{"name": "demodulation_qam256_32", "error": true, "ts": 305375812, "dur": 194988}

Since I am evaluating ARMRAL for my work and new to ARMRAL, could you kindly look into the error and help me to identify root cause(eg whether I m missing something/making some error) and fix the error soon?

It would help a lot.

Regards,

Pankaj K

---------------------------------------------------------------------------------------------------------------------------------------------------------

Failed to run command: ['/home/sitteam/PankajK/arm-ran-acceleration-library-23.04/bench/default_runner.py', '/home/sitteam/PankajK/arm_lib_2304_build_NEON_static_rel/bench_demodulation', '{"name": "demodulation_qam16_276", "args": "1 276", "reps": 300000}']
Traceback (most recent call last):
File "/home/sitteam/PankajK/arm-ran-acceleration-library-23.04/bench/default_runner.py", line 65, in <module>
main()
File "/home/sitteam/PankajK/arm-ran-acceleration-library-23.04/bench/default_runner.py", line 61, in main
run(args.exe_path, case_json)
File "/home/sitteam/PankajK/arm-ran-acceleration-library-23.04/bench/default_runner.py", line 37, in run
all_cycles = [run_perf(args) / float(reps) for _ in range(10)]
File "/home/sitteam/PankajK/arm-ran-acceleration-library-23.04/bench/default_runner.py", line 37, in <listcomp>
all_cycles = [run_perf(args) / float(reps) for _ in range(10)]
File "/home/sitteam/PankajK/arm-ran-acceleration-library-23.04/bench/default_runner.py", line 20, in run_perf
result = exec_and_check_cmd(cmd)
File "/home/sitteam/PankajK/arm-ran-acceleration-library-23.04/bench/default_runner.py", line 14, in exec_and_check_cmd
result.check_returncode()
File "/usr/lib/python3.8/subprocess.py", line 448, in check_returncode
raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '['perf', 'stat', '-x', ' ', '-e', 'cycles:u', '/home/sitteam/PankajK/arm_lib_2304_build_NEON_static_rel/bench_demodulation', '1', '276', '300000']' returned non-zero exit status 255.

Parents
  • Hi.

    Unfortunately there's not great tooling around that use-case if you're not on an Arm-based system.  Your best options are QEMU and DynamoRio, assuming you're happy with instruction counting.  However perf won't give you anything useful, and you'd also need a model of how the cores run to be able to work out anything useful which are not open source things.

    Sorry.

    Chris

Reply
  • Hi.

    Unfortunately there's not great tooling around that use-case if you're not on an Arm-based system.  Your best options are QEMU and DynamoRio, assuming you're happy with instruction counting.  However perf won't give you anything useful, and you'd also need a model of how the cores run to be able to work out anything useful which are not open source things.

    Sorry.

    Chris

Children
  • Hi Chris,

    Thanks for your reply.I am checking with my manager abt  getting Arm system access  or access to Cloud service for Arm-based Linux instance as you indicated.

    1 issue I am still facing is following:--

    As indicated in "Run tests" section of readme, since I am not developing on an AArch64 machine, I  compiled to configure the library to
    prefix the tests with `qemu-aarch64` for NEON as follows to run the tests on my x86_64,ubuntu linux machine :--

    cmake -DBUILD_TESTING=On -DARMRAL_TEST_RUNNER=qemu-aarch64 -DBUILD_EXAMPLES=On -DBUILD_SIMULATION=On -DCMAKE_INSTALL_PREFIX=/home/sitteam/PankajK/arm_lib_2304_install_neon_static_rel /home/sitteam/PankajK/arm-ran-acceleration-library-23.04 -DCMAKE_C_COMPILER=/home/sitteam/PankajK/arm_gnu_toolchain_12_2/arm-gnu-toolchain-12.2.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc -DCMAKE_CXX_COMPILER=/home/sitteam/PankajK/arm_gnu_toolchain_12_2/arm-gnu-toolchain-12.2.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-g++

    make check 

    ------------------------------------------------------------------------------

    But I got following error abt which I am unsure, I mean why qemu-aarch64 executable is not getting generated despite providing cmake option in bold as above? Requesting to provide pointers to debug/resolve  this issue. .Also can you provide more info abt "Dynamo Rio"?

    Start 56: tail_biting_convolutional_decoding
    Could not find executable qemu-aarch64
    Looked in the following places:
    qemu-aarch64
    qemu-aarch64
    Release/qemu-aarch64
    Release/qemu-aarch64
    Debug/qemu-aarch64
    Debug/qemu-aarch64
    MinSizeRel/qemu-aarch64
    MinSizeRel/qemu-aarch64
    RelWithDebInfo/qemu-aarch64
    RelWithDebInfo/qemu-aarch64
    Deployment/qemu-aarch64
    Deployment/qemu-aarch64
    Development/qemu-aarch64
    Development/qemu-aarch64
    Unable to find executable: qemu-aarch64
    56/56 Test #56: tail_biting_convolutional_decoding ...***Not Run 0.00 sec

    0% tests passed, 56 tests failed out of 56

    Total Test time (real) = 0.01 sec

    ------------------------------------------------------------------------------------------------

    Regards,

    Pankaj K

  • Hi there,

    Unfortunately qemu-aarch64 is not distributed as part of ArmRAL. Instead please see https://www.qemu.org/ for instructions on how to download and build it for your machine.

    Once you have QEMU installed you should provide the full path to the qemu-aarch64 executable in the ARMRAL_TEST_RUNNER CMake variable. Alternatively, if you place qemu-aarch64 on your executable path you can set -DARMRAL_TEST_RUNNER=qemu-aarch64  as you did before.

    Regards,

    Nick