In this article I tackle the classic question engineers developing software on custom SoCs grapple with constantly:
How do I test my software before the hardware team gets me a working silicon chip?
No ‘one size fits all’ solution is provided here (look for that alongside my pet unicorn); instead I detail an easy-to-use yet powerful approach to solve this problem for a particular development scenario.
If you are developing software for SoCs and you or your team does any of the following:
then read on. If not, keep reading anyway, as this methodology can be used effectively in a variety of situations.
Continuous integration is a software development practice where team members merge their work at least daily, and each merge is checked by an automatic build process to identify errors. This enables developers to avoid the dreaded 'merge-commit hell' that occurs when attempting to combine heavily edited and interwoven applications together, not to mention the possibility of inducing errors from not properly stress-testing code. See 'Why CI is important' article for an in-depth look for why continuous integration is so important when developing software. Developing software for SoCs, however, adds another layer of complexity in choosing what platform to run the relevant software on.
Many options exist in this space: Emulation, FPGAs, racks upon racks of hardware development boards, virtual prototypes, etc. Each has their benefits and drawbacks in certain development situations. To implement continuous integration a solution that enables fast software run-times, inexpensive scaling for large teams, and easy integration with continuous integration platforms is essential. Based on these metrics I chose virtual prototypes as the hardware model, specifically Arm's Fast Models. Keep an eye out for a future blog on why this is the best choice, as a thorough explanation is beyond the scope of this blog.
The solution proposed here is a methodology to integrate virtual prototype simulation into an existing continuous integration framework, such as Jenkins or Bamboo. This allows SoC software developers to stop worrying about physical hardware and focus their energy on writing excellent code. The Component Architecture Debug Interface (CADI) an API for simulation control and debug access to software running on Arm virtual platforms--allows python to interact with Arm Fast Models and provides a connection between hardware and software status. Python, using CADI commands, can control the execution of applications, set breakpoints, read and write to memory, access registers, view test variables, and more. This information can then be utilized to create comprehensive software tests, and can interface with an existing continuous integration framework. This article will cover how to use python's CADI platform (cleverly named PyCADI), including (1) how to setup the development environment and (2) a walk-through of example code with results.
The example use-case here is for a team that uses continuous integration to develop software on an Arm Cortex-M4. To enable automated software verification, it is standard practice for them to use a variable in every test to indicate the test result. This variable, dubbed ‘test_pass’, defaults to 0 and is set to 1 if the test passes without error. This technique helps each test plug into the team's general regression scripting.
Let’s get started.
This section is separated into the ‘First-time setup only’ and the ‘Possibly reoccurring setup’ for steps that may need to occur before testing can be run, depending on what the goal is and what variables change across tests.
The first step to running software on Fast Models is to get a Fast Model! Click the button below to obtain an evaluation license if not already downloaded. Follow the installation guide specified in the downloaded package for Windows or Linux.
Fast Models Downloads
The PyCADI library requires python 2.7.13 (very specific I know), so ensure that python 2.7.13 is installed (along with an IDE of choice). See the button below to download the correct version of python. Note: Using more recent versions of python 2.7 should work but is untested. Use with caution!
Python 2.7.13 Download
To implement breakpoints on function names and specific lines in files with my personal method, two Arm utilities are required: arm-none-eabi-objdump and arm-none-eabi-addr2line. Once downloaded and installed, ensure that the toolchain bin is added to the system’s Path. For example, on Windows this path should be similar to:
C:\Program Files (x86)\GNU Tools ARM Embedded\6 2017-q2-update\arm-none-eabi\bin
For Linux, the functions should be placed in:
/usr/bin/
In my experience this path was automatically added on Linux but not Windows. To verify the toolchain is properly in the system path, type “arm-none-eabi-objdump” into the command prompt or terminal. The help information should come up if configured correctly.
An executable application is of course necessary for testing said executable application. The details of how to create apps is not covered in this blog for software application developers. Note the absolute path to whatever application is going to be tested as PyCADI will need to locate it.
A Fast Model representing the end hardware must also be provided. Creating a Fast Model system from scratch is beyond the scope of this blog, but see the reference guides located under <install_directory>\FastModelsPortfolio_<version>\Docs for tutorials on getting up and running with Fast Models. In this example a simple Cortex-M4 core is created in System Canvas with a clock source and memory.
One important note for compiling a Fast Model system for python scripting is the build settings. With the relevant system open in System Canvas, navigate to “Project > Project Settings > Targets” and ensure that ‘CADI library’ is checked and nothing else. Then, when applying these changes and selecting “Build”, System Canvas will generate a “.so” file in Linux and a “.dll” file in Windows, both having the name “cadi_system_<system_type>”. Building with the settings in the screenshot below, the model name will be “cadi_system_Win64-Release-VC2013.dll”.
Now that the setup is complete and a model + an application are in hand, let's dive right into an example going line by line. This example will be available to download at the end of this article. Recall that the team's objective here is to check if the test passed in their regression environment. The variable ‘test_pass’ should default to 0 and change to 1 if the app checks out without error. Accordingly, ‘test_pass’ should be checked at the test start and test end.
Up first, importing the necessary modules and setting the relevant path for PyCADI's python. This path setting is primarily for Linux based systems, and fails if the tools for Fast Models are not sourced. fm.debug is the PyCADI module.
import sys, os # Set python path to Fast Models try: sys.path.append(os.path.join(os.environ['PVLIB_HOME'], 'lib', 'python27')) except KeyError as e: print "Error! Make sure you source all from the fast models directory. Try something like this:" print "$ source <FMinstallDirectory>/FastModelsTools_11.0/source_all.sh" sys.exit() import fm.debug import subprocess
Next the test variables are specified; here the absolute paths to the model and the application itself are provided. Also indicated is a register and variable to track during the test. R15 is a register of interest as the Cortex-M4's Program Counter, which can verify where breakpoints are hit in memory. The variable is of course 'test_pass' to check if the test passes. Finally the three different types of breakpoints that can be set with my implementation method are initialized: (1) At a hexidecimal address (location in program memory), (2) on a function’s start (main in this example), and (3) at the line number of a file. Note that setting a breakpoint on a file line number will only work if it is breakpointable, which sounds obvious but does not allow setting on an empty line, one with a bracket, etc. If a mainstream debugger can set a breakpoint there, so can PyCADI.
It should be noted that the method of matching the function names and line numbers to the program's memory location is my individual implementation of these features with the Arm toolchain. PyCADI natively supports setting breakpoints in these cases: When program execution reaches a memory address, when a memory location is accessed, and when a register is accessed. Setting breakpoints on function names and line numbers requires clever computation, with one example of such shown in this blog.
# Path specifications model_path = 'C:\Users\zaclas01\itm_integration_windows\Model\cadi-Win64-Release-VC2013\cadi_system_Win64-Release-VC2013.dll' app_path = 'C:\Users\zaclas01\itm_integration_windows\Applications\itm_app\startup_Cortex-M4.axf' # Variable and register to check cpu_reg = 'R15' app_var = 'test_pass' # bpt1 bpt_hex_addr = '0x000001b0' # bpt2 bpt_function_name = 'main' # bpt3 file_name = 'main.c' line_number = 97
In the code snippet below python starts interacting with the CADI port of the model. The module fm.debug creates a model object, from which a target object ‘cpu’ is obtained (the first, and only cpu, in this model), and the specified application is loaded onto the cpu. The disassembly of all application instructions is necessary in this implementation to map function names to hexadecimal locations in program memory where a breakpoint can be set.
# Load model model = fm.debug.LibraryModel(model_path) # Get cpu cpu = model.get_cpus()[0] # Load app onto cpu cpu.load_application(app_path) # Get disassembly ordered_disassembly_list = subprocess.check_output('arm-none-eabi-objdump -S %s' %app_path, shell=True).split("\n")
Speaking of breakpoints, this section details how they are set. As previously mentioned, all breakpoints here are set using the ‘cpu.add_bpt_prog(<hex_addr>)’ command, stopping when program execution reaches an address in program memory. This functionality is built into the fm.debug module of PyCADI. As all breakpoints need a known address, more processing is required to match a function name or file line number to its corresponding address in memory. This section details how my implementation goes about doing so.
The first breakpoint specified is a hex address in the program, and is simply fed into the relevant command creating a breakpoint. The hexadecimal string is converted into an integer so the fm.debug module understands the location correctly. The second breakpoint, on the start of the function ‘main’, obtains its hexadecimal location by looking for the function name in the disassembly. Once found, the hex location is extracted and a breakpoint is created. Third, by cycling through the application's instructions and seeing what line of code created that instruction, the relevant memory location is determined and set as a breakpoint. All breakpoints are added to a breakpoints list for easy processing later.
# Keep list of bpts bpts = [] #----------- Set on hex location bpt1 = cpu.add_bpt_prog(int(bpt_hex_addr,16)) #----------- Set on start of a function # Iterate over the disassembly list until the function is found for line in ordered_disassembly_list: # line at start of function has form: # 0000004c <main>: # where it is a hex value followed by the func name in <>: if line.rstrip().endswith(" <%s>:" %bpt_function_name): hex_addr = "0x"+line[:8] bpt2 = cpu.add_bpt_prog(int(hex_addr,16)) break #----------- Set on line number of file for i in range(2000): # Cycle through all instruction addresses until Arm tools finds a match addr = hex(i) raw_out = subprocess.check_output('arm-none-eabi-addr2line -e %s %s' %(app_path,addr), shell=True) # If addr has a file it came from: if '/' in raw_out: #Get file name and line number file_line_list = raw_out.rstrip().split('/')[-1].split(':') # If file name is the same if file_line_list[0] == file_name: # If line number is the same if int(file_line_list[1]) == int(line_number): # Format address to be more readable hex_addr = '0x'+addr[2:].zfill(8) bpt3 = cpu.add_bpt_prog(int(hex_addr,16)) break # Add the three breakpoints to the bpts list bpts.append(bpt1) bpts.append(bpt2) bpts.append(bpt3)
Finally the moment we have all been waiting for: Running the application! This example is set up to run the model until a breakpoint is hit or until 5 seconds passes with no breakpoint, whichever happens first. Here is a simple breakdown of the code flow:
This code sample runs the application in blocking mode; the app can also be run in non-blocking mode to allow for additional computation during app runtime. The timeout time can also be changed.
# Run model for 5 seconds; if no breakpoints are hit, exit the application print 'Starting application...' while 1: try: model.run(timeout=5) except: # On timeout, no breakpoints have been hit and the model will hang at the end of the application model.stop() break # Test stopped due to a breakpoint # Find what breakpoint was hit for bpt in bpts: if bpt.is_hit: print "breakpoint %s was hit" %bpt # Get variable status if "lin" in sys.platform: raw_out = subprocess.check_output('arm-none-eabi-objdump -t %s | grep %s' %(app_path,app_var), shell=True) elif "win" in sys.platform: raw_out = subprocess.check_output('arm-none-eabi-objdump -t %s | Findstr %s' %(app_path,app_var), shell=True) hex_location = "0x" + str(raw_out).split()[0] var_status = cpu.read_memory(int(hex_location,16),count=1)[0] # The 'count=1' indicates that the variable is one unit print 'Variable %s value is: %s' %(app_var,var_status) # Get register status reg_status = str(hex(cpu.read_register(cpu_reg))) print 'Register %s value is: %s' %(cpu_reg,reg_status) # Model done running #----------------------------------------------------------------- # Get variable status if "lin" in sys.platform: raw_out = subprocess.check_output('arm-none-eabi-objdump -t %s | grep %s' %(app_path,app_var), shell=True) elif "win" in sys.platform: raw_out = subprocess.check_output('arm-none-eabi-objdump -t %s | Findstr %s' %(app_path,app_var), shell=True) hex_location = "0x" + str(raw_out).split()[0] var_status = cpu.read_memory(int(hex_location,16),count=1)[0] # The 'count=1' indicates that the variable is one unit print 'Variable %s final value is: %s' %(app_var,var_status) # Get register status reg_status = str(hex(cpu.read_register(cpu_reg))) print 'Register %s final value is: %s' %(cpu_reg,reg_status) print 'Test End'
That is the entirety of the sample code! While being quite simple and easy to digest, the results are powerful. Here is the output when running the above code (with some additional print statements included for clarity):
Here it is clear to see that ‘test_pass’ indeed starts set to 0 and returns 1 once the test is completed, indicating a successful test. This status can be sent to other applications or a continuous integration platform in a variety of different ways, such as creating a verification text file or returning the status to another program that called the PyCADI script.
Using PyCADI with Fast Models offers a lightweight yet powerful methodology for streamlining the continuous integration process for SoC software developers. As opposed to solutions involving racks of hardware boards, FPGAs, or hardware emulators, PyCADI with Fast Models is inexpensive, simple, and fast. It can also integrate effectively with existing continuous integration platforms. Ultimately utilizing this virtual hardware scripting system enables a more powerful use of the continuous integration practice, leading to reduced merge conflicts and faster time to market. All that leads to a more enjoyable workflow and more money in the bank.
Further, because this method of scripting is done with python, all of python’s modules and resources are available when scripting. This opens the door to endless possibilities when integrating with other environments and analyzing data. There is full API documentation of PyCADI which details all possible fm.debug object parameters and settings, but for the sake of convenience I put together a quick ‘cheat-sheet’ of useful commands to get started with PyCADI and Fast Models right away. That sheet, along with a sample Cortex-M4 model, is provided for free in the zip folder below. Using the files in this zip folder along with Fast Models, python, and your own applications, you can start improving your continuous integration system in a matter of hours.
Download now, and happy coding. Rather, download now for happy coding!