Hello all,
This is my first time posting on a forum, so all feedbacks are welcome in order to improve !
I'll begin with the context information, followed by the issue encountered, and at the end, you'll find all the setup information (e.g. compiler version, and so on).
The context is as follows :
We have several applications that we want to support with different compilers : Clang (Keil), GCC, and IAR.
We also have two libraries that we offer in black box form and with an API defined in our headers so that people using our applications only focus on the application itself and not what's behind the scenes. To achieve this, we built the libraries with Keil and following the AEABI recommendations (Misc Controls added in the Linker panel : --library_interface=aeabi_clib).
Functionally speaking, it is working fine on the 3 compilers when we use the Keil libs into our applications.
The issue I'm facing is how to debug such libs.
When the application is a Keil project, so the app is Keil, and libs are Keil, the debugging experience is excellent, even with optimizations enabled. The call stack is 100% consistent, no address jumps or offsets are introduced, all function calls are perfectly followed.
However, when the application is let's say an IAR one, so the app is built with IAR, and the libs are still made with Keil, it sometimes happens that an offset is introduced, leading to some function calls not being tracable, like let's say this case :
At first, I thought it was coming from optimizations, but after removing them during investigations and regarding how the debugging experience for a 100% Keil case was with them enabled, I arrived at the conclusion that it might come from a debug information mismatch between Keil generated debug information and how the IAR debugger interprets them.
However, I also found on this github : https://github.com/ARM-software/abi-aa/blob/main/aadwarf32/aadwarf32.rst#overview that "The ABI for the Arm architecture specifies the use of DWARF 3.0-format debugging data". According to the standard itself, https://dwarfstd.org/dwarf4std.html, gdwarf 3.0 and 4.0 are compatible.
Anyway, I've also tried setting gdwarf 3.0 as the debug format for Keil libs by adding this into the Compiler Misc Control panel (and by removing the use of 'Debug Information" in the Output panel) : -g -g3 -gdwarf-3 -gstrict-dwarf
But it didn't work either way.
Does anyone know what's going on behind the scenes and how to fix this if it happens to any of you please ? I'm running out of ideas for debugging Keil libraries on IAR.
If the debug information mismatch isn't the subject to investigate, what advice and investigations tips can you give me please ?
Setup information :
Thank you for your help to understand what's happening ! I wish you a great day,
Max
the order of the tag
-g -g3 -gdwarf-3 -gstrict-dwarf
is probably best re-written as:
-gdwarf-3 -gstrict-dwarf -g3
.. but this change may not have an impact on what you are trying to do here.
As for the IAR side, maybe something like this will help?
www.iar.com/.../
Hello Kevin,Thank you for your reply.I tried, just in case, the new compiler options, unfortunately it didn't work any better.
As for the IAR side, I had already tried the suggestions at this link before, but got the same results in the call stack view : kind of an offset at some point during program execution, like a black box where debugging within is not possible: [ Function C + 0x31 ].
Do you perhaps have a tool to convert debug information from Keil to IAR ? I'd like to try giving to the IAR debugger fully compliant debug information from the lib made with Keil.I'm not really sure this would work. However, from what I've seen so far and because debugging is working on a 100% Keil project, I believe the issue either come during linking on IAR side, or from the IAR debugger's interpretation of the content in the debug sections.Hence my request to test this way.
Thank you for your help.Max
This thread also passed through my eyes and sadly I don't have a good answer to give you, but I spotted your reference to the ABI seemingly requiring DWARF-3. I believe that in practice DWARF-4 is widely used these days and is well supported by AC6, so you may be better off sticking to DWARF-4 across all tools.I raised https://github.com/ARM-software/abi-aa/issues/239 for this to be tracked on the ABI side -- maybe the answer will be that the wording should be updated to "require a minimum of DWARF-3".
Hello Stam,
I agree with Dwarf-4 being widely supported, hence the reason I first enabled the debug information through the debug panel rather than specifying it explicitly on the command line. I gave a shot to both formats, unfortunately the underlying issue remained.
Thank you for opening case 239, that'd clarify this point as you mentioned.
I would suggest you to check the following things:
1) I don't think --library_interface=aeabi_clib is an armlink option for AC6, it seems an AC5 compiler option:
https://developer.arm.com/documentation/101754/0621/armlink-Reference/armlink-Command-line-Options
https://developer.arm.com/documentation/dui0472/m/Compiler-Command-line-Options/--library-interface-lib?lang=en
2) you may make a test with the latest Arm compiler v6.21 to see if there is any difference.
3) you may use the fromelf utility with the option -g to print out the debug info of a lib or image that you want to use with IAR debugger. You may get an insight what could go wrong when IAR debugger works with that debug info on the place where the issue occurs
4) LLVM clang offers the -gsplit-dwarf compiler option, which allows the generation of split DWARF debug information into a dedicated file. This feature is available in armclang as a community feature. However, there is currently no armlink option available to enable the debugger to locate and read the debug information file, as is provided by --gdb-index in llvm lld. It's also worth noting that split-dwarf is primarily aimed at speeding up incremental builds rather than facilitating debug information distribution. Additional tools, such as GNU dwp or llvm-dwp, are required to further process the split-dwarf files. You may find the following resource helpful in understanding the process of packaging a release from a split DWARF build: https://www.productive-cpp.com/improving-cpp-builds-with-split-dwarf/. On the other hand, you may potentially use the generated debug info file to analyze the root cause of the issue you saw in IAR debugger
-gsplit-dwarf
armlink
--gdb-index
lld
Hello Chen,
Thank you for your answer and all tips & links.
1. Indeed, --library_interface=aeabi_clib is AC5 with no known port to AC6. I tried the just in case the option, but it doesn't seem to make sense. It is now removed from the project configuration.
2. Nothing has changed, the problem is still there. I also tried with IAR 9.50.1, but it's still the same.
3. I tried using fromelf again and to see where the debugging and symbol reallocation information I'm looking for was located, I can indeed confirm that it is here, in valid sections. I didn't see anything alarming.
4. Thanks for letting me know about this option! It was worth a try, even if the result was not successful.
I think you need to contact IAR and provide them with your files so that they can adapt their debugger. Even if compilation tools are DWARF compliant, they have some freedom to generate the debug information. I assume that the IAR debugger is not fully interpret the debug information provided by the Arm compilation toolset.
Hello Hans,
Indeed, I suppose we did everything we could on Keil's side to make it work. I'll keep this thread open with no accepted answer, so if IAR provides a working one, it could be shared here as well.After that, there'll be Keil from GCC to test.
Hello everyone, a little message to keep you informed of the progress of this post on the forum.
It seems to come from the C-Spy debugger. A case was opened on IAR's side to confirm (or not) the issue. IAR versions tested and impacted are 9.20.1 & 9.50.2.
The last point was to debug an AEABI Keil optimized lib with a GCC project: even with optimizations enabled, the debugging experience here is completely consistent. No issues.As stated previously, I will keep this thread open with no accepted answer until one is found.
Thanks again for all your advice and information provided so far.Max