Hello everyone,
today I'm asking for hints on a tricky problem. We have a firmware that uses RTX-Kernel running on a NXP LPC2368. Now the device that the firmware is written for should get a new lc display. My honest mission is to change the firmware in order to use the new display.
I've spent some weeks this year to do so and some time I've had the problem that the controller resets short time after start and again and again...
Everytime this behaviour occured I have deleted one or more obsolete variables (mostly global) or functions. In most cases I solved the problem by searching other obsolete variables and deleting them from source code - try and error. That is really time-killing.
While testing the firmware on wednesday, I tried to make the adopted and modified routine for writing data to display RAM a little faster. I moved an global unsigned int to the function and changed it to static unsigned char because the value it has to carry is 0x0D at a maximum.
After flashing the firmware in the controller, the controller hung at a random short time.
Yesterday I was trying to solve the problem with hanging firmware on random time and found the problem when no task is running: OS calls os_idle_demon() and was not able to return from it. I found a solution in world wide web: Creating an empty low priority task without using any os_wait functions that prevents the OS from calling the idle task. (It has something to do with incorrect interrupt states on retunring from idle task.)
Today I further tried to make the display writing function faster and changed two unsigned char inside the function from static to non-static. After flashing this firmware the controller resets again and again. I will now try to find out why the controller behaves this way.
What I found out is, that no watchdog is enabled by user (is it part of the OS?). The os_stk_overflow an os_idle_demon are not called from OS. I debug the firmware using ULINK2.
Any ideas where to search the problem for?
Best regards
Maybe writing a pattern into user defined stacks can tell you which task is overwritten and maybe even by how much. On the other hand, RTX does have such a mechanism, and it is not triggered. Maybe the stacks are the ones writing one of _your_ buffers which causes entry into abort, and thus is not a stack overrun at all?
Note that it is possible to have a stack overflow while jumping past the marker the OS may use for overflow detection. When a program declares lots of auto variables, the stack may overflow but with holes not used - for example a 100char write buffer that isn't completely filled.
A simulator that explicitly keeps track of the stack pointer can detect such a stack overflow. But an OS that is limited to a single marker word can not.
And as noted - stack overflows are bad, but it is quite easy to get similar problems by having buffer overruns or using uninitialized pointers or uninitialized array indices.
Maybe ...
It's one of those Dealy linker bugs???
Dealy? Deadly! Since I'm a native german, I don't know how to handle the last post. Is there any linker bug known that I can check my linker for?
Some more background information for you: 'Use Microlib' is enabled in the target options. I do not know if it has any relevance.
I really do not have any idea where to search. I spent the last hours running some debugger sessions and tried to get a fixed point to catch the error, but everytime the program behaves different. Now my program code is at the point it was this morning but it is not doing a reset any more... really no difference to the code some hours ago, but no reset occurs.
I wish I had a pro here right by my side! I am tired of this sick program.
Is there a chance to use any simulator with this ability with my program?
Hi Robert,
Please don't take that last post seriously.
The general consensus is that there are no serious bugs in the linker.
(Unless someone knows something different that they don't want to share.)
S(tunned) Steve is just bored and wants to throw gravel in the machinery by h(a)unting Tamir about a previous thread. Nothing you need to worry about.
I don't think anyone has asked yet but what is the source of the reset?
In other words what is the value of the RSID after the reset?
M
I think Marc Crandall is right. And maybe it is not a Reset, it just looks like a Reset.
Hi Robert Suess,
Did you implement any of the
Undef_Handler SWI_Handler PAbt_Handler DAbt_Handler
or some kind of software reset functionality?
I see, thank you for all the answers.
@S Steve: Ok, thank you for sharing your refreshing ideas. :) I was not quite sure if you where trolling around.
Should I go with the flow or should I make my own experiences, what do you think?
@Per: Thank you for enlighten me about S(tunned) Steve. ;)
@Marc: Welcome to my thread and thank you very much for your input! I will try to enforce the reset again and if successful, I will check the value of the RSID after the reset like you suggested.
To answer Johns question: It is very difficult for me to find all pieces of the puzzle since I not wrote the firmware on my own. It is a crackbrained mix of a very old firmware written for a 8bit controller, a non actual RTX USBCDC example project written for 'Keil MCB2300' and a patchwork of code snippets to make the firmware behave like it should. And no comments are in the code. Do not get me wrong, it is a great achievement that the firmware runs like it is expected to do. But for me as programmer it is hard to find errors if they occur now.
The information I can give this moment is, if it helps to clarify your question John:
; Exception Vectors ; Mapped to Address 0. ; Absolute addressing mode must be used. ; Dummy Handlers are implemented as infinite loops which can be modified. CDCVectors LDR PC, Reset_Addr LDR PC, Undef_Addr LDR PC, SWI_Addr LDR PC, PAbt_Addr LDR PC, DAbt_Addr NOP ; Reserved Vector ; LDR PC, IRQ_Addr LDR PC, [PC, #-0x0120] ; Vector from VicVectAddr LDR PC, FIQ_Addr Reset_Addr DCD Reset_Handler Undef_Addr DCD Reset_Handler;Undef_Handler SWI_Addr DCD SWI_Handler PAbt_Addr DCD Reset_Handler;PAbt_Handler DAbt_Addr DCD Reset_Handler ;DAbt_Handler DCD 0 ; Reserved Address IRQ_Addr DCD IRQ_Handler FIQ_Addr DCD FIQ_Handler IMPORT SWI_Handler EXTERN DAbt_Handler ; RoS| 29.11.11: for RT-Agent (http://www.keil.com/support/man/docs/ulink2/ulink2_ra_modifying_startup.htm) Undef_Handler B Undef_Handler ;SWI_Handler B SWI_Handler PAbt_Handler B PAbt_Handler ;DAbt_Handler B DAbt_Handler ; RoS| 29.11.11: for RT-Agent IRQ_Handler B IRQ_Handler FIQ_Handler B FIQ_Handler ; Reset Handler EXPORT Reset_Handler Reset_Handler
This is part of the modified startup file lpc2300.s. Hope this helps.
It's a pretty safe bet to say that there are no serious bugs (and probably very few minor ones).
There is one person who claimed there was a dealy bug in the linker recently. However the evidence given was stunningly flawed.
I see.
I can not make the firmware reset permanently at startup today. I will try further on monday.
Thanks again for all hints and comments!
I only reply to technical statements made by the stunned among us. Now that he finally made one (!) he will get an answer:
First of, the linker, at least the one provided with MDK 4.14 is not perfect. Do you work with CM0 devices (LPC1114)? If you do, you might have encountered a failure to link with less that 0x2000 bytes of RAM available in the scatter file - fixed _ONLY_ by changing 0x2000 to 0x5000, linking, and then going back to the original setting (0x2000), which then links as well (!).
Now that he finally made one (!) he will get an answer:
Are you trying to take the proverbial? I replied to your mis-interpretation with an appropriate and valid question in:
http://www.keil.com/forum/19955/
Look carefully at how you answered it.
Professional ... I think not.
@Robert
Your fault handlers are simple while(1)s. Maybe to proceed you could implement more informative handlers to see if you can gather more information about where this fault (if any) is coming from.
Actually I didn't examine your source properly. It looks like all of your handlers are pointing to the Reset handler.
For starters put the while(1)'s back and see if you get stuck in one of these handlers.
Like so:
; Exception Vectors ; Mapped to Address 0. ; Absolute addressing mode must be used. ; Dummy Handlers are implemented as infinite loops which can be modified. CDCVectors LDR PC, Reset_Addr LDR PC, Undef_Addr LDR PC, SWI_Addr LDR PC, PAbt_Addr LDR PC, DAbt_Addr NOP ; Reserved Vector ; LDR PC, IRQ_Addr LDR PC, [PC, #-0x0120] ; Vector from VicVectAddr LDR PC, FIQ_Addr Reset_Addr DCD Reset_Handler Undef_Addr DCD Undef_Handler SWI_Addr DCD SWI_Handler PAbt_Addr DCD PAbt_Handler DAbt_Addr DCD DAbt_Handler DCD 0 ; Reserved Address IRQ_Addr DCD IRQ_Handler FIQ_Addr DCD FIQ_Handler IMPORT SWI_Handler EXTERN DAbt_Handler ; RoS| 29.11.11: for RT-Agent (http://www.keil.com/support/man/docs/ulink2/ulink2_ra_modifying_startup.htm) Undef_Handler B Undef_Handler ;SWI_Handler B SWI_Handler PAbt_Handler B PAbt_Handler DAbt_Handler B DAbt_Handler ; RoS| 29.11.11: for RT-Agent IRQ_Handler B IRQ_Handler FIQ_Handler B FIQ_Handler ; Reset Handler EXPORT Reset_Handler Reset_Handler