Hi all, please help me understand data compressor.
I do not understand, why compressor compress the constants in flash. I believed that only compressed are the data in flash, that are used to initialised variables in ram with a given value. Why even the data in ram are compressed? But what I do not understand in all, what are my data in flash (const volatile - because can be changed by programing flash from the code in run time).
C++ code: #define SECTOR_ADDR_for_Config 0x080E0000 const volatile DISTA_konfigurace_flash_typ F __attribute__((at(SECTOR_ADDR_for_Config)))= { { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, ..... up to 1024 bytes
MAPFILE: Load Region LR$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE, COMPRESSED[0x00000190])
Execution Region ER$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE, UNINIT, COMPRESSED[0x00000190])
Base Addr Size Type Attr Idx E Section Name Object
0x080e0000 0x00000400 Data RW 311 .ARM.__AT_0x080E0000 dista_konfigurace.o
Can anyone help? Thank you, Ludek
If I accept that compiler/linker behaviour is senseless because it does not know what to do with RW constant in flash
That is SUCH a dumb thing to say!
Why the heck do you think a decent tool such as the compiler or linker should know specifics about the flash?
It's for the developer to determine what the flash should be used for, how it should be accessed and only then execute the requirements.
If you have a source file flash_config.h and flash_config.c:
flash_config.h: ===== typedef struct { uint32_t version; uint32_t size; uint32_t my_info; ... uint8_t ... ... } config_t;
extern const config_t config; =====
flash_config.c: ===== #include "flash_config.h"
const config_t config = { CONFIG_VERSION, sizeof(config_t), ... }; =====
Then you have const data - so the linker will find RO data to store in flash.
The compiler sees the const keyword, and will complain about any attempt to write.
The compiler will only see the actual values when compiling flash_config.c, and will always need to perform memory accesses to pick up the actual values for code in other source files. So no need for any volatile - your flash reprogramming isn't really an asynchronous change. And no danger that the compiler will inline any value when generating code.
If you have code in the same source file, then you have to be aware that in classical C, const just means 'read-only'. But more modern C++ standards can use directly assigned const values as actual constant expressions.
const unsigned NUM_ELEMENTS = 10; unsigned elements[NUM_ELEMENTS];
So NUM_ELEMENTS isn't a strict variable that happens to be read-only, that the produced code needs to read - it can be treated the same as:
enum { NUM_ELEMENTS = 10, }; unsigned elements[NUM_ELEMENTS];
So with more recent C++ you need to think twice when you have:
const <datatype> <varname> = <value>;
and later in the same compilation unit have code that makes use of the <varname> variable. You shouldn't expect to be able to reprogram that flash memory address and store a different value into the variable - the produced code may have already treated the original value similar to a constexpr value.
Dear w tf, (thank you for suggestion, that I am who say what flash is... I like it.), Dear Per,
Thank you for exhausting description about modern C++ and C in general. It helped me much. If I understand correctly what you tried to explain that my case could be solved by simply adding EXTERN to the HEADER, preventing variable optimalisation? Volatile is reserved for prevention of accessing data partly when interrupt occurs, for example. I am not sure if I did not experienced the variable optimalisation even with extern (when not using volatila, just only const...) - but it has been already a year or two, hard to remember and hard to replicate, since the optimalisation is hard to force to repeat the same action. So I hope, extern will help and solve my needs, I want my code to be clear, clean, correct and me to learn and understand.
NEXT: I am sending the situation, that the data for initialisation of the constant are already stored in the constant destination. As you can see, the text COMPRESSED shows, that it is the load data, not the run time data...
--datacompressor off: Load Region LR$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE)
Execution Region ER$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE, UNINIT)
compressor on: Load Region LR$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE, COMPRESSED[0x00000190])
Construct is: #define SECTOR_ADDR_for_Config 0x080E0000 //Flash starts at 0X08000000 const volatile DISTA_konfigurace_flash_typ F __attribute__((at(SECTOR_ADDR_for_Config)))= { { 0x00, 0x00, 0x00, 0x00,....
I am trying to understand, why this situation happens. I agree with w tf that I have some
knowledge gaps in code translation/linking basics. I have never needed, not even on ZX
Spectrum :-) .
Ludek
Note that your data is RW - which imply RAM. But the address represents flash. What should the linker do?
Make it into real RO data and then have the scatter file have one dedicated flash region for this data. Then there will be no copy and no compress.
And this most probably solves all your linking issues.
Absolute placement in the code works well to place a symbol on top of some hardware register/buffer. But the _compiler_ doesn't have access to your memory regions from the project file. So the compiler doesn't know the address is in flash.
The linker knows it's in flash but if the object file has already indicated RW you have forced the linker into a corner. The linker could fail. Or assume there is aliasing between readonly and writeonly memory - something that does happen sometimes for memory-mapped registers.
That's why the scatter file itself is so much more powerful than an absolute address specified in the source code.
Per,
when ever I read about volatile: publications.gbdirect.co.uk/.../const_and_volatile.html en.wikibooks.org/.../C_Programming it mention the prevention from optimalisation. Mentioning extern as a cure for that not. So I have to be sure the solution works always from definition for me. Do you have any reference for it?
thank you,Ludek
I didn't bother to look at the second link as a link since I didn't get it tagged as a link.
The first link doesn't contain any text implying you need volatile - only that volatile is needed where an unknown external part may change the value. That's the asynchronous case where main loop and ISR accesses the same variable. Or two threads. Or the variable is mapped on top of special hardware, like a UART register.
For non-volatile variables, the compiler assumes they keep the same value until the compiler loses track of the variable - such as when you call a non-inlined function. An exception is if you use extreme global optimization, in which case the compiler may call other functions and because of the global optimizations knows these functions can't change the variable.
So you do not need volatile when you accesses const data stored in another file.
But you may get issues if you have code using a const variable value in the same file where you have code:
const int charlie = 10; ... if (charlie != 10) { ... }
A newer C++ compiler could in this special case know (based on the language standard) that charlie can't change value. So it can compile the code with the value 10 hardcoded. This because newer C++ can in this case treat const as meaning "can never change"
If the C++ compiler only knows there _exists_ a const int charlie, then it needs to perform normal variable accesses to charlie. So an update of the flash page will result in the code to behave based on the new value. The code gets generated based on const meaning write-protected for the current code. So no volatile keyword needed - your flash update code will not perform any asynchronous changes.
The compiler will only see the actual values when compiling flash_config.c, and will always need to perform memory accesses to pick up the actual values for code in other source files.
That assessment is no longer universally correct. Link-time optimization is a pretty standard feature of more advanced C and C++ tool chains these days, and that can pick up on cross-module 'const' usages like that. Flagging const objects that can be changed post build-time (e.g. by modifying the final hex file after the fact, or flashing a new copy of the const segment) as "volatile" is standard practice. It's exactly what a developer is supposed to do inform a compiler that a const object's value must always be read from its actual memory, even if the source code itself never modifies it. There's really no excuse for tools not supporting that idiom: it's a language standard requirement.
Hans,
nice to hear, that I am not completely mad. But my knowledge is not bright enought to set linker to work correctly. I am greenhorn regarding scatterfile now. I am even not sure, if I am able to write it in Keil. The variable is Const Volatile placed in Flash. How to set scatterfile entry, that is will be correct and once I use compression of initdata I will not find the compressed data in my variable that is const volatile and is in flash. Do you think it is possible there and now to achieve it just to be sure it will work even in future (or when changing the tools) when, as you write, linker optimalisation could occure. Now I am facing even worse issue that seems is in some relation to const volatile and memory compression functionality that results in hardfault before reaching main when scatterload initialises variables did not correctly sum up the complete image size.. (my observation and guessing).
thank you for your time and point of view,
Per, also thank you for discussing and trying to find a solution for me.
Is there any one with solution to this case?
thank you all, Ludek