why data compressor compress constants in Flash

Thanks for all suggestions,
Ok,

If I accept that compiler/linker behaviour is senseless because it does not know what to do with RW constant in flash, what can I do in order to meet my needs:

- I need the data to be constant - not changeable by standard code
- I need it to be placed in flash - in order to store configuration of the device
- I need to place it to a given address - so that it is in next erasable flash sector that main program
- I need it not to be optimized as I experienced once - I need that all the data of the big struct can be found at the given address and all its members.
- And even more, I need some starting values in my configuration stored.

During using it (over a year or two), I faced only two problems:
- once some data were stored to registers, so I added "volatile"
- some time the default data were compressed in my struct so I set "--compress off"
I did not needed scatter file or another approach to reach it.
But now, I am facing problem (described in other thread), so I started to ask "why" some things happen (compression of const data and so on) in order to more understand and probably to solve my issue (described in other thread).

I found, that this only fullfils "const volatile _AT_ = {default data}" . But it confuses a linker a bit... so I would like to do it really correct and clean.

Can scatter file help me with this? How?

thank you,
Ludek

You can slice the flash into an arbitrary number of individual address ranges with the scatter file.

So you can create a specific flash region that corresponds with a specific flash sector.

And you can configure in the scatter file that the RO data from a specific object file should be stored in this specific flash region.

Whe the scatter file has the same load and execution address range for this flash region, then the startup file will not contain any code to try to copy the data on boot - the loader knows the data is already at the correct location.

You want the data to be const - but it's being stored in flash that really makes the data write-protected.

I would like to see the code construct where the compiler did ignore reading the const data and instead hard-coded the values when generating the code. That should normally not happen, since the compiler doesn't even know the value of variables from a different object file, unless you really go all-out with global optimization.

If I accept that compiler/linker behaviour is senseless because it does not know what to do with RW constant in flash

That is SUCH a dumb thing to say!

Why the heck do you think a decent tool such as the compiler or linker should know specifics about the flash?

It's for the developer to determine what the flash should be used for, how it should be accessed and only then execute the requirements.

Per,

thank you for clear explanation. This could help I think.
Regarding the const...
The flash does not mean it is write protect from definition.
Simple write to the flash write to the flash, once it is unlocked and prepared for write.

And second, the write to the const variable is prevented by the compiler that stops you doing it.

Regarding the construct that fills the flash with something else I will write tomorrow my case.

have a nice evening.

Ludek

If you have a source file flash_config.h and flash_config.c:

flash_config.h:
=====
typedef struct { uint32_t version; uint32_t size; uint32_t my_info; ... uint8_t ... ...
} config_t;

extern const config_t config;
=====

flash_config.c:
=====
#include "flash_config.h"

const config_t config = { CONFIG_VERSION, sizeof(config_t), ...
};
=====

Then you have const data - so the linker will find RO data to store in flash.

The compiler sees the const keyword, and will complain about any attempt to write.

The compiler will only see the actual values when compiling flash_config.c, and will always need to perform memory accesses to pick up the actual values for code in other source files. So no need for any volatile - your flash reprogramming isn't really an asynchronous change. And no danger that the compiler will inline any value when generating code.

If you have code in the same source file, then you have to be aware that in classical C, const just means 'read-only'. But more modern C++ standards can use directly assigned const values as actual constant expressions.

const unsigned NUM_ELEMENTS = 10;
unsigned elements[NUM_ELEMENTS];

So NUM_ELEMENTS isn't a strict variable that happens to be read-only, that the produced code needs to read - it can be treated the same as:

enum {
    NUM_ELEMENTS = 10,
};
unsigned elements[NUM_ELEMENTS];

So with more recent C++ you need to think twice when you have:

const <datatype> <varname> = <value>;

and later in the same compilation unit have code that makes use of the <varname> variable. You shouldn't expect to be able to reprogram that flash memory address and store a different value into the variable - the produced code may have already treated the original value similar to a constexpr value.

Dear w tf, (thank you for suggestion, that I am who say what flash is... I like it.),
Dear Per,

Thank you for exhausting description about modern C++ and C in general. It helped me much. If I understand correctly what you tried to explain that my case could be solved by simply adding EXTERN to the HEADER, preventing variable optimalisation?
Volatile is reserved for prevention of accessing data partly when interrupt occurs, for example. I am not sure if I did not experienced the variable optimalisation even with extern (when not using volatila, just only const...) - but it has been already a year or two, hard to remember and hard to replicate, since the optimalisation is hard to force to repeat the same action. So I hope, extern will help and solve my needs, I want my code to be clear, clean, correct and me to learn and understand.

NEXT: I am sending the situation, that the data for initialisation of the constant are already stored in the constant destination. As you can see, the text COMPRESSED shows, that it is the load data, not the run time data...

--datacompressor off: Load Region LR$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE)

Execution Region ER$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE, UNINIT)

Base Addr Size Type Attr Idx E Section Name Object

0x080e0000 0x00000400 Data RW 311 .ARM.__AT_0x080E0000 dista_konfigurace.o

compressor on: Load Region LR$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE, COMPRESSED[0x00000190])

Execution Region ER$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE, UNINIT, COMPRESSED[0x00000190])

Base Addr Size Type Attr Idx E Section Name Object

0x080e0000 0x00000400 Data RW 311 .ARM.__AT_0x080E0000 dista_konfigurace.o

Construct is:
#define SECTOR_ADDR_for_Config 0x080E0000 //Flash starts at 0X08000000
const volatile DISTA_konfigurace_flash_typ F __attribute__((at(SECTOR_ADDR_for_Config)))=
{ { 0x00, 0x00, 0x00, 0x00,....

I am trying to understand, why this situation happens. I agree with w tf that I have some

knowledge gaps in code translation/linking basics. I have never needed, not even on ZX

Spectrum :-) .

Ludek

Note that your data is RW - which imply RAM. But the address represents flash. What should the linker do?

Make it into real RO data and then have the scatter file have one dedicated flash region for this data. Then there will be no copy and no compress.

And this most probably solves all your linking issues.

Absolute placement in the code works well to place a symbol on top of some hardware register/buffer. But the _compiler_ doesn't have access to your memory regions from the project file. So the compiler doesn't know the address is in flash.

The linker knows it's in flash but if the object file has already indicated RW you have forced the linker into a corner. The linker could fail. Or assume there is aliasing between readonly and writeonly memory - something that does happen sometimes for memory-mapped registers.

That's why the scatter file itself is so much more powerful than an absolute address specified in the source code.

I didn't bother to look at the second link as a link since I didn't get it tagged as a link.

The first link doesn't contain any text implying you need volatile - only that volatile is needed where an unknown external part may change the value. That's the asynchronous case where main loop and ISR accesses the same variable. Or two threads. Or the variable is mapped on top of special hardware, like a UART register.

For non-volatile variables, the compiler assumes they keep the same value until the compiler loses track of the variable - such as when you call a non-inlined function. An exception is if you use extreme global optimization, in which case the compiler may call other functions and because of the global optimizations knows these functions can't change the variable.

So you do not need volatile when you accesses const data stored in another file.

But you may get issues if you have code using a const variable value in the same file where you have code:

const int charlie = 10;

...

if (charlie != 10) {
    ...
}

A newer C++ compiler could in this special case know (based on the language standard) that charlie can't change value. So it can compile the code with the value 10 hardcoded. This because newer C++ can in this case treat const as meaning "can never change"

If the C++ compiler only knows there _exists_ a const int charlie, then it needs to perform normal variable accesses to charlie. So an update of the flash page will result in the code to behave based on the new value. The code gets generated based on const meaning write-protected for the current code. So no volatile keyword needed - your flash update code will not perform any asynchronous changes.

The compiler will only see the actual values when compiling flash_config.c, and will always need to perform memory accesses to pick up the actual values for code in other source files.

That assessment is no longer universally correct. Link-time optimization is a pretty standard feature of more advanced C and C++ tool chains these days, and that can pick up on cross-module 'const' usages like that. Flagging const objects that can be changed post build-time (e.g. by modifying the final hex file after the fact, or flashing a new copy of the const segment) as "volatile" is standard practice. It's exactly what a developer is supposed to do inform a compiler that a const object's value must always be read from its actual memory, even if the source code itself never modifies it. There's really no excuse for tools not supporting that idiom: it's a language standard requirement.

Hans,

nice to hear, that I am not completely mad. But my knowledge is not bright enought to set linker to work correctly. I am greenhorn regarding scatterfile now. I am even not sure, if I am able to write it in Keil. The variable is Const Volatile placed in Flash. How to set scatterfile entry, that is will be correct and once I use compression of initdata I will not find the compressed data in my variable that is const volatile and is in flash. Do you think it is possible there and now to achieve it just to be sure it will work even in future (or when changing the tools) when, as you write, linker optimalisation could occure. Now I am facing even worse issue that seems is in some relation to const volatile and memory compression functionality that results in hardfault before reaching main when scatterload initialises variables did not correctly sum up the complete image size.. (my observation and guessing).

thank you for your time and point of view,

Ludek

Per, also thank you for discussing and trying to find a solution for me.

Is there any one with solution to this case?

thank you all,
Ludek