Hi, I have a system profile structure (set of system properties), saved in a serial dataflash. Size of a profile is about 16K. At power-up I need to extract from this profile another structure - slave profile (6K), and send it to the slave microcontroller over CAN. In order to do that at a certain point I need to initialize both structures in ram. And this is the only time when these structures are used.
The question is - what is the optimum way to allocate memory for these structures ? I am using STM32F107 with 64K of ram. If I initialize structures locally, in a 'parsing / slave initializing' function, I need to assign additional 22K to the stack, and I loose them for the application. By initializing these structures globally I also loose 22K of ram.
Would really appreciate considerations on this subject.
Gennady
I need to assign additional 22K to the stack, and I loose them for the application.
Not exactly. If other parts of your application use the stack extensively, then you don't loose it.
By initializing these structures globally I also loose 22K of ram.
Yes. If you don't reuse this memory in any way.
The question is: are you short on RAM? Do you need to worry at all? 64KB is plenty for many applications. If you really are short on RAM, you can try a few things: 1) Optimize memory-hungry algorithms. Are you sure you have to hold those structures in RAM? 2) Reuse memory. That means you have to figure out if other parts of your application can use the same memory. If it's possible, the next step is to choose how to share this memory: using the stack, or perhaps dynamic memory.
Like Mike says, is this really an issue?
"I have a system profile structure ... about 16K. At power-up I need to extract from this profile another structure - slave profile (6K)"
Why do you need to extract it? Can't you just use it in-place - within the 16K of the "system" profile?
Could you use a union to overlay this data with something mutually exclusive?
Be sure to update your cross-post on the ST forum - so that you don't waste their time giving suggestions that have already been made here!
my.st.com/.../Flat.aspx
http://bit.ly/h3lctA
It would be worth spending some time studying the Linker manual to see if it has any useful features which could help here.
I just tried a very quick search for "overlay", and found this:
http://www.keil.com/support/man/docs/armlink/armlink_cchihacc.htm
"You can use the OVERLAY attribute in a scatter-loading description file to place multiple execution regions at the same address"
As you say that the "profile" stuff is only ever used at startup, it sounds like your "overlay manager" could be quite simple - possibly it wouldn't actually need to do anything at all...?
I would make sure the data is stored in a way that it can be directly streamed from master to slave without any need for the data to be stored in RAM.
There really should be no reason for the master to fill a struct or whatever we are talking about with data that it will not make use of other than to forward to another processor.
I need to assign additional 22K to the stack, and I loose them for the application. Not exactly. If other parts of your application use the stack extensively, then you don't loose it.
I can use part of this stack in other parts, but definitely not that much. And significant part of this application is a communication with multiple slaves / ethernet, so i need quite a few global buffers placed in ram. And again, the only purpose for placing this structures in ram is parsing structure. I really would like to avoid dynamic memory allocation, so I can try some other way to share memory.
Unfortunately I have no power over the profile loaded to dataflash - it's built by the configuration application on the pc and has overlapping parts related to different slaves / properties. Also it's in json format, and has multiple fields which I don't use/need
Thank you, I will take a look at that, maybe that is what I need.
"And significant part of this application is a communication with multiple slaves / ethernet, so i need quite a few global buffers placed in ram. And again, the only purpose for placing this structures in ram is parsing structure."
Then your problem is a no-problem.
Create a single 22k+ block that you use for the temporary storage of this structure.
When you have sent it to the slave(s), you can then reuse this block for quite a number of your receive and transmit buffers.
One way you can do it is to use pointers.
Another way is to create a union:
union { struct { uint8_t my_eth0_rx_buf[ETH_RX_SIZE]; uint8_t my_eth0_tx_buf[ETH_TX_SIZE]; uint8_t my_xx_rx_buf[XX_RX_SIZE]; uint8_t my_xx_tx_buf[XX_TX_SIZE]; ... } bufs; struct { .... } temporary_ridiculously_large_slave_struct; } my_buffers;
But the big issue - why should the master parse the slave configuration? Wouldn't it be better if the master sends the raw array of bytes from EEPROM to the slaves and have them parse it?
A file server does not parse the Word documents you store on it. It just sends them out and it's up to the receiving machine to see if the file really is a valid Word document. So the file server doesn't need to be updated if you suddenly decides to store a document for a newer version of Word or a document for Microsoft Excel.
If there are a few parameters in this 22kB large block that the server needs to know about, then you should let the slaves decode the information and then report back the relevant parameters to the "file server". So the master sends an unknown blob and then gets back "our timeout settings are 1500ms" and "our addresses are xxx".
The only reason for me to initialize the profile structure in ram is that I need to parse it to the slave profile. The best way would be to have a slave structure as a substructure of a main profile structure, but I can't change the formatting, and as I mentioned in another post, it's in a .json format with multiple unrelated fields, so needed data should be extracted.
Sorry but if your master have the knowledge to cut and paste from the data stored in the dataflash, then the master should be able to do that cutting/pasting on-the-fly without the need to build a complete structure in RAM. It's even possible to compute a CRC on the data sent to the slaves without ever having had a full structure available anywhere in the dataflash or in the RAM of the master.
Anyway - how come you have this problem right now, but do already have a PC program that creates data in a format that is unsuitable to your needs? The PC have "infinite" amounts of memory and code space doesn't matter either. The normal way to implement a PC configurator for an embedded product is to have the configurator create optimally mashed data that requires the minimum amount of RAM and code in the embedded system that is expected to receive the data.
So you normally let the embedded side be responsible for the specifications and the PC developer(s) then have to fulfill the requirements. Especially since it isn't uncommon that the PC side is developed by junior developers who are not likely to view the whole picture before designing their side.
Totally Agree!
Always try to push as much complexity as far "up" the tree as possible.
I just tried to simplify my life and avoid multiple dataflash readings and calculating addresses for each piece of data.
So you normally let the embedded side be responsible for the specifications and the PC developer(s) then have to fulfill the requirements. ======= Totally Agree! Always try to push as much complexity as far "up" the tree as possible.
Can't agree more. Unfortunately, this is the way it's done for this project.
I've received a lot of great advice here and really appreciate it. Now I wil try to implement it.
Be sure that you feed back to the appropriate authorities and that they are aware of the problems that this causes!
Suggest that they strongly consider that other projects not be done this way!
Send them here if they don't beleive you!
Sometimes, the RAM/code space costs of having extra complexities in the firmware doesn't really matter.
The huge reason for having the complexities in the PC configurator is that the configurator can often be emailed out in case of a found bug, or maybe available on the company web page.
A bug in the embedded firmwares can on the other hand cost huge amounts of time and money to update. And then the customer goodwill losses if they do spend time/money updating the firmware just to later find out that you had to release yet another firmware update because of the extra complexities in the firmware...