union/struct problem

Dear Per Westermark,

Thanks for your reply,

I don't think the keil compiler does something incorrect. Keil is best compiler for arm mcus. I'm just trying to find the my fault.

"Because of union is packed"
" What makes you think/say this? "

I don't use "#pragma pack(push,1)" or __packed keywords for pack process but Are byte arrays already packed? The union includes byte array so I'm thinking that it is packed.
I have used many times differently in keil before and I didn't see not packed

for example, below,

typedef u8  unsigned char;
typedef u32 unsigned int;

u8 Buf[]={1,2,3,4,5,0};

typedef union {
    u8      Reg8[5];
    struct
    {
        u8      Select;    // 1 byte
        u32     Value32;   // 4 byte
    }Regs;
}CreditLoadRegs_t;




CreditLoadRegs_t *Ptr = (CreditLoadRegs_t *)Buf;


Ptr->Reg8[0] = Ptr->Select = 1

Ptr->Value32 = Ptr->Reg8[1] | (Ptr->Reg8[2] << 8) |
               (Ptr->Reg8[3] << 16) | (Ptr->Reg8[4] << 24)

             = 0x5432

Am I thinking wrong? if It is wrong, why?

Dear Per Westermark,

Thanks for your reply,

I don't think the keil compiler does something incorrect. Keil is best compiler for arm mcus. I'm just trying to find the my fault.

"Because of union is packed"
" What makes you think/say this? "

I don't use "#pragma pack(push,1)" or __packed keywords for pack process but Are byte arrays already packed? The union includes byte array so I'm thinking that it is packed.
I have used many times differently in keil before and I didn't see not packed

for example, below,

typedef u8  unsigned char;
typedef u32 unsigned int;

u8 Buf[]={1,2,3,4,5,0};

typedef union {
    u8      Reg8[5];
    struct
    {
        u8      Select;    // 1 byte
        u32     Value32;   // 4 byte
    }Regs;
}CreditLoadRegs_t;




CreditLoadRegs_t *Ptr = (CreditLoadRegs_t *)Buf;


Ptr->Reg8[0] = Ptr->Select = 1

Ptr->Value32 = Ptr->Reg8[1] | (Ptr->Reg8[2] << 8) |
               (Ptr->Reg8[3] << 16) | (Ptr->Reg8[4] << 24)

             = 0x5432

Am I thinking wrong? if It is wrong, why?

A byte array is packed.

But this most definitely is not a byte array:

struct {
    u8      Select;    // 1 byte
    u32     Value32;   // 4 byte
} Regs;

Value32 would want 32-bit align.
So you would end up with:

struct {
    u8      Select;    // 1 byte
    u8      pad1[3];
    u32     Value32;   // 4 byte
} Regs;

Now map that in your union, and you'll notice that you have 3 bytes of your byte array that maps to air.

Next thing - if your byte array is packed from a data structure that has a different byte order compared to what your processor uses, then even the struct was packed you would still get the wrong content seen in Value32.

struct {
    u8      Select;    // 1 byte
    u32     Value32;   // 4 byte
} Regs;

You're right. This structer is not packed. But I thought that union make it pack.

Result, I think as following

struct
    {
        u8      Select;    // 1 byte
        u8      unused[3];
        u32     Value32;   // 4 byte
    }Regs;

Reg8[0]=Select
Reg8[1]=unused[0]
Reg8[2]=unused[1]
Reg8[3]=unused[2]
Reg8[4]=LSB_Value32;

So the structer must be __packed

typedef union {
    u8      Reg8[5];
    __packed struct
    {
        u8      Select;    // 1 byte
        u32     Value32;   // 4 byte
    }Regs;
}CreditLoadRegs_t;

The union is only about overlaying data - not packing data.

And it isn't intended to overlay as a way to type-convert, but to allow conditional existence of data from a time when RAM was at a premium.

Any and all type conversion performed using unions are a kind of abuse. When done correctly, it can in some limited cases be used in a "correct" way without undefined behavior. But it is a slippery slope - the language standard didn't intend you to write to one member of the union and read out the data using a different member of the union.

you're quite right, thanks a lot

But I thought that union make it pack.

In case of doubt, and definitely in case of surprising behaviour like the one discussed here, you really should replace such thought by actually looking up documentation.

So the structer must be __packed

No. It doesn't have to be packed. The job at hand can and should be done entirely without any packed structs, unions and fishy pointer casts. You'll want to look up the term "serialization".

Dear Hans-Bernhard Broeker

No. It doesn't have to be packed. The job at hand can and should be done entirely without any packed structs, unions and fishy pointer casts. You'll want to look up the term "serialization".

Thanks your response,

Sorry, I didn't understand why it doesn't have to packed exactly. I have read a few books about standart C , embedded C and data structers in C. But I never read/see about serialization which is not packed or alligned memory in mcus.

I'm asking for learning,

if possible, can you give me a small sample or/and can you show/prefer me a source/book about this subject.

A common way of serializing data is to write code that takes a uint8_t pointer and inserts or retrieves different data types byte-by-byte in a well-defined order.

So if serializing a int32_t, the serialize function may emit it from high-to-low byte. And the deserialize function would extract and join from high-to-low. That means that the byte stream transmitted will be compatible even if the sender and receiver have different byte order.

And it also means that it doesn't matter if the sender and receiver have different rules for padding between fields in a struct.

So you might do:

p = transmit_buffer;
p = pack_u32(p,my_struct.my_u32);
p = pack_u32(p,my_struct.other_u32);
p = pack_u16(p,my_struct.some_u16);
p = pack_string(p,my_struct.string,sizeof(my_struct.string));
...

and on the other side do something like:

p = receive_buffer;
p = extract_u32(p,&my_struct.my_u32);
p = extract_u32(p,&my_struct.other_u32);
p = extract_u16(p,&my_struct.some_u16);
p = extract_string(p,my_struct.string,sizeof(my_struct.string));
...

Or maybe switch parameter/return value and do:

p = receive_buffer;
my_struct.my_u32 = extract_u32(&p);
my_struct.other_u32 = extract_u16(&p);
...

This is a situation where C++ is very handy, since it can keep track of the size of the transmit buffer and fail if trying to insert more than what fits. Or keep track of the number of available bytes in the receive buffer and fail if the code tries to extract more than was available.

With C, you need additional parameters or global variables to keep track of all the state information when packing/unpacking.

But with explicit serializing, you can get very robust code that works with different architectures, compilers, optimization levels, computer languages etc. The two sides can completely change order and data types of the internal data as long as they both agree on the order/size/type of the individual values stored in the serialized data.

Thank you a lot for sample and information,

I had already known serializing but I hadn't known its technical name as serializing.

I'm using it but usually for big string/data sending/receiving, if I'll send or receive little size data which is also combinations of difference variable type, I'm using packed structs as rx and tx buffer. (Because serializing process is slowing for parsing process)
But if there are big data size, I'm using serializing process.

__packed typedef struct{
  u8    Reg8;
  u32   Reg32;
  float Flt;
  u16   Reg16;
  .....
}Comm_t;

// sender side
Comm_t  SenderCommRegs;

UartCanUsbSenderFonc(Comm_t *cPtr);

UartCanUsbSenderFonc(&SenderCommRegs);

// receiver side
Comm_t  RecvCommRegs;

UartCanUsbRecvFonc(Comm_t *cPtr);

UartCanUsbRecvFonc(&RecvCommRegs);

in fact, I'm a little confused.

I can test it but if I use __packed keyword also, Can the keil compiler add byte/bytes for padding?

if it is not add, The union must cover struct regs. isn't it?

typedef union{
   u8 Areg8[5];       // 5 bytes u8 array
   __packed struct{
       u8  Temp8;     // 1 byte
       u16 Low16;     // 2 bytes
       u16 Hi16;      // 2 bytes
   }Regs;
}x_t;

One could presumably use sizeof() and offsetof() to arrive at a conclusion.

I would be careful as it is very easy to get caught with an alignment fault on ARM devices if you are not paying attention.

"Because serializing process is slowing for parsing process"

You'd be surprised at how fast serializing can be. When your web browser retrieves a web page, it gets serialized data. When it processes a JPG image, it's decoding serialized data. When you view a BluRay movie, the player decodes serialized data.

A "file format" is a protocol for how the data should be serialized into the file so writer and reader will be able to both understand the content. All networking happens with serialized data. It's just that in some situations, one layer of serialized data may contain "blobs" of unknown payload, that a different protocol layer will be able to work on.

In the end, you should avoid having transfers or data sharing contain data blobs where the content is raw memory dumps unless the data is known to just be a byte stream. It's a short-term cost/time-saving that tends to give lots of interesting problems.

Ok, I understand, Thanks,