union/struct problem

if you're writing about my first message, you're right.

And when data came via gprs in BufPtr, I printed it, Data which is in buffer was as expected.
But output of typedef union was not. (as if union parser is not packed.)

for example,

u8 GprsBuf[1024]={1,2,3,4,5,...};

u8 *BufPtr = GprsBuf;

for(Cnt=0;Cnt<5;Cnt++)
    printf("%c",BufPtr[Cnt]+0x30); // output = "12345"

// if I use as following

CreditLoadRegs_t  *Ptr32 = (CreditLoadRegs_t  *)BufPtr;

for(Cnt=0;Cnt<5;Cnt++)
    printf("%c",Ptr32.Reg8[Cnt]+0x30); // output is wrong

Do you have any idea?

"Because of union is packed"

What makes you think/say this?

You seems to think the Keil compiler does something incorrect. But doing things that has undefined outcome means it's expected that you might get different results on different architectures or when using different compilers. So code that "works" on one system isn't automatically correct code even on the system where it "works".

If you have two architectures with different byte order, then that 32-bit value can print completely differently. You are claiming the data is sent over the network - so the data isn't from a different part of the same program.

And different architectures have different alignment requirements, which means a union or struct would have different rules for the addition of padding. Another thing here is that on a target with 32-bit align, a union with a 32-bit member value would need to have 32-bit align of the union itself - but that would be incompatible with typecasts from pointers to 8-bit data, unless you have explicitly aligned the 8-bit data.

Any time you move data between two devices, you shouldn't rely like that on the two devices having identical align and byte order. So programs doing networking have special functions for converting to/from "network byte order". Even if you have written the code for both the sender and the receiver, you should still add proper pack/unpack of the data, to allow you to change compiler and/or architecture on one side and stay compatible with the other side.

Dear Per Westermark,

Thanks for your reply,

I don't think the keil compiler does something incorrect. Keil is best compiler for arm mcus. I'm just trying to find the my fault.

"Because of union is packed"
" What makes you think/say this? "

I don't use "#pragma pack(push,1)" or __packed keywords for pack process but Are byte arrays already packed? The union includes byte array so I'm thinking that it is packed.
I have used many times differently in keil before and I didn't see not packed

for example, below,

typedef u8  unsigned char;
typedef u32 unsigned int;

u8 Buf[]={1,2,3,4,5,0};

typedef union {
    u8      Reg8[5];
    struct
    {
        u8      Select;    // 1 byte
        u32     Value32;   // 4 byte
    }Regs;
}CreditLoadRegs_t;




CreditLoadRegs_t *Ptr = (CreditLoadRegs_t *)Buf;


Ptr->Reg8[0] = Ptr->Select = 1

Ptr->Value32 = Ptr->Reg8[1] | (Ptr->Reg8[2] << 8) |
               (Ptr->Reg8[3] << 16) | (Ptr->Reg8[4] << 24)

             = 0x5432

Am I thinking wrong? if It is wrong, why?

A byte array is packed.

But this most definitely is not a byte array:

struct {
    u8      Select;    // 1 byte
    u32     Value32;   // 4 byte
} Regs;

Value32 would want 32-bit align.
So you would end up with:

struct {
    u8      Select;    // 1 byte
    u8      pad1[3];
    u32     Value32;   // 4 byte
} Regs;

Now map that in your union, and you'll notice that you have 3 bytes of your byte array that maps to air.

Next thing - if your byte array is packed from a data structure that has a different byte order compared to what your processor uses, then even the struct was packed you would still get the wrong content seen in Value32.

struct {
    u8      Select;    // 1 byte
    u32     Value32;   // 4 byte
} Regs;

You're right. This structer is not packed. But I thought that union make it pack.

Result, I think as following

struct
    {
        u8      Select;    // 1 byte
        u8      unused[3];
        u32     Value32;   // 4 byte
    }Regs;

Reg8[0]=Select
Reg8[1]=unused[0]
Reg8[2]=unused[1]
Reg8[3]=unused[2]
Reg8[4]=LSB_Value32;

So the structer must be __packed

typedef union {
    u8      Reg8[5];
    __packed struct
    {
        u8      Select;    // 1 byte
        u32     Value32;   // 4 byte
    }Regs;
}CreditLoadRegs_t;

The union is only about overlaying data - not packing data.

And it isn't intended to overlay as a way to type-convert, but to allow conditional existence of data from a time when RAM was at a premium.

Any and all type conversion performed using unions are a kind of abuse. When done correctly, it can in some limited cases be used in a "correct" way without undefined behavior. But it is a slippery slope - the language standard didn't intend you to write to one member of the union and read out the data using a different member of the union.

you're quite right, thanks a lot

But I thought that union make it pack.

In case of doubt, and definitely in case of surprising behaviour like the one discussed here, you really should replace such thought by actually looking up documentation.

So the structer must be __packed

No. It doesn't have to be packed. The job at hand can and should be done entirely without any packed structs, unions and fishy pointer casts. You'll want to look up the term "serialization".

Dear Hans-Bernhard Broeker

No. It doesn't have to be packed. The job at hand can and should be done entirely without any packed structs, unions and fishy pointer casts. You'll want to look up the term "serialization".

Thanks your response,

Sorry, I didn't understand why it doesn't have to packed exactly. I have read a few books about standart C , embedded C and data structers in C. But I never read/see about serialization which is not packed or alligned memory in mcus.

I'm asking for learning,

if possible, can you give me a small sample or/and can you show/prefer me a source/book about this subject.

A common way of serializing data is to write code that takes a uint8_t pointer and inserts or retrieves different data types byte-by-byte in a well-defined order.

So if serializing a int32_t, the serialize function may emit it from high-to-low byte. And the deserialize function would extract and join from high-to-low. That means that the byte stream transmitted will be compatible even if the sender and receiver have different byte order.

And it also means that it doesn't matter if the sender and receiver have different rules for padding between fields in a struct.

So you might do:

p = transmit_buffer;
p = pack_u32(p,my_struct.my_u32);
p = pack_u32(p,my_struct.other_u32);
p = pack_u16(p,my_struct.some_u16);
p = pack_string(p,my_struct.string,sizeof(my_struct.string));
...

and on the other side do something like:

p = receive_buffer;
p = extract_u32(p,&my_struct.my_u32);
p = extract_u32(p,&my_struct.other_u32);
p = extract_u16(p,&my_struct.some_u16);
p = extract_string(p,my_struct.string,sizeof(my_struct.string));
...

Or maybe switch parameter/return value and do:

p = receive_buffer;
my_struct.my_u32 = extract_u32(&p);
my_struct.other_u32 = extract_u16(&p);
...

This is a situation where C++ is very handy, since it can keep track of the size of the transmit buffer and fail if trying to insert more than what fits. Or keep track of the number of available bytes in the receive buffer and fail if the code tries to extract more than was available.

With C, you need additional parameters or global variables to keep track of all the state information when packing/unpacking.

But with explicit serializing, you can get very robust code that works with different architectures, compilers, optimization levels, computer languages etc. The two sides can completely change order and data types of the internal data as long as they both agree on the order/size/type of the individual values stored in the serialized data.

Thank you a lot for sample and information,

I had already known serializing but I hadn't known its technical name as serializing.

I'm using it but usually for big string/data sending/receiving, if I'll send or receive little size data which is also combinations of difference variable type, I'm using packed structs as rx and tx buffer. (Because serializing process is slowing for parsing process)
But if there are big data size, I'm using serializing process.

__packed typedef struct{
  u8    Reg8;
  u32   Reg32;
  float Flt;
  u16   Reg16;
  .....
}Comm_t;

// sender side
Comm_t  SenderCommRegs;

UartCanUsbSenderFonc(Comm_t *cPtr);

UartCanUsbSenderFonc(&SenderCommRegs);

// receiver side
Comm_t  RecvCommRegs;

UartCanUsbRecvFonc(Comm_t *cPtr);

UartCanUsbRecvFonc(&RecvCommRegs);

in fact, I'm a little confused.

I can test it but if I use __packed keyword also, Can the keil compiler add byte/bytes for padding?

if it is not add, The union must cover struct regs. isn't it?

typedef union{
   u8 Areg8[5];       // 5 bytes u8 array
   __packed struct{
       u8  Temp8;     // 1 byte
       u16 Low16;     // 2 bytes
       u16 Hi16;      // 2 bytes
   }Regs;
}x_t;

One could presumably use sizeof() and offsetof() to arrive at a conclusion.

I would be careful as it is very easy to get caught with an alignment fault on ARM devices if you are not paying attention.

"Because serializing process is slowing for parsing process"

You'd be surprised at how fast serializing can be. When your web browser retrieves a web page, it gets serialized data. When it processes a JPG image, it's decoding serialized data. When you view a BluRay movie, the player decodes serialized data.

A "file format" is a protocol for how the data should be serialized into the file so writer and reader will be able to both understand the content. All networking happens with serialized data. It's just that in some situations, one layer of serialized data may contain "blobs" of unknown payload, that a different protocol layer will be able to work on.

In the end, you should avoid having transfers or data sharing contain data blobs where the content is raw memory dumps unless the data is known to just be a byte stream. It's a short-term cost/time-saving that tends to give lots of interesting problems.

Ok, I understand, Thanks,