为什么 C 编译器不能重新排列结构成员以消除对齐填充?

可能的复制品:
为什么海湾合作委员会不优化结构?
为什么 C + + 不使结构更紧凑

考虑一下32位 x86计算机上的以下示例:

由于对齐约束,下面的结构

struct s1 {
char a;
int b;
char c;
char d;
char e;
}

可以更有效地表示内存(12比8字节) ,如果成员按照

struct s2 {
int b;
char a;
char c;
char d;
char e;
}

我知道 C/C + + 编译器不允许这样做。我的问题是为什么语言是这样设计的。毕竟,我们可能最终会浪费大量的内存,而像 struct_ref->b这样的引用不会关心这种差异。

编辑 : 感谢你们所有人给出的非常有用的答案。您很好地解释了为什么由于语言的设计方式,重新排列不起作用。然而,这让我想到: 如果重新安排是语言的一部分,这些争论还会继续吗?假设有一些特定的重排规则,我们至少需要这个规则

  1. 我们应该只在实际需要时才重新组织结构(如果结构已经“紧凑”,则不要执行任何操作)
  2. 规则只查看结构的定义,而不查看内部结构。这样可以确保结构类型具有相同的布局,无论它是否位于另一个结构的内部
  3. 根据给定结构的定义(即规则是固定的) ,该结构的已编译内存布局是可预测的

我逐一解释你的论点:

  • 低级数据映射,“ element of less 驚” : 只要自己用紧凑的风格编写结构(就像@Perry 的答案) ,什么都没有改变(需求1)。如果出于某种奇怪的原因,您希望内部填充出现在那里,您可以使用虚拟变量手动插入它,并且/或者可以有关键字/指令。

  • 编译器差异 : 需求3消除了这个问题。事实上,从@David Heffernan 的评论来看,我们今天似乎有这个问题,因为不同的编译器垫不同?

  • 优化 : 重新排序的全部意义在于(内存)优化。我看到了很多潜力。我们可能无法一起去除所有的填充,但我不认为重新排序可以以任何方式限制优化。

  • 类型转换 : 在我看来,这是最大的问题。不过,应该有办法解决这个问题。由于这些规则在语言中是固定的,编译器能够找出成员是如何重新排序的,并作出相应的反应。如上所述,在需要完全控制的情况下,总是可以防止重新排序。另外,需求2确保类型安全的代码永远不会中断。

我之所以认为这样的规则是有意义的,是因为我发现按照结构成员的内容而不是按照它们的类型来分组更自然。而且,当我有很多内部结构时,编译器比我更容易选择最佳排序方式。最佳布局甚至可能是我无法以类型安全的方式表示的布局。另一方面,它似乎使语言更加复杂,这当然是一个缺点。

请注意,我并不是说要改变语言——只有当它可以(/应该)被设计成不同的语言时才会这样。

我知道我的问题是假设性的,但我认为这个讨论提供了对机器和语言设计较低层次的更深入的见解。

我是新来的,所以我不知道是否应该提出一个新的问题。如果是这样的话,请告诉我。

21634 次浏览

It would change the semantics of pointer operations to reorder the structure members. If you care about compact memory representation, it's your responsibility as a programmer to know your target architecture, and organize your structures accordingly.

If you were reading/writing binary data to/from C structures, reordering of the struct members would be a disaster. There would be no practical way to actually populate the structure from a buffer, for example.

C [and C++] are regarded as systems programming languages so they provide low level access to the hardware, e.g., memory by means of pointers. Programmer can access a data chunk and cast it to a structure and access various members [easily].

Another example is a struct like the one below, which stores variable sized data.

struct {
uint32_t data_size;
uint8_t  data[1]; // this has to be the last member
} _vv_a;

Keep in mind that a variable declaration, such as a struct, is designed to be a "public" representation of the variable. It's used not only by your compiler, but is also available to other compilers as representing that data type. It will probably end up in a .h file. Therefore, if a compiler is going to take liberties with the way the members within a struct are organized, then ALL compilers have to be able to follow the same rules. Otherwise, as has been mentioned, the pointer arithmetic will get confused between different compilers.

Structs are used to represent physical hardware at the very lowest levels. As such the compiler cannot move things a round to suit at that level.

However it would not be unreasonable to have a #pragma that let the compiler re-arrange purely memory based structs that are only used internally to the program. However I don't know of such a beast (but that doesn't meant squat - I'm out of touch with C/C++)

Here's a reason I didn't see so far - without standard rearrangement rules, it would break compatibility between source files.

Suppose a struct is defined in a header file, and used in two files.
Both files are compiled separately, and later linked. Compilation may be at different times (maybe you touched just one, so it had to be recompiled), possibly on different computers (if the files are on a network drive) or even different compiler versions.
If at one time, the compiler would decide to reorder, and at another it won't, the two files won't agree on where the fields are.

As an example, think of the stat system call and struct stat.
When you install Linux (for example), you get libC, which includes stat, which was compiled by someone sometime.
You then compile an application with your compiler, with your optimization flags, and expect both to agree on the struct's layout.

Your case is very specific as it would require the first element of a struct to be put re-ordered. This is not possible, since the element that is defined first in a struct must always be at offset 0. A lot of (bogus) code would break if this would be allowed.

More generally pointers of subobjects that live inside the same bigger object must always allow for pointer comparison. I can imagine that some code that uses this feature would break if you'd invert the order. And for that comparison the knowledge of the compiler at the point of definition wouldn't help: a pointer to a subobject doesn't have a "mark" do which larger object it belongs. When passed to another function just as such, all information of a possible context is lost.

There are multiple reasons why the C compiler cannot automatically reorder the fields:

  • The C compiler doesn't know whether the struct represents the memory structure of objects beyond the current compilation unit (for example: a foreign library, a file on disc, network data, CPU page tables, ...). In such a case the binary structure of data is also defined in a place inaccessible to the compiler, so reordering the struct fields would create a data type that is inconsistent with the other definitions. For example, the header of a file in a ZIP file contains multiple misaligned 32-bit fields. Reordering the fields would make it impossible for C code to directly read or write the header (assuming the ZIP implementation would like to access the data directly):

    struct __attribute__((__packed__)) LocalFileHeader {
    uint32_t signature;
    uint16_t minVersion, flag, method, modTime, modDate;
    uint32_t crc32, compressedSize, uncompressedSize;
    uint16_t nameLength, extraLength;
    };
    

    The packed attribute prevents the compiler from aligning the fields according to their natural alignment, and it has no relation to the problem of field ordering. It would be possible to reorder the fields of LocalFileHeader so that the structure has both minimal size and has all fields aligned to their natural alignment. However, the compiler cannot choose to reorder the fields because it does not know that the struct is actually defined by the ZIP file specification.

  • C is an unsafe language. The C compiler doesn't know whether the data will be accessed via a different type than the one seen by the compiler, for example:

    struct S {
    char a;
    int b;
    char c;
    };
    
    
    struct S_head {
    char a;
    };
    
    
    struct S_ext {
    char a;
    int b;
    char c;
    int d;
    char e;
    };
    
    
    struct S s;
    struct S_head *head = (struct S_head*)&s;
    fn1(head);
    
    
    struct S_ext ext;
    struct S *sp = (struct S*)&ext;
    fn2(sp);
    

    This is a widely used low-level programming pattern, especially if the header contains the type ID of data located just beyond the header.

  • If a struct type is embedded in another struct type, it is impossible to inline the inner struct:

    struct S {
    char a;
    int b;
    char c, d, e;
    };
    
    
    struct T {
    char a;
    struct S s; // Cannot inline S into T, 's' has to be compact in memory
    char b;
    };
    

    This also means that moving some fields from S to a separate struct disables some optimizations:

    // Cannot fully optimize S
    struct BC { int b; char c; };
    struct S {
    char a;
    struct BC bc;
    char d, e;
    };
    
  • Because most C compilers are optimizing compilers, reordering struct fields would require new optimizations to be implemented. It is questionable whether those optimizations would be able to do better than what programmers are able to write. Designing data structures by hand is much less time consuming than other compiler tasks such as register allocation, function inlining, constant folding, transformation of a switch statement into binary search, etc. Thus the benefits to be gained by allowing the compiler to optimize data structures appear to be less tangible than traditional compiler optimizations.

Not being a member of WG14, I can't say anything definitive, but I have my own ideas:

  1. It would violate the principle of least surprise - there may be a damned good reason why I want to lay my elements out in a specific order, regardless of whether or not it's the most space-efficient, and I would not want the compiler to rearrange those elements;

  2. It has the potential to break a non-trivial amount of existing code - there's a lot of legacy code out there that relies on things like the address of the struct being the same as the address of the first member (saw a lot of classic MacOS code that made that assumption);

The C99 Rationale directly addresses the second point ("Existing code is important, existing implementations are not") and indirectly addresses the first ("Trust the programmer").

C is designed and intended to make it possible to write non-portable hardware and format dependent code in a high level language. Rearrangement of structure contents behind the back of the programmer would destroy that ability.

Observe this actual code from NetBSD's ip.h:


/*
* Structure of an internet header, naked of options.
*/
struct ip {
#if BYTE_ORDER == LITTLE_ENDIAN
unsigned int ip_hl:4,       /* header length */
ip_v:4;        /* version */
#endif
#if BYTE_ORDER == BIG_ENDIAN
unsigned int ip_v:4,        /* version */
ip_hl:4;       /* header length */
#endif
u_int8_t  ip_tos;       /* type of service */
u_int16_t ip_len;       /* total length */
u_int16_t ip_id;        /* identification */
u_int16_t ip_off;       /* fragment offset field */
u_int8_t  ip_ttl;       /* time to live */
u_int8_t  ip_p;         /* protocol */
u_int16_t ip_sum;       /* checksum */
struct    in_addr ip_src, ip_dst; /* source and dest address */
} __packed;

That structure is identical in layout to the header of an IP datagram. It is used to directly interpret blobs of memory blatted in by an ethernet controller as IP datagram headers. Imagine if the compiler arbitrarily re-arranged the contents out from under the author -- it would be a disaster.

And yes, it isn't precisely portable (and there's even a non-portable gcc directive given there via the __packed macro) but that's not the point. C is specifically designed to make it possible to write non-portable high level code for driving hardware. That's its function in life.

suppose you have a header a.h with

struct s1 {
char a;
int b;
char c;
char d;
char e;
}

and this is part of a separate library (of which you only have the compiled binaries compiled by a unknown compiler) and you wish to use this struct to communicate with this library,

if the compiler is allowed to reorder the members in whichever way it pleases this will be impossible as the client compiler doesn't know whether to use the struct as-is or optimized (and then does b go in front or in the back) or even fully padded with every member aligned on 4 byte intervals

to solve this you can define a deterministic algorithm for compacting but that requires all compilers to implement it and that the algorithm is a good one (in terms of efficiency). it is easier to just agree on padding rules than it is on reordering

it is easy to add a #pragma that prohibits the optimization for when you need the layout of to a specific struct be exactly what you need so that is no issue