如何修复编译 > 2GB 代码时的 GCC 编译错误？

The x86-64 ABI used by Linux defines a "large model" specifically to avoid such size limitations, which includes 64-bit relocation types for the GOT and PLT. (See the table in section 4.4.2, and the instruction sequences in 3.5.5 which show how they are used.)

Since your functions are occupying 2.8 GB, you are out of luck, because gcc doesn't support large models. What you can do, is to reorganize your code in such a way that would allow you to split it into shared libraries which you would dynamically link.

If that is not possible, as someone suggested, instead of putting your data into code (compiling and linking it), since it is huge, you can load it at run time (either as a normal file, or you can mmap it).

EDIT

Seems like the large model is supported by gcc 4.6 (see this page). You can try that, but the above still applies about reorganizing your code.

If I read your errors correctly, what makes you carry over the limit is the initialized data section (if it was the code, you would have far more errors IMHO). Do you have big arrays of global data? If it is the case, I'd restructure the program so that they are allocated dynamically. If the data is initialized, I'd read it from a configuration file.

BTW seeing this:

(.text+0x20): undefined reference to `main'

I think you have another problem.

The error occurs because you have too much CODE, not data! This is indicated by for example __libc_csu_fini (which is a function) being referenced from _start and the relocation is truncated to fit. This means that _start (the program's true entry point) is trying to call that function via a SIGNED 32-bit offset, which has only a range of 2 GB. Since the total amount of your object code is ~2.8 GB, the facts check out.

If you could redesign your data structures, much of your code could be "compressed" by rewriting the huge expressions as simple loops.

Also, you could compute csc[] in a different program, store the results in a file, and just load them when necessary.

With a program of that side, cache misses for code are very likely to exceed the costs of looping at runtime. I would recommend that you go back to your code generator, and have it generate some compact representation for what it wants evaluated (ie, one likely to fit in D-cache), then execute that with an interpreter in your program. You could also see if you can factor out smaller kernels that still have a significant number of operations, then use those as 'instructions' in the interpreted code.

So, you already have a program that produces this text:

prefactor = +s.ds8*s.ds10*ti[0]->value();
expr = ( - 5/243.*(s.x14*s.x15*csc[49300] + 9/10.*s.x14*s.x15*csc[49301] +
1/10.*s.x14*s.x15*csc[49302] - 3/5.*s.x14*s.x15*csc[49303] -...

and

double csc19295 =       + s.ds0*s.ds1*s.ds2 * ( -
32*s.x12pow2*s.x15*s.x34*s.mbpow2*s.mWpowinv2 -
32*s.x12pow2*s.x15*s.x35*s.mbpow2*s.mWpowinv2 -
32*s.x12pow2*s.x15*s.x35*s.x45*s.mWpowinv2 -...

right?

If all your functions have a similar "format" (multiply n numbers m times and add the results - or something similar) then I think you can do this:

• change the generator program to output offsets instead of strings (i.e. instead of the string "s.ds0" it will produce offsetof(ProcessVars, ds0)
• create an array of such offsets
• write an evaluator which accepts the array above and the base addresses of the structure pointers and produces an result

The array+evaluator will represent the same logic as one of your functions, but only the evaluator will be code. The array is "data" and can be either generated at runtime or saved on disk and read i chunks or with a memory mapped file.

For your particular example in func1 imagine how you would rewrite the function via an evaluator if you had access to the base address of s and csc and also a vector like representation of the constants and the offsets you need to add to the base addresses to get to x14, ds8 and csc[51370]

You need to create a new form of "data" that will describe how to process the actual data you pass to your huge number of functions.

A couple of suggestions: - Optimize for size (-Os). Make your inline function calls, normal function calls. Enable string pooling.

Try splitting the things into different DLL's (shared objects, .so for linux, .dylib for Mac OS X). Make sure that they can be unloaded. Then implement something to load things on demand, and free them when not needed.

If not, split your code into different executables, and use something to communicate between them (pipes, sockets, even writing / reading to file). Clumsy, but what options do you have?

Totally alternative: - Use a dynamic language with JIT. Right on top of my head - use LuaJIT - and rewrite (regenerate?) a lot of these expressions in Lua, or other such languages and runtimes that allow code to be garbage collected.

LuaJIT is quite efficient, sometimes beating C/C++ for certain things, but often very close (sometimes can be slow due to poor garbage collection yet there). Check for yourself:

http://luajit.org/performance_x86.html

Download the scimark2.lua file from there, and compare it with the "C" version (google it) - often results are very close.

Those expressions look a lot like an alternating series to me. I don't know what the rest of the code looks like, but it doesn't seem like it'd be that hard to derive the generating expression. It'd probably be worth it at execution time too, especially if you have 2.8 GB of 2 KB unrolled code.

I think everybody agrees there should be a different way to do what you want to do. Compiling hundreds of megabyte (gigabytes?) of code, linking it into a multi-gigabyte sized executable and running it just sounds very inefficient.

If I understand your problem correctly, you use some sort of code generator, G, to generate a bunch of functions func1...N which take a bunch of maps csc1...M as input. What you want to do is to calculated csc1...M, and run a loop of 1,000,000 times for different inputs and each time find s = func1 + func2 + ... + funcN. You didn't specify how fucn1...N are related to csc1...M though.

If all that is true, it seems that you should be able to turn the problem on its head in different way which can potentially be much more manageable and even possibly faster (i.e. letting your machine's cache to actually function).

Besides the practical problem of the object files sizes, your current program will not be efficient since it does not localize access to the data (too many huge maps) and has no localized code execution (too many very long functions).

How about breaking your program into 3 phase: Phase 1 build csc1...M and storing them. Phase 2 build one func at a time, run it 1,000,000 times with each input and store the results. Phase 3 find the sum of the results of the stored func1...N outcomes for each run out of 1,000,000 times. The good part about this solution is that it can be easily made parallel across several independent machines.

Edit: @bbtrb, could you make one func and one csc available somehwere? They seem to be highly regular and compressible. For instance, func1 seems to be just a sum of expressions each consisting of 1 coefficient, 2 indexes to the variables in s and 1 index into csc. So it can be reduced to a nice loop. If you make complete examples available, I'm sure ways can be found to compress them into loops rather than long expressions.

This looks like the result of code generation gone wrong, perhaps by symbolic algebra and/or manual unrolling. Symbolic manipulations are well known to grow exponentially in the depth of the expression tree or computational graph. It is likely that automatic differentiation can be used here, which would make the code size quite small and also speed up execution dramatically.

The linker is attempting to generate 32-bit relocation offsets within a binary that has somehow exceeded these limitations. Try reduce the main program's address space requirements.

Can you split some/most of the object code into one or more libraries (also compiled with -fpic / -fPIC)? Then generate a non-static binary that links against these libs. The libraries will live in discrete memory blocks and your relocation offsets will be dynamic/absolute (64-bit) rather than relative (32-bit).

It looks to me like the code is doing numerical integration using some kind of adaptive depth method. Unfortunately, the code generator (or rather the author of the code generator) is so stupid as to generate one function per patch rather than one per type of patch. As such, it's produced too much code to be compiled, and even if it could be compiled its execution would be painful because nothing's ever shared anywhere ever. (Can you imagine the pain resulting by having to load each page of object code from disk because nothing is ever shared and so it's always a candidate for the OS to evict. To say nothing of instruction caches, which are going to be useless.)

The fix is to stop unrolling everything; for this sort of code, you want to maximize sharing as the overhead of extra instructions to access data in more complex patterns will be absorbed by the cost of dealing with the (presumably) large underlying dataset anyway. It's also possible that the code generator will even do this by default, and that the scientist saw some options for unrolling (with the note that these sometimes improve speed) and turned them all on at once and is now insisting that this resulting mess be accepted by the computer, rather than accepting the machine's real restrictions and using the numerically correct version that is generated by default. But if the code generator won't do it, get one that will (or hack the existing code).

The bottom line: compiling and linking 2.8GB of code doesn't work and shouldn't be forced to work. Find another way.