Wednesday, August 30, 2006

Faster Build Times - Reducing Header Dependencies

Long compile times are a pain in the neck. They waste time, cause frustration, interrupt your ‘flow’ etc. They do cause a lot of programmers to learn how to juggle, which is a slight positive, but lets proceed with the assumption that long compile times are bad.

As I’ve already mentioned in the past, a lot of compilation time goes into parsing headers. Using pre-compiled headers is a good way to reduce them, but there are a couple of other ways.

You can wrap your header with the following kind of pre-processor directives:

#ifndef _MYFILENAME_H_

#define _MYFILENAME_H_

… normal header contents go here …

#endif // _MYFILENAME_H_

So what does this do? Well the defined value “_MYFILENAME_H_” is arbitrary, it’s just some unique value, the filename with some underscores is commonly used. The first time this is encountered, the parser will process the contents of the header, but also define the property. If the header gets included again (for the current cpp file it’s compiling) the value will already be defined and the contents of the header will be skipped. This is a time saving, and also avoids cyclic loops of headers.

But wait! For that trick to work, the parser still has to read the header to get to the point where the #endif is, so it still has to read the whole file.

Microsoft have an answer for this. You can put:

#pragma once

at the top of your file, and the parser will only include the file once. The parser knows that if it encounters this pragma for a second time for one cpp file, it can just stop there. This is good, but it only works on the Microsoft compiler, and so might not meet your needs. Also even though we’re preventing repeated parsing of the same file, we could still be including a lot of files.

The best thing is for us to reduce the amount of headers getting included. Most people know this, but it’s still a very common problem. A header can need to include another header for several reasons, including:

  • So a class/struct from another file can be embedded in a class.
  • So that functions defined in the header can ‘use’ another data types' functions and data.
  • So pointers to other classes/structs can be used in structures and function prototypes.

The last bullet point here is interesting. The header that’s doing the including is not needing to know the size of the other structure, or call functions related to it. In this case, we can just make a forward declaration instead of including the other header. So to embed a pointer to the class 'Foo', instead of:

#include “foo.h”

we can just say:

class Foo;

This tells the compiler “there’s a class called Foo, and all you need to know right now is that it exists and I’m going to be dealing with pointers to it”. You’ll then likely need to put the #include into one or more cpp files where the pointer gets used. This can seem like a pointless exercise at first, but it reduces the overall amount of headers which need including, by breaking the chains of headers which rely on other headers.

You can also think about the other cases – do I need to implement this function in the header, or can it be moved to the cpp, allowing me to remove the include.

If you want faster build times, turn on build timing so you can see how long your compiles take. In .NET, go to “Tools->Options->Projects and Solutions->VC++ Project Settings->Build Timing” and turn it on. Then turn on “Project->Properties->C/C++->Advanced->Show Includes” to show a list of all the included files when you compile. Then look at the build output and find header include chains you think you can break up.

It can be tricky to reduce header dependencies, it requires many large rebuilds of your project. But assuming you can find some optimisations, you get paid back in shorter compile times and the benefits they bring.


Matt Godbolt said...

I remember all too well the pain of messy header dependencies, seemingly worse in the Games Industry. We used all manner of tricks at Argonaut Games including a very unsettling '#include all the files into one file and compile that' for some libraries - horrible, but quite effective in some cases.

Further reading on the kinds of tricks you can do I'd recommend looking at John Lakos's Large Scale C++ design.

It's always worth checking your header files for redundant includes and obvious bottlenecks - after my stint coding for Xbox and PS2 I set up a company making C++ tools, including a #include visualiser called IncludeManager which we find invaluable for this kind of thing. I wish just we'd have had it at Argonaut!

Mark Pope said...

Thanks Matt, your program looks great.