This document explains how to extend Boost.Build to accomodate your local requirements. Let's start with a simple but realistic example.

Say you're writing an application that generates C++ code. If you ever did this, you know that it's not nice. Embedding large portions of C++ code in string literals is very awkward. A much better solution is:

It's quite easy to achieve. You write special verbatim files that are just C++, except that the very first line of the file contains the name of a variable that should be generated. A simple tool is created that takes a verbatim file and creates a cpp file with a single char* variable whose name is taken from the first line of the verbatim file and whose value is the file's properly quoted content.

Let's see what Boost.Build can do.

First off, Boost.Build has no idea about "verbatim files". So, you must register a new target type. The following code does it:

import type ;
type.register VERBATIM : verbatim ;

The first parameter to type.register gives the name of the declared type. By convention, it's uppercase. The second parameter is the suffix for files of this type. So, if Boost.Build sees code.verbatim in a list of sources, it knows that it's of type VERBATIM.

Next, you tell Boost.Build that the verbatim files can be transformed into C++ files in one build step. A generator is a template for a build step that transforms targets of one type (or set of types) into another. Our generator will be called verbatim.inline-file; it transforms VERBATIM files into CPP files:

import generators ;
generators.register-standard verbatim.inline-file : VERBATIM : CPP ;

Lastly, you have to inform Boost.Build about the shell commands used to make that transformation. That's done with an actions declaration.

actions inline-file
{
    "./inline-file.py" $(<) $(>)
}

Now, we're ready to tie it all together. Put all the code above in file verbatim.jam, add import verbatim ; to Jamroot.jam, and it's possible to write the following in your Jamfile:

exe codegen : codegen.cpp class_template.verbatim usage.verbatim ;

The listed verbatim files will be automatically converted into C++ source files, compiled and then linked to the codegen executable.

In subsequent sections, we will extend this example, and review all the mechanisms in detail. The complete code is available in the example/customization directory.

The first thing we did in the intruduction was declaring a new target type:

import type ;
type.register VERBATIM : verbatim ;

The type is the most important property of a target. Boost.Build can automatically generate necessary build actions only because you specify the desired type (using the different main target rules), and because Boost.Build can guess the type of sources from their extensions.

The first two parameters for the type.register rule are the name of new type and the list of extensions associated with it. A file with an extension from the list will have the given target type. In the case where a target of the declared type is generated from other sources, the first specified extension will be used.

Sometimes you want to change the suffix used for generated targets depending on build properties, such as toolset. For example, some compiler uses extension elf for executable files. You can use the type.set-generated-target-suffix rule:

type.set-generated-target-suffix EXE : <toolset>elf : elf ;

A new target type can be inherited from an existing one.

type.register PLUGIN : : SHARED_LIB ;

The above code defines a new type derived from SHARED_LIB. Initially, the new type inherits all the properties of the base type - in particular generators and suffix. Typically, you'll change the new type in some way. For example, using type.set-generated-target-suffix you can set the suffix for the new type. Or you can write special a generator for the new type. For example, it can generate additional metainformation for the plugin. In either way, the PLUGIN type can be used whenever SHARED_LIB can. For example, you can directly link plugins to an application.

A type can be defined as "main", in which case Boost.Build will automatically declare a main target rule for building targets of that type. More details can be found later.

class verbatim-scanner : common-scanner
{
    rule pattern ( )
    {
        return "//###include[ ]*\"([^\"]*)\"" ;
    }
}

scanner.register verbatim-scanner : include ;

type.set-scanner VERBATIM : verbatim-scanner ;

This section will describe how Boost.Build can be extended to support new tools.

For each additional tool, a Boost.Build object called generator must be created. That object has specific types of targets that it accepts and produces. Using that information, Boost.Build is able to automatically invoke the generator. For example, if you declare a generator that takes a target of the type D and produces a target of the type OBJ, when placing a file with extention .d in a list of sources will cause Boost.Build to invoke your generator, and then to link the resulting object file into an application. (Of course, this requires that you specify that the .d extension corresponds to the D type.)

Each generator should be an instance of a class derived from the generator class. In the simplest case, you don't need to create a derived class, but simply create an instance of the generator class. Let's review the example we've seen in the introduction.

import generators ;
generators.register-standard verbatim.inline-file : VERBATIM : CPP ;
actions inline-file
{
    "./inline-file.py" $(<) $(>)
}

We declare a standard generator, specifying its id, the source type and the target type. When invoked, the generator will create a target of type CPP with a source target of type VERBATIM as the only source. But what command will be used to actually generate the file? In bjam, actions are specified using named "actions" blocks and the name of the action block should be specified when creating targets. By convention, generators use the same name of the action block as their own id. So, in above example, the "inline-file" actions block will be used to convert the source into the target.

There are two primary kinds of generators: standard and composing, which are registered with the generators.register-standard and the generators.register-composing rules, respectively. For example:

generators.register-standard verbatim.inline-file : VERBATIM : CPP ;
generators.register-composing mex.mex : CPP LIB : MEX ;

The first (standard) generator takes a single source of type VERBATIM and produces a result. The second (composing) generator takes any number of sources, which can have either the CPP or the LIB type. Composing generators are typically used for generating top-level target type. For example, the first generator invoked when building an exe target is a composing generator corresponding to the proper linker.

You should also know about two specific functions for registering generators: generators.register-c-compiler and generators.register-linker. The first sets up header dependecy scanning for C files, and the seconds handles various complexities like searched libraries. For that reason, you should always use those functions when adding support for compilers and linkers.

(Need a note about UNIX)

Custom generator classes

The standard generators allows you to specify source and target types, an action, and a set of flags. If you need anything more complex, you need to create a new generator class with your own logic. Then, you have to create an instance of that class and register it. Here's an example how you can create your own generator class:

class custom-generator : generator
{
    rule __init__ ( * : * )
    {
        generator.__init__ $(1) : $(2) : $(3) : $(4) : $(5) : $(6) : $(7) : $(8) : $(9) ;
    }

}

generators.register
  [ new custom-generator verbatim.inline-file : VERBATIM : CPP ] ;

This generator will work exactly like the verbatim.inline-file generator we've defined above, but it's possible to customize the behaviour by overriding methods of the generator class.

There are two methods of interest. The run method is responsible for the overall process - it takes a number of source targets, converts them to the right types, and creates the result. The generated-targets method is called when all sources are converted to the right types to actually create the result.

The generated-targets method can be overridden when you want to add additional properties to the generated targets or use additional sources. For a real-life example, suppose you have a program analysis tool that should be given a name of executable and the list of all sources. Naturally, you don't want to list all source files manually. Here's how the generated-targets method can find the list of sources automatically:

class itrace-generator : generator {
....
    rule generated-targets ( sources + : property-set : project name ? )
    {
        local leaves ;
        local temp = [ virtual-target.traverse $(sources[1]) : : include-sources ] ;
        for local t in $(temp)
        {
            if ! [ $(t).action ]
            {
                leaves += $(t) ;
            }
        }
        return [ generator.generated-targets $(sources) $(leafs)
          : $(property-set) : $(project) $(name) ] ;
    }
}
generators.register [ new itrace-generator nm.itrace : EXE : ITRACE ] ;

The generated-targets method will be called with a single source target of type EXE. The call to virtual-target.traverse will return all targets the executable depends on, and we further find files that are not produced from anything. The found targets are added to the sources.

The run method can be overriden to completely customize the way the generator works. In particular, the conversion of sources to the desired types can be completely customized. Here's another real example. Tests for the Boost Python library usually consist of two parts: a Python program and a C++ file. The C++ file is compiled to Python extension that is loaded by the Python program. But in the likely case that both files have the same name, the created Python extension must be renamed. Otherwise, the Python program will import itself, not the extension. Here's how it can be done:

rule run ( project name ? : property-set : sources * )
{
    local python ;
    for local s in $(sources)
    {
        if [ $(s).type ] = PY
        {
            python = $(s) ;
        }
    }
    
    local libs ;
    for local s in $(sources)
    {
        if [ type.is-derived [ $(s).type ] LIB ]
        {
            libs += $(s) ;
        }
    }

    local new-sources ;
    for local s in $(sources)
    {
        if [ type.is-derived [ $(s).type ] CPP ]
        {
            local name = [ $(s).name ] ;    # get the target's basename
            if $(name) = [ $(python).name ]
            {
                name = $(name)_ext ;        # rename the target
            }
            new-sources += [ generators.construct $(project) $(name) :
              PYTHON_EXTENSION : $(property-set) : $(s) $(libs) ] ;
        }
    }

    result = [ construct-result $(python) $(new-sources) : $(project) $(name)
                 : $(property-set) ] ;
}

First, we separate all source into python files, libraries and C++ sources. For each C++ source we create a separate Python extension by calling generators.construct and passing the C++ source and the libraries. At this point, we also change the extension's name, if necessary.

Often, we need to control the options passed the invoked tools. This is done with features. Consider an example:

# Declare a new free feature
import feature : feature ;
feature verbatim-options : : free ;

# Cause the value of the 'verbatim-options' feature to be
# available as 'OPTIONS' variable inside verbatim.inline-file
import toolset : flags ;
flags verbatim.inline-file OPTIONS <verbatim-options> ;

# Use the "OPTIONS" variable
actions inline-file
{
    "./inline-file.py" $(OPTIONS) $(<) $(>)
}

We first define a new feature. Then, the flags invocation says that whenever verbatin.inline-file action is run, the value of the verbatim-options feature will be added to the OPTIONS variable, and can be used inside the action body. You'd need to consult online help (--help) to find all the features of the toolset.flags rule.

Although you can define any set of features and interpret their values in any way, Boost.Build suggests the following coding standard for designing features.

Most features should have a fixed set of values that is portable (tool neutral) across the class of tools they are designed to work with. The user does not have to adjust the values for a exact tool. For example, <optimization>speed has the same meaning for all C++ compilers and the user does not have to worry about the exact options passed to the compiler's command line.

Besides such portable features there are special 'raw' features that allow the user to pass any value to the command line parameters for a particular tool, if so desired. For example, the <cxxflags> feature allows you to pass any command line options to a C++ compiler. The <include> feature allows you to pass any string preceded by -I and the interpretation is tool-specific. (See the section called “ Can I get capture external program output using a Boost.Jam variable? ” for an example of very smart usage of that feature). Of course one should always strive to use portable features, but these are still be provided as a backdoor just to make sure Boost.Build does not take away any control from the user.

Using portable features is a good idea because:

Steps for adding a feauture

Adding a feature requires three steps:

Another example

Here's another example. Let's see how we can make a feature that refers to a target. For example, when linking dynamic libraries on Windows, one sometimes needs to specify a "DEF file", telling what functions should be exported. It would be nice to use this file like this:

        lib a : a.cpp : <def-file>a.def ;

Actually, this feature is already supported, but anyway...

Variants and composite features.

Sometimes you want to create a shortcut for some set of features. For example, release is a value of <variant> and is a shortcut for a set of features.

It is possible to define your own build variants. For example:

variant crazy : <optimization>speed <inlining>off
                <debug-symbols>on <profiling>on ;

will define a new variant with the specified set of properties. You can also extend an existing variant:

variant super_release : release : <define>USE_ASM ;

In this case, super_release will expand to all properties specified by release, and the additional one you've specified.

You are not restricted to using the variant feature only. Here's example that defines a brand new feature:

feature parallelism : mpi fake none : composite link-incompatible ;
feature.compose <parallelism>mpi : <library>/mpi//mpi/<parallelism>none ;
feature.compose <parallelism>fake : <library>/mpi//fake/<parallelism>none ;

This will allow you to specify the value of feature parallelism, which will expand to link to the necessary library.

A main target rule (e.g “exe” Or “lib”) creates a top-level target. It's quite likely that you'll want to declare your own and there are two ways to do that.

The first way applies when your target rule should just produce a target of specific type. In that case, a rule is already defined for you! When you define a new type, Boost.Build automatically defines a corresponding rule. The name of the rule is obtained from the name of the type, by downcasing all letters and replacing underscores with dashes. For example, if you create a module obfuscate.jam containing:

import type ;
type.register OBFUSCATED_CPP  : ocpp ;

import generators ;
generators.register-standard obfuscate.file : CPP : OBFUSCATED_CPP ;

and import that module, you'll be able to use the rule "obfuscated-cpp" in Jamfiles, which will convert source to the OBFUSCATED_CPP type.

The second way is to write a wrapper rule that calls any of the existing rules. For example, suppose you have only one library per directory and want all cpp files in the directory to be compiled into that library. You can achieve this effect using:

lib codegen : [ glob *.cpp ] ;

If you want to make it even simpler, you could add the following definition to the Jamroot.jam file:

rule glib ( name : extra-sources * : requirements * )
{
    lib $(name) : [ glob *.cpp ] $(extra-sources) : $(requirements) ;
}

allowing you to reduce the Jamfile to just

glib codegen ;

Note that because you can associate a custom generator with a target type, the logic of building can be rather complicated. For example, the boostbook module declares a target type BOOSTBOOK_MAIN and a custom generator for that type. You can use that as example if your main target rule is non-trivial.

If your extensions will be used only on one project, they can be placed in a separate .jam file and imported by your Jamroot.jam. If the extensions will be used on many projects, users will thank you for a finishing touch.

The using rule provides a standard mechanism for loading and configuring extensions. To make it work, your module should provide an init rule. The rule will be called with the same parameters that were passed to the using rule. The set of allowed parameters is determined by you. For example, you can allow the user to specify paths, tool versions, and other options.

Here are some guidelines that help to make Boost.Build more consistent: