kumquat-buildroot/docs/manual/customize-outside-br.adoc

// -*- mode:doc -*- ;
// vim: set syntax=asciidoc:

[[outside-br-custom]]
=== Keeping customizations outside of Buildroot

As already briefly mentioned in xref:customize-dir-structure[], you can
place project-specific customizations in two locations:

 * directly within the Buildroot tree, typically maintaining them using
   branches in a version control system so that upgrading to a newer
   Buildroot release is easy.

 * outside of the Buildroot tree, using the _br2-external_ mechanism.
   This mechanism allows to keep package recipes, board support and
   configuration files outside of the Buildroot tree, while still
   having them nicely integrated in the build logic. We call this
   location a _br2-external tree_. This section explains how to use
   the br2-external mechanism and what to provide in a br2-external
   tree.

One can tell Buildroot to use one or more br2-external trees by setting
the +BR2_EXTERNAL+ make variable set to the path(s) of the br2-external
tree(s) to use. It can be passed to any Buildroot +make+ invocation. It
is automatically saved in the hidden +.br2-external.mk+ file in the output
directory. Thanks to this, there is no need to pass +BR2_EXTERNAL+ at
every +make+ invocation. It can however be changed at any time by
passing a new value, and can be removed by passing an empty value.

.Note
The path to a br2-external tree can be either absolute or relative.
If it is passed as a relative path, it is important to note that it is
interpreted relative to the main Buildroot source directory, *not* to
the Buildroot output directory.

.Note:
If using an br2-external tree from before Buildroot 2016.11, you need to
convert it before you can use it with Buildroot 2016.11 onward. See
xref:br2-external-converting[] for help on doing so.

Some examples:

-----
buildroot/ $ make BR2_EXTERNAL=/path/to/foo menuconfig
-----

From now on, definitions from the +/path/to/foo+ br2-external tree
will be used:

-----
buildroot/ $ make
buildroot/ $ make legal-info
-----

We can switch to another br2-external tree at any time:

-----
buildroot/ $ make BR2_EXTERNAL=/where/we/have/bar xconfig
-----

We can also use multiple br2-external trees:

----
buildroot/ $ make BR2_EXTERNAL=/path/to/foo:/where/we/have/bar menuconfig
----

Or disable the usage of any br2-external tree:

-----
buildroot/ $ make BR2_EXTERNAL= xconfig
-----

==== Layout of a br2-external tree

A br2-external tree must contain at least those three files, described
in the following chapters:

 * +external.desc+
 * +external.mk+
 * +Config.in+

Apart from those mandatory files, there may be additional and optional
content that may be present in a br2-external tree, like the +configs/+
or +provides/+ directories. They are described in the following chapters
as well.

A complete example br2-external tree layout is also described later.

===== The +external.desc+ file

That file describes the br2-external tree: the _name_ and _description_
for that br2-external tree.

The format for this file is line based, with each line starting by a
keyword, followed by a colon and one or more spaces, followed by the
value assigned to that keyword. There are two keywords currently
recognised:

 * +name+, mandatory, defines the name for that br2-external tree. That
   name must only use ASCII characters in the set +[A-Za-z0-9_]+; any
   other character is forbidden. Buildroot sets the variable
   +BR2_EXTERNAL_$(NAME)_PATH+ to the absolute path of the br2-external
   tree, so that you can use it to refer to your br2-external tree. This
   variable is available both in Kconfig, so you can use it to source your
   Kconfig files (see below) and in the Makefile, so that you can use it
   to include other Makefiles (see below) or refer to other files (like
   data files) from your br2-external tree.
+
.Note:
Since it is possible to use multiple br2-external trees at once, this
  name is used by Buildroot to generate variables for each of those trees.
  That name is used to identify your br2-external tree, so try to come up
  with a name that really describes your br2-external tree, in order for
  it to be relatively unique, so that it does not clash with another name
  from another br2-external tree, especially if you are planning on
  somehow sharing your br2-external tree with third parties or using
  br2-external trees from third parties.

 * +desc+, optional, provides a short description for that br2-external
   tree. It shall fit on a single line, is mostly free-form (see below),
   and is used when displaying information about a br2-external tree (e.g.
   above the list of defconfig files, or as the prompt in the menuconfig);
   as such, it should relatively brief (40 chars is probably a good upper
   limit). The description is available in the +BR2_EXTERNAL_$(NAME)_DESC+
   variable.

Examples of names and the corresponding +BR2_EXTERNAL_$(NAME)_PATH+
variables:

  * +FOO+ -> +BR2_EXTERNAL_FOO_PATH+
  * +BAR_42+ -> +BR2_EXTERNAL_BAR_42_PATH+

In the following examples, it is assumed the name to be set to +BAR_42+.

.Note:
Both +BR2_EXTERNAL_$(NAME)_PATH+ and `BR2_EXTERNAL_$(NAME)_DESC` are
  available in the Kconfig files and the Makefiles. They are also
  exported in the environment so are available in post-build, post-image
  and in-fakeroot scripts.

===== The +Config.in+ and +external.mk+ files

Those files (which may each be empty) can be used to define package
recipes (i.e. +foo/Config.in+ and +foo/foo.mk+ like for packages bundled
in Buildroot itself) or other custom configuration options or make logic.

Buildroot automatically includes the +Config.in+ from each br2-external
tree to make it appear in the top-level configuration menu, and includes
the +external.mk+ from each br2-external tree with the rest of the
makefile logic.

The main usage of this is to store package recipes. The recommended way
to do this is to write a +Config.in+ file that looks like:

------
source "$BR2_EXTERNAL_BAR_42_PATH/package/package1/Config.in"
source "$BR2_EXTERNAL_BAR_42_PATH/package/package2/Config.in"
------

Then, have an +external.mk+ file that looks like:

------
include $(sort $(wildcard $(BR2_EXTERNAL_BAR_42_PATH)/package/*/*.mk))
------

And then in +$(BR2_EXTERNAL_BAR_42_PATH)/package/package1+ and
+$(BR2_EXTERNAL_BAR_42_PATH)/package/package2+ create normal
Buildroot package recipes, as explained in xref:adding-packages[].
If you prefer, you can also group the packages in subdirectories
called <boardname> and adapt the above paths accordingly.

You can also define custom configuration options in +Config.in+ and
custom make logic in +external.mk+.

===== The +configs/+ directory

One can store Buildroot defconfigs in the +configs+ subdirectory of
the br2-external tree. Buildroot will automatically show them in the
output of +make list-defconfigs+ and allow them to be loaded with the
normal +make <name>_defconfig+ command. They will be visible in the
'make list-defconfigs' output, below an +External configs+ label that
contains the name of the br2-external tree they are defined in.

.Note:
If a defconfig file is present in more than one br2-external tree, then
the one from the last br2-external tree is used. It is thus possible
to override a defconfig bundled in Buildroot or another br2-external
tree.

===== The +provides/+ directory

For some packages, Buildroot provides a choice between two (or more)
implementations of API-compatible such packages. For example, there is
a choice to choose either libjpeg ot jpeg-turbo; there is one between
openssl or libressl; there is one to select one of the known,
pre-configured toolchains...

It is possible for a br2-external to extend those choices, by providing
a set of files that define those alternatives:

* +provides/toolchains.in+ defines the pre-configured toolchains, which
  will then be listed in the toolchain selection;
* +provides/jpeg.in+ defines the alternative libjpeg implementations;
* +provides/openssl.in+ defines the alternative openssl implementations;
* +provides/skeleton.in+ defines the alternative skeleton implementations;
* +provides/init.in+ defines the alternative init system implementations, this
  can be used to select a default skeleton for your init.

===== Free-form content

One can store all the board-specific configuration files there, such
as the kernel configuration, the root filesystem overlay, or any other
configuration file for which Buildroot allows to set the location (by
using the +BR2_EXTERNAL_$(NAME)_PATH+ variable). For example, you
could set the paths to a global patch directory, to a rootfs overlay
and to the kernel configuration file as follows (e.g. by running
`make menuconfig` and filling in these options):

----
BR2_GLOBAL_PATCH_DIR=$(BR2_EXTERNAL_BAR_42_PATH)/patches/
BR2_ROOTFS_OVERLAY=$(BR2_EXTERNAL_BAR_42_PATH)/board/<boardname>/overlay/
BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE=$(BR2_EXTERNAL_BAR_42_PATH)/board/<boardname>/kernel.config
----

===== Additional Linux kernel extensions

Additional Linux kernel extensions (see xref:linux-kernel-ext[]) can
be added by storing them in the `linux/` directory at the root of a
br2-external tree.

===== Example layout

Here is an example layout using all features of br2-external (the sample
content is shown for the file above it, when it is relevant to explain
the br2-external tree; this is all entirely made up just for the sake of
illustration, of course):

----
/path/to/br2-ext-tree/
  |- external.desc
  |     |name: BAR_42
  |     |desc: Example br2-external tree
  |     `----
  |
  |- Config.in
  |     |source "$BR2_EXTERNAL_BAR_42_PATH/toolchain/toolchain-external-mine/Config.in.options"
  |     |source "$BR2_EXTERNAL_BAR_42_PATH/package/pkg-1/Config.in"
  |     |source "$BR2_EXTERNAL_BAR_42_PATH/package/pkg-2/Config.in"
  |     |source "$BR2_EXTERNAL_BAR_42_PATH/package/my-jpeg/Config.in"
  |     |
  |     |config BAR_42_FLASH_ADDR
  |     |    hex "my-board flash address"
  |     |    default 0x10AD
  |     `----
  |
  |- external.mk
  |     |include $(sort $(wildcard $(BR2_EXTERNAL_BAR_42_PATH)/package/*/*.mk))
  |     |include $(sort $(wildcard $(BR2_EXTERNAL_BAR_42_PATH)/toolchain/*/*.mk))
  |     |
  |     |flash-my-board:
  |     |    $(BR2_EXTERNAL_BAR_42_PATH)/board/my-board/flash-image \
  |     |        --image $(BINARIES_DIR)/image.bin \
  |     |        --address $(BAR_42_FLASH_ADDR)
  |     `----
  |
  |- package/pkg-1/Config.in
  |     |config BR2_PACKAGE_PKG_1
  |     |    bool "pkg-1"
  |     |    help
  |     |      Some help about pkg-1
  |     `----
  |- package/pkg-1/pkg-1.hash
  |- package/pkg-1/pkg-1.mk
  |     |PKG_1_VERSION = 1.2.3
  |     |PKG_1_SITE = /some/where/to/get/pkg-1
  |     |PKG_1_LICENSE = blabla
  |     |
  |     |define PKG_1_INSTALL_INIT_SYSV
  |     |    $(INSTALL) -D -m 0755 $(PKG_1_PKGDIR)/S99my-daemon \
  |     |                          $(TARGET_DIR)/etc/init.d/S99my-daemon
  |     |endef
  |     |
  |     |$(eval $(autotools-package))
  |     `----
  |- package/pkg-1/S99my-daemon
  |
  |- package/pkg-2/Config.in
  |- package/pkg-2/pkg-2.hash
  |- package/pkg-2/pkg-2.mk
  |
  |- provides/jpeg.in
  |     |config BR2_PACKAGE_MY_JPEG
  |     |    bool "my-jpeg"
  |     `----
  |- package/my-jpeg/Config.in
  |     |config BR2_PACKAGE_PROVIDES_JPEG
  |     |    default "my-jpeg" if BR2_PACKAGE_MY_JPEG
  |     `----
  |- package/my-jpeg/my-jpeg.mk
  |     |# This is a normal package .mk file
  |     |MY_JPEG_VERSION = 1.2.3
  |     |MY_JPEG_SITE = https://example.net/some/place
  |     |MY_JPEG_PROVIDES = jpeg
  |     |$(eval $(autotools-package))
  |     `----
  |
  |- provides/init.in
  |     |config BR2_INIT_MINE
  |     |    bool "my custom init"
  |     |    select BR2_PACKAGE_MY_INIT
  |     |    select BR2_PACKAGE_SKELETON_INIT_MINE if BR2_ROOTFS_SKELETON_DEFAULT
  |     `----
  |
  |- provides/skeleton.in
  |     |config BR2_ROOTFS_SKELETON_MINE
  |     |    bool "my custom skeleton"
  |     |    select BR2_PACKAGE_SKELETON_MINE
  |     `----
  |- package/skeleton-mine/Config.in
  |     |config BR2_PACKAGE_SKELETON_MINE
  |     |    bool
  |     |    select BR2_PACKAGE_HAS_SKELETON
  |     |
  |     |config BR2_PACKAGE_PROVIDES_SKELETON
  |     |    default "skeleton-mine" if BR2_PACKAGE_SKELETON_MINE
  |     `----
  |- package/skeleton-mine/skeleton-mine.mk
  |     |SKELETON_MINE_ADD_TOOLCHAIN_DEPENDENCY = NO
  |     |SKELETON_MINE_ADD_SKELETON_DEPENDENCY = NO
  |     |SKELETON_MINE_PROVIDES = skeleton
  |     |SKELETON_MINE_INSTALL_STAGING = YES
  |     |$(eval $(generic-package))
  |     `----
  |
  |- provides/toolchains.in
  |     |config BR2_TOOLCHAIN_EXTERNAL_MINE
  |     |    bool "my custom toolchain"
  |     |    depends on BR2_some_arch
  |     |    select BR2_INSTALL_LIBSTDCPP
  |     `----
  |- toolchain/toolchain-external-mine/Config.in.options
  |     |if BR2_TOOLCHAIN_EXTERNAL_MINE
  |     |config BR2_TOOLCHAIN_EXTERNAL_PREFIX
  |     |    default "arch-mine-linux-gnu"
  |     |config BR2_PACKAGE_PROVIDES_TOOLCHAIN_EXTERNAL
  |     |    default "toolchain-external-mine"
  |     |endif
  |     `----
  |- toolchain/toolchain-external-mine/toolchain-external-mine.mk
  |     |TOOLCHAIN_EXTERNAL_MINE_SITE = https://example.net/some/place
  |     |TOOLCHAIN_EXTERNAL_MINE_SOURCE = my-toolchain.tar.gz
  |     |$(eval $(toolchain-external-package))
  |     `----
  |
  |- linux/Config.ext.in
  |     |config BR2_LINUX_KERNEL_EXT_EXAMPLE_DRIVER
  |     |    bool "example-external-driver"
  |     |    help
  |     |      Example external driver
  |     |---
  |- linux/linux-ext-example-driver.mk
  |
  |- configs/my-board_defconfig
  |     |BR2_GLOBAL_PATCH_DIR="$(BR2_EXTERNAL_BAR_42_PATH)/patches/"
  |     |BR2_ROOTFS_OVERLAY="$(BR2_EXTERNAL_BAR_42_PATH)/board/my-board/overlay/"
  |     |BR2_ROOTFS_POST_IMAGE_SCRIPT="$(BR2_EXTERNAL_BAR_42_PATH)/board/my-board/post-image.sh"
  |     |BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="$(BR2_EXTERNAL_BAR_42_PATH)/board/my-board/kernel.config"
  |     `----
  |
  |- patches/linux/0001-some-change.patch
  |- patches/linux/0002-some-other-change.patch
  |- patches/busybox/0001-fix-something.patch
  |
  |- board/my-board/kernel.config
  |- board/my-board/overlay/var/www/index.html
  |- board/my-board/overlay/var/www/my.css
  |- board/my-board/flash-image
  `- board/my-board/post-image.sh
        |#!/bin/sh
        |generate-my-binary-image \
        |    --root ${BINARIES_DIR}/rootfs.tar \
        |    --kernel ${BINARIES_DIR}/zImage \
        |    --dtb ${BINARIES_DIR}/my-board.dtb \
        |    --output ${BINARIES_DIR}/image.bin
        `----
----

The br2-external tree will then be visible in the menuconfig (with
the layout expanded):

----
External options  --->
    *** Example br2-external tree (in /path/to/br2-ext-tree/)
    [ ] pkg-1
    [ ] pkg-2
    (0x10AD) my-board flash address
----

If you are using more than one br2-external tree, it would look like
(with the layout expanded and the second one with name +FOO_27+ but no
+desc:+ field in +external.desc+):

----
External options  --->
    Example br2-external tree  --->
        *** Example br2-external tree (in /path/to/br2-ext-tree)
        [ ] pkg-1
        [ ] pkg-2
        (0x10AD) my-board flash address
    FOO_27  --->
        *** FOO_27 (in /path/to/another-br2-ext)
        [ ] foo
        [ ] bar
----

Additionally, the jpeg provider will be visible in the jpeg choice:

----
Target packages  --->
    Libraries  --->
        Graphics  --->
            [*] jpeg support
                jpeg variant ()  --->
                    ( ) jpeg
                    ( ) jpeg-turbo
                        *** jpeg from: Example br2-external tree ***
                    (X) my-jpeg
                        *** jpeg from: FOO_27 ***
                    ( ) another-jpeg
----

And similarly for the toolchains:

----
Toolchain  --->
    Toolchain ()  --->
        ( ) Custom toolchain
            *** Toolchains from: Example br2-external tree ***
        (X) my custom toolchain
----

.Note
The toolchain options in +toolchain/toolchain-external-mine/Config.in.options+
will not appear in the `Toolchain` menu. They must be explicitly included
from within the br2-external's top-level +Config.in+ and will thus appear
in the `External options` menu.