GitLab

This reverts commit ca412786.

The equivs package is used to create dummy Debian packages which can
be useful in development to satisfy dependencies without installing the
real package. Note that this package is not the recommended way of
dealing with broken dependencies: a bug report should be filed instead.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>

Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>

Match the same compression options as unpack and allow to have an uncompressed
tar when removing the `compression` parameter.
Signed-off-by: Corentin Noël <corentin.noel@collabora.com>

Match the same compression options as unpack and allow to have an uncompressed
tar when removing the `compression` parameter.
Signed-off-by: Corentin Noël <corentin.noel@collabora.com>

The readme got updated, so refresh the man page
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>

Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>

The readme has got a bit out-of-sync with the help command; update the
readme for the new options and descriptions.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>

Fakemachine now allows the virtualisation backend to be chosen when
creating the machine, so allow the backend to be chosen by the user.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>

This property is a boolean value indicating if `apt update` will be run
before install package. Default value is 'true'.
Signed-off-by: Trung Do <trung1.dothanh@toshiba.co.jp>
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>

The command-line argument parsing library already prints any parsing
errors; there's no need to print the error again ourselves.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Added stages "Verify" and "PreMachine" allowing to use secret files
outside of recipe directory tree (both absolute and relative paths
are supported).
Verification stage ensures that configuration files from options
are existing on file system.

This is needed for cases if secret files are not the part of Debos
recipe directory and useful for CI.
Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Allow to use certificate and private key for client authentication.
Those action parameters falls thru `debootstrap` for `wget` allowing
to authenticate client on server side.
Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Testify is currently not built unless the manual tests are ran at which
point testify compiliation fails due to requiring a newer version of
golang than in buster. Unfortunately the main branch is no longer
compatible, so install from source to get the unit tests running again.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>

F2FS requires a few tweaks to be usable within debos:
  * its command-line option for setting the fs label is different from
    other mkfs utils
  * parted doesn't recognize "f2fs" as a valid filesystem, therefore it
    shouldn't appear on the command line

With these simple changes, it becomes possible to use F2FS partitions
with debos.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@gmail.com>

For the run action, debos does not complain even if both "script" and
"command" are empty which can occur due to a typo or accidentally
missing the tag.

As the documentation already mentions that one of "script" or
"command" is mandatory for the run action, let's detect the invalid
scenario and complain to the user instead of silently continuing.
Signed-off-by: Punit Agrawal <punit1.agrawal@toshiba.co.jp>

Debos currently sets the partition type to filesystem type and
does not allow the partition type to be set from the recipe.

In some situations setting the partition type is required, so add the
property `PartType` to the Partition and set the partition type to the value
of that property. If the `PartType` property is unset, the original partition
type is retained thus not breaking backwards compatibility.

For msdos, the partition type should be hexadecimal and 2-characters long.
For gpt, the partition GUID should be in GUID format and 36-characters long.
Some examples and further reading is included in the documentation.

Resolves: #98
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>

Debos currently sets the GPT partition label to the `Name` property of the
Partition and does not allow the GPT partition label to be set from the recipe.

In some situations setting the GPT partition label is required, so add the
property `PartLabel` to the Partition and set the GPT partition label to the value
of that property. By default, the `PartLabel` property is set to the `Name`
property of the Partition to not break backwards compatibility.

Resolves: #202
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>

Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

Fakemachine is subject to panic and causes Debos to exit success due to
the current logic of the exitcode. For example, the fakemachine function
CopyFileTo() panics if the file is missing.

In the case of a panic, the function never returns. Thus, the exitcode
cannot be set to 1 and Debos exits with 0.

This commit reverses the logic of the exitcode: it is initialized to 1
(i.e. failure), and it is set to 0 (i.e. success) only if Debos has
reached the end of the things it has to do (or for the help message).
Signed-off-by: Gaël PORTAY <gael.portay@collabora.com>

Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

The timeout is fixed at 10 seconds, and I see it quite often.

The man page for pacman.conf(5) reads:

 DisableDownloadTimeout

    Disable defaults for low speed limit and timeout on downloads. Use
    this if you have issues downloading files with proxy and/or security
    gateway.
Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

Before this commit, we hard-coded the 3 repositories 'core', 'extra',
and 'community', and we only allowed the user define the mirror and
mirror layout for those.

This is not really a good idea, as none of these repositories are
required to exist, or even to be named that way. These are just
conventions. An Archlinux derivative could as well decide to have their
repositories named 'main', 'contrib' and 'non-free', for example.

Hence it seems more flexible to have a 'repositories' property instead,
and let user define the repository name and the repository server. This
way, there's no need for a mirror and mirror-layout properties.

With this change, a basic recipe to build an Archlinux OS can be:

    architecture: amd64

    actions:
      - action: pacstrap
        repositories:
          - name: core
            server: http://mirrors.kernel.org/archlinux/$repo/os/$arch
          - name: extra
            server: http://mirrors.kernel.org/archlinux/$repo/os/$arch

      - action: pack
        compression: gz
        file: recipe.tar.gz

More complicated, a recipe that allows to build both ArchLinux or
Manjaro can be defined like that:

    {{  $arch := or .arch "amd64" -}}
    {{- $distrib := or .distrib "archlinux" -}}
    {{- $mirror := or .mirror "http://mirrors.kernel.org/archlinux

" -}}
    {{- $layout := or .layout "$repo/os/$arch" -}}

    {{- if eq $distrib "manjaro" -}}
    {{- $mirror = or .mirror "http://repo.manjaro.org.uk" -}}
    {{- $layout = or .layout "stable/$repo/$arch" -}}
    {{- end -}}

    architecture: {{ $arch }}

    actions:
      - action: pacstrap
        repositories:
          - name: core
            server: {{ $mirror }}/{{ $layout }}
          - name: extra
            server: {{ $mirror }}/{{ $layout }}
          - name: community
            server: {{ $mirror }}/{{ $layout }}

      - action: pack
        compression: gz
        file: {{ $distrib }}-base.tar.gz

And it can be invoked in the following ways:

    # build archlinux:
    debos recipe.yaml

    # build manjaro:
    debos -t distrib:manjaro recipe.yaml

Introducing the 'repository' property has the consequence of removing
the 'mirror' property, which can't make anymore. It means that debos
doesn't try to handle the mirror list anymore, it's left to the user.

By default, ArchLinux leaves you with no mirror configured, so pacman
fails until you manually enable a mirror in /etc/pacman.d/mirrolist.

Manjaro, on the other hand, leaves you with a mirrorlist where all the
mirrors are enabled, and possibly NOT sorted. Watch the pacstrap logs
for the messages:

    Pacstrap | ::INFO Internet connection appears to be down
    Pacstrap | ::INFO Mirror ranking is not available
    Pacstrap | ::INFO Mirror list is generated using random method
    Pacstrap | ::INFO Writing mirror list

One last word.

Remember that these repositories are only used to create the pacman.conf
file used during the pacstrap action. pacstrap itself will install a
default pacman config and a default mirror list in the bootstrapped
system, hence subsequent calls to the `pacman` action will use those.

So if you want to use a particular mirror, defining it in the pacstrap
action is not enough, you will also need to enable it in the
bootstrapped system.
Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

With this change, pacman-key will populate the pacman keyring in the
chroot with EVERY keyring found in /usr/share/pacman/keyrings on the
host.

For example, assuming you're building a Manjaro image, you need to
populate the chroot with the two keyrings 'archlinux' and 'manjaro'.
Assuming that both keyrings are installed on the host, then you have two
ways to achieve that
- run `pacman-key --populate archlinux manjaro`, which install exactly
  these two keyrings in the chroot.
- run `pacman-key --populate`, which installs every keyring found in
  /usr/share/pacman/keyrings.

This commit makes the assumption that the host has already all the
keyrings needed in /usr/share/pacman/keyrings, and that it's desirable
to install them all in the chroot.
Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

ArchLinux's default mirror layout is $repo/os/$arch, however in
ArchLinux world the mirror layout doesn't seem to be enforced by
anything, so there's some variety out there.

Within ArchLinux itself, it seems that there are at least two different
existing layouts, as seen in mkosi sources:

    if platform.machine() == "aarch64":
        server = f"Server = {args.mirror}/$arch/$repo"
    else:
        server = f"Server = {args.mirror}/$repo/os/$arch"

Additionally, Manjaro mirrors also do it differently:

    {args.mirror}/$branch/$repo/$arch

So we can't hardcode the mirror layout in debos anymore.

This commit let user enter a string that defines the mirror layout. If
not set, debos defaults to ArchLinux's default layout '$repo/os/$arch'.
Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

I experience regular failures due to timeout, for example:

  error: failed retrieving file 'libssh2-1.8.2-1-x86_64.pkg.tar.xz' from http://mirrors.kernel.org/archlinux :
  Connection timed out after 10001 milliseconds

The pacman manual mentions:

  --disable-download-timeout
    Disable defaults for low speed limit and timeout on downloads. Use this
    if you have issues downloading files with proxy and/or security gateway.

I do run a proxy most of the time, but even when I don't I still see
this issue. So let's disable this timeout.

Ref: <https://www.archlinux.org/pacman/pacman.8.html

>
Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>

Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>