Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>

Aptly is currently re-publishing (dropping and publishing again)
repositories when the project configuration is changed. This is
performed to update the components and architectures of the published
distribution based on the updated project configuration.

OBS/aptly contains a hack allowing the usage of slashes `/` in the
publish distribution field, which we're currently using to have the
snapshots published in the same prefix and using a distribution name
with the form: `$release/snapshots/$timestamp`. This makes the snapshots
folder to be inside the same `dists` directory where the repositories
are published.

There's a bug in the above implementation, where:
* OBS project configuration is updated, triggering a re-publishing of
  the distribution.
* Aptly drops the published distribution.
* If aptly doesn't have any other distribution published in the same
  prefix, it will remove the entire `dists` folder, with all the
  snapshots in it.
* Aptly p...

The installed aptly version now allows to disable bzip2 compression for
index files. Enable this to speed up aptly publishing, as speed
performance is more important than Package file size in this case.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

The `bs_aptly_snapshot_policy` script run snapshot retention policies.
Each snapshot retention policy is defined by a YAML file, stored by
default at `/srv/obs/aptly/snapshot.policy.d/`.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

We don't use the latest symlink, just the latest.txt timestamp. So drop
it.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

The old reprepro publisher hook script, made the assumption that
-security and -updates suffixes always existed, but this is not the case
with aptly. This commit updates the aptly publisher hook script to check
if the snapshot directory exists for those suffixes and ignores it if it
doesn't. In that case, a warning is gently printed to stderr.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

In order to rsync this properly, the symlink needs to be relative,
otherwise it's not found.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Ariel D'Alessandro authored Jun 13, 2022 and Sjoerd Simons committed Jun 16, 2022

The latest published snapshot required latest/ dir and latest.txt file
to point to it. Generate those.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Ariel D'Alessandro authored Jun 13, 2022 and Sjoerd Simons committed Jun 16, 2022

Add a function to read aptly global configuration options.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Ariel D'Alessandro authored Jun 13, 2022 and Sjoerd Simons committed Jun 16, 2022

Reprepro was setting the snapshot Codename (a.k.a. Suite) to
snapshots/$distro/$timestamp. Let's use the same convention for
compatability.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

The required aptly version includes custom collabora patches. Install
it instead of the upstream version.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

The gpg-key needs to be passed as an argument when a repository is
re-published. This is now missing, so the wrong gpg-key is being used.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

We want to publish sources by default, which needs to be added to the
architectures list on publishing.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Generating Contents indexes takes ages and and require a lot of data.
Basically, aptly scan through the contents of each .deb file.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

This aptly script performs a database cleanup to remove information
about unreferenced packages. Also deletes files in the package pool that
aren't used by packages anymore.

Aptly documentation suggests that it's a good idea to run this command
after massive deletion of mirrors, snapshots or local repos.

This commit adds the script to be called in a weekly basis by cron.
Note that with this configuration it will be called each Sunday at
00:00 hours.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Cron will be used to run periodic jobs on the backend. Let's add this to
the processed launched and monitored by supervisor.

Such cron functionality is required by aptly, which has been now
integrated to OBS. A few related scripts need to be run on the backend
to perform maintenance tasks.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Package binaries are appended with the release suffix, so the target
filenames need to be updated. For some strange reason, this wasn't
working before, as the obs-build submodule wasn't properly initialized.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

This commit adds helper functions to the aptly module in order to remove
and drop published snapshots.

The script `src/backend/bs_aptly_drop_snapshot` is added as the default
reference logic to drop snapshots that were created using
`src/backend/bs_published_hook_aptly_snapshot`.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

This commit adds test scripts for reprepro to aptly repository
migration.

While here, rename aptly test.sh script as aptly.sh to be more specific
about its functionality.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Split common functions from aptly test that could be used by further
test scripts.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

The script `./src/backend/bs_aptly_migration` can be used to migrate
reprepro repositories to aptly.

Note that reprepro repositories and snapshots cleanup is not implemented
by this script.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Extend the testing configuration to define repositories for both
reprepro and aptly.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

This is the default reprepro snaphost hook script. Let's add it until
OBS reprepro support is finally dropped.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

This function cleans up all contents from a repository.

In order to re-create snapshots from reprepro, we may wanna reset a
repository, then add all packages from an old reprepro snapshot and
create the aptly snapshot from there.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Update the prefix used in the aptly test, to match the one used by
reprepro repositories in general.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

These functions will be used from other perl scripts as well. Let's make
it available from the aptly module.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Andrej Shadura authored May 02, 2022 and Sjoerd Simons committed May 12, 2022

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Ariel D'Alessandro authored Mar 29, 2022 and Sjoerd Simons committed May 12, 2022

This commit adds tests for the OBS aptly integration.

See tests/aptly/README.md for instructions on how to run the test suite.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Ariel D'Alessandro authored Feb 03, 2022 and Sjoerd Simons committed May 12, 2022

This commit integratates aptly backend. The following features are
supported:

* OBS repositories gets automatically created/removed on aptly.
* Each arch with the publish flag enabled in a repository, get published
  on aptly.
* Package binaries get added to the aptly repository right after they
  get built. Old versions are automatically removed.
* Every time a package is published, a snapshot is taken and published.

Related documentation is added to `README.aptly.md`:
* aptly configuration files.
* snapshot and db-cleanup scripts.
Signed-off-by: Ariel D'Alessandro <ariel.dalessandro@collabora.com>

Andrej Shadura authored May 12, 2022 and Sjoerd Simons committed May 12, 2022

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Andrej Shadura authored May 12, 2022 and Sjoerd Simons committed May 12, 2022

Secrets are owned by root and are not world-readable by default, so
the frontend cannot access them when it’s not running as root.
Not all versions of docker-compose support setting access rights for
secrets, so instead of wrangling with them, just copy secrets and
re-own them.
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Andrej Shadura authored May 04, 2022 and Sjoerd Simons committed May 04, 2022

The can_register check is actually only suitable for preventing new
unverified registrations; in SSO mode, we normally trust the SSO
provider have performed the checks and only gives us users we’re
supposed to let in.

Ideally, this should be a separate set of settings to allow e.g.
optionally requiring confirmation on SSO logins or to configure
different levels of trust per SSO provider.
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

The auth hash can be quite large, and with session storage in cookies,
the cookie can easily reach the 4 KB limit. Work around this issue
by only storing the part of the hash we currently use.
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

Add a new "hash type" for invalid passwords, which is never equal to
normal passwords, but nevertheless can be changed without being known by
the user.

This "invalid" password can only be set by directly setting the password
hash type. When updating the password using update_password method, it will
always be upgrade it to the strongest hash type, sha256crypt.

To allow changing this "invalid" password to a normal one, stop
requiring a non-empty current password in the password change dialog
when changing a password from an "invalid" one. Don’t show the current
password box either, as it is not used anyway in this case, making
it better not to show it to avoid confusion.
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>

The redirect should only happen to the referrer URL pointing at our
domain, never to the external ones.
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>