Setting up cross-project testing
Cross-project testing enables you to ensure the compatibility and reliability of interconnected repositories. And with a little bit of configuration on Packit and tmt parts, it can be easily automated within Packit!
Big kudos to Martin Pitt who experimented with these settings in
cockpit repositories and wrote an amazing blog post about it!
If you are interested in all the details, make sure to read the
full blog post. Alternatively, the following sections
will describe the needed configuration for setting it up.
After determining the dependency (or dependencies) you want to test, the configuration is relatively easy:
- For the dependent repository, Packit should be configured to build every commit of the
mainbranch, or any other similar branch containing recent changes, into a dedicated Copr repository.
- For the dependency repository, you should add a separate Packit test job with a tmt test plan to run on all pull requests.
Let's break down these steps using examples from the
cockpit setup (where
cockpit-podman depends on the
Configuring Packit builds on commits of the dependent repository
The goal here is to build the most recent code (usually in the
main branch) of the repo in a dedicated Copr repository,
which can be easily done using Packit.
Firstly a new dedicated Copr project needs to be created and configured as needed.
For Packit to build in this project, the
Packit allowed forge projects needs to be filled as well.
Then you can configure a Packit
copr_build job to do a build in the new Copr repo for each commit that lands in the desired
branch of the dependent repo (in this example
main branch of
- job: copr_build
You will need to approve the
packit Copr user to be able to trigger builds in the Copr repo.
The additional details of the setup are described here.
Configuring the test plan
After that, a new test plan
plans/<your-project>.fmf needs to be added to the dependency repository, which selects the
tests you want to run from this repo. Unfortunately, it is impossible to auto-import all plans
at the moment (but this may be useful since you can select a subset of tests that apply to the tested dependency).
So this is mostly a copy of the dependent repo test plan, with one modification: they should not run by default,
but only in a
revdeps tmt context. The test plan can then look like this (an example from
# reverse dependency test
when: revdeps == yes
summary: Run cockpit-podman system tests
# ... possibly more test plans here
For the other plans in this repository, you should mark it to not run in the
when: revdeps == yes
Configuring the Packit test job
Finally, to plug everything together, a Packit test job needs to be configured that will run the
revdeps context test plan
against the builds from
main branch of the dependent repo by specifying
The key here is to install the latest
main build from the dependency: we are making sure we are not breaking
Make sure to also give it a meaningful
identifier so that you can differentiate the statuses:
- job: tests
- type: repository-file