Skip to main content

2019

· 16 min read

Let's see what happened in Packit in 2019.

Initial version 0.1.0 of packit is out! (2019-03-08)

We would like to announce general availability of the initial version of packit, titled '0.1.0'.

Since this is our first release, we would like to ask you to be patient if you encounter any issues. We work hard on packit's usability. If you feel like that packit is doing something weird or if anything is unclear, don't hesitate and reach out to us by creating a new GitHub issue.

The initial release contains two commands:

  • packit propose-update — Opens a pull request in dist-git for the latest upstream release of a selected repository.
  • packit watch-releases — Watches events for all the upstream releases and performs propose-update for those who use packit.

Installation​

$ dnf install --enablerepo=updates-testing packit

Or

$ pip3 install --user packitos

Or (if you're brave)

$ pip3 install --user git+https://github.com/packit-service/packit

Requirements​

Present features have strict requirements on the upstream projects:

  • You need to have a packit config file present in the upstream repo.

  • You need to have spec file present in the upstream repo.

This workflow is suitable for people who are both upstream and downstream maintainers of the particular project. If you don't fit into that bucket, then packit might not be ready for you, yet. Please wait till we land more source-git related functionality into packit.

propose-update​

I'm going to demonstrate this functionality on ogr, our library for git forges, which powers packit.

It was recently approved for Fedora, so we can use packit to bring the initial version of ogr into Fedora Rawhide, 30 and 29.

Do we have everything?​

Let's see guide for the propose-update command on what we need:

0. The upstream repository with a valid upstream release.​

$ git remote -v
origin  git@github.com:TomasTomecek/ogr.git (fetch)
origin  git@github.com:TomasTomecek/ogr.git (push)
upstream        https://github.com/packit-service/ogr.git (fetch)
upstream        https://github.com/packit-service/ogr.git (push)

Yup.

$ git tag --list
0.0.1
0.0.2
0.0.3

$ git checkout 0.0.3
Note: checking out '0.0.3'.

And the tag name is matching the version in a spec file:

$ grep Version python-ogr.spec
Version:        0.0.3

1. Packit config file placed in the upstream repository.​

$ ll .packit.yaml
-rw-rw-r--. 1 tt tt 177 Mar  1 17:44 .packit.yaml

Check.

2. Spec file present in the upstream repository.​

$ ll python-ogr.spec
-rw-rw-r--. 1 tt tt 1.3K Mar  1 17:43 python-ogr.spec

👍

3. Pagure API tokens for Fedora Dist-git.​

$ env | grep TOKEN
PAGURE_USER_TOKEN=will
PAGURE_FORK_TOKEN=not
GITHUB_TOKEN=share, sorry

4. Valid Fedora Kerberos ticket.​

$ kinit ttomecek@FEDORAPROJECT.ORG
Password for ttomecek@FEDORAPROJECT.ORG:

$ klist
Ticket cache: KEYRING:persistent:1024:krb_ccache_g0t1Ty3Ah
Default principal: ttomecek@FEDORAPROJECT.ORG

Valid starting       Expires              Service principal
03/01/2019 18:12:25  03/02/2019 18:12:19  krbtgt/FEDORAPROJECT.ORG@FEDORAPROJECT.ORG
        renew until 03/08/2019 18:12:19

We're all set!

Time to shine​

We are still in the "ogr" upstream git repository.

$ packit propose-update
INFO: Running 'anitya' versioneer
ERROR: Failed to determine latest upstream version!
Check that the package exists on https://release-monitoring.org.
using "master" dist-git branch
syncing ./python-ogr.spec
INFO: Downloading file from URL https://files.pythonhosted.org/packages/source/o/ogr/ogr-0.0.3.tar.gz
100%[=============================>]    17.95K  eta 00:00:00
downloaded archive: /tmp/tmp2e65b0xt/ogr-0.0.3.tar.gz
uploading to the lookaside cache
PR created: https://src.fedoraproject.org/rpms/python-ogr/pull-request/1

Mind-blowing, isn't it? Now we have latest python-ogr in Fedora Rawhide by running only a single command.

I have also added ogr into release-monitoring as packit suggests.

Once we are okay with the changes, we have to merge the pull request. That's our responsibility, as maintainers.

Building in koji​

Time to build the package (packit doesn't support building in koji, yet)

$ fedpkg clone python-ogr
Cloning into 'python-ogr'...
remote: Counting objects: 8, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 8 (delta 0), reused 5 (delta 0)
Receiving objects: 100% (8/8), done.

$ cd python-ogr

$ git log
commit c298df5e540ba1d010366e102c1c75d4f5b0b0cc (HEAD -> master, origin/master, origin/HEAD)
Author: Tomas Tomecek <ttomecek@redhat.com>
Date:   Fri Mar 1 18:15:00 2019 +0100

    [packit] 0.0.3 upstream release

    more info

    Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>

commit 7d5ab1471ca0ee2a6c0254410b83beaa83b80f0b
Author: Gwyn Ciesla <limb@fedoraproject.org>
Date:   Fri Mar 1 15:18:34 2019 +0000

    Added the README

Yup, that's our commit. more info was added there by accident, this is already fixed in packit.

$ fedpkg build
Building python-ogr-0.0.3-1.fc31 for rawhide
Created task: 33125435
Task info: https://koji.fedoraproject.org/koji/taskinfo?taskID=33125435
Watching tasks (this may be safely interrupted)...
33125435 build (rawhide, /rpms/python-ogr.git:c298df5e540ba1d010366e102c1c75d4f5b0b0cc): free
33125435 build (rawhide, /rpms/python-ogr.git:c298df5e540ba1d010366e102c1c75d4f5b0b0cc): free -> open (buildvm-14.phx2.fedoraproject.org)
  33125451 buildArch (python-ogr-0.0.3-1.fc31.src.rpm, noarch): open (buildvm-14.phx2.fedoraproject.org)
  33125436 buildSRPMFromSCM (/rpms/python-ogr.git:c298df5e540ba1d010366e102c1c75d4f5b0b0cc): closed
33125435 build (rawhide, /rpms/python-ogr.git:c298df5e540ba1d010366e102c1c75d4f5b0b0cc): open (buildvm-14.phx2.fedoraproject.org) -> closed
  0 free  1 open  2 done  0 failed
  33125464 tagBuild (noarch): closed
  33125451 buildArch (python-ogr-0.0.3-1.fc31.src.rpm, noarch): open (buildvm-14.phx2.fedoraproject.org) -> closed
  0 free  0 open  4 done  0 failed

33125435 build (rawhide, /rpms/python-ogr.git:c298df5e540ba1d010366e102c1c75d4f5b0b0cc) completed successfully

That was rough, can't wait to do this with packit.

Let's do Fedora 30 now:

$ packit propose-update --dist-git-branch f30
INFO: Running 'anitya' versioneer
using "f30" dist-git branch
syncing ./python-ogr.spec
INFO: Downloading file from URL https://files.pythonhosted.org/packages/source/o/ogr/ogr-0.0.3.tar.gz
100%[=============================>]    17.95K  eta 00:00:00
downloaded archive: /tmp/tmpl5xxq22x/ogr-0.0.3.tar.gz
uploading to the lookaside cache
PR created: https://src.fedoraproject.org/rpms/python-ogr/pull-request/3

And so on...

Conclusion​

As you can see, packit is useful for us right away.

We'll be delighted if you try it out and let us know what you think.

Packit 0.2.0 is here! (2019-03-19)

Our sprint nears an end which means we have released a new version of packit - 0.2.0! You can expect a new release after every sprint (i.e. every 2 weeks).

The 0.2.0 version has a bunch of new features and improvements: you can find a complete list in the changelog. We also have a detailed documentation for all the workflows packit covers.

Let's get through what's new:

  1. We have decided to rename two keys in our config file so they are more descriptive. Old names still work but they are deprecated:
    • package_name → downstream_package_name
    • upstream_name → upstream_project_name
  2. You don't need to touch dist-git at all when getting your new upstream release into Fedora, you can stay in your upstream repository and just fire off a bunch of packit calls:
    • packit propose-update to create a pull request in Fedora dist-git with the selected upstream release
    • packit build to build the new upstream release once the pull request is merged
    • and finally, packit create-update creates a new bodhi update (if you chose a stable Fedora release)
  3. Packit now has a srpm command which creates an SRPM out of the local content of your upstream repository.
  4. You can now use packit to sync files from your dist-git repo back into upstream (mainly to keep spec files in sync). sync-from-downstream is the command.
  5. Command propose-update received numerous improvements:
    • You can pick upstream version to use.
    • Packit will NOT check out the git ref with the upstream release if you specify --local-content
    • It's possible to force packit to execute fedpkg new-sources using --force-new-sources and bypass the caching mechanism.

Installation​

Please make sure you are installing 0.2.0:

$ dnf install --enablerepo=updates-testing packit

Or

$ pip3 install --user packitos

You can also install packit from master branch, if you are brave enough:

$ pip3 install --user git+https://github.com/packit-service/packit

How are we using packit?​

I'd like to show you how we used packit to bring a new upstream release of ogr into Fedora, a library which packit is using.

Once we have performed an upstream release of ogr, we can propose an update in dist-git:

$ git clone https://github.com/packit-service/ogr && cd ogr/

$ packit propose-update
INFO: Running 'anitya' versioneer
Version in upstream registries is '0.0.3'.
Version in spec file is '0.0.3'.
Picking version of the latest release from the upstream registry over spec file.
Checking out upstream version 0.0.3
Using 'master' dist-git branch
Cloning repo: https://src.fedoraproject.org/rpms/python-ogr.git -> /tmp/tmpb9xlvdhj
Syncing /home/tt/g/user-cont/ogr/python-ogr.spec
Archive ogr-0.0.3.tar.gz found in lookaside cache (skipping upload).
ERROR    Cmd('git') failed due to: exit code(1)
  cmdline: git commit -s -m [packit] 0.0.3 upstream release -m Upstream tag: 0.0.3
Upstream commit: 059d21080a7849acff4626b6e0ec61830d537ac4

  stdout: 'On branch 0.0.3-master-update
nothing to commit, working tree clean'

Whoops, it seems that I have messed up, I forgot to bump the spec file in the upstream repo when doing the release. I will bump it locally and utilize --local-content argument:

$ rpmdev-bumpspec -n 0.1.0 -c 'New upstream release: 0.1.0' *.spec

$ packit propose-update --local-content
INFO: Running 'anitya' versioneer
Version in upstream registries is '0.0.3'.
Version in spec file is '0.1.0'.
Picking version of the latest release from the upstream registry over spec file.
Using 'master' dist-git branch
Cloning repo: https://src.fedoraproject.org/rpms/python-ogr.git -> /tmp/tmpd9j4se27
Syncing /home/tt/g/user-cont/ogr/python-ogr.spec
Archive ogr-0.1.0.tar.gz found in lookaside cache (skipping upload).
INFO: Downloading file from URL https://files.pythonhosted.org/packages/source/o/ogr/ogr-0.1.0.tar.gz
100%[=============================>]    20.25K  eta 00:00:00
Downloaded archive: '/tmp/tmpd9j4se27/ogr-0.1.0.tar.gz'
About to upload to lookaside cache
won't be doing kinit, no credentials provided
PR created: https://src.fedoraproject.org/rpms/python-ogr/pull-request/6

Once the scratch build is done and tests passed we merged and built it:

$ packit build
Using 'master' dist-git branch
Cloning repo: https://src.fedoraproject.org/rpms/python-ogr.git -> /tmp/tmprp3cmdjy
Building python-ogr-0.1.0-1.fc31 for rawhide
Created task: 33616980
Task info: https://koji.fedoraproject.org/koji/taskinfo?taskID=33616980

We have done the same for F30 and F29.

The previous commands were run in the directory of the upstream repository. Packit also accepts path to your upstream clone, or even URL. So let's create a bodhi update for python-ogr by specifying the upstream repo URL:

$ cd $HOME

$ packit create-update --dist-git-branch f29 https://github.com/packit-service/ogr
Cloning repo: https://github.com/packit-service/ogr -> /tmp/tmpdkdadmn_
Koji builds for package python-ogr and koji tag f29-updates-candidate:
 - python-ogr-0.1.0-1.fc29
Cloning repo: https://src.fedoraproject.org/rpms/python-ogr.git -> /tmp/tmpn1809ec9
Bodhi update FEDORA-2019-78948e62d2:
- https://bodhi.fedoraproject.org/updates/FEDORA-2019-78948e62d2
- stable_karma: 3
- unstable_karma: -3
- notes:
New upstream release: 0.1.0

And that's it, no need to access dist-git any more.

Please give packit a try and let us know what you think.

Packit 0.3.0 (2019-04-11)

In the previous post we promised to provide a new release every 2 weeks and we are already breaking this promise as it's been 3 weeks since then. We decided to wait with the release to merge several pull requests related to source-git support.

Now the good news. You can find a complete list of new features and improvements of version 0.3.0 in the changelog.

Features​

  • You can now specify your own hooks or actions to replace default packit behaviour. (More information can be found in the documentation).
  • Packit supports pagure.io-based upstream projects.
  • Commands propose-update and sync-from-downstream supports copying directories.
  • A new command status! It displays useful upstream/downstream info.
  • Packit now supports Source-git. The functionality is not available, yet - we will add a CLI interface for it in the next release.
  • You can now have a config file for packit in your home directory(~/.config/packit.yaml).
  • Packit installed from an RPM now has manpages.

packit status example​

$ packit status
Cloning repo: https://src.fedoraproject.org/rpms/packit.git -> /tmp/tmp84we_6n8
Downstream PRs: No open PRs.
f29: 0.2.0
f30: 0.2.0
master: 0.2.0

Packit 0.4.0 & 0.4.1 (2019-05-18)

It's been over a month since we released packit "0.3.0". Here comes packit 0.4.0 (and patch release 0.4.1) and as always they bring a lot of new features and improvements.

You can find a complete list in the changelog.

Packit as a service​

  • We have Packit as a service running in OpenShift and also a GitHub App, which uses it. Unfortunately it's still not in the Marketplace, so we have been the only one using it so far. The service/app submits builds in copr and once they're done it adds a GitHub status and comment with instructions how to install the builds. The service is now configurable via jobs defined in configuration file.
  • Packit is now able to check GPG signatures of the upstream commits against configured fingerprints.

CLI​

  • srpm command now works also with Source-git.
  • status command now access remote APIs asynchronously in parallel, which should speed up the execution.
  • CLI has new --dry-run option to not perform any remote changes (pull requests or comments).
  • Fedmsg parsing has been unified into a single listen-to-fedmsg command.

Packit 0.4.2 (2019-06-26)

Another relase after a month since 0.4.1, this time mostly with bug fixes.

We've been busy polishing our GitHub App recently, therefore we had no resources for new features.

See CHANGELOG for more details.

September 2020

Week 36 (August 31th - September 4th)​

Week 37 (September 7th - September 11th)​

  • Stage now uses Tokman to get access tokens for GitHub, which should resolve race condition when running parallel jobs (Tokman by Hunor, ogr integration by Matej).
  • Franta has addressed problems with Testing-Farm cluster with custom response on PRs that links to more information (pinned info, packit-service#798).

Week 38 (September 14th - September 18th)​

Week 39 (September 21st - September 25th)​

  • Packit-service can be configured to work with private namespaces. This is plumbing work which we need right now for CentOS Stream. We are not planning to enable this for GitHub - packit-service will still work only for public repositories, private ones are ignored. packit-service#831
  • If git tag contains more information than just version (e.g. pkg_name-v1.2.3), it is possible to use upstream_tag_template to extract version from the tag, which will be used in a subsequent task. doc packit#959
  • Added support for globbing pattern in upstream_ref. doc packit#960
  • Packit --remote is global option now and available for all commands. Because of this sync-from-downstream --remote was renamed to --remote-to-push. Remote can now be specified in the user's config (via upstream_git_remote parameter). packit#977

Following bugs were fixed:

  • Packit dropping leading zeros in version. packit#814
  • Packit CLI issue caused by picking incorrect copr project name. packit#971

Week 40 (September 28th - October 2nd)​

  • Packit-service is now explicitly checking if requested copr-build targets exist and if not, the user is informed about it. packit-service#835
  • We have improved the way how packit updates %setup line in a spec file - you are now able to set content of -n option via archive_root_dir_template config option, it defaults to {upstream-pkg-name}. doc packit#834
  • Packit is able to generate a patch file with format-patch without leading a/ and b/ in the patch diff. Required for patches in dist-git which are applied with -p0.
  • Contribution guidelines were updated, now we have one shared link.

Week 44-48 (November) 2019

With this blog post we'd like to continue with the idea of openly communicating changes in Packit. Since most of the developers use Packit as the GitHub App (which uses code from this repository - Packit Service), this blog will be about changes in all the parts, i.e. the GitHub App, the Packit Service and Packit itself.

Continuous Deployment (CD)​

At the moment the workflow is that one of us manually triggers production container image build at the end of a week. This image is then automatically deployed into our production instance of the service over a weekend (Sun/Mon night) so that everyone can start a week with all the amazing stuff we added the previous week. In case an issue makes it through our staging instance into production uncaught, we can easily rollback on Monday. The same person also writes down what's changed in Packit (service/app) since previous deployment.

Changes in production (since the end of November)​

Previous post is almost half a year old so we won't list all the changes since then, but only since last deployment, i.e. since end of December.

Packit​

Previous deployment was running packit-0.7.1. We haven't released a newer version since then, but in the service we install Packit from the Git repository (we have a separate stable git branch for our production deployment). From the most visible changes, Packit now:

  • better handles Create-archive action
  • is able to work in a repo with detached head
  • logs output from subprocesses in realtime
  • syncs config file and spec file by default in Propose-update action
  • hadles patches with undecodable chars

Packit Service​

Now:

  • better reports Copr builds
    • uses separate commit status for srpm build and every chroot
    • clears test farm commit statuses when new build is triggered
    • better handles failed Copr builds
  • gracefully handles no config file in the repo
  • better handles when no (copr build) targets are specified in config file
  • better checks whitelist of users
  • does not create duplicate tickets in our notification repo when a new user install the app

Changes not visible to end users:

  • using Requre for integration/E2E tests
  • Fedora messaging consumer part of the service has been improved and moved to separate repo/image
  • using FAS instead of Fedora Badges for checking whether a user is Fedora packager
  • many improved logs
  • many bugs squashed
  • lot's of code refactored

December 2019

Week 49​

ogr & packit​

  • ogr-0.9.0 has been released greatly restructured. (#291)
  • packit status (CLI) now shows also latest Copr builds. (#579)
  • Target aliases (currently fedora-development, fedora-stable, fedora-all) can now be used in the packit config file. (#619)
  • When doing a new update in Fedora dist-git, packit now by default creates a new pull request instead of pushing directly to dist-git. (#622)

packit service​

  • Does not set test checks when tests are not configured. (#275)
  • Supports target aliases and dist-git branches aliases. (#277, #285)
  • Nicely formats errors from OpenShift API. (#283)
  • Runs Copr build when user adds a /packit build comment into a PR. (#290)

Week 50​

packit​

  • If there is no upstream_package_name/downstream_package_name given in .packit.yaml, they now default to the name of the GitHub repo. (#624)
  • If no jobs are defined in .packit.yaml packit by default runs build job on fedora-stable targets and propose_downstream on fedora-all branches. (#625)
  • build command has nicer output. (#630)
  • Smaller fixes. (#630, #636)

packit service​

  • Creates a new issue when propose-update fails. (#300)
  • Better reports failed submitting of a Copr build. (#301)