Monitoring of Packit Service
To record Packit Service-related metrics from Celery tasks we are going to use
Prometheus Pushgateway which is
deployed in our cluster.
It can collect the metrics from the workers and provide the
/metrics endpoint for Prometheus.
There is a Prometheus instance running in OpenShift PSI, which is going to
/metrics endpoint and then it will be possible
to visualize them. Therefore the
/metrics endpoint needs to be publicly
accessible - it is exposed on https://workers.packit.dev/metrics for metrics
of the production instance and https://workers.stg.packit.dev/metrics
for metrics of the staging instance.
We use nginx (definition) to serve as a reverse
proxy for the pushgateway, which enables us to allow only
GET requests and
forward these to pushgateway (workers can send
POST requests internally).
To record celery related metrics from Celery tasks we are going to use Flower Prometheus Integration.