Re: [atomic-devel] CI/CD thoughts

[Deshi Xiao <dxiao redhat com>]

Indeed, Atomic is new distribution then fedora and centos, I would like Atomic upstream can consider to create a unique CI build to deliver the image.


-- 
Xiao Deshi


----- Original Message -----
From: "Colin Walters" <walters verbum org>
To: atomic-devel projectatomic io
Sent: Tuesday, December 2, 2014 10:53:11 AM
Subject: [atomic-devel] CI/CD thoughts

https://bugzilla.redhat.com/show_bug.cgi?id=1169151 and a discussion about CI in CentOS had me thinking a bit today about this.

Currently Atomic is structured as part of the parent distribution.  For Fedora, we use Fedora's processes, for CentOS, CentOS's, etc.  The code is shared, and obviously these distributions share some tools (but not all).

Neither Fedora nor CentOS use a CI/CD model.  In my opinion, this is due to RPM (and the stack of things on top) - you can't do CI if you have no way to roll back to older versions.  Now with Atomic we're actually delivering software via Docker and OSTree, we have an opportunity to change that.  But that's a longer post.

So the vast majority of projects I've found address this by doing CI outside of the RPM process before the official builds.

Other projects like oVirt and OpenStack live outside of the distribution, and both have their own CI infrastructure.  http://jenkins.ovirt.org/ and https://jenkins.openstack.org/ respectively.

I think it's notable that oVirt upstream uses custom Jenkins jobs to make RPMs: http://jenkins.ovirt.org/job/ovirt-engine_master_create-rpms-quick_merged/label=el7/lastBuild/consoleFull

So, if we live inside the parent distribution, what resources are available?

CentOS resources:
 - A new CI system based on Jenkins, it does arbitrary stuff post-release: http://wiki.centos.org/QaWiki/PubHardware

Fedora resources:
 - Jenkins instance: http://jenkins.cloud.fedoraproject.org/ 
 - "Taskotron", basically buildbot running rpmlint and depchecks: https://taskotron.fedoraproject.org/

One thing that came up in the CentOS discussion was to ignore the RPM vs CI/CD problem and focus on testing of the RPMs post-build.  That approach would argue for a set of tests post-build.

A good example project for that is: https://github.com/autotest/autotest-docker
Which in turn depends on autotest, which has its own whole grid/scheduling/task system.   Not sure if there's any real intersection with Jenkins there or if it would basically need its own dedicated pool.

There is the path of Docker-in-Docker which brings massive amounts of flexibility, but I don't really want to spend too much energy on thatthere myself as IMO the main thing that needs coverage is Host->Docker.  Most of the things that are testable with DID are already tested upstream.