[atomic-devel] RFC Upstream Docker Layered Image Build tooling

[Adam Miller <maxamillion fedoraproject org>]

Hello all,
    Something that is in the works in Fedora right now is the Docker
Layered Image Build Service[0], and we've been looking at using
OSBS[1][2] for this but the more we dig into it there is uncertainty
about the OpenShift[3] component being the best solution for a build
system. OpenShift is a great Platform as a Service(PaaS) cloud
environment that happens to feature the logic for scheduling container
builds, but that function exists to enable other functionality in the
PaaS. Meanwhile from an administrative perspective, we have to
run/maintain the entire PaaS to use an extremely tiny subset of it's
functionality.
    I spoke with KB from the CentOS team to try and get some
notes/pointers on OSBS because as I understood it, the CentOS team
already gone through a setup of the system before. As we spoke I
mentioned my concerns about the overhead and complexity of using a
full blown PaaS just for build scheduling and it turns out the CentOS
team shares those concerns. Also, the CentOS team largely just uses
atomic-reactor[4] on it's own outside of OSBS currently because of the
PaaS overhead (at this time, they are still using a version under the
previous name of "dock" but that's more or less just a side detail).
Using atomic-reactor as close to at face value as possible is
something that we on the Fedora Team are also interested in pursuing
because the atomic-reactor functionality is great for layered builds,
it's simple/concise, has a very clean code base, and it has a very
good python API.
    Largely as a side note, I attempted to dig into things a little
bit and found that there was originally a smaller in scope
builder/scheduler service being worked on called dbs[5][6][7] which is
based around Celery[8] but this project was left behind in favor of
using OpenShift for OSBS.
    All that being said, I wanted to start a discussion around the
topic to see if there is something that we could all work on
collectively upstream to target the very specific need of performing
these kinds of builds using atomic-reactor in a way that Fedora,
CentOS, and $others could benefit from using and collaborating on
contributions. Whether that be to investigate dbs and possibly
"resurrect" that work or fork (if necessary) to continue development,
or pursue something entirely new. The end goal would be that we could
collaborate on as much as possible and not re-invent build tooling for
the different distros. Also, it might be nice if this tooling could
build nulecule[9] apps.

Please let me know what your thoughts are on the topic. If there's
something I'm not thinking of or if the idea of removing OpenShift
from the build tooling seems completely crazy, please let me know why
because it's certainly possible I'm missing something and I'm always
open to new ideas.

Thank you,
-AdamM

[0] - https://fedoraproject.org/wiki/Changes/Layered_Docker_Image_Build_Service
[1] - https://github.com/projectatomic/osbs-client
[2] - https://github.com/projectatomic/ansible-osbs
[3] - https://github.com/openshift/origin
[4] - https://github.com/projectatomic/atomic-reactor
[5] - https://github.com/DBuildService/dbs-server
[6] - https://github.com/DBuildService/dbs-worker
[7] - https://github.com/DBuildService/dbs-client
[8] - http://www.celeryproject.org/
[9] - https://github.com/projectatomic/nulecule