[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [atomic-devel] RFC Upstream Docker Layered Image Build tooling

When you say "run / maintain the entire PaaS", can you describe the portions of the PaaS that you don't want to run?  You need worker nodes (with Docker installed) to build docker images, an agent on each node (to manage the docker image builds), and a central server to have an API for starting / scheduling / running builds.  For a small deployment, that's what OpenShift is providing.  Is it concerns with SDN (which you don't need for builds), or the registry (which you don't need), or the router (which you don't need)?  I would hope that you would be able to run just the bits you need for image builds in OpenShift on their own.

I can certainly understand there are gaps in build functionality.  Part of the original goal here was to build something that would be useful to people who needed to build images in a farm, with a long term goal of being able to manage other types of workloads.  I don't think it's true that the build system exists only to serve the PaaS - we plan on using it for other things, like hosting builds securely for others.

Are you saying that long term, you don't believe that a generic compute service that happens to have very close ties to Docker, has integrated metering, usage, security, and quota, and is supported on a wide range of platforms, is *not* a good tool to use to build images?

On Wed, Jul 15, 2015 at 10:15 AM, Adam Miller <maxamillion fedoraproject org> wrote:
Hello all,
    Something that is in the works in Fedora right now is the Docker
Layered Image Build Service[0], and we've been looking at using
OSBS[1][2] for this but the more we dig into it there is uncertainty
about the OpenShift[3] component being the best solution for a build
system. OpenShift is a great Platform as a Service(PaaS) cloud
environment that happens to feature the logic for scheduling container
builds, but that function exists to enable other functionality in the
PaaS. Meanwhile from an administrative perspective, we have to
run/maintain the entire PaaS to use an extremely tiny subset of it's
    I spoke with KB from the CentOS team to try and get some
notes/pointers on OSBS because as I understood it, the CentOS team
already gone through a setup of the system before. As we spoke I
mentioned my concerns about the overhead and complexity of using a
full blown PaaS just for build scheduling and it turns out the CentOS
team shares those concerns. Also, the CentOS team largely just uses
atomic-reactor[4] on it's own outside of OSBS currently because of the
PaaS overhead (at this time, they are still using a version under the
previous name of "dock" but that's more or less just a side detail).
Using atomic-reactor as close to at face value as possible is
something that we on the Fedora Team are also interested in pursuing
because the atomic-reactor functionality is great for layered builds,
it's simple/concise, has a very clean code base, and it has a very
good python API.
    Largely as a side note, I attempted to dig into things a little
bit and found that there was originally a smaller in scope
builder/scheduler service being worked on called dbs[5][6][7] which is
based around Celery[8] but this project was left behind in favor of
using OpenShift for OSBS.
    All that being said, I wanted to start a discussion around the
topic to see if there is something that we could all work on
collectively upstream to target the very specific need of performing
these kinds of builds using atomic-reactor in a way that Fedora,
CentOS, and $others could benefit from using and collaborating on
contributions. Whether that be to investigate dbs and possibly
"resurrect" that work or fork (if necessary) to continue development,
or pursue something entirely new. The end goal would be that we could
collaborate on as much as possible and not re-invent build tooling for
the different distros. Also, it might be nice if this tooling could
build nulecule[9] apps.

Please let me know what your thoughts are on the topic. If there's
something I'm not thinking of or if the idea of removing OpenShift
from the build tooling seems completely crazy, please let me know why
because it's certainly possible I'm missing something and I'm always
open to new ideas.

Thank you,

[0] - https://fedoraproject.org/wiki/Changes/Layered_Docker_Image_Build_Service
[1] - https://github.com/projectatomic/osbs-client
[2] - https://github.com/projectatomic/ansible-osbs
[3] - https://github.com/openshift/origin
[4] - https://github.com/projectatomic/atomic-reactor
[5] - https://github.com/DBuildService/dbs-server
[6] - https://github.com/DBuildService/dbs-worker
[7] - https://github.com/DBuildService/dbs-client
[8] - http://www.celeryproject.org/
[9] - https://github.com/projectatomic/nulecule

Clayton Coleman | Lead Engineer, OpenShift

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]