Re: [atomic-devel] 3 kinds of "docker build"

[Sam Thursfield <sam thursfield codethink co uk>]

Hi guys. First post here I think although I've been watching with 
interest for a bit.

On 16/10/14 14:10, Colin Walters wrote:
> One thing I find strange about STI myself is that the unit of
> interaction is the Docker image, and the source code is an external
> thing you supply.
>
> To me, Docker images are *caches* of source code built in a specific
> configuration.  Source code is git repositories, pinned to specific
> revisions.

This is (one of) the ideas behind Baserock[1]. Baserock considers 
everything to be throwaway except the input source code, which is kept 
in Git, and the build tool and build environment... which are generated 
from source code kept in Git.

[1]: http://wiki.baserock.org -- feel free to ask for clarification 
about things on the website that don't make sense, I'm aware it's not 
great for "I have no idea what this thing is" readers right now.

...

> While I understand the rationale behind STI's focus on the
> build-one-thing, a case I was hitting is this:
>
> I have two Docker images (apps I'm working on), with a common set of
> packages shared between them, both deriving from a well-known framework
> (base image).  The common set here might include say a configuration or
> utility library I wrote.  If I'm working on one of my apps and fix
> something in the utility library, I want to potentially rebuild/retest
> both images.

That's an interesting use case. Morph, the build tool Baserock uses, 
could be adapted into being an OK tool for this situation. It has pretty 
good caching of builds so if you build two systems that share a lot of 
stuff, the shared things only build once (or not at all, if there were 
no new commits).

There is a work-in-progress extension to allow Morph to deploy to 
Docker. If someone was interested in trying it out I'd be game for 
finishing it off :)

> Here's a real-world example:
> https://github.com/stackforge/kolla/blob/master/docker/fedora-rdo-base/Dockerfile
>
> Now that Docker file is just an *assembly*, whereas I as an app author
> will want to build from source.  Say the author of Glance also needs to
> update "python-jsonpatch" (to pick a random example).  Clearly in the
> glance Dockerfile they could test this by just doing a yum install of
> their custom RPM, but I think a more intelligent build system would be
> capable of building the python-jsonpatch from *source*, in addition to
> taking a custom RPM.

The main problem with Morph as a developer tool is that it's very 
paranoid about build-dependencies. So to follow your Glance example, if 
the developer hacks on python-jsonpatch, Morph will rebuild that from 
source, but then also rebuild everything above it in the build graph. 
That said ... for Python packages rebuilds are not at all slow. But if 
there are big compiled projects in the build graph this approach doesn't 
really fit a developer workflow.

Sam


--
Sam Thursfield, Codethink Ltd.
Office telephone: +44 161 236 5575