Re: [atomic-devel] 3 kinds of "docker build"

[Colin Walters <walters verbum org>]

On Thu, Oct 16, 2014, at 12:42 PM, Sam Thursfield wrote:
> Hi guys. First post here I think although I've been watching with 
> interest for a bit.

Welcome!

> This is (one of) the ideas behind Baserock[1]. Baserock considers 
> everything to be throwaway except the input source code, which is kept 
> in Git, and the build tool and build environment... which are generated 
> from source code kept in Git.

Yes, though when I was talking to Rob last weekend, I think lorry needs
to fix the submodule handling.  What gnome-continuous does here has
worked out quite well; it does recursive mirroring, and crucially
rewrites the submodule references to refer to the local mirror.  See:
https://git.gnome.org/browse/gnome-continuous/tree/src/js/vcs.js#n187

> That's an interesting use case. Morph, the build tool Baserock uses, 
> could be adapted into being an OK tool for this situation. It has pretty 
> good caching of builds so if you build two systems that share a lot of 
> stuff, the shared things only build once (or not at all, if there were 
> no new commits).

Are caches here like packages, or more like bitbake's sstate cache?

> The main problem with Morph as a developer tool is that it's very 
> paranoid about build-dependencies. So to follow your Glance example, if 
> the developer hacks on python-jsonpatch, Morph will rebuild that from 
> source, but then also rebuild everything above it in the build graph. 

Ther'es no one right answer here - it needs to be tunable. 
gnome-continuous only rebuilds modules which changed, which has worked
out quite well for its intended use case, but we've certainly seen
breakage when e.g. vala changes and then only later is geary built or
something.  On the other hand you do at least need to periodically check
everything builds from source.

That's just source -> artifact policy, a whole other level is the
artifact -> Docker image policy.  Should the distribution's Docker base
images be updated for every RPM change, or only security?  And what
about layered images?  There's almost certainly going to need to be some
way to communicate that a new image build is security-sensitive, and
layers really should rebuild.

Anyways so...circling back to my desire for some tool that lets me build
multiple containers with common artifacts, I was basically thinking of
something like "mockchain in Docker".  mockchain being:
http://skvidal.wordpress.com/2012/04/20/mockchain-use-cases-and-examples/

This is actually one of the nicest things about COPR - you can upload
*multiple* sources and it will (eventually through brute force) build
them in a working order.