[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [atomic-devel] 3 kinds of "docker build"

On 17/10/14 14:33, Colin Walters wrote:
On Thu, Oct 16, 2014, at 12:42 PM, Sam Thursfield wrote:
Hi guys. First post here I think although I've been watching with
interest for a bit.


Thanks :)

This is (one of) the ideas behind Baserock[1]. Baserock considers
everything to be throwaway except the input source code, which is kept
in Git, and the build tool and build environment... which are generated
from source code kept in Git.

Yes, though when I was talking to Rob last weekend, I think lorry needs
to fix the submodule handling.  What gnome-continuous does here has
worked out quite well; it does recursive mirroring, and crucially
rewrites the submodule references to refer to the local mirror.  See:

OK. In Baserock we end up rewriting the .gitmodule file manually to have a keyed URL that points to a Git mirror server (e.g. [1]). Then we also set 'git config submodule.foo.url=xxx' on local checkouts.

It'd be nice to do this a better way :) the fact that we have a Git mirror *and* a local Git cache makes it a bit tricky, perhaps.

[1]: http://git.baserock.org/cgi-bin/cgit.cgi/delta/gstreamer.git/tree/.gitmodules?h=baserock/genivi/baseline

That's an interesting use case. Morph, the build tool Baserock uses,
could be adapted into being an OK tool for this situation. It has pretty
good caching of builds so if you build two systems that share a lot of
stuff, the shared things only build once (or not at all, if there were
no new commits).

Are caches here like packages, or more like bitbake's sstate cache?

What's a package? :)

If you have all the source code of a system, and construct a build graph from it, you can then use the cached artifacts to save yourself building things that were already built. Given a single cached artifact, you can't go back and construct a build graph from that.

Given a built system (i.e. the end result of a `morph build`) and all the input Git repos you can reconstruct the exact build graph that produced that system, so the cached build artifacts *are* a real cache ... but it may cost you a lot of time in rebuilding stuff if you empty it!

The main problem with Morph as a developer tool is that it's very
paranoid about build-dependencies. So to follow your Glance example, if
the developer hacks on python-jsonpatch, Morph will rebuild that from
source, but then also rebuild everything above it in the build graph.

Ther'es no one right answer here - it needs to be tunable.
gnome-continuous only rebuilds modules which changed, which has worked
out quite well for its intended use case, but we've certainly seen
breakage when e.g. vala changes and then only later is geary built or
something.  On the other hand you do at least need to periodically check
everything builds from source.

Yeah; I guess for continuous delivery you need both.

That's just source -> artifact policy, a whole other level is the
artifact -> Docker image policy.  Should the distribution's Docker base
images be updated for every RPM change, or only security?  And what
about layered images?  There's almost certainly going to need to be some
way to communicate that a new image build is security-sensitive, and
layers really should rebuild.

I guess if the images in the Docker Registry don't get updated often enough, people start to add 'yum update' to their Dockerfiles and Fedora's hosting bill increases. ...

I don't really think the Docker Registry or the whole "golden master" approach is a step forward, but we shall see!

Anyways so...circling back to my desire for some tool that lets me build
multiple containers with common artifacts, I was basically thinking of
something like "mockchain in Docker".  mockchain being:

This is actually one of the nicest things about COPR - you can upload
*multiple* sources and it will (eventually through brute force) build
them in a working order.

mockchain is a bit over my head, as an relative outsider to working on RPM-based distros. COPR does look incredibly handy if I ever need to publish my own Fedora package repository!


Sam Thursfield, Codethink Ltd.
Office telephone: +44 161 236 5575

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]