[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [atomic-devel] atomic-reactor and building in a container

> So the filesystem of final images would be in `/dest` and TAGGED_SWITCH_ROOT, would do the final commit?

no, please refer to this for details


I don't know how I forget to give you the link before

the syntax is SWITCH_ROOT <other_image> <root1> <root2> ...
where <other_image> can be scratch, busybox, fedora, ..etc.
it will be used as the filesystem root (preserving its layers)
but <root1> <root2>... will be copied to / in a new layer and commit

let me give you an example 

FROM fedora
RUN yum install busybox && mkdir -p /dest/{bin,sbin}/ && cp /sbin/busybox /dest/sbin/
SWITCH_ROOT scratch /dest/
RUN /sbin/busybox --install -s /
# or as in < https://github.com/fedora-cloud/Fedora-Dockerfiles/blob/master/busybox/makeimage.sh
# RUN for i in $(/sbin/busybox --list);do ln -s /sbin/busybox /bin/$i; done 
##### END

docker build -t myfedora/busybox .

the final image myfedora/busybox would just have fedora's busybox because it was from scratch

another example when you want to eliminate compilers and unnecessary layers ..etc 

FROM fedora
RUN yum install -y git nodejs golang 
RUN mkdir -p /dest/{bin,data}/ && \
  git clone myUrl myproj && cd myproj && \
  golang build -o /dest/bin/go-foo && \
  cp data /dest/data/
SWITCH_ROOT busybox /dest/
##### END

docker build -t myfedora/go-foo .

would be a very small busybox with my statically linked go-foo

you can replace "SWITCH_ROOT busybox /dest/" with "SWITCH_ROOT fedora /dest/"
in case of python/nodejs where having a proper dynamically linked base system is a better option

the result in would be adding an extra layer over the already fetched/cached base image (busybox/fedora)

> ideally come up with a solution (and implementation).

the implementation is quite simple we start with a building container and a working container both initially refer to the same thing
with first "SWITCH_ROOT" (or if we change it to "PIVOT_ROOT") we start a new container from <other_image> which will be the working container (leaving the build container as is)
let's say we start a container from busybox image, then we use "docker cp" to copy the <root1> <root2> .. from build container to working container then we commit and cache
all next run/adds/copies are in working container then when done we commit and tag the working container and saves its image

we can have multiple images from same docker file using "TAGGED_SWITCH_ROOT" (or if we change it to "TAGGED_PIVOT_ROOT")

TAGGED_SWITCH_ROOT <tag-suffix> <other_image> <root1> <root2> ...

will do the same except when done it will commit and tag with extra "-<tag-suffix>" after the passed tag, for example

docker build -t myfedora/go-foo .

might result in myfedora/go-foo (from SWITCH_ROOT) and myfedora/go-foo-server (from "TAGGED_SWITCH_ROOT server ...")

note: there always be at most 2  containers, the initial build container and the current active working container,
in case of multiple TAGGED_SWITCH_ROOT <tag-suffix> <other_image> <root1>
the source of <root1> will always refer to the initial build container (not the previously active working container)
the builder has to keep track of at most 2 containers during the build of the docker file no matter how many images it will build.

please tell me what you like more PIVOT or SWITCH

On Thu, Sep 3, 2015 at 11:07 AM, Tomas Tomecek <ttomecek redhat com> wrote:
Quoting Muayyad AlSadi (2015-08-28 14:00:33)
> quote from https://github.com/projectatomic/atomic-reactor
> > Features: build inside a docker container (so your builds are separated
> between each other)

The main reason behind this is that we run atomic-reactor in openshift using the
custom build strategy. Not sure if this new method would require changes in

> I was consulting with upstream docker and found they have a milestone to
> separate builder outside of docker, ie "docker build" at some point would
> an external app would build an image and pass it to docker daemon to load
> it or push it docker registry

This is what they announced on last Dockercon US.

> they recently introduced CP api that allow docker client to copy files
> between containers/host
> https://docs.docker.com/reference/commandline/cp/
> one of those builders is dockramp
> https://github.com/jlhawn/dockramp

I tried it a couple days ago, unfortunately got tracebacks only.

Other than that, it looks really promising.

> and there we are discussing how to do something similar to atomic-reactor,
> but using extended Dockerfiles only
> we are discussing a way to pivot/switch containers root in away similar to
> rpm's spec-file, the docker file is a usual dockerfile that starts with a
> build image (or stock image followed by yum install gcc golang ...)
> then do the build then switch the root to a minimal/base image (without
> compiler) that is sufficient to run it, with something like this
> RUN make install SERVER_PREFIX=/dest/server CLIENT_PREFIX=/dest/client
> RUN cp /bin/server-extra /dest/server/bin/
> RUN cp /bin/server-replicator /dest/replicator/bin/
> RUN cp /bin/client-extra /dest/client/bin/
> SWITCH_ROOT busybox /dest/server
> TAGGED_SWITCH_ROOT client busybox /dest/client
> TAGGED_SWITCH_ROOT replicator busybox /dest/replicator
> it would build 3 images: foobar, foobar-client, foobar-replicator

So the filesystem of final images would be in `/dest` and TAGGED_SWITCH_ROOT
would do the final commit?

I think I saw a couple of similar ideas in docker's issue tracker.

If you could provide more detailed description of the design, we can talk and
ideally come up with a solution (and implementation).

Thank you for your proposal!

Tomáš Tomeček
Software Engineer
Developer Experience

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]