Re: [atomic-devel] looking for feedback on running kubernetes in system containers

[Spyros Trigazis <strigazi gmail com>]

Hello,

I'm working on using system containers for our kubernetes configuration.

What I found a little strange in a solution like [1], is that we don't specify
the version of the component. eg if i do atomic install --system jasonbrooks/kubernetes-kubelet:rawhide
I don't control and I don't know the kubelet version.

If we move to system containers, which containers we need to use?
Is ti recommended to maintain our own container images based on [2]?

Cheers,
Spyros

[1] http://www.projectatomic.io/blog/2017/05/testing-system-containerized-kubeadm/

[2] https://github.com/projectatomic/atomic-system-containers

   

On 4 May 2017 at 15:53, Spyros Trigazis <strigazi gmail com> wrote:

Hi Jason,

I tried etcd from your branch [2] and with a single change [1], the etcd
system container was a drop-in replacement instead of using etcd
from the package.

Cheers,
Spyros

[1] I needed to set ETCD_INITIAL_CLUSTER="" since we use
the option ETCD_DISCOVERY and they conflict.
ETCD_INITIAL_CLUSTER is coming from [3]. Maybe we shouldn't
set all these options by default.
[2] https://github.com/jasonbrooks/atomic-system-containers/tree/kube-containers commit f99eaaa7a4a74595bd1be23c470d4ab1dd1a66dd

[3] https://github.com/jasonbrooks/atomic-system-containers/blob/kube-containers/etcd/etcd-env.sh#L13

On 3 May 2017 at 10:47, Giuseppe Scrivano <gscrivan redhat com> wrote:

Hi Jason,

Jason Brooks <jbrooks redhat com> writes:

> I've experimented w/ making more changes to the ansible like these --
> adapting the scripts to the system containers rather than the reverse,
> but I started thinking it'd be easier to adapt the system containers
> to be more of a drop-in replacement, leaving them to be configured as
> much like the regular packages as possible. So, things like making
> etcd configurable by editing a conf file vs. limiting configuration to
> --set commands. Do you think it's worthwhile to try and make system
> containers work this way, or would we be losing out on some system
> containers goodness through this?

yes, this is something that was requested by different users and it
makes sense to change the etcd container to support reading its
configuration from a file as well.

Now that we support copy of files to the host, we can create this file
at installation time and still allow multiple instances of the etcd
container.

It could be placed on the host accordingly to the container name like:
/etc/$NAME/etcd.conf.

The Docker container is already using this feature:

https://github.com/projectatomic/atomic-system-containers/commit/316a5c7587648ae431b6d34255454d407063844b

And for renaming the files (to take into account the container name):

https://github.com/ashcrow/atomic-system-containers/blob/e15222e1240131458d08b5dd85a73ffb1652b79c/docker-centos/manifest.json

For etcd we will need to create the bind mount as well.  In the Docker
case it wasn't needed since we are already bind mounting /etc.

Giuseppe