Re: [atomic-devel] Kubernetes manual setup

[Chris Negus <cnegus redhat com>]

I started writing the content for Kubernetes on Fedora, based on all of your comments. Here's the approach I'm taking:

* Documenting kubeadm approach. It seems to work nicely as Jason describes.
* Pointing to minikube as an alternative vanilla Kubernetes setup for Fedora.
* Pointing to minishift, openshift-ansible, "oc cluster up" as alternatives for trying Kubernetes via OpenShift
* Demonstrating both Fedora and Fedora Atomic, stressing the value of Atomic for deploying containers.

Speak up if you think this is wrong. I'll share the doc once it's a bit more solid so everyone can have at it.

Colin, we're cutting the manual Kube setup (using system containers) from RHEL docs this coming release. Would it still be of value to showcase an example using Kube from system containers for the upstream case?

-- Chris Negus


----- Original Message -----
> On Wed, Feb 21, 2018, at 12:34 PM, Chris Negus wrote:
> 
> > In my mind, this means that someone trying out vanilla Kubernetes will
> > start with some OS outside of the Fedora/RHEL/CentOS ecosystem. My
> > question is, is it okay to let this content die? Or should we encourage
> > some way to still manually use Kubernetes on Fedora (Atomic or not)?
> 
> This is a pretty deep question with a lot of implications and history
> involved.
> Let me start by stating something that's fairly obvious: Kubernetes is pretty
> popular!  There are multiple "distros" of Kubernetes now, including now *two*
> products owned by Red Hat.
> 
> My personal opinion (now) is that we are primarily producing a base operating
> system; we need to
> support these different Kubernetes distributions; as well as non-Kubernetes
> cases.
> Including Kubernetes directly in AH was clearly a historical mistake, one
> that
> we are just finally digging ourselves out from.  Part of the issue here is
> that
> in Atomic we are introducing *two* entirely new ways to deliver software;
> via system containers as well as package layering now.  And Kubernetes could
> be done via both.
> In practice, I think system containers for Kubernetes make a lot of sense
> because
> then you get a natural decoupling of the base OS from your container service;
> they
> live at different cadences.  But it's been a challenge because it's quite
> different
> from all of the other ways to run Kube.
> 
> Kubernetes *also* is an excellent use case for the Fedora "Modularity" effort
> - having
> just one version of Kubernetes included in the "Everything" repository
> doesn't make
> sense when upstream supports multiple.
> The way I would describe this then is that we are providing technology that
> is intended
> to support this use case, and we also need to be participating in issues that
> cross
> the OS/Kubernetes boundary (for example: SELinux policy, host update
> management).
> So I think our current efforts at having system containers for upstream
> Kubernetes maintained by
> "us" are indeed the right approach - we need to ensure the basics work.
> Ideally
> of course we have more people gaining traction/buy-in in the upstream
> Kubernetes community.
> In practice I think a whole lot of that is mostly "non-Atomic" CentOS/RHEL
> using RPMs
> or just plain dropping binaries in /opt - hopefully we can get some of the
> upstream
> community using the technologies we've developed here.  But it's tricky
> because
> we have other Kubernetes distributions like OpenShift which are also a
> primary use case.
> 
> Anyways another important thing here is: we need to *clearly* distinguish the
> "dev/test"
> scenario from the "production" use case.  For the "dev/test" case there's
> minikube/minishift
> as well as `oc cluster up` which I use a lot (and a major bonus of using
> Linux as
> a desktop including Fedora Atomic Workstation is that you can `oc cluster up`
> directly
> on metal and avoid virt overhead).
> 
>