Re: [atomic-devel] Kubeadm vs. SELinux

[Devan Goodwin <dgoodwin redhat com>]

On Wed, Nov 23, 2016 at 9:42 AM, Daniel J Walsh <dwalsh redhat com> wrote:
>
>
> On 11/22/2016 07:26 PM, Josh Berkus wrote:
>> On 11/22/2016 03:27 PM, Clayton Coleman wrote:
>>> Copying Devan as well since he's been working with kubeadm for a while.
>>>
>>>> On Nov 22, 2016, at 5:25 PM, Jason Brooks <jbrooks redhat com> wrote:
>>>>
>>>>> On Tue, Nov 22, 2016 at 2:38 PM, Daniel J Walsh <dwalsh redhat com> wrote:
>>>>>
>>>>>
>>>>>> On 11/22/2016 05:15 PM, Josh Berkus wrote:
>>>>>> Currently, it is not possible to run Kubeadm with SELinux enabled.
>>>>>>
>>>>>> This is bad; it means that Kubernetes' official installation
>>>>>> instructions include `setenforce 0`.  But it's hard to argue the point
>>>>>> when a kubeadm install -- soon to be the main install option for
>>>>>> Kubernetes, and the only one which currently works on Atomic -- simply
>>>>>> doesn't work with SELinux enabled.
>>>>>>
>>>>>> The current blocker is that kubeadm init will hang forever at this stage:
>>>>>>
>>>>>> <master/apiclient> created API client, waiting for the control plane to
>>>>>> become ready
>>>>>>
>>>>>>
>>>>>> The errors shown in the journal are here:
>>>>>>
>>>>>> https://gist.github.com/jberkus/4e926c76fbf772ffee4eb774cb0a4c60
>>>>>>
>>>>>> That's on Fedora 25 Atomic.  I've had the exact same experience on
>>>>>> CentOS 7 and RHEL 7, although the error messages are not identical.
>>>>>>
>>>>>> Seems like this is on us to fix, if we want people to keep SELinux
>>>>>> enforcing. I don;t know if we need to push patches to Kubeadm, or to
>>>>>> SELinux, or both.
>>>>>>
>>>>> What AVC's are you seeing?  Where is the bugzilla for this?
>>>>>
>>>>> ausearch -m avc -ts recent
>>>> https://paste.fedoraproject.org/488671/79856867/
>>>>
>>>> This is from a kubeadm that's packaged up in a copr:
>>>> https://copr.fedorainfracloud.org/coprs/jasonbrooks/kube-release/
>>>>
>>>> The kubernetes project provides rpms for centos and ubuntu, and there
>>>> are a few things about the way they pkg it that conflict w/ atomic.
>>>> Some more info at
>>>> https://jebpages.com/2016/11/01/installing-kubernetes-on-centos-atomic-host-with-kubeadm/.
>>>>
>> In addition to this, please note that setenforce 0 is not required on
>> the workers nodes, just on the master.  The kubelet nodes work fine with
>> just relabeling the /var/lib/kubelet directory.
>>
>> It would be really nice if we could somehow do that relabeling as part
>> of the installation package, but I don't see how; it would need to be a
>> patch/fork on kubeadm instead.
>>
>>
> Relabeling of /var/lib/kublet should not be necessary either.  What
> label is being applied?  What AVC's show up when it is
> not applied?
>
> Just setting random labels has the potential for opening security
> holes.   We need to get this directory labeled in such
> a way that not all containers can read/write it.


Is this relabeling only required on Atomic?