Re: [atomic-devel] Extending Atomic Host and 'rpm-ostree pkg-add'

[Daniel J Walsh <dwalsh redhat com>]

Right it would be great if you and Giuseppe could work together on this,
for a prototype to handle using an rpm

package to track files put onto the system using `atomic install`

We should open a public google docs to allow people to comment on the
design.  I think both of you are doing

good work, but we could probably work faster if we collaborate.


On 11/23/2016 07:31 AM, Jakub Filak wrote:
> Giuseppe has opened a pull request that adds support for generating an rpm
> package from a system container and enhancing his code to install arbitrary
> files is just a matter of 3 lines in Python.
>
>
> On 11/22/2016 05:08 PM, Daniel J Walsh wrote:
>> Very nice.  Probably need some work on the Name field. 
>>
>> Do you have the example code?
> The code is available on github: https://github.com/jfilak/af
>
> It's a shell script and it's a bit dummy.
>
>> Does the rpm get built on the host and then installed. 
>>
> If the host have rpmbuild command, it does.
>
>> I would be nice if this could all be generated for the developer so the
>> developer
>>
>> does not need to do much to generate the rpm. 
>>
> Container developers just need to place their files to /exports/hostfs/ in
> the image and 'af install' can do the rest.
>
>> One interesting idea would be to add a postuninstall to the rpm of
>>
>> atomic uninstall whether
>>
> Add anything you need to %postun via an environment variable:
> https://github.com/jfilak/af/commit/fffd9d73429475930f3c6c3b889a80ed7ad33738
>
>> On 11/22/2016 09:25 AM, Jakub Filak wrote:
>>> 'rpm -qf /some/file/path' must either return a valid package name or exit
>>> with an error code. That's expected behavior of this command.
>>>
>>> 'rpm -qfi /some/file/path' prints out detailed description of the package
>>> that owns the file and this description can contain container details.
>>>
>>> Here is an example of description of an rpm package created by my script:
>>>
>>> Name        : C_wether___docker.io-fedora
>>> Version     : latest
>>> Release     : 0
>>> Architecture: noarch
>>> Install Date: Mon 14 Nov 2016 03:11:20 AM CET
>>> Group       : Unspecified
>>> Size        : 14
>>> License     : None
>>> Signature   : (none)
>>> Source RPM  : C_weter___docker.io-fedora-latest-0.src.rpm
>>> Build Date  : Mon 14 Nov 2016 03:11:20 AM CET
>>> Build Host  : 4d7d6e02109a
>>> Relocations : (not relocatable)
>>> URL         : https://github.com/jfilak/af
>>> Summary     : Host files from docker.io/fedora:latest
>>> Description :
>>> Files delivered by Docker container : wether
>>> The container was created from Docker image : docker.io/fedora:latest
>>>
>>> The package was created by these steps:
>>>
>>> sudo docker run -it --rm --name wether fedora sh
>>> mkdir -p /exports/hostfs/opt/filak/
>>> echo "Hello, world!" > /exports/hostfs/opt/filak/jakub.txt
>>>
>>> In another terminal:
>>> sudo ./af install --rpm wether
>>>
>>>
>>>
>>> On 11/18/2016 05:17 PM, Daniel J Walsh wrote:
>>>> We want admin to some how know that
>>>>
>>>> rpm -qf /etc/systemd/system/continer.service
>>>>
>>>> Was created by CONTAINER-FOOBAR.
>>>>
>>>> Having the container create an rpm on the fly that takes into case the
>>>> name of the container.
>>>>
>>>> On 11/18/2016 10:49 AM, Jakub Filak wrote:
>>>>> Yes, you are true about the chroot. I realized it a bit late and I didn't
>>>>> want to spent too much time on a proof-of-concept script.
>>>>>
>>>>> Could you please tell me more about the attributes you have on mind? I am
>>>>> afraid that I am caught in my use case and I cannot see anything beyond that.
>>>>>
>>>>>
>>>>> On 11/18/2016 04:24 PM, Daniel J Walsh wrote:
>>>>>> I think you should be able to do this totally with a chroot /host
>>>>>>
>>>>>> rather then nsenter.
>>>>>>
>>>>>>  A little trick I have been playing with for scripts executed in the chroot.
>>>>>>
>>>>>> Then you could just copy the rpm out of the container on to /host/run
>>>>>> and then execute
>>>>>>
>>>>>> the ./install.sh command to execute the appropriate rpm commands on the
>>>>>> host. 
>>>>>>
>>>>>> The difficult part is creating the rpm with attributes back to the
>>>>>> container. 
>>>>>>
>>>>>>
>>>>>> On 11/18/2016 09:26 AM, Jakub Filak wrote:
>>>>>>> The script was initially developed on Fedora Rawhide, so it works there.
>>>>>>>
>>>>>>> I had to add a couple of hacks to be able to run it on Atomic. Mainly
>>>>>>> because Atomic does not have rpm-build, which itself has tons of
>>>>>>> dependencies, and I decided to run the script in a container. Soon I found
>>>>>>> out that not only missing rpm-build is a problem but "rpm -i" does not work
>>>>>>> on Atomic. However I overcome this problem with a wrapper for rpm [0].
>>>>>>>
>>>>>>> The bottom line is that to make the script working on the current Atomic,
>>>>>>> you must install rpm-build (possibly in a privileged Fedora container that
>>>>>>> shares PID NS with the host) and issue the below command:
>>>>>>>
>>>>>>> $ PATH="./atomic-host:$PATH" ./af install --rpm <container_name>
>>>>>>>
>>>>>>> On Fedora you just need to run only:
>>>>>>> $ ./af install --rpm <container_name>
>>>>>>>
>>>>>>>
>>>>>>> Full example:
>>>>>>>
>>>>>>> [host] $ docker pull elcolio/etcd
>>>>>>> [host] $ docker run -d --name etcd elcolio/etcd
>>>>>>> [host] $ docker run --privileged --pid=host -it --rm fedora sh
>>>>>>> [cntr] $ dnf install rpm-build git
>>>>>>> [cntr] $ cd tmp && git clone https://github.com/jfilak/af && cd af
>>>>>>> [cntr] $ PATH="./atomic-host:$PATH" ./af install --rpm etcd
>>>>>>> [cntr] $ exit
>>>>>>> [host] $ rpm -qf /etc/etcd.conf
>>>>>>> C_etcd___docker.io_elcolio_etcd-latest.0.noarch
>>>>>>>
>>>>>>> For sake of simplicity, I assume that the /exports/hostfs/etc/etcd.conf file
>>>>>>> exists within the container.
>>>>>>>
>>>>>>>
>>>>>>> Jakub
>>>>>>>
>>>>>>> PS: The script is just a proof of concept that I created over night.
>>>>>>>
>>>>>>>
>>>>>>> 0: https://github.com/jfilak/af/blob/master/atomic-host/rpm
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 11/18/2016 01:52 PM, Daniel J Walsh wrote:
>>>>>>>> Seems like a simple fix.  rpm-ostree should be modified to support file
>>>>>>>> path rpms as well as
>>>>>>>>
>>>>>>>> rpm repositories.  But will this work on a traditional rpm based system
>>>>>>>> like RHEL or Fedora
>>>>>>>>
>>>>>>>> Workstation?
>>>>>>>>
>>>>>>>>
>>>>>>>> On 11/18/2016 03:35 AM, Jakub Filak wrote:
>>>>>>>>> I've been playing with privileged containers delivering services for D-Bus
>>>>>>>>> system bus. These D-Bus services must be enabled by a configuration file
>>>>>>>>> placed in the /etc/dbus-1/system.d/ directory. Therefore my containers must
>>>>>>>>> install files on Atomic host and this action creates system files not owned
>>>>>>>>> by any rpm package.
>>>>>>>>>
>>>>>>>>> Last week I wrote a script that creates an intermediate rpm package from
>>>>>>>>> files in the /exports/hostfs/ directory in a container and installs the
>>>>>>>>> package to a host [0]. Unfortunately, the script uses 'ostree admin unlock'
>>>>>>>>> before running 'rpm -i ...', so my changes to rpm database disappear with
>>>>>>>>> reboot. Using 'rpm-ostree pkg-add ...' could make the changes persistent but
>>>>>>>>> the command does not accept local files.
>>>>>>>>>
>>>>>>>>> My question is that will it be possible to install local rpm files via
>>>>>>>>> 'rpm-ostree pkg-add'?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Jakub
>>>>>>>>>
>>>>>>>>> 0: https://github.com/jfilak/af
>>>>>>>>>