Re: [atomic-devel] Storage use cases for the "atomic" command

[Vivek Goyal <vgoyal redhat com>]

On Fri, Apr 15, 2016 at 03:53:20PM +0300, Marius Vollmer wrote:
> Hi,
> 
> I am working on this, as you know:
> 
>     https://github.com/cockpit-project/cockpit/wiki/Atomic:-Docker-Storage
> 
> and after getting my misunderstandings of docker-storage-setup cleared
> up, my next idea is to change the "atomic" command to pretty directly
> support those use cases.
> 
> 
> We need two operations:
> 
>  - Take a block device and use it for storing Docker images and
>    containers.
> 
>  - Reset the docker storage setup so that block devices that were
>    previously used for storing Docker images and containers are
>    available again for other uses.

I will be very interested in two operations.


- Status of current configuration
- Reset the configuration.
	- Help users move from loop devices to proper block devices
	- Help user tear town existing thin pool and setup a fresh one.

Lot of people have run into issues with second item and anything we can
do to make user's life easier there, will help a lot.

> 
> We need the following information:
> 
>  - A list of block devices that are currently used to store Docker
>    images and containers.
> 
>  - A notification when that list changes.
> 
> (As a proper API, of course.)
> 
> 
> What about the following?
> 
>  - We implement "atomic container-storage add DEV" and "atomic
>    container-storage reset".
> 
>  - We make "atomic" maintain a file in /var with the status of the
>    Docker storage pool, in JSON, with atomic updates.
> 
> Clients like Cockpit can watch the status file with inotify or similar
> and get notifications that way.
> 
> The "add" operation would append the given block device to the DEVS line
> in /etc/sysconfig/docker-storage-setup" and run docker-storage-setup.
> 
> The "reset" operation would
>  - systemctl stop docker
>  - rm -rf /var/lib/docker
>  - set DEVS="" in /etc/sysconfig/docker-storage-setup
>  - run "docker-storage-setup reset"
> 
> With the "devicemapper" backend using a thin pool, "d-s-s reset" would
>  - rm /etc/sysconfig/docker-storage
>  - remove the thinpool
>  - vgreduce all pvs that are now completely empty

This last step is interesting. There are few things to think about.

- Disks might still be busy as they are in use by root lv or some other
  lv sharing same volume group as thin pool.

- We might have to call "pvremove" explicitly. Not sure if vgreduce will
  do it automatically.

- If disk is free, then we will have to remove partition also from disk
  otherwise when dss is run next time, it will complain that disk is
  already partitioned.

- Wipe all the signatures from the disk so that when dss is run next time
  it does not complain.

Lot of people want to just reset the docker metadata and thin pool and 
not necessarily free up disk. I am wondering if we should add one more
option to reset. Simple reset will just cleanup docker data and remove
thin pool and a deep reset will try to free up the disks and wipe
partition table and all signatures.

> 
> I don't yet know what "d-s-s reset" could do for other backends or a
> devicemapper with loopbacks.

For overlay, I think d-s-s reset will simply remove
/etc/sysconfig/docker-storage and that should be enough.

Thanks
Vivek