Re: [atomic-devel] Running docker-storage-setup from a UI

[Vivek Goyal <vgoyal redhat com>]

On Wed, Apr 13, 2016 at 10:36:40AM +0300, Marius Vollmer wrote:
> Vivek Goyal <vgoyal redhat com> writes:
> 
> > On Tue, Apr 12, 2016 at 03:09:08PM +0300, Marius Vollmer wrote:
> >
> >> - Docker-storage-setup needs to run non-interactively, but I think it
> >>   can't do that right now, and asks confirmation for various things.
> >
> > It already runs non-interactively. It is a service which runs on boot.
> 
> Yes, I noticed that, too.  So any user-prompting from
> docker-storage-setup is a bug?  I get a prompt from pvcreate sometimes,
> for example when the random data on the new partition has a filesystem
> signature at the beginning.  With what I know now, this is probably
> unintended.  Should I file an issue?  Lvm likes to add confirmation
> prompts like this over time... :)

I think if it runs as service, then no terminal is attached and it
assumes a default answer. I guess yes. Give it a try.

> 
> Why does d-s-s run during boot, actually?  Shouldn't people run it
> interactively after changing /e/s/d-s-s?  Or does it only run on first
> boot?

It runs whenever docker is run. So if docker is configured to run on 
every boot, it will run every boot. There are multiple reasons to 
do that.

-  Get things right automatically during first boot.

  By default when we boot first time an image, we want thin pool to be
  setup before docker runs first time. On atomic images, we have free
  space in root volume group and this succeeds without any configuration
  required from user.

- Wait for thin pool to come up

  If user had configured a thin pool but thin pool is on a slow disk or
  if disk takes little longer to come up, now we have logic to wait for
  thin pool to come up before docker starts. Otherwise docker will fail
  right away saying thin pool does not exist.

- Automatically grow pvs backing volume group

  Apart from setting thin pool, dss also automatically grows pvs backing
  root volume group. And virtual size of a disk might have been increased
  since last reboot and dss will do it over next boot.

> 
> >>   Would it be acceptable to add a "--force-wipe" option to d-s-s, and
> >>   maybe others?  I can do that at the same time as I write the code for
> >>   the UI.
> >
> > So --force-wipe is a new functionality. What will it do?
> 
> It would not do any safety checks while turning a block device into a
> physical volume, with the assumption that the user has already
> independently confirmed that all devices in DEVS are ok to be used for
> the docker storage pool, no matter what they currently contain.
> 
> I now see that d-s-s refuses to touch a device that already has a
> partition table on it.  The --force-wipe option would override this and
> write a new one.
> 
> If you think that a --force-wipe option is not appropriate for d-s-s,
> Cockpit can wipe the devices itself.  But I think the option is nicer.

I don't know, wiping partitions is always risky by default. User might
lose important data. If cockpit can do it, that would be great. If there
is a strong demand for this, we can add it, but nobody has asked for it
yet.

> 
> >>   So I am thinking there could be something like
> >>
> >>     # docker-storage-setup status
> >>     /dev/vda: ok, shared with OS
> >>     /dev/sda: ok
> >>     /dev/sdb: Not yet set up!
> >>
> >>     The Docker storage pool is not fully set up.  Run
> >>     "docker-storage-setup" to complete the set up.
> >
> > I think first what would be nice is to display current docker
> > configuration.
> 
> Certainly.  I will work first on what I need, of course, but I am hoping
> that others chime in and extend the status output with other useful
> things.
> 
> > What does "shared with OS" mean? Does it mean, it is in same volume
> > group where rootfs is?
> 
> Yes, pretty much, but I am already changing my mind on the details.
> This will all be properly documented once I have working code.
> 
> 
> Thinking out loud:
> 
> I want to have three flags for each block device: "pool" if it holds any
> bits of the docker storage pool, "config" if it is listed in /e/s/d-s-s,
> and "other" if it holds any bits of other logical volumes.  That would
> allow the UI to show the correct current state based on the "pool" flag.
> The human readable output might highlight devices that are "config" but
> not "pool", but I am not sure yet what the UI will do about them.  Maybe
> nothing, maybe pre-select them in the "Additional storage" pane.

So all this current config detection can be easily done outside d-s-s
as well. I think it probably would be better if you run various lvm
commands to figure out volumes, thin-pools, pvs and display whatever
information is useful?

> 
> I think a reliable "other" flag will be interesting for a potential
> "docker-storage-setup reset" operation.  This would remove the current
> pool and create a new one, so that the user can meaningfully remove
> devices from DEVS again.

If user removes the thin pool, a new one will be created. "lvremove
docker-vg/docker-pool". I really don't see a need for a new command 
or option.

Don't want to make a simple bash script too complicated until and unless
it is really really needed.

Thanks
Vivek