Re: [atomic-devel] Refspec used by "atomic host upgrade"

[Colin Walters <walters verbum org>]

On Thu, Jul 7, 2016, at 07:21 AM, Marius Vollmer wrote:
> Hi,
> 
> when creating our Atomic test images, we want to first run "atomic host
> upgrade" and then "ostree checkout $ref /var/local-tree".  We do this so
> that we can modify /var/local-tree via rpm and boot into it to run the
> tests.
>
> 
> What should $ref be so that /var/local-tree will be the same as what
> "atomic host upgrade" has just downloaded and deployed for the next
> boot?

One intent of ostree is to clearly and concisely define the base system.
If I say I'm running "fedora/24/x86_64/docker-host", that should be
unique - i.e. no one else should choose that name.  Now unlike
Docker there's no defacto centralized naming registry (in fact
ostree intentionally has a more distributed "remote" model), but anyways
the idea is the branches are unique and descriptive.

This design does make it more awkward to manipulate
the system locally, because conceptually then, the system *isn't*
"fedora/24/x86_64/docker-host".  Unlike docker, OSTree doesn't
natively have a model for derivation.  For rpm-ostree though,
the layered packages are clearly distinguished.

All of that philosophy aside, there are many ways to do this:

#1) Create a local webserver and archive repo with new updates
      and change the url= line in the /etc/ostree/remotes.d file

This one involves a bit more data movement because you'd have
to pull the data from the system into an "archive-z2" repository.

This is how the unit tests for rpm-ostree work:
https://github.com/projectatomic/rpm-ostree/blob/master/tests/common/libtest.sh#L303

#2)  Generate a local commit and *rebase* to it

The current rpm-ostree vagrant-based tests do something like this:

https://github.com/projectatomic/rpm-ostree/blob/master/tests/vmcheck/test.sh#L98

#3) Generate a local commit that overrides the origin remote

This is what you're trying to do now...but it's really battling
the system.  Here's how to generate a quick local commit that
appears to be like the remote:

ref=centos-atomic-continuous:centos-atomic-host/7/x86_64/devel/continuous
cd /sysroot/ostree/repo/tmp
ostree checkout --fsync=false ${ref} work
mkdir -p mnt
rofiles-fuse work mnt
echo somenewbinary > mnt/usr/bin/anewbinary
fusermount -u mnt
ostree commit --link-checkout-speedup -b ${ref} -s '' --tree=dir=work

But the problem here is that `atomic host upgrade` is going to talk
to the remote webserver.

Also, this will fail if the remote is configured for GPG verification.

> We used dig it out of /usr/share/rpm-ostree/treefile.json, giving

There is a DBus API for this, and we recently added `status --json`:
https://github.com/projectatomic/rpm-ostree/pull/315

> Now we need to prefix the remote as well, like this:
> 
>     ostree checkout \
>       rhel-atomic-host-ostree:rhel-atomic-host/7/x86_64/standard \
>       /var/local-tree
> 
> "atomic host upgrade" has a way to compute this refspec.  How can we
> reliably get the same value?

It's in the .origin file.

I think in the big picture, what you're trying to do here should be part of the
core model - with package layering we now cleanly support layering,
this is *overriding* - but in the end "it's just files", the challenge is mostly
around presentation in the command line etc.

For now though I'd look at #1 or #2.