Re: [atomic-devel] Meeting item for 20170802: Atomic Host + Modularity - Enabling CI/CD

[Josh Boyer <jwboyer redhat com>]

On Thu, Aug 3, 2017 at 8:59 AM, Dusty Mabe <dusty dustymabe com> wrote:
>
>
> On 08/03/2017 08:32 AM, Josh Boyer wrote:
>> On Wed, Aug 2, 2017 at 11:15 AM, Dusty Mabe <dusty dustymabe com> wrote:
>>> Recently we talked about adding more CI/CD to the Atomic Host
>>> processes. Since we release every two weeks we'd like to automatically
>>> run tests on every change that goes in to Atomic Host and prevent
>>> it from going into Atomic Host if it causes test failures.
>>>
>>> This was suggested to the Atomic Working Group from the new CI Working
>>> Group [1] in the 2017-07-12 meeting [2]. Stef, from the CI working
>>> group also sent a mail to the cloud list about it [3].
>>>
>>> Here are some of the things we want to be able to do:
>>>
>>> 1  Have a rigorous definition (including specific versions, buildroot) of
>>>    what goes into an Atomic Host, including dependencies.
>>>
>>> 2  Triggering the CI/CD pipeline based on a change to definition
>>>    of what goes into Atomic Host.
>>>
>>> 3  A way to revert a package in the Atomic Host compose to an earlier version.
>>>
>>> 4  A place to store higher level tests along with rigorous definition
>>>    of Atomic Host, including them being versioned.
>>>
>>> 5  Landing multiple changes that need to land together to pass testing.
>>>
>>>
>>> Out of that list we think we require that:
>>>
>>> 1R The definition of what is composed into Atomic Host artifacts should
>>>    include specific versions of packages, and all dependencies included.
>>
>> Is this an output of or input to the compose?  I ask because
>> dependencies will change, so assuming they are static would be bad.
>
> I believe we want this to be an input, but an input that can be updated by
> bots. i.e. a new version of rpmxyz is available and a bot opens a PR to
> add it to the definition of atomic host and tests run on that PR. This is
> not set in stone, however. Definitely looking from input here, especially
> from people who know more about modularity.

Hm.  OK.  I can see that working if it's automated as you suggest.  I
guess I would say that the goal of a "rigorous" definition is still
met, it's just that the definition changes a lot.

>> They can also vary by architecture.
>
> This is something that was previously glossed over in our discussions. Thanks
> for bringing it up. Does modularity handle this at all today?

Good question, and I don't know the answer.  I suspect it would have
to, particularly for things like low-level modules where the packages
are close to the architecture sets.  Atomic Host itself is certainly
in that category.  The higher level modules (e.g. httpd) are likely
identical in content and RPM NVRs though.

>>> 2R The definition of what is composed in an Atomic Host should be stored
>>>    in a git repository so that changes can be detected easily. The
>>>    CI/CD pipeline can be triggered off of changes to this reposiroty.
>>
>> If the answer to the above question is "output of", then this item is
>> essentially a package manifest for everything in the compose, correct?
>
> Answer was 'input of'. We can debate on whether that is a good or a bad thing :)

I was going for the easier way out with "output of", but input isn't
inherently bad either.  I guess for CI/CD, it doesn't matter much
either way.  The important thing there is that the changes are made
available to the pipeline and it can act on those at the appropriate
time.

josh

>>> 3R A mechanism to make a future composed Atomic Host artifact, contain
>>>    an earlier (in RPM NVR parlance) version of a package.
>>>
>>> 4R The high level functional Atomic Host tests should live in the same
>>>    git repository with the rigorous definition of what goes into an Atomic
>>>    Host.
>>>
>>> 5R A mechanism to tell the CI Pipeline that multiple dist-git repository
>>>    changes (i.e. multiple changing RPMs) should be built and tested together.
>>>
>>> We believe that moving to an Atomic Host defined by a module [4] can
>>> satisfy our needs and enable the improved control of artifacts within
>>> Atomic Host as well as the improved triggering and coupling with a
>>> testing pipeline that will help us move forward. Since modularity is new
>>> this will take some experimentation to prove out. Let's chat about this
>>> in the Atomic Working Group meeting today at 1700UTC in #fedora-meeting-1.
>>>
>>> Dusty
>>>
>>>
>>> [1] https://fedoraproject.org/wiki/CI
>>> [2] https://meetbot.fedoraproject.org/fedora-meeting-1/2017-07-12/fedora_atomic_wg.2017-07-12-17.00.log.html
>>> [3] https://lists.fedoraproject.org/archives/list/cloud lists fedoraproject org/message/ZS7XY7NHXGGOPOB2YKNQUWSDUGCMYIL5/
>>> [4] https://docs.pagure.org/modularity/
>>>
>> _______________________________________________
>> Atomic Working Group mailing list -- atomic lists fedoraproject org
>> To unsubscribe send an email to atomic-leave lists fedoraproject org
>>