[atomic-devel] packages that are tested by a-h-t

[Dusty Mabe <dusty dustymabe com>]

The atomic-host-tests [1] is a repo of tests to run against atomic host.
We run them against ostrees and images that we produce.

What is the list of rpms that are explicitly tested by this set of tests?
Basically I want to report test results and give more rich information
about whether or not something broke.

For example, we explicitly test the docker package in a-h-t. If we run a-h-t
against a tree where there is a new docker rpm, then we can be fairly certain
we didn't break something (or if we did, we should add a new test for that).
So running a-h-t for a new docker package gives a pretty good indication of
if that package is good.

Another example is if we have a package that is broken (say vi) within atomic
host, but the a-h-t don't explicitly test it. For these packages i'd basically
like to report 'this update was not explicitly tested, but note that this update
did not break atomic host during testing'. Or something along those lines.

In order to do this I need a list of rpms that we can pretty confidently say are
"good" as a result of the test. For the rest of the rpms, we can't say they're good,
but we can say that we at least didn't detect that they were bad.

Dusty 

[1] https://github.com/projectatomic/atomic-host-tests