On 13.11.2015 17:47, Matthew Miller wrote: > On Fri, Nov 13, 2015 at 10:41 AM, Amanda Carter <acarter redhat com> wrote: >> Hey folks, creating a separate thread for this longer term discussion. We're getting ready to release our first 2 week atomic update on Tuesday and Dusty Mabe has raised 2 potential release blockers that were not part of automated testing. It's good that he caught them, but it's also a bit of a stroke of luck. Since there is no official QE for this release, who should own verifying that there are no release blocking bugs prior to every automated release and escalating if there are? If no one raises the blocker, we'll have no way to block the release. > > Other than adding "making sure that anything found manually gets tests > added so it doesn't reoccur", I don't have much to add to this except > for saying that I also recognize the need. > > Paul Frields tells me that no one on his team (Fedora Infrastructure > Engineering) is in the right position to do this overall. That team -- > along with Fedora Release Engineering -- has been working on making > sure that the release artifacts get constructed, but aren't experts in > what goes _in_ them. (It's nice that we're at the point where the > problems are on the inside, I guess!) > > I think it needs to be someone who is deeply connected to the teams > working on Atomic development, but who is also given this is a core > responsibility and guaranteed the time for making sure things work in > Fedora specifically. > > I hope that over time we can grow community around this, but that's in > a chicken and egg situation. (I can elaborate on that if it's not > obvious to everyone...) The Cockpit team has automated tests that could feed into this. These run thousands of tests per day. Our tests run on top an operating system, rather than randomly components. That is, the Cockpit tests would currently run on top of an Atomic Host release. We do currently have one of our verify operating systems that pulls Fedora updates-testing and tests against that. We could have a verify machine that pulls in from a release candidate ostree repo and provide some testing that way. Obviously this has nothing to do with 'ownership' but might be helpful. Stef
Attachment:
signature.asc
Description: OpenPGP digital signature