Re: [atomic-devel] How to handle crashes

[Jakub Filak <jfilak redhat com>]

Derek,

see my answers inline:

On 10/24/2016 08:02 PM, Derek Carr wrote:
> Hi Jakub,
> 
> I subscribed to your issue upstream, apologies for missing your earlier notes. 
> 
> Is there an exhaustive list of things that ABRT can detect that is documented?

We have this list:
http://abrt.readthedocs.io/en/latest/supported_langs.html

> 
> The document shows Linux kernel items, but they do overlap with what
> NodeProblemDetector has at this point.

They do overlap, but ABRT's list of known problems is bigger.
https://github.com/abrt/abrt/blob/master/src/lib/kernel.c#L84

Hmm, maybe I should open a pull request in node-problem-detector with a
patches updating their problem patterns list:

https://github.com/kubernetes/node-problem-detector/blob/master/config/kernel-monitor.json

> 
> I am not sure if I have a use case for the language specific add-ons as this
> would run on the actual node, and the language frameworks running end-user
> applications would be in the node, and problems with them don't reflect
> problems with the node per se.

Make sense. ABRT can be configured to detect problems on the node and in the
end-user containers and report only the problems on the node to
node-problem-detector.

Do uncaught Python exceptions or C/C++ core files appear on the node too?

> Is there any Go specific support?

Not yet, https://github.com/abrt/abrt/issues/1189

However, I'm not sure if it's even possible to detect go panics on OS level.
It wasn't possible in Ruby, but it is possible now ;)


Regards,
Jakub

> 
> Thanks,
> Derek
> 
> On Mon, Oct 24, 2016 at 7:07 AM, Jakub Filak <jfilak redhat com
> <mailto:jfilak redhat com>> wrote:
> 
>     I've asked node-problem-detector upstream for help on engaging ABRT in
>     node-problem-detector:
>     https://github.com/kubernetes/node-problem-detector/issues/35
>     <https://github.com/kubernetes/node-problem-detector/issues/35>
> 
> 
>     On 10/21/2016 09:35 AM, Jakub Filak wrote:
>     > I've created a Docker file that produces an image with ABRT configured to
>     > detect Kernel oopses in systemd-journal, vmcores on host and registers
>     > /proc/sys/kernel/core_pattern to detect core files:
>     > https://github.com/jfilak/docker-abrt/tree/atomic_minimal
>     <https://github.com/jfilak/docker-abrt/tree/atomic_minimal>
>     >
>     > Detecting those problems is not a rocket science. However, with ABRT
>     > employed you don't need to take care about this part and you can focus on
>     > propagation of events to appropriate destinations.
>     >
>     > Dominika, how can we teach ABRT to report the detected problems to
>     > NodeProblemDetector?
>     >
>     > Is there a command that ABRT can execute or should we connect to a TCP
>     port?
>     >
>     >
>     > Regards,
>     > Jakub
>     >
>     >
>     > On 09/15/2016 02:34 AM, Derek Carr wrote:
>     >> Dominika has been looking into node problem detector on our team, the
>     issue
>     >> we have found is while we like how it can report NodeConditions back into
>     >> cluster state, it's current kernel monitoring support is insufficient
>     >> until https://github.com/kubernetes/node-problem-detector/issues/14
>     <https://github.com/kubernetes/node-problem-detector/issues/14>
>     >>
>     >> It would be neat if we can plug something more intelligent into the
>     current
>     >> framework that could do more.
>     >>
>     >> On Wednesday, September 14, 2016, Jeremy Eder <jeder redhat com
>     <mailto:jeder redhat com>
>     >> <mailto:jeder redhat com <mailto:jeder redhat com>>> wrote:
>     >>
>     >>     Anyone know?  There's a node-problem-detector proposed in
>     Kubernetes but
>     >>     ... abrt is far more comprehensive.
>     >>     https://github.com/kubernetes/node-problem-detector
>     <https://github.com/kubernetes/node-problem-detector>
>     >>     <https://github.com/kubernetes/node-problem-detector
>     <https://github.com/kubernetes/node-problem-detector>>
>     >>
>     >>     The difference is that node-problem-detector has hooks to call
>     back to
>     >>     the kubernetes control plane to inform it that a node has problems.
>     >>     We could create an abrt container that does the same for RH-based
>     ecosystem.
>     >>
>     >>     On Fri, Sep 9, 2016 at 11:21 AM, Jeremy Eder <jeder redhat com
>     <mailto:jeder redhat com>
>     >>     <javascript:_e(%7B%7D,'cvml','jeder redhat com
>     <mailto:jeder redhat com>');>> wrote:
>     >>
>     >>         Hmm, appears this was not integrated into Fedora Atomic?  Is
>     there a
>     >>         plan to do so?
>     >>
>     >>         On Fri, Mar 20, 2015 at 5:50 AM, Jakub Filak
>     <jfilak redhat com <mailto:jfilak redhat com>
>     >>         <javascript:_e(%7B%7D,'cvml','jfilak redhat com
>     <mailto:jfilak redhat com>');>> wrote:
>     >>
>     >>             __
>     >>
>     >>             Hello,
>     >>
>     >>
>     >>
>     >>             I've been working on integration of ABRT with Project Atomic
>     >>             and, today, my work landed in Fedora 22 [1].
>     >>
>     >>
>     >>
>     >>             To enable abrt core_dump helper on Atomic hosts, it is
>     necessary
>     >>             to install abrt-atomic package and enable
>     abrt-coredump-helper
>     >>             service. After doing so core dump files will be stored in
>     >>             sub-directories of /var/tmp/abrt/.
>     >>
>     >>
>     >>
>     >>             You can find more technical details here:
>     >>
>     >>           
>      https://github.com/abrt/abrt/wiki/Containers-and-chroots#abrt---project-atomic
>     <https://github.com/abrt/abrt/wiki/Containers-and-chroots#abrt---project-atomic>
>     >>           
>      <https://github.com/abrt/abrt/wiki/Containers-and-chroots#abrt---project-atomic
>     <https://github.com/abrt/abrt/wiki/Containers-and-chroots#abrt---project-atomic>>
>     >>
>     >>
>     >>
>     >>
>     >>
>     >>             Should I write a new proposal for the oversight repository or
>     >>             should I just open a new pull request for fedora-atomic
>     repository?
>     >>
>     >>
>     >>
>     >>
>     >>
>     >>
>     >>
>     >>             Regards,
>     >>
>     >>             Jakub
>     >>
>     >>
>     >>
>     >>
>     >>
>     >>             1:
>     >>           
>      https://admin.fedoraproject.org/updates/gnome-abrt-1.1.0-1.fc22,abrt-2.5.0-2.fc22,libreport-2.5.0-1.fc22
>     <https://admin.fedoraproject.org/updates/gnome-abrt-1.1.0-1.fc22,abrt-2.5.0-2.fc22,libreport-2.5.0-1.fc22>
>     >>           
>      <https://admin.fedoraproject.org/updates/gnome-abrt-1.1.0-1.fc22,abrt-2.5.0-2.fc22,libreport-2.5.0-1.fc22
>     <https://admin.fedoraproject.org/updates/gnome-abrt-1.1.0-1.fc22,abrt-2.5.0-2.fc22,libreport-2.5.0-1.fc22>>
>     >>
>     >>
>     >>
>     >>
>     >>         --
>     >>
>     >>         -- Jeremy Eder
>     >>
>     >>
>     >>
>     >>
>     >>     --
>     >>
>     >>     -- Jeremy Eder
>     >>
>     >
> 
>