Skip to content

Draft: Track job failures correctly

Edmund Smith requested to merge eds/t37702a into main

Up until now, we have used Health::Complete to indicate job success in Gitlab pipelines, and Heath::Incomplete to indicate job failure. This is not in fact how LAVA uses these health values, because whether a job completed does not directly indicate whether the tests within it passed. An incomplete job will always be a failure, because it will have aborted. But a complete job may contain many (even all) test cases having failed, even if the job itself did not abort.

The actual logic for whether the job should be reported to Gitlab as failed is unfortunately slightly more complicated. We need to identify results messages within the log, and check whether any of them are failures. That means we need to make sure we check the logs, before we report any successful status to Gitlab.

The biggest change is that the monitor mode now outputs the logs (for all jobs) directly. There's no particular reason not to do this, since it has to query the logs anyway in order to accurately report. success or failure.

This MR is draft pending final testing of the submit command, and full testing of the monitor command.

Edited by Edmund Smith

Merge request reports