[uss_quafilier] Set severity of Successful flight deletion to info during cleanup#1404
[uss_quafilier] Set severity of Successful flight deletion to info during cleanup#1404the-glu wants to merge 1 commit intointeruss:mainfrom
Conversation
|
|
||
| ## Cleanup | ||
| ### ⚠️ Successful flight deletion check | ||
| ### ℹ️ Successful flight deletion check |
There was a problem hiding this comment.
Since the requirement DeleteFlightSuccess specifies that a USS must remove a flight when instructed, I don't think Low severity is an appropriate categorization of this failure. A requirement is violated, but that violation doesn't warrant stopping the test scenario/cleanup -- that seems to match the description of Medium severity, so I don't think we want to change the severity of this check (or any of the others where a requirement is violated but we still want to continue test scenario/cleanup execution).
There was a problem hiding this comment.
Ok, but then for #1401, for "probably shouldn't fail regardless of {NotPlanned, Closed} in any situation since this is merely cleanup.", what should we do with fails?
Because it could be for multiple reasons (either dues to USS not failling the specs or something else), should we make the test smarter to differentiate both cases?
| ### [Restore virtual USS availability test step](../set_uss_available.md) | ||
|
|
||
| ### 🛑 Successful flight deletion check | ||
| ### ℹ️ Successful flight deletion check |
There was a problem hiding this comment.
While I agree High severity is inappropriate (we do not want to interrupt the cleanup), it seems like a requirement is violated so Low is inappropriate also -- it seems like Medium severity fits here.
There should be very few circumstances in which a check during cleanup should have High severity, as failing that check will prevent all the rest of the cleanup. |
This PR marks all flight deletion check during cleanup as informative, as failling the check may be considered informative, as described in #1401 .
Question: Should we perform the same changes for cleanup of others things, like OPs ?
Edit: This also contribute to #1400 to avoid stopping during flight cleanup