The idea is to read the file that is used by ExUnit to write failed tests, so next time when you open summary of tests, you can already see failed tests even though they were run from the CLI. https://github.com/elixir-lang/elixir/blob/927b10df80ee1c1c7396e68efe00d06bc3e80420/lib/mix/lib/mix/tasks/test.ex#L751-L767