[REPORT]Persist report and run-log files to an external space after all tests executions

[nirmalchandra]

The Problem

In many projects, Jenkins-slaves, containerized environments, or VMs (like those used in GitLab and GitHub) are the backbone of CI/CD pipelines. These environments run the critical jobs, but there's a catch: the test result reports often aren’t saved for long-term reference, and access is restricted or controlled, leaving reports inaccessible. Sometimes, even the target folder (where JUnit reports and .class files are generated) is out of reach.

These reports—ranging from CSV files to HTML-Dashboard summary file and log files—are crucial for investigating test failures. But with limited access or short retention, developers and SDETs struggle to conduct thorough analyses. The inability to easily retrieve evidence hinders the ability to back up test failures and diagnose root causes, making the process more frustrating and less efficient.

Solution:

Implement a mechanism to allow configurable external space to save/persist the reports(csv, html and log files).

Example 1:
Use an external git repo "test-reports" to push the test-reports to this repo.

config properties entry:

reports.repo=https://github.com/authorjapps/test-reports
reports.repo.token:<use an authentication token>
reports.repo.max.upload.limit.mb=2   #<----- can be configured for more or less if needed

AC1:
Limit the "upload" to 2mb max of each file by default, unless configured for different size via reports.repo.max.upload.limit.mb

AC2:
If due to some reason, the upload did not succeed, then do not fail the build,
instead log an Error message (WARN level) e.g. "Something went wrong while pushing reports files to the external storage. Please fix the connectivity and rerun later. You can safely ignore this failure message as it had no effect on the CI build".

AC3:
Push only the below 4 files:

- target/surefire-reports/*.xml <--- 1 XMl file (pick via <*.log> pattern, if more files present, then validate via AC1)
- target/zerocode-junit-interactive-fuzzy-search.html
- target/zerocode-junit-granular-report.csv
- target/logs/*.log <--- 1 log file (pick via <*.log> pattern, as this is custom named per project. Also, validate via AC1>

[xc-x]

Hi, I would like to do this issue. Can you please assgin to me?

[authorjapps]

Hi, I would like to do this issue. Can you please assgin to me?

Hello @xc-x , thanks for your interest. Now assigned to you.
Also, can you list down the high level implementation steps in this ticket, prior to or while implementing it?
(this will help the collaborators while reviewing)

[javiertuya]

@xc-x @authorjapps @nirmalchandra A suggestion:

Recently, I was looking for a way to push several files using the GitHub api, but I only found how to push a single file. It could be possible using some API methods to manipulate the Git Tree, but it seems to be very low level and I didn't found enough documentation.

Maybe the safest approach is working in the workspace (from inside the GHA container or Jenkins slave) using Eclipse JGit. You can do the same actions that you could do from the CLI. If you look at this repo: https://github.com/javiertuya/dashgit, the dashgit-updater project contains a class giis.qabot.ci.clients.GitLocal that provides common git commands (clone, checkout, commit, push, merge, etc.). It could be useful to upload the files to any repo (GitHub, GitLab...)

[rkshpanigrahi]

Hi @authorjapps Could you please assign this issue to me if it is available.

[authorjapps]

Hi @authorjapps Could you please assign this issue to me if it is available.

Done.

@rkshpanigrahi

When you're ready, please add a write-up here describing the approach you'll take to implement the solution. This will help other collaborators share any ideas or suggestions. Once that's done, you can proceed with raising a PR.

[rkshpanigrahi]

Hi @authorjapps

Here is my approach to implement the solution.
Please let me know if any corrections are needed.

1. Define config properties for repo URL, auth token, and max file size (default 2MB).

   reports.repo=https://github.com/authorjapps/test-reports
   reports.repo.token:
   reports.repo.max.upload.limit.mb=2 #<----- can be configured for more or less if needed

2. After test execution, push following files to external repo. Check each file's size against reports.repo.max.upload.limit.mb and skip if it exceeds the limit.

   • target/surefire-reports/.xml <--- 1 XMl file (pick via <.log> pattern, if more files present, then validate via AC1)
   • target/zerocode-junit-interactive-fuzzy-search.html
   • target/zerocode-junit-granular-report.csv
   • target/logs/.log <--- 1 log file (pick via <.log> pattern, as this is custom named per project. Also, validate via AC1>

3. Once all test completed push the files to the configured external repo

4. Going to use Eclipse JGit to push selected files to test-reports repo.

5. On upload failure, log a WARN-level message without failing the CI build