List invalid or tampered immutable files in verify command #2637

turmelclem · 2025-07-15T07:35:03Z

Content

improve client library and CLI for verify command to list missing, tampered and/or non verifiable files

verify command can now be use with a range
a flag can be use (--allow-missing) to not throw error if immutable file is missing

Pre-submit checklist

Branch
- Tests are provided (if possible)
- Crates versions are updated (if relevant)
- CHANGELOG file is updated (if relevant)
- Commit sequence broadly makes sense
- Key commits have useful messages
PR
- All check jobs of the CI have succeeded
- Self-reviewed the diff
- Useful pull request description
- Reviewer requested
Documentation
- Update README file (if relevant)
- Update documentation website (if relevant)
- Add dev blog post (if relevant)
- Add ADR blog post or Dev ADR entry (if relevant)
- No new TODOs introduced

Comments

Issue(s)

Closes #2618

examples/client-cardano-database-v2/src/main.rs

github-actions · 2025-07-15T07:38:55Z

Test Results

4 files ± 0 154 suites ±0 23m 37s ⏱️ +18s
2 134 tests +16 2 134 ✅ +16 0 💤 ±0 0 ❌ ±0
6 530 runs +64 6 530 ✅ +64 0 💤 ±0 0 ❌ ±0

Results for commit 9eaf920. ± Comparison against base commit 2ab36d3.

This pull request removes 1 and adds 17 tests. Note that renamed tests count towards both.

mithril-client ‑ cardano_database_client::proving::tests::compute_merkle_proof::compute_merkle_proof_succeeds

mithril-client ‑ cardano_database_client::proving::tests::download_and_verify_digests::download_and_verify_digest_should_return_digest_map_acording_to_beacon
mithril-client ‑ cardano_database_client::proving::tests::list_immutable_files_not_verified::should_return_empty_list_when_no_tampered_files
mithril-client ‑ cardano_database_client::proving::tests::list_immutable_files_not_verified::should_return_list_with_non_verifiable
mithril-client ‑ cardano_database_client::proving::tests::list_immutable_files_not_verified::should_return_list_with_tampered_files
mithril-client ‑ message::tests::cardano_database_message::compute_cardano_database_message_should_fail_if_immutable_is_missing_and_allow_missing_not_set
mithril-client ‑ message::tests::cardano_database_message::compute_cardano_database_message_should_fail_if_immutable_is_tampered
mithril-client ‑ message::tests::cardano_database_message::compute_cardano_database_message_should_fail_if_immutables_are_missing_and_tampered
mithril-client ‑ message::tests::cardano_database_message::compute_cardano_database_message_should_fail_if_there_is_more_local_immutable_than_verified_digest
mithril-client ‑ message::tests::cardano_database_message::compute_cardano_database_message_should_success_if_immutable_is_missing_and_allow_missing_is_set
mithril-client ‑ message::tests::cardano_database_message::compute_cardano_database_message_succeeds
…

♻️ This comment has been updated with latest results.

examples/client-cardano-database-v2/src/main.rs

mithril-client-cli/src/utils/cardano_db.rs

mithril-client/src/cardano_database_client/proving.rs

Copilot

Pull Request Overview

This PR enhances the Cardano database snapshot verification by listing missing or tampered immutable files instead of failing silently. It refactors the digest download and Merkle-proof verification into a unified flow, introduces detailed error types with file listings, and updates the CLI to surface these details in both JSON and human-readable formats.

Refactored compute_merkle_proof into download_and_verify_digests, returning VerifiedDigests (digest map + Merkle tree).
Extended MessageBuilder::compute_cardano_database_message to accept VerifiedDigests, handle an allow_missing flag, and return rich error information (ImmutableFilesLists).
Updated CLI commands to write JSON error files and print counts of missing/tampered files.

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
mithril-client/tests/cardano_db_snapshot_list_get_download_verify.rs	Updated tests to use `download_and_verify_digests` and `VerifiedDigests`.
mithril-client/src/message.rs	Added `ImmutableFilesLists`, `ComputeCardanoDatabaseMessageError`, and enhanced `compute_cardano_database_message`.
mithril-client/src/cardano_database_client/proving.rs	Replaced `compute_merkle_proof` with `download_and_verify_digests` returning `VerifiedDigests`.
mithril-client/src/cardano_database_client/api.rs	Updated public API to expose `download_and_verify_digests`.
mithril-client-cli/src/commands/cardano_db/verify.rs	Integrated new error handling in the `verify` command and JSON output.

mithril-client/src/message.rs

Alenar

LGTM 🐎

examples/client-cardano-database-v2/src/main.rs

Alenar · 2025-07-22T09:40:05Z

mithril-client-cli/src/commands/cardano_db/download/v2.rs

+        let options = ComputeCardanoDatabaseMessageOptions {
+            db_dir: restoration_options.db_dir.clone(),
+            immutable_file_range: restoration_options.immutable_file_range,
+            allow_missing: false,
+        };


I wonder if this should be part of the client library directly to reduce the number of arguments of compute_cardano_database_message.

mithril-client/src/cardano_database_client/mod.rs

mithril-client/src/cardano_database_client/proving.rs

mithril-client/src/message.rs

mithril-client/src/cardano_database_client/proving.rs

jpraynaud · 2025-07-22T09:16:26Z

mithril-client/src/cardano_database_client/proving.rs

+        }
+
+        #[tokio::test]
+        async fn download_and_verify_digest_should_return_digest_map_acording_to_beacon() {


Suggested change

async fn download_and_verify_digest_should_return_digest_map_acording_to_beacon() {

async fn download_and_verify_digest_should_return_digest_map_according_to_beacon()

mithril-client/src/cardano_database_client/proving.rs

mithril-client-cli/src/commands/cardano_db/download/v2.rs

jpraynaud · 2025-07-22T10:04:20Z

mithril-client/src/message.rs

        pub async fn compute_cardano_database_message(
            &self,
-            certificate: &MithrilCertificate,
-            merkle_proof: &MKProof,
-        ) -> MithrilResult<ProtocolMessage> {
-            let mut message = certificate.protocol_message.clone();
-            message.set_message_part(
-                ProtocolMessagePartKey::CardanoDatabaseMerkleRoot,
-                merkle_proof.root().to_hex(),
-            );
+            certificate: &CertificateMessage,
+            cardano_database_snapshot: &CardanoDatabaseSnapshotMessage,
+            immutable_file_range: &ImmutableFileRange,
+            allow_missing: bool,
+            database_dir: &Path,
+            verified_digests: &VerifiedDigests,
+        ) -> Result<ProtocolMessage, ComputeCardanoDatabaseMessageError> {


IMO the proving part (Merkle proof computation) should be kept in the proving module.
Also this function is too complex (too many inputs and lines).

as you see we now have added three new parameters since verified_digests are a dedicated step (that was previously done by compute_merkle_proof) so we need digests, range, and allow_missing flag. They may be reorganize in a struct ?

if we keep compute_merkle_proof in the proving.rs file has a exposed method for API, we will have to decide where to put the listing of missing immutable files and also the verification of the proof (tampered and non verifiable files)

see explored possibilities here

…ests and verified them against certificate

…de the snapshotmessage computation

…d to compute_cardano_database_message

…ano database message with a structured error

… immutables files

… files

…red immutables files in case of ImmutableFilesVerification error

…o handle the new typed error

verify command fail

… verify command

…rdano_database_message function

…f proof computation error

…e files

…verification result

…, fix dummy tool import

github-advanced-security bot found potential problems Jul 15, 2025

View reviewed changes

examples/client-cardano-database-v2/src/main.rs Fixed Show fixed Hide fixed

examples/client-cardano-database-v2/src/main.rs Fixed Show fixed Hide fixed

examples/client-cardano-database-v2/src/main.rs Fixed Show fixed Hide fixed

turmelclem force-pushed the ctl/2618-list-invalid-or-missing-immutable-files-in-verify-command branch from 48bf6a1 to 68915c2 Compare July 15, 2025 08:47

github-advanced-security bot found potential problems Jul 15, 2025

View reviewed changes

turmelclem force-pushed the ctl/2618-list-invalid-or-missing-immutable-files-in-verify-command branch from 68915c2 to 169a685 Compare July 15, 2025 08:54

github-advanced-security bot found potential problems Jul 15, 2025

View reviewed changes

mithril-client-cli/src/utils/cardano_db.rs Fixed Show fixed Hide fixed

turmelclem force-pushed the ctl/2618-list-invalid-or-missing-immutable-files-in-verify-command branch 5 times, most recently from 6a98b96 to 2af282f Compare July 16, 2025 14:05

github-advanced-security bot found potential problems Jul 16, 2025

View reviewed changes

mithril-client/src/cardano_database_client/proving.rs Fixed Show fixed Hide fixed

turmelclem force-pushed the ctl/2618-list-invalid-or-missing-immutable-files-in-verify-command branch 3 times, most recently from 54a2b7c to cc140f0 Compare July 16, 2025 16:22

turmelclem self-assigned this Jul 17, 2025

turmelclem force-pushed the ctl/2618-list-invalid-or-missing-immutable-files-in-verify-command branch 6 times, most recently from 2eae876 to 65218e9 Compare July 17, 2025 15:55

turmelclem requested review from Copilot, jpraynaud, Alenar and dlachaume July 17, 2025 15:55

Copilot AI reviewed Jul 17, 2025

View reviewed changes

mithril-client/src/message.rs Outdated Show resolved Hide resolved

mithril-client/src/message.rs Outdated Show resolved Hide resolved

mithril-client/src/message.rs Outdated Show resolved Hide resolved

turmelclem temporarily deployed to testing-preview July 17, 2025 16:05 — with GitHub Actions Inactive

turmelclem force-pushed the ctl/2618-list-invalid-or-missing-immutable-files-in-verify-command branch from 65218e9 to b33c6f7 Compare July 18, 2025 08:32

turmelclem temporarily deployed to testing-preview July 18, 2025 08:42 — with GitHub Actions Inactive

Alenar approved these changes Jul 22, 2025

View reviewed changes

turmelclem force-pushed the ctl/2618-list-invalid-or-missing-immutable-files-in-verify-command branch from 2776820 to 69aaf2e Compare July 22, 2025 12:17

github-advanced-security bot found potential problems Jul 22, 2025

View reviewed changes

turmelclem force-pushed the ctl/2618-list-invalid-or-missing-immutable-files-in-verify-command branch 2 times, most recently from ceed11c to 75988a6 Compare July 22, 2025 12:29

turmelclem temporarily deployed to testing-preview July 22, 2025 12:52 — with GitHub Actions Inactive

jpraynaud reviewed Jul 22, 2025

View reviewed changes

turmelclem force-pushed the ctl/2618-list-invalid-or-missing-immutable-files-in-verify-command branch from feabf8c to 6c220c5 Compare July 23, 2025 12:47

turmelclem temporarily deployed to testing-preview July 23, 2025 16:01 — with GitHub Actions Inactive

turmelclem added 18 commits July 25, 2025 14:50

feature(client-lib, client-cli): introduce a function to download dig…

0e76447

…ests and verified them against certificate

refactor(client-lib): move merkleproof creation and verification insi…

ccd8595

…de the snapshotmessage computation

refactor(client-lib): remove compute_merkle_proof since logic is move…

0898dc1

…d to compute_cardano_database_message

feature(client-lib): list missing immutable files when computing card…

2992846

…ano database message with a structured error

feature(client-lib): add function in VerifiedDigests to list tampered…

b07677c

… immutables files

feature(client-lib): compute cardano database message verify tampered…

8fd5c7a

… files

feature(client-lib): display 10 elements of each missing and/or tampe…

3b4bb5e

…red immutables files in case of ImmutableFilesVerification error

feature(client-cli): adapt call of compute_cardano_database_message t…

e979632

…o handle the new typed error

feature(client-cli): display missing and/or tampered immutables when

d9a9367

verify command fail

feature(client-lib,client-cli): add arg --allow-missing in cardano-db…

29831c0

… verify command

fix(examples): realign immutables range with release-preprod environment

266b2a3

feature(client-lib, examples): adapt examples with the new compute_ca…

0b3de45

…rdano_database_message function

docs: update mithril client doc with new verify command arguments

da1ee2c

feature(client-lib): return also list of non verified files in case o…

0fca1a1

…f proof computation error

feature(client-cli): improve verify command output with non verifiabl…

857cc80

…e files

refactoring(client-lib,client-cli): improve name of cardano database …

ee28690

…verification result

refactoring(client-lib): reorganize imports

2605f94

feature(client-lib): rename ComputeCardanoDatabaseMessageError errors…

9eaf920

…, fix dummy tool import

turmelclem force-pushed the ctl/2618-list-invalid-or-missing-immutable-files-in-verify-command branch from 6c220c5 to 9eaf920 Compare July 25, 2025 12:59

turmelclem temporarily deployed to testing-preview July 25, 2025 13:08 — with GitHub Actions Inactive

turmelclem mentioned this pull request Jul 28, 2025

List the invalid/missing immutable files in the verify command in client CLI #2618

Open

25 tasks

	async fn download_and_verify_digest_should_return_digest_map_acording_to_beacon() {
	async fn download_and_verify_digest_should_return_digest_map_according_to_beacon()

List invalid or tampered immutable files in verify command #2637

Are you sure you want to change the base?

List invalid or tampered immutable files in verify command #2637

Uh oh!

Conversation

turmelclem commented Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Content

Pre-submit checklist

Comments

Issue(s)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Alenar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Alenar Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jpraynaud Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jpraynaud Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

turmelclem Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

turmelclem commented Jul 15, 2025 •

edited

Loading

github-actions bot commented Jul 15, 2025 •

edited

Loading

turmelclem Jul 24, 2025 •

edited

Loading