Skip to content
42 changes: 21 additions & 21 deletions core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,50 +126,50 @@

# Contribution Process

The Stellar Protocol, like most software in the world, continues to evolve over time to meet the
The Stellar protocol, like most software in the world, continues to evolve over time to meet the
needs of our network's participants and to drive technology forward into new territory. Given the
importance of the reliability and safety of the network, we ask that all of those who have ideas
towards pushing Stellar's protocol development forward adhere to the following:

- Consider your idea and how it serves the fundamental goals of the Stellar Network and aligns with
values of the Stellar Protocol (which are listed below). If you cannot show how your proposal
- Consider your idea and how it serves the fundamental goals of the Stellar network and aligns with
values of the Stellar protocol (which are listed below). If you cannot show how your proposal
aligns with those goals and values, it's unlikely to ever be implemented.
- Gather feedback from discussion on the dev mailing list and other forums, and utilize it to begin
a draft proposal, otherwise known as a CAP (Core Advancement Proposal).
- Follow the proposal process listed below.

## Stellar Network Goals
- **The Stellar Network should be secure and reliable, and should bias towards safety, simplicity,
## Stellar network Goals
- **The Stellar network should be secure and reliable, and should bias towards safety, simplicity,
reliability, and performance over new functionality.**
- **The Stellar Network should run at scale and at low cost to all participants of the network.**
* In support of this, the Stellar Network should support off-chain transactions, e.g. Starlight.
- **The Stellar network should run at scale and at low cost to all participants of the network.**
* In support of this, the Stellar network should support off-chain transactions, e.g. Starlight.
* An explicit non-goal is limiting the hardware requirements of stellar-core to a personal
computer.
- **The Stellar Network should facilitate simplicity and interoperability with other protocols and
- **The Stellar network should facilitate simplicity and interoperability with other protocols and
networks.**
* In support of this, the Stellar Network should facilitate side-chain transactions to enable
* In support of this, the Stellar network should facilitate side-chain transactions to enable
sub-networks.
- **The Stellar Network should enable cross-border payments, i.e. payments via exchange of assets,
- **The Stellar network should enable cross-border payments, i.e. payments via exchange of assets,
throughout the globe, enabling users to make payments between assets in a manner that is fast,
cheap, and highly usable.**
- In support of this, the Stellar Network should support an orderbook that values simplicity
- In support of this, the Stellar network should support an orderbook that values simplicity
over functionality, and one that primarily serves to enable cross-border payments.
- In support of this, the Stellar Network should facilitate liquidity as a means to enabling
- In support of this, the Stellar network should facilitate liquidity as a means to enabling
- cross-border payments.
- In support of this, the Stellar Network should enable asset issuance, but as a means of
- In support of this, the Stellar network should enable asset issuance, but as a means of
- enabling cross-border payments.
- **The Stellar Network should support decentralization wherever possible, but not at the expense
- **The Stellar network should support decentralization wherever possible, but not at the expense
of the majority of its values.**
- There should be no privileged actors — we should support egalitarianism and everyone
participating on the same playing field.
- **The Stellar Network should enable users to easily exchange their non-Stellar based assets to
- **The Stellar network should enable users to easily exchange their non-Stellar based assets to
Stellar-based assets, and vice versa.**
- **The Stellar Network should make it easy for developers of Stellar projects to create highly
- **The Stellar network should make it easy for developers of Stellar projects to create highly
usable products.**

## Stellar Protocol Development Values
- **The Stellar Protocol should serve the goals of the Stellar Network.**
- **The Stellar Protocol should bias towards simplicity.**
- **The Stellar protocol should serve the goals of the Stellar network.**
- **The Stellar protocol should bias towards simplicity.**
- When possible, solutions should be considered outside of core protocol changes such as via
[SEPs (Stellar Ecosystem Proposals)](../ecosystem/README.md) to minimize complexity in the
Stellar protocol.
Expand All @@ -187,12 +187,12 @@ towards pushing Stellar's protocol development forward adhere to the following:
- Unobservable tx semantics (eg. performance or bug fixes)
- Horizon semantics
- Public APIs, Client Libraries/SDKs.
- **The Stellar Protocol should be clear, concise, and opinionated.**
- **The Stellar protocol should be clear, concise, and opinionated.**
- New operations and functionality should be opinionated, and straightforward to use.
- There should ideally be only one obvious way to accomplish a given task.
- **The Stellar Protocol should bias towards broad use cases, and bias against niche
- **The Stellar protocol should bias towards broad use cases, and bias against niche
functionality.**
- **The Stellar Protocol should bias towards user safety.**
- **The Stellar protocol should bias towards user safety.**

## CAP Process
These are the steps from [idea to deployment](https://www.youtube.com/watch?v=Otbml6WIQPo) on how
Expand Down
28 changes: 14 additions & 14 deletions core/cap-0076.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,25 +13,26 @@ Protocol version: 24

## Simple Summary

Fix the entries that have been archived in corrupted state due to a bug in protocol 23 *and have never had corruption observed*. Also update the fee pool to reflect the unintentional XLM burns.
Fixes the entries that have been archived in corrupted state due to a bug in protocol 23 *and have never had corruption observed*. Also updates the fee pool to reflect the unintentional XLM burns.

## Working Group

As specified in the Preamble.

## Motivation

Protocol 23 has introduced a mechanism for evicting the persistent contract data and code entries into Hot Archive (see [CAP-62](./cap-0062.md) for details). However, due to an implementation bug, at the moment of the archival (i.e. when the entry gets removed from the live bucket list and is moved to the Hot Archive) an arbitrary historical state has been used instead of the most recent state. That means that some entries have been archived with a state that they had at some point at time, but not the most recent state.
Protocol 23 introduced a mechanism to evict persistent contract data and code entries into the Hot Archive (see [CAP-62](./cap-0062.md) for details). However, an implementation bug at the moment of archival (i.e. when the entry gets deleted from the live bucket list and added to the Hot Archive) used an arbitrary historical state instead of the most recent state. As a result, some entries were archived with stale data from a point in time that was not the latest snapshot.

Consider the following example that illustrates how the bug manifests. Imagine a contract `C` that has a balance of 10 XLM. `C` then proceeds to spend 9 XLM and is left with 1 XLM balance. Then `C` doesn't perform any more XLM operations for a while and its balance entry gets archived. Due to a bug, the entry state where it had 10 XLM balance could get archived. If that was the case, then when someone would restore `C` balance, they would get back 10 XLM balance, thus effectively minting 9 XLM. Note, that until the entry has been restored its state is not observable on-chain and thus the 9 XLM mint is only meaningful from the moment of restoration.

The actual bug impact on Stellar Mainnet has been that 478 ledger entries have been archived with an incorrect state before validators disabled the state eviction at the protocol entries. Luckily, 394 entries out of these have never been restored (at least at the moment of writing this CAP; however, the restoration has been disabled at the overlay level for these entries and it's highly unlikely that any more restorations would have happened until protocol 24 upgrade).
On Stellar mainnet, the bug archived incorrect state for 478 ledger entries before validators disabled state eviction at the protocol level in ledger `59313516`. Luckily, 394 of those entries have never been restored (at least at the moment of writing this CAP; however, the restoration has been disabled at the overlay level for these entries and it's highly unlikely that any more restorations would have happened until the protocol 24 upgrade).

The 394 entries that have never been restored are hashed into ledger as a part of the Hot Archive. However, as mentioned above, archived entries don't have any other observable on-chain impact (i.e. it's not possible for any transaction to read or modify their value). Because of that it is possible to amend the state of the corrupted entries to the correct value without risking to break any on-chain logic or invariants.
The 394 entries that have never been restored are hashed into the ledger as a part of the Hot Archive. However, as mentioned above, archived entries don't have any other observable on-chain impact (i.e. it's not possible for any transaction to read or modify their value). Because of that it is possible to amend the state of the corrupted entries to the correct value without risking to break any on-chain logic or invariants.

Amending the corrupted entries would be beneficial to the network as it avoids all sorts of issues that would occur if entries were restored in the corrupted state (such as unexpected token mints or burns, or protocols assuming invalid state). While it is a rare and unusual precedent for validators to make any changes to the on-chain state they don't own (i.e. anything beyond the network configuration itself), this particular kind of amendment has a very limited and verifiable scope. It is possible for any observer to ingest the correct (i.e. pre-archival) state for every amended entry from the history and then verify that it matches the state of the entry after protocol 24 upgrade. It is also possible to verify that only the affected entries get amended via replaying the history and examining the Hot Archive hash after the upgrade. These factors make it not possible for validators to maliciously update the entries that haven't been corrupted, or updating the corrupted entries to incorrect values *without that being visible for any external observer*.

Note, that this CAP does not allow amendment of any state that has actually been observed due to restoration. The only change that concerns restored corrupted entries is an amendment of the fee pool that is done just in order to reflect the total XLM balance on the network.
> [!NOTE]
> This CAP does not allow amendment of any state that has actually been observed due to restoration. The only change concerning restored corrupted entries is an amendment of the fee pool, implemented just in order to reflect the networks' total XLM balance.

### Goals Alignment

Expand All @@ -49,7 +50,7 @@ The fee pool is amended by adding `31879035` stroops in order to account for the

### The list of the corrupted entries

The full list of the corrupted entries (including those that had been restored) is attached to this repository in [corrupted_hot_archive_entries.csv](./../contents/cap-0076/corrupted_hot_archive_entries.csv) file.
The full list of the corrupted entries (including those that had been restored) is attached to this repository in [corrupted_hot_archive_entries.csv](../contents/cap-0076/corrupted_hot_archive_entries.csv) file.

The file contains 478 rows with 5 columns:

Expand All @@ -65,26 +66,25 @@ The file contains 478 rows with 5 columns:

The entry amendments may strictly happen only for the entries that are still in the Hot Archive at the moment of protocol upgrade, and that still are in the exact state that has been observed at the moment of the corrupted archival.

Specifically, for every `ledger_key` in [corrupted_hot_archive_entries](./../contents/cap-0076/corrupted_hot_archive_entries.csv) a check for an entry to be amendable will be performed. The check requires that all of the following conditions are true:
Specifically, for every `ledger_key` in [corrupted_hot_archive_entries](../contents/cap-0076/corrupted_hot_archive_entries.csv) a check for an entry to be amendable will be performed. The check requires that all of the following conditions are true:

- There is no entry corresponding to the `ledger_key` in the current live state
- `ledger_key` is not being archived in the upgrade ledger
- An entry `E` corresponding to the `ledger_key` exists in the Hot Archive
- `E == archived_entry` (where `archived_entry` is the corresponding value from the corrupted_hot_archive_entries table)

If the entry is amendable, then a new ledger entry with value ``correct_entry` - base64 encoded `LedgerEntry` XDR, the correct value that should have been archived (i.e. the value of the affected entry at the moment when it has been archived)
` (from the table) will be written to the Hot Archive and thus become the most recent state of the entry that will be used for the restoration.
If the entry is amendable, then a new ledger entry with value `correct_entry` (defined above) from the table will be written to the Hot Archive and thus become the most recent state of the entry that will be used for the restoration.

Note, that since the amendment only occurs in the Hot Archive, the change will not be reflected in `LedgerCloseMeta`, as it can not contain Hot Archive changes. The change will only be observable due to the `bucketListHash` change in the upgrade ledger header. After the upgrade the amended changes will be observable at the moment of restoration (and they will have the state that matches the state they had prior to archival).

#### Fee pool amendment

`31879035` stroops will be added to the `feePool` in the `LedgerHeader`. This change reflects the XLM burn that occurred due to bug. Specifically, two XLM contract balances have been restored with a balance that is lower than the balance they had before archival. The specific XLM burns are as follows (these can also be verified in [corrupted_hot_archive_entries](./../contents/cap-0076/corrupted_hot_archive_entries.csv)):
`31879035` stroops will be added to the `feePool` in the `LedgerHeader`. This change reflects the XLM burn that occurred due to bug. Specifically, two XLM contract balances have been restored with a balance that is lower than the balance they had before archival. The specific XLM burns are as follows (these can also be verified in [corrupted_hot_archive_entries](../contents/cap-0076/corrupted_hot_archive_entries.csv)):

- Contract `CAS3J7GYLGXMF6TDJBBYYSE3HQ6BBSMLNUQ34T6TZMYMW2EVH34XOWMA` had XLM balance of `291100005` stroops before archival, but had `290100005` stroops at the moment of restoration (`1000000` stroops burned)
- Contract `CAS3J7GYLGXMF6TDJBBYYSE3HQ6BBSMLNUQ34T6TZMYMW2EVH34XOWMA` had XLM balance of `291100005` stroops before archival, but had `290100005` stroops at the moment of restoration (`1000000` stroops burned)
- Contract `CDLMAKG5TSJA6FGP7LLC2FKJRQW6DQYMEPP6FURFVULDEQMP3PRZ4ISI` had XLM balance of `1173140246` stroops before archival, but had `1142261211` stroops balance at the moment of restoration (`30879035` stroops burned)

Thus `1000000 + 30879035 = 31879035` stroops have been burned, which is reflected via `feePool`.
Thus `1000000 + 30879035 = 31879035` stroops were burned, which is reflected via `feePool`.

### Backwards Incompatibilities

Expand All @@ -99,13 +99,13 @@ The upgrade ledger will need to do a few hundred additional disk lookups, but th
As mentioned in the 'Motivation' section this CAP performs modification of the ledger state that is not directly owned by the validators, which comes with the inherent risk of malicious modifications, or non-malicious erroneous modifications that lead to the further state corruption that can then be abused by the attackers.

In order to resolve these concerns to some degree, and in order to ensure that the amendments only bring back the entries to their valid state, the Stellar Core build that performs the upgrade will contain the tools that allow anyone to ensure that:
- During the replay of protocol 23 *only* the entries from [corrupted_hot_archive_entries](./../contents/cap-0076/corrupted_hot_archive_entries.csv) table are incorrectly incorrectly archived, and that their correct and archived states match those in the table
- During the replay of protocol 23 *only* the entries from [corrupted_hot_archive_entries](../contents/cap-0076/corrupted_hot_archive_entries.csv) table are incorrectly archived, and that their correct and archived states match those in the table
- Ensure that *every* entry from the table has indeed been incorrectly archived
- Ensure that during the protocol upgrade only the entries from the table have been updated, and that the update has brought them back to the correct state

## Test Cases

The audit process described in the section above is implemented in Stellar Core. It will have an optional config called `PATH_TO_PROTOCOL_23_CORRUPTION_FILE`, which is the file path to the [corrupted_hot_archive_entries](./../contents/cap-0076/corrupted_hot_archive_entries.csv) file. When set, Stellar Core will add a series of asserts to verify the correctness of the file when replaying protocol 23 ledgers, as described in the Security Concerns section. In order to independently verify the correctness of the affected archived keys, set this config option and run catchup starting from the initial protocol 23 ledger, 58762517.
The audit process described in the section above is implemented in Stellar Core. It will have an optional config called `PATH_TO_PROTOCOL_23_CORRUPTION_FILE`, which is the file path to the [corrupted_hot_archive_entries](../contents/cap-0076/corrupted_hot_archive_entries.csv) file. When set, Stellar Core will add a series of asserts to verify the correctness of the file when replaying protocol 23 ledgers, as described in the Security Concerns section. In order to independently verify the correctness of the affected archived keys, set this config option and run catchup starting from the initial protocol 23 ledger, `58762517`.

## Implementation

Expand Down
Loading
Loading