- 
                Notifications
    You must be signed in to change notification settings 
- Fork 168
Extendend whitelist options #1644
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
b1ff17f    to
    de26071      
    Compare
  
    cf3fcab    to
    cc0f1a7      
    Compare
  
            
          
                cardano-db-sync/src/Cardano/DbSync/Era/Universal/Insert/Other.hs
              
                Outdated
          
            Show resolved
            Hide resolved
        
      There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! I added a few questions also
cc0f1a7    to
    ffd68e0      
    Compare
  
    72083ce    to
    ea4514c      
    Compare
  
    ea4514c    to
    78418bf      
    Compare
  
    | Run into following error message with executable built from this branch, on  
 and it is hanging like that for quite some time, even though I used  Longer log:  | 
| Another question would be - should we have validator for stake addresses in config whitelisting ? I used this and no errors on  | 
| 
 what version of cardano-node are you using at the moment? I can't even get things to start syncing locally. I can see in the code where the issue is happening. Will try to get sanchonet working today 👍 
 The problem with that is if you add the stake address before starting a fresh sync, there is no way to validate if it's existence as it's not been yet created. | 
| 
 @Cmdv not sure if this is right - it seemed to work with this version. | 
| I am using... node: db-sync: Network:  Network Configs: wget https://book.play.dev.cardano.org/environments/sanchonet/config.json \
  https://book.play.dev.cardano.org/environments/sanchonet/db-sync-config.json \
  https://book.play.dev.cardano.org/environments/sanchonet/submit-api-config.json \
  https://book.play.dev.cardano.org/environments/sanchonet/topology.json \
  https://book.play.dev.cardano.org/environments/sanchonet/byron-genesis.json \
  https://book.play.dev.cardano.org/environments/sanchonet/shelley-genesis.json \
  https://book.play.dev.cardano.org/environments/sanchonet/alonzo-genesis.json \
  https://book.play.dev.cardano.org/environments/sanchonet/conway-genesis.jsondb-sync config:   "insert_options": {
    "tx_out": {
      "value": "enable"
    },
    "ledger": "enable",
    "shelley": {
      "enable": true,
      "stake_addresses": ["e083d27f03b197750ee41b7dca02c806718164b934b0fbfa747349e8f2"]
    },
    "multi_asset": {
      "enable": true
    },
    "metadata": {
      "enable": true
    },
    "plutus": {
      "enable": true
    },
    "governance": "enable",
    "offchain_pool_data": "enable",
    "json_type": "text"
  },Logs: So If I am reading it correctly - it breaks on querying / inserting first record in  
 Query on database synced without whitelisting: sancho=# select * from gov_action_proposal limit 3;
 id | tx_id | index | prev_gov_action_proposal |  deposit   | return_address | expiration | voting_anchor_id |        type         |                                                                                      description                                                                                      | param_proposal | ratified_epoch | enacted_epoch | dropped_epoch | expired_epoch 
----+-------+-------+--------------------------+------------+----------------+------------+------------------+---------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+----------------+---------------+---------------+---------------
  1 |  1878 |     0 |                          | 1000000000 |           1024 |        251 |                1 | TreasuryWithdrawals | {"tag": "TreasuryWithdrawals", "contents": [[[{"network": "Testnet", "credential": {"keyHash": "248f556b733c3ef24899ae0609d3796198d5470192304c4894dd85cb"}}, 10000000000000]], null]} |                |                |               |               |              
  2 |  1948 |     0 |                          | 1000000000 |           1024 |        252 |                2 | InfoAction          | {"tag": "InfoAction"}                                                                                                                                                                 |                |                |               |               |              
  3 |  4116 |     0 |                          | 1000000000 |           1044 |        253 |                1 | TreasuryWithdrawals | {"tag": "TreasuryWithdrawals", "contents": [[[{"network": "Testnet", "credential": {"keyHash": "248f556b733c3ef24899ae0609d3796198d5470192304c4894dd85cb"}}, 51000000]], null]}       |                |                |               |               |              
(3 rows) | 
| OK, I think I got it something more. So basically adding  sancho=# select * from gov_action_proposal limit 3;
 id | tx_id | index | prev_gov_action_proposal |  deposit   | return_address | expiration | voting_anchor_id |        type         |                                                                                      description                                                                                      | param_proposal | ratified_epoch | enacted_epoch | dropped_epoch | expired_epoch 
----+-------+-------+--------------------------+------------+----------------+------------+------------------+---------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+----------------+---------------+---------------+---------------
  1 |  1878 |     0 |                          | 1000000000 |           1024 |        251 |                1 | TreasuryWithdrawals | {"tag": "TreasuryWithdrawals", "contents": [[[{"network": "Testnet", "credential": {"keyHash": "248f556b733c3ef24899ae0609d3796198d5470192304c4894dd85cb"}}, 10000000000000]], null]} |                |                |               |               |              in this case  sancho=# select * from stake_address where id=1024;
  id  |                           hash_raw                           |                               view                               | script_hash 
------+--------------------------------------------------------------+------------------------------------------------------------------+-------------
 1024 | \xe0192a8eb77a6f405d8196dd96db13950c48300a0ff8b6f42e7901a2ef | stake_test1uqvj4r4h0fh5qhvpjmwedkcnj5xysvq2plutdapw0yq69mccrcnd7 | 
(1 row)to        "stake_addresses": ["e083d27f03b197750ee41b7dca02c806718164b934b0fbfa747349e8f2","e0192a8eb77a6f405d8196dd96db13950c48300a0ff8b6f42e7901a2ef"]Moved error past epoch  So if stake address for  | 
2772d9f    to
    efa38c2      
    Compare
  
    | @Cmdv  With latest version on this branch I am left stuck at  I see that tables listed below are empty while the non whitelisted version of DB have records for them ( the common thing is that they have  select * from constitution;
 id | gov_action_proposal_id | voting_anchor_id | script_hash 
----+------------------------+------------------+-------------
(0 rows)
select * from new_committee_info;
 id | gov_action_proposal_id | quorum_numerator | quorum_denominator 
----+------------------------+------------------+--------------------
(0 rows)
select * from new_committee;
 id | gov_action_proposal_id | deleted_members | added_members | quorum_numerator | quorum_denominator 
----+------------------------+-----------------+---------------+------------------+--------------------
(0 rows)
select * from new_committee_member;
 id | gov_action_proposal_id | committee_hash_id | expiration_epoch 
----+------------------------+-------------------+------------------
(0 rows)
select * from voting_procedure;
 id | tx_id | index | gov_action_proposal_id | voter_role | drep_voter | pool_voter | vote | voting_anchor_id | committee_voter 
----+-------+-------+------------------------+------------+------------+------------+------+------------------+-----------------
(0 rows)Also do not see records in those tables ( contrary to non whitelisted DB version ) EDIT:     "shelley": {
      "enable": true,
      "stake_addresses": ["e083d27f03b197750ee41b7dca02c806718164b934b0fbfa747349e8f2"]
    },sancho_regular=# select * from stake_address where id=5;
 id |                           hash_raw                           |                               view                               | script_hash 
----+--------------------------------------------------------------+------------------------------------------------------------------+-------------
  5 | \xe083d27f03b197750ee41b7dca02c806718164b934b0fbfa747349e8f2 | stake_test1uzpaylcrkxth2rhyrd7u5qkgqeccze9exjc0h7n5wdy73usd0pxpl | 
(1 row)
It seems like there might be also issue with  sancho_regular=# select * from tx_out where stake_address_id=5;
 id | tx_id | index | address | address_has_script | payment_cred | stake_address_id | value | data_hash | inline_datum_id | reference_script_id 
----+-------+-------+---------+--------------------+--------------+------------------+-------+-----------+-----------------+---------------------
(0 rows)
sancho_regular=# select count(*) from tx_out where stake_address_id IS NOT NULL;
 count 
-------
  6210
(1 row)and on whitelisted by  sancho_test_config_shelley_whitelist=# select count(*) from tx_out;
 count  
--------
 103980
(1 row)
sancho_test_config_shelley_whitelist=# select count(*) from tx_out where stake_address_id IS NOT NULL;
 count 
-------
     0
(1 row)So we should get rid off all those records with NULL  | 
| This is related to  Assuming that config below is correct:   "insert_options": {
    "tx_out": {
      "value": "enable"
    },
    "ledger": "enable",
    "shelley": {
      "enable": true
    },
    "multi_asset": {
      "enable": true
    },
    "metadata": {
      "enable": true
    },
   "plututs": {
      "enable": true,
      "script_hashes":["34250edd1e9836f5378702fbf9416b709bc140e04f668cc355208518","67f33146617a5e61936081db3b2117cbf59bd2123748f58ac9678656", "bccf2a430c016bc960fbf31b02694011cd399d20da8882aac9d33611", "1b539b68e5389bdc3fe511c7694583e3755f58d4b38aef50384f4349"]
    },
    "governance": "enable",
    "offchain_pool_data": "enable",
    "json_type": "text"
  },I am seeing entries in  sancho_test_config_shelley_whitelist_plutus=# SELECT id, tx_id, hash, type, json
FROM script
WHERE hash NOT IN (
    '\x34250edd1e9836f5378702fbf9416b709bc140e04f668cc355208518', -- hash for 'timelock' - 1st entry in script_hashes
    '\x67f33146617a5e61936081db3b2117cbf59bd2123748f58ac9678656', -- hash for 'plutusV1' - 2nd entry in script_hashes
    '\xbccf2a430c016bc960fbf31b02694011cd399d20da8882aac9d33611', -- hash for 'plutusV2' - 3rd entry in script_hashes
    '\x1b539b68e5389bdc3fe511c7694583e3755f58d4b38aef50384f4349'  -- hash for 'plutusV3' - 4th entry in script_hashes
)
ORDER BY type, id;
 id | tx_id |                            hash                            |   type   |                                                                                                                                                          json                                                                                                                                                           
----+-------+------------------------------------------------------------+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  6 | 17132 | \xeeb006239ad2562556b736c3990f20968fcf89bb6d4f7292aaa0c30a | timelock | {"type": "atLeast", "scripts": [{"type": "sig", "keyHash": "ff750767ac2496a659b8325bf7064f92d969672d931ce0bc0290f6ba"}, {"type": "sig", "keyHash": "b7b9594ef50295739c95144238654413221df009f388cbf9b8ca73b7"}, {"type": "sig", "keyHash": "0253cc2bc1ed8176c675f454dd730fae5bfaa147b73924bde70d786a"}], "required": 2}
 24 | 28937 | \x6b0261483fa1fe471c6f941c2a917d75164cbd96a93a9e42f62efd1c | timelock | {"type": "sig", "keyHash": "6c6fe0b26f5bfa67ece9221683ea4e7ad19bdf14db8a9fae7b294b5e"}
  9 | 19924 | \x56b0f0b597150e619c76bed60683f3b1e42d7bc0685ed951b882bfc5 | plutusV2 | 
 10 | 19924 | \x86bff95ba20e9d1d1b34899a56d86bbacc9fed999260b27dcc92d128 | plutusV2 | 
 11 | 19927 | \x0689163fa0280c34d6e3d78657e2b06980be7ac77854f0dd5201bf05 | plutusV2 | 
 12 | 21241 | \x8275a6621fdb65e60c117fddd5f162d43adf6364c2645cd865bf5482 | plutusV2 | 
 14 | 24872 | \x04b8c921d534b5debae418ae679a83ec5e628b8e680af1a241788790 | plutusV2 |
... | 
| @ArturWieczorek thanks a lot for the very thorough reports 🙏 
 I wonder if it has something with stopping and restarting as I got to 301 last week and that was the tip but now we're on 306. So this morning I restarted and like you it was just not moving forward. I can't fully tell it's it's node not providing more data or something is going on on our side. Will investigate more. 
 So it's doesn't just whitelist against script hash, it also checks payment creds. Which was the existing functionality. I'm unsure as to the reasoning but it's functionality was kept. New code here | 
37b3270    to
    911028b      
    Compare
  
    911028b    to
    e285b01      
    Compare
  
    6b3e580    to
    1fc6558      
    Compare
  
    1fc6558    to
    c4f7e83      
    Compare
  
    | @ArturWieczorek could you double check this PR for me when you next get the chance? 👍 | 
| On    "insert_options": {
     "preset": "only_gov"
  }with: I ran into  [db-sync-node:Info:77] [2024-07-25 11:23:36.81 UTC] Starting epoch 393
[db-sync-node:Info:77] [2024-07-25 11:23:36.81 UTC] Insert Conway Block: epoch 393, slot 33955211, block 1698926, hash 2bbcce48b0580dd58831de1fb1349133ae130b37b0462a3d72728e07892119b6
[db-sync-node:Info:85] [2024-07-25 11:23:36.85 UTC] Asynchronously wrote a ledger snapshot to ledger-state/sancho_only_gov/33955151-163957d025-392.lstate in 0.039110436s.
[db-sync-node:Info:77] [2024-07-25 11:23:37.60 UTC] Inserted 0 EpochStake for EpochNo 394
[db-sync-node:Info:77] [2024-07-25 11:23:38.84 UTC] Insert Conway Block: epoch 393, slot 33976998, block 1700000, hash 48490c35b1cbd36546ec51ccd8b89954b23be572455cf4e2857b4797f550bb60
[db-sync-node:Info:77] [2024-07-25 11:23:45.53 UTC] Removing old epoch boundary file ledger-state/sancho_only_gov/33436789-a577f01423-386.lstate
[db-sync-node:Info:77] [2024-07-25 11:23:45.54 UTC] Removing 0 orphaned rewards
[db-sync-node:Warning:77] [2024-07-25 11:23:45.54 UTC] validateEpochRewards: rewards spendable in epoch 394 expected total of 3 but got 0
[db-sync-node:Info:77] [2024-07-25 11:23:45.54 UTC] Persistant SQL Statement Cache size is 44
[db-sync-node:Info:77] [2024-07-25 11:23:45.54 UTC] 
Cache Statistics:
  Stake Addresses: cache sizes: 0 and 0, hit rate: 0%, hits: 0, misses: 97
  Pools: cache size: 0, hit rate: 0%, hits: 0, misses: 1703194
  Datums: cache capacity: 250000, cache size: 0, hits: 0, misses: 0
  Multi Assets: cache capacity: 250000, cache size: 0, hits: 0, misses: 0
  Previous Block: hit rate: 49%, hits: 1703193, misses: 1703194
  TxId: cache size: 1390, cache capacity: 300000, hits: 0, misses: 0
[db-sync-node:Info:77] [2024-07-25 11:23:45.54 UTC] Starting epoch 394
[db-sync-node:Info:77] [2024-07-25 11:23:45.54 UTC] Insert Conway Block: epoch 394, slot 34041601, block 1703194, hash 2588a19fd69700dae6edaf56ee64116ccb19508b66faff27b8b840f096d042f1
[db-sync-node:Error:77] [2024-07-25 11:23:45.57 UTC] Error SNErrDefault: "resolveGovActionProposal.queryGovActionProposalId missing GovAction (TxKey {unTxKey = SqlBackendKey {unSqlBackendKey = 1283}}, 0)"
[db-sync-node:Info:77] [2024-07-25 11:23:45.57 UTC] Shutting down DB thread | 
| @Cmdv  Looks like  
 sancho_new_test_config_full=# select * from tx_cbor;
 id | tx_id | bytes 
----+-------+-------
(0 rows)EDIT: @Cmdv Looks like it is correct behaviour. According to docs: 
 | 
| @Cmdv This is my config - is it correct ?   "insert_options": {
    "multi_asset": {
      "enable": true,
      "policies": [
        "6c969320597b755454ff3653ad09725d590c570827a129aeb4385526",
        "065270479316f1d92e00f7f9f095ebeaac9d009c878dc35ce36d3404"
      ]
    }
  }and here are queries that show issue with filtering not happening. preview_13_3_0_0_ma_filter=# SELECT count(*) FROM multi_asset WHERE policy IN ('\x6c969320597b755454ff3653ad09725d590c570827a129aeb4385526', '\x065270479316f1d92e00f7f9f095ebeaac9d009c878dc35ce36d3404');
 count 
-------
    34
(1 row)
preview_13_3_0_0_ma_filter=# SELECT count(*) FROM multi_asset WHERE policy NOT IN ('\x6c969320597b755454ff3653ad09725d590c570827a129aeb4385526', '\x065270479316f1d92e00f7f9f095ebeaac9d009c878dc35ce36d3404');
 count 
-------
  1719
(1 row) | 
7b8ec05    to
    f64d632      
    Compare
  
    | @Cmdv So far it looks good:     "multi_asset": {
      "enable": true,
      "policies": [
        "3da414859ddd0955cf6694a20cbbc000d60c4cb49dcbf46ad543568a",
        "c70ef82c7ff32b5dda84e3519d8d2b8cbd29c1d2a93cc2e183797143"
      ]
    },and DB: sancho_test_policies=# select * from multi_asset;
 id |                           policy                           |                              name                              |                 fingerprint                  
----+------------------------------------------------------------+----------------------------------------------------------------+----------------------------------------------
  1 | \x3da414859ddd0955cf6694a20cbbc000d60c4cb49dcbf46ad543568a | \x37f9a73217d919ef97332211ade0afd42732e38b11577c82bbddfd9f2301 | asset16vajwx08w7wgwazxu9fe6lvvpxakgtxftmfc4w
  2 | \x3da414859ddd0955cf6694a20cbbc000d60c4cb49dcbf46ad543568a | \xe8a0e694ab11691ed96275dbb8e02c6ab7ae9af840e1e119830ae4e72300 | asset1uqx7htnvmw9az0qwvdhc5e06ep6qg535uwne2f
  3 | \x3da414859ddd0955cf6694a20cbbc000d60c4cb49dcbf46ad543568a | \x1c64222c787d10d13247d24b92563a28da34edcbdd918a1c280d02aa2300 | asset1jm3gee8mm3mmrep49c0qx7xmz0wugulx8fcs7l
  4 | \x3da414859ddd0955cf6694a20cbbc000d60c4cb49dcbf46ad543568a | \xeda822c2552732f57e749ed4d9c6056a0989884b12ac863301e83ecd2300 | asset129za0pu050u8ulhh0a08zdkwde8ued5cdgpcj2
  5 | \xc70ef82c7ff32b5dda84e3519d8d2b8cbd29c1d2a93cc2e183797143 | \x5e249e4b897fe25aea28119031cbb7931ccd4ae135caec5c90fabe3d2301 | asset146w8yfx0c5gue58v44nasvz7rx6c52wmapdprn
  6 | \xc70ef82c7ff32b5dda84e3519d8d2b8cbd29c1d2a93cc2e183797143 | \x5ea4fa27f12cda53293b37f721092eedad3f3221832c3a6c706e8d3a2300 | asset1enqrads9txxszt3kt4mx5sqwyq5ur8lne8yfak
  7 | \xc70ef82c7ff32b5dda84e3519d8d2b8cbd29c1d2a93cc2e183797143 | \x72f5eeb05dcacbab058506ef94443c0d9a513c61b6a4e442513692702301 | asset1en2qcllrneuh7fc26h3kcwfst3syz5kr204era
  8 | \xc70ef82c7ff32b5dda84e3519d8d2b8cbd29c1d2a93cc2e183797143 | \x0826e33c43dce633a67a41528d87352f7b940282d8e5a5a29f8c8abd2300 | asset13gddghj7j2y53tfdjlvcmfc762g56r6wfe26rr
(8 rows)VS database synced with "standard" config: sancho_normal=# select * from multi_asset where policy = '\x3da414859ddd0955cf6694a20cbbc000d60c4cb49dcbf46ad543568a' or policy = '\xc70ef82c7ff32b5dda84e3519d8d2b8cbd29c1d2a93cc2e183797143';
 id |                           policy                           |                              name                              |                 fingerprint                  
----+------------------------------------------------------------+----------------------------------------------------------------+----------------------------------------------
  1 | \x3da414859ddd0955cf6694a20cbbc000d60c4cb49dcbf46ad543568a | \x37f9a73217d919ef97332211ade0afd42732e38b11577c82bbddfd9f2301 | asset16vajwx08w7wgwazxu9fe6lvvpxakgtxftmfc4w
  2 | \x3da414859ddd0955cf6694a20cbbc000d60c4cb49dcbf46ad543568a | \xe8a0e694ab11691ed96275dbb8e02c6ab7ae9af840e1e119830ae4e72300 | asset1uqx7htnvmw9az0qwvdhc5e06ep6qg535uwne2f
  3 | \x3da414859ddd0955cf6694a20cbbc000d60c4cb49dcbf46ad543568a | \x1c64222c787d10d13247d24b92563a28da34edcbdd918a1c280d02aa2300 | asset1jm3gee8mm3mmrep49c0qx7xmz0wugulx8fcs7l
  4 | \x3da414859ddd0955cf6694a20cbbc000d60c4cb49dcbf46ad543568a | \xeda822c2552732f57e749ed4d9c6056a0989884b12ac863301e83ecd2300 | asset129za0pu050u8ulhh0a08zdkwde8ued5cdgpcj2
  5 | \xc70ef82c7ff32b5dda84e3519d8d2b8cbd29c1d2a93cc2e183797143 | \x5e249e4b897fe25aea28119031cbb7931ccd4ae135caec5c90fabe3d2301 | asset146w8yfx0c5gue58v44nasvz7rx6c52wmapdprn
  6 | \xc70ef82c7ff32b5dda84e3519d8d2b8cbd29c1d2a93cc2e183797143 | \x5ea4fa27f12cda53293b37f721092eedad3f3221832c3a6c706e8d3a2300 | asset1enqrads9txxszt3kt4mx5sqwyq5ur8lne8yfak
 38 | \xc70ef82c7ff32b5dda84e3519d8d2b8cbd29c1d2a93cc2e183797143 | \x72f5eeb05dcacbab058506ef94443c0d9a513c61b6a4e442513692702301 | asset1en2qcllrneuh7fc26h3kcwfst3syz5kr204era
 55 | \xc70ef82c7ff32b5dda84e3519d8d2b8cbd29c1d2a93cc2e183797143 | \x0826e33c43dce633a67a41528d87352f7b940282d8e5a5a29f8c8abd2300 | asset13gddghj7j2y53tfdjlvcmfc762g56r6wfe26rr
(8 rows) | 
f4954e2    to
    9d2590c      
    Compare
  
    | For     "shelley": {
      "enable": true,
      "stake_addresses": ["e092a56d8920af4451021a07a0d59f7d0d8a1f3359158c98e1d9cf2be4", "e003d205532089ad2f7816892e2ef42849b7b52788e41b3fd43a6e01cf"]
    },This does not seem right : sancho_test_address_filtering=# select * from stake_address;
 id |                           hash_raw                           |                               view                               | script_hash 
----+--------------------------------------------------------------+------------------------------------------------------------------+-------------
  1 | \xe092a56d8920af4451021a07a0d59f7d0d8a1f3359158c98e1d9cf2be4 | stake_test1uzf22mvfyzh5g5gzrgr6p4vl05xc58enty2cex8pm88jheqrpmt2u | 
  2 | \xe008f121b36abf2bcdd7daa2551c1e6653413a78c419e170d3319924d3 | stake_test1uqy0zgdnd2ljhnwhm23928q7vef5zwnccsv7zuxnxxvjf5c0swgr8 | 
  3 | \xe0e0d4c1ce0503af576155012f78f5601fe3593a93e4e7690b69813c3b | stake_test1ursdfswwq5p674mp25qj7784vq07xkf6j0jww6gtdxqncwcpy58kc | 
  4 | \xe072715e17c335384378239b830eca11de0f76f436c231337395a4acdf | stake_test1upe8zhshcv6nssmcywdcxrk2z80q7ah5xmprzvmnjkj2ehch9nrjd | 
  5 | \xe0c66ad2e4be6c8b6f4ff32b08be6d8b0c757782a6d116239a6903bf59 | stake_test1urrx45hyhekgkm607v4s30nd3vx82auz5mg3vgu6dypm7kgcnxs8z | 
  6 | \xe06d77fe1ae0aa1037e054657f1d91cd503efd0538f2566704d232c723 | stake_test1upkh0ls6uz4pqdlq23jh78v3e4gralg98re9vecy6gevwgcd32r5a | 
  7 | \xe0b17951d65a7ec99d008d3bc888b1986088b26c44792631bdbb29c8bd | stake_test1uzchj5wktflvn8gq35au3z93npsg3vnvg3ujvvdahv5u30gxuggdt | 
  8 | \xe09b632f3bc249952490afcd6557cd3be2f09f15637b097f24ae5bb093 | stake_test1uzdkxtemcfye2fys4lxk247d8030p8c4vdasjley4edmpycgmrxs2 | 
  9 | \xe0176b66617c48f76b3ee4a86e488b51e6969e0b5e81ed2147039159bf | stake_test1uqtkkenp03y0w6e7uj5xujyt28nfd8stt6q76g28qwg4n0c3tkc8s | 
 10 | \xe0c5ffc5c5445bb4cc9d8dc986768700959e7901e0916788d2512e08e9 | stake_test1urzll3w9g3dmfnya3hycva58qz2eu7gpuzgk0zxj2yhq36g466keq | 
 11 | \xe0c13582aec9a44fcc6d984be003c5058c660e1d2ff1370fd8b49ba73f | stake_test1urqntq4wexjylnrdnp97qq79qkxxvrsa9lcnwr7ckjd6w0cr04y4p | 
 12 | \xe0df1992dd82b1e2c3f1389ce800701cc49300de7ce71b7af57cb4feca | stake_test1ur03nykas2c79sl38zwwsqrsrnzfxqx70nn3k7h40j60ajsle7at7 | 
 13 | \xe01bf1e138f2f8beabc963c94cc28ee8ed4b41744601f2edaf50b21efd | stake_test1uqdlrcfc7tuta27fv0y5es5wark5kst5gcql9md02zepalg9yxxuz | 
 14 | \xe0ccf6ffabac700363af242c39fee6228e3a53b5fc7dae4c5675af8904 | stake_test1urx0dlat43cqxca0yskrnlhxy28r55a4l376unzkwkhcjpqeqdp0r | 
 15 | \xe07b0a8d15d72c6d497a43ae127b3d03b1dc34e1f18fcd86acd5457d77 | stake_test1upas4rg46ukx6jt6gwhpy7eaqwcacd8p7x8ump4v64zh6acrjhemn | 
 16 | \xe0992d25f1234f161763f5faaeea857baf06de242547bde90895019cde | stake_test1uzvj6f03yd83v9mr7ha2a6590whsdh3yy4rmm6ggj5qeehstm86lq | 
 17 | \xe031e66d448f7fe215f77ab30eb7409585f9196c7037487130b8c81eaa | stake_test1uqc7vm2y3al7y90h02esad6qjkzljxtvwqm5sufshrypa2sulwjzz | 
 18 | \xe0480b7d18a26a3d1c8097038cc2268066f88d09266ef771b17f375afd | stake_test1upyqklgc5f4r68yqjupces3xspn03rgfyeh0wud30um44lg583k50 | 
 19 | \xe023066388b51eefb6f405ae8bae44d11e1efc15062748b628f5c8be0b | stake_test1uq3svcugk50wldh5qkhghtjy6y0palq4qcn53d3g7hytuzcav7rp0 | 
 20 | \xe0e7cd215fa76c70b9987c4f983766ee63dc8e31cdd8cbd491a2807426 | stake_test1urnu6g2l5ak8pwvc038esdmxae3aer33ehvvh4y352q8gfszxrpxq | 
 21 | \xe08141d8651e0139b3bfc9c31f76d885431f83957329d75e0e53a65c2a | stake_test1uzq5rkr9rcqnnvale8p37akcs4p3lqu4wv5awhsw2wn9c2sey6zya | 
 22 | \xe08fb4b1cbf9dd257d83859ab5833541ade22c3d01ca02e22d5500ed10 | stake_test1uz8mfvwtl8wj2lvrskdttqe4gxk7ytpaq89q9c3d25qw6yqj5c6wh | 
 23 | \xe003d205532089ad2f7816892e2ef42849b7b52788e41b3fd43a6e01cf | stake_test1uqpayp2nyzy66tmcz6yjuth59pym0df83rjpk0758fhqrnc8dx6fl | 
 24 | \xe06f7b4bb2492b2ca9ebd5da5cc28ff453b35c71997a643957a7a1de9d | stake_test1uphhkjajfy4je20t6hd9es5073fmxhr3n9axgw2h57saa8gpvk6j8 | 
 25 | \xe03bdb8a34e61405507e0321511bf4afbcfa40a8ad87f97112a5777145 | stake_test1uqaahz35uc2q25r7qvs4zxl547705s9g4krljugj54mhz3gk5lc36 | 
 26 | \xe00062f6f5b4951b27d1c8948b7f5f07a434f6f7e5d629380e1648946c | stake_test1uqqx9ah4kj23kf73ez2gkl6lq7jrfahhuhtzjwqwzeyfgmq380acx | 
 27 | \xe0e356224fb5a313bbe98353e4ccd02f396baaafafbf85393bdbf680ad | stake_test1ur34vgj0kk338wlfsdf7fnxs9uukh24047lc2wfmm0mgptgq7849c | 
 28 | \xe0609ffae0a55723de7a69397c77f29c19bf0ef58f2c782877b8abbba1 | stake_test1upsfl7hq54tj8hn6dyuhcaljnsvm7rh43uk8s2rhhz4mhgg7slyup | 
 29 | \xe010ae59b652be66c7fd26a9dce502e56f3a64d77a531465383b1e30be | stake_test1uqg2ukdk22lxd3lay65aeegzu4hn5exh0ff3gefc8v0rp0sgkn67g | 
 30 | \xe01cf186c852a7e3644d84fdbe03ef221bd7647470c341bd11e9c90057 | stake_test1uqw0rpkg22n7xezdsn7muql0ygdawer5wrp5r0g3a8ysq4c72ax2e | 
 31 | \xe021af6d5199f45b1ea76320e598ccbf4e4d38c842e64f53e3280f2a58 | stake_test1uqs67m23n869k848vvswtxxvha8y6wxggtny75lr9q8j5kqgqsq3u | 
 32 | \xe09f01ce7841ab8eb33bb1baf48462cc9565aba9e85714b4c543827c18 | stake_test1uz0srnncgx4cavemkxa0fprzej2kt2afapt3fdx9gwp8cxqa5qrpn | 
(32 rows)
sancho_test_address_filtering=# select * from stake_registration;
 id | addr_id | cert_index | epoch_no | tx_id | deposit 
----+---------+------------+----------+-------+---------
  1 |       1 |          0 |      393 |  1352 | 2000000
  2 |      23 |          0 |      437 | 23126 | 2000000
(2 rows)
Should not  | 
| Looks like there might be issue with  sancho_test_address_filtering=# select * from reward_rest LIMIT 3;
 addr_id |      type       |    amount    | spendable_epoch | earned_epoch 
---------+-----------------+--------------+-----------------+--------------
       2 | proposal_refund | 100000000000 |             395 |          394
       1 | proposal_refund | 100000000000 |             411 |          410
       3 | proposal_refund | 100000000000 |             412 |          411
(3 rows)
sancho_test_address_filtering=# select * from stake_registration;
 id | addr_id | cert_index | epoch_no | tx_id | deposit 
----+---------+------------+----------+-------+---------
  1 |       1 |          0 |      392 |  1238 | 2000000
(1 row)but in filter there is only 1 address:   "insert_options": {
    "shelley": {
      "enable": true,
      "stake_addresses": ["e0e0d4c1ce0503af576155012f78f5601fe3593a93e4e7690b69813c3b"]
    },
  }, | 
9d2590c    to
    2227452      
    Compare
  
    
Description
This PR fixes #1600 adding more whitelists to the configs
The most complex whitelist was the Shelley Stake Address as it stored in the table
stake_address.The functionality is if the stake address doesn't match in the whitelist on any of the inserts into the following tables, then we don't insert at all:
Checklist
fourmoluon version 0.10.1.0 (which can be run withscripts/fourmolize.sh)Migrations
If there is a breaking change, especially a big one, please add a justification here. Please elaborate
more what the migration achieves, what it cannot achieve or why a migration is not possible.