openethereum/ethcore/src/state_db.rs

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

526 lines
18 KiB
Rust
Raw Normal View History

2020-09-22 14:53:52 +02:00
// Copyright 2015-2020 Parity Technologies (UK) Ltd.
// This file is part of OpenEthereum.
2020-09-22 14:53:52 +02:00
// OpenEthereum is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
2020-09-22 14:53:52 +02:00
// OpenEthereum is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
2020-09-22 14:53:52 +02:00
// along with OpenEthereum. If not, see <http://www.gnu.org/licenses/>.
`Client` refactoring (#7038) * Improves `BestBlock` comment * Improves `TraceDB` comment * Improves `journaldb::Algorithm` comment. Probably the whole enum should be renamed to `Strategy` or something alike. * Comments some of the `Client`'s fields * Deglobs client imports * Fixes comments * Extracts `import_lock` to `Importer` struct * Extracts `verifier` to `Importer` struct * Extracts `block_queue` to `Importer` struct * Extracts `miner` to `Importer` struct * Extracts `ancient_verifier` to `Importer` struct * Extracts `rng` to `Importer` struct * Extracts `import_old_block` to `Importer` struct * Adds `Nonce` trait * Adds `Balance` trait * Adds `ChainInfo` trait * Fixes imports for tests using `chain_info` method * Adds `BlockInfo` trait * Adds more `ChainInfo` imports * Adds `BlockInfo` imports * Adds `ReopenBlock` trait * Adds `PrepareOpenBlock` trait * Fixes import in tests * Adds `CallContract` trait * Fixes imports in tests using `call_contract` method * Adds `TransactionInfo` trait * Adds `RegistryInfo` trait * Fixes imports in tests using `registry_address` method * Adds `ScheduleInfo` trait * Adds `ImportSealedBlock` trait * Fixes imports in test using `import_sealed_block` method * Adds `BroadcastProposalBlock` trait * Migrates `Miner` to static dispatch * Fixes tests * Moves `calculate_enacted_retracted` to `Importer` * Moves import-related methods to `Importer` * Removes redundant `import_old_block` wrapper * Extracts `import_block*` into separate trait * Fixes tests * Handles `Pending` in `LightFetch` * Handles `Pending` in filters * Handles `Pending` in `ParityClient` * Handles `Pending` in `EthClient` * Removes `BlockId::Pending`, partly refactors dependent code * Adds `StateInfo` trait * Exports `StateOrBlock` and `BlockChain` types from `client` module * Refactors `balance` RPC using generic API * Refactors `storage_at` RPC using generic API * Makes `MinerService::pending_state`'s return type dynamic * Adds `StateOrBlock` and `BlockChain` types * Adds impl of `client::BlockChain` for `Client` * Exports `StateInfo` trait from `client` module * Missing `self` use To be fixed up to "Adds impl of `client::BlockChain` for `Client`" * Adds `number_to_id` and refactors dependent RPC methods * Refactors `code_at` using generic API * Adds `StateClient` trait * Refactors RPC to use `StateClient` trait * Reverts `client::BlockChain` trait stuff, refactors methods to accept `StateOrBlock` * Refactors TestClient * Adds helper function `block_number_to_id` * Uses `block_number_to_id` instead of local function * Handles `Pending` in `list_accounts` and `list_storage_keys` * Attempt to use associated types for state instead of trait objects * Simplifies `state_at_beginning` * Extracts `call` and `call_many` into separate trait * Refactors `build_last_hashes` to accept reference * Exports `Call` type from the module * Refactors `call` and `call_many` to accept state and header * Exports `state_at` in `StateClient` * Exports `pending_block_header` from `MinerService` * Refactors RPC `call` method using new API * Adds missing parentheses * Refactors `parity::call` to use new call API * Update .gitlab-ci.yml fix gitlab lint * Fixes error handling * Refactors `traces::call` and `call_many` to use new call API * Refactors `call_contract` * Refactors `block_header` * Refactors internal RPC method `block` * Moves `estimate_gas` to `Call` trait, refactors parameters * Refactors `estimate_gas` in RPC * Refactors `uncle` * Refactors RPC `transaction` * Covers missing branches * Makes it all compile, fixes compiler grumbles * Adds casts in `blockchain` module * Fixes `PendingBlock` tests, work on `MinerService` * Adds test stubs for StateClient and EngineInfo * Makes `state_db` public * Adds missing impls for `TestBlockChainClient` * Adds trait documentation * Adds missing docs to the `state_db` module * Fixes trivial compilation errors * Moves `code_hash` method to a `BlockInfo` trait * Refactors `Verifier` to be generic over client * Refactors `TransactionFilter` to be generic over client * Refactors `Miner` and `Client` to reflect changes in verifier and txfilter API * Moves `ServiceTransactionChecker` back to `ethcore` * Fixes trait bounds in `Miner` API * Fixes `Client` * Fixes lifetime bound in `FullFamilyParams` * Adds comments to `FullFamilyParams` * Fixes imports in `ethcore` * Fixes BlockNumber handling in `code_at` and `replay_block_transactions` * fix compile issues * First step to redundant trait merge * Fixes compilation error in RPC tests * Adds mock `State` as a stub for `TestClient` * Handles `StateOrBlock::State` in `TestBlockChainClient::balance` * Fixes `transaction_count` RPC * Fixes `transaction_count` * Moves `service_transaction.json` to the `contracts` subfolder * Fixes compilation errors in tests * Refactors client to use `AccountData` * Refactors client to use `BlockChain` * Refactors miner to use aggregate traits * Adds `SealedBlockImporter` trait * Refactors miner to use `SealedBlockImporter` trait * Removes unused imports * Simplifies `RegistryInfo::registry_address` * Fixes indentation * Removes commented out trait bound
2018-03-03 18:42:13 +01:00
//! State database abstraction. For more info, see the doc for `StateDB`
use std::{
collections::{BTreeMap, HashSet, VecDeque},
io,
sync::Arc,
};
use ethereum_types::{Address, H256};
use hash_db::HashDB;
use journaldb::JournalDB;
use keccak_hasher::KeccakHasher;
2020-09-22 12:41:04 +02:00
use kvdb::{DBTransaction, DBValue};
use lru_cache::LruCache;
use memory_cache::MemoryLruCache;
use parking_lot::Mutex;
use types::BlockNumber;
use state::{self, Account};
2016-10-14 14:44:11 +02:00
const STATE_CACHE_BLOCKS: usize = 12;
// The percentage of supplied cache size to go to accounts.
const ACCOUNT_CACHE_RATIO: usize = 90;
2016-10-11 19:37:31 +02:00
/// Shared canonical state cache.
struct AccountCache {
/// DB Account cache. `None` indicates that account is known to be missing.
2016-10-07 00:28:42 +02:00
// When changing the type of the values here, be sure to update `mem_used` and
// `new`.
accounts: LruCache<Address, Option<Account>>,
2016-10-11 19:37:31 +02:00
/// Information on the modifications in recently committed blocks; specifically which addresses
/// changed in which block. Ordered by block number.
modifications: VecDeque<BlockChanges>,
}
/// Buffered account cache item.
struct CacheQueueItem {
/// Account address.
address: Address,
/// Acccount data or `None` if account does not exist.
account: SyncAccount,
2016-10-11 19:37:31 +02:00
/// Indicates that the account was modified before being
/// added to the cache.
modified: bool,
}
#[derive(Debug)]
/// Accumulates a list of accounts changed in a block.
struct BlockChanges {
/// Block number.
number: BlockNumber,
/// Block hash.
hash: H256,
/// Parent block hash.
parent: H256,
/// A set of modified account addresses.
accounts: HashSet<Address>,
/// Block is part of the canonical chain.
is_canon: bool,
}
/// State database abstraction.
2016-10-11 19:37:31 +02:00
/// Manages shared global state cache which reflects the canonical
/// state as it is on the disk. All the entries in the cache are clean.
/// A clone of `StateDB` may be created as canonical or not.
2016-10-11 19:37:31 +02:00
/// For canonical clones local cache is accumulated and applied
/// in `sync_cache`
/// For non-canonical clones local cache is dropped.
///
/// Global cache propagation.
/// After a `State` object has been committed to the trie it
/// propagates its local cache into the `StateDB` local cache
/// using `add_to_account_cache` function.
/// Then, after the block has been added to the chain the local cache in the
/// `StateDB` is propagated into the global cache.
pub struct StateDB {
2016-10-11 19:37:31 +02:00
/// Backing database.
2020-07-29 10:36:15 +02:00
db: Box<dyn JournalDB>,
2016-10-11 19:37:31 +02:00
/// Shared canonical state cache.
account_cache: Arc<Mutex<AccountCache>>,
/// DB Code cache. Maps code hashes to shared bytes.
code_cache: Arc<Mutex<MemoryLruCache<H256, Arc<Vec<u8>>>>>,
2016-10-11 19:37:31 +02:00
/// Local dirty cache.
local_cache: Vec<CacheQueueItem>,
2016-10-07 00:28:42 +02:00
cache_size: usize,
2016-10-11 19:37:31 +02:00
/// Hash of the block on top of which this instance was created or
/// `None` if cache is disabled
parent_hash: Option<H256>,
/// Hash of the committing block or `None` if not committed yet.
commit_hash: Option<H256>,
/// Number of the committing block or `None` if not committed yet.
commit_number: Option<BlockNumber>,
}
impl StateDB {
2016-10-07 00:28:42 +02:00
/// Create a new instance wrapping `JournalDB` and the maximum allowed size
/// of the LRU cache in bytes. Actual used memory may (read: will) be higher due to bookkeeping.
// TODO: make the cache size actually accurate by moving the account storage cache
// into the `AccountCache` structure as its own `LruCache<(Address, H256), H256>`.
2020-07-29 10:36:15 +02:00
pub fn new(db: Box<dyn JournalDB>, cache_size: usize) -> StateDB {
let acc_cache_size = cache_size * ACCOUNT_CACHE_RATIO / 100;
let code_cache_size = cache_size - acc_cache_size;
let cache_items = acc_cache_size / ::std::mem::size_of::<Option<Account>>();
2020-08-05 06:08:03 +02:00
StateDB {
db: db,
account_cache: Arc::new(Mutex::new(AccountCache {
accounts: LruCache::new(cache_items),
modifications: VecDeque::new(),
})),
code_cache: Arc::new(Mutex::new(MemoryLruCache::new(code_cache_size))),
local_cache: Vec::new(),
2016-10-07 00:28:42 +02:00
cache_size: cache_size,
parent_hash: None,
commit_hash: None,
commit_number: None,
2020-08-05 06:08:03 +02:00
}
}
2020-08-05 06:08:03 +02:00
/// Journal all recent operations under the given era and ID.
pub fn journal_under(
&mut self,
batch: &mut DBTransaction,
now: u64,
id: &H256,
) -> io::Result<u32> {
let records = self.db.journal_under(batch, now, id)?;
2016-10-11 19:37:31 +02:00
self.commit_hash = Some(id.clone());
self.commit_number = Some(now);
Ok(records)
}
2020-08-05 06:08:03 +02:00
/// Mark a given candidate from an ancient era as canonical, enacting its removals from the
/// backing database and reverting any non-canonical historical commit's insertions.
pub fn mark_canonical(
&mut self,
batch: &mut DBTransaction,
end_era: u64,
canon_id: &H256,
) -> io::Result<u32> {
self.db.mark_canonical(batch, end_era, canon_id)
}
2020-08-05 06:08:03 +02:00
2016-10-11 19:37:31 +02:00
/// Propagate local cache into the global cache and synchonize
/// the global cache with the best block state.
/// This function updates the global cache by removing entries
/// that are invalidated by chain reorganization. `sync_cache`
/// should be called after the block has been committed and the
/// blockchain route has ben calculated.
pub fn sync_cache(&mut self, enacted: &[H256], retracted: &[H256], is_best: bool) {
trace!(
"sync_cache id = (#{:?}, {:?}), parent={:?}, best={}",
self.commit_number,
self.commit_hash,
self.parent_hash,
is_best
);
let mut cache = self.account_cache.lock();
2017-10-15 15:10:20 +02:00
let cache = &mut *cache;
2020-08-05 06:08:03 +02:00
2016-10-11 19:37:31 +02:00
// Purge changes from re-enacted and retracted blocks.
// Filter out commiting block if any.
let mut clear = false;
for block in enacted
.iter()
.filter(|h| self.commit_hash.as_ref().map_or(true, |p| *h != p))
{
clear = clear || {
if let Some(ref mut m) = cache.modifications.iter_mut().find(|m| &m.hash == block) {
2016-10-11 19:37:31 +02:00
trace!("Reverting enacted block {:?}", block);
m.is_canon = true;
for a in &m.accounts {
trace!("Reverting enacted address {:?}", a);
cache.accounts.remove(a);
}
false
} else {
true
}
};
}
2020-08-05 06:08:03 +02:00
2016-10-11 19:37:31 +02:00
for block in retracted {
clear = clear || {
if let Some(ref mut m) = cache.modifications.iter_mut().find(|m| &m.hash == block) {
2016-10-11 19:37:31 +02:00
trace!("Retracting block {:?}", block);
m.is_canon = false;
for a in &m.accounts {
trace!("Retracted address {:?}", a);
cache.accounts.remove(a);
}
false
} else {
true
}
};
}
if clear {
// We don't know anything about the block; clear everything
trace!("Wiping cache");
cache.accounts.clear();
cache.modifications.clear();
}
2020-08-05 06:08:03 +02:00
2016-10-11 19:37:31 +02:00
// Propagate cache only if committing on top of the latest canonical state
// blocks are ordered by number and only one block with a given number is marked as canonical
// (contributed to canonical state cache)
if let (Some(ref number), Some(ref hash), Some(ref parent)) =
(self.commit_number, self.commit_hash, self.parent_hash)
{
if cache.modifications.len() == STATE_CACHE_BLOCKS {
cache.modifications.pop_back();
}
let mut modifications = HashSet::new();
trace!("committing {} cache entries", self.local_cache.len());
for account in self.local_cache.drain(..) {
if account.modified {
modifications.insert(account.address.clone());
}
if is_best {
let acc = account.account.0;
2016-10-11 19:37:31 +02:00
if let Some(&mut Some(ref mut existing)) =
cache.accounts.get_mut(&account.address)
{
2018-01-02 09:43:08 +01:00
if let Some(new) = acc {
2016-10-11 19:37:31 +02:00
if account.modified {
existing.overwrite_with(new);
}
continue;
}
2020-08-05 06:08:03 +02:00
}
2018-01-02 09:43:08 +01:00
cache.accounts.insert(account.address, acc);
2016-10-11 19:37:31 +02:00
}
2020-08-05 06:08:03 +02:00
}
2016-10-11 19:37:31 +02:00
// Save modified accounts. These are ordered by the block number.
let block_changes = BlockChanges {
accounts: modifications,
number: *number,
hash: hash.clone(),
is_canon: is_best,
parent: parent.clone(),
};
let insert_at = cache
.modifications
.iter()
.enumerate()
.find(|&(_, m)| m.number < *number)
.map(|(i, _)| i);
2016-10-11 19:37:31 +02:00
trace!("inserting modifications at {:?}", insert_at);
if let Some(insert_at) = insert_at {
cache.modifications.insert(insert_at, block_changes);
} else {
cache.modifications.push_back(block_changes);
}
2020-08-05 06:08:03 +02:00
}
}
`Client` refactoring (#7038) * Improves `BestBlock` comment * Improves `TraceDB` comment * Improves `journaldb::Algorithm` comment. Probably the whole enum should be renamed to `Strategy` or something alike. * Comments some of the `Client`'s fields * Deglobs client imports * Fixes comments * Extracts `import_lock` to `Importer` struct * Extracts `verifier` to `Importer` struct * Extracts `block_queue` to `Importer` struct * Extracts `miner` to `Importer` struct * Extracts `ancient_verifier` to `Importer` struct * Extracts `rng` to `Importer` struct * Extracts `import_old_block` to `Importer` struct * Adds `Nonce` trait * Adds `Balance` trait * Adds `ChainInfo` trait * Fixes imports for tests using `chain_info` method * Adds `BlockInfo` trait * Adds more `ChainInfo` imports * Adds `BlockInfo` imports * Adds `ReopenBlock` trait * Adds `PrepareOpenBlock` trait * Fixes import in tests * Adds `CallContract` trait * Fixes imports in tests using `call_contract` method * Adds `TransactionInfo` trait * Adds `RegistryInfo` trait * Fixes imports in tests using `registry_address` method * Adds `ScheduleInfo` trait * Adds `ImportSealedBlock` trait * Fixes imports in test using `import_sealed_block` method * Adds `BroadcastProposalBlock` trait * Migrates `Miner` to static dispatch * Fixes tests * Moves `calculate_enacted_retracted` to `Importer` * Moves import-related methods to `Importer` * Removes redundant `import_old_block` wrapper * Extracts `import_block*` into separate trait * Fixes tests * Handles `Pending` in `LightFetch` * Handles `Pending` in filters * Handles `Pending` in `ParityClient` * Handles `Pending` in `EthClient` * Removes `BlockId::Pending`, partly refactors dependent code * Adds `StateInfo` trait * Exports `StateOrBlock` and `BlockChain` types from `client` module * Refactors `balance` RPC using generic API * Refactors `storage_at` RPC using generic API * Makes `MinerService::pending_state`'s return type dynamic * Adds `StateOrBlock` and `BlockChain` types * Adds impl of `client::BlockChain` for `Client` * Exports `StateInfo` trait from `client` module * Missing `self` use To be fixed up to "Adds impl of `client::BlockChain` for `Client`" * Adds `number_to_id` and refactors dependent RPC methods * Refactors `code_at` using generic API * Adds `StateClient` trait * Refactors RPC to use `StateClient` trait * Reverts `client::BlockChain` trait stuff, refactors methods to accept `StateOrBlock` * Refactors TestClient * Adds helper function `block_number_to_id` * Uses `block_number_to_id` instead of local function * Handles `Pending` in `list_accounts` and `list_storage_keys` * Attempt to use associated types for state instead of trait objects * Simplifies `state_at_beginning` * Extracts `call` and `call_many` into separate trait * Refactors `build_last_hashes` to accept reference * Exports `Call` type from the module * Refactors `call` and `call_many` to accept state and header * Exports `state_at` in `StateClient` * Exports `pending_block_header` from `MinerService` * Refactors RPC `call` method using new API * Adds missing parentheses * Refactors `parity::call` to use new call API * Update .gitlab-ci.yml fix gitlab lint * Fixes error handling * Refactors `traces::call` and `call_many` to use new call API * Refactors `call_contract` * Refactors `block_header` * Refactors internal RPC method `block` * Moves `estimate_gas` to `Call` trait, refactors parameters * Refactors `estimate_gas` in RPC * Refactors `uncle` * Refactors RPC `transaction` * Covers missing branches * Makes it all compile, fixes compiler grumbles * Adds casts in `blockchain` module * Fixes `PendingBlock` tests, work on `MinerService` * Adds test stubs for StateClient and EngineInfo * Makes `state_db` public * Adds missing impls for `TestBlockChainClient` * Adds trait documentation * Adds missing docs to the `state_db` module * Fixes trivial compilation errors * Moves `code_hash` method to a `BlockInfo` trait * Refactors `Verifier` to be generic over client * Refactors `TransactionFilter` to be generic over client * Refactors `Miner` and `Client` to reflect changes in verifier and txfilter API * Moves `ServiceTransactionChecker` back to `ethcore` * Fixes trait bounds in `Miner` API * Fixes `Client` * Fixes lifetime bound in `FullFamilyParams` * Adds comments to `FullFamilyParams` * Fixes imports in `ethcore` * Fixes BlockNumber handling in `code_at` and `replay_block_transactions` * fix compile issues * First step to redundant trait merge * Fixes compilation error in RPC tests * Adds mock `State` as a stub for `TestClient` * Handles `StateOrBlock::State` in `TestBlockChainClient::balance` * Fixes `transaction_count` RPC * Fixes `transaction_count` * Moves `service_transaction.json` to the `contracts` subfolder * Fixes compilation errors in tests * Refactors client to use `AccountData` * Refactors client to use `BlockChain` * Refactors miner to use aggregate traits * Adds `SealedBlockImporter` trait * Refactors miner to use `SealedBlockImporter` trait * Removes unused imports * Simplifies `RegistryInfo::registry_address` * Fixes indentation * Removes commented out trait bound
2018-03-03 18:42:13 +01:00
/// Conversion method to interpret self as `HashDB` reference
2020-07-29 10:36:15 +02:00
pub fn as_hash_db(&self) -> &dyn HashDB<KeccakHasher, DBValue> {
self.db.as_hash_db()
}
2020-08-05 06:08:03 +02:00
`Client` refactoring (#7038) * Improves `BestBlock` comment * Improves `TraceDB` comment * Improves `journaldb::Algorithm` comment. Probably the whole enum should be renamed to `Strategy` or something alike. * Comments some of the `Client`'s fields * Deglobs client imports * Fixes comments * Extracts `import_lock` to `Importer` struct * Extracts `verifier` to `Importer` struct * Extracts `block_queue` to `Importer` struct * Extracts `miner` to `Importer` struct * Extracts `ancient_verifier` to `Importer` struct * Extracts `rng` to `Importer` struct * Extracts `import_old_block` to `Importer` struct * Adds `Nonce` trait * Adds `Balance` trait * Adds `ChainInfo` trait * Fixes imports for tests using `chain_info` method * Adds `BlockInfo` trait * Adds more `ChainInfo` imports * Adds `BlockInfo` imports * Adds `ReopenBlock` trait * Adds `PrepareOpenBlock` trait * Fixes import in tests * Adds `CallContract` trait * Fixes imports in tests using `call_contract` method * Adds `TransactionInfo` trait * Adds `RegistryInfo` trait * Fixes imports in tests using `registry_address` method * Adds `ScheduleInfo` trait * Adds `ImportSealedBlock` trait * Fixes imports in test using `import_sealed_block` method * Adds `BroadcastProposalBlock` trait * Migrates `Miner` to static dispatch * Fixes tests * Moves `calculate_enacted_retracted` to `Importer` * Moves import-related methods to `Importer` * Removes redundant `import_old_block` wrapper * Extracts `import_block*` into separate trait * Fixes tests * Handles `Pending` in `LightFetch` * Handles `Pending` in filters * Handles `Pending` in `ParityClient` * Handles `Pending` in `EthClient` * Removes `BlockId::Pending`, partly refactors dependent code * Adds `StateInfo` trait * Exports `StateOrBlock` and `BlockChain` types from `client` module * Refactors `balance` RPC using generic API * Refactors `storage_at` RPC using generic API * Makes `MinerService::pending_state`'s return type dynamic * Adds `StateOrBlock` and `BlockChain` types * Adds impl of `client::BlockChain` for `Client` * Exports `StateInfo` trait from `client` module * Missing `self` use To be fixed up to "Adds impl of `client::BlockChain` for `Client`" * Adds `number_to_id` and refactors dependent RPC methods * Refactors `code_at` using generic API * Adds `StateClient` trait * Refactors RPC to use `StateClient` trait * Reverts `client::BlockChain` trait stuff, refactors methods to accept `StateOrBlock` * Refactors TestClient * Adds helper function `block_number_to_id` * Uses `block_number_to_id` instead of local function * Handles `Pending` in `list_accounts` and `list_storage_keys` * Attempt to use associated types for state instead of trait objects * Simplifies `state_at_beginning` * Extracts `call` and `call_many` into separate trait * Refactors `build_last_hashes` to accept reference * Exports `Call` type from the module * Refactors `call` and `call_many` to accept state and header * Exports `state_at` in `StateClient` * Exports `pending_block_header` from `MinerService` * Refactors RPC `call` method using new API * Adds missing parentheses * Refactors `parity::call` to use new call API * Update .gitlab-ci.yml fix gitlab lint * Fixes error handling * Refactors `traces::call` and `call_many` to use new call API * Refactors `call_contract` * Refactors `block_header` * Refactors internal RPC method `block` * Moves `estimate_gas` to `Call` trait, refactors parameters * Refactors `estimate_gas` in RPC * Refactors `uncle` * Refactors RPC `transaction` * Covers missing branches * Makes it all compile, fixes compiler grumbles * Adds casts in `blockchain` module * Fixes `PendingBlock` tests, work on `MinerService` * Adds test stubs for StateClient and EngineInfo * Makes `state_db` public * Adds missing impls for `TestBlockChainClient` * Adds trait documentation * Adds missing docs to the `state_db` module * Fixes trivial compilation errors * Moves `code_hash` method to a `BlockInfo` trait * Refactors `Verifier` to be generic over client * Refactors `TransactionFilter` to be generic over client * Refactors `Miner` and `Client` to reflect changes in verifier and txfilter API * Moves `ServiceTransactionChecker` back to `ethcore` * Fixes trait bounds in `Miner` API * Fixes `Client` * Fixes lifetime bound in `FullFamilyParams` * Adds comments to `FullFamilyParams` * Fixes imports in `ethcore` * Fixes BlockNumber handling in `code_at` and `replay_block_transactions` * fix compile issues * First step to redundant trait merge * Fixes compilation error in RPC tests * Adds mock `State` as a stub for `TestClient` * Handles `StateOrBlock::State` in `TestBlockChainClient::balance` * Fixes `transaction_count` RPC * Fixes `transaction_count` * Moves `service_transaction.json` to the `contracts` subfolder * Fixes compilation errors in tests * Refactors client to use `AccountData` * Refactors client to use `BlockChain` * Refactors miner to use aggregate traits * Adds `SealedBlockImporter` trait * Refactors miner to use `SealedBlockImporter` trait * Removes unused imports * Simplifies `RegistryInfo::registry_address` * Fixes indentation * Removes commented out trait bound
2018-03-03 18:42:13 +01:00
/// Conversion method to interpret self as mutable `HashDB` reference
2020-07-29 10:36:15 +02:00
pub fn as_hash_db_mut(&mut self) -> &mut dyn HashDB<KeccakHasher, DBValue> {
self.db.as_hash_db_mut()
}
2020-08-05 06:08:03 +02:00
/// Clone the database.
pub fn boxed_clone(&self) -> StateDB {
StateDB {
db: self.db.boxed_clone(),
account_cache: self.account_cache.clone(),
code_cache: self.code_cache.clone(),
2016-10-11 19:37:31 +02:00
local_cache: Vec::new(),
2016-10-07 00:28:42 +02:00
cache_size: self.cache_size,
2016-10-11 19:37:31 +02:00
parent_hash: None,
commit_hash: None,
commit_number: None,
}
2020-08-05 06:08:03 +02:00
}
/// Clone the database for a canonical state.
2016-10-11 19:37:31 +02:00
pub fn boxed_clone_canon(&self, parent: &H256) -> StateDB {
StateDB {
db: self.db.boxed_clone(),
account_cache: self.account_cache.clone(),
code_cache: self.code_cache.clone(),
2016-10-11 19:37:31 +02:00
local_cache: Vec::new(),
2016-10-07 00:28:42 +02:00
cache_size: self.cache_size,
2016-10-11 19:37:31 +02:00
parent_hash: Some(parent.clone()),
commit_hash: None,
commit_number: None,
2020-08-05 06:08:03 +02:00
}
}
2020-08-05 06:08:03 +02:00
/// Check if pruning is enabled on the database.
pub fn is_pruned(&self) -> bool {
self.db.is_pruned()
}
2020-08-05 06:08:03 +02:00
/// Heap size used.
pub fn get_sizes(&self, sizes: &mut BTreeMap<String, usize>) {
self.db.get_sizes(sizes);
sizes.insert(
String::from("account_cache_len"),
self.account_cache.lock().accounts.len(),
);
sizes.insert(
String::from("code_cache_size"),
self.code_cache.lock().current_size(),
);
}
2020-08-05 06:08:03 +02:00
/// Returns underlying `JournalDB`.
2020-07-29 10:36:15 +02:00
pub fn journal_db(&self) -> &dyn JournalDB {
&*self.db
}
2020-08-05 06:08:03 +02:00
2016-10-07 00:28:42 +02:00
/// Query how much memory is set aside for the accounts cache (in bytes).
pub fn cache_size(&self) -> usize {
self.cache_size
}
2020-08-05 06:08:03 +02:00
2016-10-11 19:37:31 +02:00
/// Check if the account can be returned from cache by matching current block parent hash against canonical
/// state and filtering out account modified in later blocks.
fn is_allowed(
addr: &Address,
parent_hash: &H256,
modifications: &VecDeque<BlockChanges>,
) -> bool {
2016-10-11 19:37:31 +02:00
if modifications.is_empty() {
return true;
}
// Ignore all accounts modified in later blocks
// Modifications contains block ordered by the number
// We search for our parent in that list first and then for
// all its parent until we hit the canonical block,
// checking against all the intermediate modifications.
let mut parent = parent_hash;
for m in modifications {
2016-10-11 19:37:31 +02:00
if &m.hash == parent {
if m.is_canon {
return true;
}
parent = &m.parent;
}
if m.accounts.contains(addr) {
trace!(
"Cache lookup skipped for {:?}: modified in a later block",
addr
);
return false;
2020-08-05 06:08:03 +02:00
}
2016-10-11 19:37:31 +02:00
}
trace!(
"Cache lookup skipped for {:?}: parent hash is unknown",
addr
);
false
2016-10-11 19:37:31 +02:00
}
}
impl state::Backend for StateDB {
2020-07-29 10:36:15 +02:00
fn as_hash_db(&self) -> &dyn HashDB<KeccakHasher, DBValue> {
self.db.as_hash_db()
}
2020-08-05 06:08:03 +02:00
2020-07-29 10:36:15 +02:00
fn as_hash_db_mut(&mut self) -> &mut dyn HashDB<KeccakHasher, DBValue> {
self.db.as_hash_db_mut()
}
2020-08-05 06:08:03 +02:00
fn add_to_account_cache(&mut self, addr: Address, data: Option<Account>, modified: bool) {
self.local_cache.push(CacheQueueItem {
address: addr,
account: SyncAccount(data),
modified: modified,
})
}
2020-08-05 06:08:03 +02:00
fn cache_code(&self, hash: H256, code: Arc<Vec<u8>>) {
let mut cache = self.code_cache.lock();
2020-08-05 06:08:03 +02:00
cache.insert(hash, code);
}
2020-08-05 06:08:03 +02:00
fn get_cached_account(&self, addr: &Address) -> Option<Option<Account>> {
self.parent_hash.as_ref().and_then(|parent_hash| {
let mut cache = self.account_cache.lock();
if !Self::is_allowed(addr, parent_hash, &cache.modifications) {
return None;
}
cache
.accounts
.get_mut(addr)
.map(|a| a.as_ref().map(|a| a.clone_basic()))
})
}
2020-08-05 06:08:03 +02:00
fn get_cached<F, U>(&self, a: &Address, f: F) -> Option<U>
where
F: FnOnce(Option<&mut Account>) -> U,
{
self.parent_hash.as_ref().and_then(|parent_hash| {
let mut cache = self.account_cache.lock();
if !Self::is_allowed(a, parent_hash, &cache.modifications) {
return None;
}
cache.accounts.get_mut(a).map(|c| f(c.as_mut()))
})
}
2020-08-05 06:08:03 +02:00
fn get_cached_code(&self, hash: &H256) -> Option<Arc<Vec<u8>>> {
let mut cache = self.code_cache.lock();
2020-08-05 06:08:03 +02:00
cache.get_mut(hash).map(|code| code.clone())
}
}
/// Sync wrapper for the account.
struct SyncAccount(Option<Account>);
/// That implementation is safe because account is never modified or accessed in any way.
/// We only need `Sync` here to allow `StateDb` to be kept in a `RwLock`.
/// `Account` is `!Sync` by default because of `RefCell`s inside it.
unsafe impl Sync for SyncAccount {}
2016-10-11 19:37:31 +02:00
#[cfg(test)]
mod tests {
use ethereum_types::{Address, H256, U256};
use kvdb::DBTransaction;
use state::{Account, Backend};
Private transactions integration pr (#6422) * Private transaction message added * Empty line removed * Private transactions logic removed from client into the separate module * Fixed compilation after merge with head * Signed private transaction message added as well * Comments after the review fixed * Private tx execution * Test update * Renamed some methods * Fixed some tests * Reverted submodules * Fixed build * Private transaction message added * Empty line removed * Private transactions logic removed from client into the separate module * Fixed compilation after merge with head * Signed private transaction message added as well * Comments after the review fixed * Encrypted private transaction message and signed reply added * Private tx execution * Test update * Main scenario completed * Merged with the latest head * Private transactions API * Comments after review fixed * Parameters for private transactions added to parity arguments * New files added * New API methods added * Do not process packets from unconfirmed peers * Merge with ptm_ss branch * Encryption and permissioning with key server added * Fixed compilation after merge * Version of Parity protocol incremented in order to support private transactions * Doc strings for constants added * Proper format for doc string added * fixed some encryptor.rs grumbles * Private transactions functionality moved to the separate crate * Refactoring in order to remove late initialisation * Tests fixed after moving to the separate crate * Fetch method removed * Sync test helpers refactored * Interaction with encryptor refactored * Contract address retrieving via substate removed * Sensible gas limit for private transactions implemented * New private contract with nonces added * Parsing of the response from key server fixed * Build fixed after the merge, native contracts removed * Crate renamed * Tests moved to the separate directory * Handling of errors reworked in order to use error chain * Encodable macro added, new constructor replaced with default * Native ethabi usage removed * Couple conversions optimized * Interactions with client reworked * Errors omitting removed * Fix after merge * Fix after the merge * private transactions improvements in progress * private_transactions -> ethcore/private-tx * making private transactions more idiomatic * private-tx encryptor uses shared FetchClient and is more idiomatic * removed redundant tests, moved integration tests to tests/ dir * fixed failing service test * reenable add_notify on private tx provider * removed private_tx tests from sync module * removed commented out code * Use plain password instead of unlocking account manager * remove dead code * Link to the contract changed * Transaction signature chain replay protection module created * Redundant type conversion removed * Contract address returned by private provider * Test fixed * Addressing grumbles in PrivateTransactions (#8249) * Tiny fixes part 1. * A bunch of additional comments and todos. * Fix ethsync tests. * resolved merge conflicts * final private tx pr (#8318) * added cli option that enables private transactions * fixed failing test * fixed failing test * fixed failing test * fixed failing test
2018-04-09 16:14:33 +02:00
use test_helpers::get_temp_state_db;
2020-08-05 06:08:03 +02:00
#[test]
fn state_db_smoke() {
let _ = ::env_logger::try_init();
2020-08-05 06:08:03 +02:00
2017-04-06 19:26:17 +02:00
let state_db = get_temp_state_db();
let root_parent = H256::random();
let address = Address::random();
let h0 = H256::random();
let h1a = H256::random();
let h1b = H256::random();
let h2a = H256::random();
let h2b = H256::random();
let h3a = H256::random();
let h3b = H256::random();
let mut batch = DBTransaction::new();
2020-08-05 06:08:03 +02:00
// blocks [ 3a(c) 2a(c) 2b 1b 1a(c) 0 ]
2017-06-11 16:41:16 +02:00
// balance [ 5 5 4 3 2 2 ]
let mut s = state_db.boxed_clone_canon(&root_parent);
s.add_to_account_cache(address, Some(Account::new_basic(2.into(), 0.into())), false);
s.journal_under(&mut batch, 0, &h0).unwrap();
s.sync_cache(&[], &[], true);
2020-08-05 06:08:03 +02:00
let mut s = state_db.boxed_clone_canon(&h0);
s.journal_under(&mut batch, 1, &h1a).unwrap();
s.sync_cache(&[], &[], true);
2020-08-05 06:08:03 +02:00
let mut s = state_db.boxed_clone_canon(&h0);
s.add_to_account_cache(address, Some(Account::new_basic(3.into(), 0.into())), true);
s.journal_under(&mut batch, 1, &h1b).unwrap();
s.sync_cache(&[], &[], false);
2020-08-05 06:08:03 +02:00
let mut s = state_db.boxed_clone_canon(&h1b);
s.add_to_account_cache(address, Some(Account::new_basic(4.into(), 0.into())), true);
s.journal_under(&mut batch, 2, &h2b).unwrap();
s.sync_cache(&[], &[], false);
2020-08-05 06:08:03 +02:00
let mut s = state_db.boxed_clone_canon(&h1a);
s.add_to_account_cache(address, Some(Account::new_basic(5.into(), 0.into())), true);
s.journal_under(&mut batch, 2, &h2a).unwrap();
s.sync_cache(&[], &[], true);
2020-08-05 06:08:03 +02:00
let mut s = state_db.boxed_clone_canon(&h2a);
s.journal_under(&mut batch, 3, &h3a).unwrap();
s.sync_cache(&[], &[], true);
2020-08-05 06:08:03 +02:00
let s = state_db.boxed_clone_canon(&h3a);
assert_eq!(
s.get_cached_account(&address).unwrap().unwrap().balance(),
&U256::from(5)
);
2020-08-05 06:08:03 +02:00
let s = state_db.boxed_clone_canon(&h1a);
assert!(s.get_cached_account(&address).is_none());
2020-08-05 06:08:03 +02:00
let s = state_db.boxed_clone_canon(&h2b);
assert!(s.get_cached_account(&address).is_none());
2020-08-05 06:08:03 +02:00
let s = state_db.boxed_clone_canon(&h1b);
assert!(s.get_cached_account(&address).is_none());
2020-08-05 06:08:03 +02:00
// reorg to 3b
// blocks [ 3b(c) 3a 2a 2b(c) 1b 1a 0 ]
let mut s = state_db.boxed_clone_canon(&h2b);
s.journal_under(&mut batch, 3, &h3b).unwrap();
s.sync_cache(
&[h1b.clone(), h2b.clone(), h3b.clone()],
&[h1a.clone(), h2a.clone(), h3a.clone()],
true,
);
let s = state_db.boxed_clone_canon(&h3a);
assert!(s.get_cached_account(&address).is_none());
}
}