* to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
122 lines
3.6 KiB
Rust
122 lines
3.6 KiB
Rust
// Copyright 2015, 2016 Ethcore (UK) Ltd.
|
|
// This file is part of Parity.
|
|
|
|
// Parity is free software: you can redistribute it and/or modify
|
|
// it under the terms of the GNU General Public License as published by
|
|
// the Free Software Foundation, either version 3 of the License, or
|
|
// (at your option) any later version.
|
|
|
|
// Parity is distributed in the hope that it will be useful,
|
|
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
// GNU General Public License for more details.
|
|
|
|
// You should have received a copy of the GNU General Public License
|
|
// along with Parity. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
//! Snapshot test helpers. These are used to build blockchains and state tries
|
|
//! which can be queried before and after a full snapshot/restore cycle.
|
|
|
|
use account_db::AccountDBMut;
|
|
use rand::Rng;
|
|
use snapshot::account::Account;
|
|
|
|
use util::hash::{FixedHash, H256};
|
|
use util::hashdb::HashDB;
|
|
use util::trie::{Alphabet, StandardMap, SecTrieDBMut, TrieMut, ValueMode};
|
|
use util::trie::{TrieDB, TrieDBMut};
|
|
use util::rlp::SHA3_NULL_RLP;
|
|
|
|
// the proportion of accounts we will alter each tick.
|
|
const ACCOUNT_CHURN: f32 = 0.01;
|
|
|
|
/// This structure will incrementally alter a state given an rng.
|
|
pub struct StateProducer {
|
|
state_root: H256,
|
|
storage_seed: H256,
|
|
}
|
|
|
|
impl StateProducer {
|
|
/// Create a new `StateProducer`.
|
|
pub fn new() -> Self {
|
|
StateProducer {
|
|
state_root: SHA3_NULL_RLP,
|
|
storage_seed: H256::zero(),
|
|
}
|
|
}
|
|
|
|
/// Tick the state producer. This alters the state, writing new data into
|
|
/// the database.
|
|
pub fn tick<R: Rng>(&mut self, rng: &mut R, db: &mut HashDB) {
|
|
// modify existing accounts.
|
|
let mut accounts_to_modify: Vec<_> = {
|
|
let trie = TrieDB::new(&*db, &self.state_root).unwrap();
|
|
trie.iter()
|
|
.filter(|_| rng.gen::<f32>() < ACCOUNT_CHURN)
|
|
.map(|(k, v)| (H256::from_slice(&k), v.to_owned()))
|
|
.collect()
|
|
};
|
|
|
|
// sweep once to alter storage tries.
|
|
for &mut (ref mut address_hash, ref mut account_data) in &mut accounts_to_modify {
|
|
let mut account = Account::from_thin_rlp(&*account_data);
|
|
let acct_db = AccountDBMut::from_hash(db, *address_hash);
|
|
fill_storage(acct_db, account.storage_root_mut(), &mut self.storage_seed);
|
|
*account_data = account.to_thin_rlp();
|
|
}
|
|
|
|
// sweep again to alter account trie.
|
|
let mut trie = TrieDBMut::from_existing(db, &mut self.state_root).unwrap();
|
|
|
|
for (address_hash, account_data) in accounts_to_modify {
|
|
trie.insert(&address_hash[..], &account_data).unwrap();
|
|
}
|
|
|
|
// add between 0 and 5 new accounts each tick.
|
|
let new_accs = rng.gen::<u32>() % 5;
|
|
|
|
for _ in 0..new_accs {
|
|
let address_hash = H256::random();
|
|
let balance: usize = rng.gen();
|
|
let nonce: usize = rng.gen();
|
|
let acc = ::account::Account::new_basic(balance.into(), nonce.into()).rlp();
|
|
trie.insert(&address_hash[..], &acc).unwrap();
|
|
}
|
|
}
|
|
|
|
/// Get the current state root.
|
|
pub fn state_root(&self) -> H256 {
|
|
self.state_root
|
|
}
|
|
}
|
|
|
|
/// Fill the storage of an account.
|
|
pub fn fill_storage(mut db: AccountDBMut, root: &mut H256, seed: &mut H256) {
|
|
let map = StandardMap {
|
|
alphabet: Alphabet::All,
|
|
min_key: 6,
|
|
journal_key: 6,
|
|
value_mode: ValueMode::Random,
|
|
count: 100,
|
|
};
|
|
{
|
|
let mut trie = if *root == SHA3_NULL_RLP {
|
|
SecTrieDBMut::new(&mut db, root)
|
|
} else {
|
|
SecTrieDBMut::from_existing(&mut db, root).unwrap()
|
|
};
|
|
|
|
for (k, v) in map.make_with(seed) {
|
|
trie.insert(&k, &v).unwrap();
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Compare two state dbs.
|
|
pub fn compare_dbs(one: &HashDB, two: &HashDB) {
|
|
let keys = one.keys();
|
|
|
|
for (key, _) in keys {
|
|
assert_eq!(one.get(&key).unwrap(), two.get(&key).unwrap());
|
|
}
|
|
} |