openethereum/ethcore/snapshot/snapshot-tests/src/state.rs

212 lines
6.6 KiB
Rust
Raw Normal View History

// Copyright 2015-2019 Parity Technologies (UK) Ltd.
// This file is part of Parity Ethereum.
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
// Parity Ethereum is free software: you can redistribute it and/or modify
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Parity Ethereum is distributed in the hope that it will be useful,
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Parity Ethereum. If not, see <http://www.gnu.org/licenses/>.
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
//! State snapshotting tests.
use std::sync::Arc;
use std::sync::atomic::AtomicBool;
use keccak_hash::{KECCAK_NULL_RLP, keccak};
use common_types::{
Move more types out of ethcore (#10880) * WIP move errors, pod_account and state account to own crates * Sort out dependencies, fix broken code and tests Remove botched ethcore-error crate * remove template line * fix review feedback * Remove test-only AccountDBMut::new * Extract AccountDB to account-db * Move Substate to state-account – wip * Add lib.rs * cleanup * test failure * test failure 2 * third time's the charm * Add factories crate * Use new factories crate * Use factories crate * Extract trace * Fix tests * Sort out parity-util-mem and parking_lot * cleanup * WIP port over the rest of state from ethcore * Collect all impls for Machine * some notes * Rename pod-account to pod * Move PodState to pod crate * Use PodState from pod crate * Fix use clause for json tests * Sort out evmbin * Add missing code and use PodState * Move code that depends on Machine and Executive to own module * Sort out cloning errors, fix ethcore to use new state crate * Do without funky From impls * Fix ethcore tests * Fixes around the project to use new state crate * Add back the more specific impls of StateOrBlock From conversions * Move execute to freestanding function and remove it from trait Sort out the error handling in executive_state by moving the result types from state to ethcore Undo the verbose code added to work around the StateOrBlock From conversions * cleanup * Fix "error: enum variants on type aliases are experimental" * Bring back the state tests Fix whitespace * remove ethcore/state/mod.rs * cleanup * cleanup * Cleanup state-account errors * Fix more todos Add module docs * Add error.rs * Fixup Cargo.lock * Smaller ethcore API is fine * Add `to-pod-full` feature to state-account Fix evmbin * Fix a few more test failures * Fix RPC test build * Baptize the new trait * Remove resolved TODOs * Rename state-account to account-state * Do not re-export the trace crate * Don't export state_db from ethcore * Let private-tx use StateDB. :( * Remove ethcore/src/pod_state.rs * Inner type does not need to be pub/pub(crate) * optimise imports * Revert "Inner type does not need to be pub/pub(crate)" This reverts commit 2f839f8a0f72f71334da64620f57e6dd6039f06b. * Move DatabaseExtras to ethcore-blockchain * Add database_extra module to ethcore-blockchain * Remove to-pod-full feature * cosmetics * New crate: state-db * Add new crate * Move PreverifiedBlock and BlockError to types * Sort out the merge * Add missing `license` meta data keys * wip * wip client-traits * merge conflict * verification crate type checks * Move impls for CommonParams to common_types Fix misc stuff in ethcore * Fix tests * Implement VerifyingEngine for all engines except Ethash Temporarily sort out error handling Move more types to common_types * Split Engine in two and move code around * cleanup * verification: don't rexport common_types * Use error from common_types * Consolidate error types * VerifyingEngine use Errors from common_types * verification: Use error type from common_types * SnapshotError moved to common_types * Move more code from Engne to VerifyingEngine Add a VerifyingClient trait: BlockInfo + CallContract Whitespace * Add MAX_UNCLE_AGE const * Port over remaining code from ethcore/verification * Use errors from common_types * Fix the confusing "io" naming * Move more types into common_types * Add todos * Experiment with Engine trait outside ethcore * Hook up types from common_types in ethcore Don't use verification crate Don't use client-traits crate * Revert to impl Engine for Arc<Ethash> and add note to explain why Revert moving ClientIoMessage to common_types Fix build * Remove ClientIoMessage from common_types * Cleanup * More cleanup * Sort error handling changes in the rest of parity * Remove unused code * Remove WIP types * Cleanup todos not tackled here * remove cruft * Fix some whitespace and a merge error * ethcore tests * test failures * Restore Engine impls to master to make review a bit easier * cleanup * whitespace * applied review suggestions * types does not depend on rustc-hex * ethash engine moved to engine module * applied review suggestion
2019-07-18 12:27:08 +02:00
basic_account::BasicAccount,
errors::{EthcoreError as Error, SnapshotError},
snapshot::{ManifestData, Progress},
Move more types out of ethcore (#10880) * WIP move errors, pod_account and state account to own crates * Sort out dependencies, fix broken code and tests Remove botched ethcore-error crate * remove template line * fix review feedback * Remove test-only AccountDBMut::new * Extract AccountDB to account-db * Move Substate to state-account – wip * Add lib.rs * cleanup * test failure * test failure 2 * third time's the charm * Add factories crate * Use new factories crate * Use factories crate * Extract trace * Fix tests * Sort out parity-util-mem and parking_lot * cleanup * WIP port over the rest of state from ethcore * Collect all impls for Machine * some notes * Rename pod-account to pod * Move PodState to pod crate * Use PodState from pod crate * Fix use clause for json tests * Sort out evmbin * Add missing code and use PodState * Move code that depends on Machine and Executive to own module * Sort out cloning errors, fix ethcore to use new state crate * Do without funky From impls * Fix ethcore tests * Fixes around the project to use new state crate * Add back the more specific impls of StateOrBlock From conversions * Move execute to freestanding function and remove it from trait Sort out the error handling in executive_state by moving the result types from state to ethcore Undo the verbose code added to work around the StateOrBlock From conversions * cleanup * Fix "error: enum variants on type aliases are experimental" * Bring back the state tests Fix whitespace * remove ethcore/state/mod.rs * cleanup * cleanup * Cleanup state-account errors * Fix more todos Add module docs * Add error.rs * Fixup Cargo.lock * Smaller ethcore API is fine * Add `to-pod-full` feature to state-account Fix evmbin * Fix a few more test failures * Fix RPC test build * Baptize the new trait * Remove resolved TODOs * Rename state-account to account-state * Do not re-export the trace crate * Don't export state_db from ethcore * Let private-tx use StateDB. :( * Remove ethcore/src/pod_state.rs * Inner type does not need to be pub/pub(crate) * optimise imports * Revert "Inner type does not need to be pub/pub(crate)" This reverts commit 2f839f8a0f72f71334da64620f57e6dd6039f06b. * Move DatabaseExtras to ethcore-blockchain * Add database_extra module to ethcore-blockchain * Remove to-pod-full feature * cosmetics * New crate: state-db * Add new crate * Move PreverifiedBlock and BlockError to types * Sort out the merge * Add missing `license` meta data keys * wip * wip client-traits * merge conflict * verification crate type checks * Move impls for CommonParams to common_types Fix misc stuff in ethcore * Fix tests * Implement VerifyingEngine for all engines except Ethash Temporarily sort out error handling Move more types to common_types * Split Engine in two and move code around * cleanup * verification: don't rexport common_types * Use error from common_types * Consolidate error types * VerifyingEngine use Errors from common_types * verification: Use error type from common_types * SnapshotError moved to common_types * Move more code from Engne to VerifyingEngine Add a VerifyingClient trait: BlockInfo + CallContract Whitespace * Add MAX_UNCLE_AGE const * Port over remaining code from ethcore/verification * Use errors from common_types * Fix the confusing "io" naming * Move more types into common_types * Add todos * Experiment with Engine trait outside ethcore * Hook up types from common_types in ethcore Don't use verification crate Don't use client-traits crate * Revert to impl Engine for Arc<Ethash> and add note to explain why Revert moving ClientIoMessage to common_types Fix build * Remove ClientIoMessage from common_types * Cleanup * More cleanup * Sort error handling changes in the rest of parity * Remove unused code * Remove WIP types * Cleanup todos not tackled here * remove cruft * Fix some whitespace and a merge error * ethcore tests * test failures * Restore Engine impls to master to make review a bit easier * cleanup * whitespace * applied review suggestions * types does not depend on rustc-hex * ethash engine moved to engine module * applied review suggestion
2019-07-18 12:27:08 +02:00
};
use snapshot::{
test_helpers::to_fat_rlps,
chunk_state, StateRebuilder, SNAPSHOT_SUBPARTS,
io::{PackedReader, PackedWriter, SnapshotReader, SnapshotWriter},
};
Upgrade ethereum types (#10670) * cargo upgrade "ethereum-types" --all --allow-prerelease * [ethash] fix compilation errors * [ethkey] fix compilation errors * [journaldb] fix compilation errors * [dir] fix compilation errors * [ethabi] update to 0.7 * wip * [eip-712] fix compilation errors * [ethjson] fix compilation errors * [Cargo.toml] add TODO to remove patches * [ethstore] fix compilation errors * use patched keccak-hash with new primitive-types * wip * [ethcore-network-devp2p] fix compilation errors * [vm] fix compilation errors * [common-types, evm, wasm] fix compilation errors * [ethcore-db] Require AsRef instead of Deref for keys * [ethcore-blockchain] fix some compilation errors * [blooms-db] fix compilation errors Thanks a lot @dvdplm :) * we don't need no rlp ethereum feature * [ethcore] fix some compilation errors * [parity-ipfs-api] fix compilation error * [ethcore-light] fix compilation errors * [Cargo.lock] update parity-common * [ethcore-private-tx] fix some compilation errors * wip * [ethcore-private-tx] fix compilation errors * [parity-updater] fix compilation errors * [parity-rpc] fix compilation errors * [parity-bin] fix other compilation errors * update to new ethereum-types * update keccak-hash * [fastmap] fix compilation in tests * [blooms-db] fix compilation in tests * [common-types] fix compilation in tests * [triehash-ethereum] fix compilation in tests * [ethkey] fix compilation in tests * [pwasm-run-test] fix compilation errors * [wasm] fix compilation errors * [ethjson] fix compilation in tests * [eip-712] fix compilation in tests * [ethcore-blockchain] fix compilation in tests * [ethstore] fix compilation in tests * [ethstore-accounts] fix compilation in tests * [parity-hash-fetch] fix compilation in tests * [parity-whisper] fix compilation in tests * [ethcore-miner] fix compilation in tests * [ethcore-network-devp2p] fix compilation in tests * [*] upgrade rand to 0.6 * [evm] get rid of num-bigint conversions * [ethcore] downgrade trie-standardmap and criterion * [ethcore] fix some warnings * [ethcore] fix compilation in tests * [evmbin] fix compilation in tests * [updater] fix compilation in tests * [ethash] fix compilation in tests * [ethcore-secretstore] fix compilation in tests * [ethcore-sync] fix compilation in tests * [parity-rpc] fix compilation in tests * [ethcore] finally fix compilation in tests FUCK YEAH!!! * [ethstore] lazy_static is unused * [ethcore] fix test * fix up bad merge * [Cargo.toml] remove unused patches * [*] replace some git dependencies with crates.io * [Cargo.toml] remove unused lazy_static * [*] clean up * [ethcore] fix transaction_filter_deprecated test * [private-tx] fix serialization tests * fix more serialization tests * [ethkey] fix smoky test * [rpc] fix tests, please? * [ethcore] remove commented out code * Apply suggestions from code review Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * [ethstore] remove unused dev-dependency * [ethcore] remove resolved TODO * [*] resolve keccak-hash TODO * [*] s/Address::default()/Address::zero() * [rpc] remove Subscribers::new_test * [rpc] remove EthPubSubClient::new_test * [ethcore] use trie-standardmap from crates.io * [dir] fix db_root_path * [ethcore] simplify snapshot::tests::helpers::fill_storage * Apply suggestions from code review Co-Authored-By: David <dvdplm@gmail.com> * [ethcore-secretstore] resolve TODO in serialization * [ethcore-network-devp2p] resolve TODO in save_key * [Cargo.lock] update triehash * [*] use ethabi from crates.io * [ethkey] use secp256k1 from master branch * [Cargo.lock] update eth-secp256k1
2019-06-03 15:36:21 +02:00
use rand::SeedableRng;
use rand_xorshift::XorShiftRng;
use ethereum_types::H256;
2017-10-17 17:24:47 +02:00
use journaldb::{self, Algorithm};
use kvdb_rocksdb::{Database, DatabaseConfig};
use parking_lot::Mutex;
use tempdir::TempDir;
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
use crate::helpers::StateProducer;
Upgrade ethereum types (#10670) * cargo upgrade "ethereum-types" --all --allow-prerelease * [ethash] fix compilation errors * [ethkey] fix compilation errors * [journaldb] fix compilation errors * [dir] fix compilation errors * [ethabi] update to 0.7 * wip * [eip-712] fix compilation errors * [ethjson] fix compilation errors * [Cargo.toml] add TODO to remove patches * [ethstore] fix compilation errors * use patched keccak-hash with new primitive-types * wip * [ethcore-network-devp2p] fix compilation errors * [vm] fix compilation errors * [common-types, evm, wasm] fix compilation errors * [ethcore-db] Require AsRef instead of Deref for keys * [ethcore-blockchain] fix some compilation errors * [blooms-db] fix compilation errors Thanks a lot @dvdplm :) * we don't need no rlp ethereum feature * [ethcore] fix some compilation errors * [parity-ipfs-api] fix compilation error * [ethcore-light] fix compilation errors * [Cargo.lock] update parity-common * [ethcore-private-tx] fix some compilation errors * wip * [ethcore-private-tx] fix compilation errors * [parity-updater] fix compilation errors * [parity-rpc] fix compilation errors * [parity-bin] fix other compilation errors * update to new ethereum-types * update keccak-hash * [fastmap] fix compilation in tests * [blooms-db] fix compilation in tests * [common-types] fix compilation in tests * [triehash-ethereum] fix compilation in tests * [ethkey] fix compilation in tests * [pwasm-run-test] fix compilation errors * [wasm] fix compilation errors * [ethjson] fix compilation in tests * [eip-712] fix compilation in tests * [ethcore-blockchain] fix compilation in tests * [ethstore] fix compilation in tests * [ethstore-accounts] fix compilation in tests * [parity-hash-fetch] fix compilation in tests * [parity-whisper] fix compilation in tests * [ethcore-miner] fix compilation in tests * [ethcore-network-devp2p] fix compilation in tests * [*] upgrade rand to 0.6 * [evm] get rid of num-bigint conversions * [ethcore] downgrade trie-standardmap and criterion * [ethcore] fix some warnings * [ethcore] fix compilation in tests * [evmbin] fix compilation in tests * [updater] fix compilation in tests * [ethash] fix compilation in tests * [ethcore-secretstore] fix compilation in tests * [ethcore-sync] fix compilation in tests * [parity-rpc] fix compilation in tests * [ethcore] finally fix compilation in tests FUCK YEAH!!! * [ethstore] lazy_static is unused * [ethcore] fix test * fix up bad merge * [Cargo.toml] remove unused patches * [*] replace some git dependencies with crates.io * [Cargo.toml] remove unused lazy_static * [*] clean up * [ethcore] fix transaction_filter_deprecated test * [private-tx] fix serialization tests * fix more serialization tests * [ethkey] fix smoky test * [rpc] fix tests, please? * [ethcore] remove commented out code * Apply suggestions from code review Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * [ethstore] remove unused dev-dependency * [ethcore] remove resolved TODO * [*] resolve keccak-hash TODO * [*] s/Address::default()/Address::zero() * [rpc] remove Subscribers::new_test * [rpc] remove EthPubSubClient::new_test * [ethcore] use trie-standardmap from crates.io * [dir] fix db_root_path * [ethcore] simplify snapshot::tests::helpers::fill_storage * Apply suggestions from code review Co-Authored-By: David <dvdplm@gmail.com> * [ethcore-secretstore] resolve TODO in serialization * [ethcore-network-devp2p] resolve TODO in save_key * [Cargo.lock] update triehash * [*] use ethabi from crates.io * [ethkey] use secp256k1 from master branch * [Cargo.lock] update eth-secp256k1
2019-06-03 15:36:21 +02:00
const RNG_SEED: [u8; 16] = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16];
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
#[test]
fn snap_and_restore() {
use hash_db::{HashDB, EMPTY_PREFIX};
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
let mut producer = StateProducer::new();
Upgrade ethereum types (#10670) * cargo upgrade "ethereum-types" --all --allow-prerelease * [ethash] fix compilation errors * [ethkey] fix compilation errors * [journaldb] fix compilation errors * [dir] fix compilation errors * [ethabi] update to 0.7 * wip * [eip-712] fix compilation errors * [ethjson] fix compilation errors * [Cargo.toml] add TODO to remove patches * [ethstore] fix compilation errors * use patched keccak-hash with new primitive-types * wip * [ethcore-network-devp2p] fix compilation errors * [vm] fix compilation errors * [common-types, evm, wasm] fix compilation errors * [ethcore-db] Require AsRef instead of Deref for keys * [ethcore-blockchain] fix some compilation errors * [blooms-db] fix compilation errors Thanks a lot @dvdplm :) * we don't need no rlp ethereum feature * [ethcore] fix some compilation errors * [parity-ipfs-api] fix compilation error * [ethcore-light] fix compilation errors * [Cargo.lock] update parity-common * [ethcore-private-tx] fix some compilation errors * wip * [ethcore-private-tx] fix compilation errors * [parity-updater] fix compilation errors * [parity-rpc] fix compilation errors * [parity-bin] fix other compilation errors * update to new ethereum-types * update keccak-hash * [fastmap] fix compilation in tests * [blooms-db] fix compilation in tests * [common-types] fix compilation in tests * [triehash-ethereum] fix compilation in tests * [ethkey] fix compilation in tests * [pwasm-run-test] fix compilation errors * [wasm] fix compilation errors * [ethjson] fix compilation in tests * [eip-712] fix compilation in tests * [ethcore-blockchain] fix compilation in tests * [ethstore] fix compilation in tests * [ethstore-accounts] fix compilation in tests * [parity-hash-fetch] fix compilation in tests * [parity-whisper] fix compilation in tests * [ethcore-miner] fix compilation in tests * [ethcore-network-devp2p] fix compilation in tests * [*] upgrade rand to 0.6 * [evm] get rid of num-bigint conversions * [ethcore] downgrade trie-standardmap and criterion * [ethcore] fix some warnings * [ethcore] fix compilation in tests * [evmbin] fix compilation in tests * [updater] fix compilation in tests * [ethash] fix compilation in tests * [ethcore-secretstore] fix compilation in tests * [ethcore-sync] fix compilation in tests * [parity-rpc] fix compilation in tests * [ethcore] finally fix compilation in tests FUCK YEAH!!! * [ethstore] lazy_static is unused * [ethcore] fix test * fix up bad merge * [Cargo.toml] remove unused patches * [*] replace some git dependencies with crates.io * [Cargo.toml] remove unused lazy_static * [*] clean up * [ethcore] fix transaction_filter_deprecated test * [private-tx] fix serialization tests * fix more serialization tests * [ethkey] fix smoky test * [rpc] fix tests, please? * [ethcore] remove commented out code * Apply suggestions from code review Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * [ethstore] remove unused dev-dependency * [ethcore] remove resolved TODO * [*] resolve keccak-hash TODO * [*] s/Address::default()/Address::zero() * [rpc] remove Subscribers::new_test * [rpc] remove EthPubSubClient::new_test * [ethcore] use trie-standardmap from crates.io * [dir] fix db_root_path * [ethcore] simplify snapshot::tests::helpers::fill_storage * Apply suggestions from code review Co-Authored-By: David <dvdplm@gmail.com> * [ethcore-secretstore] resolve TODO in serialization * [ethcore-network-devp2p] resolve TODO in save_key * [Cargo.lock] update triehash * [*] use ethabi from crates.io * [ethkey] use secp256k1 from master branch * [Cargo.lock] update eth-secp256k1
2019-06-03 15:36:21 +02:00
let mut rng = XorShiftRng::from_seed(RNG_SEED);
let mut old_db = journaldb::new_memory_db();
let db_cfg = DatabaseConfig::with_columns(ethcore_db::NUM_COLUMNS);
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
for _ in 0..150 {
producer.tick(&mut rng, &mut old_db);
}
let tempdir = TempDir::new("").unwrap();
let snap_file = tempdir.path().join("SNAP");
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
let state_root = producer.state_root();
let writer = Mutex::new(PackedWriter::new(&snap_file).unwrap());
let mut state_hashes = Vec::new();
for part in 0..SNAPSHOT_SUBPARTS {
let mut hashes = chunk_state(&old_db, &state_root, &writer, &Progress::default(), Some(part), 0).unwrap();
state_hashes.append(&mut hashes);
}
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
writer.into_inner().finish(ManifestData {
2017-03-24 14:02:04 +01:00
version: 2,
state_hashes,
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
block_hashes: Vec::new(),
state_root,
block_number: 1000,
Upgrade ethereum types (#10670) * cargo upgrade "ethereum-types" --all --allow-prerelease * [ethash] fix compilation errors * [ethkey] fix compilation errors * [journaldb] fix compilation errors * [dir] fix compilation errors * [ethabi] update to 0.7 * wip * [eip-712] fix compilation errors * [ethjson] fix compilation errors * [Cargo.toml] add TODO to remove patches * [ethstore] fix compilation errors * use patched keccak-hash with new primitive-types * wip * [ethcore-network-devp2p] fix compilation errors * [vm] fix compilation errors * [common-types, evm, wasm] fix compilation errors * [ethcore-db] Require AsRef instead of Deref for keys * [ethcore-blockchain] fix some compilation errors * [blooms-db] fix compilation errors Thanks a lot @dvdplm :) * we don't need no rlp ethereum feature * [ethcore] fix some compilation errors * [parity-ipfs-api] fix compilation error * [ethcore-light] fix compilation errors * [Cargo.lock] update parity-common * [ethcore-private-tx] fix some compilation errors * wip * [ethcore-private-tx] fix compilation errors * [parity-updater] fix compilation errors * [parity-rpc] fix compilation errors * [parity-bin] fix other compilation errors * update to new ethereum-types * update keccak-hash * [fastmap] fix compilation in tests * [blooms-db] fix compilation in tests * [common-types] fix compilation in tests * [triehash-ethereum] fix compilation in tests * [ethkey] fix compilation in tests * [pwasm-run-test] fix compilation errors * [wasm] fix compilation errors * [ethjson] fix compilation in tests * [eip-712] fix compilation in tests * [ethcore-blockchain] fix compilation in tests * [ethstore] fix compilation in tests * [ethstore-accounts] fix compilation in tests * [parity-hash-fetch] fix compilation in tests * [parity-whisper] fix compilation in tests * [ethcore-miner] fix compilation in tests * [ethcore-network-devp2p] fix compilation in tests * [*] upgrade rand to 0.6 * [evm] get rid of num-bigint conversions * [ethcore] downgrade trie-standardmap and criterion * [ethcore] fix some warnings * [ethcore] fix compilation in tests * [evmbin] fix compilation in tests * [updater] fix compilation in tests * [ethash] fix compilation in tests * [ethcore-secretstore] fix compilation in tests * [ethcore-sync] fix compilation in tests * [parity-rpc] fix compilation in tests * [ethcore] finally fix compilation in tests FUCK YEAH!!! * [ethstore] lazy_static is unused * [ethcore] fix test * fix up bad merge * [Cargo.toml] remove unused patches * [*] replace some git dependencies with crates.io * [Cargo.toml] remove unused lazy_static * [*] clean up * [ethcore] fix transaction_filter_deprecated test * [private-tx] fix serialization tests * fix more serialization tests * [ethkey] fix smoky test * [rpc] fix tests, please? * [ethcore] remove commented out code * Apply suggestions from code review Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * [ethstore] remove unused dev-dependency * [ethcore] remove resolved TODO * [*] resolve keccak-hash TODO * [*] s/Address::default()/Address::zero() * [rpc] remove Subscribers::new_test * [rpc] remove EthPubSubClient::new_test * [ethcore] use trie-standardmap from crates.io * [dir] fix db_root_path * [ethcore] simplify snapshot::tests::helpers::fill_storage * Apply suggestions from code review Co-Authored-By: David <dvdplm@gmail.com> * [ethcore-secretstore] resolve TODO in serialization * [ethcore-network-devp2p] resolve TODO in save_key * [Cargo.lock] update triehash * [*] use ethabi from crates.io * [ethkey] use secp256k1 from master branch * [Cargo.lock] update eth-secp256k1
2019-06-03 15:36:21 +02:00
block_hash: H256::zero(),
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
}).unwrap();
let db_path = tempdir.path().join("db");
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
let db = {
let new_db = Arc::new(Database::open(&db_cfg, &db_path.to_string_lossy()).unwrap());
let mut rebuilder = StateRebuilder::new(new_db.clone(), Algorithm::OverlayRecent);
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
let reader = PackedReader::new(&snap_file).unwrap().unwrap();
let flag = AtomicBool::new(true);
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
for chunk_hash in &reader.manifest().state_hashes {
let raw = reader.chunk(*chunk_hash).unwrap();
let chunk = snappy::decompress(&raw).unwrap();
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
rebuilder.feed(&chunk, &flag).unwrap();
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
}
assert_eq!(rebuilder.state_root(), state_root);
Upgrade ethereum types (#10670) * cargo upgrade "ethereum-types" --all --allow-prerelease * [ethash] fix compilation errors * [ethkey] fix compilation errors * [journaldb] fix compilation errors * [dir] fix compilation errors * [ethabi] update to 0.7 * wip * [eip-712] fix compilation errors * [ethjson] fix compilation errors * [Cargo.toml] add TODO to remove patches * [ethstore] fix compilation errors * use patched keccak-hash with new primitive-types * wip * [ethcore-network-devp2p] fix compilation errors * [vm] fix compilation errors * [common-types, evm, wasm] fix compilation errors * [ethcore-db] Require AsRef instead of Deref for keys * [ethcore-blockchain] fix some compilation errors * [blooms-db] fix compilation errors Thanks a lot @dvdplm :) * we don't need no rlp ethereum feature * [ethcore] fix some compilation errors * [parity-ipfs-api] fix compilation error * [ethcore-light] fix compilation errors * [Cargo.lock] update parity-common * [ethcore-private-tx] fix some compilation errors * wip * [ethcore-private-tx] fix compilation errors * [parity-updater] fix compilation errors * [parity-rpc] fix compilation errors * [parity-bin] fix other compilation errors * update to new ethereum-types * update keccak-hash * [fastmap] fix compilation in tests * [blooms-db] fix compilation in tests * [common-types] fix compilation in tests * [triehash-ethereum] fix compilation in tests * [ethkey] fix compilation in tests * [pwasm-run-test] fix compilation errors * [wasm] fix compilation errors * [ethjson] fix compilation in tests * [eip-712] fix compilation in tests * [ethcore-blockchain] fix compilation in tests * [ethstore] fix compilation in tests * [ethstore-accounts] fix compilation in tests * [parity-hash-fetch] fix compilation in tests * [parity-whisper] fix compilation in tests * [ethcore-miner] fix compilation in tests * [ethcore-network-devp2p] fix compilation in tests * [*] upgrade rand to 0.6 * [evm] get rid of num-bigint conversions * [ethcore] downgrade trie-standardmap and criterion * [ethcore] fix some warnings * [ethcore] fix compilation in tests * [evmbin] fix compilation in tests * [updater] fix compilation in tests * [ethash] fix compilation in tests * [ethcore-secretstore] fix compilation in tests * [ethcore-sync] fix compilation in tests * [parity-rpc] fix compilation in tests * [ethcore] finally fix compilation in tests FUCK YEAH!!! * [ethstore] lazy_static is unused * [ethcore] fix test * fix up bad merge * [Cargo.toml] remove unused patches * [*] replace some git dependencies with crates.io * [Cargo.toml] remove unused lazy_static * [*] clean up * [ethcore] fix transaction_filter_deprecated test * [private-tx] fix serialization tests * fix more serialization tests * [ethkey] fix smoky test * [rpc] fix tests, please? * [ethcore] remove commented out code * Apply suggestions from code review Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * [ethstore] remove unused dev-dependency * [ethcore] remove resolved TODO * [*] resolve keccak-hash TODO * [*] s/Address::default()/Address::zero() * [rpc] remove Subscribers::new_test * [rpc] remove EthPubSubClient::new_test * [ethcore] use trie-standardmap from crates.io * [dir] fix db_root_path * [ethcore] simplify snapshot::tests::helpers::fill_storage * Apply suggestions from code review Co-Authored-By: David <dvdplm@gmail.com> * [ethcore-secretstore] resolve TODO in serialization * [ethcore-network-devp2p] resolve TODO in save_key * [Cargo.lock] update triehash * [*] use ethabi from crates.io * [ethkey] use secp256k1 from master branch * [Cargo.lock] update eth-secp256k1
2019-06-03 15:36:21 +02:00
rebuilder.finalize(1000, H256::zero()).unwrap();
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
new_db
};
let new_db = journaldb::new(db, Algorithm::OverlayRecent, ethcore_db::COL_STATE);
assert_eq!(new_db.earliest_era(), Some(1000));
let keys = old_db.keys();
Snapshot creation and restoration (#1679) * to_rlp takes self by-reference * clean up some derefs * out-of-order insertion for blockchain * implement block rebuilder without verification * group block chunk header into struct * block rebuilder does verification * integrate snapshot service with client service; flesh out implementation more * initial implementation of snapshot service * remove snapshottaker trait * snapshot writer trait with packed and loose implementations * write chunks using "snapshotwriter" in service * have snapshot taking use snapshotwriter * implement snapshot readers * back up client dbs when replacing * use snapshot reader in snapshot service * describe offset format * use new get_db_path in parity, allow some errors in service * blockchain formatting * implement parity snapshot * implement snapshot restore * force blocks to be submitted in order * fix bug loading block hashes in packed reader * fix seal field loading * fix uncle hash computation * fix a few bugs * store genesis state in db. reverse block chunk order in packed writer * allow out-of-order import for blocks * bring restoration types together * only snapshot the last 30000 blocks * restore into overlaydb instead of journaldb * commit version to database * use memorydbs and commit directly * fix trie test compilation * fix failing tests * sha3_null_rlp, not H256::zero * move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc * port archivedb to new overlaydb * add deletion mode tests for overlaydb * use new overlaydb, check state root at end * share chain info between state and block snapshotting * create blocks snapshot using blockchain directly * allow snapshot from arbitrary block, remove panickers from snapshot creation * begin test framework * blockchain chunking test * implement stateproducer::tick * state snapshot test * create block and state chunks concurrently, better restoration informant * fix tests * add deletion mode tests for overlaydb * address comments * more tests * Fix up tests. * remove a few printlns * add a little more documentation to `commit` * fix tests * fix ref_overlaydb test names * snapshot command skeleton * revert ref_overlaydb renaming * reimplement snapshot commands * fix many errors * everything but inject * get ethcore compiling * get snapshot tests passing again * instrument snapshot commands again * fix fallout from other changes, mark snapshots as experimental * optimize injection patterns * do two injections * fix up tests * take snapshots from 1000 blocks efore * address minor comments * fix a few io crate related errors * clarify names about total difficulty [ci skip]
2016-08-05 17:00:46 +02:00
for key in keys.keys() {
assert_eq!(old_db.get(&key, EMPTY_PREFIX).unwrap(), new_db.as_hash_db().get(&key, EMPTY_PREFIX).unwrap());
}
}
2016-11-11 18:24:12 +01:00
#[test]
fn get_code_from_prev_chunk() {
use std::collections::HashSet;
use rlp::RlpStream;
use ethereum_types::{H256, U256};
use hash_db::{HashDB, EMPTY_PREFIX};
2016-11-11 18:24:12 +01:00
use account_db::{AccountDBMut, AccountDB};
let code = b"this is definitely code";
let mut used_code = HashSet::new();
let mut acc_stream = RlpStream::new_list(4);
acc_stream.append(&U256::default())
.append(&U256::default())
.append(&KECCAK_NULL_RLP)
.append(&keccak(code));
2016-11-11 18:24:12 +01:00
let (h1, h2) = (H256::random(), H256::random());
// two accounts with the same code, one per chunk.
// first one will have code inlined,
// second will just have its hash.
let thin_rlp = acc_stream.out();
let acc: BasicAccount = ::rlp::decode(&thin_rlp).expect("error decoding basic account");
2016-11-11 18:24:12 +01:00
2017-01-03 17:05:27 +01:00
let mut make_chunk = |acc, hash| {
let mut db = journaldb::new_memory_db();
AccountDBMut::from_hash(&mut db, hash).insert(EMPTY_PREFIX, &code[..]);
let p = Progress::default();
let fat_rlp = to_fat_rlps(&hash, &acc, &AccountDB::from_hash(&db, hash), &mut used_code, usize::max_value(), usize::max_value(), &p).unwrap();
2016-11-11 18:24:12 +01:00
let mut stream = RlpStream::new_list(1);
2017-03-28 16:23:09 +02:00
stream.append_raw(&fat_rlp[0], 1);
2016-11-11 18:24:12 +01:00
stream.out()
};
2017-01-03 17:05:27 +01:00
let chunk1 = make_chunk(acc.clone(), h1);
let chunk2 = make_chunk(acc, h2);
2016-11-11 18:24:12 +01:00
let tempdir = TempDir::new("").unwrap();
let db_cfg = DatabaseConfig::with_columns(ethcore_db::NUM_COLUMNS);
let new_db = Arc::new(Database::open(&db_cfg, tempdir.path().to_str().unwrap()).unwrap());
2016-11-11 18:24:12 +01:00
{
let mut rebuilder = StateRebuilder::new(new_db.clone(), Algorithm::OverlayRecent);
let flag = AtomicBool::new(true);
2016-11-11 18:24:12 +01:00
rebuilder.feed(&chunk1, &flag).unwrap();
rebuilder.feed(&chunk2, &flag).unwrap();
rebuilder.finalize(1000, H256::random()).unwrap();
}
2016-11-11 18:24:12 +01:00
let state_db = journaldb::new(new_db, Algorithm::OverlayRecent, ethcore_db::COL_STATE);
assert_eq!(state_db.earliest_era(), Some(1000));
}
#[test]
fn checks_flag() {
let mut producer = StateProducer::new();
Upgrade ethereum types (#10670) * cargo upgrade "ethereum-types" --all --allow-prerelease * [ethash] fix compilation errors * [ethkey] fix compilation errors * [journaldb] fix compilation errors * [dir] fix compilation errors * [ethabi] update to 0.7 * wip * [eip-712] fix compilation errors * [ethjson] fix compilation errors * [Cargo.toml] add TODO to remove patches * [ethstore] fix compilation errors * use patched keccak-hash with new primitive-types * wip * [ethcore-network-devp2p] fix compilation errors * [vm] fix compilation errors * [common-types, evm, wasm] fix compilation errors * [ethcore-db] Require AsRef instead of Deref for keys * [ethcore-blockchain] fix some compilation errors * [blooms-db] fix compilation errors Thanks a lot @dvdplm :) * we don't need no rlp ethereum feature * [ethcore] fix some compilation errors * [parity-ipfs-api] fix compilation error * [ethcore-light] fix compilation errors * [Cargo.lock] update parity-common * [ethcore-private-tx] fix some compilation errors * wip * [ethcore-private-tx] fix compilation errors * [parity-updater] fix compilation errors * [parity-rpc] fix compilation errors * [parity-bin] fix other compilation errors * update to new ethereum-types * update keccak-hash * [fastmap] fix compilation in tests * [blooms-db] fix compilation in tests * [common-types] fix compilation in tests * [triehash-ethereum] fix compilation in tests * [ethkey] fix compilation in tests * [pwasm-run-test] fix compilation errors * [wasm] fix compilation errors * [ethjson] fix compilation in tests * [eip-712] fix compilation in tests * [ethcore-blockchain] fix compilation in tests * [ethstore] fix compilation in tests * [ethstore-accounts] fix compilation in tests * [parity-hash-fetch] fix compilation in tests * [parity-whisper] fix compilation in tests * [ethcore-miner] fix compilation in tests * [ethcore-network-devp2p] fix compilation in tests * [*] upgrade rand to 0.6 * [evm] get rid of num-bigint conversions * [ethcore] downgrade trie-standardmap and criterion * [ethcore] fix some warnings * [ethcore] fix compilation in tests * [evmbin] fix compilation in tests * [updater] fix compilation in tests * [ethash] fix compilation in tests * [ethcore-secretstore] fix compilation in tests * [ethcore-sync] fix compilation in tests * [parity-rpc] fix compilation in tests * [ethcore] finally fix compilation in tests FUCK YEAH!!! * [ethstore] lazy_static is unused * [ethcore] fix test * fix up bad merge * [Cargo.toml] remove unused patches * [*] replace some git dependencies with crates.io * [Cargo.toml] remove unused lazy_static * [*] clean up * [ethcore] fix transaction_filter_deprecated test * [private-tx] fix serialization tests * fix more serialization tests * [ethkey] fix smoky test * [rpc] fix tests, please? * [ethcore] remove commented out code * Apply suggestions from code review Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * [ethstore] remove unused dev-dependency * [ethcore] remove resolved TODO * [*] resolve keccak-hash TODO * [*] s/Address::default()/Address::zero() * [rpc] remove Subscribers::new_test * [rpc] remove EthPubSubClient::new_test * [ethcore] use trie-standardmap from crates.io * [dir] fix db_root_path * [ethcore] simplify snapshot::tests::helpers::fill_storage * Apply suggestions from code review Co-Authored-By: David <dvdplm@gmail.com> * [ethcore-secretstore] resolve TODO in serialization * [ethcore-network-devp2p] resolve TODO in save_key * [Cargo.lock] update triehash * [*] use ethabi from crates.io * [ethkey] use secp256k1 from master branch * [Cargo.lock] update eth-secp256k1
2019-06-03 15:36:21 +02:00
let mut rng = XorShiftRng::from_seed(RNG_SEED);
let mut old_db = journaldb::new_memory_db();
let db_cfg = DatabaseConfig::with_columns(ethcore_db::NUM_COLUMNS);
for _ in 0..10 {
producer.tick(&mut rng, &mut old_db);
}
let tempdir = TempDir::new("").unwrap();
let snap_file = tempdir.path().join("SNAP");
let state_root = producer.state_root();
let writer = Mutex::new(PackedWriter::new(&snap_file).unwrap());
let state_hashes = chunk_state(&old_db, &state_root, &writer, &Progress::default(), None, 0).unwrap();
writer.into_inner().finish(ManifestData {
2017-03-24 14:02:04 +01:00
version: 2,
state_hashes,
block_hashes: Vec::new(),
state_root,
block_number: 0,
Upgrade ethereum types (#10670) * cargo upgrade "ethereum-types" --all --allow-prerelease * [ethash] fix compilation errors * [ethkey] fix compilation errors * [journaldb] fix compilation errors * [dir] fix compilation errors * [ethabi] update to 0.7 * wip * [eip-712] fix compilation errors * [ethjson] fix compilation errors * [Cargo.toml] add TODO to remove patches * [ethstore] fix compilation errors * use patched keccak-hash with new primitive-types * wip * [ethcore-network-devp2p] fix compilation errors * [vm] fix compilation errors * [common-types, evm, wasm] fix compilation errors * [ethcore-db] Require AsRef instead of Deref for keys * [ethcore-blockchain] fix some compilation errors * [blooms-db] fix compilation errors Thanks a lot @dvdplm :) * we don't need no rlp ethereum feature * [ethcore] fix some compilation errors * [parity-ipfs-api] fix compilation error * [ethcore-light] fix compilation errors * [Cargo.lock] update parity-common * [ethcore-private-tx] fix some compilation errors * wip * [ethcore-private-tx] fix compilation errors * [parity-updater] fix compilation errors * [parity-rpc] fix compilation errors * [parity-bin] fix other compilation errors * update to new ethereum-types * update keccak-hash * [fastmap] fix compilation in tests * [blooms-db] fix compilation in tests * [common-types] fix compilation in tests * [triehash-ethereum] fix compilation in tests * [ethkey] fix compilation in tests * [pwasm-run-test] fix compilation errors * [wasm] fix compilation errors * [ethjson] fix compilation in tests * [eip-712] fix compilation in tests * [ethcore-blockchain] fix compilation in tests * [ethstore] fix compilation in tests * [ethstore-accounts] fix compilation in tests * [parity-hash-fetch] fix compilation in tests * [parity-whisper] fix compilation in tests * [ethcore-miner] fix compilation in tests * [ethcore-network-devp2p] fix compilation in tests * [*] upgrade rand to 0.6 * [evm] get rid of num-bigint conversions * [ethcore] downgrade trie-standardmap and criterion * [ethcore] fix some warnings * [ethcore] fix compilation in tests * [evmbin] fix compilation in tests * [updater] fix compilation in tests * [ethash] fix compilation in tests * [ethcore-secretstore] fix compilation in tests * [ethcore-sync] fix compilation in tests * [parity-rpc] fix compilation in tests * [ethcore] finally fix compilation in tests FUCK YEAH!!! * [ethstore] lazy_static is unused * [ethcore] fix test * fix up bad merge * [Cargo.toml] remove unused patches * [*] replace some git dependencies with crates.io * [Cargo.toml] remove unused lazy_static * [*] clean up * [ethcore] fix transaction_filter_deprecated test * [private-tx] fix serialization tests * fix more serialization tests * [ethkey] fix smoky test * [rpc] fix tests, please? * [ethcore] remove commented out code * Apply suggestions from code review Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * [ethstore] remove unused dev-dependency * [ethcore] remove resolved TODO * [*] resolve keccak-hash TODO * [*] s/Address::default()/Address::zero() * [rpc] remove Subscribers::new_test * [rpc] remove EthPubSubClient::new_test * [ethcore] use trie-standardmap from crates.io * [dir] fix db_root_path * [ethcore] simplify snapshot::tests::helpers::fill_storage * Apply suggestions from code review Co-Authored-By: David <dvdplm@gmail.com> * [ethcore-secretstore] resolve TODO in serialization * [ethcore-network-devp2p] resolve TODO in save_key * [Cargo.lock] update triehash * [*] use ethabi from crates.io * [ethkey] use secp256k1 from master branch * [Cargo.lock] update eth-secp256k1
2019-06-03 15:36:21 +02:00
block_hash: H256::zero(),
}).unwrap();
let tempdir = TempDir::new("").unwrap();
let db_path = tempdir.path().join("db");
{
let new_db = Arc::new(Database::open(&db_cfg, &db_path.to_string_lossy()).unwrap());
let mut rebuilder = StateRebuilder::new(new_db.clone(), Algorithm::OverlayRecent);
let reader = PackedReader::new(&snap_file).unwrap().unwrap();
let flag = AtomicBool::new(false);
for chunk_hash in &reader.manifest().state_hashes {
let raw = reader.chunk(*chunk_hash).unwrap();
let chunk = snappy::decompress(&raw).unwrap();
match rebuilder.feed(&chunk, &flag) {
Err(Error::Snapshot(SnapshotError::RestorationAborted)) => {},
_ => panic!("unexpected result when feeding with flag off"),
}
}
}
2017-01-03 17:05:27 +01:00
}