openethereum/ethcore/snapshot/src/watcher.rs

201 lines
5.1 KiB
Rust
Raw Normal View History

// Copyright 2015-2019 Parity Technologies (UK) Ltd.
// This file is part of Parity Ethereum.
2016-09-02 18:28:47 +02:00
// Parity Ethereum is free software: you can redistribute it and/or modify
2016-09-02 18:28:47 +02:00
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Parity Ethereum is distributed in the hope that it will be useful,
2016-09-02 18:28:47 +02:00
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Parity Ethereum. If not, see <http://www.gnu.org/licenses/>.
2016-09-02 18:28:47 +02:00
//! Watcher for snapshot-related chain events.
use std::sync::Arc;
use client_traits::{BlockInfo, ChainNotify};
use common_types::{
Make ClientIoMessage generic over the Client (#10981) * Add client-traits crate Move the BlockInfo trait to new crate * New crate `machine` Contains code extracted from ethcore that defines `Machine`, `Externalities` and other execution related code. * Use new machine and client-traits crates in ethcore * Use new crates machine and client-traits instead of ethcore where appropriate * Fix tests * Don't re-export so many types from ethcore::client * Fixing more fallout from removing re-export * fix test * More fallout from not re-exporting types * Add some docs * cleanup * import the macro edition style * Tweak docs * Add missing import * remove unused ethabi_derive imports * Use latest ethabi-contract * Move many traits from ethcore/client/traits to client-traits crate Initial version of extracted Engine trait * Move snapshot related traits to the engine crate (eew) * Move a few snapshot related types to common_types Cleanup Executed as exported from machine crate * fix warning * Gradually introduce new engine crate: snapshot * ethcore typechecks with new engine crate * Sort out types outside ethcore * Add an EpochVerifier to ethash and use that in Engine.epoch_verifier() Cleanup * Document pub members * Sort out tests Sort out default impls for EpochVerifier * Add test-helpers feature and move EngineSigner impl to the right place * Sort out tests * Sort out tests and refactor verification types * Fix missing traits * More missing traits Fix Histogram * Fix tests and cleanup * cleanup * Put back needed logger import * Don't rexport common_types from ethcore/src/client Don't export ethcore::client::* * Remove files no longer used Use types from the engine crate Explicit exports from engine::engine * Get rid of itertools * Move a few more traits from ethcore to client-traits: BlockChainReset, ScheduleInfo, StateClient * Move ProvingBlockChainClient to client-traits * Don't re-export ForkChoice and Transition from ethcore * Address grumbles: sort imports, remove commented out code * Fix merge resolution error * Extract the Clique engine to own crate * Extract NullEngine and the block_reward module from ethcore * Extract InstantSeal engine to own crate * Extract remaining engines * Extract executive_state to own crate so it can be used by engine crates * Remove snapshot stuff from the engine crate * Put snapshot traits back in ethcore * cleanup * Remove stuff from ethcore * Don't use itertools * itertools in aura is legit-ish * More post-merge fixes * Re-export less types in client * cleanup * Extract spec to own crate * Put back the test-helpers from basic-authority * Fix ethcore benchmarks * Reduce the public api of ethcore/verification * WIP * Add Cargo.toml * Fix compilation outside ethcore * Audit uses of import_verified_blocks() and remove unneeded calls Cleanup * cleanup * Remove unused imports from ethcore * Cleanup * remove double semi-colons * Add missing generic param * More missing generics * Update ethcore/block-reward/Cargo.toml Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * Update ethcore/engines/basic-authority/Cargo.toml Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * Update ethcore/engines/ethash/Cargo.toml Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * Update ethcore/engines/clique/src/lib.rs Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * signers is already a ref * Add an EngineType enum to tighten up Engine.name() * Introduce Snapshotting enum to distinguish the type of snapshots a chain uses * Rename supports_warp to snapshot_mode * Missing import * Update ethcore/src/snapshot/consensus/mod.rs Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * missing import * Fix import * double semi * Fix merge problem * cleanup * Parametrise `ClientIoMessage` with `()` for the light client * Add impl Tick for () * Address review feedback * Move ClientIoMessage to common-types * remove superseeded fixme * fix merge conflict errors
2019-08-28 10:09:42 +02:00
ids::BlockId,
io_message::ClientIoMessage,
chain_notify::NewBlocks,
Make ClientIoMessage generic over the Client (#10981) * Add client-traits crate Move the BlockInfo trait to new crate * New crate `machine` Contains code extracted from ethcore that defines `Machine`, `Externalities` and other execution related code. * Use new machine and client-traits crates in ethcore * Use new crates machine and client-traits instead of ethcore where appropriate * Fix tests * Don't re-export so many types from ethcore::client * Fixing more fallout from removing re-export * fix test * More fallout from not re-exporting types * Add some docs * cleanup * import the macro edition style * Tweak docs * Add missing import * remove unused ethabi_derive imports * Use latest ethabi-contract * Move many traits from ethcore/client/traits to client-traits crate Initial version of extracted Engine trait * Move snapshot related traits to the engine crate (eew) * Move a few snapshot related types to common_types Cleanup Executed as exported from machine crate * fix warning * Gradually introduce new engine crate: snapshot * ethcore typechecks with new engine crate * Sort out types outside ethcore * Add an EpochVerifier to ethash and use that in Engine.epoch_verifier() Cleanup * Document pub members * Sort out tests Sort out default impls for EpochVerifier * Add test-helpers feature and move EngineSigner impl to the right place * Sort out tests * Sort out tests and refactor verification types * Fix missing traits * More missing traits Fix Histogram * Fix tests and cleanup * cleanup * Put back needed logger import * Don't rexport common_types from ethcore/src/client Don't export ethcore::client::* * Remove files no longer used Use types from the engine crate Explicit exports from engine::engine * Get rid of itertools * Move a few more traits from ethcore to client-traits: BlockChainReset, ScheduleInfo, StateClient * Move ProvingBlockChainClient to client-traits * Don't re-export ForkChoice and Transition from ethcore * Address grumbles: sort imports, remove commented out code * Fix merge resolution error * Extract the Clique engine to own crate * Extract NullEngine and the block_reward module from ethcore * Extract InstantSeal engine to own crate * Extract remaining engines * Extract executive_state to own crate so it can be used by engine crates * Remove snapshot stuff from the engine crate * Put snapshot traits back in ethcore * cleanup * Remove stuff from ethcore * Don't use itertools * itertools in aura is legit-ish * More post-merge fixes * Re-export less types in client * cleanup * Extract spec to own crate * Put back the test-helpers from basic-authority * Fix ethcore benchmarks * Reduce the public api of ethcore/verification * WIP * Add Cargo.toml * Fix compilation outside ethcore * Audit uses of import_verified_blocks() and remove unneeded calls Cleanup * cleanup * Remove unused imports from ethcore * Cleanup * remove double semi-colons * Add missing generic param * More missing generics * Update ethcore/block-reward/Cargo.toml Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * Update ethcore/engines/basic-authority/Cargo.toml Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * Update ethcore/engines/ethash/Cargo.toml Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * Update ethcore/engines/clique/src/lib.rs Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * signers is already a ref * Add an EngineType enum to tighten up Engine.name() * Introduce Snapshotting enum to distinguish the type of snapshots a chain uses * Rename supports_warp to snapshot_mode * Missing import * Update ethcore/src/snapshot/consensus/mod.rs Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * missing import * Fix import * double semi * Fix merge problem * cleanup * Parametrise `ClientIoMessage` with `()` for the light client * Add impl Tick for () * Address review feedback * Move ClientIoMessage to common-types * remove superseeded fixme * fix merge conflict errors
2019-08-28 10:09:42 +02:00
};
use ethereum_types::H256;
use ethcore_io::IoChannel;
use log::{trace, warn};
use parking_lot::Mutex;
2016-09-02 18:28:47 +02:00
// helper trait for transforming hashes to numbers and checking if syncing.
trait Oracle: Send + Sync {
2016-09-02 18:28:47 +02:00
fn to_number(&self, hash: H256) -> Option<u64>;
fn is_major_importing(&self) -> bool;
2016-09-02 18:28:47 +02:00
}
struct StandardOracle<F> where F: 'static + Send + Sync + Fn() -> bool {
client: Arc<dyn BlockInfo>,
sync_status: F,
}
impl<F> Oracle for StandardOracle<F>
where F: Send + Sync + Fn() -> bool
{
2016-09-02 18:28:47 +02:00
fn to_number(&self, hash: H256) -> Option<u64> {
self.client.block_header(BlockId::Hash(hash)).map(|h| h.number())
2016-09-02 18:28:47 +02:00
}
fn is_major_importing(&self) -> bool {
(self.sync_status)()
}
2016-09-02 18:28:47 +02:00
}
2016-09-05 12:17:21 +02:00
// helper trait for broadcasting a block to take a snapshot at.
trait Broadcast: Send + Sync {
fn take_at(&self, num: Option<u64>);
}
impl<C: 'static> Broadcast for Mutex<IoChannel<ClientIoMessage<C>>> {
2016-09-05 12:17:21 +02:00
fn take_at(&self, num: Option<u64>) {
let num = match num {
Some(n) => n,
None => return,
};
2016-09-05 14:25:56 +02:00
trace!(target: "snapshot_watcher", "broadcast: {}", num);
2016-10-30 09:56:34 +01:00
if let Err(e) = self.lock().send(ClientIoMessage::TakeSnapshot(num)) {
2016-09-05 12:17:21 +02:00
warn!("Snapshot watcher disconnected from IoService: {}", e);
}
}
}
2016-09-02 18:28:47 +02:00
/// A `ChainNotify` implementation which will trigger a snapshot event
/// at certain block numbers.
pub struct Watcher {
oracle: Box<dyn Oracle>,
broadcast: Box<dyn Broadcast>,
2016-09-02 18:28:47 +02:00
period: u64,
history: u64,
}
impl Watcher {
/// Create a new `Watcher` which will trigger a snapshot event
/// once every `period` blocks, but only after that block is
/// `history` blocks old.
pub fn new<F, C>(client: Arc<dyn BlockInfo>, sync_status: F, channel: IoChannel<ClientIoMessage<C>>, period: u64, history: u64) -> Self
where
F: 'static + Send + Sync + Fn() -> bool,
C: 'static + Send + Sync,
{
2016-09-02 18:28:47 +02:00
Watcher {
oracle: Box::new(StandardOracle { client, sync_status }),
2016-10-30 09:56:34 +01:00
broadcast: Box::new(Mutex::new(channel)),
period,
history,
2016-09-02 18:28:47 +02:00
}
}
}
impl ChainNotify for Watcher {
fn new_blocks(&self, new_blocks: NewBlocks) {
if self.oracle.is_major_importing() || new_blocks.has_more_blocks_to_import { return }
trace!(target: "snapshot_watcher", "{} imported", new_blocks.imported.len());
2016-09-05 14:25:56 +02:00
let highest = new_blocks.imported.into_iter()
2016-09-02 18:28:47 +02:00
.filter_map(|h| self.oracle.to_number(h))
.filter(|&num| num >= self.period + self.history)
.map(|num| num - self.history)
.filter(|num| num % self.period == 0)
.fold(0, ::std::cmp::max);
2016-09-05 12:17:21 +02:00
match highest {
0 => self.broadcast.take_at(None),
_ => self.broadcast.take_at(Some(highest)),
2016-09-02 18:28:47 +02:00
}
}
}
#[cfg(test)]
mod tests {
use std::collections::HashMap;
use std::time::Duration;
2016-09-02 18:28:47 +02:00
use client_traits::ChainNotify;
use common_types::chain_notify::{NewBlocks, ChainRoute};
2016-09-02 18:28:47 +02:00
Upgrade ethereum types (#10670) * cargo upgrade "ethereum-types" --all --allow-prerelease * [ethash] fix compilation errors * [ethkey] fix compilation errors * [journaldb] fix compilation errors * [dir] fix compilation errors * [ethabi] update to 0.7 * wip * [eip-712] fix compilation errors * [ethjson] fix compilation errors * [Cargo.toml] add TODO to remove patches * [ethstore] fix compilation errors * use patched keccak-hash with new primitive-types * wip * [ethcore-network-devp2p] fix compilation errors * [vm] fix compilation errors * [common-types, evm, wasm] fix compilation errors * [ethcore-db] Require AsRef instead of Deref for keys * [ethcore-blockchain] fix some compilation errors * [blooms-db] fix compilation errors Thanks a lot @dvdplm :) * we don't need no rlp ethereum feature * [ethcore] fix some compilation errors * [parity-ipfs-api] fix compilation error * [ethcore-light] fix compilation errors * [Cargo.lock] update parity-common * [ethcore-private-tx] fix some compilation errors * wip * [ethcore-private-tx] fix compilation errors * [parity-updater] fix compilation errors * [parity-rpc] fix compilation errors * [parity-bin] fix other compilation errors * update to new ethereum-types * update keccak-hash * [fastmap] fix compilation in tests * [blooms-db] fix compilation in tests * [common-types] fix compilation in tests * [triehash-ethereum] fix compilation in tests * [ethkey] fix compilation in tests * [pwasm-run-test] fix compilation errors * [wasm] fix compilation errors * [ethjson] fix compilation in tests * [eip-712] fix compilation in tests * [ethcore-blockchain] fix compilation in tests * [ethstore] fix compilation in tests * [ethstore-accounts] fix compilation in tests * [parity-hash-fetch] fix compilation in tests * [parity-whisper] fix compilation in tests * [ethcore-miner] fix compilation in tests * [ethcore-network-devp2p] fix compilation in tests * [*] upgrade rand to 0.6 * [evm] get rid of num-bigint conversions * [ethcore] downgrade trie-standardmap and criterion * [ethcore] fix some warnings * [ethcore] fix compilation in tests * [evmbin] fix compilation in tests * [updater] fix compilation in tests * [ethash] fix compilation in tests * [ethcore-secretstore] fix compilation in tests * [ethcore-sync] fix compilation in tests * [parity-rpc] fix compilation in tests * [ethcore] finally fix compilation in tests FUCK YEAH!!! * [ethstore] lazy_static is unused * [ethcore] fix test * fix up bad merge * [Cargo.toml] remove unused patches * [*] replace some git dependencies with crates.io * [Cargo.toml] remove unused lazy_static * [*] clean up * [ethcore] fix transaction_filter_deprecated test * [private-tx] fix serialization tests * fix more serialization tests * [ethkey] fix smoky test * [rpc] fix tests, please? * [ethcore] remove commented out code * Apply suggestions from code review Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * [ethstore] remove unused dev-dependency * [ethcore] remove resolved TODO * [*] resolve keccak-hash TODO * [*] s/Address::default()/Address::zero() * [rpc] remove Subscribers::new_test * [rpc] remove EthPubSubClient::new_test * [ethcore] use trie-standardmap from crates.io * [dir] fix db_root_path * [ethcore] simplify snapshot::tests::helpers::fill_storage * Apply suggestions from code review Co-Authored-By: David <dvdplm@gmail.com> * [ethcore-secretstore] resolve TODO in serialization * [ethcore-network-devp2p] resolve TODO in save_key * [Cargo.lock] update triehash * [*] use ethabi from crates.io * [ethkey] use secp256k1 from master branch * [Cargo.lock] update eth-secp256k1
2019-06-03 15:36:21 +02:00
use ethereum_types::{H256, U256, BigEndianHash};
2016-09-02 18:28:47 +02:00
use super::{Broadcast, Oracle, Watcher};
2016-09-02 18:28:47 +02:00
struct TestOracle(HashMap<H256, u64>);
impl Oracle for TestOracle {
2016-09-02 18:28:47 +02:00
fn to_number(&self, hash: H256) -> Option<u64> {
self.0.get(&hash).cloned()
}
fn is_major_importing(&self) -> bool { false }
2016-09-02 18:28:47 +02:00
}
2016-09-05 12:17:21 +02:00
struct TestBroadcast(Option<u64>);
impl Broadcast for TestBroadcast {
fn take_at(&self, num: Option<u64>) {
if num != self.0 {
panic!("Watcher broadcast wrong number. Expected {:?}, found {:?}", self.0, num);
2016-09-02 18:28:47 +02:00
}
}
}
2016-09-05 12:17:21 +02:00
// helper harness for tests which expect a notification.
fn harness(numbers: Vec<u64>, period: u64, history: u64, expected: Option<u64>) {
const DURATION_ZERO: Duration = Duration::from_millis(0);
Upgrade ethereum types (#10670) * cargo upgrade "ethereum-types" --all --allow-prerelease * [ethash] fix compilation errors * [ethkey] fix compilation errors * [journaldb] fix compilation errors * [dir] fix compilation errors * [ethabi] update to 0.7 * wip * [eip-712] fix compilation errors * [ethjson] fix compilation errors * [Cargo.toml] add TODO to remove patches * [ethstore] fix compilation errors * use patched keccak-hash with new primitive-types * wip * [ethcore-network-devp2p] fix compilation errors * [vm] fix compilation errors * [common-types, evm, wasm] fix compilation errors * [ethcore-db] Require AsRef instead of Deref for keys * [ethcore-blockchain] fix some compilation errors * [blooms-db] fix compilation errors Thanks a lot @dvdplm :) * we don't need no rlp ethereum feature * [ethcore] fix some compilation errors * [parity-ipfs-api] fix compilation error * [ethcore-light] fix compilation errors * [Cargo.lock] update parity-common * [ethcore-private-tx] fix some compilation errors * wip * [ethcore-private-tx] fix compilation errors * [parity-updater] fix compilation errors * [parity-rpc] fix compilation errors * [parity-bin] fix other compilation errors * update to new ethereum-types * update keccak-hash * [fastmap] fix compilation in tests * [blooms-db] fix compilation in tests * [common-types] fix compilation in tests * [triehash-ethereum] fix compilation in tests * [ethkey] fix compilation in tests * [pwasm-run-test] fix compilation errors * [wasm] fix compilation errors * [ethjson] fix compilation in tests * [eip-712] fix compilation in tests * [ethcore-blockchain] fix compilation in tests * [ethstore] fix compilation in tests * [ethstore-accounts] fix compilation in tests * [parity-hash-fetch] fix compilation in tests * [parity-whisper] fix compilation in tests * [ethcore-miner] fix compilation in tests * [ethcore-network-devp2p] fix compilation in tests * [*] upgrade rand to 0.6 * [evm] get rid of num-bigint conversions * [ethcore] downgrade trie-standardmap and criterion * [ethcore] fix some warnings * [ethcore] fix compilation in tests * [evmbin] fix compilation in tests * [updater] fix compilation in tests * [ethash] fix compilation in tests * [ethcore-secretstore] fix compilation in tests * [ethcore-sync] fix compilation in tests * [parity-rpc] fix compilation in tests * [ethcore] finally fix compilation in tests FUCK YEAH!!! * [ethstore] lazy_static is unused * [ethcore] fix test * fix up bad merge * [Cargo.toml] remove unused patches * [*] replace some git dependencies with crates.io * [Cargo.toml] remove unused lazy_static * [*] clean up * [ethcore] fix transaction_filter_deprecated test * [private-tx] fix serialization tests * fix more serialization tests * [ethkey] fix smoky test * [rpc] fix tests, please? * [ethcore] remove commented out code * Apply suggestions from code review Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * [ethstore] remove unused dev-dependency * [ethcore] remove resolved TODO * [*] resolve keccak-hash TODO * [*] s/Address::default()/Address::zero() * [rpc] remove Subscribers::new_test * [rpc] remove EthPubSubClient::new_test * [ethcore] use trie-standardmap from crates.io * [dir] fix db_root_path * [ethcore] simplify snapshot::tests::helpers::fill_storage * Apply suggestions from code review Co-Authored-By: David <dvdplm@gmail.com> * [ethcore-secretstore] resolve TODO in serialization * [ethcore-network-devp2p] resolve TODO in save_key * [Cargo.lock] update triehash * [*] use ethabi from crates.io * [ethkey] use secp256k1 from master branch * [Cargo.lock] update eth-secp256k1
2019-06-03 15:36:21 +02:00
let hashes: Vec<_> = numbers.clone().into_iter().map(|x| BigEndianHash::from_uint(&U256::from(x))).collect();
2016-09-05 12:17:21 +02:00
let map = hashes.clone().into_iter().zip(numbers).collect();
2016-09-02 18:28:47 +02:00
let watcher = Watcher {
oracle: Box::new(TestOracle(map)),
2016-09-05 12:17:21 +02:00
broadcast: Box::new(TestBroadcast(expected)),
period,
history,
2016-09-02 18:28:47 +02:00
};
watcher.new_blocks(NewBlocks::new(
2016-09-02 18:28:47 +02:00
hashes,
vec![],
ChainRoute::default(),
2016-09-02 18:28:47 +02:00
vec![],
2016-12-08 12:03:34 +01:00
vec![],
DURATION_ZERO,
false
));
2016-09-02 18:28:47 +02:00
}
2016-09-05 12:17:21 +02:00
// helper
2016-09-02 18:28:47 +02:00
#[test]
fn should_not_fire() {
2016-09-05 12:17:21 +02:00
harness(vec![0], 5, 0, None);
2016-09-02 18:28:47 +02:00
}
#[test]
fn fires_once_for_two() {
2016-09-05 12:17:21 +02:00
harness(vec![14, 15], 10, 5, Some(10));
2016-09-02 18:28:47 +02:00
}
#[test]
fn finds_highest() {
2016-09-05 12:17:21 +02:00
harness(vec![15, 25], 10, 5, Some(20));
2016-09-02 18:28:47 +02:00
}
#[test]
fn doesnt_fire_before_history() {
2016-09-05 12:17:21 +02:00
harness(vec![10, 11], 10, 5, None);
2016-09-02 18:28:47 +02:00
}
}