* version: bump beta to 2.2.2 * Add experimental RPCs flag (#9928) * WiP * Enable experimental RPCs. * Keep existing blocks when restoring a Snapshot (#8643) * Rename db_restore => client * First step: make it compile! * Second step: working implementation! * Refactoring * Fix tests * PR Grumbles * PR Grumbles WIP * Migrate ancient blocks interating backward * Early return in block migration if snapshot is aborted * Remove RwLock getter (PR Grumble I) * Remove dependency on `Client`: only used Traits * Add test for recovering aborted snapshot recovery * Add test for migrating old blocks * Fix build * PR Grumble I * PR Grumble II * PR Grumble III * PR Grumble IV * PR Grumble V * PR Grumble VI * Fix one test * Fix test * PR Grumble * PR Grumbles * PR Grumbles II * Fix tests * Release RwLock earlier * Revert Cargo.lock * Update _update ancient block_ logic: set local in `commit` * Update typo in ethcore/src/snapshot/service.rs Co-Authored-By: ngotchac <ngotchac@gmail.com> * Adjust requests costs for light client (#9925) * PIP Table Cost relative to average peers instead of max peers * Add tracing in PIP new_cost_table * Update stat peer_count * Use number of leeching peers for Light serve costs * Fix test::light_params_load_share_depends_on_max_peers (wrong type) * Remove (now) useless test * Remove `load_share` from LightParams.Config Prevent div. by 0 * Add LEECHER_COUNT_FACTOR * PR Grumble: u64 to u32 for f64 casting * Prevent u32 overflow for avg_peer_count * Add tests for LightSync::Statistics * Fix empty steps (#9939) * Don't send empty step twice or empty step then block. * Perform basic validation of locally sealed blocks. * Don't include empty step twice. * prevent silent errors in daemon mode, closes #9367 (#9946) * Fix a deadlock (#9952) * Update informant: - decimal in Mgas/s - print every 5s (not randomly between 5s and 10s) * Fix dead-lock in `blockchain.rs` * Update locks ordering * Fix light client informant while syncing (#9932) * Add `is_idle` to LightSync to check importing status * Use SyncStateWrapper to make sure is_idle gets updates * Update is_major_import to use verified queue size as well * Add comment for `is_idle` * Add Debug to `SyncStateWrapper` * `fn get` -> `fn into_inner` * ci: rearrange pipeline by logic (#9970) * ci: rearrange pipeline by logic * ci: rename docs script * fix docker build (#9971) * Deny unknown fields for chainspec (#9972) * Add deny_unknown_fields to chainspec * Add tests and fix existing one * Remove serde_ignored dependency for chainspec * Fix rpc test eth chain spec * Fix starting_nonce_test spec * Improve block and transaction propagation (#9954) * Refactor sync to add priority tasks. * Send priority tasks notifications. * Propagate blocks, optimize transactions. * Implement transaction propagation. Use sync_channel. * Tone down info. * Prevent deadlock by not waiting forever for sync lock. * Fix lock order. * Don't use sync_channel to prevent deadlocks. * Fix tests. * Fix unstable peers and slowness in sync (#9967) * Don't sync all peers after each response * Update formating * Fix tests: add `continue_sync` to `Sync_step` * Update ethcore/sync/src/chain/mod.rs Co-Authored-By: ngotchac <ngotchac@gmail.com> * fix rpc middlewares * fix Cargo.lock * json: resolve merge in spec * rpc: fix starting_nonce_test * ci: allow nightl job to fail
411 lines
12 KiB
Rust
411 lines
12 KiB
Rust
// Copyright 2015-2018 Parity Technologies (UK) Ltd.
|
|
// This file is part of Parity.
|
|
|
|
// Parity is free software: you can redistribute it and/or modify
|
|
// it under the terms of the GNU General Public License as published by
|
|
// the Free Software Foundation, either version 3 of the License, or
|
|
// (at your option) any later version.
|
|
|
|
// Parity is distributed in the hope that it will be useful,
|
|
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
// GNU General Public License for more details.
|
|
|
|
// You should have received a copy of the GNU General Public License
|
|
// along with Parity. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
//! A provider for the PIP protocol. This is typically a full node, who can
|
|
//! give as much data as necessary to its peers.
|
|
|
|
use std::sync::Arc;
|
|
|
|
use ethcore::blockchain_info::BlockChainInfo;
|
|
use ethcore::client::{BlockChainClient, ProvingBlockChainClient, ChainInfo, BlockInfo as ClientBlockInfo};
|
|
use ethcore::ids::BlockId;
|
|
use ethcore::encoded;
|
|
use ethereum_types::H256;
|
|
use parking_lot::RwLock;
|
|
use transaction::PendingTransaction;
|
|
|
|
use cht::{self, BlockInfo};
|
|
use client::{LightChainClient, AsLightClient};
|
|
use transaction_queue::TransactionQueue;
|
|
|
|
use request;
|
|
|
|
/// Maximum allowed size of a headers request.
|
|
pub const MAX_HEADERS_PER_REQUEST: u64 = 512;
|
|
|
|
/// Defines the operations that a provider for the light subprotocol must fulfill.
|
|
pub trait Provider: Send + Sync {
|
|
/// Provide current blockchain info.
|
|
fn chain_info(&self) -> BlockChainInfo;
|
|
|
|
/// Find the depth of a common ancestor between two blocks.
|
|
/// If either block is unknown or an ancestor can't be found
|
|
/// then return `None`.
|
|
fn reorg_depth(&self, a: &H256, b: &H256) -> Option<u64>;
|
|
|
|
/// Earliest block where state queries are available.
|
|
/// If `None`, no state queries are servable.
|
|
fn earliest_state(&self) -> Option<u64>;
|
|
|
|
/// Provide a list of headers starting at the requested block,
|
|
/// possibly in reverse and skipping `skip` at a time.
|
|
///
|
|
/// The returned vector may have any length in the range [0, `max`], but the
|
|
/// results within must adhere to the `skip` and `reverse` parameters.
|
|
fn block_headers(&self, req: request::CompleteHeadersRequest) -> Option<request::HeadersResponse> {
|
|
use request::HashOrNumber;
|
|
|
|
if req.max == 0 { return None }
|
|
|
|
let best_num = self.chain_info().best_block_number;
|
|
let start_num = match req.start {
|
|
HashOrNumber::Number(start_num) => start_num,
|
|
HashOrNumber::Hash(hash) => match self.block_header(BlockId::Hash(hash)) {
|
|
None => {
|
|
trace!(target: "pip_provider", "Unknown block hash {} requested", hash);
|
|
return None;
|
|
}
|
|
Some(header) => {
|
|
let num = header.number();
|
|
let canon_hash = self.block_header(BlockId::Number(num))
|
|
.map(|h| h.hash());
|
|
|
|
if req.max == 1 || canon_hash != Some(hash) {
|
|
// Non-canonical header or single header requested.
|
|
return Some(::request::HeadersResponse {
|
|
headers: vec![header],
|
|
})
|
|
}
|
|
|
|
num
|
|
}
|
|
}
|
|
};
|
|
|
|
let max = ::std::cmp::min(MAX_HEADERS_PER_REQUEST, req.max);
|
|
|
|
let headers: Vec<_> = (0_u64..max)
|
|
.map(|x: u64| x.saturating_mul(req.skip.saturating_add(1)))
|
|
.take_while(|&x| if req.reverse { x < start_num } else { best_num.saturating_sub(start_num) >= x })
|
|
.map(|x| if req.reverse { start_num.saturating_sub(x) } else { start_num.saturating_add(x) })
|
|
.map(|x| self.block_header(BlockId::Number(x)))
|
|
.take_while(|x| x.is_some())
|
|
.flat_map(|x| x)
|
|
.collect();
|
|
|
|
if headers.is_empty() {
|
|
None
|
|
} else {
|
|
Some(::request::HeadersResponse { headers })
|
|
}
|
|
}
|
|
|
|
/// Get a block header by id.
|
|
fn block_header(&self, id: BlockId) -> Option<encoded::Header>;
|
|
|
|
/// Get a transaction index by hash.
|
|
fn transaction_index(&self, req: request::CompleteTransactionIndexRequest)
|
|
-> Option<request::TransactionIndexResponse>;
|
|
|
|
/// Fulfill a block body request.
|
|
fn block_body(&self, req: request::CompleteBodyRequest) -> Option<request::BodyResponse>;
|
|
|
|
/// Fulfill a request for block receipts.
|
|
fn block_receipts(&self, req: request::CompleteReceiptsRequest) -> Option<request::ReceiptsResponse>;
|
|
|
|
/// Get an account proof.
|
|
fn account_proof(&self, req: request::CompleteAccountRequest) -> Option<request::AccountResponse>;
|
|
|
|
/// Get a storage proof.
|
|
fn storage_proof(&self, req: request::CompleteStorageRequest) -> Option<request::StorageResponse>;
|
|
|
|
/// Provide contract code for the specified (block_hash, code_hash) pair.
|
|
fn contract_code(&self, req: request::CompleteCodeRequest) -> Option<request::CodeResponse>;
|
|
|
|
/// Provide a header proof from a given Canonical Hash Trie as well as the
|
|
/// corresponding header.
|
|
fn header_proof(&self, req: request::CompleteHeaderProofRequest) -> Option<request::HeaderProofResponse>;
|
|
|
|
/// Provide pending transactions.
|
|
fn transactions_to_propagate(&self) -> Vec<PendingTransaction>;
|
|
|
|
/// Provide a proof-of-execution for the given transaction proof request.
|
|
/// Returns a vector of all state items necessary to execute the transaction.
|
|
fn transaction_proof(&self, req: request::CompleteExecutionRequest) -> Option<request::ExecutionResponse>;
|
|
|
|
/// Provide epoch signal data at given block hash. This should be just the
|
|
fn epoch_signal(&self, req: request::CompleteSignalRequest) -> Option<request::SignalResponse>;
|
|
}
|
|
|
|
// Implementation of a light client data provider for a client.
|
|
impl<T: ProvingBlockChainClient + ?Sized> Provider for T {
|
|
fn chain_info(&self) -> BlockChainInfo {
|
|
ChainInfo::chain_info(self)
|
|
}
|
|
|
|
fn reorg_depth(&self, a: &H256, b: &H256) -> Option<u64> {
|
|
self.tree_route(a, b).map(|route| route.index as u64)
|
|
}
|
|
|
|
fn earliest_state(&self) -> Option<u64> {
|
|
Some(self.pruning_info().earliest_state)
|
|
}
|
|
|
|
fn block_header(&self, id: BlockId) -> Option<encoded::Header> {
|
|
ClientBlockInfo::block_header(self, id)
|
|
}
|
|
|
|
fn transaction_index(&self, req: request::CompleteTransactionIndexRequest)
|
|
-> Option<request::TransactionIndexResponse>
|
|
{
|
|
use ethcore::ids::TransactionId;
|
|
|
|
self.transaction_receipt(TransactionId::Hash(req.hash)).map(|receipt| request::TransactionIndexResponse {
|
|
num: receipt.block_number,
|
|
hash: receipt.block_hash,
|
|
index: receipt.transaction_index as u64,
|
|
})
|
|
}
|
|
|
|
fn block_body(&self, req: request::CompleteBodyRequest) -> Option<request::BodyResponse> {
|
|
BlockChainClient::block_body(self, BlockId::Hash(req.hash))
|
|
.map(|body| ::request::BodyResponse { body })
|
|
}
|
|
|
|
fn block_receipts(&self, req: request::CompleteReceiptsRequest) -> Option<request::ReceiptsResponse> {
|
|
BlockChainClient::block_receipts(self, &req.hash)
|
|
.map(|x| ::request::ReceiptsResponse { receipts: x.receipts })
|
|
}
|
|
|
|
fn account_proof(&self, req: request::CompleteAccountRequest) -> Option<request::AccountResponse> {
|
|
self.prove_account(req.address_hash, BlockId::Hash(req.block_hash)).map(|(proof, acc)| {
|
|
::request::AccountResponse {
|
|
proof,
|
|
nonce: acc.nonce,
|
|
balance: acc.balance,
|
|
code_hash: acc.code_hash,
|
|
storage_root: acc.storage_root,
|
|
}
|
|
})
|
|
}
|
|
|
|
fn storage_proof(&self, req: request::CompleteStorageRequest) -> Option<request::StorageResponse> {
|
|
self.prove_storage(req.address_hash, req.key_hash, BlockId::Hash(req.block_hash)).map(|(proof, item) | {
|
|
::request::StorageResponse {
|
|
proof,
|
|
value: item,
|
|
}
|
|
})
|
|
}
|
|
|
|
fn contract_code(&self, req: request::CompleteCodeRequest) -> Option<request::CodeResponse> {
|
|
self.state_data(&req.code_hash)
|
|
.map(|code| ::request::CodeResponse { code })
|
|
}
|
|
|
|
fn header_proof(&self, req: request::CompleteHeaderProofRequest) -> Option<request::HeaderProofResponse> {
|
|
let cht_number = match cht::block_to_cht_number(req.num) {
|
|
Some(cht_num) => cht_num,
|
|
None => {
|
|
debug!(target: "pip_provider", "Requested CHT proof with invalid block number");
|
|
return None;
|
|
}
|
|
};
|
|
|
|
let mut needed = None;
|
|
|
|
// build the CHT, caching the requested header as we pass through it.
|
|
let cht = {
|
|
let block_info = |id| {
|
|
let hdr = self.block_header(id);
|
|
let td = self.block_total_difficulty(id);
|
|
|
|
match (hdr, td) {
|
|
(Some(hdr), Some(td)) => {
|
|
let info = BlockInfo {
|
|
hash: hdr.hash(),
|
|
parent_hash: hdr.parent_hash(),
|
|
total_difficulty: td,
|
|
};
|
|
|
|
if hdr.number() == req.num {
|
|
needed = Some((hdr, td));
|
|
}
|
|
|
|
Some(info)
|
|
}
|
|
_ => None,
|
|
}
|
|
};
|
|
|
|
match cht::build(cht_number, block_info) {
|
|
Some(cht) => cht,
|
|
None => return None, // incomplete CHT.
|
|
}
|
|
};
|
|
|
|
let (needed_hdr, needed_td) = needed.expect("`needed` always set in loop, number checked before; qed");
|
|
|
|
// prove our result.
|
|
match cht.prove(req.num, 0) {
|
|
Ok(Some(proof)) => Some(::request::HeaderProofResponse {
|
|
proof,
|
|
hash: needed_hdr.hash(),
|
|
td: needed_td,
|
|
}),
|
|
Ok(None) => None,
|
|
Err(e) => {
|
|
debug!(target: "pip_provider", "Error looking up number in freshly-created CHT: {}", e);
|
|
None
|
|
}
|
|
}
|
|
}
|
|
|
|
fn transaction_proof(&self, req: request::CompleteExecutionRequest) -> Option<request::ExecutionResponse> {
|
|
use transaction::Transaction;
|
|
|
|
let id = BlockId::Hash(req.block_hash);
|
|
let nonce = match self.nonce(&req.from, id) {
|
|
Some(nonce) => nonce,
|
|
None => return None,
|
|
};
|
|
let transaction = Transaction {
|
|
nonce,
|
|
gas: req.gas,
|
|
gas_price: req.gas_price,
|
|
action: req.action,
|
|
value: req.value,
|
|
data: req.data,
|
|
}.fake_sign(req.from);
|
|
|
|
self.prove_transaction(transaction, id)
|
|
.map(|(_, proof)| ::request::ExecutionResponse { items: proof })
|
|
}
|
|
|
|
fn transactions_to_propagate(&self) -> Vec<PendingTransaction> {
|
|
BlockChainClient::transactions_to_propagate(self)
|
|
.into_iter()
|
|
.map(|tx| tx.pending().clone())
|
|
.collect()
|
|
}
|
|
|
|
fn epoch_signal(&self, req: request::CompleteSignalRequest) -> Option<request::SignalResponse> {
|
|
self.epoch_signal(req.block_hash).map(|signal| request::SignalResponse {
|
|
signal,
|
|
})
|
|
}
|
|
}
|
|
|
|
/// The light client "provider" implementation. This wraps a `LightClient` and
|
|
/// a light transaction queue.
|
|
pub struct LightProvider<L> {
|
|
client: Arc<L>,
|
|
txqueue: Arc<RwLock<TransactionQueue>>,
|
|
}
|
|
|
|
impl<L> LightProvider<L> {
|
|
/// Create a new `LightProvider` from the given client and transaction queue.
|
|
pub fn new(client: Arc<L>, txqueue: Arc<RwLock<TransactionQueue>>) -> Self {
|
|
LightProvider {
|
|
client,
|
|
txqueue,
|
|
}
|
|
}
|
|
}
|
|
|
|
// TODO: draw from cache (shared between this and the RPC layer)
|
|
impl<L: AsLightClient + Send + Sync> Provider for LightProvider<L> {
|
|
fn chain_info(&self) -> BlockChainInfo {
|
|
self.client.as_light_client().chain_info()
|
|
}
|
|
|
|
fn reorg_depth(&self, _a: &H256, _b: &H256) -> Option<u64> {
|
|
None
|
|
}
|
|
|
|
fn earliest_state(&self) -> Option<u64> {
|
|
None
|
|
}
|
|
|
|
fn block_header(&self, id: BlockId) -> Option<encoded::Header> {
|
|
self.client.as_light_client().block_header(id)
|
|
}
|
|
|
|
fn transaction_index(&self, _req: request::CompleteTransactionIndexRequest)
|
|
-> Option<request::TransactionIndexResponse>
|
|
{
|
|
None
|
|
}
|
|
|
|
fn block_body(&self, _req: request::CompleteBodyRequest) -> Option<request::BodyResponse> {
|
|
None
|
|
}
|
|
|
|
fn block_receipts(&self, _req: request::CompleteReceiptsRequest) -> Option<request::ReceiptsResponse> {
|
|
None
|
|
}
|
|
|
|
fn account_proof(&self, _req: request::CompleteAccountRequest) -> Option<request::AccountResponse> {
|
|
None
|
|
}
|
|
|
|
fn storage_proof(&self, _req: request::CompleteStorageRequest) -> Option<request::StorageResponse> {
|
|
None
|
|
}
|
|
|
|
fn contract_code(&self, _req: request::CompleteCodeRequest) -> Option<request::CodeResponse> {
|
|
None
|
|
}
|
|
|
|
fn header_proof(&self, _req: request::CompleteHeaderProofRequest) -> Option<request::HeaderProofResponse> {
|
|
None
|
|
}
|
|
|
|
fn transaction_proof(&self, _req: request::CompleteExecutionRequest) -> Option<request::ExecutionResponse> {
|
|
None
|
|
}
|
|
|
|
fn epoch_signal(&self, _req: request::CompleteSignalRequest) -> Option<request::SignalResponse> {
|
|
None
|
|
}
|
|
|
|
fn transactions_to_propagate(&self) -> Vec<PendingTransaction> {
|
|
let chain_info = self.chain_info();
|
|
self.txqueue.read()
|
|
.ready_transactions(chain_info.best_block_number, chain_info.best_block_timestamp)
|
|
}
|
|
}
|
|
|
|
impl<L: AsLightClient> AsLightClient for LightProvider<L> {
|
|
type Client = L::Client;
|
|
|
|
fn as_light_client(&self) -> &L::Client {
|
|
self.client.as_light_client()
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use ethcore::client::{EachBlockWith, TestBlockChainClient};
|
|
use super::Provider;
|
|
|
|
#[test]
|
|
fn cht_proof() {
|
|
let client = TestBlockChainClient::new();
|
|
client.add_blocks(2000, EachBlockWith::Nothing);
|
|
|
|
let req = ::request::CompleteHeaderProofRequest {
|
|
num: 1500,
|
|
};
|
|
|
|
assert!(client.header_proof(req.clone()).is_none());
|
|
|
|
client.add_blocks(48, EachBlockWith::Nothing);
|
|
|
|
assert!(client.header_proof(req.clone()).is_some());
|
|
}
|
|
}
|