openethereum/parity/blockchain.rs

487 lines
14 KiB
Rust
Raw Normal View History

2016-12-11 19:30:54 +01:00
// Copyright 2015, 2016 Parity Technologies (UK) Ltd.
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// This file is part of Parity.
// Parity is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Parity is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Parity. If not, see <http://www.gnu.org/licenses/>.
use std::str::{FromStr, from_utf8};
use std::{io, fs};
use std::io::{BufReader, BufRead};
2016-09-15 16:56:10 +02:00
use std::time::{Instant, Duration};
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
use std::thread::sleep;
use std::sync::Arc;
use rustc_serialize::hex::FromHex;
use io::{PanicHandler, ForwardPanic};
2016-11-27 11:57:05 +01:00
use util::{ToPretty, Uint, U256, H256, Address, Hashable};
2016-09-01 14:55:07 +02:00
use rlp::PayloadInfo;
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
use ethcore::service::ClientService;
use ethcore::client::{Mode, DatabaseCompactionProfile, VMType, BlockImportError, BlockChainClient, BlockId};
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
use ethcore::error::ImportError;
use ethcore::miner::Miner;
use ethcore::verification::queue::VerifierSettings;
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
use cache::CacheConfig;
2016-09-15 16:56:10 +02:00
use informant::{Informant, MillisecondDuration};
use params::{SpecType, Pruning, Switch, tracing_switch_to_bool, fatdb_switch_to_bool};
use io_handler::ImportIoHandler;
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
use helpers::{to_client_config, execute_upgrades};
use dir::Directories;
use user_defaults::UserDefaults;
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
use fdlimit;
#[derive(Debug, PartialEq)]
pub enum DataFormat {
Hex,
Binary,
}
impl Default for DataFormat {
fn default() -> Self {
DataFormat::Binary
}
}
impl FromStr for DataFormat {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s {
"binary" | "bin" => Ok(DataFormat::Binary),
"hex" => Ok(DataFormat::Hex),
x => Err(format!("Invalid format: {}", x))
}
}
}
#[derive(Debug, PartialEq)]
pub enum BlockchainCmd {
Import(ImportBlockchain),
Export(ExportBlockchain),
2016-11-27 11:11:56 +01:00
ExportState(ExportState),
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
}
#[derive(Debug, PartialEq)]
pub struct ImportBlockchain {
pub spec: SpecType,
pub cache_config: CacheConfig,
pub dirs: Directories,
pub file_path: Option<String>,
pub format: Option<DataFormat>,
pub pruning: Pruning,
pub pruning_history: u64,
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
pub compaction: DatabaseCompactionProfile,
pub wal: bool,
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
pub tracing: Switch,
pub fat_db: Switch,
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
pub vm_type: VMType,
pub check_seal: bool,
pub with_color: bool,
pub verifier_settings: VerifierSettings,
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
}
#[derive(Debug, PartialEq)]
pub struct ExportBlockchain {
pub spec: SpecType,
pub cache_config: CacheConfig,
pub dirs: Directories,
pub file_path: Option<String>,
pub format: Option<DataFormat>,
pub pruning: Pruning,
pub pruning_history: u64,
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
pub compaction: DatabaseCompactionProfile,
pub wal: bool,
pub fat_db: Switch,
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
pub tracing: Switch,
pub from_block: BlockId,
pub to_block: BlockId,
pub check_seal: bool,
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
}
2016-11-27 11:11:56 +01:00
#[derive(Debug, PartialEq)]
pub struct ExportState {
pub spec: SpecType,
pub cache_config: CacheConfig,
pub dirs: Directories,
pub file_path: Option<String>,
pub format: Option<DataFormat>,
pub pruning: Pruning,
pub pruning_history: u64,
pub compaction: DatabaseCompactionProfile,
pub wal: bool,
pub fat_db: Switch,
pub tracing: Switch,
pub at: BlockId,
2016-11-27 11:11:56 +01:00
pub storage: bool,
pub code: bool,
2016-11-27 18:16:43 +01:00
pub min_balance: Option<U256>,
pub max_balance: Option<U256>,
2016-11-27 11:11:56 +01:00
}
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
pub fn execute(cmd: BlockchainCmd) -> Result<String, String> {
match cmd {
BlockchainCmd::Import(import_cmd) => execute_import(import_cmd),
BlockchainCmd::Export(export_cmd) => execute_export(export_cmd),
2016-11-27 11:11:56 +01:00
BlockchainCmd::ExportState(export_cmd) => execute_export_state(export_cmd),
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
}
}
fn execute_import(cmd: ImportBlockchain) -> Result<String, String> {
2016-09-15 16:56:10 +02:00
let timer = Instant::now();
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// Setup panic handler
let panic_handler = PanicHandler::new_in_arc();
// create dirs used by parity
try!(cmd.dirs.create_dirs(false, false));
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// load spec file
let spec = try!(cmd.spec.spec());
// load genesis hash
let genesis_hash = spec.genesis_header().hash();
// database paths
let db_dirs = cmd.dirs.database(genesis_hash, spec.fork_name.clone());
// user defaults path
let user_defaults_path = db_dirs.user_defaults_path();
// load user defaults
let mut user_defaults = try!(UserDefaults::load(&user_defaults_path));
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
fdlimit::raise_fd_limit();
// select pruning algorithm
let algorithm = cmd.pruning.to_algorithm(&user_defaults);
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// check if tracing is on
let tracing = try!(tracing_switch_to_bool(cmd.tracing, &user_defaults));
// check if fatdb is on
let fat_db = try!(fatdb_switch_to_bool(cmd.fat_db, &user_defaults, algorithm));
// prepare client and snapshot paths.
let client_path = db_dirs.client_path(algorithm);
let snapshot_path = db_dirs.snapshot_path();
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// execute upgrades
try!(execute_upgrades(&db_dirs, algorithm, cmd.compaction.compaction_profile(db_dirs.fork_path().as_path())));
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// prepare client config
let mut client_config = to_client_config(
&cmd.cache_config,
Mode::Active,
tracing,
fat_db,
cmd.compaction,
cmd.wal,
cmd.vm_type,
"".into(),
algorithm,
cmd.pruning_history,
cmd.check_seal
);
client_config.queue.verifier_settings = cmd.verifier_settings;
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// build client
let service = try!(ClientService::start(
client_config,
&spec,
&client_path,
&snapshot_path,
&cmd.dirs.ipc_path(),
Arc::new(Miner::with_spec(&spec)),
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
).map_err(|e| format!("Client service error: {:?}", e)));
2016-11-15 19:07:23 +01:00
// free up the spec in memory.
drop(spec);
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
panic_handler.forward_from(&service);
let client = service.client();
let mut instream: Box<io::Read> = match cmd.file_path {
Some(f) => Box::new(try!(fs::File::open(&f).map_err(|_| format!("Cannot open given file: {}", f)))),
None => Box::new(io::stdin()),
};
const READAHEAD_BYTES: usize = 8;
let mut first_bytes: Vec<u8> = vec![0; READAHEAD_BYTES];
let mut first_read = 0;
let format = match cmd.format {
Some(format) => format,
None => {
first_read = try!(instream.read(&mut first_bytes).map_err(|_| "Error reading from the file/stream."));
match first_bytes[0] {
0xf9 => DataFormat::Binary,
_ => DataFormat::Hex,
}
}
};
let informant = Informant::new(client.clone(), None, None, None, cmd.with_color);
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
try!(service.register_io_handler(Arc::new(ImportIoHandler {
info: Arc::new(informant),
})).map_err(|_| "Unable to register informant handler".to_owned()));
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
let do_import = |bytes| {
while client.queue_info().is_full() { sleep(Duration::from_secs(1)); }
match client.import_block(bytes) {
Err(BlockImportError::Import(ImportError::AlreadyInChain)) => {
trace!("Skipping block already in chain.");
}
Err(e) => {
return Err(format!("Cannot import block: {:?}", e));
},
Ok(_) => {},
}
Ok(())
};
match format {
DataFormat::Binary => {
loop {
let mut bytes = if first_read > 0 {first_bytes.clone()} else {vec![0; READAHEAD_BYTES]};
let n = if first_read > 0 {
first_read
} else {
try!(instream.read(&mut bytes).map_err(|_| "Error reading from the file/stream."))
};
if n == 0 { break; }
first_read = 0;
let s = try!(PayloadInfo::from(&bytes).map_err(|e| format!("Invalid RLP in the file/stream: {:?}", e))).total();
bytes.resize(s, 0);
try!(instream.read_exact(&mut bytes[n..]).map_err(|_| "Error reading from the file/stream."));
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
try!(do_import(bytes));
}
}
DataFormat::Hex => {
for line in BufReader::new(instream).lines() {
let s = try!(line.map_err(|_| "Error reading from the file/stream."));
let s = if first_read > 0 {from_utf8(&first_bytes).unwrap().to_owned() + &(s[..])} else {s};
first_read = 0;
let bytes = try!(s.from_hex().map_err(|_| "Invalid hex in file/stream."));
try!(do_import(bytes));
}
}
}
client.flush_queue();
// save user defaults
user_defaults.pruning = algorithm;
user_defaults.tracing = tracing;
2016-11-27 12:58:14 +01:00
user_defaults.fat_db = fat_db;
try!(user_defaults.save(&user_defaults_path));
2016-09-15 16:56:10 +02:00
let report = client.report();
let ms = timer.elapsed().as_milliseconds();
Ok(format!("Import completed in {} seconds, {} blocks, {} blk/s, {} transactions, {} tx/s, {} Mgas, {} Mgas/s",
ms / 1000,
report.blocks_imported,
(report.blocks_imported * 1000) as u64 / ms,
report.transactions_applied,
(report.transactions_applied * 1000) as u64 / ms,
report.gas_processed / From::from(1_000_000),
(report.gas_processed / From::from(ms * 1000)).low_u64(),
).into())
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
}
2016-11-27 11:11:56 +01:00
fn start_client(
dirs: Directories,
spec: SpecType,
pruning: Pruning,
pruning_history: u64,
tracing: Switch,
fat_db: Switch,
compaction: DatabaseCompactionProfile,
wal: bool,
cache_config: CacheConfig) -> Result<ClientService, String> {
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// create dirs used by parity
2016-11-27 11:11:56 +01:00
try!(dirs.create_dirs(false, false));
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// load spec file
2016-11-27 11:11:56 +01:00
let spec = try!(spec.spec());
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// load genesis hash
let genesis_hash = spec.genesis_header().hash();
// database paths
2016-11-27 11:11:56 +01:00
let db_dirs = dirs.database(genesis_hash, spec.fork_name.clone());
// user defaults path
let user_defaults_path = db_dirs.user_defaults_path();
// load user defaults
let user_defaults = try!(UserDefaults::load(&user_defaults_path));
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
fdlimit::raise_fd_limit();
// select pruning algorithm
2016-11-27 11:11:56 +01:00
let algorithm = pruning.to_algorithm(&user_defaults);
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// check if tracing is on
2016-11-27 11:11:56 +01:00
let tracing = try!(tracing_switch_to_bool(tracing, &user_defaults));
// check if fatdb is on
2016-11-27 11:11:56 +01:00
let fat_db = try!(fatdb_switch_to_bool(fat_db, &user_defaults, algorithm));
// prepare client and snapshot paths.
let client_path = db_dirs.client_path(algorithm);
let snapshot_path = db_dirs.snapshot_path();
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// execute upgrades
2016-11-27 11:11:56 +01:00
try!(execute_upgrades(&db_dirs, algorithm, compaction.compaction_profile(db_dirs.fork_path().as_path())));
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
// prepare client config
2016-11-27 11:11:56 +01:00
let client_config = to_client_config(&cache_config, Mode::Active, tracing, fat_db, compaction, wal, VMType::default(), "".into(), algorithm, pruning_history, true);
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
let service = try!(ClientService::start(
2016-11-27 18:26:23 +01:00
client_config,
&spec,
&client_path,
&snapshot_path,
&dirs.ipc_path(),
Arc::new(Miner::with_spec(&spec)),
).map_err(|e| format!("Client service error: {:?}", e)));
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
2016-11-15 19:07:23 +01:00
drop(spec);
2016-11-27 11:11:56 +01:00
Ok(service)
}
fn execute_export(cmd: ExportBlockchain) -> Result<String, String> {
// Setup panic handler
let service = try!(start_client(cmd.dirs, cmd.spec, cmd.pruning, cmd.pruning_history, cmd.tracing, cmd.fat_db, cmd.compaction, cmd.wal, cmd.cache_config));
let panic_handler = PanicHandler::new_in_arc();
let format = cmd.format.unwrap_or_default();
2016-11-15 19:07:23 +01:00
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
panic_handler.forward_from(&service);
let client = service.client();
let mut out: Box<io::Write> = match cmd.file_path {
Some(f) => Box::new(try!(fs::File::create(&f).map_err(|_| format!("Cannot write to file given: {}", f)))),
None => Box::new(io::stdout()),
};
let from = try!(client.block_number(cmd.from_block).ok_or("From block could not be found"));
let to = try!(client.block_number(cmd.to_block).ok_or("To block could not be found"));
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
for i in from..(to + 1) {
2016-11-27 11:11:56 +01:00
if i % 10000 == 0 {
info!("#{}", i);
}
let b = try!(client.block(BlockId::Number(i)).ok_or("Error exporting incomplete chain"));
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
match format {
DataFormat::Binary => { out.write(&b).expect("Couldn't write to stream."); }
DataFormat::Hex => { out.write_fmt(format_args!("{}", b.pretty())).expect("Couldn't write to stream."); }
}
}
Ok("Export completed.".into())
}
2016-11-27 11:11:56 +01:00
fn execute_export_state(cmd: ExportState) -> Result<String, String> {
// Setup panic handler
let service = try!(start_client(cmd.dirs, cmd.spec, cmd.pruning, cmd.pruning_history, cmd.tracing, cmd.fat_db, cmd.compaction, cmd.wal, cmd.cache_config));
let panic_handler = PanicHandler::new_in_arc();
panic_handler.forward_from(&service);
let client = service.client();
let mut out: Box<io::Write> = match cmd.file_path {
Some(f) => Box::new(try!(fs::File::create(&f).map_err(|_| format!("Cannot write to file given: {}", f)))),
None => Box::new(io::stdout()),
};
let mut last: Option<Address> = None;
let at = cmd.at;
let mut i = 0usize;
2016-11-27 12:58:14 +01:00
out.write_fmt(format_args!("{{ \"state\": [", )).expect("Couldn't write to stream.");
2016-11-27 11:11:56 +01:00
loop {
let accounts = try!(client.list_accounts(at, last.as_ref(), 1000).ok_or("Specified block not found"));
if accounts.is_empty() {
break;
}
for account in accounts.into_iter() {
2016-11-27 18:16:43 +01:00
let balance = client.balance(&account, at).unwrap_or_else(U256::zero);
if cmd.min_balance.map_or(false, |m| balance < m) || cmd.max_balance.map_or(false, |m| balance > m) {
2016-11-28 01:32:40 +01:00
last = Some(account);
2016-11-27 18:16:43 +01:00
continue; //filtered out
}
2016-11-27 11:11:56 +01:00
if i != 0 {
out.write(b",").expect("Write error");
}
2016-11-27 18:16:43 +01:00
out.write_fmt(format_args!("\n\"0x{}\": {{\"balance\": \"{:x}\", \"nonce\": \"{:x}\"", account.hex(), balance, client.nonce(&account, at).unwrap_or_else(U256::zero))).expect("Write error");
2016-11-27 11:11:56 +01:00
let code = client.code(&account, at).unwrap_or(None).unwrap_or_else(Vec::new);
if !code.is_empty() {
out.write_fmt(format_args!(", \"code_hash\": \"0x{}\"", code.sha3().hex())).expect("Write error");
if cmd.code {
out.write_fmt(format_args!(", \"code\": \"{}\"", code.to_hex())).expect("Write error");
}
}
let storage_root = client.storage_root(&account, at).unwrap_or(::util::SHA3_NULL_RLP);
if storage_root != ::util::SHA3_NULL_RLP {
out.write_fmt(format_args!(", \"storage_root\": \"0x{}\"", storage_root.hex())).expect("Write error");
if cmd.storage {
2016-11-27 11:57:05 +01:00
out.write_fmt(format_args!(", \"storage\": {{")).expect("Write error");
let mut last_storage: Option<H256> = None;
loop {
let keys = try!(client.list_storage(at, &account, last_storage.as_ref(), 1000).ok_or("Specified block not found"));
if keys.is_empty() {
break;
}
let mut si = 0;
for key in keys.into_iter() {
if si != 0 {
out.write(b",").expect("Write error");
}
out.write_fmt(format_args!("\n\t\"0x{}\": \"0x{}\"", key.hex(), client.storage_at(&account, &key, at).unwrap_or_else(Default::default).hex())).expect("Write error");
si += 1;
last_storage = Some(key);
}
}
out.write(b"\n}").expect("Write error");
2016-11-27 11:11:56 +01:00
}
}
out.write(b"}").expect("Write error");
i += 1;
if i % 10000 == 0 {
2016-11-27 18:16:43 +01:00
info!("Account #{}", i);
2016-11-27 11:11:56 +01:00
}
last = Some(account);
}
}
out.write_fmt(format_args!("\n]}}")).expect("Write error");
Ok("Export completed.".into())
}
cli overhaul (#1600) * cli commands * cleanup parity/signer * cleanup parity/signer * remove redundant import of signer crate from main.rs * cli cleanup in progress * cli cleanup in progress * moved few commonly used functions to separate methods with tests * cleaning up blockchain import in progress * cleaning up blockchain import in progress2 * cleaning up blockchain import in progress3 * tests for database compaction profile parsing * cleaning up blockchain import in progress4 * cleaning up blockchain import in progress5 * blockchain import * export blockchain in progress * cleanup execute_export * Configuration::to_duration cleaned up * removed unused code, tests for to_duration * cleanup Configuration::mode function * parsing some of the cli params in params.rs * rpc and signer are no longer optional * move importing extern crates to main.rs file * swipe dies from rpc module * swipe dies from dapps * finding deprecated * several tests and fixes for parity * parity cleanup in progress * cleanup price parsing * parity cleanup in progress * swiped all dies * parity cleanup in progress * replace usages of from_str with parse() in parity/params.rs * removed few more from_str * split parity/params.rs into params and helpers * removed wildcard import from configuration.rs * cleanup directories/path creation * cleaning up run cmd * moved LoggerConfig * defaults for cli params * fixed indention in raise_fd_limit * tests for rpc_apis * tests for default ipc and rpc settings * ipc socket * cleanup in progress * account service * cleanup miner config * BlockChain commands use Directiores structure now * client_config * network settings and dapps configuration * removing warnings * default logger config * fixed client_path * overhaul * fixing export && import * default export DataFormat * import and export also upgrade db * fixed export && import * polishing pr * polishing pr * fixed custom bootnodes * fixed daemonize on windows * fixed setting up enable network * finished pr * fixed compiling on windows * Fixed warning; windows build * Better cache management * Fixed tests on windows * Fixed test * Restored pruning method names * --cache alias * Fixed more tests * Ensure default options actually listed as valid [ci:skip]
2016-07-25 16:09:47 +02:00
#[cfg(test)]
mod test {
use super::DataFormat;
#[test]
fn test_data_format_parsing() {
assert_eq!(DataFormat::Binary, "binary".parse().unwrap());
assert_eq!(DataFormat::Binary, "bin".parse().unwrap());
assert_eq!(DataFormat::Hex, "hex".parse().unwrap());
}
}