Go to file
David 8c2199dd2a
Snapshot restoration overhaul (#11219)
* Comments and todos
Use `snapshot_sync` as logging target

* fix compilation

* More todos, more logs

* Fix picking snapshot peer: prefer the one with the highest block number
More docs, comments, todos

* Adjust WAIT_PEERS_TIMEOUT to be a multiple of MAINTAIN_SYNC_TIMER to try to fix snapshot startup problems
Docs, todos, comments

* Tabs

* Formatting

* Don't build new rlp::EMPTY_LIST_RLP instances

* Dial down debug logging

* Don't warn about missing hashes in the manifest: it's normal
Log client version on peer connect

* Cleanup

* Do not skip snapshots further away than 30k block from the highest block seen

Currently we look for peers that seed snapshots that are close to the highest block seen on the network (where "close" means withing 30k blocks). When a node starts up we wait for some time (5sec, increased here to 10sec) to let peers connect and if we have found a suitable peer to sync a snapshot from at the end of that delay, we start the download; if none is found and --warp-barrier is used we stall, otherwise we start a slow-sync.
When looking for a suitable snapshot, we use the highest block seen on the network to check if a peer has a snapshot that is within 30k blocks of that highest block number. This means that in a situation where all available snapshots are older than that, we will often fail to start a snapshot at all. What's worse is that the longer we delay starting a snapshot sync (to let more peers connect, in the hope of finding a good snapshot), the more likely we are to have seen a high block and thus the more likely we become to accept a snapshot.
This commit removes this comparison with the highest blocknumber criteria entirely and picks the best snapshot we find in 10sec.

* lockfile

* Add a `ChunkType::Dupe` variant so that we do not disconnect a peer if they happen to send us a duplicate chunk (just ignore the chunk and keep going)
Resolve some documentation todos, add more

* tweak log message

* Don't warp sync twice
Check if our own block is beyond the given warp barrier (can happen after we've completed a warp sync but are not quite yet synced up to the tip) and if so, don't sync.
More docs, resolve todos.
Dial down some `sync` debug level logging to trace

* Avoid iterating over all snapshot block/state hashes to find the next work item

Use a HashSet instead of a Vec and remove items from the set as chunks are processed. Calculate and store the total number of chunks in the `Snapshot`  struct instead of counting pending chunks each time.

* Address review grumbles

* Log correct number of bytes written to disk

* Revert ChunkType::Dup change

* whitespace grumble

* Cleanup debugging code

* Fix docs

* Fix import and a typo

* Fix test impl

* Use `indexmap::IndexSet` to ensure chunk hashes are accessed in order

* Revert increased SNAPSHOT_MANIFEST_TIMEOUT: 5sec should be enough
2019-10-31 16:07:21 +01:00
.cargo Enable aesni (#10756) 2019-06-18 20:24:21 +02:00
.github docs: Update Readme with TOC, Contributor Guideline. Update Cargo package descriptions (#10652) 2019-06-25 08:15:13 +02:00
accounts Crypto primitives removed from ethkey (#11174) 2019-10-23 13:03:46 +02:00
chainspec docs: Update Readme with TOC, Contributor Guideline. Update Cargo package descriptions (#10652) 2019-06-25 08:15:13 +02:00
cli-signer Upgrade jsonrpc to latest (#11206) 2019-10-26 11:26:04 +02:00
docs Update README.md and Changelogs (#10866) 2019-07-12 15:50:58 +02:00
ethash Update ethereum types to 0.8.0 version (#11139) 2019-10-08 14:18:44 +02:00
ethcore Snapshot restoration overhaul (#11219) 2019-10-31 16:07:21 +01:00
evmbin Update ethereum types to 0.8.0 version (#11139) 2019-10-08 14:18:44 +02:00
ipfs Upgrade jsonrpc to latest (#11206) 2019-10-26 11:26:04 +02:00
json Step duration map configuration parameter ported from the POA Network fork (#10902) 2019-10-28 14:39:18 +01:00
miner Use provided usd-per-eth value if an endpoint is specified (#11209) 2019-10-29 12:18:30 +01:00
parity Use provided usd-per-eth value if an endpoint is specified (#11209) 2019-10-29 12:18:30 +01:00
parity-clib Upgrade to jsonrpc v14 (#11151) 2019-10-10 15:56:22 +01:00
rpc Upgrade jsonrpc to latest (#11206) 2019-10-26 11:26:04 +02:00
scripts Fix docker centos build (#11226) 2019-10-31 14:59:51 +01:00
secret-store Upgrade jsonrpc to latest (#11206) 2019-10-26 11:26:04 +02:00
updater [dependencies] bump rand 0.7 (#11022) 2019-10-18 15:12:16 +02:00
util Snapshot restoration overhaul (#11219) 2019-10-31 16:07:21 +01:00
.dockerignore improved dockerfile builds (#5659) 2017-05-23 12:25:41 +02:00
.editorconfig Fix checksums and auto-update push (#7846) 2018-02-13 19:31:06 +01:00
.gitignore Move the C/C++ example to another directory (#9032) 2018-07-04 13:06:53 +03:00
.gitlab-ci.yml retry on gitlab system failures (#11222) 2019-10-30 19:16:18 +01:00
.gitmodules WASM contracts MVP (#5679) 2017-07-10 17:42:10 +02:00
Cargo.lock Snapshot restoration overhaul (#11219) 2019-10-31 16:07:21 +01:00
Cargo.toml Upgrade jsonrpc to latest (#11206) 2019-10-26 11:26:04 +02:00
CHANGELOG.md Update README.md and Changelogs (#10866) 2019-07-12 15:50:58 +02:00
LICENSE Change to library. 2015-11-24 21:05:08 +01:00
license_header misc: bump license header to 2019 (#10135) 2019-01-07 11:33:07 +01:00
README.md Type annotation for next_key() matching of json filter options (#11192) 2019-10-23 14:20:47 +02:00
rustfmt.toml Use memmap for dag cache (#6193) 2017-09-25 19:45:33 +02:00
SECURITY.md Updated security@parity.io key (#10939) 2019-08-05 14:09:32 +01:00

Parity Ethereum

The Fastest and most Advanced Ethereum Client.

» Download the latest release «

Table of Contents

  1. Description
  2. Technical Overview
  3. Building
    3.1 Building Dependencies
    3.2 Building from Source Code
    3.3 Simple One-Line Installer for Mac and Linux
    3.4 Starting Parity Ethereum
  4. Testing
  5. Documentation
  6. Toolchain
  7. Community
  8. Contributing
  9. License

1. Description

Built for mission-critical use: Miners, service providers, and exchanges need fast synchronisation and maximum uptime. Parity Ethereum provides the core infrastructure essential for speedy and reliable services.

  • Clean, modular codebase for easy customisation
  • Advanced CLI-based client
  • Minimal memory and storage footprint
  • Synchronise in hours, not days with Warp Sync
  • Modular for light integration into your service or product

2. Technical Overview

Parity Ethereum's goal is to be the fastest, lightest, and most secure Ethereum client. We are developing Parity Ethereum using the sophisticated and cutting-edge Rust programming language. Parity Ethereum is licensed under the GPLv3 and can be used for all your Ethereum needs.

By default, Parity Ethereum runs a JSON-RPC HTTP server on port :8545 and a Web-Sockets server on port :8546. This is fully configurable and supports a number of APIs.

If you run into problems while using Parity Ethereum, check out the wiki for documentation, feel free to file an issue in this repository, or hop on our Gitter or Riot chat room to ask a question. We are glad to help! For security-critical issues, please refer to the security policy outlined in SECURITY.md.

Parity Ethereum's current beta-release is 2.6. You can download it at the releases page or follow the instructions below to build from source. Please, mind the CHANGELOG.md for a list of all changes between different versions.

3. Building

3.1 Build Dependencies

Parity Ethereum requires latest stable Rust version to build.

We recommend installing Rust through rustup. If you don't already have rustup, you can install it like this:

  • Linux:

    $ curl https://sh.rustup.rs -sSf | sh
    

    Parity Ethereum also requires gcc, g++, pkg-config, file, make, and cmake packages to be installed.

  • OSX:

    $ curl https://sh.rustup.rs -sSf | sh
    

    clang is required. It comes with Xcode command line tools or can be installed with homebrew.

  • Windows: Make sure you have Visual Studio 2015 with C++ support installed. Next, download and run the rustup installer from https://static.rust-lang.org/rustup/dist/x86_64-pc-windows-msvc/rustup-init.exe, start "VS2015 x64 Native Tools Command Prompt", and use the following command to install and set up the msvc toolchain:

    $ rustup default stable-x86_64-pc-windows-msvc
    

Once you have rustup installed, then you need to install:

Make sure that these binaries are in your PATH. After that, you should be able to build Parity Ethereum from source.

3.2 Build from Source Code

# download Parity Ethereum code
$ git clone https://github.com/paritytech/parity-ethereum
$ cd parity-ethereum

# build in release mode
$ cargo build --release --features final

This produces an executable in the ./target/release subdirectory.

Note: if cargo fails to parse manifest try:

$ ~/.cargo/bin/cargo build --release

Note, when compiling a crate and you receive errors, it's in most cases your outdated version of Rust, or some of your crates have to be recompiled. Cleaning the repository will most likely solve the issue if you are on the latest stable version of Rust, try:

$ cargo clean

This always compiles the latest nightly builds. If you want to build stable or beta, do a

$ git checkout stable

or

$ git checkout beta

3.3 Simple One-Line Installer for Mac and Linux

bash <(curl https://get.parity.io -L)

The one-line installer always defaults to the latest beta release. To install a stable release, run:

bash <(curl https://get.parity.io -L) -r stable

3.4 Starting Parity Ethereum

Manually

To start Parity Ethereum manually, just run

$ ./target/release/parity

so Parity Ethereum begins syncing the Ethereum blockchain.

Using systemd service file

To start Parity Ethereum as a regular user using systemd init:

  1. Copy ./scripts/parity.service to your systemd user directory (usually ~/.config/systemd/user).
  2. Copy release to bin folder, write sudo install ./target/release/parity /usr/bin/parity
  3. To configure Parity Ethereum, write a /etc/parity/config.toml config file, see Configuring Parity Ethereum for details.

4. Testing

Download the required test files: git submodule update --init --recursive. You can run tests with the following commands:

  • All packages

    cargo test --all
    
  • Specific package

    cargo test --package <spec>
    

Replace <spec> with one of the packages from the package list (e.g. cargo test --package evmbin).

You can show your logs in the test output by passing --nocapture (i.e. cargo test --package evmbin -- --nocapture)

5. Documentation

Official website: https://parity.io

Be sure to check out our wiki for more information.

Viewing documentation for Parity Ethereum packages

You can generate documentation for Parity Ethereum Rust packages that automatically opens in your web browser using rustdoc with Cargo (of the The Rustdoc Book), by running the the following commands:

  • All packages

    cargo doc --document-private-items --open
    
  • Specific package

    cargo doc --package <spec> -- --document-private-items --open
    

Use--document-private-items to also view private documentation and --no-deps to exclude building documentation for dependencies.

Replacing <spec> with one of the following from the details section below (i.e. cargo doc --package parity-ethereum --open):

Package List

  • Parity Ethereum (EthCore) Client Application
    parity-ethereum
    
  • Parity Ethereum Account Management, Key Management Tool, and Keys Generator
    ethcore-accounts, ethkey-cli, ethstore, ethstore-cli
    
  • Parity Chain Specification
    chainspec
    
  • Parity CLI Signer Tool & RPC Client
    cli-signer parity-rpc-client
    
  • Parity Ethereum Ethash & ProgPoW Implementations
    ethash
    
  • Parity (EthCore) Library
    ethcore
    
    • Parity Ethereum Blockchain Database, Test Generator, Configuration, Caching, Importing Blocks, and Block Information
      ethcore-blockchain
      
    • Parity Ethereum (EthCore) Contract Calls and Blockchain Service & Registry Information
      ethcore-call-contract
      
    • Parity Ethereum (EthCore) Database Access & Utilities, Database Cache Manager
      ethcore-db
      
    • Parity Ethereum Virtual Machine (EVM) Rust Implementation
      evm
      
    • Parity Ethereum (EthCore) Light Client Implementation
      ethcore-light
      
    • Parity Smart Contract based Node Filter, Manage Permissions of Network Connections
      node-filter
      
    • Parity Private Transactions
      ethcore-private-tx
      
    • Parity Ethereum (EthCore) Client & Network Service Creation & Registration with the I/O Subsystem
      ethcore-service
      
    • Parity Ethereum (EthCore) Blockchain Synchronization
      ethcore-sync
      
    • Parity Ethereum Common Types
      common-types
      
    • Parity Ethereum Virtual Machines (VM) Support Library
      vm
      
    • Parity Ethereum WASM Interpreter
      wasm
      
    • Parity Ethereum WASM Test Runner
      pwasm-run-test
      
    • Parity EVM Implementation
      evmbin
      
    • Parity Ethereum IPFS-compatible API
      parity-ipfs-api
      
    • Parity Ethereum JSON Deserialization
      ethjson
      
    • Parity Ethereum State Machine Generalization for Consensus Engines
      parity-machine
      
  • Parity Ethereum (EthCore) Miner Interface
    ethcore-miner parity-local-store price-info ethcore-stratum using_queue
    
  • Parity Ethereum (EthCore) Logger Implementation
    ethcore-logger
    
  • C bindings library for the Parity Ethereum client
    parity-clib
    
  • Parity Ethereum JSON-RPC Servers
    parity-rpc
    
  • Parity Ethereum (EthCore) Secret Store
    ethcore-secretstore
    
  • Parity Updater Service
    parity-updater parity-hash-fetch
    
  • Parity Core Libraries (Parity Util)
    ethcore-bloom-journal blooms-db dir eip-712 fake-fetch fastmap fetch ethcore-io
    journaldb keccak-hasher len-caching-lock macros memory-cache memzero
    migration-rocksdb ethcore-network ethcore-network-devp2p panic_hook
    patricia-trie-ethereum registrar rlp_compress rlp_derive parity-runtime stats
    time-utils triehash-ethereum unexpected parity-version
    

Contributing to documentation for Parity Ethereum packages

Document source code for Parity Ethereum packages by annotating the source code with documentation comments.

Example (generic documentation comment):

/// Summary
///
/// Description
///
/// # Panics
///
/// # Errors
///
/// # Safety
///
/// # Examples
///
/// Summary of Example 1
///
/// ```rust
/// // insert example 1 code here for use with documentation as tests
/// ```
///

6. Toolchain

In addition to the Parity Ethereum client, there are additional tools in this repository available:

  • evmbin - Parity Ethereum EVM Implementation.
  • ethstore - Parity Ethereum Key Management.
  • ethkey - Parity Ethereum Keys Generator.

The following tool is available in a separate repository:

  • ethabi - Parity Ethereum Encoding of Function Calls. Docs here
  • whisper - Parity Ethereum Whisper-v2 PoC Implementation.

7. Community

Join the chat!

Questions? Get in touch with us on Gitter: Gitter: Parity Gitter: Parity.js Gitter: Parity/Miners Gitter: Parity-PoA

Alternatively, join our community on Matrix: Riot: +Parity

8. Contributing

An introduction has been provided in the "So You Want to be a Core Developer" presentation slides by Hernando Castano. Additional guidelines are provided in CONTRIBUTING.

Contributor Code of Conduct

CODE_OF_CONDUCT

9. License

LICENSE