* ethcore: add clique engine (#9981)
* fix broken sync
* correct seal fields
* ethcore: fix comment
* parity: remove duplicate params
* clique: fix whitespaces
* ethcore: fix goerli chain spec
* refactor signer_snapshot into pending/finalized state
* move close_block_extra_data after seal is applied
* refactor most of the logic into the signer_snapshot
* clique: refactor locking logic out of the consensus engine interface
* Fix jsonspec and add an unittest
* Replace space with tabs
* Unbroke sync
* Fix broken sync
* 1/2 state tracking without votes
* 2/2 implement vote tracking
* ci: use travis for goerli
* ci: setup a clique network
* ci: sync a görli node
* add clique deploy script
* ci: fix paths in clique deploy script
* ci: use docker compose
* ci: fix travis job names
* ci: fix build deps
* ci: massively reduce tests
* Revert "ci: massively reduce tests"
This reverts commit 6369f0b069ed2607a7e9f2e1d85489bacdc43384.
* ci: run cargo test directly
* ci: separate build and test stages
* ci: cache rust installation
* ci: simplify ci stages
* ci: make clique deploy script executable
* ci: shutdown goerli sync after 20min
* ci: remove slow sync stage
* ci: use timeout to finish jobs
* ci: fix build path
* ci: use absolute paths to end this confusion
* ci: add geth and parity to path
* ci: be more verbose
* ci: allow for more relaxed caching timeout
* ci: update repositories for custom ppa
* ci: fix typo in file name
* ci: fix docker compose file
* ci: add ethkey to docker
* ci: make sure deploy script is up to date with upstream
* ci: stop docker container after certain time
* ci: force superuser to update permissions on docker files
* ci: reduce run time of script to ~30 min
* ci: remove duplicate caching in travis
* remove trace statements
* clique: add more validation involving the recent signer list
* ethcore: enable constantinople for rinkeby
* ethcore: fix whitespaces in rinkeby spec
* ethcore: reformat goerli.json
* Revert "ci: remove duplicate caching in travis"
This reverts commit a562838d3d194d37f9871dcbe00b783637978f89.
* tmp commit
* another tmp commit
* it builds!
* add sealing capabilities
* add seal_header hook to allow separation of block seal/importing code paths
* clique: remove populate_from_parent.
* add panic
* make turn delay random
* initialize OpenBlock properly in 'enact'
* misc: remove duplicate lines
* misc: fix license headers
* misc: convert spaces to tabs
* misc: fix tabs
* Update Cargo.toml
* Update Cargo.toml
* Update Cargo.toml
* clique: ensure validator restores state before trying to seal
* clique: make 'state' return an Error. Make some error messages more clear
* Fix compile error after rebase & toolchain upgrade
* fix a bunch of import warnings
* Refactor code
* Fix permissions
* Refactoring syncing
* Implement full validator checks
* Refactor util functions to seperate file
* mining 1
* ethcore: add chainspec for kotti
* ethcore: rename pre-goerli configs
* ethcore: load kotti chain spec
* cli: add kotti to params
* Implement working local sealing
* making sealing & syncing work together
* Relax timestamp checking
* ethcore: prepare for the real goerli to launch
* Implement NOTURN wiggle properly & cleanupnup warnings
* Implement vote casting
* Update docs & skip signing if no signer
* Optimize step-service interval
* Record state on local sealed block
* Fix script filemode
* Cleaning up codebase
* restore enact trace logging
* Delete clique.sh and move sync.sh
* remove travis.yml
* Remove dead code
* Cleanup compile warning
* address review comments
* adding more comments and removing unwrap()
* ci: remove sync script
* Address review comments
* fix compile error
* adding better debugging for timing
* Implement an dedicated thread for sealing timing
* fix(add helper for timestamp overflows) (#10330)
* fix(add helper timestamp overflows)
* fix(simplify code)
* fix(make helper private)
* snap: official image / test (#10168)
* official image / test
* fix / test
* bit more necromancy
* fix paths
* add source bin/df /test
* add source bin/df /test2
* something w paths /test
* something w paths /test
* add source-type /test
* show paths /test
* copy plugin /test
* plugin -> nil
* install rhash
* no questions while installing rhash
* publish snap only for release
* fix(docker): fix not receives SIGINT (#10059)
* fix(docker): fix not receives SIGINT
* fix: update with reviews
* update with review
* update
* update
* Don't add discovery initiators to the node table (#10305)
* Don't add discovery initiators to the node table
* Use enums for tracking state of the nodes in discovery
* Dont try to ping ourselves
* Fix minor nits
* Update timeouts when observing an outdated node
* Extracted update_bucket_record from update_node
* Fixed typo
* Fix two final nits from @todr
* change docker image based on debian instead of ubuntu due to the chan… (#10336)
* change docker image based on debian instead of ubuntu due to the changes of the build container
* role back docker build image and docker deploy image to ubuntu:xenial based (#10338)
* Bundle protocol and packet_id together in chain sync (#10315)
Define a new `enum` where devp2p subprotocol packet ids (currently eth and par) are defined. Additionally provide functionality to query id value and protocol of a given id object.
* snap: prefix version and populate candidate channel (#10343)
* snap: populate candidate releases with beta snaps to avoid stale channel
* snap: prefix version with v*
* addressing review comments
* engine: fix copyright header
* scripts: restore permissions on sign command
* ethcore: enforce tabs
* ethcore: enforce tabs
* ethcore: enforce tabs
* addressing comments
* addressing comments
* addressing more comments
* addressing more comments
* addressing more comments
* addressing more comments
* addressing more comments
* json-spec: fix clique epoch to non-zero u64
* ci: enable travis for parity goerli
* ci: don't separate build and test step
* ci: don't run c++ tests on travis
* ci: simplify cargo test to squeeze into travis timeout
* ci: don't run tests on travis at all
* style(fixes)
* fix(add tests)
* fix(recent_signer bug)
* fix(complete all tests)
* fix(nits)
* fix(simplify asserts)
* fix(cliqueState): simplify code
* fix(nits)
* docs(comments what's need to fixed)
* fix(revert unintended changes)
* fix(tests)
* fix(logs): voting logs
* fix(readability + more logs)
* fix(sync)
* docs(add missing licens header)
* fix(log): info! -> trace!
* docs(fix nits) + fix(remove assert)
* perf(use counter instead of vec)
* fix(remove needless block in match)
* fix(faulty comment)
* grumbles(docs for tests)
* fix(nits)
* fix(revert_vote): only remove vote when votes == 0
* fix(vote counter): checked arithmetics
* fix(simplify tests)
* fix(nits)
* fix(clique): err types
* fix(clique utils): make use of errors
* fix(cleanup nits)
* fix(clique sealing): don't read state no signer
* fix(replace Vec<Signers> with BTreeSet<Signers>)
* fix(tests): BTreeSet and more generic helpers
* fix(nits)
* fix(ethcore_block_seal): remove needless `Box`
* fix(faulty log): info -> trace
* fix(checked SystemTime): prevent SystemTime panics
* style(chain cfg): space after `:`
* style(fn enact): fix whitespace
* docs(clique): StepService
* docs(nit): fix faulty comment
* docs(fix typo)
* style(fix bad indentation)
* fix(bad regex match)
* grumble(on_seal_block): make `&mut` to avoid clone
* docs(on_seal_block): fix faulty documentation
* Delete .travis.yml
* docs: remove eth hf references in spec
* Update client.rs
* fix(nits)
* fix(clique step): `RwLock` -> `AtomicBool`
* fix(clique): use `Duration::as_millis`
* Clean up some Clique documentation
* Add trace information to eth_estimateGas (#10519)
* Add trace information to eth_estimateGas
* replace unwrap better version
* change vm::Error formatter to more user-friendly
* remove extra error format
* use map_or instead sequence of map/unwrap_or
* fix(light/on_demand): extract as a trait
* fix(grumble): OnDemand remove needless trait bound
If system uptime is less than the duration in the test, thread
will panic due to: 'overflow when subtracting duration from instant'.
Changed duration to 20 seconds to make it unlikely the test will
fail due to this condition.
Since no operations that carry a similar risk of panic occur outside
of the tests, it doesn't seem warranted to depend on an external crate
to fix this.
* PIP Table Cost relative to average peers instead of max peers
* Add tracing in PIP new_cost_table
* Update stat peer_count
* Use number of leeching peers for Light serve costs
* Fix test::light_params_load_share_depends_on_max_peers (wrong type)
* Remove (now) useless test
* Remove `load_share` from LightParams.Config
Prevent div. by 0
* Add LEECHER_COUNT_FACTOR
* PR Grumble: u64 to u32 for f64 casting
* Prevent u32 overflow for avg_peer_count
* Add tests for LightSync::Statistics
* Rename db_restore => client
* First step: make it compile!
* Second step: working implementation!
* Refactoring
* Fix tests
* PR Grumbles
* PR Grumbles WIP
* Migrate ancient blocks interating backward
* Early return in block migration if snapshot is aborted
* Remove RwLock getter (PR Grumble I)
* Remove dependency on `Client`: only used Traits
* Add test for recovering aborted snapshot recovery
* Add test for migrating old blocks
* Fix build
* PR Grumble I
* PR Grumble II
* PR Grumble III
* PR Grumble IV
* PR Grumble V
* PR Grumble VI
* Fix one test
* Fix test
* PR Grumble
* PR Grumbles
* PR Grumbles II
* Fix tests
* Release RwLock earlier
* Revert Cargo.lock
* Update _update ancient block_ logic: set local in `commit`
* Update typo in ethcore/src/snapshot/service.rs
Co-Authored-By: ngotchac <ngotchac@gmail.com>
* Fixing and disabling some tests for windows 10 compatibility.
* Few adjustment for windows in tests (eg bigger timeout for keyserver tests)
* Spaces and temporary single thread ci (to be able to spot the error).
* aura: emit ancestry actions for finalizing blocks
* aura: refactor is_epoch_end to get finalized blocks as argument
* ethcore: add is_epoch_end_light method to Engine
The full client now tracks finality by querying the engine on each block import,
and it also persists the finalization state to the DB. For the light client
current it doesn't persist finality information and only keeps track of finality
for epoch signals, by calling `is_epoch_end_light`. This method implements the
previously existing logic of building finality for all the blocks in the current
epoch and then checking the finalized blocks against the transition store.
* ethcore: allow finalizing current block
* aura: fix construction of finality proof
* aura: fix warnings
- missing docs for is_epoch_end_light
- unused method unfinalized_hashes in RollingFinality
* aura: fix clone on copy types
* OnDemand no longer loop until there is a query.
All peer known at the time will be queried, and the query fail if all
return no reply.
Returning the failure is done through an empty Vec of reply (the type
of the oneshot channel remains unchanged).
Before this commit the query were send randomly to any peer until there
is a reply (for a query that got no result it was an issue, for other
queries it was quering multiple times the same peers).
After this commit the first query is random but next queries
follows hashmap iterator order.
Test no_capability was broken by this commit (the pending query was
removed).
* OnDemand no longer loop until there is a query.
All peer known at the time will be queried, and the query fail if all
return no reply.
Returning the failure is done through an empty Vec of reply (the type
of the oneshot channel remains unchanged).
Before this commit the query were send randomly to any peer until there
is a reply (for a query that got no result it was an issue, for other
queries it was quering multiple times the same peers).
After this commit the first query is random but next queries
follows hashmap iterator order.
Test no_capability was broken by this commit (the pending query was
removed). If adding some kind of timeout mechanism it could be restored.
* Comment plus better field names.
* No panick on dropped oneshot channel.
* Use Set to avoid counter heuristic
* Cli option `on_demand_nb_retry` for maximum number of retry when doing
on demand query in light client.
* Missing test update for previous commit
* Add a timeout (only when there is no peer to query), that way we do not
set number of query to minimum current number peer or configured number
of query : that way capability test was restored.
* Adding an error type for on_demand, it helps having variant of error
reported at rpc level : choice of rpc error code error might not be
right.
* Duration as constant is nice
* Switch to duration in main too
* Fix indentation (sorry for that).
* Fix error management (bad merge in previous commit)
* Lots of english corrections, major change on the new command parameters :
- use standard '-' instead of '_'
- renaming nb_retry params to 'on-demand-retry-count'
* Add EIP-1014 transition config flag
* Remove EIP-86 configs
* Change CREATE2 opcode index to 0xf5
* Move salt to the last item in the stack
* Change sendersaltandaddress scheme to comply with current EIP-1014
* Fix json configs
* Fix create2 test
* Fix deprecated comments
* Light client on-demand request for headers range.
* Cache headers in HeaderWithAncestors response.
Also fulfills request locally if all headers are in cache.
* LightFetch::logs fetches missing headers on demand.
* LightFetch::logs limit the number of headers requested at a time.
* LightFetch::logs refactor header fetching logic.
* Enforce limit on header range length in light client logs request.
* Fix light request tests after struct change.
* Respond to review comments.
* Add a `fastmap` crate that provides the H256FastMap specialized HashMap
* Use `fastmap` instead of `plain_hasher`
* Update submodules for Reasons™
* Submodule update
Closes#9255
This PR also removes the limit of max 64 transactions per packet, currently we only attempt to prevent the packet size to go over 8MB. This will only be the case for super-large transactions or high-block-gas-limit chains.
Patching this is important only for chains that have blocks that can fit more than 4k transactions (over 86M block gas limit)
For mainnet, we should actually see a tiny bit faster propagation since instead of computing 4k pending set, we only need `4 * 8M / 21k = 1523` transactions.
Running some tests on `dekompile` node right now, to check how it performs in the wild.
The `patricia_trie` crate is generic over the hasher (by way of HashDB) and node encoding scheme. Adds a new `patricia_trie_ethereum` crate with concrete impls for Keccak/RLP.
* new blooms database
* fixed conflict in Cargo.lock
* removed bloomchain
* cleanup in progress
* all tests passing in trace db with new blooms-db
* added trace_blooms to BlockChainDB interface, fixed db flushing
* BlockChainDB no longer exposes RwLock in the interface
* automatically flush blooms-db after every insert
* blooms-db uses io::BufReader to read files, wrap blooms-db into Mutex, cause fs::File is just a shared file handle
* fix json_tests
* blooms-db can filter multiple possibilities at the same time
* removed enum trace/db.rs CacheId
* lint fixes
* fixed tests
* kvdb-rocksdb uses fs-swap crate
* update Cargo.lock
* use fs::rename
* fixed failing test on linux
* fix tests
* use fs_swap
* fixed failing test on linux
* cleanup after swap
* fix tests
* fixed osx permissions
* simplify parity database opening functions
* added migration to blooms-db
* address @niklasad1 grumbles
* fix license and authors field of blooms-db Cargo.toml
* restore blooms-db after snapshot
* Unordered iterator.
* Use unordered and limited set if full not required.
* Split timeout work into smaller timers.
* Avoid collecting all pending transactions when mining
* Remove println.
* Use priority ordering in eth-filter.
* Fix ethcore-miner tests and tx propagation.
* Review grumbles addressed.
* Add test for unordered not populating the cache.
* Fix ethcore tests.
* Fix light tests.
* Fix ethcore-sync tests.
* Fix RPC tests.
* Update `add_license` script
* run script
* add `remove duplicate lines script` and run it
* Revert changes `English spaces`
* strip whitespaces
* Revert `GPL` in files with `apache/mit license`
* don't append `gpl license` in files with other lic
* Don't append `gpl header` in files with other lic.
* re-ran script
* include c and cpp files too
* remove duplicate header
* rebase nit
* Add light client TODO item
* Move existing total-difficulty-based fork choice check to Engine
* Abstract total difficulty and block provider as Machine::BlockMetadata and Machine::BlockProvider
* Decouple "generate_metadata" logic to Engine
* Use fixed BlockMetadata and BlockProvider type for null and instantseal
In this way they can use total difficulty fork choice check
* Extend blockdetails with metadatas and finalized info
* Extra data update: mark_finalized and update_metadatas
* Check finalized block in Blockchain
* Fix a test constructor in verification mod
* Add total difficulty trait
* Fix type import
* Db migration to V13 with metadata column
* Address grumbles
* metadatas -> metadata
* Use generic type for update_metadata to avoid passing HashMap all around
* Remove metadata in blockdetails
* [WIP] Implement a generic metadata architecture
* [WIP] Metadata insertion logic in BlockChain
* typo: Value -> Self::Value
* [WIP] Temporarily remove Engine::is_new_best interface
So that we don't have too many type errors.
* [WIP] Fix more type errors
* [WIP] ExtendedHeader::PartialEq
* [WIP] Change metadata type Option<Vec<u8>> to Vec<u8>
* [WIP] Remove Metadata Error
* [WIP] Clean up error conversion
* [WIP] finalized -> is_finalized
* [WIP] Mark all fields in ExtrasInsert as pub
* [WIP] Remove unused import
* [WIP] Keep only local metadata info
* Mark metadata as optional
* [WIP] Revert metadata db change in BlockChain
* [WIP] Put finalization in unclosed state
* Use metadata interface in BlockDetail
* [WIP] Fix current build failures
* [WIP] Remove unused blockmetadata struct
* Remove DB migration info
* [WIP] Typo
* Use ExtendedHeader to implement fork choice check
* Implement is_new_best using Ancestry iterator
* Use expect instead of panic
* [WIP] Add ancestry Engine support via on_new_block
* Fix tests
* Emission of ancestry actions
* use_short_version should take account of metadata
* Engine::is_new_best -> Engine::fork_choice
* Use proper expect format as defined in #1026
* panic -> expect
* ancestry_header -> ancestry_with_metadata
* Boxed iterator -> &mut iterator
* Fix tests
* is_new_best -> primitive_fork_choice
* Document how fork_choice works
* Engine::fork_choice -> Engine::primitive_fork_choice
* comment: clarify types of finalization where Engine::primitive_fork_choice works
* Expose FinalizationInfo to Engine
* Fix tests due to merging
* Remove TotalDifficulty trait
* Do not pass FinalizationInfo to Engine
If there's finalized blocks in from route, choose the old branch without calling `Engine::fork_choice`.
* Fix compile
* Fix unused import
* Remove is_to_route_finalized
When no block reorg passes a finalized block, this variable is always false.
* Address format grumbles
* Fix docs: mark_finalized returns None if block hash is not found
`blockchain` mod does not yet have an Error type, so we still temporarily use None here.
* Fix inaccurate tree_route None expect description
* rlp::decode returns Result
* Fix journaldb to handle rlp::decode Result
* Fix ethcore to work with rlp::decode returning Result
* Light client handles rlp::decode returning Result
* Fix tests in rlp_derive
* Fix tests
* Cleanup
* cleanup
* Allow panic rather than breaking out of iterator
* Let decoding failures when reading from disk blow up
* syntax
* Fix the trivial grumbles
* Fix failing tests
* Make Account::from_rlp return Result
* Syntx, sigh
* Temp-fix for decoding failures
* Header::decode returns Result
Handle new return type throughout the code base.
* Do not continue reading from the DB when a value could not be read
* Fix tests
* Handle header decoding in light_sync
* Handling header decoding errors
* Let the DecodeError bubble up unchanged
* Remove redundant error conversion
rlp::decode returns Result
Make a best effort to handle decoding errors gracefully throughout the code, using `expect` where the value is guaranteed to be valid (and in other places where it makes sense).
* WIP
* Convert Ethcore error to use error_chain
* Use error_chain for ImportError and BlockImportError
* Fix error pattern matches for error_chain in miner
* Implement explicit From for AccountsError
* Fix pattern matches for ErrorKinds
* Handle ethcore error_chain in light client
* Explicitly define Result type to avoid shadowing
* Fix remaining Error pattern matches
* Fix tab space formatting
* Helps if the tests compile
* Fix error chain matching after merge
* WIP
* Replace Rlp with UntrustedRlp in views, explicity unwrap with expect
First pass to get it to compile. Need to figure out whether to do this or to propogate Errors upstream, which would require many more changes to dependent code. If we do this way we are assuming that the views are always used in a context where the rlp is trusted to be valid e.g. when reading from our own DB. So need to fid out whether views are used with data received from an untrusted (e.g. extrernal peer).
* Remove original Rlp impl, rename UntrustedRlp -> Rlp
* Create rlp views with view! macro to record debug info
Views are assumed to be over valid rlp, so if there is a decoding error we record where the view was created in the first place and report it in the expect
* Use $crate in view! macro to avoid import, fix tests
* Expect valid rlp in decode functions for now
* Replace spaces with tabs in new file
* Add doc tests for creating views with macro
* Update rlp docs to reflect removing of UntrustedRlp
* Replace UntrustedRlp usages in private-tx merge
