* Move snapshot to own crate
Sort out imports
* WIP cargo toml
* Make snapshotting generic over the client
Sort out tests
* Sort out types from blockchain and client
* Sort out sync
* Sort out imports and generics
* Sort out main binary
* Fix sync test-helpers
* Sort out import for secret-store
* Sort out more imports
* Fix easy todos
* cleanup
* Cleanup
* remove unneded workspace member
* cleanup
* Sort out test-helpers dependency on account-db
* Update ethcore/client-traits/src/lib.rs
Co-Authored-By: Niklas Adolfsson <niklasadolfsson1@gmail.com>
* Update ethcore/snapshot/Cargo.toml
* Add client-traits crate
Move the BlockInfo trait to new crate
* New crate `machine`
Contains code extracted from ethcore that defines `Machine`, `Externalities` and other execution related code.
* Use new machine and client-traits crates in ethcore
* Use new crates machine and client-traits instead of ethcore where appropriate
* Fix tests
* Don't re-export so many types from ethcore::client
* Fixing more fallout from removing re-export
* fix test
* More fallout from not re-exporting types
* Add some docs
* cleanup
* import the macro edition style
* Tweak docs
* Add missing import
* remove unused ethabi_derive imports
* Use latest ethabi-contract
* Move many traits from ethcore/client/traits to client-traits crate
Initial version of extracted Engine trait
* Move snapshot related traits to the engine crate (eew)
* Move a few snapshot related types to common_types
Cleanup Executed as exported from machine crate
* fix warning
* Gradually introduce new engine crate: snapshot
* ethcore typechecks with new engine crate
* Sort out types outside ethcore
* Add an EpochVerifier to ethash and use that in Engine.epoch_verifier()
Cleanup
* Document pub members
* Sort out tests
Sort out default impls for EpochVerifier
* Add test-helpers feature and move EngineSigner impl to the right place
* Sort out tests
* Sort out tests and refactor verification types
* Fix missing traits
* More missing traits
Fix Histogram
* Fix tests and cleanup
* cleanup
* Put back needed logger import
* Don't rexport common_types from ethcore/src/client
Don't export ethcore::client::*
* Remove files no longer used
Use types from the engine crate
Explicit exports from engine::engine
* Get rid of itertools
* Move a few more traits from ethcore to client-traits: BlockChainReset, ScheduleInfo, StateClient
* Move ProvingBlockChainClient to client-traits
* Don't re-export ForkChoice and Transition from ethcore
* Address grumbles: sort imports, remove commented out code
* Fix merge resolution error
* Extract the Clique engine to own crate
* Extract NullEngine and the block_reward module from ethcore
* Extract InstantSeal engine to own crate
* Extract remaining engines
* Extract executive_state to own crate so it can be used by engine crates
* Remove snapshot stuff from the engine crate
* Put snapshot traits back in ethcore
* cleanup
* Remove stuff from ethcore
* Don't use itertools
* itertools in aura is legit-ish
* More post-merge fixes
* Re-export less types in client
* cleanup
* Extract spec to own crate
* Put back the test-helpers from basic-authority
* Fix ethcore benchmarks
* Reduce the public api of ethcore/verification
* WIP
* Add Cargo.toml
* Fix compilation outside ethcore
* Audit uses of import_verified_blocks() and remove unneeded calls
Cleanup
* cleanup
* Remove unused imports from ethcore
* Cleanup
* remove double semi-colons
* Add missing generic param
* More missing generics
* Update ethcore/block-reward/Cargo.toml
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update ethcore/engines/basic-authority/Cargo.toml
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update ethcore/engines/ethash/Cargo.toml
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update ethcore/engines/clique/src/lib.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* signers is already a ref
* Add an EngineType enum to tighten up Engine.name()
* Add CHAINID opcode
* Introduce Snapshotting enum to distinguish the type of snapshots a chain uses
* Rename supports_warp to snapshot_mode
* Missing import
* Add chain_id wherever we instantiate EnvInfo
* more missing chain_id
* Tell serde to ignore the chain_id field on Env
* Update ethcore/src/snapshot/consensus/mod.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Use the chain_id from the machine by adding chain_id() to the Ext trait
* cleanup
* add missing impl
cleanup
* missing import
* Fix import
* Add transition marker for EIP 1344
* double semi
* Fix merge problem
* cleanup
* merge conflict error
* Fix a few warnings
* Update ethcore/vm/src/schedule.rs
Co-Authored-By: Andronik Ordian <write@reusable.software>
* more merge fallout
* Add client-traits crate
Move the BlockInfo trait to new crate
* New crate `machine`
Contains code extracted from ethcore that defines `Machine`, `Externalities` and other execution related code.
* Use new machine and client-traits crates in ethcore
* Use new crates machine and client-traits instead of ethcore where appropriate
* Fix tests
* Don't re-export so many types from ethcore::client
* Fixing more fallout from removing re-export
* fix test
* More fallout from not re-exporting types
* Add some docs
* cleanup
* import the macro edition style
* Tweak docs
* Add missing import
* remove unused ethabi_derive imports
* Use latest ethabi-contract
* Move many traits from ethcore/client/traits to client-traits crate
Initial version of extracted Engine trait
* Move snapshot related traits to the engine crate (eew)
* Move a few snapshot related types to common_types
Cleanup Executed as exported from machine crate
* fix warning
* Gradually introduce new engine crate: snapshot
* ethcore typechecks with new engine crate
* Sort out types outside ethcore
* Add an EpochVerifier to ethash and use that in Engine.epoch_verifier()
Cleanup
* Document pub members
* Sort out tests
Sort out default impls for EpochVerifier
* Add test-helpers feature and move EngineSigner impl to the right place
* Sort out tests
* Sort out tests and refactor verification types
* Fix missing traits
* More missing traits
Fix Histogram
* Fix tests and cleanup
* cleanup
* Put back needed logger import
* Don't rexport common_types from ethcore/src/client
Don't export ethcore::client::*
* Remove files no longer used
Use types from the engine crate
Explicit exports from engine::engine
* Get rid of itertools
* Move a few more traits from ethcore to client-traits: BlockChainReset, ScheduleInfo, StateClient
* Move ProvingBlockChainClient to client-traits
* Don't re-export ForkChoice and Transition from ethcore
* Address grumbles: sort imports, remove commented out code
* Fix merge resolution error
* Extract the Clique engine to own crate
* Extract NullEngine and the block_reward module from ethcore
* Extract InstantSeal engine to own crate
* Extract remaining engines
* Extract executive_state to own crate so it can be used by engine crates
* Remove snapshot stuff from the engine crate
* Put snapshot traits back in ethcore
* cleanup
* Remove stuff from ethcore
* Don't use itertools
* itertools in aura is legit-ish
* More post-merge fixes
* Re-export less types in client
* cleanup
* Extract spec to own crate
* Put back the test-helpers from basic-authority
* Fix ethcore benchmarks
* Reduce the public api of ethcore/verification
* WIP
* Add Cargo.toml
* Fix compilation outside ethcore
* Audit uses of import_verified_blocks() and remove unneeded calls
Cleanup
* cleanup
* Remove unused imports from ethcore
* Cleanup
* remove double semi-colons
* Add missing generic param
* More missing generics
* Update ethcore/block-reward/Cargo.toml
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update ethcore/engines/basic-authority/Cargo.toml
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update ethcore/engines/ethash/Cargo.toml
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update ethcore/engines/clique/src/lib.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* signers is already a ref
* Add an EngineType enum to tighten up Engine.name()
* Introduce Snapshotting enum to distinguish the type of snapshots a chain uses
* Rename supports_warp to snapshot_mode
* Missing import
* Update ethcore/src/snapshot/consensus/mod.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* missing import
* Fix import
* double semi
* Fix merge problem
* cleanup
* Parametrise `ClientIoMessage` with `()` for the light client
* Add impl Tick for ()
* Address review feedback
* Move ClientIoMessage to common-types
* remove superseeded fixme
* fix merge conflict errors
* Add client-traits crate
Move the BlockInfo trait to new crate
* New crate `machine`
Contains code extracted from ethcore that defines `Machine`, `Externalities` and other execution related code.
* Use new machine and client-traits crates in ethcore
* Use new crates machine and client-traits instead of ethcore where appropriate
* Fix tests
* Don't re-export so many types from ethcore::client
* Fixing more fallout from removing re-export
* fix test
* More fallout from not re-exporting types
* Add some docs
* cleanup
* import the macro edition style
* Tweak docs
* Add missing import
* remove unused ethabi_derive imports
* Use latest ethabi-contract
* Move many traits from ethcore/client/traits to client-traits crate
Initial version of extracted Engine trait
* Move snapshot related traits to the engine crate (eew)
* Move a few snapshot related types to common_types
Cleanup Executed as exported from machine crate
* fix warning
* Gradually introduce new engine crate: snapshot
* ethcore typechecks with new engine crate
* Sort out types outside ethcore
* Add an EpochVerifier to ethash and use that in Engine.epoch_verifier()
Cleanup
* Document pub members
* Sort out tests
Sort out default impls for EpochVerifier
* Add test-helpers feature and move EngineSigner impl to the right place
* Sort out tests
* Sort out tests and refactor verification types
* Fix missing traits
* More missing traits
Fix Histogram
* Fix tests and cleanup
* cleanup
* Put back needed logger import
* Don't rexport common_types from ethcore/src/client
Don't export ethcore::client::*
* Remove files no longer used
Use types from the engine crate
Explicit exports from engine::engine
* Get rid of itertools
* Move a few more traits from ethcore to client-traits: BlockChainReset, ScheduleInfo, StateClient
* Move ProvingBlockChainClient to client-traits
* Don't re-export ForkChoice and Transition from ethcore
* Address grumbles: sort imports, remove commented out code
* Fix merge resolution error
* Extract the Clique engine to own crate
* Extract NullEngine and the block_reward module from ethcore
* Extract InstantSeal engine to own crate
* Extract remaining engines
* Extract executive_state to own crate so it can be used by engine crates
* Remove snapshot stuff from the engine crate
* Put snapshot traits back in ethcore
* cleanup
* Remove stuff from ethcore
* Don't use itertools
* itertools in aura is legit-ish
* More post-merge fixes
* Re-export less types in client
* cleanup
* Update ethcore/block-reward/Cargo.toml
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update ethcore/engines/basic-authority/Cargo.toml
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update ethcore/engines/ethash/Cargo.toml
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update ethcore/engines/clique/src/lib.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* signers is already a ref
* Add an EngineType enum to tighten up Engine.name()
* Introduce Snapshotting enum to distinguish the type of snapshots a chain uses
* Rename supports_warp to snapshot_mode
* Missing import
* Update ethcore/src/snapshot/consensus/mod.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* remove double-semicolons
* Add client-traits crate
Move the BlockInfo trait to new crate
* New crate `machine`
Contains code extracted from ethcore that defines `Machine`, `Externalities` and other execution related code.
* Use new machine and client-traits crates in ethcore
* Use new crates machine and client-traits instead of ethcore where appropriate
* Fix tests
* Don't re-export so many types from ethcore::client
* Fixing more fallout from removing re-export
* fix test
* More fallout from not re-exporting types
* Add some docs
* cleanup
* import the macro edition style
* Tweak docs
* Add missing import
* remove unused ethabi_derive imports
* Use latest ethabi-contract
* Move many traits from ethcore/client/traits to client-traits crate
Initial version of extracted Engine trait
* Move snapshot related traits to the engine crate (eew)
* Move a few snapshot related types to common_types
Cleanup Executed as exported from machine crate
* fix warning
* Gradually introduce new engine crate: snapshot
* ethcore typechecks with new engine crate
* Sort out types outside ethcore
* Add an EpochVerifier to ethash and use that in Engine.epoch_verifier()
Cleanup
* Document pub members
* Sort out tests
Sort out default impls for EpochVerifier
* Add test-helpers feature and move EngineSigner impl to the right place
* Sort out tests
* Sort out tests and refactor verification types
* Fix missing traits
* More missing traits
Fix Histogram
* Fix tests and cleanup
* cleanup
* Put back needed logger import
* Don't rexport common_types from ethcore/src/client
Don't export ethcore::client::*
* Remove files no longer used
Use types from the engine crate
Explicit exports from engine::engine
* Get rid of itertools
* Move a few more traits from ethcore to client-traits: BlockChainReset, ScheduleInfo, StateClient
* Move ProvingBlockChainClient to client-traits
* Don't re-export ForkChoice and Transition from ethcore
* Address grumbles: sort imports, remove commented out code
* Fix merge resolution error
* merge failure
* Add client-traits crate
Move the BlockInfo trait to new crate
* New crate `machine`
Contains code extracted from ethcore that defines `Machine`, `Externalities` and other execution related code.
* Use new machine and client-traits crates in ethcore
* Use new crates machine and client-traits instead of ethcore where appropriate
* Fix tests
* Don't re-export so many types from ethcore::client
* Fixing more fallout from removing re-export
* fix test
* More fallout from not re-exporting types
* Add some docs
* cleanup
* import the macro edition style
* Tweak docs
* Add missing import
* remove unused ethabi_derive imports
* Use latest ethabi-contract
* Better logging when backfilling ancient blocks fail
Print total blocks imported, closes#10792
* `finalize()` doesn't need Engine
Pull out call to migrated_blocks() from replace_client_db()
* More logs
* Clarify that the percentage may be misleading
* Remove replace_client_db() and replace with a straight call to restore_db()
* Include the parent_hash in UnlinkedAncientBlockChain errors
* Add a new RestorationStatus varian: Finalizing (as it can take a loooong while)
Call abort_restore() when restoration fails
* Add missing cases for new variant
* typos
* Typo and derive Debug
* Do not attempt to salvage existing blocks unless they form a complete chain back to genesis
* Fix test
* Revert "Fix test"
This reverts commit f027d4b4cb7b6c23fceec528c1711886ba9cfe4e.
* Fix test again
* Update comment
* Be careful about locks
* fix test failure
* Do not defer returning an error when the chain is broken
* Review feedback
* no hex formatting for Option
* fix(ManageNetwork): replace Range -> RangeIncls
Fixes `TODO: Range should be changed to RangeInclusive once stable (https://github.com/rust-lang/rust/pull/50758)`
* fix(tests)
* fix(grumbles): off-by-one error in debug_asserts
* RangeInclusive::end() is inclusive which means that if start and end is equal the `debug_assert(range.end() >
range.start()` will fail which is shouldn't
Fix: new blocks notifications sometimes missing in pubsub RPC
Implement new struct to pass to `new_blocks()` with extra parameter - `has_more_blocks_to_import`, which was previously used to determine whether the notification should be sent. Now it's up to each implementation to decide what to do.
Updated all implementations to behave as before, except `eth_pubsub`, which will send notification even when the queue is not empty.
Update tests.
* Add `is_idle` to LightSync to check importing status
* Use SyncStateWrapper to make sure is_idle gets updates
* Update is_major_import to use verified queue size as well
* Add comment for `is_idle`
* Add Debug to `SyncStateWrapper`
* `fn get` -> `fn into_inner`
This PR is fixing deadlock for #8918
It avoids some recursive calls on light_sync by making state check optional for Informant.
The current behavior is to display the information when informant checks if block is major version.
This change a bit the informant behavior, but not on most cases.
To remember where and how this kind of deadlock are likely to happen (not seen with Parkinglot deadlock detection because it uses std condvar), I am adding a description of the deadlock.
Also, for the reviewers there may be better solution than modifying the informant.
### Thread1
- ethcore/sync/light_sync/mod.rs
A call to the light handler through any Io (having a loop of rpc query running on like client makes the dead lock way more likely).
At the end of those calls we systematically call `maintain_sync` method.
Here maintain_sync locks `state` (it is the deadlock cause), with a write purpose
`maintain_sync` -> `begin_search` with the state locked open
`begin_search` -> lightcliennt `flush_queue` method
- ethcore/light/src/client/mod.rs
`flush_queue` -> `flush` on queue (HeaderQueue aka VerificationQueue of headers)
- ethcore/src/verification/queue/mod.rs
Condition there is some unverified or verifying content
`flush` wait on a condvar until the queue is empty. The only way to unlock condvar is that worker is empty and unlock it (so thread 2 is Verification worker).
### Thread2
A verification worker at the end of a verify loop (new block).
- ethcore/src/verification/queue/mod.rs
thread loops on `verify` method.
End of loop condition is_ready -> Import the block immediately
calls `set_sync` on QueueSignal which send a BlockVerified ClientIoMessage in inner channel (IoChannel of ClientIoMessage) using `send_sync`
- util/io/src/service_mio.rs
IoChannel `send_sync` method calls all handlers with `message` method; one of the handlers is ImportBlocks IoHandler (with a single inner Client service field)
- ethcore/light/src/client/service.rs
`message` trigger inner method `import_verified`
- core/light/src/client/mod.rs
`import_verified` at the very end notify the listeners of a new_headers, one of the listeners is Informant `listener` method
- parity/informant.rs
`newHeaders` run up to call to `is_major_importing` on its target (again clinet)
- ethcore/sync/src/light_sync/mod.rs
Here `is_major_importing` tries to get state lock (read purpose only) but cannot because of previous state lock, thus deadlock
* Keep all enacted blocks notify in order
* Collect is unnecessary
* Update ChainNotify to use ChainRouteType
* Fix all ethcore fn defs
* Wrap the type within ChainRoute
* Fix private-tx and sync api
* Fix secret_store API
* Fix updater API
* Fix rpc api
* Fix informant api
* Eagerly cache enacted/retracted and remove contain_enacted/retracted
* Fix indent
* tests: should use full expr form for struct constructor
* Use into_enacted_retracted to further avoid copy
* typo: not a function
* rpc/tests: ChainRoute -> ChainRoute::new
* Improves `BestBlock` comment
* Improves `TraceDB` comment
* Improves `journaldb::Algorithm` comment.
Probably the whole enum should be renamed to `Strategy` or something alike.
* Comments some of the `Client`'s fields
* Deglobs client imports
* Fixes comments
* Extracts `import_lock` to `Importer` struct
* Extracts `verifier` to `Importer` struct
* Extracts `block_queue` to `Importer` struct
* Extracts `miner` to `Importer` struct
* Extracts `ancient_verifier` to `Importer` struct
* Extracts `rng` to `Importer` struct
* Extracts `import_old_block` to `Importer` struct
* Adds `Nonce` trait
* Adds `Balance` trait
* Adds `ChainInfo` trait
* Fixes imports for tests using `chain_info` method
* Adds `BlockInfo` trait
* Adds more `ChainInfo` imports
* Adds `BlockInfo` imports
* Adds `ReopenBlock` trait
* Adds `PrepareOpenBlock` trait
* Fixes import in tests
* Adds `CallContract` trait
* Fixes imports in tests using `call_contract` method
* Adds `TransactionInfo` trait
* Adds `RegistryInfo` trait
* Fixes imports in tests using `registry_address` method
* Adds `ScheduleInfo` trait
* Adds `ImportSealedBlock` trait
* Fixes imports in test using `import_sealed_block` method
* Adds `BroadcastProposalBlock` trait
* Migrates `Miner` to static dispatch
* Fixes tests
* Moves `calculate_enacted_retracted` to `Importer`
* Moves import-related methods to `Importer`
* Removes redundant `import_old_block` wrapper
* Extracts `import_block*` into separate trait
* Fixes tests
* Handles `Pending` in `LightFetch`
* Handles `Pending` in filters
* Handles `Pending` in `ParityClient`
* Handles `Pending` in `EthClient`
* Removes `BlockId::Pending`, partly refactors dependent code
* Adds `StateInfo` trait
* Exports `StateOrBlock` and `BlockChain` types from `client` module
* Refactors `balance` RPC using generic API
* Refactors `storage_at` RPC using generic API
* Makes `MinerService::pending_state`'s return type dynamic
* Adds `StateOrBlock` and `BlockChain` types
* Adds impl of `client::BlockChain` for `Client`
* Exports `StateInfo` trait from `client` module
* Missing `self` use
To be fixed up to "Adds impl of `client::BlockChain` for `Client`"
* Adds `number_to_id` and refactors dependent RPC methods
* Refactors `code_at` using generic API
* Adds `StateClient` trait
* Refactors RPC to use `StateClient` trait
* Reverts `client::BlockChain` trait stuff, refactors methods to accept `StateOrBlock`
* Refactors TestClient
* Adds helper function `block_number_to_id`
* Uses `block_number_to_id` instead of local function
* Handles `Pending` in `list_accounts` and `list_storage_keys`
* Attempt to use associated types for state instead of trait objects
* Simplifies `state_at_beginning`
* Extracts `call` and `call_many` into separate trait
* Refactors `build_last_hashes` to accept reference
* Exports `Call` type from the module
* Refactors `call` and `call_many` to accept state and header
* Exports `state_at` in `StateClient`
* Exports `pending_block_header` from `MinerService`
* Refactors RPC `call` method using new API
* Adds missing parentheses
* Refactors `parity::call` to use new call API
* Update .gitlab-ci.yml
fix gitlab lint
* Fixes error handling
* Refactors `traces::call` and `call_many` to use new call API
* Refactors `call_contract`
* Refactors `block_header`
* Refactors internal RPC method `block`
* Moves `estimate_gas` to `Call` trait, refactors parameters
* Refactors `estimate_gas` in RPC
* Refactors `uncle`
* Refactors RPC `transaction`
* Covers missing branches
* Makes it all compile, fixes compiler grumbles
* Adds casts in `blockchain` module
* Fixes `PendingBlock` tests, work on `MinerService`
* Adds test stubs for StateClient and EngineInfo
* Makes `state_db` public
* Adds missing impls for `TestBlockChainClient`
* Adds trait documentation
* Adds missing docs to the `state_db` module
* Fixes trivial compilation errors
* Moves `code_hash` method to a `BlockInfo` trait
* Refactors `Verifier` to be generic over client
* Refactors `TransactionFilter` to be generic over client
* Refactors `Miner` and `Client` to reflect changes in verifier and txfilter API
* Moves `ServiceTransactionChecker` back to `ethcore`
* Fixes trait bounds in `Miner` API
* Fixes `Client`
* Fixes lifetime bound in `FullFamilyParams`
* Adds comments to `FullFamilyParams`
* Fixes imports in `ethcore`
* Fixes BlockNumber handling in `code_at` and `replay_block_transactions`
* fix compile issues
* First step to redundant trait merge
* Fixes compilation error in RPC tests
* Adds mock `State` as a stub for `TestClient`
* Handles `StateOrBlock::State` in `TestBlockChainClient::balance`
* Fixes `transaction_count` RPC
* Fixes `transaction_count`
* Moves `service_transaction.json` to the `contracts` subfolder
* Fixes compilation errors in tests
* Refactors client to use `AccountData`
* Refactors client to use `BlockChain`
* Refactors miner to use aggregate traits
* Adds `SealedBlockImporter` trait
* Refactors miner to use `SealedBlockImporter` trait
* Removes unused imports
* Simplifies `RegistryInfo::registry_address`
* Fixes indentation
* Removes commented out trait bound
* do more validation of imported headers in light client
* generalize informant with traits
* informant implementation for light client
* make comment into TODO
* fix broken test
* disable full checking of headers in light client in sync tests
