* VerificationQueue don't spawn up extra threads
In the verification queue we spawn up worker threads to do the work.
However, if `num-verifiers` is specified we still spawn the maximum
number of threads which consume extra memory.
There is one catch though when `--scale-verifiers` is specified then
we can't do it because all threads are created upon initilization AFAIK.
In my opinion, is doesn't to use both `num-verifiers` and
`scale-verifiers` they are kind of contradictory!
* Fix nits in logic and add tests for verification
* refactor(verification queue) - rm hardcoded const
* Address grumbles in new tests
* Remove hardcoded `MAX_VERIFIERS` constant and replace it by relying
entirely on `num_cpu` crate instead inorder to support CPUs that have
more cores/logical cores
* ethcore: don't validate difficulty when ignoring seal check
* ethcore: fix block verification test
* ethcore: document skipped verifications when check_seal is disabled
* blockchain insert expects owned block instead of block reference
* reduce a number of times a block is deserialized
* removed cached uncle_bytes from block
* removed is_finalized from OpenBlock
* removed unused parity_machine::WithMetadata trait
* removed commented out code
* remove unused metadata from block
* remove unused metadata from block
* BlockDetails extras may have at most 5 elements
* Store recently rejected transactions.
* Don't cache AlreadyImported rejections.
* Make the size of transaction verification queue dependent on pool size.
* Add a test for recently rejected.
* Fix logging for recently rejected.
* Make rejection cache smaller.
* obsolete test removed
* obsolete test removed
* Construct cache with_capacity.
* new blooms database
* fixed conflict in Cargo.lock
* removed bloomchain
* cleanup in progress
* all tests passing in trace db with new blooms-db
* added trace_blooms to BlockChainDB interface, fixed db flushing
* BlockChainDB no longer exposes RwLock in the interface
* automatically flush blooms-db after every insert
* blooms-db uses io::BufReader to read files, wrap blooms-db into Mutex, cause fs::File is just a shared file handle
* fix json_tests
* blooms-db can filter multiple possibilities at the same time
* removed enum trace/db.rs CacheId
* lint fixes
* fixed tests
* kvdb-rocksdb uses fs-swap crate
* update Cargo.lock
* use fs::rename
* fixed failing test on linux
* fix tests
* use fs_swap
* fixed failing test on linux
* cleanup after swap
* fix tests
* fixed osx permissions
* simplify parity database opening functions
* added migration to blooms-db
* address @niklasad1 grumbles
* fix license and authors field of blooms-db Cargo.toml
* restore blooms-db after snapshot
* Update `add_license` script
* run script
* add `remove duplicate lines script` and run it
* Revert changes `English spaces`
* strip whitespaces
* Revert `GPL` in files with `apache/mit license`
* don't append `gpl license` in files with other lic
* Don't append `gpl header` in files with other lic.
* re-ran script
* include c and cpp files too
* remove duplicate header
* rebase nit
* Add light client TODO item
* Move existing total-difficulty-based fork choice check to Engine
* Abstract total difficulty and block provider as Machine::BlockMetadata and Machine::BlockProvider
* Decouple "generate_metadata" logic to Engine
* Use fixed BlockMetadata and BlockProvider type for null and instantseal
In this way they can use total difficulty fork choice check
* Extend blockdetails with metadatas and finalized info
* Extra data update: mark_finalized and update_metadatas
* Check finalized block in Blockchain
* Fix a test constructor in verification mod
* Add total difficulty trait
* Fix type import
* Db migration to V13 with metadata column
* Address grumbles
* metadatas -> metadata
* Use generic type for update_metadata to avoid passing HashMap all around
* Remove metadata in blockdetails
* [WIP] Implement a generic metadata architecture
* [WIP] Metadata insertion logic in BlockChain
* typo: Value -> Self::Value
* [WIP] Temporarily remove Engine::is_new_best interface
So that we don't have too many type errors.
* [WIP] Fix more type errors
* [WIP] ExtendedHeader::PartialEq
* [WIP] Change metadata type Option<Vec<u8>> to Vec<u8>
* [WIP] Remove Metadata Error
* [WIP] Clean up error conversion
* [WIP] finalized -> is_finalized
* [WIP] Mark all fields in ExtrasInsert as pub
* [WIP] Remove unused import
* [WIP] Keep only local metadata info
* Mark metadata as optional
* [WIP] Revert metadata db change in BlockChain
* [WIP] Put finalization in unclosed state
* Use metadata interface in BlockDetail
* [WIP] Fix current build failures
* [WIP] Remove unused blockmetadata struct
* Remove DB migration info
* [WIP] Typo
* Use ExtendedHeader to implement fork choice check
* Implement is_new_best using Ancestry iterator
* Use expect instead of panic
* [WIP] Add ancestry Engine support via on_new_block
* Fix tests
* Emission of ancestry actions
* use_short_version should take account of metadata
* Engine::is_new_best -> Engine::fork_choice
* Use proper expect format as defined in #1026
* panic -> expect
* ancestry_header -> ancestry_with_metadata
* Boxed iterator -> &mut iterator
* Fix tests
* is_new_best -> primitive_fork_choice
* Document how fork_choice works
* Engine::fork_choice -> Engine::primitive_fork_choice
* comment: clarify types of finalization where Engine::primitive_fork_choice works
* Expose FinalizationInfo to Engine
* Fix tests due to merging
* Remove TotalDifficulty trait
* Do not pass FinalizationInfo to Engine
If there's finalized blocks in from route, choose the old branch without calling `Engine::fork_choice`.
* Fix compile
* Fix unused import
* Remove is_to_route_finalized
When no block reorg passes a finalized block, this variable is always false.
* Address format grumbles
* Fix docs: mark_finalized returns None if block hash is not found
`blockchain` mod does not yet have an Error type, so we still temporarily use None here.
* Fix inaccurate tree_route None expect description
* rlp::decode returns Result
* Fix journaldb to handle rlp::decode Result
* Fix ethcore to work with rlp::decode returning Result
* Light client handles rlp::decode returning Result
* Fix tests in rlp_derive
* Fix tests
* Cleanup
* cleanup
* Allow panic rather than breaking out of iterator
* Let decoding failures when reading from disk blow up
* syntax
* Fix the trivial grumbles
* Fix failing tests
* Make Account::from_rlp return Result
* Syntx, sigh
* Temp-fix for decoding failures
* Header::decode returns Result
Handle new return type throughout the code base.
* Do not continue reading from the DB when a value could not be read
* Fix tests
* Handle header decoding in light_sync
* Handling header decoding errors
* Let the DecodeError bubble up unchanged
* Remove redundant error conversion
* Don't panic in import_block if invalid rlp
* Remove redundant type annotation
* Replace RLP header view usage with safe decoding
Using the view will panic with invalid RLP. Here we use Rlp decoding directly which will return a `Result<_, DecoderError>`. While this path currently should not have any invalid RLP - it makes it safer if ever called with invalid RLP from other code paths.
* Fetch logs by hash in blockchain database
* Fix tests
* Add unit test for branch block logs fetching
* Add docs that blocks must already be sorted
* Handle branch block cases properly
* typo: empty -> is_empty
* Remove return_empty_if_none by using a closure
* Use BTreeSet to avoid sorting again
* Move is_canon to BlockChain
* typo: pass value by reference
* Use loop and wrap inside blocks to simplify the code
Borrowed from https://github.com/paritytech/parity/pull/8463#discussion_r183453326
* typo: missed a comment
* WIP
* Convert Ethcore error to use error_chain
* Use error_chain for ImportError and BlockImportError
* Fix error pattern matches for error_chain in miner
* Implement explicit From for AccountsError
* Fix pattern matches for ErrorKinds
* Handle ethcore error_chain in light client
* Explicitly define Result type to avoid shadowing
* Fix remaining Error pattern matches
* Fix tab space formatting
* Helps if the tests compile
* Fix error chain matching after merge
* WIP
* Replace Rlp with UntrustedRlp in views, explicity unwrap with expect
First pass to get it to compile. Need to figure out whether to do this or to propogate Errors upstream, which would require many more changes to dependent code. If we do this way we are assuming that the views are always used in a context where the rlp is trusted to be valid e.g. when reading from our own DB. So need to fid out whether views are used with data received from an untrusted (e.g. extrernal peer).
* Remove original Rlp impl, rename UntrustedRlp -> Rlp
* Create rlp views with view! macro to record debug info
Views are assumed to be over valid rlp, so if there is a decoding error we record where the view was created in the first place and report it in the expect
* Use $crate in view! macro to avoid import, fix tests
* Expect valid rlp in decode functions for now
* Replace spaces with tabs in new file
* Add doc tests for creating views with macro
* Update rlp docs to reflect removing of UntrustedRlp
* Replace UntrustedRlp usages in private-tx merge
* Private transaction message added
* Empty line removed
* Private transactions logic removed from client into the separate module
* Fixed compilation after merge with head
* Signed private transaction message added as well
* Comments after the review fixed
* Private tx execution
* Test update
* Renamed some methods
* Fixed some tests
* Reverted submodules
* Fixed build
* Private transaction message added
* Empty line removed
* Private transactions logic removed from client into the separate module
* Fixed compilation after merge with head
* Signed private transaction message added as well
* Comments after the review fixed
* Encrypted private transaction message and signed reply added
* Private tx execution
* Test update
* Main scenario completed
* Merged with the latest head
* Private transactions API
* Comments after review fixed
* Parameters for private transactions added to parity arguments
* New files added
* New API methods added
* Do not process packets from unconfirmed peers
* Merge with ptm_ss branch
* Encryption and permissioning with key server added
* Fixed compilation after merge
* Version of Parity protocol incremented in order to support private transactions
* Doc strings for constants added
* Proper format for doc string added
* fixed some encryptor.rs grumbles
* Private transactions functionality moved to the separate crate
* Refactoring in order to remove late initialisation
* Tests fixed after moving to the separate crate
* Fetch method removed
* Sync test helpers refactored
* Interaction with encryptor refactored
* Contract address retrieving via substate removed
* Sensible gas limit for private transactions implemented
* New private contract with nonces added
* Parsing of the response from key server fixed
* Build fixed after the merge, native contracts removed
* Crate renamed
* Tests moved to the separate directory
* Handling of errors reworked in order to use error chain
* Encodable macro added, new constructor replaced with default
* Native ethabi usage removed
* Couple conversions optimized
* Interactions with client reworked
* Errors omitting removed
* Fix after merge
* Fix after the merge
* private transactions improvements in progress
* private_transactions -> ethcore/private-tx
* making private transactions more idiomatic
* private-tx encryptor uses shared FetchClient and is more idiomatic
* removed redundant tests, moved integration tests to tests/ dir
* fixed failing service test
* reenable add_notify on private tx provider
* removed private_tx tests from sync module
* removed commented out code
* Use plain password instead of unlocking account manager
* remove dead code
* Link to the contract changed
* Transaction signature chain replay protection module created
* Redundant type conversion removed
* Contract address returned by private provider
* Test fixed
* Addressing grumbles in PrivateTransactions (#8249)
* Tiny fixes part 1.
* A bunch of additional comments and todos.
* Fix ethsync tests.
* resolved merge conflicts
* final private tx pr (#8318)
* added cli option that enables private transactions
* fixed failing test
* fixed failing test
* fixed failing test
* fixed failing test
* Improves `BestBlock` comment
* Improves `TraceDB` comment
* Improves `journaldb::Algorithm` comment.
Probably the whole enum should be renamed to `Strategy` or something alike.
* Comments some of the `Client`'s fields
* Deglobs client imports
* Fixes comments
* Extracts `import_lock` to `Importer` struct
* Extracts `verifier` to `Importer` struct
* Extracts `block_queue` to `Importer` struct
* Extracts `miner` to `Importer` struct
* Extracts `ancient_verifier` to `Importer` struct
* Extracts `rng` to `Importer` struct
* Extracts `import_old_block` to `Importer` struct
* Adds `Nonce` trait
* Adds `Balance` trait
* Adds `ChainInfo` trait
* Fixes imports for tests using `chain_info` method
* Adds `BlockInfo` trait
* Adds more `ChainInfo` imports
* Adds `BlockInfo` imports
* Adds `ReopenBlock` trait
* Adds `PrepareOpenBlock` trait
* Fixes import in tests
* Adds `CallContract` trait
* Fixes imports in tests using `call_contract` method
* Adds `TransactionInfo` trait
* Adds `RegistryInfo` trait
* Fixes imports in tests using `registry_address` method
* Adds `ScheduleInfo` trait
* Adds `ImportSealedBlock` trait
* Fixes imports in test using `import_sealed_block` method
* Adds `BroadcastProposalBlock` trait
* Migrates `Miner` to static dispatch
* Fixes tests
* Moves `calculate_enacted_retracted` to `Importer`
* Moves import-related methods to `Importer`
* Removes redundant `import_old_block` wrapper
* Extracts `import_block*` into separate trait
* Fixes tests
* Handles `Pending` in `LightFetch`
* Handles `Pending` in filters
* Handles `Pending` in `ParityClient`
* Handles `Pending` in `EthClient`
* Removes `BlockId::Pending`, partly refactors dependent code
* Adds `StateInfo` trait
* Exports `StateOrBlock` and `BlockChain` types from `client` module
* Refactors `balance` RPC using generic API
* Refactors `storage_at` RPC using generic API
* Makes `MinerService::pending_state`'s return type dynamic
* Adds `StateOrBlock` and `BlockChain` types
* Adds impl of `client::BlockChain` for `Client`
* Exports `StateInfo` trait from `client` module
* Missing `self` use
To be fixed up to "Adds impl of `client::BlockChain` for `Client`"
* Adds `number_to_id` and refactors dependent RPC methods
* Refactors `code_at` using generic API
* Adds `StateClient` trait
* Refactors RPC to use `StateClient` trait
* Reverts `client::BlockChain` trait stuff, refactors methods to accept `StateOrBlock`
* Refactors TestClient
* Adds helper function `block_number_to_id`
* Uses `block_number_to_id` instead of local function
* Handles `Pending` in `list_accounts` and `list_storage_keys`
* Attempt to use associated types for state instead of trait objects
* Simplifies `state_at_beginning`
* Extracts `call` and `call_many` into separate trait
* Refactors `build_last_hashes` to accept reference
* Exports `Call` type from the module
* Refactors `call` and `call_many` to accept state and header
* Exports `state_at` in `StateClient`
* Exports `pending_block_header` from `MinerService`
* Refactors RPC `call` method using new API
* Adds missing parentheses
* Refactors `parity::call` to use new call API
* Update .gitlab-ci.yml
fix gitlab lint
* Fixes error handling
* Refactors `traces::call` and `call_many` to use new call API
* Refactors `call_contract`
* Refactors `block_header`
* Refactors internal RPC method `block`
* Moves `estimate_gas` to `Call` trait, refactors parameters
* Refactors `estimate_gas` in RPC
* Refactors `uncle`
* Refactors RPC `transaction`
* Covers missing branches
* Makes it all compile, fixes compiler grumbles
* Adds casts in `blockchain` module
* Fixes `PendingBlock` tests, work on `MinerService`
* Adds test stubs for StateClient and EngineInfo
* Makes `state_db` public
* Adds missing impls for `TestBlockChainClient`
* Adds trait documentation
* Adds missing docs to the `state_db` module
* Fixes trivial compilation errors
* Moves `code_hash` method to a `BlockInfo` trait
* Refactors `Verifier` to be generic over client
* Refactors `TransactionFilter` to be generic over client
* Refactors `Miner` and `Client` to reflect changes in verifier and txfilter API
* Moves `ServiceTransactionChecker` back to `ethcore`
* Fixes trait bounds in `Miner` API
* Fixes `Client`
* Fixes lifetime bound in `FullFamilyParams`
* Adds comments to `FullFamilyParams`
* Fixes imports in `ethcore`
* Fixes BlockNumber handling in `code_at` and `replay_block_transactions`
* fix compile issues
* First step to redundant trait merge
* Fixes compilation error in RPC tests
* Adds mock `State` as a stub for `TestClient`
* Handles `StateOrBlock::State` in `TestBlockChainClient::balance`
* Fixes `transaction_count` RPC
* Fixes `transaction_count`
* Moves `service_transaction.json` to the `contracts` subfolder
* Fixes compilation errors in tests
* Refactors client to use `AccountData`
* Refactors client to use `BlockChain`
* Refactors miner to use aggregate traits
* Adds `SealedBlockImporter` trait
* Refactors miner to use `SealedBlockImporter` trait
* Removes unused imports
* Simplifies `RegistryInfo::registry_address`
* Fixes indentation
* Removes commented out trait bound
* aura: broadcast empty step message instead of sealing empty block
* aura: add empty_step messages to seal
* aura: include parent_hash in empty step message
* aura: verify received empty step messages
* aura: verify empty step messages in block
* aura: fix dead lock on empty_steps
* aura: fix EmptyStep Encodable
* aura: take number of empty steps into account in chain score
* aura: use empty step signers for finality
* aura: add empty "empty step" messages to seal when reading from spec
* aura: fix EmptyStep rlp encoding
* aura: use Vec<u8> instead of Bytes
* aura: fix block empty step verification
* Update .gitlab-ci.yml
fix lint
* aura: fix accumulation of empty step signatures for finality
* aura: include empty steps in seal signature
* aura: configurable max number of empty steps
* engine: pass block header to seal_fields method
This is necessary to make the number of seal fields dynamic,
e.g. activating a transition on a certain block number that changes
the seal.
* aura: add transition to enable empty step messages
* aura: clear old empty step messages on verify_block_external
* aura: ignore empty step messages from the future
* aura: report skipped primaries when empty steps are not enabled
* aura: fix tests
* aura: report misbehavior
* aura: add tests for rolling finality with multiple signatures
* engine: fix validator set test
In this test the block validation wasn't failing because the block was in the
future (expected failure) but was instead failing because the author of the
block isn't the expected authority. Since we added reporting of blocks produced
by the wrong authority this test started failing.
* aura: reward all the authors of empty step messages
* aura: fix reward attribution for new blocks
* aura: add tests for empty steps broadcasting and inclusion in blocks
* aura: reduce size of empty step messages in seal
* aura: add test for empty step inclusion in blocks
* aura: add test for rewarding of empty steps
* aura: add test for empty steps validation
* aura: fix rlp encoding of sealed empty step
* aura: fix grumbles
* updated ethereum-types and tiny-keccak
* Updated several deps
* Updated several more dependencies
* Modify dummy file to trigger ci
* fixed update of memmap to 0.6 in ethash crate
* Fixed fetch after update to latest reqwest
* Updated jsonrpc-core with fixes for serde
* add expects in util/version/build.rs
* Move miner away from ethcore.
* Fix ethcore to use miner/transaction.
* Fix tests and warnings.
* fixed incorrect merge of the test in the documentation
* move common forks and parameters to common params
* port specs over to new format
* fix RPC tests
* parity-machine skeleton
* remove block type
* extract out ethereum-specific methods into EthereumMachine
* beginning to integrate Machine into engines. dealing with stale transitions in Ethash
* initial porting to machine
* move block reward back into engine
* abstract block reward logic
* move last hash and DAO HF logic into machine
* begin making engine function parameters generic
* abstract epoch verifier and ethash block reward logic
* instantiate special ethereummachine for ethash in spec
* optional full verification in verify_block_family
* re-instate tx_filter in a way that works for all engines
* fix warnings
* fix most tests, further generalize engine trait
* uncomment nullengine, get ethcore tests compiling
* fix warnings
* update a bunch of specs
* re-enable engine signer, validator set, and transition handler
* migrate basic_authority engine
* move last hashes into executedblock
* port tendermint
* make all ethcore tests pass
* json-tests compilation
* fix RPC tests: change in gas limit for new block changed PoW hash
* fix minor grumbles
* validate chainspecs
* fix broken import
* fix transaction verification for pre-homestead
