Commit Graph

68 Commits

Author SHA1 Message Date
Niklas Adolfsson 98b7c07171 Update `license header` and `scripts` (#8666)
* Update `add_license` script

* run script

* add `remove duplicate lines script` and run it

* Revert changes `English spaces`

* strip whitespaces

* Revert `GPL` in files with `apache/mit license`

* don't append `gpl license` in files with other lic

* Don't append `gpl header` in files with other lic.

* re-ran script

* include c and cpp files too

* remove duplicate header

* rebase nit
2018-06-04 10:19:50 +02:00
Wei Tang 0ecbb3ec02
Fork choice and metadata framework for Engine (#8401)
* Add light client TODO item

* Move existing total-difficulty-based fork choice check to Engine

* Abstract total difficulty and block provider as Machine::BlockMetadata and Machine::BlockProvider

* Decouple "generate_metadata" logic to Engine

* Use fixed BlockMetadata and BlockProvider type for null and instantseal

In this way they can use total difficulty fork choice check

* Extend blockdetails with metadatas and finalized info

* Extra data update: mark_finalized and update_metadatas

* Check finalized block in Blockchain

* Fix a test constructor in verification mod

* Add total difficulty trait

* Fix type import

* Db migration to V13 with metadata column

* Address grumbles

* metadatas -> metadata

* Use generic type for update_metadata to avoid passing HashMap all around

* Remove metadata in blockdetails

* [WIP] Implement a generic metadata architecture

* [WIP] Metadata insertion logic in BlockChain

* typo: Value -> Self::Value

* [WIP] Temporarily remove Engine::is_new_best interface

So that we don't have too many type errors.

* [WIP] Fix more type errors

* [WIP] ExtendedHeader::PartialEq

* [WIP] Change metadata type Option<Vec<u8>> to Vec<u8>

* [WIP] Remove Metadata Error

* [WIP] Clean up error conversion

* [WIP] finalized -> is_finalized

* [WIP] Mark all fields in ExtrasInsert as pub

* [WIP] Remove unused import

* [WIP] Keep only local metadata info

* Mark metadata as optional

* [WIP] Revert metadata db change in BlockChain

* [WIP] Put finalization in unclosed state

* Use metadata interface in BlockDetail

* [WIP] Fix current build failures

* [WIP] Remove unused blockmetadata struct

* Remove DB migration info

* [WIP] Typo

* Use ExtendedHeader to implement fork choice check

* Implement is_new_best using Ancestry iterator

* Use expect instead of panic

* [WIP] Add ancestry Engine support via on_new_block

* Fix tests

* Emission of ancestry actions

* use_short_version should take account of metadata

* Engine::is_new_best -> Engine::fork_choice

* Use proper expect format as defined in #1026

* panic -> expect

* ancestry_header -> ancestry_with_metadata

* Boxed iterator -> &mut iterator

* Fix tests

* is_new_best -> primitive_fork_choice

* Document how fork_choice works

* Engine::fork_choice -> Engine::primitive_fork_choice

* comment: clarify types of finalization where Engine::primitive_fork_choice works

* Expose FinalizationInfo to Engine

* Fix tests due to merging

* Remove TotalDifficulty trait

* Do not pass FinalizationInfo to Engine

If there's finalized blocks in from route, choose the old branch without calling `Engine::fork_choice`.

* Fix compile

* Fix unused import

* Remove is_to_route_finalized

When no block reorg passes a finalized block, this variable is always false.

* Address format grumbles

* Fix docs: mark_finalized returns None if block hash is not found

`blockchain` mod does not yet have an Error type, so we still temporarily use None here.

* Fix inaccurate tree_route None expect description
2018-05-16 14:58:01 +08:00
David 842b75c0e6 Decoding headers can fail (#8570)
* rlp::decode returns Result

* Fix journaldb to handle rlp::decode Result

* Fix ethcore to work with rlp::decode returning Result

* Light client handles rlp::decode returning Result

* Fix tests in rlp_derive

* Fix tests

* Cleanup

* cleanup

* Allow panic rather than breaking out of iterator

* Let decoding failures when reading from disk blow up

* syntax

* Fix the trivial grumbles

* Fix failing tests

* Make Account::from_rlp return Result

* Syntx, sigh

* Temp-fix for decoding failures

* Header::decode returns Result

Handle new return type throughout the code base.

* Do not continue reading from the DB when a value could not be read

* Fix tests

* Handle header decoding in light_sync

* Handling header decoding errors

* Let the DecodeError bubble up unchanged

* Remove redundant error conversion
2018-05-09 12:05:56 +02:00
David 28c731881f
Rlp decode returns Result (#8527)
rlp::decode returns Result

Make a best effort to handle decoding errors gracefully throughout the code, using `expect` where the value is guaranteed to be valid (and in other places where it makes sense).
2018-05-08 11:22:12 +02:00
Andrew Jones 14361cc7b1 Move ethcore::Error to error_chain (#8386)
* WIP

* Convert Ethcore error to use error_chain

* Use error_chain for ImportError and BlockImportError

* Fix error pattern matches for error_chain in miner

* Implement explicit From for AccountsError

* Fix pattern matches for ErrorKinds

* Handle ethcore error_chain in light client

* Explicitly define Result type to avoid shadowing

* Fix remaining Error pattern matches

* Fix tab space formatting

* Helps if the tests compile

* Fix error chain matching after merge
2018-04-19 11:52:54 +02:00
Andrew Jones a04c5b180a Replace legacy Rlp with UntrustedRlp and use in ethcore rlp views (#8316)
* WIP

* Replace Rlp with UntrustedRlp in views, explicity unwrap with expect

First pass to get it to compile. Need to figure out whether to do this or to propogate Errors upstream, which would require many more changes to dependent code. If we do this way we are assuming that the views are always used in a context where the rlp is trusted to be valid e.g. when reading from our own DB. So need to fid out whether views are used with data received from an untrusted (e.g. extrernal peer).

* Remove original Rlp impl, rename UntrustedRlp -> Rlp

* Create rlp views with view! macro to record debug info

Views are assumed to be over valid rlp, so if there is a decoding error we record where the view was created in the first place and report it in the expect

* Use $crate in view! macro to avoid import, fix tests

* Expect valid rlp in decode functions for now

* Replace spaces with tabs in new file

* Add doc tests for creating views with macro

* Update rlp docs to reflect removing of UntrustedRlp

* Replace UntrustedRlp usages in private-tx merge
2018-04-16 15:52:12 +02:00
Andrew Jones e3f7b70c38 Replace all Rlp usages with UntrustedRlp except for ethcore views (#8233)
* Replace Rlp with UntrustedRlp and unsafely unwrap

All Rlp methods return Result<_,DecoderError> now, so for this first
pass each will be marked with `expect("TODO")`. In the next pass we can
categorise figure out how to handle each case.

* Handle DecoderError for tendermint message

* Unwrap rlp results in TestBlockcChainClient

Rlp should be valid since created manually in tests

* Replace `use rlp::*` with explicit imports

* Remove rlp decode unwraps from light cli request

* Structured rlp encoding for curr best and latest in header chain

* Propogate decoder errors from send_packet

* Fix body uncles rlp index

* Use BodyView in sync and `expect` rlp errors

* Revert bbf28f removing original Rlp for this phase

This can be done again in the next phase, in order that we can leave the ethcore views unchanged

* Restore legacy Rlp and UntrustedRlp

Use legacy Rlp for ethcore views. Will redo replacing Rlp with UntrustedRlp in  a subsequent PR

* Fix tests

* Replace boilerplate Encodable/Decodable with derive

* Use BlockView instead of Rlp, remove unwrap

* Remove rlp test_cli unwraps by using BlockView instead of Rlp directly

* Remove unneccesary change to use borrowed hash

* Construct sync block using new_from_header_and_body
2018-03-29 11:19:45 +02:00
Pierre Krieger 04931618ed
Implement hardcoded sync in the light client (#8075)
* Implement hardcoded sync

* Fix concerns

* Remove artifact

* Fix cli tests

* Fix compilation

* Update hardcoded sync block

* Don't use any data fetch for the light service
2018-03-27 13:56:59 +02:00
Pierre Krieger 113c35af0a Remove the time dependency where possible (#8100) 2018-03-14 12:29:52 +01:00
Dmitry Kashitsyn 9d7d6f7108 `Client` refactoring (#7038)
* Improves `BestBlock` comment

* Improves `TraceDB` comment

* Improves `journaldb::Algorithm` comment.

Probably the whole enum should be renamed to `Strategy` or something alike.

* Comments some of the `Client`'s fields

* Deglobs client imports

* Fixes comments

* Extracts `import_lock` to `Importer` struct

* Extracts `verifier` to `Importer` struct

* Extracts `block_queue` to `Importer` struct

* Extracts `miner` to `Importer` struct

* Extracts `ancient_verifier` to `Importer` struct

* Extracts `rng` to `Importer` struct

* Extracts `import_old_block` to `Importer` struct

* Adds `Nonce` trait

* Adds `Balance` trait

* Adds `ChainInfo` trait

* Fixes imports for tests using `chain_info` method

* Adds `BlockInfo` trait

* Adds more `ChainInfo` imports

* Adds `BlockInfo` imports

* Adds `ReopenBlock` trait

* Adds `PrepareOpenBlock` trait

* Fixes import in tests

* Adds `CallContract` trait

* Fixes imports in tests using `call_contract` method

* Adds `TransactionInfo` trait

* Adds `RegistryInfo` trait

* Fixes imports in tests using `registry_address` method

* Adds `ScheduleInfo` trait

* Adds `ImportSealedBlock` trait

* Fixes imports in test using `import_sealed_block` method

* Adds `BroadcastProposalBlock` trait

* Migrates `Miner` to static dispatch

* Fixes tests

* Moves `calculate_enacted_retracted` to `Importer`

* Moves import-related methods to `Importer`

* Removes redundant `import_old_block` wrapper

* Extracts `import_block*` into separate trait

* Fixes tests

* Handles `Pending` in `LightFetch`

* Handles `Pending` in filters

* Handles `Pending` in `ParityClient`

* Handles `Pending` in `EthClient`

* Removes `BlockId::Pending`, partly refactors dependent code

* Adds `StateInfo` trait

* Exports `StateOrBlock` and `BlockChain` types from `client` module

* Refactors `balance` RPC using generic API

* Refactors `storage_at` RPC using generic API

* Makes `MinerService::pending_state`'s return type dynamic

* Adds `StateOrBlock` and `BlockChain` types

* Adds impl of `client::BlockChain` for `Client`

* Exports `StateInfo` trait from `client` module

* Missing `self` use

To be fixed up to "Adds impl of `client::BlockChain` for `Client`"

* Adds `number_to_id` and refactors dependent RPC methods

* Refactors `code_at` using generic API

* Adds `StateClient` trait

* Refactors RPC to use `StateClient` trait

* Reverts `client::BlockChain` trait stuff, refactors methods to accept `StateOrBlock`

* Refactors TestClient

* Adds helper function `block_number_to_id`

* Uses `block_number_to_id` instead of local function

* Handles `Pending` in `list_accounts` and `list_storage_keys`

* Attempt to use associated types for state instead of trait objects

* Simplifies `state_at_beginning`

* Extracts `call` and `call_many` into separate trait

* Refactors `build_last_hashes` to accept reference

* Exports `Call` type from the module

* Refactors `call` and `call_many` to accept state and header

* Exports `state_at` in `StateClient`

* Exports `pending_block_header` from `MinerService`

* Refactors RPC `call` method using new API

* Adds missing parentheses

* Refactors `parity::call` to use new call API

* Update .gitlab-ci.yml

fix gitlab lint

* Fixes error handling

* Refactors `traces::call` and `call_many` to use new call API

* Refactors `call_contract`

* Refactors `block_header`

* Refactors internal RPC method `block`

* Moves `estimate_gas` to `Call` trait, refactors parameters

* Refactors `estimate_gas` in RPC

* Refactors `uncle`

* Refactors RPC `transaction`

* Covers missing branches

* Makes it all compile, fixes compiler grumbles

* Adds casts in `blockchain` module

* Fixes `PendingBlock` tests, work on `MinerService`

* Adds test stubs for StateClient and EngineInfo

* Makes `state_db` public

* Adds missing impls for `TestBlockChainClient`

* Adds trait documentation

* Adds missing docs to the `state_db` module

* Fixes trivial compilation errors

* Moves `code_hash` method to a `BlockInfo` trait

* Refactors `Verifier` to be generic over client

* Refactors `TransactionFilter` to be generic over client

* Refactors `Miner` and `Client` to reflect changes in verifier and txfilter API

* Moves `ServiceTransactionChecker` back to `ethcore`

* Fixes trait bounds in `Miner` API

* Fixes `Client`

* Fixes lifetime bound in `FullFamilyParams`

* Adds comments to `FullFamilyParams`

* Fixes imports in `ethcore`

* Fixes BlockNumber handling in `code_at` and `replay_block_transactions`

* fix compile issues

* First step to redundant trait merge

* Fixes compilation error in RPC tests

* Adds mock `State` as a stub for `TestClient`

* Handles `StateOrBlock::State` in `TestBlockChainClient::balance`

* Fixes `transaction_count` RPC

* Fixes `transaction_count`

* Moves `service_transaction.json` to the `contracts` subfolder

* Fixes compilation errors in tests

* Refactors client to use `AccountData`

* Refactors client to use `BlockChain`

* Refactors miner to use aggregate traits

* Adds `SealedBlockImporter` trait

* Refactors miner to use `SealedBlockImporter` trait

* Removes unused imports

* Simplifies `RegistryInfo::registry_address`

* Fixes indentation

* Removes commented out trait bound
2018-03-03 18:42:13 +01:00
Marek Kotewicz c060d9584d
ethabi version 5 (#7723)
* Refactor updater to use ethabi-derive

* Grumble: do_call type alias

* Empty commit to trigger test re-run

* migration to ethabi-5.0

* migration to ethabi-5.0 in progress

* use ethabi_deriven to generate TransactAcl contract

* use ethabi_deriven to generate Registry contract

* hash-fetch uses ethabi_derive, removed retain cycle from updater, fixed #7720

* node-filter crate uses ethabi_derive to generate peer_set contract interface

* use LruCache in node-filter instead of HashMap

* validator_set engine uses ethabi_derive

* ethcore does not depend on native_contracts

* miner does no depend on native_contracts

* secret_store does not use native_contracts (in progress)

* removed native-contracts

* ethcore and updater does not depend on futures

* updated ethereum-types

* fixed all warnings caused by using new version of ethereum-types

* updated ethabi_derive && ethabi_contract to get rid of warnings

* removed another retain cycle in updater, fixed following minor version on update

* moved contracts out of native_contracts res

* updated ethabi_contract

* fixed failing test

* fixed failing test

* there is no need to create two contracts of the same kind any more

* simplify updater::ReleaseTrack conversion into u8 and add several tests for it

* applied review suggestions

* applied review suggestions
2018-02-09 09:32:06 +01:00
Marek Kotewicz e95b093483 dissolve util (#7460)
* ethereum-types refactor in progress

* ethereum-types refactor in progress

* ethereum-types refactor in progress

* ethereum-types refactor in progress

* ethereum-types refactor finished

* removed obsolete util/src/lib.rs

* removed commented out code
2018-01-10 15:35:18 +03:00
debris 6dc50d01b5 KeyValueDB trait uses errors instead of strings 2017-10-16 12:11:35 +02:00
debris 616cafb04a renamed kvdb_memorydb::in_memory -> kvdb_memorydb::create 2017-10-15 16:17:15 +02:00
debris eb526b7769 separated kvdb into 3 crates: kvdb, kvdb-memorydb && kvdb-rocksdb, #6693 2017-10-12 15:36:27 +02:00
Marek Kotewicz 6279ff32f5 Separate migrations from util (#6690)
* separate migration from util and make its dependencies into libs:

* snappy
* kvdb
* error
* common

* renamed common -> macros

* util error does not depend on snappy module

* ethsync does not depend on util nor ethcore_error

* nibbleslice and nibblevec merged with patricia_trie crate

* removed unused dependencies from util

* util journaldb traits does not need to be public

* util_error

* fixed ethcore compile error

* ignore .swo files

* Update chain.rs
2017-10-10 20:01:27 +02:00
Robert Habermeier ad39446e87 Revert "fixed master (#6465)"
This reverts commit 899538ae25.
2017-09-05 17:54:05 +02:00
Marek Kotewicz 899538ae25 fixed master (#6465)
* fixed master

* Revert "Merge pull request #6370 from paritytech/light-poa"

This reverts commit 3c60f99def, reversing
changes made to b731ccea18.
2017-09-05 14:53:09 +02:00
Fredrik Harrysson dccdedc001 Merge branch 'master' into fo-6418-dont-export-bigint 2017-09-05 10:48:54 +02:00
Fredrik e54784bfeb Merge branch 'master' into fo-6418-dont-export-bigint
# Conflicts:
#	dapps/src/tests/helpers/registrar.rs
#	ethcore/evm/src/interpreter/shared_cache.rs
#	ethcore/light/src/client/header_chain.rs
#	ethcore/light/src/client/mod.rs
#	ethcore/light/src/net/mod.rs
#	ethcore/light/src/on_demand/request.rs
#	ethcore/light/src/on_demand/tests.rs
#	ethcore/light/src/provider.rs
#	ethcore/node_filter/src/lib.rs
#	ethcore/src/block.rs
#	ethcore/src/blockchain/blockchain.rs
#	ethcore/src/client/test_client.rs
#	ethcore/src/engines/authority_round/mod.rs
#	ethcore/src/engines/basic_authority.rs
#	ethcore/src/engines/mod.rs
#	ethcore/src/engines/tendermint/mod.rs
#	ethcore/src/engines/validator_set/contract.rs
#	ethcore/src/engines/validator_set/multi.rs
#	ethcore/src/engines/validator_set/safe_contract.rs
#	ethcore/src/engines/vote_collector.rs
#	ethcore/src/miner/external.rs
#	ethcore/src/miner/miner.rs
#	ethcore/src/miner/service_transaction_checker.rs
#	ethcore/src/miner/work_notify.rs
#	ethcore/src/pod_account.rs
#	ethcore/src/pod_state.rs
#	ethcore/src/snapshot/block.rs
#	ethcore/src/snapshot/consensus/work.rs
#	ethcore/src/snapshot/mod.rs
#	ethcore/src/snapshot/service.rs
#	ethcore/src/spec/spec.rs
#	ethcore/src/state/backend.rs
#	ethcore/src/trace/db.rs
#	ethcore/src/verification/queue/mod.rs
#	ethcore/src/verification/verification.rs
#	parity/informant.rs
#	rpc/src/v1/helpers/dispatch.rs
#	rpc/src/v1/helpers/light_fetch.rs
#	rpc/src/v1/helpers/signing_queue.rs
#	rpc/src/v1/impls/eth.rs
#	rpc/src/v1/impls/eth_filter.rs
#	rpc/src/v1/impls/eth_pubsub.rs
#	rpc/src/v1/impls/light/eth.rs
#	rpc/src/v1/impls/signing.rs
#	rpc/src/v1/tests/helpers/miner_service.rs
#	rpc/src/v1/tests/helpers/snapshot_service.rs
#	rpc/src/v1/tests/helpers/sync_provider.rs
#	rpc/src/v1/tests/mocked/eth.rs
#	stratum/src/lib.rs
#	sync/src/blocks.rs
#	sync/src/chain.rs
#	sync/src/light_sync/mod.rs
#	sync/src/tests/helpers.rs
#	sync/src/tests/snapshot.rs
#	updater/src/updater.rs
#	util/src/lib.rs
#	util/triehash/src/lib.rs
2017-09-04 18:32:55 +02:00
Fredrik 308f36a532 Don't reexport bigint from util 2017-09-04 16:36:49 +02:00
Robert Habermeier ffde22e711 move genesis_epoch_data function to spec 2017-09-04 12:46:32 +02:00
Robert Habermeier e69e8254db instantiate genesis epoch data if nonexistant in header_chain 2017-09-04 12:37:03 +02:00
Robert Habermeier 372779915f Merge branch 'master' into light-poa 2017-09-04 12:30:37 +02:00
Hawstein 7b8af30590 remove re-export of parking_lot in util (#6435) 2017-09-02 20:09:13 +02:00
debris 94f717a255 Merge branch 'master' into keccak_fn 2017-08-31 11:53:26 +02:00
debris 7849fff41e finished 2017-08-31 11:35:41 +02:00
Robert Habermeier abfa8e95a2 Merge branch 'master' into light-poa 2017-08-30 18:18:52 +02:00
Axel Chalon 6b5ad69c22 Remove reexport of HeapSizeOf in util (#6419) 2017-08-30 16:04:47 +02:00
Robert Habermeier c79ecee094 only load ancestry from chain closure in engine 2017-08-24 16:29:31 +02:00
Robert Habermeier 6ab0fc4e14 epoch transitions in the header chain DB 2017-08-19 17:07:31 +09:00
Robert Habermeier 7d348e2260 light client fixes (#6148)
* light client fixes

* fix memory-lru-cache

* clear pending reqs on disconnect
2017-07-26 15:48:00 +02:00
Vurich 3d8dc11442 Upgrade `elastic-array` to 0.9.0
This is a huge change, which includes some changes to replace code that
originally cloned to reuse allocations instead. The updated
`elastic-array` crate renames its consuming `Vec`-conversion method to
`into_vec`, which means that I can do a simple
`sed -i 's/to_vec/into_vec/'` and then fix the compilation errors.

This commit is probably a minor performance win and definitely a
significant readability win.
2017-06-29 13:05:33 +02:00
Robert Habermeier daf1495c4e Filters and block RPCs for the light client (#5320)
* block_hash method for LightChainClient

* abstraction and futures-based eth_filter

* log fetching for light client

* add eth-filter delegate

* eth_block fetching RPCs

* return default accounts from on_demand

* fix early exit

* BlockNumber -> BlockId

* early exit for no known block number.
2017-04-12 12:07:54 +02:00
Adrian Brink 645b8e4b0b Fix failing tests 2017-04-07 21:35:42 +02:00
Adrian Brink fa88ee148a Add header to cache 2017-04-07 21:35:41 +02:00
Adrian Brink 5e33fe0aa7 Add cache to HeaderChain struct. 2017-04-07 21:35:41 +02:00
Adrian Brink 9552ca7bb3 Change to snakecase 2017-04-07 21:35:41 +02:00
Adrian Brink 3632a98062 Clean up 2017-04-07 21:35:41 +02:00
Adrian Brink 23c76caafa Add header to cache 2017-04-07 21:35:41 +02:00
Adrian Brink 8a7ca6f0ba Add caching to block_header() 2017-04-07 21:34:29 +02:00
Adrian Brink 8ea25eeb3c Add cache to HeaderChain struct. 2017-04-07 21:34:29 +02:00
Robert Habermeier 0d8a2c8c44 CHT key optimization 2017-04-05 11:30:51 +02:00
Robert Habermeier ac7f1f6719 fix header chain tests 2017-03-23 14:15:13 +01:00
Robert Habermeier 0d110ed47c apply pending changes to chain after DB commit 2017-03-23 04:00:22 +01:00
Robert Habermeier 0abbd7ac97 Merge branch 'pip-msg' into lightcli 2017-03-22 22:08:39 +01:00
Robert Habermeier 21771aa1a6 don't keep headers in memory to avoid DoS 2017-03-21 20:23:58 +01:00
Robert Habermeier d013a13be6 header_chain writes to database 2017-03-21 19:45:52 +01:00
Robert Habermeier 41effadb94 RLP encoding and decoding for requests 2017-03-06 12:21:06 +01:00
Robert Habermeier 3b023c82b7 fetch gas price corpus from network when needed 2017-02-17 17:08:46 +01:00