Commit Graph

353 Commits

Author SHA1 Message Date
Andrew Jones
a04c5b180a Replace legacy Rlp with UntrustedRlp and use in ethcore rlp views (#8316)
* WIP

* Replace Rlp with UntrustedRlp in views, explicity unwrap with expect

First pass to get it to compile. Need to figure out whether to do this or to propogate Errors upstream, which would require many more changes to dependent code. If we do this way we are assuming that the views are always used in a context where the rlp is trusted to be valid e.g. when reading from our own DB. So need to fid out whether views are used with data received from an untrusted (e.g. extrernal peer).

* Remove original Rlp impl, rename UntrustedRlp -> Rlp

* Create rlp views with view! macro to record debug info

Views are assumed to be over valid rlp, so if there is a decoding error we record where the view was created in the first place and report it in the expect

* Use $crate in view! macro to avoid import, fix tests

* Expect valid rlp in decode functions for now

* Replace spaces with tabs in new file

* Add doc tests for creating views with macro

* Update rlp docs to reflect removing of UntrustedRlp

* Replace UntrustedRlp usages in private-tx merge
2018-04-16 15:52:12 +02:00
Pierre Krieger
fac356c701 More code refactoring to integrate Duration (#8322)
* More code refactoring to integrate Duration

* Fix typo

* Fix tests

* More test fix
2018-04-14 21:35:58 +02:00
Tomasz Drwięga
1cd93e4ceb New Transaction Queue implementation (#8074)
* Implementation of Verifier, Scoring and Ready.

* Queue in progress.

* TransactionPool.

* Prepare for txpool release.

* Miner refactor [WiP]

* WiP reworking miner.

* Make it compile.

* Add some docs.

* Split blockchain access to a separate file.

* Work on miner API.

* Fix ethcore tests.

* Refactor miner interface for sealing/work packages.

* Implement next nonce.

* RPC compiles.

* Implement couple of missing methdods for RPC.

* Add transaction queue listeners.

* Compiles!

* Clean-up and parallelize.

* Get rid of RefCell in header.

* Revert "Get rid of RefCell in header."

This reverts commit 0f2424c9b7319a786e1565ea2a8a6d801a21b4fb.

* Override Sync requirement.

* Fix status display.

* Unify logging.

* Extract some cheap checks.

* Measurements and optimizations.

* Fix scoring bug, heap size of bug and add cache

* Disable tx queueing and parallel verification.

* Make ethcore and ethcore-miner compile again.

* Make RPC compile again.

* Bunch of txpool tests.

* Migrate transaction queue tests.

* Nonce Cap

* Nonce cap cache and tests.

* Remove stale future transactions from the queue.

* Optimize scoring and write some tests.

* Simple penalization.

* Clean up and support for different scoring algorithms.

* Add CLI parameters for the new queue.

* Remove banning queue.

* Disable debug build.

* Change per_sender limit to be 1% instead of 5%

* Avoid cloning when propagating transactions.

* Remove old todo.

* Post-review fixes.

* Fix miner options default.

* Implement back ready transactions for light client.

* Get rid of from_pending_block

* Pass rejection reason.

* Add more details to drop.

* Rollback heap size of.

* Avoid cloning hashes when propagating and include more details on rejection.

* Fix tests.

* Introduce nonces cache.

* Remove uneccessary hashes allocation.

* Lower the mem limit.

* Re-enable parallel verification.

* Add miner log. Don't check the type if not below min_gas_price.

* Add more traces, fix disabling miner.

* Fix creating pending blocks twice on AuRa authorities.

* Fix tests.

* re-use pending blocks in AuRa

* Use reseal_min_period to prevent too frequent update_sealing.

* Fix log to contain hash not sender.

* Optimize local transactions.

* Fix aura tests.

* Update locks comments.

* Get rid of unsafe Sync impl.

* Review fixes.

* Remove excessive matches.

* Fix compilation errors.

* Use new pool in private transactions.

* Fix private-tx test.

* Fix secret store tests.

* Actually use gas_floor_target

* Fix config tests.

* Fix pool tests.

* Address grumbles.
2018-04-13 17:34:27 +02:00
Tomasz Drwięga
9f775a7673 Remove RefCell from Header (#8227)
* Cache RLP and header hashes.

* Refactor header - WiP

* Avoid decoding laster header.

* Pre-compute hashes for Sealed/Locked block.

* Use accrue bloom. Closes ##8241
2018-04-03 17:01:28 +09:00
Tomasz Drwięga
9108a3bb50 Bump ethabi & ethereum-types. (#8258)
* Bump ethabi & ethereum-types.

* Fix test.

* Fix hex encodings.
2018-04-02 12:12:52 +01:00
Ryan Leung
9c9ddaccec use constant durations (#8278)
* use constant durations

* fix CI

* address comments
2018-04-02 10:47:56 +02:00
Andrew Jones
e3f7b70c38 Replace all Rlp usages with UntrustedRlp except for ethcore views (#8233)
* Replace Rlp with UntrustedRlp and unsafely unwrap

All Rlp methods return Result<_,DecoderError> now, so for this first
pass each will be marked with `expect("TODO")`. In the next pass we can
categorise figure out how to handle each case.

* Handle DecoderError for tendermint message

* Unwrap rlp results in TestBlockcChainClient

Rlp should be valid since created manually in tests

* Replace `use rlp::*` with explicit imports

* Remove rlp decode unwraps from light cli request

* Structured rlp encoding for curr best and latest in header chain

* Propogate decoder errors from send_packet

* Fix body uncles rlp index

* Use BodyView in sync and `expect` rlp errors

* Revert bbf28f removing original Rlp for this phase

This can be done again in the next phase, in order that we can leave the ethcore views unchanged

* Restore legacy Rlp and UntrustedRlp

Use legacy Rlp for ethcore views. Will redo replacing Rlp with UntrustedRlp in  a subsequent PR

* Fix tests

* Replace boilerplate Encodable/Decodable with derive

* Use BlockView instead of Rlp, remove unwrap

* Remove rlp test_cli unwraps by using BlockView instead of Rlp directly

* Remove unneccesary change to use borrowed hash

* Construct sync block using new_from_header_and_body
2018-03-29 11:19:45 +02:00
Pierre Krieger
04931618ed
Implement hardcoded sync in the light client (#8075)
* Implement hardcoded sync

* Fix concerns

* Remove artifact

* Fix cli tests

* Fix compilation

* Update hardcoded sync block

* Don't use any data fetch for the light service
2018-03-27 13:56:59 +02:00
Pierre Krieger
113c35af0a Remove the time dependency where possible (#8100) 2018-03-14 12:29:52 +01:00
Marek Kotewicz
de72643898
ethcore-service (#8089)
* ethcore test::helpers cleanup

* ethcore-service
2018-03-13 11:49:57 +01:00
Pierre Krieger
eeee90def5 Abstract devp2p (#8048)
* Rename ethcore-network to ethcore-network-devp2p

* Fix typo

* Extract generic traits into util/network

* Simplify util/network

* Fix devp2p tests

* Remove old feature

* Fix RPC tests
2018-03-05 11:56:35 +01:00
Rando
33284e988e
Bump master to 1.11.0 (#8021)
* Bump master to 1.11.0

* Bump price-info

* Bump mac installer version

* Fix gitlab builds
2018-03-03 22:02:33 +01:00
Dmitry Kashitsyn
9d7d6f7108 Client refactoring (#7038)
* Improves `BestBlock` comment

* Improves `TraceDB` comment

* Improves `journaldb::Algorithm` comment.

Probably the whole enum should be renamed to `Strategy` or something alike.

* Comments some of the `Client`'s fields

* Deglobs client imports

* Fixes comments

* Extracts `import_lock` to `Importer` struct

* Extracts `verifier` to `Importer` struct

* Extracts `block_queue` to `Importer` struct

* Extracts `miner` to `Importer` struct

* Extracts `ancient_verifier` to `Importer` struct

* Extracts `rng` to `Importer` struct

* Extracts `import_old_block` to `Importer` struct

* Adds `Nonce` trait

* Adds `Balance` trait

* Adds `ChainInfo` trait

* Fixes imports for tests using `chain_info` method

* Adds `BlockInfo` trait

* Adds more `ChainInfo` imports

* Adds `BlockInfo` imports

* Adds `ReopenBlock` trait

* Adds `PrepareOpenBlock` trait

* Fixes import in tests

* Adds `CallContract` trait

* Fixes imports in tests using `call_contract` method

* Adds `TransactionInfo` trait

* Adds `RegistryInfo` trait

* Fixes imports in tests using `registry_address` method

* Adds `ScheduleInfo` trait

* Adds `ImportSealedBlock` trait

* Fixes imports in test using `import_sealed_block` method

* Adds `BroadcastProposalBlock` trait

* Migrates `Miner` to static dispatch

* Fixes tests

* Moves `calculate_enacted_retracted` to `Importer`

* Moves import-related methods to `Importer`

* Removes redundant `import_old_block` wrapper

* Extracts `import_block*` into separate trait

* Fixes tests

* Handles `Pending` in `LightFetch`

* Handles `Pending` in filters

* Handles `Pending` in `ParityClient`

* Handles `Pending` in `EthClient`

* Removes `BlockId::Pending`, partly refactors dependent code

* Adds `StateInfo` trait

* Exports `StateOrBlock` and `BlockChain` types from `client` module

* Refactors `balance` RPC using generic API

* Refactors `storage_at` RPC using generic API

* Makes `MinerService::pending_state`'s return type dynamic

* Adds `StateOrBlock` and `BlockChain` types

* Adds impl of `client::BlockChain` for `Client`

* Exports `StateInfo` trait from `client` module

* Missing `self` use

To be fixed up to "Adds impl of `client::BlockChain` for `Client`"

* Adds `number_to_id` and refactors dependent RPC methods

* Refactors `code_at` using generic API

* Adds `StateClient` trait

* Refactors RPC to use `StateClient` trait

* Reverts `client::BlockChain` trait stuff, refactors methods to accept `StateOrBlock`

* Refactors TestClient

* Adds helper function `block_number_to_id`

* Uses `block_number_to_id` instead of local function

* Handles `Pending` in `list_accounts` and `list_storage_keys`

* Attempt to use associated types for state instead of trait objects

* Simplifies `state_at_beginning`

* Extracts `call` and `call_many` into separate trait

* Refactors `build_last_hashes` to accept reference

* Exports `Call` type from the module

* Refactors `call` and `call_many` to accept state and header

* Exports `state_at` in `StateClient`

* Exports `pending_block_header` from `MinerService`

* Refactors RPC `call` method using new API

* Adds missing parentheses

* Refactors `parity::call` to use new call API

* Update .gitlab-ci.yml

fix gitlab lint

* Fixes error handling

* Refactors `traces::call` and `call_many` to use new call API

* Refactors `call_contract`

* Refactors `block_header`

* Refactors internal RPC method `block`

* Moves `estimate_gas` to `Call` trait, refactors parameters

* Refactors `estimate_gas` in RPC

* Refactors `uncle`

* Refactors RPC `transaction`

* Covers missing branches

* Makes it all compile, fixes compiler grumbles

* Adds casts in `blockchain` module

* Fixes `PendingBlock` tests, work on `MinerService`

* Adds test stubs for StateClient and EngineInfo

* Makes `state_db` public

* Adds missing impls for `TestBlockChainClient`

* Adds trait documentation

* Adds missing docs to the `state_db` module

* Fixes trivial compilation errors

* Moves `code_hash` method to a `BlockInfo` trait

* Refactors `Verifier` to be generic over client

* Refactors `TransactionFilter` to be generic over client

* Refactors `Miner` and `Client` to reflect changes in verifier and txfilter API

* Moves `ServiceTransactionChecker` back to `ethcore`

* Fixes trait bounds in `Miner` API

* Fixes `Client`

* Fixes lifetime bound in `FullFamilyParams`

* Adds comments to `FullFamilyParams`

* Fixes imports in `ethcore`

* Fixes BlockNumber handling in `code_at` and `replay_block_transactions`

* fix compile issues

* First step to redundant trait merge

* Fixes compilation error in RPC tests

* Adds mock `State` as a stub for `TestClient`

* Handles `StateOrBlock::State` in `TestBlockChainClient::balance`

* Fixes `transaction_count` RPC

* Fixes `transaction_count`

* Moves `service_transaction.json` to the `contracts` subfolder

* Fixes compilation errors in tests

* Refactors client to use `AccountData`

* Refactors client to use `BlockChain`

* Refactors miner to use aggregate traits

* Adds `SealedBlockImporter` trait

* Refactors miner to use `SealedBlockImporter` trait

* Removes unused imports

* Simplifies `RegistryInfo::registry_address`

* Fixes indentation

* Removes commented out trait bound
2018-03-03 18:42:13 +01:00
Pierre Krieger
d4205da484 Extract the hard dependency on rocksdb from the light client (#8034)
* Extract the hard dependency on rocksdb from the light client

* Remove TODO
2018-03-01 19:53:15 +01:00
Nikolay Volf
684322cd6f Kovan WASM fork code (#7849)
* kovan fork code

* introduce ethcore level vm_factory and let it fail

* fix json tests

* wasmcosts as option

* review changes

* wasm costs in parser

* fix evm tests

* review fixes

* fix test

* remove redundant json field
2018-02-19 12:27:42 +01:00
Pierre Krieger
364a1fa5a8 Fix a division by zero in light client RPC handler (#7917) 2018-02-17 21:54:39 +01:00
Marek Kotewicz
f905a1676f triehash optimisations (#7920) 2018-02-16 20:24:16 +01:00
Tomasz Drwięga
236a4aac22 Pending transactions subscription (#7906)
* Pending transactions subscription.

* Pending transactions Pub-Sub

* Add light client support.

* Remove redundant Ok
2018-02-16 16:51:34 +01:00
André Silva
aab63c339d Aura: Broadcast empty step messages instead of creating empty blocks (#7605)
* aura: broadcast empty step message instead of sealing empty block

* aura: add empty_step messages to seal

* aura: include parent_hash in empty step message

* aura: verify received empty step messages

* aura: verify empty step messages in block

* aura: fix dead lock on empty_steps

* aura: fix EmptyStep Encodable

* aura: take number of empty steps into account in chain score

* aura: use empty step signers for finality

* aura: add empty "empty step" messages to seal when reading from spec

* aura: fix EmptyStep rlp encoding

* aura: use Vec<u8> instead of Bytes

* aura: fix block empty step verification

* Update .gitlab-ci.yml

fix lint

* aura: fix accumulation of empty step signatures for finality

* aura: include empty steps in seal signature

* aura: configurable max number of empty steps

* engine: pass block header to seal_fields method

This is necessary to make the number of seal fields dynamic,
e.g. activating a transition on a certain block number that changes
the seal.

* aura: add transition to enable empty step messages

* aura: clear old empty step messages on verify_block_external

* aura: ignore empty step messages from the future

* aura: report skipped primaries when empty steps are not enabled

* aura: fix tests

* aura: report misbehavior

* aura: add tests for rolling finality with multiple signatures

* engine: fix validator set test

In this test the block validation wasn't failing because the block was in the
future (expected failure) but was instead failing because the author of the
block isn't the expected authority. Since we added reporting of blocks produced
by the wrong authority this test started failing.

* aura: reward all the authors of empty step messages

* aura: fix reward attribution for new blocks

* aura: add tests for empty steps broadcasting and inclusion in blocks

* aura: reduce size of empty step messages in seal

* aura: add test for empty step inclusion in blocks

* aura: add test for rewarding of empty steps

* aura: add test for empty steps validation

* aura: fix rlp encoding of sealed empty step

* aura: fix grumbles
2018-02-15 01:39:29 +01:00
Marek Kotewicz
c060d9584d
ethabi version 5 (#7723)
* Refactor updater to use ethabi-derive

* Grumble: do_call type alias

* Empty commit to trigger test re-run

* migration to ethabi-5.0

* migration to ethabi-5.0 in progress

* use ethabi_deriven to generate TransactAcl contract

* use ethabi_deriven to generate Registry contract

* hash-fetch uses ethabi_derive, removed retain cycle from updater, fixed #7720

* node-filter crate uses ethabi_derive to generate peer_set contract interface

* use LruCache in node-filter instead of HashMap

* validator_set engine uses ethabi_derive

* ethcore does not depend on native_contracts

* miner does no depend on native_contracts

* secret_store does not use native_contracts (in progress)

* removed native-contracts

* ethcore and updater does not depend on futures

* updated ethereum-types

* fixed all warnings caused by using new version of ethereum-types

* updated ethabi_derive && ethabi_contract to get rid of warnings

* removed another retain cycle in updater, fixed following minor version on update

* moved contracts out of native_contracts res

* updated ethabi_contract

* fixed failing test

* fixed failing test

* there is no need to create two contracts of the same kind any more

* simplify updater::ReleaseTrack conversion into u8 and add several tests for it

* applied review suggestions

* applied review suggestions
2018-02-09 09:32:06 +01:00
Pierre Krieger
2c60a53fef Randomize the peer we dispatch to (#7844) 2018-02-08 21:36:46 +01:00
Marek Kotewicz
d430e7f4e2
Updated dependencies (#7628)
* Updated few dependencies

* Updated parking_lot to 0.5

* Updated backtrace to 0.3.5

* Updated crossbeam to 0.3
2018-01-19 13:46:11 +01:00
Robert Habermeier
2c426defd9
remove check_hangup hack now that is_canceled exists (#7620) 2018-01-18 18:19:04 +01:00
Robert Habermeier
c7aed335ce update back-references more aggressively after answering from cache (#7578) 2018-01-17 09:45:36 +01:00
Tomasz Drwięga
f044b61f42 Extract some parts of miner from ethcore. (#7353)
* Move miner away from ethcore.

* Fix ethcore to use miner/transaction.

* Fix tests and warnings.

* fixed incorrect merge of the test in the documentation
2018-01-11 17:49:10 +01:00
Marek Kotewicz
e95b093483 dissolve util (#7460)
* ethereum-types refactor in progress

* ethereum-types refactor in progress

* ethereum-types refactor in progress

* ethereum-types refactor in progress

* ethereum-types refactor finished

* removed obsolete util/src/lib.rs

* removed commented out code
2018-01-10 15:35:18 +03:00
André Silva
e114b0b28d Upgrade to RocksDB 5.8.8 and tune settings to reduce space amplification (#7348)
* kvdb-rocksdb: update to RocksDB 5.8.8

* kvdb-rocksdb: tune RocksDB options

* Switch to level-style compaction
* Increase default block size (16K), and use bigger blocks for HDDs (64K)
* Increase default file size base (64MB SSDs, 256MB HDDs)
* Create a single block cache shared across all column families
* Tune compaction settings using RocksDB helper functions, taking into account
  memory budget spread across all columns
* Configure backgrounds jobs based on the number of CPUs
* Set some default recommended settings

* ethcore: remove unused config blockchain.db_cache_size

* parity: increase default value for db_cache_size

* kvdb-rocksdb: enable compression on all levels

* kvdb-rocksdb: set global db_write_bufer_size

* kvdb-rocksdb: reduce db_write_bufer_size to force earlier flushing

* kvdb-rocksdb: use master branch for rust-rocksdb dependency
2018-01-03 11:00:37 +01:00
Robert Habermeier
81212c5031 Merge branch 'master' into upload-crates 2017-11-17 16:42:41 +01:00
Marek Kotewicz
0230a44b15
Merge pull request #7040 from paritytech/squashed_network_error_chain
squashed ethcore-network changes which introduce error-chain
2017-11-15 18:18:25 +01:00
debris
a22c48b6b0 removed redundant imports 2017-11-14 17:47:41 +01:00
debris
3cf52dac59 use error-chain in ethcore-network 2017-11-13 16:48:58 +01:00
Robert Habermeier
ec5519ccd1 rename hash crate to keccak-hash 2017-11-10 20:08:28 +01:00
Robert Habermeier
2288bcd1b4 Merge branch 'master' of github.com:paritytech/parity into upload-crates 2017-11-10 18:42:18 +01:00
Robert Habermeier
be092e7c09 prepare cargo configuration for upload of crates 2017-11-10 18:31:31 +01:00
David Dorgan
bdf1e6afcc Revert "Sync parity-ewf with parity"
This reverts commit f7ba063b22.
2017-10-24 13:31:58 +01:00
David Dorgan
f7ba063b22 Sync parity-ewf with parity 2017-10-24 12:31:50 +01:00
Robert Habermeier
965dff3d32 light: get local transactions by hash 2017-10-24 07:30:53 +02:00
debris
95f0384510 Merge branch 'master' into ipc_cleanup 2017-10-16 18:19:41 +02:00
debris
16d84f8ecf finished removing ipc 2017-10-16 18:18:43 +02:00
debris
6dc50d01b5 KeyValueDB trait uses errors instead of strings 2017-10-16 12:11:35 +02:00
debris
2ce0eae406 Merge branch 'master' into kvdb_split 2017-10-16 10:01:12 +02:00
Dmitry Kashitsyn
92a9fd3359 Uses dash instead of underscore in crate name 2017-10-16 11:05:54 +07:00
Dmitry Kashitsyn
4df541e47a Refactors references to MemoryLruCache in ethcore (#6693) 2017-10-15 21:47:06 +07:00
debris
616cafb04a renamed kvdb_memorydb::in_memory -> kvdb_memorydb::create 2017-10-15 16:17:15 +02:00
debris
5c220ab64b Merge branch 'master' into kvdb_split 2017-10-13 12:18:49 +02:00
Arkadiy Paronyan
f3ea2f354e v1.9.0 (#6727) 2017-10-12 23:44:02 +02:00
debris
eb526b7769 separated kvdb into 3 crates: kvdb, kvdb-memorydb && kvdb-rocksdb, #6693 2017-10-12 15:36:27 +02:00
Marek Kotewicz
6279ff32f5 Separate migrations from util (#6690)
* separate migration from util and make its dependencies into libs:

* snappy
* kvdb
* error
* common

* renamed common -> macros

* util error does not depend on snappy module

* ethsync does not depend on util nor ethcore_error

* nibbleslice and nibblevec merged with patricia_trie crate

* removed unused dependencies from util

* util journaldb traits does not need to be public

* util_error

* fixed ethcore compile error

* ignore .swo files

* Update chain.rs
2017-10-10 20:01:27 +02:00
Robert Habermeier
b010fb5004 Light Client: fetch transactions/receipts by transaction hash (#6641)
* rpc: transaction/receipt requests made async

* rpc: light client fetches transaction and uncle by hash/index

* on_demand: request type for transaction index

* serve transaction index requests in light protocol

* add a test for transaction index serving

* fetch transaction and receipts by hash on light client

* fix decoding tests

* light: more lenient cost table parsing (backwards compatible)

* fix tests and warnings

* LES -> PIP

* Update provider.rs

* proper doc comments for public functions
2017-10-08 18:19:27 +02:00
Robert Habermeier
bc167a211b Generalize engine trait (#6591)
* move common forks and parameters to common params

* port specs over to new format

* fix RPC tests

* parity-machine skeleton

* remove block type

* extract out ethereum-specific methods into EthereumMachine

* beginning to integrate Machine into engines. dealing with stale transitions in Ethash

* initial porting to machine

* move block reward back into engine

* abstract block reward logic

* move last hash and DAO HF logic into machine

* begin making engine function parameters generic

* abstract epoch verifier and ethash block reward logic

* instantiate special ethereummachine for ethash in spec

* optional full verification in verify_block_family

* re-instate tx_filter in a way that works for all engines

* fix warnings

* fix most tests, further generalize engine trait

* uncomment nullengine, get ethcore tests compiling

* fix warnings

* update a bunch of specs

* re-enable engine signer, validator set, and transition handler

* migrate basic_authority engine

* move last hashes into executedblock

* port tendermint

* make all ethcore tests pass

* json-tests compilation

* fix RPC tests: change in gas limit for new block changed PoW hash

* fix minor grumbles

* validate chainspecs

* fix broken import

* fix transaction verification for pre-homestead
2017-09-26 14:19:08 +02:00