Commit Graph

50 Commits

Author SHA1 Message Date
David
ffeaee778c
Pause pruning while snapshotting (#11178)
* WIP. Typos and logging.

* Format todos

* Pause pruning while a snapshot is under way
Logs, docs and todos

* Allocate memory for the full chunk

* Name snapshotting threads

* Ensure `taking_snapshot` is set to false whenever and however `take_snapshot`returns
Rename `take_at` to `request_snapshot_at`
Cleanup

* Let "in_progress" deletion fail
Fix tests

* Just use an atomic

* Review grumbles

* Finish the sentence

* Resolve a few todos and clarify comments.

* Calculate progress rate since last update

* Lockfile

* Fix tests

* typo

* Reinstate default snapshotting frequency
Cut down on the logging noise

* address grumble

* Log memory use with `journal_size()` and explain why.
2019-10-24 16:46:32 +02:00
Anton Gavrilov
f48780c29b
Update ethereum types to 0.8.0 version (#11139)
* Ethereum types updated to 0.8 version

* Fix for rand usage in rpc

* Cargo lock fixed after rebase

* TODO added
2019-10-08 14:18:44 +02:00
David
2b8b8851ab Update kvdb, kvdb-rocksdb and h2 (#11091)
* Update a few dependencies

Updates two dependencies: `kvdb-rocksdb` and `h2`. Brings in `parking_lot 0.9` which is unintended but possibly fine.

* Bump parking_lot to 0.9
Bump kvdb-memorydb to 0.2 (from git atm)

* New kvdb-memorydb is not breaking

* Remove [patch]
2019-09-27 23:29:14 +02:00
David
48629c2bd4
Move snapshot related traits to their proper place (#11012)
* Move snapshot to own crate
Sort out imports

* WIP cargo toml

* Make snapshotting generic over the client
Sort out tests

* Sort out types from blockchain and client

* Sort out sync

* Sort out imports and generics

* Sort out main binary

* Fix sync test-helpers

* Sort out import for secret-store

* Sort out more imports

* Fix easy todos

* cleanup

* Move SnapshotClient and SnapshotWriter to their proper places
Sort out the circular dependency between snapshot and ethcore by moving all snapshot tests to own crate, snapshot-tests

* cleanup

* Cleanup

* fix merge issues

* Update ethcore/snapshot/snapshot-tests/Cargo.toml

Co-Authored-By: Andronik Ordian <write@reusable.software>

* Sort out botched merge

* Ensure snapshot-tests run

* Docs

* Fix grumbles
2019-09-10 22:44:33 +02:00
David
d193ddde19
Extract snapshot to own crate (#11010)
* Move snapshot to own crate
Sort out imports

* WIP cargo toml

* Make snapshotting generic over the client
Sort out tests

* Sort out types from blockchain and client

* Sort out sync

* Sort out imports and generics

* Sort out main binary

* Fix sync test-helpers

* Sort out import for secret-store

* Sort out more imports

* Fix easy todos

* cleanup

* Cleanup

* remove unneded workspace member

* cleanup

* Sort out test-helpers dependency on account-db

* Update ethcore/client-traits/src/lib.rs

Co-Authored-By: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* Update ethcore/snapshot/Cargo.toml
2019-09-03 11:29:25 +02:00
David
cd26526868
Make ClientIoMessage generic over the Client (#10981)
* Add client-traits crate
Move the BlockInfo trait to new crate

* New crate `machine`
Contains code extracted from ethcore that defines `Machine`, `Externalities` and other execution related code.

* Use new machine and client-traits crates in ethcore

* Use new crates machine and client-traits instead of ethcore where appropriate

* Fix tests

* Don't re-export so many types from ethcore::client

* Fixing more fallout from removing re-export

* fix test

* More fallout from not re-exporting types

* Add some docs

* cleanup

* import the macro edition style

* Tweak docs

* Add missing import

* remove unused ethabi_derive imports

* Use latest ethabi-contract

* Move many traits from ethcore/client/traits to client-traits crate
Initial version of extracted Engine trait

* Move snapshot related traits to the engine crate (eew)

* Move a few snapshot related types to common_types
Cleanup Executed as exported from machine crate

* fix warning

* Gradually introduce new engine crate: snapshot

* ethcore typechecks with new engine crate

* Sort out types outside ethcore

* Add an EpochVerifier to ethash and use that in Engine.epoch_verifier()
Cleanup

* Document pub members

* Sort out tests
Sort out default impls for EpochVerifier

* Add test-helpers feature and move EngineSigner impl to the right place

* Sort out tests

* Sort out tests and refactor verification types

* Fix missing traits

* More missing traits
Fix Histogram

* Fix tests and cleanup

* cleanup

* Put back needed logger import

* Don't rexport common_types from ethcore/src/client
Don't export ethcore::client::*

* Remove files no longer used
Use types from the engine crate
Explicit exports from engine::engine

* Get rid of itertools

* Move a few more traits from ethcore to client-traits: BlockChainReset, ScheduleInfo, StateClient

* Move ProvingBlockChainClient to client-traits

* Don't re-export ForkChoice and Transition from ethcore

* Address grumbles: sort imports, remove commented out code

* Fix merge resolution error

* Extract the Clique engine to own crate

* Extract NullEngine and the block_reward module from ethcore

* Extract InstantSeal engine to own crate

* Extract remaining engines

* Extract executive_state to own crate so it can be used by engine crates

* Remove snapshot stuff from the engine crate

* Put snapshot traits back in ethcore

* cleanup

* Remove stuff from ethcore

* Don't use itertools

* itertools in aura is legit-ish

* More post-merge fixes

* Re-export less types in client

* cleanup

* Extract spec to own crate

* Put back the test-helpers from basic-authority

* Fix ethcore benchmarks

* Reduce the public api of ethcore/verification

* WIP

* Add Cargo.toml

* Fix compilation outside ethcore

* Audit uses of import_verified_blocks() and remove unneeded calls
Cleanup

* cleanup

* Remove unused imports from ethcore

* Cleanup

* remove double semi-colons

* Add missing generic param

* More missing generics

* Update ethcore/block-reward/Cargo.toml

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update ethcore/engines/basic-authority/Cargo.toml

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update ethcore/engines/ethash/Cargo.toml

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update ethcore/engines/clique/src/lib.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* signers is already a ref

* Add an EngineType enum to tighten up Engine.name()

* Introduce Snapshotting enum to distinguish the type of snapshots a chain uses

* Rename supports_warp to snapshot_mode

* Missing import

* Update ethcore/src/snapshot/consensus/mod.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* missing import

* Fix import

* double semi

* Fix merge problem

* cleanup

* Parametrise `ClientIoMessage` with `()` for the light client

* Add impl Tick for ()

* Address review feedback

* Move ClientIoMessage to common-types

* remove superseeded fixme

* fix merge conflict errors
2019-08-28 10:09:42 +02:00
Niklas Adolfsson
dab2a6bd4b fix compile warnings (#10993)
* fix warnings

* fix: failing build, use `spec` as dev-dependency
2019-08-27 17:29:33 +02:00
David
fbf425c4e2 Extract spec to own crate (#10978)
* Add client-traits crate
Move the BlockInfo trait to new crate

* New crate `machine`
Contains code extracted from ethcore that defines `Machine`, `Externalities` and other execution related code.

* Use new machine and client-traits crates in ethcore

* Use new crates machine and client-traits instead of ethcore where appropriate

* Fix tests

* Don't re-export so many types from ethcore::client

* Fixing more fallout from removing re-export

* fix test

* More fallout from not re-exporting types

* Add some docs

* cleanup

* import the macro edition style

* Tweak docs

* Add missing import

* remove unused ethabi_derive imports

* Use latest ethabi-contract

* Move many traits from ethcore/client/traits to client-traits crate
Initial version of extracted Engine trait

* Move snapshot related traits to the engine crate (eew)

* Move a few snapshot related types to common_types
Cleanup Executed as exported from machine crate

* fix warning

* Gradually introduce new engine crate: snapshot

* ethcore typechecks with new engine crate

* Sort out types outside ethcore

* Add an EpochVerifier to ethash and use that in Engine.epoch_verifier()
Cleanup

* Document pub members

* Sort out tests
Sort out default impls for EpochVerifier

* Add test-helpers feature and move EngineSigner impl to the right place

* Sort out tests

* Sort out tests and refactor verification types

* Fix missing traits

* More missing traits
Fix Histogram

* Fix tests and cleanup

* cleanup

* Put back needed logger import

* Don't rexport common_types from ethcore/src/client
Don't export ethcore::client::*

* Remove files no longer used
Use types from the engine crate
Explicit exports from engine::engine

* Get rid of itertools

* Move a few more traits from ethcore to client-traits: BlockChainReset, ScheduleInfo, StateClient

* Move ProvingBlockChainClient to client-traits

* Don't re-export ForkChoice and Transition from ethcore

* Address grumbles: sort imports, remove commented out code

* Fix merge resolution error

* Extract the Clique engine to own crate

* Extract NullEngine and the block_reward module from ethcore

* Extract InstantSeal engine to own crate

* Extract remaining engines

* Extract executive_state to own crate so it can be used by engine crates

* Remove snapshot stuff from the engine crate

* Put snapshot traits back in ethcore

* cleanup

* Remove stuff from ethcore

* Don't use itertools

* itertools in aura is legit-ish

* More post-merge fixes

* Re-export less types in client

* cleanup

* Extract spec to own crate

* Put back the test-helpers from basic-authority

* Fix ethcore benchmarks

* Reduce the public api of ethcore/verification

* Update ethcore/block-reward/Cargo.toml

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update ethcore/engines/basic-authority/Cargo.toml

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update ethcore/engines/ethash/Cargo.toml

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update ethcore/engines/clique/src/lib.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* signers is already a ref

* Add an EngineType enum to tighten up Engine.name()

* Introduce Snapshotting enum to distinguish the type of snapshots a chain uses

* Rename supports_warp to snapshot_mode

* Missing import

* Update ethcore/src/snapshot/consensus/mod.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* missing import

* Fix import

* double semi
2019-08-23 15:32:58 +02:00
David
2af3140a26
Extract engines to own crates (#10966)
* Add client-traits crate
Move the BlockInfo trait to new crate

* New crate `machine`
Contains code extracted from ethcore that defines `Machine`, `Externalities` and other execution related code.

* Use new machine and client-traits crates in ethcore

* Use new crates machine and client-traits instead of ethcore where appropriate

* Fix tests

* Don't re-export so many types from ethcore::client

* Fixing more fallout from removing re-export

* fix test

* More fallout from not re-exporting types

* Add some docs

* cleanup

* import the macro edition style

* Tweak docs

* Add missing import

* remove unused ethabi_derive imports

* Use latest ethabi-contract

* Move many traits from ethcore/client/traits to client-traits crate
Initial version of extracted Engine trait

* Move snapshot related traits to the engine crate (eew)

* Move a few snapshot related types to common_types
Cleanup Executed as exported from machine crate

* fix warning

* Gradually introduce new engine crate: snapshot

* ethcore typechecks with new engine crate

* Sort out types outside ethcore

* Add an EpochVerifier to ethash and use that in Engine.epoch_verifier()
Cleanup

* Document pub members

* Sort out tests
Sort out default impls for EpochVerifier

* Add test-helpers feature and move EngineSigner impl to the right place

* Sort out tests

* Sort out tests and refactor verification types

* Fix missing traits

* More missing traits
Fix Histogram

* Fix tests and cleanup

* cleanup

* Put back needed logger import

* Don't rexport common_types from ethcore/src/client
Don't export ethcore::client::*

* Remove files no longer used
Use types from the engine crate
Explicit exports from engine::engine

* Get rid of itertools

* Move a few more traits from ethcore to client-traits: BlockChainReset, ScheduleInfo, StateClient

* Move ProvingBlockChainClient to client-traits

* Don't re-export ForkChoice and Transition from ethcore

* Address grumbles: sort imports, remove commented out code

* Fix merge resolution error

* Extract the Clique engine to own crate

* Extract NullEngine and the block_reward module from ethcore

* Extract InstantSeal engine to own crate

* Extract remaining engines

* Extract executive_state to own crate so it can be used by engine crates

* Remove snapshot stuff from the engine crate

* Put snapshot traits back in ethcore

* cleanup

* Remove stuff from ethcore

* Don't use itertools

* itertools in aura is legit-ish

* More post-merge fixes

* Re-export less types in client

* cleanup

* Update ethcore/block-reward/Cargo.toml

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update ethcore/engines/basic-authority/Cargo.toml

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update ethcore/engines/ethash/Cargo.toml

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update ethcore/engines/clique/src/lib.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* signers is already a ref

* Add an EngineType enum to tighten up Engine.name()

* Introduce Snapshotting enum to distinguish the type of snapshots a chain uses

* Rename supports_warp to snapshot_mode

* Missing import

* Update ethcore/src/snapshot/consensus/mod.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* remove double-semicolons
2019-08-22 18:25:49 +02:00
Anton Gavrilov
66e4410be7
Private contract migration and offchain state sync (#10748)
* Temp storage for the private state added

* Temp storage for the private state added

* Request message added

* Store and retrieve offchain state logic

* State sync cache

* Private state column added to key value db

* Private state column added to key value db

* Indexing stored states via its hash

* Works with errors changed

* Private state stored into the local db

* Access to private state db added to sync io

* Private state db file added

* Rlp packets for retrieiving private state data added

* Handling of private sync completed message

* Test code fixed

* External flag for offchain storing added

* Test for private state sync added

* Saving private state logic corrected

* Migration code corrected

* Fixes after merge with master

* Merge with head

* Additional checks for slices

* Log for private state retrieval added

* Limit time of retrieving private states

* Store required hashes for every request, mark them stale if needed

* Store requested private state hashes and check received data

* Log stale requests

* State insertion fix

* Refactoring of how logging passed to state store

* Heapsize removed, syncing hashes structure reworked

* Check state length returned by contract

* Get rid of OverlayDB

* hash-db version updated

* Test fixed

* One more test fixed
2019-08-16 14:45:52 +02:00
Marek Kotewicz
45978bc2bd unify loading spec && further spec cleanups (#10948)
* bundle_* macros for spec

* fixed failing tests

* renamed bundle.rs to chain.rs

* removed unused derives

* updated spec/chain.rs description
2019-08-07 16:52:48 +02:00
David
1ef9d5b52f Move more types out of ethcore (#10880)
* WIP move errors, pod_account and state account to own crates

* Sort out dependencies, fix broken code and tests
Remove botched ethcore-error crate

* remove template line

* fix review feedback

* Remove test-only AccountDBMut::new

* Extract AccountDB to account-db

* Move Substate to state-account – wip

* Add lib.rs

* cleanup

* test failure

* test failure 2

* third time's the charm

* Add factories crate

* Use new factories crate

* Use factories crate

* Extract trace

* Fix tests

* Sort out parity-util-mem and parking_lot

* cleanup

* WIP port over the rest of state from ethcore

* Collect all impls for Machine

* some notes

* Rename pod-account to pod

* Move PodState to pod crate

* Use PodState from pod crate

* Fix use clause for json tests

* Sort out evmbin

* Add missing code and use PodState

* Move code that depends on Machine and Executive to own module

* Sort out cloning errors, fix ethcore to use new state crate

* Do without funky From impls

* Fix ethcore tests

* Fixes around the project to use new state crate

* Add back the more specific impls of StateOrBlock From conversions

* Move execute to freestanding function and remove it from trait
Sort out the error handling in executive_state by moving the result types from state to ethcore
Undo the verbose code added to work around the StateOrBlock From conversions

* cleanup

* Fix "error: enum variants on type aliases are experimental"

* Bring back the state tests
Fix whitespace

* remove ethcore/state/mod.rs

* cleanup

* cleanup

* Cleanup state-account errors

* Fix more todos
Add module docs

* Add error.rs

* Fixup Cargo.lock

* Smaller ethcore API is fine

* Add `to-pod-full` feature to state-account
Fix evmbin

* Fix a few more test failures

* Fix RPC test build

* Baptize the new trait

* Remove resolved TODOs

* Rename state-account to account-state

* Do not re-export the trace crate

* Don't export state_db from ethcore

* Let private-tx use StateDB. :(

* Remove ethcore/src/pod_state.rs

* Inner type does not need to be pub/pub(crate)

* optimise imports

* Revert "Inner type does not need to be pub/pub(crate)"

This reverts commit 2f839f8a0f72f71334da64620f57e6dd6039f06b.

* Move DatabaseExtras to ethcore-blockchain

* Add database_extra module to ethcore-blockchain

* Remove to-pod-full feature

* cosmetics

* New crate: state-db

* Add new crate

* Move PreverifiedBlock and BlockError to types

* Sort out the merge

* Add missing `license` meta data keys

* wip

* wip client-traits

* merge conflict

* verification crate type checks

* Move impls for CommonParams to common_types
Fix misc stuff in ethcore

* Fix tests

* Implement VerifyingEngine for all engines except Ethash
Temporarily sort out error handling
Move more types to common_types

* Split Engine in two and move code around

* cleanup

* verification: don't rexport common_types

* Use error from common_types

* Consolidate error types

* VerifyingEngine use Errors from common_types

* verification: Use error type from common_types

* SnapshotError moved to common_types

* Move more code from Engne to VerifyingEngine
Add a VerifyingClient trait: BlockInfo + CallContract
Whitespace

* Add MAX_UNCLE_AGE const

* Port over remaining code from ethcore/verification

* Use errors from common_types

* Fix the confusing "io" naming

* Move more types into common_types

* Add todos

* Experiment with Engine trait outside ethcore

* Hook up types from common_types in ethcore
Don't use verification crate
Don't use client-traits crate

* Revert to impl Engine for Arc<Ethash> and add note to explain why
Revert moving ClientIoMessage to common_types
Fix build

* Remove ClientIoMessage from common_types

* Cleanup

* More cleanup

* Sort error handling changes in the rest of parity

* Remove unused code

* Remove WIP types

* Cleanup todos not tackled here

* remove cruft

* Fix some whitespace and a merge error

* ethcore tests

* test failures

* Restore Engine impls to master to make review a bit easier

* cleanup

* whitespace

* applied review suggestions

* types does not depend on rustc-hex

* ethash engine moved to engine module

* applied review suggestion
2019-07-18 12:27:08 +02:00
David
0f337171bf Extract state-db from ethcore (#10858)
* WIP move errors, pod_account and state account to own crates

* Sort out dependencies, fix broken code and tests
Remove botched ethcore-error crate

* remove template line

* fix review feedback

* Remove test-only AccountDBMut::new

* Extract AccountDB to account-db

* Move Substate to state-account – wip

* Add lib.rs

* cleanup

* test failure

* test failure 2

* third time's the charm

* Add factories crate

* Use new factories crate

* Use factories crate

* Extract trace

* Fix tests

* Sort out parity-util-mem and parking_lot

* cleanup

* WIP port over the rest of state from ethcore

* Collect all impls for Machine

* some notes

* Rename pod-account to pod

* Move PodState to pod crate

* Use PodState from pod crate

* Fix use clause for json tests

* Sort out evmbin

* Add missing code and use PodState

* Move code that depends on Machine and Executive to own module

* Sort out cloning errors, fix ethcore to use new state crate

* Do without funky From impls

* Fix ethcore tests

* Fixes around the project to use new state crate

* Add back the more specific impls of StateOrBlock From conversions

* Move execute to freestanding function and remove it from trait
Sort out the error handling in executive_state by moving the result types from state to ethcore
Undo the verbose code added to work around the StateOrBlock From conversions

* cleanup

* Fix "error: enum variants on type aliases are experimental"

* Bring back the state tests
Fix whitespace

* remove ethcore/state/mod.rs

* cleanup

* cleanup

* Cleanup state-account errors

* Fix more todos
Add module docs

* Add error.rs

* Fixup Cargo.lock

* Smaller ethcore API is fine

* Add `to-pod-full` feature to state-account
Fix evmbin

* Fix a few more test failures

* Fix RPC test build

* Baptize the new trait

* Remove resolved TODOs

* Rename state-account to account-state

* Do not re-export the trace crate

* Don't export state_db from ethcore

* Let private-tx use StateDB. :(

* Remove ethcore/src/pod_state.rs

* Inner type does not need to be pub/pub(crate)

* optimise imports

* Revert "Inner type does not need to be pub/pub(crate)"

This reverts commit 2f839f8a0f72f71334da64620f57e6dd6039f06b.

* Move DatabaseExtras to ethcore-blockchain

* Add database_extra module to ethcore-blockchain

* Remove to-pod-full feature

* cosmetics

* New crate: state-db

* Add new crate

* Sort out the merge

* Fix unclean merge

* Add license
2019-07-08 22:01:47 +01:00
Marek Kotewicz
1887080990 updated project to ansi_term 0.11 (#10799) 2019-06-27 13:34:40 +02:00
Marek Kotewicz
3a2c173fe1 Removed redundant ethcore-service error type (#10788) 2019-06-26 14:16:58 +02:00
Luke Schoen
6022c47b53 docs: Update Readme with TOC, Contributor Guideline. Update Cargo package descriptions (#10652)
* docs: Update Readme with TOC, Contributor Guide and License sections

* docs: Simplify package descriptions in Readme

* docs: Fix typos

* docs: Update Cargo package configs adding missing descriptions

* fix: Remove machine since was accidental from diff branch. Fix contribting indentation.

* fix: Fix formatting

* remove details snippet since not code and formats badly

* review-fix: Remove duplicate description

* fix dot point formatting

* fix: Add missing description to vm package

* fix: Remove duplicate Code of Conduct and Contributor guidelines

* docs: Update Contributing for consistency with other repos

* docs: Update Readme to link to existing Code of Conduct and Contributor guidelines in .github folder

* refactor: Add missing space

* fix: Fix links to be markdown format

* review-fix: Update CONTRIBUTING.md to link to Parity Ethereum Style Guide

* docs: Fix type in .github/CONTRIBUTING.md

Co-Authored-By: David <dvdplm@gmail.com>

* docs: Fix typo in .github/CONTRIBUTING.md

Co-Authored-By: David <dvdplm@gmail.com>

* docs: Rephrase parity-clib/Cargo.toml

Co-Authored-By: David <dvdplm@gmail.com>

* review-fix: Fix whole paragraph

* review-fix: Add comment to prevent direct pushes to master branch

* review-fix: Change some rules to recommendations

* Update .github/CONTRIBUTING.md

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* review-fix: Replace feature reduction with breaking changes dot point

* review-fix: Rephrase what to do when reviewing PR

* review-fix: Update parity-rpc package description and module rustdocs

* docs: Add missing fullstop

* review-fix: Update rustdoc section to show all packages first as default

* review-fix: Rename Parity Util to Parity Core Libraries

* review-fix: Rename readme too for the C bindings name change

* review-fix: Remove some docs since we do not teach the reader Rust

* review-fix: Wrap Parity Ethereum specific packages in a details section

* review-fix: Separate tools that are in this repo vs those that are not

* review-fix: Add link to ethabi on crates.io

* review-fix: Remove unnecessary extra line talking about pushing to master

* review-fix: Remove useless summary of parts of the rust book documentation section

* docs: Update .github/CONTRIBUTING.md to specify two reviewers required

Co-Authored-By: David <dvdplm@gmail.com>

* docs: Update README.md with simplification

Co-Authored-By: David <dvdplm@gmail.com>

* docs: Update README.md renaming to Test Runner instead of just Run

Co-Authored-By: David <dvdplm@gmail.com>

* review-fix: Remove hardware wallet from docs

* review-fix: Remove veto PR section of contributor doc

* review-fix: Change to test generator title
2019-06-25 08:15:13 +02:00
Wei Tang
02eb046774
Move Engine::register_client to be before other I/O handler registration (#10767) 2019-06-24 15:10:42 +02:00
David
4ee217ba7c Remove unused code (#10762) 2019-06-19 15:08:21 +01:00
David
d2120ded56 Add a way to signal shutdown to snapshotting threads (#10744)
* Add a way to signal shutdown to snapshotting threads

* Pass Progress to fat_rlps() so we can abort from there too.

* Checking for abort in a single spot

* Remove nightly-only weak/strong counts

* fix warning

* Fix tests

* Add dummy impl to abort snapshots

* Add another dummy impl for TestSnapshotService

* Remove debugging code

* Return error instead of the odd Ok(())
Switch to AtomicU64
2019-06-19 10:13:09 +02:00
Andrew Jones
bf55db4c7e Die error_chain, die (#10747)
* Replace error chain for network error

* Fix usages and add manual From impls

* OnDemand Error and remove remaining dependencies

* Die error_chain, die.

* DIE

* Hasta la vista, baby
2019-06-17 08:44:59 +02:00
Andronik Ordian
dae5d75dd6 Upgrade ethereum types (#10670)
* cargo upgrade "ethereum-types" --all --allow-prerelease

* [ethash] fix compilation errors

* [ethkey] fix compilation errors

* [journaldb] fix compilation errors

* [dir] fix compilation errors

* [ethabi] update to 0.7

* wip

* [eip-712] fix compilation errors

* [ethjson] fix compilation errors

* [Cargo.toml] add TODO to remove patches

* [ethstore] fix compilation errors

* use patched keccak-hash with new primitive-types

* wip

* [ethcore-network-devp2p] fix compilation errors

* [vm] fix compilation errors

* [common-types, evm, wasm] fix compilation errors

* [ethcore-db] Require AsRef instead of Deref for keys

* [ethcore-blockchain] fix some compilation errors

* [blooms-db] fix compilation errors

Thanks a lot @dvdplm :)

* we don't need no rlp ethereum feature

* [ethcore] fix some compilation errors

* [parity-ipfs-api] fix compilation error

* [ethcore-light] fix compilation errors

* [Cargo.lock] update parity-common

* [ethcore-private-tx] fix some compilation errors

* wip

* [ethcore-private-tx] fix compilation errors

* [parity-updater] fix compilation errors

* [parity-rpc] fix compilation errors

* [parity-bin] fix other compilation errors

* update to new ethereum-types

* update keccak-hash

* [fastmap] fix compilation in tests

* [blooms-db] fix compilation in tests

* [common-types] fix compilation in tests

* [triehash-ethereum] fix compilation in tests

* [ethkey] fix compilation in tests

* [pwasm-run-test] fix compilation errors

* [wasm] fix compilation errors

* [ethjson] fix compilation in tests

* [eip-712] fix compilation in tests

* [ethcore-blockchain] fix compilation in tests

* [ethstore] fix compilation in tests

* [ethstore-accounts] fix compilation in tests

* [parity-hash-fetch] fix compilation in tests

* [parity-whisper] fix compilation in tests

* [ethcore-miner] fix compilation in tests

* [ethcore-network-devp2p] fix compilation in tests

* [*] upgrade rand to 0.6

* [evm] get rid of num-bigint conversions

* [ethcore] downgrade trie-standardmap and criterion

* [ethcore] fix some warnings

* [ethcore] fix compilation in tests

* [evmbin] fix compilation in tests

* [updater] fix compilation in tests

* [ethash] fix compilation in tests

* [ethcore-secretstore] fix compilation in tests

* [ethcore-sync] fix compilation in tests

* [parity-rpc] fix compilation in tests

* [ethcore] finally fix compilation in tests

FUCK YEAH!!!

* [ethstore] lazy_static is unused

* [ethcore] fix test

* fix up bad merge

* [Cargo.toml] remove unused patches

* [*] replace some git dependencies with crates.io

* [Cargo.toml] remove unused lazy_static

* [*] clean up

* [ethcore] fix transaction_filter_deprecated test

* [private-tx] fix serialization tests

* fix more serialization tests

* [ethkey] fix smoky test

* [rpc] fix tests, please?

* [ethcore] remove commented out code

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* [ethstore] remove unused dev-dependency

* [ethcore] remove resolved TODO

* [*] resolve keccak-hash TODO

* [*] s/Address::default()/Address::zero()

* [rpc] remove Subscribers::new_test

* [rpc] remove EthPubSubClient::new_test

* [ethcore] use trie-standardmap from crates.io

* [dir] fix db_root_path

* [ethcore] simplify snapshot::tests::helpers::fill_storage

* Apply suggestions from code review

Co-Authored-By: David <dvdplm@gmail.com>

* [ethcore-secretstore] resolve TODO in serialization

* [ethcore-network-devp2p] resolve TODO in save_key

* [Cargo.lock] update triehash

* [*] use ethabi from crates.io

* [ethkey] use secp256k1 from master branch

* [Cargo.lock] update eth-secp256k1
2019-06-03 15:36:21 +02:00
Andrew Jones
7d26a82232 private-tx: replace error_chain (#10510)
* Update to vanilla tx pool error

* private-tx: remove error-chain, implement Error, derive Display

* private-tx: replace ErrorKind and bail!

* private-tx: add missing From impls and other compiler errors

* private-tx: use original tx-pool error

* Don't be silly cargo
2019-03-27 14:46:05 +01:00
Andrew Jones
97cb010df8 Silence Error::cause deprecations (#10438) 2019-03-02 13:18:18 +01:00
Tomasz Drwięga
d5c19f8719 Deprecate account management (#10213)
* Extract accounts from ethcore.

* Fix ethcore.

* Get rid of AccountProvider in test_helpers

* Fix rest of the code.

* Re-use EngineSigner, fix tests.

* Simplify EngineSigner to always have an Address.

* Fix RPC tests.

* Add deprecation notice to RPCs.

* Feature to disable accounts.

* extract accounts in RPC

* Run with accounts in tests.

* Fix RPC compilation and tests.

* Fix compilation of the binary.

* Fix compilation of the binary.

* Fix compilation with accounts enabled.

* Fix tests.

* Update submodule.

* Remove android.

* Use derive for Default

* Don't build secretstore by default.

* Add link to issue.

* Refresh Cargo.lock.

* Fix miner tests.

* Update rpc/Cargo.toml

Co-Authored-By: tomusdrw <tomusdrw@users.noreply.github.com>

* Fix private tests.
2019-02-07 14:34:24 +01:00
Anton Gavrilov
45d7c60608 Call private contract methods from another private contract (read-only) (#10086)
* Patch available private contracts during private transaction

* Key acl ABI added

* Work with secret store keys moved to the separate struct

* Private tx test refactored

* Revert "Private tx test refactored"

This reverts commit 476c132d692c7a886bc7b7cd7fe47b3d7692bd63.

* Test for calling private contract from another one added

* Test fixed

* Redundant tab removed

* ACL contract processing fixed, test added

* Merge with head

* Expect replaced with closure
2019-02-07 12:39:04 +01:00
Afri Schoedon
7c335e8764
misc: bump license header to 2019 (#10135)
* misc: bump license header to 2019

* misc: remove_duplicate_empty_lines.sh

* misc: run license header script

* commit cargo lock
2019-01-07 11:33:07 +01:00
Tomasz Drwięga
3650f2d51c Extract blockchain from ethcore (#10114)
* Split blockchain & db from ethcore.

* Clean up blockchain deps.

* Missing docs.

* Fix blockchain tests.

* Make other crates compile.

* Remove some re-exports.

* Remove types re-export from ethcore.

* Remove EVM dependency from transaction.

* Merge ethcore-transaction with common-types.

* Clean-up ethcore deps a bit.

* remove ethcore from cargo.toml

* Update ethcore/blockchain/src/lib.rs

Co-Authored-By: tomusdrw <tomusdrw@users.noreply.github.com>

* Address review comments.

* Update DB comment.

* Add tracking issue to the TODO and fix typo.

* Common naming for common types.

* Update ethcore/db/src/keys.rs

Co-Authored-By: tomusdrw <tomusdrw@users.noreply.github.com>

* Update ethcore/blockchain/src/generator.rs

Co-Authored-By: tomusdrw <tomusdrw@users.noreply.github.com>

* Try to fix beta tests.
2019-01-04 14:05:46 +01:00
Tomasz Drwięga
ff0095ac5e Move a bunch of stuff around (#10101)
* Move devtools.

* Merge stop_guard & rename memzero

* Move price-info to miner.

* Group account management

* Clean up workspace members.

* Move local store closer to miner.

* Move clib examples.

* Move registrar and hash-fetch

* Move rpc_cli/rpc_client

* Move stratum closer to miner.

* Fix naming convention of crates.

* Update Cpp examples path.

* Fix paths for clib-example.

* Fix removing build.
2018-12-28 17:33:49 +08:00
Tomasz Drwięga
0b5bbf6048 Improve block and transaction propagation (#9954)
* Refactor sync to add priority tasks.

* Send priority tasks notifications.

* Propagate blocks, optimize transactions.

* Implement transaction propagation. Use sync_channel.

* Tone down info.

* Prevent deadlock by not waiting forever for sync lock.

* Fix lock order.

* Don't use sync_channel to prevent deadlocks.

* Fix tests.
2018-11-28 11:30:05 +01:00
Nicolas Gotchac
9475a2e474 Keep existing blocks when restoring a Snapshot (#8643)
* Rename db_restore => client

* First step: make it compile!

* Second step: working implementation!

* Refactoring

* Fix tests

* PR Grumbles

* PR Grumbles WIP

* Migrate ancient blocks interating backward

* Early return in block migration if snapshot is aborted

* Remove RwLock getter (PR Grumble I)

* Remove dependency on `Client`: only used Traits

* Add test for recovering aborted snapshot recovery

* Add test for migrating old blocks

* Fix build

* PR Grumble I

* PR Grumble II

* PR Grumble III

* PR Grumble IV

* PR Grumble V

* PR Grumble VI

* Fix one test

* Fix test

* PR Grumble

* PR Grumbles

* PR Grumbles II

* Fix tests

* Release RwLock earlier

* Revert Cargo.lock

* Update _update ancient block_ logic: set local in `commit`

* Update typo in ethcore/src/snapshot/service.rs

Co-Authored-By: ngotchac <ngotchac@gmail.com>
2018-11-18 00:06:34 +01:00
David
72fd1fa58d
Fetch parity-common crates from crates.io (#9410)
* Fetch `parity-common` crates from crates.io

* Add doc tests from `patricia-trie` to `patricia-trie-ethereum`
Fix/update a few deps

* [ethkey] upgrade ethereum-types

* [whisper] update deps

* [network] deps

* [network-devp2p] deps

* [journaldb] deps

* [fastmap] deps

* [miner] deps and test fixes

* [machine] deps

* [json] deps

* [hw] deps

* [ethash] deps

* [registrar] deps

* Update a few more dependencies with new ethabi-*

* [updater] Update deps

* deps

* [ethcore] Update deps

* Use new parity-snappy and parity-rocksdb crates

* Updated submodules

* Use parity-snappy 0.1

* Use kvdb-rocksdb 0.1.2

* Don't use latest ethereum/tests

* Fix merge conflicts errors

* Remove superseeded comment

* Address grumbles: add newlines, add/remove spaces
2018-09-04 20:13:51 +02:00
André Silva
c0eb30b604
use kvdb-* and parity-snappy crates from crates.io (#9441)
* use kvdb-* and parity-snappy crates from crates.io

* update rocksdb-sys and snappy-sys
2018-09-03 17:40:11 +01:00
Anton Gavrilov
1073d56245 Private packets verification and queue refactoring (#8715)
* Verify private transaction before propagating

* Private transactions queue reworked with tx pool queue direct usage

* Styling fixed

* Prevent resending private packets to the sender

* Process signed private transaction packets via io queue

* Test fixed

* Build and test fixed after merge

* Comments after review fixed

* Signed transaction taken from verified

* Fix after merge

* Pool scoring generalized in order to use externally

* Lib refactored according to the review comments

* Ready state refactored

* Redundant bound and copying removed

* Fixed build after the merge

* Forgotten case reworked

* Review comments fixed

* Logging reworked, target added

* Fix after merge
2018-08-29 20:31:04 +08:00
Tomasz Drwięga
346913b7f6 Better logging when mining own transactions. (#9363) 2018-08-17 16:01:32 +01:00
Nick Sanders
e590874a81 Update log -> 0.4, env_logger -> 0.5. (#9294)
* Rename a few types & methods.

* Change `(Log)Builder::format` (closure) arg.
2018-08-10 00:04:10 +01:00
Tomasz Drwięga
5f523f6966 Offload cull to IoWorker. (#9099) 2018-07-13 12:23:57 +02:00
Afri Schoedon
da5de4a6ff db: remove wal disabling / fast-and-loose option. (#8963)
* parity: highlight --fast-and-loose is not recommended.

* parity: remove fast-and-loose option, ref #1765

* db: remove db wal from options, it's always enabled

* db: remove wal from rocksdb helpers

* cli: fix wallet import test

* ethcore: fix client config tests

* parity: proper deprecated handling for fast-and-loose

* ethcore-client: fix config

* parity: mark fast-and-loose removed

* parity: fix test_find_deprecated

* parity: fix type for fast-and-loose flag

* lock file
2018-07-10 17:33:25 +02:00
David
c7f608ec74
Delete crates from parity-ethereum and fetch them from parity-common instead (#9083)
Use crates from parity-common: hashdb, keccak-hash, kvdb, kvdb-memorydb, kvdb-rocksdb, memorydb, parity-bytes, parity-crypto, path, patricia_trie, plain_hasher, rlp, target, test-support, trie-standardmap, triehash
2018-07-10 14:59:19 +02:00
David
4145be863b Bump error-chain and quick_error versions (#8972)
* Update error-chain to 0.12 to silence warnings

* Bump quick_error crate to 1.2.2  to silence linter warnings
2018-06-25 14:20:44 +02:00
Marek Kotewicz
458afcd230 new blooms database (#8712)
* new blooms database

* fixed conflict in Cargo.lock

* removed bloomchain

* cleanup in progress

* all tests passing in trace db with new blooms-db

* added trace_blooms to BlockChainDB interface, fixed db flushing

* BlockChainDB no longer exposes RwLock in the interface

* automatically flush blooms-db after every insert

* blooms-db uses io::BufReader to read files, wrap blooms-db into Mutex, cause fs::File is just a shared file handle

* fix json_tests

* blooms-db can filter multiple possibilities at the same time

* removed enum trace/db.rs CacheId

* lint fixes

* fixed tests

* kvdb-rocksdb uses fs-swap crate

* update Cargo.lock

* use fs::rename

* fixed failing test on linux

* fix tests

* use fs_swap

* fixed failing test on linux

* cleanup after swap

* fix tests

* fixed osx permissions

* simplify parity database opening functions

* added migration to blooms-db

* address @niklasad1 grumbles

* fix license and authors field of blooms-db Cargo.toml

* restore blooms-db after snapshot
2018-06-20 15:13:07 +02:00
Wei Tang
09ee6e1477 Fix subcrate test compile (#8862)
* Fix test compile for light, node_filter, service in ethcore

* Fix ipfs, local-store, rpc, secret_store, updater subcrate test compile
2018-06-11 12:26:49 +02:00
Niklas Adolfsson
98b7c07171 Update license header and scripts (#8666)
* Update `add_license` script

* run script

* add `remove duplicate lines script` and run it

* Revert changes `English spaces`

* strip whitespaces

* Revert `GPL` in files with `apache/mit license`

* don't append `gpl license` in files with other lic

* Don't append `gpl header` in files with other lic.

* re-ran script

* include c and cpp files too

* remove duplicate header

* rebase nit
2018-06-04 10:19:50 +02:00
Nicolas Gotchac
7fcb082cad Shutdown the Snapshot Service early (#8658)
* Shutdown the Snapshot Service when shutting down the runner

* Rename `service` to `client_service`

* Fix tests
2018-05-29 13:23:15 +03:00
Tomasz Drwięga
24838bbcd3 Don't block sync when importing old blocks (#8530)
* Alter IO queueing.

* Don't require IoMessages to be Clone

* Ancient blocks imported via IoChannel.

* Get rid of private transactions io message.

* Get rid of deadlock and fix disconnected handler.

* Revert to old disconnect condition.

* Fix tests.

* Fix deadlock.
2018-05-09 08:49:34 +02:00
Pierre Krieger
fac356c701 More code refactoring to integrate Duration (#8322)
* More code refactoring to integrate Duration

* Fix typo

* Fix tests

* More test fix
2018-04-14 21:35:58 +02:00
Tomasz Drwięga
1cd93e4ceb New Transaction Queue implementation (#8074)
* Implementation of Verifier, Scoring and Ready.

* Queue in progress.

* TransactionPool.

* Prepare for txpool release.

* Miner refactor [WiP]

* WiP reworking miner.

* Make it compile.

* Add some docs.

* Split blockchain access to a separate file.

* Work on miner API.

* Fix ethcore tests.

* Refactor miner interface for sealing/work packages.

* Implement next nonce.

* RPC compiles.

* Implement couple of missing methdods for RPC.

* Add transaction queue listeners.

* Compiles!

* Clean-up and parallelize.

* Get rid of RefCell in header.

* Revert "Get rid of RefCell in header."

This reverts commit 0f2424c9b7319a786e1565ea2a8a6d801a21b4fb.

* Override Sync requirement.

* Fix status display.

* Unify logging.

* Extract some cheap checks.

* Measurements and optimizations.

* Fix scoring bug, heap size of bug and add cache

* Disable tx queueing and parallel verification.

* Make ethcore and ethcore-miner compile again.

* Make RPC compile again.

* Bunch of txpool tests.

* Migrate transaction queue tests.

* Nonce Cap

* Nonce cap cache and tests.

* Remove stale future transactions from the queue.

* Optimize scoring and write some tests.

* Simple penalization.

* Clean up and support for different scoring algorithms.

* Add CLI parameters for the new queue.

* Remove banning queue.

* Disable debug build.

* Change per_sender limit to be 1% instead of 5%

* Avoid cloning when propagating transactions.

* Remove old todo.

* Post-review fixes.

* Fix miner options default.

* Implement back ready transactions for light client.

* Get rid of from_pending_block

* Pass rejection reason.

* Add more details to drop.

* Rollback heap size of.

* Avoid cloning hashes when propagating and include more details on rejection.

* Fix tests.

* Introduce nonces cache.

* Remove uneccessary hashes allocation.

* Lower the mem limit.

* Re-enable parallel verification.

* Add miner log. Don't check the type if not below min_gas_price.

* Add more traces, fix disabling miner.

* Fix creating pending blocks twice on AuRa authorities.

* Fix tests.

* re-use pending blocks in AuRa

* Use reseal_min_period to prevent too frequent update_sealing.

* Fix log to contain hash not sender.

* Optimize local transactions.

* Fix aura tests.

* Update locks comments.

* Get rid of unsafe Sync impl.

* Review fixes.

* Remove excessive matches.

* Fix compilation errors.

* Use new pool in private transactions.

* Fix private-tx test.

* Fix secret store tests.

* Actually use gas_floor_target

* Fix config tests.

* Fix pool tests.

* Address grumbles.
2018-04-13 17:34:27 +02:00
Marek Kotewicz
86446d713a ethcore-sync (#8347) 2018-04-10 12:13:49 +02:00
Anton Gavrilov
e6f75bccfe Private transactions integration pr (#6422)
* Private transaction message added

* Empty line removed

* Private transactions logic removed from client into the separate module

* Fixed compilation after merge with head

* Signed private transaction message added as well

* Comments after the review fixed

* Private tx execution

* Test update

* Renamed some methods

* Fixed some tests

* Reverted submodules

* Fixed build

* Private transaction message added

* Empty line removed

* Private transactions logic removed from client into the separate module

* Fixed compilation after merge with head

* Signed private transaction message added as well

* Comments after the review fixed

* Encrypted private transaction message and signed reply added

* Private tx execution

* Test update

* Main scenario completed

* Merged with the latest head

* Private transactions API

* Comments after review fixed

* Parameters for private transactions added to parity arguments

* New files added

* New API methods added

* Do not process packets from unconfirmed peers

* Merge with ptm_ss branch

* Encryption and permissioning with key server added

* Fixed compilation after merge

* Version of Parity protocol incremented in order to support private transactions

* Doc strings for constants added

* Proper format for doc string added

* fixed some encryptor.rs grumbles

* Private transactions functionality moved to the separate crate

* Refactoring in order to remove late initialisation

* Tests fixed after moving to the separate crate

* Fetch method removed

* Sync test helpers refactored

* Interaction with encryptor refactored

* Contract address retrieving via substate removed

* Sensible gas limit for private transactions implemented

* New private contract with nonces added

* Parsing of the response from key server fixed

* Build fixed after the merge, native contracts removed

* Crate renamed

* Tests moved to the separate directory

* Handling of errors reworked in order to use error chain

* Encodable macro added, new constructor replaced with default

* Native ethabi usage removed

* Couple conversions optimized

* Interactions with client reworked

* Errors omitting removed

* Fix after merge

* Fix after the merge

* private transactions improvements in progress

* private_transactions -> ethcore/private-tx

* making private transactions more idiomatic

* private-tx encryptor uses shared FetchClient and is more idiomatic

* removed redundant tests, moved integration tests to tests/ dir

* fixed failing service test

* reenable add_notify on private tx provider

* removed private_tx tests from sync module

* removed commented out code

* Use plain password instead of unlocking account manager

* remove dead code

* Link to the contract changed

* Transaction signature chain replay protection module created

* Redundant type conversion removed

* Contract address returned by private provider

* Test fixed

* Addressing grumbles in PrivateTransactions (#8249)

* Tiny fixes part 1.

* A bunch of additional comments and todos.

* Fix ethsync tests.

* resolved merge conflicts

* final private tx pr (#8318)

* added cli option that enables private transactions

* fixed failing test

* fixed failing test

* fixed failing test

* fixed failing test
2018-04-09 16:14:33 +02:00
Wei Tang
c039ab79b5 Decouple rocksdb dependency from ethcore (#8320)
* Move client DB opening logic to CLI

* Move restoration db open logic to CLI

This adds KeyValueDBHandler which handles opening a new database, thus allow us to move the restoration db open logic
out of ethcore.

* Move rocksdb's compactionprofile conversion to CLI

* Move kvdb_rocksdb as test dependency for ethcore

* Fix tests due to interface change

* Fix service tests

* Remove unused migration dep for ethcore
2018-04-09 14:21:37 +02:00
Marek Kotewicz
de72643898
ethcore-service (#8089)
* ethcore test::helpers cleanup

* ethcore-service
2018-03-13 11:49:57 +01:00