Commit Graph

62 Commits

Author SHA1 Message Date
Wei Tang 897a94641e Unify and limit rocksdb dependency places (#8371)
* secret_store: remove kvdb_rocksdb dependency

* cli: init db mod for open dispatch

* cli: move db, client_db, restoration_db, secretstore_db to a separate mod

* migration: rename to migration-rocksdb and remove ethcore-migrations

* ethcore: re-move kvdb-rocksdb dep to test

* mark test_helpers as test only and fix migration mod naming

* Move restoration_db_handler to test_helpers_internal

* Fix missing preambles in test_helpers_internal and rocksdb/helpers

* Move test crates downward

* Fix missing docs

* cli, db::open_db: move each argument to a separate line

* Use featuregate instead of dead code for `open_secretstore_db`

* Move pathbuf import to open_secretstore_db

Because it's only used there behind a feature gate
2018-04-13 21:14:53 +02:00
Tomasz Drwięga 1cd93e4ceb New Transaction Queue implementation (#8074)
* Implementation of Verifier, Scoring and Ready.

* Queue in progress.

* TransactionPool.

* Prepare for txpool release.

* Miner refactor [WiP]

* WiP reworking miner.

* Make it compile.

* Add some docs.

* Split blockchain access to a separate file.

* Work on miner API.

* Fix ethcore tests.

* Refactor miner interface for sealing/work packages.

* Implement next nonce.

* RPC compiles.

* Implement couple of missing methdods for RPC.

* Add transaction queue listeners.

* Compiles!

* Clean-up and parallelize.

* Get rid of RefCell in header.

* Revert "Get rid of RefCell in header."

This reverts commit 0f2424c9b7319a786e1565ea2a8a6d801a21b4fb.

* Override Sync requirement.

* Fix status display.

* Unify logging.

* Extract some cheap checks.

* Measurements and optimizations.

* Fix scoring bug, heap size of bug and add cache

* Disable tx queueing and parallel verification.

* Make ethcore and ethcore-miner compile again.

* Make RPC compile again.

* Bunch of txpool tests.

* Migrate transaction queue tests.

* Nonce Cap

* Nonce cap cache and tests.

* Remove stale future transactions from the queue.

* Optimize scoring and write some tests.

* Simple penalization.

* Clean up and support for different scoring algorithms.

* Add CLI parameters for the new queue.

* Remove banning queue.

* Disable debug build.

* Change per_sender limit to be 1% instead of 5%

* Avoid cloning when propagating transactions.

* Remove old todo.

* Post-review fixes.

* Fix miner options default.

* Implement back ready transactions for light client.

* Get rid of from_pending_block

* Pass rejection reason.

* Add more details to drop.

* Rollback heap size of.

* Avoid cloning hashes when propagating and include more details on rejection.

* Fix tests.

* Introduce nonces cache.

* Remove uneccessary hashes allocation.

* Lower the mem limit.

* Re-enable parallel verification.

* Add miner log. Don't check the type if not below min_gas_price.

* Add more traces, fix disabling miner.

* Fix creating pending blocks twice on AuRa authorities.

* Fix tests.

* re-use pending blocks in AuRa

* Use reseal_min_period to prevent too frequent update_sealing.

* Fix log to contain hash not sender.

* Optimize local transactions.

* Fix aura tests.

* Update locks comments.

* Get rid of unsafe Sync impl.

* Review fixes.

* Remove excessive matches.

* Fix compilation errors.

* Use new pool in private transactions.

* Fix private-tx test.

* Fix secret store tests.

* Actually use gas_floor_target

* Fix config tests.

* Fix pool tests.

* Address grumbles.
2018-04-13 17:34:27 +02:00
Marek Kotewicz 86446d713a ethcore-sync (#8347) 2018-04-10 12:13:49 +02:00
Wei Tang c039ab79b5 Decouple rocksdb dependency from ethcore (#8320)
* Move client DB opening logic to CLI

* Move restoration db open logic to CLI

This adds KeyValueDBHandler which handles opening a new database, thus allow us to move the restoration db open logic
out of ethcore.

* Move rocksdb's compactionprofile conversion to CLI

* Move kvdb_rocksdb as test dependency for ethcore

* Fix tests due to interface change

* Fix service tests

* Remove unused migration dep for ethcore
2018-04-09 14:21:37 +02:00
Marek Kotewicz ee93be80c0
removed old migrations (#7974)
* removed old migrations

* improve SimpleMigration

* fixed migration tests

* fixed redundant whitespace

* add ToV13 migration which removes bloom groups

* bump CURRENT_VERSION of db
2018-02-22 14:53:10 +01:00
Marek Kotewicz f43e355254
Replace RandomTempDir with tempdir::Tempdir (#7624)
* Remove GuardedTempResult

* Remove RandomTempPath

* ethcore does not depend on devtools
2018-01-19 17:32:53 +01:00
Tomasz Drwięga f044b61f42 Extract some parts of miner from ethcore. (#7353)
* Move miner away from ethcore.

* Fix ethcore to use miner/transaction.

* Fix tests and warnings.

* fixed incorrect merge of the test in the documentation
2018-01-11 17:49:10 +01:00
Marek Kotewicz e95b093483 dissolve util (#7460)
* ethereum-types refactor in progress

* ethereum-types refactor in progress

* ethereum-types refactor in progress

* ethereum-types refactor in progress

* ethereum-types refactor finished

* removed obsolete util/src/lib.rs

* removed commented out code
2018-01-10 15:35:18 +03:00
André Silva e114b0b28d Upgrade to RocksDB 5.8.8 and tune settings to reduce space amplification (#7348)
* kvdb-rocksdb: update to RocksDB 5.8.8

* kvdb-rocksdb: tune RocksDB options

* Switch to level-style compaction
* Increase default block size (16K), and use bigger blocks for HDDs (64K)
* Increase default file size base (64MB SSDs, 256MB HDDs)
* Create a single block cache shared across all column families
* Tune compaction settings using RocksDB helper functions, taking into account
  memory budget spread across all columns
* Configure backgrounds jobs based on the number of CPUs
* Set some default recommended settings

* ethcore: remove unused config blockchain.db_cache_size

* parity: increase default value for db_cache_size

* kvdb-rocksdb: enable compression on all levels

* kvdb-rocksdb: set global db_write_bufer_size

* kvdb-rocksdb: reduce db_write_bufer_size to force earlier flushing

* kvdb-rocksdb: use master branch for rust-rocksdb dependency
2018-01-03 11:00:37 +01:00
Nicolas Ochem 2e12a2db50 Fix #6209 - introduce standalone dir crate
* created the dir crate in util
* moved code from ethstore/src/dir/paths.rs to dir crate
* rename dir module in ethstore to accounts_dir to distinguish it
  from the dir crate
* changes after @tomusdrw on #6952
2017-12-26 00:54:34 -08:00
Marek Kotewicz d80dd81d77
parity-version pr reopen (#7136)
* parity-version module split from util

removed unused util deps and features

trigger buildbot again

only kvdb links rocksdb

snappy linker issues

* rm snappy

* fixed old version imports
2017-12-22 09:37:39 -04:00
Tomasz Drwięga a5a7c1827a
Some more. 2017-12-21 15:37:10 +01:00
Arkadiy Paronyan dde6baedec
Removed ethcore-util dependency from ethcore-network (#7180)
* Removed ethcore-util dependency

* Removed snappy
2017-12-02 10:08:04 +01:00
debris 3cf52dac59 use error-chain in ethcore-network 2017-11-13 16:48:58 +01:00
Dmitry Kashitsyn 1516fc1c57 Adds validate_node_url() and refactors boot node check (#6907) 2017-11-06 12:46:11 +07:00
Dmitry Kashitsyn f9e588dd7b Refactors parity/parity to use journaldb crate (#6693) 2017-10-17 11:46:54 +07:00
debris eb526b7769 separated kvdb into 3 crates: kvdb, kvdb-memorydb && kvdb-rocksdb, #6693 2017-10-12 15:36:27 +02:00
Marek Kotewicz 6279ff32f5 Separate migrations from util (#6690)
* separate migration from util and make its dependencies into libs:

* snappy
* kvdb
* error
* common

* renamed common -> macros

* util error does not depend on snappy module

* ethsync does not depend on util nor ethcore_error

* nibbleslice and nibblevec merged with patricia_trie crate

* removed unused dependencies from util

* util journaldb traits does not need to be public

* util_error

* fixed ethcore compile error

* ignore .swo files

* Update chain.rs
2017-10-10 20:01:27 +02:00
Fredrik 308f36a532 Don't reexport bigint from util 2017-09-04 16:36:49 +02:00
Joseph Mark b5f1524e78 Refactor --allow-ips to handle custom ip-ranges (#6144)
* Add checks for additional reserved ip addresses

100.64.0.0/10 and 240.0.0.0/4 are both reserved but not currently
filtered.

* Add check for special purpose addresses

192.0.0.0/24 - Used for the IANA IPv4 Special Purpose Address Registry

* Refactor ip_utils (#5872)

* Add checks for all ipv4 special use addresses
* Add comprehensive ipv4 test cases

* Refactor Ipv6 address checks (#5872)

* Refactor AllowIP (#5872)

* Add IpFilter struct to wrap predefined filter (AllowIP) with custom
allow/block filters.
* Refactor parsing of --allow-ips to handle custom filters.
* Move AllowIP/IpFilter from ethsync to ethcore-network where they
are used.

* Revert Cargo.lock

* Tests for custom ip filters (#5872)

* Add "none" as a valid argument for --allow-ips to allow narrow
custom ranges, eg.: --allow-ips="none 10.0.0.0/8"
* Add tests for parsing filter arguments and node endpoints.
* Add ipnetwork crate to dev dependencies for testing.

* Add ipv6 filter tests (#5872)

* Revert parity-ui-precompiled to master

* Fix minor detail in usage.txt (#5872)

* Spaces to tabs

* Rename IpFilter::new() to ::default()

* Small readability improvements

* Test (#5872)

* Revert "Test (#5872)"

This reverts commit 7a8906430a6dad633fe29df3dca57f1630851fa9.
2017-07-28 19:06:39 +02:00
Tomasz Drwięga a24b6ad983 Use standard paths for Ethash cache (#5881)
* Use cache path to store ethash files.

* Fixing tests, more flexible API.

* Use AsRef<Path> everywhere.

* Fixing ethcore tests.

* Fix RPC tests.
2017-07-10 12:57:40 +02:00
Nikolay Volf e1fef5c732 Update dependencies and bigint api (#5685)
* update to latest bigint

* bump elastic array and deps

* fix rlp tests

* also update all smallvec deps

* fix doc test

* reduce parking in attempt to fix CI bug

* fix from/into electum bug

* remove duplicate imports
2017-05-24 12:31:33 +02:00
Tomasz Drwięga f47cbe0be6 Adding CLI options: port shift and unsafe expose. (#5677)
* Adding CLI option for port shift and unsafe expose.

* Fixing IPC path.

* Fix hosts when attempting to expose on all interfaces.

* Fixing test.

* Fix typo.
2017-05-23 12:24:32 +02:00
Marek Kotewicz d530cc86f3 splitting part of util into smaller crates (#4956)
* split path module from util

* moved RotatingLogger from util to logger crate

* fix tests

* fix tests

* use only one version of ansi_term crate
2017-03-22 06:23:40 +01:00
Gav Wood 3041c95408 Chain-selection from UI (#4859)
* First little bits for chain-selection.

* Provide RPCs and get settings through to user defaults.

* Hasty stash.

* Fix updater accidentally redownloading.

* Finish up.

* Add JS tests.

* Hypervisor should never run a binary modified before itself.

* Style.

* Help tweak.

* Fix test compile.

* Fix JS test

* Build fix for tests.

* Revert default chain name

* Another test

* Use spec name via client.

* Fix mock up.

* whitespace

[ci:skip]

* whitespace

[ci:skip]

* remove exit/restart endpoints.
2017-03-13 12:10:53 +01:00
Gav Wood 8404edb656 Fix whitespace (#4299)
* Fix whitespace.

* Update copyright years/owner.

* Push release only for tags.
2017-01-25 18:51:41 +01:00
Robert Habermeier 203fd8a471 Memory-based pruning history size (#4114)
* prune states based on memory param

* pruning memory CLI and usage in sync

* return purged value from memorydb

* calculate memory used incrementally in overlayrecentdb

* refactor shared history pruning code in client

* Fixed usage alignment

* journal_size function for fast memory calculation
2017-01-20 13:25:53 +01:00
Tomasz Drwięga a076ffaf8c Using local path on Windows (#4017)
* Using non-roaming path on Windows

* Fixing test
2017-01-05 13:12:54 +00:00
Robert Habermeier 8125b5690c Port `try` macro to new `?` operator. (#3962)
* initial untry sweep

* restore try in ipc codegen, fix inference

* change a few missed try instances
2016-12-27 12:53:56 +01:00
arkpar e2371632c5 Base dir option 2016-12-15 21:56:45 +01:00
arkpar be5fefe772 Merge branch 'master' of github.com:ethcore/parity into new-dirs 2016-12-14 00:04:11 +01:00
arkpar 6e3e1ea88b XDG paths 2016-12-13 23:38:29 +01:00
arkpar 6601fde328 New data paths with upgrade 2016-12-12 16:51:07 +01:00
Gav Wood 17cc2be4b5
Rename ethcore -> parity in license. 2016-12-11 19:30:54 +01:00
Gav Wood 5f1fcf95e0
Make *ID names consistent with std Rust (Id) 2016-12-09 23:01:43 +01:00
keorn 5e7c21ad4a add a cli flag 2016-12-05 19:23:03 +00:00
Tomasz Drwięga c17ef2c1a4 Fixing parsing passwords from file (#3367)
* fixing parsing passwords as inputs

* Fixing typo in test
2016-11-12 10:06:40 +01:00
Tomasz Drwięga 4bf75d6f2f Remove some unwraps from parity/helpers (#3364)
* Fixing no stdin

* Removing unwrap when reading passwords
2016-11-11 16:53:51 +01:00
Gav Wood 744501c454 Mode improvements for UI (#3109)
* `--mode=off` now works.

* Add Mode::Off as a persistent CLI option.

* "last" not "auto" as default.

* Commit accidentally unsaved file.

* Whitespace

[ci:skip]

* Mode CLI parse fix

* or offline

* Save mode when it gets changed.

* Fix Offline mode

* Fix up mode operations.

* Make passive default, but not overriding.

* Fix test

* Maybe not everyone wants to run an archive node...
2016-11-05 10:38:00 +01:00
Arkadiy Paronyan 1a5bae8ef1 Extended network options (#2845)
* More network configuration options

* Filter UDP requests

* Fixed tests

* Fixed test warning
2016-10-24 18:25:27 +02:00
Arkadiy Paronyan 44a560e964 CLI option to skip seal check when importing (#2842) 2016-10-24 15:09:13 +02:00
Tomasz Drwięga cceca916a1 [frontport] CLI to specify queue ordering strategy (#2494) (#2623)
* CLI to specify queue ordering strategy (#2494)

* Alter gas priorities to include gas_price also

* CLI options and tests

* Adding ordering by gas

* whitespace

Conflicts:
	ethcore/src/miner/miner.rs
	ethcore/src/miner/mod.rs
	ethcore/src/miner/transaction_queue.rs
	parity/cli/usage.txt
	parity/configuration.rs

* fix build
2016-10-15 14:46:33 +02:00
Robert Habermeier f28b8352c1 Configurable history size in master (#2606)
* split journaldb commit into two functions: journal_under and mark_canonical

* use new commit mechanism in client

* Configurable history size in master

* Reduce DB history

* Configurable history size

* Set min history size

* Test

* Fixed a test and reduced the limit
2016-10-14 15:44:56 +03:00
Robert Habermeier 4bcc9e3b49 Merge pull request #2516 from ethcore/canon-cache-size
Mostly configurable canonical cache size
2016-10-12 21:48:29 +02:00
Robert Habermeier ea68546616 Network-specific nodes file (#2569)
* network-specific nodes.json

* save nodes.json periodically

* squash warnings
2016-10-11 18:42:20 +02:00
Arkadiy Paronyan 26d7712d30 TX queue gas limit config and allow local transactions over the gas limit (#2553)
* Gas limit config; Allow local transactions over the limit

* Fix typo

[ci:skip]
2016-10-10 23:04:43 +02:00
Robert Habermeier 299ceb8092 Merge branch 'master' into canon-cache-size 2016-10-07 12:42:51 +02:00
Robert Habermeier 8a0e98d4cc configurable jump table cache size 2016-10-07 12:18:42 +02:00
keorn 5f0ed9ddce Trim password from file (#2503)
* trim password

* indicate trimming in doc
2016-10-07 09:39:16 +02:00
Robert Habermeier 745a50dfdf configurable state cache size 2016-10-07 00:28:42 +02:00