grassrootseconomics/openethereum
8
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

New Transaction Queue implementation (#8074)

Browse Source 
* Implementation of Verifier, Scoring and Ready.

* Queue in progress.

* TransactionPool.

* Prepare for txpool release.

* Miner refactor [WiP]

* WiP reworking miner.

* Make it compile.

* Add some docs.

* Split blockchain access to a separate file.

* Work on miner API.

* Fix ethcore tests.

* Refactor miner interface for sealing/work packages.

* Implement next nonce.

* RPC compiles.

* Implement couple of missing methdods for RPC.

* Add transaction queue listeners.

* Compiles!

* Clean-up and parallelize.

* Get rid of RefCell in header.

* Revert "Get rid of RefCell in header."

This reverts commit 0f2424c9b7319a786e1565ea2a8a6d801a21b4fb.

* Override Sync requirement.

* Fix status display.

* Unify logging.

* Extract some cheap checks.

* Measurements and optimizations.

* Fix scoring bug, heap size of bug and add cache

* Disable tx queueing and parallel verification.

* Make ethcore and ethcore-miner compile again.

* Make RPC compile again.

* Bunch of txpool tests.

* Migrate transaction queue tests.

* Nonce Cap

* Nonce cap cache and tests.

* Remove stale future transactions from the queue.

* Optimize scoring and write some tests.

* Simple penalization.

* Clean up and support for different scoring algorithms.

* Add CLI parameters for the new queue.

* Remove banning queue.

* Disable debug build.

* Change per_sender limit to be 1% instead of 5%

* Avoid cloning when propagating transactions.

* Remove old todo.

* Post-review fixes.

* Fix miner options default.

* Implement back ready transactions for light client.

* Get rid of from_pending_block

* Pass rejection reason.

* Add more details to drop.

* Rollback heap size of.

* Avoid cloning hashes when propagating and include more details on rejection.

* Fix tests.

* Introduce nonces cache.

* Remove uneccessary hashes allocation.

* Lower the mem limit.

* Re-enable parallel verification.

* Add miner log. Don't check the type if not below min_gas_price.

* Add more traces, fix disabling miner.

* Fix creating pending blocks twice on AuRa authorities.

* Fix tests.

* re-use pending blocks in AuRa

* Use reseal_min_period to prevent too frequent update_sealing.

* Fix log to contain hash not sender.

* Optimize local transactions.

* Fix aura tests.

* Update locks comments.

* Get rid of unsafe Sync impl.

* Review fixes.

* Remove excessive matches.

* Fix compilation errors.

* Use new pool in private transactions.

* Fix private-tx test.

* Fix secret store tests.

* Actually use gas_floor_target

* Fix config tests.

* Fix pool tests.

* Address grumbles.
This commit is contained in:
Tomasz Drwięga
2018-04-13 17:34:27 +02:00
committed by Marek Kotewicz
parent 03b96a7c0a
commit 1cd93e4ceb
105 changed files with 5185 additions and 5784 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
ethcore/src/engines/authority_round/mod.rs
View File

@@ -38,7 +38,7 @@ use super::validator_set::{ValidatorSet, SimpleList, new_validator_set};
use self::finality::RollingFinality;
use ethkey::{public_to_address, recover, verify_address, Signature};
use ethkey::{self, Signature};
use io::{IoContext, IoHandler, TimerToken, IoService};
use itertools::{self, Itertools};
use rlp::{encode, Decodable, DecoderError, Encodable, RlpStream, UntrustedRlp};
@@ -292,14 +292,14 @@ impl EmptyStep {
let message = keccak(empty_step_rlp(self.step, &self.parent_hash));
let correct_proposer = step_proposer(validators, &self.parent_hash, self.step);
verify_address(&correct_proposer, &self.signature.into(), &message)
ethkey::verify_address(&correct_proposer, &self.signature.into(), &message)
.map_err(|e| e.into())
}
fn author(&self) -> Result<Address, Error> {
let message = keccak(empty_step_rlp(self.step, &self.parent_hash));
let public = recover(&self.signature.into(), &message)?;
Ok(public_to_address(&public))
let public = ethkey::recover(&self.signature.into(), &message)?;
Ok(ethkey::public_to_address(&public))
}
fn sealed(&self) -> SealedEmptyStep {
@@ -555,7 +555,7 @@ fn verify_external(header: &Header, validators: &ValidatorSet, empty_steps_trans
};
let header_seal_hash = header_seal_hash(header, empty_steps_rlp);
!verify_address(&correct_proposer, &proposer_signature, &header_seal_hash)?
!ethkey::verify_address(&correct_proposer, &proposer_signature, &header_seal_hash)?
};
if is_invalid_proposer {
@@ -824,7 +824,10 @@ impl Engine<EthereumMachine> for AuthorityRound {
fn generate_seal(&self, block: &ExecutedBlock, parent: &Header) -> Seal {
// first check to avoid generating signature most of the time
// (but there's still a race to the `compare_and_swap`)
if !self.can_propose.load(AtomicOrdering::SeqCst) { return Seal::None; }
if !self.can_propose.load(AtomicOrdering::SeqCst) {
trace!(target: "engine", "Aborting seal generation. Can't propose.");
return Seal::None;
}
let header = block.header();
let parent_step: U256 = header_step(parent, self.empty_steps_transition)
@@ -1305,7 +1308,7 @@ impl Engine<EthereumMachine> for AuthorityRound {
}
fn sign(&self, hash: H256) -> Result<Signature, Error> {
self.signer.read().sign(hash).map_err(Into::into)
Ok(self.signer.read().sign(hash)?)
}
fn snapshot_components(&self) -> Option<Box<::snapshot::SnapshotComponents>> {

6
ethcore/src/engines/basic_authority.rs
View File

@@ -19,7 +19,7 @@
use std::sync::{Weak, Arc};
use ethereum_types::{H256, H520, Address};
use parking_lot::RwLock;
use ethkey::{recover, public_to_address, Signature};
use ethkey::{self, Signature};
use account_provider::AccountProvider;
use block::*;
use engines::{Engine, Seal, ConstructedVerifier, EngineError};
@@ -61,7 +61,7 @@ fn verify_external(header: &Header, validators: &ValidatorSet) -> Result<(), Err
// Check if the signature belongs to a validator, can depend on parent state.
let sig = UntrustedRlp::new(&header.seal()[0]).as_val::<H520>()?;
let signer = public_to_address(&recover(&sig.into(), &header.bare_hash())?);
let signer = ethkey::public_to_address(&ethkey::recover(&sig.into(), &header.bare_hash())?);
if *header.author() != signer {
return Err(EngineError::NotAuthorized(*header.author()).into())
@@ -185,7 +185,7 @@ impl Engine<EthereumMachine> for BasicAuthority {
}
fn sign(&self, hash: H256) -> Result<Signature, Error> {
self.signer.read().sign(hash).map_err(Into::into)
Ok(self.signer.read().sign(hash)?)
}
fn snapshot_components(&self) -> Option<Box<::snapshot::SnapshotComponents>> {

26
ethcore/src/engines/mod.rs
View File

@@ -48,7 +48,7 @@ use error::Error;
use header::{Header, BlockNumber};
use snapshot::SnapshotComponents;
use spec::CommonParams;
use transaction::{UnverifiedTransaction, SignedTransaction};
use transaction::{self, UnverifiedTransaction, SignedTransaction};
use ethkey::Signature;
use parity_machine::{Machine, LocalizedMachine as Localized};
@@ -387,14 +387,28 @@ pub trait EthEngine: Engine<::machine::EthereumMachine> {
}
/// Verify a particular transaction is valid.
fn verify_transaction_unordered(&self, t: UnverifiedTransaction, header: &Header) -> Result<SignedTransaction, Error> {
///
/// Unordered verification doesn't rely on the transaction execution order,
/// i.e. it should only verify stuff that doesn't assume any previous transactions
/// has already been verified and executed.
///
/// NOTE This function consumes an `UnverifiedTransaction` and produces `SignedTransaction`
/// which implies that a heavy check of the signature is performed here.
fn verify_transaction_unordered(&self, t: UnverifiedTransaction, header: &Header) -> Result<SignedTransaction, transaction::Error> {
self.machine().verify_transaction_unordered(t, header)
}
/// Additional verification for transactions in blocks.
// TODO: Add flags for which bits of the transaction to check.
// TODO: consider including State in the params.
fn verify_transaction_basic(&self, t: &UnverifiedTransaction, header: &Header) -> Result<(), Error> {
/// Perform basic/cheap transaction verification.
///
/// This should include all cheap checks that can be done before
/// actually checking the signature, like chain-replay protection.
///
/// NOTE This is done before the signature is recovered so avoid
/// doing any state-touching checks that might be expensive.
///
/// TODO: Add flags for which bits of the transaction to check.
/// TODO: consider including State in the params.
fn verify_transaction_basic(&self, t: &UnverifiedTransaction, header: &Header) -> Result<(), transaction::Error> {
self.machine().verify_transaction_basic(t, header)
}

14
ethcore/src/engines/tendermint/mod.rs
View File

@@ -37,7 +37,7 @@ use bytes::Bytes;
use error::{Error, BlockError};
use header::{Header, BlockNumber};
use rlp::UntrustedRlp;
use ethkey::{Message, public_to_address, recover, Signature};
use ethkey::{self, Message, Signature};
use account_provider::AccountProvider;
use block::*;
use engines::{Engine, Seal, EngineError, ConstructedVerifier};
@@ -518,8 +518,8 @@ impl Engine<EthereumMachine> for Tendermint {
let message: ConsensusMessage = rlp.as_val().map_err(fmt_err)?;
if !self.votes.is_old_or_known(&message) {
let msg_hash = keccak(rlp.at(1).map_err(fmt_err)?.as_raw());
let sender = public_to_address(
&recover(&message.signature.into(), &msg_hash).map_err(fmt_err)?
let sender = ethkey::public_to_address(
&ethkey::recover(&message.signature.into(), &msg_hash).map_err(fmt_err)?
);
if !self.is_authority(&sender) {
@@ -614,7 +614,7 @@ impl Engine<EthereumMachine> for Tendermint {
};
let address = match self.votes.get(&precommit) {
Some(a) => a,
None => public_to_address(&recover(&precommit.signature.into(), &precommit_hash)?),
None => ethkey::public_to_address(&ethkey::recover(&precommit.signature.into(), &precommit_hash)?),
};
if !self.validators.contains(header.parent_hash(), &address) {
return Err(EngineError::NotAuthorized(address.to_owned()).into());
@@ -669,7 +669,7 @@ impl Engine<EthereumMachine> for Tendermint {
let verifier = Box::new(EpochVerifier {
subchain_validators: list,
recover: |signature: &Signature, message: &Message| {
Ok(public_to_address(&::ethkey::recover(&signature, &message)?))
Ok(ethkey::public_to_address(&ethkey::recover(&signature, &message)?))
},
});
@@ -690,7 +690,7 @@ impl Engine<EthereumMachine> for Tendermint {
}
fn sign(&self, hash: H256) -> Result<Signature, Error> {
self.signer.read().sign(hash).map_err(Into::into)
Ok(self.signer.read().sign(hash)?)
}
fn snapshot_components(&self) -> Option<Box<::snapshot::SnapshotComponents>> {
@@ -1026,7 +1026,7 @@ mod tests {
let client = generate_dummy_client_with_spec_and_accounts(Spec::new_test_tendermint, Some(tap.clone()));
let engine = client.engine();
client.miner().set_engine_signer(v1.clone(), "1".into()).unwrap();
client.miner().set_author(v1.clone(), Some("1".into())).unwrap();
let notify = Arc::new(TestNotify::default());
client.add_notify(notify.clone());

4
ethcore/src/engines/validator_set/contract.rs
View File

@@ -169,8 +169,8 @@ mod tests {
let validator_contract = "0000000000000000000000000000000000000005".parse::<Address>().unwrap();
// Make sure reporting can be done.
client.miner().set_gas_floor_target(1_000_000.into());
client.miner().set_engine_signer(v1, "".into()).unwrap();
client.miner().set_gas_range_target((1_000_000.into(), 1_000_000.into()));
client.miner().set_author(v1, Some("".into())).unwrap();
// Check a block that is a bit in future, reject it but don't report the validator.
let mut header = Header::default();

8
ethcore/src/engines/validator_set/multi.rs
View File

@@ -171,22 +171,22 @@ mod tests {
client.engine().register_client(Arc::downgrade(&client) as _);
// Make sure txs go through.
client.miner().set_gas_floor_target(1_000_000.into());
client.miner().set_gas_range_target((1_000_000.into(), 1_000_000.into()));
// Wrong signer for the first block.
client.miner().set_engine_signer(v1, "".into()).unwrap();
client.miner().set_author(v1, Some("".into())).unwrap();
client.transact_contract(Default::default(), Default::default()).unwrap();
::client::EngineClient::update_sealing(&*client);
assert_eq!(client.chain_info().best_block_number, 0);
// Right signer for the first block.
client.miner().set_engine_signer(v0, "".into()).unwrap();
client.miner().set_author(v0, Some("".into())).unwrap();
::client::EngineClient::update_sealing(&*client);
assert_eq!(client.chain_info().best_block_number, 1);
// This time v0 is wrong.
client.transact_contract(Default::default(), Default::default()).unwrap();
::client::EngineClient::update_sealing(&*client);
assert_eq!(client.chain_info().best_block_number, 1);
client.miner().set_engine_signer(v1, "".into()).unwrap();
client.miner().set_author(v1, Some("".into())).unwrap();
::client::EngineClient::update_sealing(&*client);
assert_eq!(client.chain_info().best_block_number, 2);
// v1 is still good.

6
ethcore/src/engines/validator_set/safe_contract.rs
View File

@@ -484,7 +484,7 @@ mod tests {
client.engine().register_client(Arc::downgrade(&client) as _);
let validator_contract = "0000000000000000000000000000000000000005".parse::<Address>().unwrap();
client.miner().set_engine_signer(v1, "".into()).unwrap();
client.miner().set_author(v1, Some("".into())).unwrap();
// Remove "1" validator.
let tx = Transaction {
nonce: 0.into(),
@@ -512,11 +512,11 @@ mod tests {
assert_eq!(client.chain_info().best_block_number, 1);
// Switch to the validator that is still there.
client.miner().set_engine_signer(v0, "".into()).unwrap();
client.miner().set_author(v0, Some("".into())).unwrap();
::client::EngineClient::update_sealing(&*client);
assert_eq!(client.chain_info().best_block_number, 2);
// Switch back to the added validator, since the state is updated.
client.miner().set_engine_signer(v1, "".into()).unwrap();
client.miner().set_author(v1, Some("".into())).unwrap();
let tx = Transaction {
nonce: 2.into(),
gas_price: 0.into(),