Compare commits
No commits in common. "master" and "lash/sendscript" have entirely different histories.
master
...
lash/sends
1
.gitignore
vendored
1
.gitignore
vendored
@ -6,4 +6,3 @@ build/
|
|||||||
*.egg-info
|
*.egg-info
|
||||||
*.sqlite
|
*.sqlite
|
||||||
.coverage
|
.coverage
|
||||||
.chaind
|
|
||||||
|
|||||||
@ -1,2 +0,0 @@
|
|||||||
- 0.3.2
|
|
||||||
* Change license to AGPL3 and copyright waived to public domain
|
|
||||||
141
LICENSE
141
LICENSE
@ -1,5 +1,5 @@
|
|||||||
GNU AFFERO GENERAL PUBLIC LICENSE
|
GNU GENERAL PUBLIC LICENSE
|
||||||
Version 3, 19 November 2007
|
Version 3, 29 June 2007
|
||||||
|
|
||||||
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
||||||
Everyone is permitted to copy and distribute verbatim copies
|
Everyone is permitted to copy and distribute verbatim copies
|
||||||
@ -7,15 +7,17 @@
|
|||||||
|
|
||||||
Preamble
|
Preamble
|
||||||
|
|
||||||
The GNU Affero General Public License is a free, copyleft license for
|
The GNU General Public License is a free, copyleft license for
|
||||||
software and other kinds of works, specifically designed to ensure
|
software and other kinds of works.
|
||||||
cooperation with the community in the case of network server software.
|
|
||||||
|
|
||||||
The licenses for most software and other practical works are designed
|
The licenses for most software and other practical works are designed
|
||||||
to take away your freedom to share and change the works. By contrast,
|
to take away your freedom to share and change the works. By contrast,
|
||||||
our General Public Licenses are intended to guarantee your freedom to
|
the GNU General Public License is intended to guarantee your freedom to
|
||||||
share and change all versions of a program--to make sure it remains free
|
share and change all versions of a program--to make sure it remains free
|
||||||
software for all its users.
|
software for all its users. We, the Free Software Foundation, use the
|
||||||
|
GNU General Public License for most of our software; it applies also to
|
||||||
|
any other work released this way by its authors. You can apply it to
|
||||||
|
your programs, too.
|
||||||
|
|
||||||
When we speak of free software, we are referring to freedom, not
|
When we speak of free software, we are referring to freedom, not
|
||||||
price. Our General Public Licenses are designed to make sure that you
|
price. Our General Public Licenses are designed to make sure that you
|
||||||
@ -24,34 +26,44 @@ them if you wish), that you receive source code or can get it if you
|
|||||||
want it, that you can change the software or use pieces of it in new
|
want it, that you can change the software or use pieces of it in new
|
||||||
free programs, and that you know you can do these things.
|
free programs, and that you know you can do these things.
|
||||||
|
|
||||||
Developers that use our General Public Licenses protect your rights
|
To protect your rights, we need to prevent others from denying you
|
||||||
with two steps: (1) assert copyright on the software, and (2) offer
|
these rights or asking you to surrender the rights. Therefore, you have
|
||||||
you this License which gives you legal permission to copy, distribute
|
certain responsibilities if you distribute copies of the software, or if
|
||||||
and/or modify the software.
|
you modify it: responsibilities to respect the freedom of others.
|
||||||
|
|
||||||
A secondary benefit of defending all users' freedom is that
|
For example, if you distribute copies of such a program, whether
|
||||||
improvements made in alternate versions of the program, if they
|
gratis or for a fee, you must pass on to the recipients the same
|
||||||
receive widespread use, become available for other developers to
|
freedoms that you received. You must make sure that they, too, receive
|
||||||
incorporate. Many developers of free software are heartened and
|
or can get the source code. And you must show them these terms so they
|
||||||
encouraged by the resulting cooperation. However, in the case of
|
know their rights.
|
||||||
software used on network servers, this result may fail to come about.
|
|
||||||
The GNU General Public License permits making a modified version and
|
|
||||||
letting the public access it on a server without ever releasing its
|
|
||||||
source code to the public.
|
|
||||||
|
|
||||||
The GNU Affero General Public License is designed specifically to
|
Developers that use the GNU GPL protect your rights with two steps:
|
||||||
ensure that, in such cases, the modified source code becomes available
|
(1) assert copyright on the software, and (2) offer you this License
|
||||||
to the community. It requires the operator of a network server to
|
giving you legal permission to copy, distribute and/or modify it.
|
||||||
provide the source code of the modified version running there to the
|
|
||||||
users of that server. Therefore, public use of a modified version, on
|
|
||||||
a publicly accessible server, gives the public access to the source
|
|
||||||
code of the modified version.
|
|
||||||
|
|
||||||
An older license, called the Affero General Public License and
|
For the developers' and authors' protection, the GPL clearly explains
|
||||||
published by Affero, was designed to accomplish similar goals. This is
|
that there is no warranty for this free software. For both users' and
|
||||||
a different license, not a version of the Affero GPL, but Affero has
|
authors' sake, the GPL requires that modified versions be marked as
|
||||||
released a new version of the Affero GPL which permits relicensing under
|
changed, so that their problems will not be attributed erroneously to
|
||||||
this license.
|
authors of previous versions.
|
||||||
|
|
||||||
|
Some devices are designed to deny users access to install or run
|
||||||
|
modified versions of the software inside them, although the manufacturer
|
||||||
|
can do so. This is fundamentally incompatible with the aim of
|
||||||
|
protecting users' freedom to change the software. The systematic
|
||||||
|
pattern of such abuse occurs in the area of products for individuals to
|
||||||
|
use, which is precisely where it is most unacceptable. Therefore, we
|
||||||
|
have designed this version of the GPL to prohibit the practice for those
|
||||||
|
products. If such problems arise substantially in other domains, we
|
||||||
|
stand ready to extend this provision to those domains in future versions
|
||||||
|
of the GPL, as needed to protect the freedom of users.
|
||||||
|
|
||||||
|
Finally, every program is threatened constantly by software patents.
|
||||||
|
States should not allow patents to restrict development and use of
|
||||||
|
software on general-purpose computers, but in those that do, we wish to
|
||||||
|
avoid the special danger that patents applied to a free program could
|
||||||
|
make it effectively proprietary. To prevent this, the GPL assures that
|
||||||
|
patents cannot be used to render the program non-free.
|
||||||
|
|
||||||
The precise terms and conditions for copying, distribution and
|
The precise terms and conditions for copying, distribution and
|
||||||
modification follow.
|
modification follow.
|
||||||
@ -60,7 +72,7 @@ modification follow.
|
|||||||
|
|
||||||
0. Definitions.
|
0. Definitions.
|
||||||
|
|
||||||
"This License" refers to version 3 of the GNU Affero General Public License.
|
"This License" refers to version 3 of the GNU General Public License.
|
||||||
|
|
||||||
"Copyright" also means copyright-like laws that apply to other kinds of
|
"Copyright" also means copyright-like laws that apply to other kinds of
|
||||||
works, such as semiconductor masks.
|
works, such as semiconductor masks.
|
||||||
@ -537,45 +549,35 @@ to collect a royalty for further conveying from those to whom you convey
|
|||||||
the Program, the only way you could satisfy both those terms and this
|
the Program, the only way you could satisfy both those terms and this
|
||||||
License would be to refrain entirely from conveying the Program.
|
License would be to refrain entirely from conveying the Program.
|
||||||
|
|
||||||
13. Remote Network Interaction; Use with the GNU General Public License.
|
13. Use with the GNU Affero General Public License.
|
||||||
|
|
||||||
Notwithstanding any other provision of this License, if you modify the
|
|
||||||
Program, your modified version must prominently offer all users
|
|
||||||
interacting with it remotely through a computer network (if your version
|
|
||||||
supports such interaction) an opportunity to receive the Corresponding
|
|
||||||
Source of your version by providing access to the Corresponding Source
|
|
||||||
from a network server at no charge, through some standard or customary
|
|
||||||
means of facilitating copying of software. This Corresponding Source
|
|
||||||
shall include the Corresponding Source for any work covered by version 3
|
|
||||||
of the GNU General Public License that is incorporated pursuant to the
|
|
||||||
following paragraph.
|
|
||||||
|
|
||||||
Notwithstanding any other provision of this License, you have
|
Notwithstanding any other provision of this License, you have
|
||||||
permission to link or combine any covered work with a work licensed
|
permission to link or combine any covered work with a work licensed
|
||||||
under version 3 of the GNU General Public License into a single
|
under version 3 of the GNU Affero General Public License into a single
|
||||||
combined work, and to convey the resulting work. The terms of this
|
combined work, and to convey the resulting work. The terms of this
|
||||||
License will continue to apply to the part which is the covered work,
|
License will continue to apply to the part which is the covered work,
|
||||||
but the work with which it is combined will remain governed by version
|
but the special requirements of the GNU Affero General Public License,
|
||||||
3 of the GNU General Public License.
|
section 13, concerning interaction through a network will apply to the
|
||||||
|
combination as such.
|
||||||
|
|
||||||
14. Revised Versions of this License.
|
14. Revised Versions of this License.
|
||||||
|
|
||||||
The Free Software Foundation may publish revised and/or new versions of
|
The Free Software Foundation may publish revised and/or new versions of
|
||||||
the GNU Affero General Public License from time to time. Such new versions
|
the GNU General Public License from time to time. Such new versions will
|
||||||
will be similar in spirit to the present version, but may differ in detail to
|
be similar in spirit to the present version, but may differ in detail to
|
||||||
address new problems or concerns.
|
address new problems or concerns.
|
||||||
|
|
||||||
Each version is given a distinguishing version number. If the
|
Each version is given a distinguishing version number. If the
|
||||||
Program specifies that a certain numbered version of the GNU Affero General
|
Program specifies that a certain numbered version of the GNU General
|
||||||
Public License "or any later version" applies to it, you have the
|
Public License "or any later version" applies to it, you have the
|
||||||
option of following the terms and conditions either of that numbered
|
option of following the terms and conditions either of that numbered
|
||||||
version or of any later version published by the Free Software
|
version or of any later version published by the Free Software
|
||||||
Foundation. If the Program does not specify a version number of the
|
Foundation. If the Program does not specify a version number of the
|
||||||
GNU Affero General Public License, you may choose any version ever published
|
GNU General Public License, you may choose any version ever published
|
||||||
by the Free Software Foundation.
|
by the Free Software Foundation.
|
||||||
|
|
||||||
If the Program specifies that a proxy can decide which future
|
If the Program specifies that a proxy can decide which future
|
||||||
versions of the GNU Affero General Public License can be used, that proxy's
|
versions of the GNU General Public License can be used, that proxy's
|
||||||
public statement of acceptance of a version permanently authorizes you
|
public statement of acceptance of a version permanently authorizes you
|
||||||
to choose that version for the Program.
|
to choose that version for the Program.
|
||||||
|
|
||||||
@ -633,29 +635,40 @@ the "copyright" line and a pointer to where the full notice is found.
|
|||||||
Copyright (C) <year> <name of author>
|
Copyright (C) <year> <name of author>
|
||||||
|
|
||||||
This program is free software: you can redistribute it and/or modify
|
This program is free software: you can redistribute it and/or modify
|
||||||
it under the terms of the GNU Affero General Public License as published by
|
it under the terms of the GNU General Public License as published by
|
||||||
the Free Software Foundation, either version 3 of the License, or
|
the Free Software Foundation, either version 3 of the License, or
|
||||||
(at your option) any later version.
|
(at your option) any later version.
|
||||||
|
|
||||||
This program is distributed in the hope that it will be useful,
|
This program is distributed in the hope that it will be useful,
|
||||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||||
GNU Affero General Public License for more details.
|
GNU General Public License for more details.
|
||||||
|
|
||||||
You should have received a copy of the GNU Affero General Public License
|
You should have received a copy of the GNU General Public License
|
||||||
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||||
|
|
||||||
Also add information on how to contact you by electronic and paper mail.
|
Also add information on how to contact you by electronic and paper mail.
|
||||||
|
|
||||||
If your software can interact with users remotely through a computer
|
If the program does terminal interaction, make it output a short
|
||||||
network, you should also make sure that it provides a way for users to
|
notice like this when it starts in an interactive mode:
|
||||||
get its source. For example, if your program is a web application, its
|
|
||||||
interface could display a "Source" link that leads users to an archive
|
<program> Copyright (C) <year> <name of author>
|
||||||
of the code. There are many ways you could offer source, and different
|
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
|
||||||
solutions will be better for different programs; see section 13 for the
|
This is free software, and you are welcome to redistribute it
|
||||||
specific requirements.
|
under certain conditions; type `show c' for details.
|
||||||
|
|
||||||
|
The hypothetical commands `show w' and `show c' should show the appropriate
|
||||||
|
parts of the General Public License. Of course, your program's commands
|
||||||
|
might be different; for a GUI interface, you would use an "about box".
|
||||||
|
|
||||||
You should also get your employer (if you work as a programmer) or school,
|
You should also get your employer (if you work as a programmer) or school,
|
||||||
if any, to sign a "copyright disclaimer" for the program, if necessary.
|
if any, to sign a "copyright disclaimer" for the program, if necessary.
|
||||||
For more information on this, and how to apply and follow the GNU AGPL, see
|
For more information on this, and how to apply and follow the GNU GPL, see
|
||||||
<https://www.gnu.org/licenses/>.
|
<https://www.gnu.org/licenses/>.
|
||||||
|
|
||||||
|
The GNU General Public License does not permit incorporating your program
|
||||||
|
into proprietary programs. If your program is a subroutine library, you
|
||||||
|
may consider it more useful to permit linking proprietary applications with
|
||||||
|
the library. If this is what you want to do, use the GNU Lesser General
|
||||||
|
Public License instead of this License. But first, please read
|
||||||
|
<https://www.gnu.org/licenses/why-not-lgpl.html>.
|
||||||
|
|||||||
@ -1 +1 @@
|
|||||||
include *requirements.txt LICENSE WAIVER WAIVER.asc CHANGELOG chaind/eth/data/config/* chaind/eth/data/config/syncer/*
|
include *requirements.txt LICENSE chaind_eth/data/config/* chaind_eth/data/config/syncer/*
|
||||||
|
|||||||
17
WAIVER
17
WAIVER
@ -1,17 +0,0 @@
|
|||||||
# Copyright waiver for the python package "chaind-eth"
|
|
||||||
|
|
||||||
I dedicate any and all copyright interest in this software to the
|
|
||||||
public domain. I make this dedication for the benefit of the public at
|
|
||||||
large and to the detriment of my heirs and successors. I intend this
|
|
||||||
dedication to be an overt act of relinquishment in perpetuity of all
|
|
||||||
present and future rights to this software under copyright law.
|
|
||||||
|
|
||||||
To the best of my knowledge and belief, my contributions are either
|
|
||||||
originally authored by me or are derived from prior works which I have
|
|
||||||
verified are also in the public domain and are not subject to claims
|
|
||||||
of copyright by other parties.
|
|
||||||
|
|
||||||
To the best of my knowledge and belief, no individual, business,
|
|
||||||
organization, government, or other entity has any copyright interest
|
|
||||||
in my contributions, and I affirm that I will not make contributions
|
|
||||||
that are otherwise encumbered.
|
|
||||||
29
WAIVER.asc
29
WAIVER.asc
@ -1,29 +0,0 @@
|
|||||||
-----BEGIN PGP MESSAGE-----
|
|
||||||
|
|
||||||
owGVU2tQVFUc3wUUuAyIsFAzhBzQYSoWGl4aqQyPDHBQEhphHAru3j27e+Tuvct9
|
|
||||||
sK6kmRiPLAKaDVnwQeZA8rLIBBkoYKJhKLNgBK2BIBoGkSkrVJRH5+6CWn3qw87s
|
|
||||||
vef//z3PLXO1lznJ/X+Y6ijuqXtJ3mTPqdamxybt3ZFK5f7ethHEswYTh7Q6ARhJ
|
|
||||||
lAc5oGE5IOggMJgEHcsAA0nlkFoIAigdiRh1MBR0AQSRBNRQjShSgIBkTPinBiRN
|
|
||||||
A+ohGGIEyEFe+oPREA94ViMYSQ4CgZXgCYOoohEF1Kwew4aAJKAnc6BtdAUaYfZV
|
|
||||||
MSrIQA0SAKuxabMtkwJBk5wWWvltuHhZ4JAeMtZZvQnoIOJ46wAvUhTkeZbjJTpJ
|
|
||||||
oLSFCYnHCDGKSsIDLM5CACRlxeEgjZhcEfE6KzL2ZICcAQoiEkzSOfZOGLBd6VCi
|
|
||||||
0oiCiK1ao+Btyh6PQGTUOOhHYdGkMYQgXmFXrPKr2nMY1khD9YpBFRYBNUrpgGIZ
|
|
||||||
7FIlSpKxOQwJEd7lCBYDIgbLwZ2IuD8O4j0T0EOAg5TmMDFuGSvkWD0wcAi/NrJc
|
|
||||||
Dg+MOkTpcC46Mg8S2DvSIDwmrZA0z9pqfBi8rTVb7XiCYQWcrmo/xGlhsxRNIj1P
|
|
||||||
YA+PLGIRrKQQ3ydOQJD/H34ZiV2N8pBaJGklUIk8YnCPSmxWSzLooLU4JdBKjTFS
|
|
||||||
P0rJrI0NP0kV6Ujeek//ez8JbOPfgSqt9EmA1GgQp8ciSQE/GRG+4JJT60X9xwJh
|
|
||||||
HZGSsLIaEY8LYShRr8Ic6hCixC7cQSZ3kvl6+zs4Bc7WGoI3FPVNFO1Z/SrX2Enf
|
|
||||||
ooxwdl9947jDXXZ+Kf5OfkV5bdDHQe09bp/HD9zsEmoOeyqqA7Lzi1/tq0Qu33Rl
|
|
||||||
NDvdstTeyDBv/wyevOVVNKsMvNSg2DY22ZgSM2E8U1vuvfVUofLMVbusgxdaXd+o
|
|
||||||
mnb28Y7Maus5l/tR5ahHKXi5/H76jO/tL9boxczm152PeuV0fD/zZGaip5/M+7n9
|
|
||||||
fy33Za13mGkvhcMp+4wuy1Pxx2cjzAsPPjFHlQ26X+r4aX39z/mbEoMmHVroGr+y
|
|
||||||
3aZtTxMDMz5/mhq/9vRq7Y10/eXC2rzOyMvHqzxqA4/IN9w/N3qjfzHafD6u1zHd
|
|
||||||
F+R+tStOefqDCKdM/4SRN0M615HDB54N8NjYM11Qtlii1x57cWRnVsJZ4mpdU11r
|
|
||||||
2hMTsd3JHrvvUkfq3fSOhrsN7xz2ubhcMpdnb5e2WHA7fGFve/ODZ+a2tJGWwhPj
|
|
||||||
UVtBaP3Cp98eG2ssTrVEzQ9vV3WcCkd31iafUMgz5C5Rw0uT393r8zNMmJIuftg7
|
|
||||||
7tU1tYdtGok26GYaPJu7w14ruBy9nPrC2crTiqFNruu6B833Rso3H3hedXSs4td3
|
|
||||||
Kw7FKN7qTwhdlF2L6rBMnzRnH0p+zy3ix8KnwjZf3zckJLaAuWuKXbH9JfOZQTWy
|
|
||||||
66HvV+9MW6j+o7F0/GZVRtmV+JihQLrlynwNyIyct3Q2vD3KDsZ9mfrb4hZoGQge
|
|
||||||
88uOC1tK+Rs=
|
|
||||||
=Wuh/
|
|
||||||
-----END PGP MESSAGE-----
|
|
||||||
@ -1,41 +0,0 @@
|
|||||||
# external imports
|
|
||||||
from hexathon import strip_0x
|
|
||||||
from chainqueue.cache import (
|
|
||||||
CacheTx,
|
|
||||||
NoopNormalizer,
|
|
||||||
)
|
|
||||||
from chainlib.eth.tx import unpack
|
|
||||||
from chainlib.encode import TxHexNormalizer
|
|
||||||
|
|
||||||
|
|
||||||
class Normalizer(TxHexNormalizer, NoopNormalizer):
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
super(Normalizer, self).__init__()
|
|
||||||
self.address = self.wallet_address
|
|
||||||
self.hash = self.tx_hash
|
|
||||||
#self.value = self.noop
|
|
||||||
|
|
||||||
|
|
||||||
def value(self, v):
|
|
||||||
hexathon.to_int(v)
|
|
||||||
|
|
||||||
|
|
||||||
eth_normalizer = Normalizer()
|
|
||||||
|
|
||||||
|
|
||||||
class EthCacheTx(CacheTx):
|
|
||||||
|
|
||||||
def __init__(self, chain_spec):
|
|
||||||
super(EthCacheTx, self).__init__(chain_spec)
|
|
||||||
|
|
||||||
|
|
||||||
def deserialize(self, signed_tx):
|
|
||||||
signed_tx_bytes = bytes.fromhex(strip_0x(signed_tx))
|
|
||||||
tx = unpack(signed_tx_bytes, self.chain_spec)
|
|
||||||
self.hash = eth_normalizer.hash(tx['hash'])
|
|
||||||
self.sender = eth_normalizer.address(tx['from'])
|
|
||||||
self.recipient = eth_normalizer.address(tx['to'])
|
|
||||||
self.nonce = eth_normalizer.value(tx['nonce'])
|
|
||||||
self.value = eth_normalizer.value(tx['value'])
|
|
||||||
self.src = signed_tx
|
|
||||||
@ -1,9 +0,0 @@
|
|||||||
[session]
|
|
||||||
socket_path =
|
|
||||||
runtime_dir =
|
|
||||||
id =
|
|
||||||
data_dir =
|
|
||||||
dispatch_delay = 0.01
|
|
||||||
|
|
||||||
[tx]
|
|
||||||
digest_size = 32
|
|
||||||
@ -1,13 +0,0 @@
|
|||||||
# external imports
|
|
||||||
from chainlib.eth.tx import raw
|
|
||||||
|
|
||||||
|
|
||||||
class EthDispatcher:
|
|
||||||
|
|
||||||
def __init__(self, conn):
|
|
||||||
self.conn = conn
|
|
||||||
|
|
||||||
|
|
||||||
def send(self, payload):
|
|
||||||
o = raw(payload)
|
|
||||||
self.conn.do(o)
|
|
||||||
@ -1,113 +0,0 @@
|
|||||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
# standard imports
|
|
||||||
import os
|
|
||||||
import logging
|
|
||||||
import sys
|
|
||||||
import datetime
|
|
||||||
|
|
||||||
# external imports
|
|
||||||
from hexathon import (
|
|
||||||
add_0x,
|
|
||||||
strip_0x,
|
|
||||||
)
|
|
||||||
from chaind import Environment
|
|
||||||
import chainlib.eth.cli
|
|
||||||
from chainlib.chain import ChainSpec
|
|
||||||
from chainqueue.db import dsn_from_config
|
|
||||||
from chainqueue.sql.backend import SQLBackend
|
|
||||||
from chainqueue.enum import StatusBits
|
|
||||||
from chaind.sql.session import SessionIndex
|
|
||||||
from chainqueue.adapters.eth import EthAdapter
|
|
||||||
from chainlib.eth.gas import price
|
|
||||||
from chainlib.eth.connection import EthHTTPConnection
|
|
||||||
from crypto_dev_signer.eth.transaction import EIP155Transaction
|
|
||||||
|
|
||||||
DEFAULT_GAS_FACTOR = 1.1
|
|
||||||
|
|
||||||
|
|
||||||
logging.basicConfig(level=logging.WARNING)
|
|
||||||
logg = logging.getLogger()
|
|
||||||
|
|
||||||
script_dir = os.path.dirname(os.path.realpath(__file__))
|
|
||||||
config_dir = os.path.join(script_dir, '..', 'data', 'config')
|
|
||||||
|
|
||||||
arg_flags = chainlib.eth.cli.argflag_std_write
|
|
||||||
argparser = chainlib.eth.cli.ArgumentParser(arg_flags)
|
|
||||||
argparser.add_argument('--backend', type=str, default='sql', help='Backend to use (currently only "sql")')
|
|
||||||
argparser.add_positional('session_id', required=False, type=str, help='Ethereum address of recipient')
|
|
||||||
args = argparser.parse_args()
|
|
||||||
extra_args = {
|
|
||||||
'backend': None,
|
|
||||||
'session_id': 'SESSION_ID',
|
|
||||||
}
|
|
||||||
|
|
||||||
env = Environment(domain='eth', env=os.environ)
|
|
||||||
|
|
||||||
config = chainlib.eth.cli.Config.from_args(args, arg_flags, extra_args=extra_args, base_config_dir=config_dir)
|
|
||||||
|
|
||||||
if config.get('SESSION_DATA_DIR') == None:
|
|
||||||
config.add(env.data_dir, 'SESSION_DATA_DIR', exists_ok=True)
|
|
||||||
|
|
||||||
chain_spec = ChainSpec.from_chain_str(config.get('CHAIN_SPEC'))
|
|
||||||
|
|
||||||
tx_getter = None
|
|
||||||
session_method = None
|
|
||||||
if config.get('_BACKEND') == 'sql':
|
|
||||||
from chainqueue.sql.query import get_tx_cache as tx_getter
|
|
||||||
from chainqueue.runnable.sql import setup_backend
|
|
||||||
from chainqueue.db.models.base import SessionBase
|
|
||||||
setup_backend(config, debug=config.true('DATABASE_DEBUG'))
|
|
||||||
session_method = SessionBase.create_session
|
|
||||||
else:
|
|
||||||
raise NotImplementedError('backend {} not implemented'.format(config.get('_BACKEND')))
|
|
||||||
|
|
||||||
if config.get('DATABASE_ENGINE') == 'sqlite':
|
|
||||||
config.add(os.path.join(config.get('SESSION_DATA_DIR'), config.get('DATABASE_NAME') + '.sqlite'), 'DATABASE_NAME', exists_ok=True)
|
|
||||||
|
|
||||||
wallet = chainlib.eth.cli.Wallet()
|
|
||||||
wallet.from_config(config)
|
|
||||||
|
|
||||||
rpc = chainlib.eth.cli.Rpc(wallet=wallet)
|
|
||||||
conn = rpc.connect_by_config(config)
|
|
||||||
|
|
||||||
dsn = dsn_from_config(config)
|
|
||||||
backend = SQLBackend(dsn, debug=config.true('DATABASE_DEBUG'), error_parser=rpc.error_parser)
|
|
||||||
session_index_backend = SessionIndex(config.get('SESSION_ID'))
|
|
||||||
adapter = EthAdapter(backend, session_index_backend=session_index_backend)
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
before = datetime.datetime.utcnow() - adapter.pending_retry_threshold
|
|
||||||
txs = session_index_backend.get(chain_spec, adapter, status=StatusBits.IN_NETWORK, not_status=StatusBits.FINAL | StatusBits.OBSOLETE, before=before)
|
|
||||||
|
|
||||||
o = price()
|
|
||||||
r = conn.do(o, error_parser=rpc.error_parser)
|
|
||||||
gas_price = strip_0x(r)
|
|
||||||
try:
|
|
||||||
gas_price = int(gas_price, 16)
|
|
||||||
except ValueError:
|
|
||||||
gas_price = int(gas_price)
|
|
||||||
logg.info('got current gas price {}'.format(gas_price))
|
|
||||||
|
|
||||||
signer = rpc.get_signer()
|
|
||||||
|
|
||||||
db_session = adapter.create_session()
|
|
||||||
for tx_hash in txs:
|
|
||||||
tx_bytes = bytes.fromhex(strip_0x(txs[tx_hash]))
|
|
||||||
tx = adapter.translate(tx_bytes, chain_spec)
|
|
||||||
tx_gas_price = int(tx['gasPrice'])
|
|
||||||
if tx_gas_price < gas_price:
|
|
||||||
tx['gasPrice'] = gas_price
|
|
||||||
else:
|
|
||||||
tx['gasPrice'] = int(tx['gasPrice'] * DEFAULT_GAS_FACTOR)
|
|
||||||
tx_obj = EIP155Transaction(tx, tx['nonce'], chain_spec.chain_id())
|
|
||||||
new_tx_bytes = signer.sign_transaction_to_wire(tx_obj)
|
|
||||||
logg.debug('add tx {} with gas price changed from {} to {}: {}'.format(tx_hash, tx_gas_price, tx['gasPrice'], new_tx_bytes.hex()))
|
|
||||||
adapter.add(new_tx_bytes, chain_spec, session=db_session)
|
|
||||||
|
|
||||||
db_session.close()
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
||||||
@ -1,165 +0,0 @@
|
|||||||
# standard imports
|
|
||||||
import os
|
|
||||||
import logging
|
|
||||||
import signal
|
|
||||||
|
|
||||||
# external imports
|
|
||||||
import chainlib.eth.cli
|
|
||||||
from chainlib.eth.cli.arg import (
|
|
||||||
Arg,
|
|
||||||
ArgFlag,
|
|
||||||
process_args,
|
|
||||||
)
|
|
||||||
from chainlib.eth.cli.config import (
|
|
||||||
Config,
|
|
||||||
process_config,
|
|
||||||
)
|
|
||||||
from chainqueue.cli.arg import (
|
|
||||||
apply_arg as apply_arg_queue,
|
|
||||||
apply_flag as apply_flag_queue,
|
|
||||||
)
|
|
||||||
from chaind.cli.arg import (
|
|
||||||
apply_arg,
|
|
||||||
apply_flag,
|
|
||||||
)
|
|
||||||
from chaind.session import SessionController
|
|
||||||
from chaind.setup import Environment
|
|
||||||
from chaind.error import (
|
|
||||||
NothingToDoError,
|
|
||||||
ClientGoneError,
|
|
||||||
ClientBlockError,
|
|
||||||
ClientInputError,
|
|
||||||
)
|
|
||||||
from chainqueue import (
|
|
||||||
Store,
|
|
||||||
Status,
|
|
||||||
)
|
|
||||||
from chainqueue.error import DuplicateTxError
|
|
||||||
from chainqueue.store.fs import (
|
|
||||||
IndexStore,
|
|
||||||
CounterStore,
|
|
||||||
)
|
|
||||||
from chainqueue.cache import CacheTokenTx
|
|
||||||
from chainlib.encode import TxHexNormalizer
|
|
||||||
from chainlib.chain import ChainSpec
|
|
||||||
from chaind.adapters.fs import ChaindFsAdapter
|
|
||||||
from chaind.dispatch import DispatchProcessor
|
|
||||||
from chainqueue.data import config_dir as chainqueue_config_dir
|
|
||||||
from chaind.data import config_dir as chaind_config_dir
|
|
||||||
from chainlib.eth.cli.log import process_log
|
|
||||||
from chaind.cli.config import process_config as process_config_local
|
|
||||||
|
|
||||||
# local imports
|
|
||||||
from chaind.eth.cache import EthCacheTx
|
|
||||||
from chaind.eth.settings import ChaindSettings
|
|
||||||
from chaind.eth.dispatch import EthDispatcher
|
|
||||||
from chaind.eth.settings import process_settings
|
|
||||||
from chaind.settings import (
|
|
||||||
process_queue,
|
|
||||||
process_socket,
|
|
||||||
process_dispatch,
|
|
||||||
)
|
|
||||||
|
|
||||||
logging.basicConfig(level=logging.WARNING)
|
|
||||||
logg = logging.getLogger()
|
|
||||||
|
|
||||||
script_dir = os.path.dirname(os.path.realpath(__file__))
|
|
||||||
config_dir = os.path.join(script_dir, '..', 'data', 'config')
|
|
||||||
|
|
||||||
env = Environment(domain='eth', env=os.environ)
|
|
||||||
|
|
||||||
arg_flags = ArgFlag()
|
|
||||||
arg_flags = apply_flag_queue(arg_flags)
|
|
||||||
arg_flags = apply_flag(arg_flags)
|
|
||||||
|
|
||||||
arg = Arg(arg_flags)
|
|
||||||
arg = apply_arg_queue(arg)
|
|
||||||
arg = apply_arg(arg)
|
|
||||||
|
|
||||||
flags = arg_flags.STD_READ | arg_flags.QUEUE | arg_flags.STATE | arg_flags.SESSION
|
|
||||||
|
|
||||||
argparser = chainlib.eth.cli.ArgumentParser()
|
|
||||||
argparser = process_args(argparser, arg, flags)
|
|
||||||
args = argparser.parse_args()
|
|
||||||
|
|
||||||
logg = process_log(args, logg)
|
|
||||||
|
|
||||||
config = Config()
|
|
||||||
config.add_schema_dir(chainqueue_config_dir)
|
|
||||||
config.add_schema_dir(chaind_config_dir)
|
|
||||||
config = process_config(config, arg, args, flags)
|
|
||||||
config = process_config_local(config, arg, args, flags)
|
|
||||||
config.add('eth', 'CHAIND_ENGINE', False)
|
|
||||||
config.add('sync', 'CHAIND_COMPONENT', False)
|
|
||||||
logg.debug('config loaded:\n{}'.format(config))
|
|
||||||
|
|
||||||
settings = ChaindSettings(include_sync=True)
|
|
||||||
settings = process_settings(settings, config)
|
|
||||||
settings = process_queue(settings, config)
|
|
||||||
settings = process_socket(settings, config)
|
|
||||||
settings = process_dispatch(settings, config)
|
|
||||||
logg.debug('settings loaded:\n{}'.format(settings))
|
|
||||||
|
|
||||||
tx_normalizer = TxHexNormalizer().tx_hash
|
|
||||||
token_cache_store = CacheTokenTx(settings.get('CHAIN_SPEC'), normalizer=tx_normalizer)
|
|
||||||
|
|
||||||
dispatcher = EthDispatcher(settings.get('CONN'))
|
|
||||||
processor = DispatchProcessor(settings.get('CHAIN_SPEC'), settings.dir_for('queue'), dispatcher)
|
|
||||||
ctrl = SessionController(settings, processor.process)
|
|
||||||
|
|
||||||
signal.signal(signal.SIGINT, ctrl.shutdown)
|
|
||||||
signal.signal(signal.SIGTERM, ctrl.shutdown)
|
|
||||||
|
|
||||||
logg.info('session id is ' + settings.get('SESSION_ID'))
|
|
||||||
logg.info('session socket path is ' + settings.get('SESSION_SOCKET_PATH'))
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
global dispatcher, settings
|
|
||||||
|
|
||||||
queue_adapter = ChaindFsAdapter(
|
|
||||||
settings.get('CHAIN_SPEC'),
|
|
||||||
settings.dir_for('queue'),
|
|
||||||
EthCacheTx,
|
|
||||||
dispatcher,
|
|
||||||
store_sync=False,
|
|
||||||
)
|
|
||||||
|
|
||||||
while True:
|
|
||||||
v = None
|
|
||||||
client_socket = None
|
|
||||||
try:
|
|
||||||
(client_socket, v) = ctrl.get()
|
|
||||||
except ClientGoneError:
|
|
||||||
break
|
|
||||||
except ClientBlockError:
|
|
||||||
continue
|
|
||||||
except ClientInputError:
|
|
||||||
continue
|
|
||||||
except NothingToDoError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
if v == None:
|
|
||||||
ctrl.process(settings.get('CONN'))
|
|
||||||
#queue_adapter = create_adapter(settings, dispatcher)
|
|
||||||
continue
|
|
||||||
|
|
||||||
result_data = None
|
|
||||||
r = 0 # no error
|
|
||||||
try:
|
|
||||||
result_data = queue_adapter.put(v.hex())
|
|
||||||
except DuplicateTxError as e:
|
|
||||||
logg.error('tx already exists: {}'.format(e))
|
|
||||||
r = 1
|
|
||||||
except ValueError as e:
|
|
||||||
logg.error('adapter rejected input {}: "{}"'.format(v.hex(), e))
|
|
||||||
continue
|
|
||||||
|
|
||||||
if r == 0:
|
|
||||||
queue_adapter.enqueue(result_data)
|
|
||||||
|
|
||||||
ctrl.respond_put(client_socket, r, extra_data=result_data)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
||||||
@ -1,207 +0,0 @@
|
|||||||
# standard imports
|
|
||||||
import os
|
|
||||||
import logging
|
|
||||||
import sys
|
|
||||||
import datetime
|
|
||||||
import enum
|
|
||||||
import re
|
|
||||||
import stat
|
|
||||||
import socket
|
|
||||||
|
|
||||||
# external imports
|
|
||||||
import chainlib.eth.cli
|
|
||||||
from chainlib.eth.cli.arg import (
|
|
||||||
Arg,
|
|
||||||
ArgFlag,
|
|
||||||
process_args,
|
|
||||||
)
|
|
||||||
from chainlib.eth.cli.config import (
|
|
||||||
Config,
|
|
||||||
process_config,
|
|
||||||
)
|
|
||||||
from chaind.setup import Environment
|
|
||||||
from chainlib.eth.gas import price
|
|
||||||
from chainlib.chain import ChainSpec
|
|
||||||
from hexathon import strip_0x
|
|
||||||
from chainqueue.cli.arg import (
|
|
||||||
apply_arg as apply_arg_queue,
|
|
||||||
apply_flag as apply_flag_queue,
|
|
||||||
)
|
|
||||||
from chainqueue.data import config_dir as chainqueue_config_dir
|
|
||||||
from chaind.data import config_dir as chaind_config_dir
|
|
||||||
from chaind.cli.arg import (
|
|
||||||
apply_arg,
|
|
||||||
apply_flag,
|
|
||||||
)
|
|
||||||
from chainlib.eth.cli.log import process_log
|
|
||||||
from chaind.settings import process_queue
|
|
||||||
from chaind.settings import ChaindSettings
|
|
||||||
from chaind.error import TxSourceError
|
|
||||||
from chainlib.error import (
|
|
||||||
InitializationError,
|
|
||||||
SignerMissingException,
|
|
||||||
)
|
|
||||||
from chaind.cli.config import process_config as process_config_local
|
|
||||||
|
|
||||||
# local imports
|
|
||||||
from chaind.eth.token.process import Processor
|
|
||||||
from chaind.eth.token.gas import GasTokenResolver
|
|
||||||
from chaind.eth.cli.csv import CSVProcessor
|
|
||||||
from chaind.eth.cli.output import (
|
|
||||||
Outputter,
|
|
||||||
OpMode,
|
|
||||||
)
|
|
||||||
from chaind.eth.settings import process_settings
|
|
||||||
|
|
||||||
logg = logging.getLogger()
|
|
||||||
|
|
||||||
|
|
||||||
def process_settings_local(settings, config):
|
|
||||||
# if settings.get('SIGNER') == None:
|
|
||||||
# raise SignerMissingException('signer missing')
|
|
||||||
return settings
|
|
||||||
|
|
||||||
|
|
||||||
env = Environment(domain='eth', env=os.environ)
|
|
||||||
|
|
||||||
arg_flags = ArgFlag()
|
|
||||||
arg_flags = apply_flag_queue(arg_flags)
|
|
||||||
arg_flags = apply_flag(arg_flags)
|
|
||||||
|
|
||||||
arg = Arg(arg_flags)
|
|
||||||
arg = apply_arg_queue(arg)
|
|
||||||
arg = apply_arg(arg)
|
|
||||||
arg.set_long('s', 'send-rpc')
|
|
||||||
|
|
||||||
flags = arg_flags.STD_WRITE | arg_flags.TOKEN | arg_flags.SOCKET_CLIENT | arg_flags.STATE | arg_flags.WALLET | arg_flags.SESSION
|
|
||||||
|
|
||||||
argparser = chainlib.eth.cli.ArgumentParser()
|
|
||||||
argparser = process_args(argparser, arg, flags)
|
|
||||||
argparser.add_argument('source', help='Transaction source file')
|
|
||||||
args = argparser.parse_args()
|
|
||||||
|
|
||||||
logg = process_log(args, logg)
|
|
||||||
|
|
||||||
config = Config()
|
|
||||||
config.add_schema_dir(chainqueue_config_dir)
|
|
||||||
config.add_schema_dir(chaind_config_dir)
|
|
||||||
config = process_config(config, arg, args, flags)
|
|
||||||
config = process_config_local(config, arg, args, flags)
|
|
||||||
config.add(args.source, '_SOURCE', False)
|
|
||||||
config.add('queue', 'CHAIND_COMPONENT', False)
|
|
||||||
config.add('eth', 'CHAIND_ENGINE', False)
|
|
||||||
logg.debug('config loaded:\n{}'.format(config))
|
|
||||||
|
|
||||||
try:
|
|
||||||
settings = ChaindSettings(include_sync=True)
|
|
||||||
settings = process_settings(settings, config)
|
|
||||||
settings = process_queue(settings, config)
|
|
||||||
settings = process_settings_local(settings, config)
|
|
||||||
except InitializationError as e:
|
|
||||||
sys.stderr.write('Initialization error: ' + str(e) + '\n')
|
|
||||||
sys.exit(1)
|
|
||||||
logg.debug('settings loaded:\n{}'.format(settings))
|
|
||||||
|
|
||||||
mode = OpMode.STDOUT
|
|
||||||
|
|
||||||
re_unix = r'^ipc://(/.+)'
|
|
||||||
m = re.match(re_unix, config.get('SESSION_SOCKET_PATH', ''))
|
|
||||||
if m != None:
|
|
||||||
config.add(m.group(1), 'SESSION_SOCKET_PATH', exists_ok=True)
|
|
||||||
r = 0
|
|
||||||
try:
|
|
||||||
stat_info = os.stat(config.get('SESSION_SOCKET_PATH'))
|
|
||||||
if not stat.S_ISSOCK(stat_info.st_mode):
|
|
||||||
r = 1
|
|
||||||
except FileNotFoundError:
|
|
||||||
r = 1
|
|
||||||
|
|
||||||
if r > 0:
|
|
||||||
sys.stderr.write('{} is not a socket\n'.format(config.get('SESSION_SOCKET_PATH')))
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
mode = OpMode.UNIX
|
|
||||||
|
|
||||||
logg.info('using mode {}'.format(mode.value))
|
|
||||||
|
|
||||||
if config.get('_SOURCE') == None:
|
|
||||||
sys.stderr.write('source data missing\n')
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
|
|
||||||
class SocketSender:
|
|
||||||
|
|
||||||
def __init__(self, settings):
|
|
||||||
self.path = settings.get('SESSION_SOCKET_PATH')
|
|
||||||
|
|
||||||
def send(self, tx):
|
|
||||||
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
|
||||||
err = None
|
|
||||||
try:
|
|
||||||
s.connect(self.path)
|
|
||||||
except FileNotFoundError as e:
|
|
||||||
err = e
|
|
||||||
if err != None:
|
|
||||||
s.close()
|
|
||||||
raise err
|
|
||||||
s.sendall(tx.encode('utf-8'))
|
|
||||||
r = s.recv(68)
|
|
||||||
s.close()
|
|
||||||
return r
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
conn = settings.get('CONN')
|
|
||||||
token_resolver = None
|
|
||||||
if settings.get('TOKEN_MODULE') != None:
|
|
||||||
import importlib
|
|
||||||
m = importlib.import_module(settings.get('TOKEN_MODULE'))
|
|
||||||
m = m.TokenResolver
|
|
||||||
else:
|
|
||||||
from chaind.eth.token.gas import GasTokenResolver
|
|
||||||
m = GasTokenResolver
|
|
||||||
token_resolver = m(
|
|
||||||
settings.get('CHAIN_SPEC'),
|
|
||||||
settings.get('SENDER_ADDRESS'),
|
|
||||||
settings.get('SIGNER'),
|
|
||||||
settings.get('GAS_ORACLE'),
|
|
||||||
settings.get('NONCE_ORACLE'),
|
|
||||||
)
|
|
||||||
|
|
||||||
processor = Processor(token_resolver, config.get('_SOURCE'), use_checksum=not config.get('_UNSAFE'))
|
|
||||||
processor.add_processor(CSVProcessor())
|
|
||||||
|
|
||||||
sends = None
|
|
||||||
try:
|
|
||||||
sends = processor.load(conn)
|
|
||||||
except TxSourceError as e:
|
|
||||||
sys.stderr.write('processing error: {}. processors: {}\n'.format(str(e), str(processor)))
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
sender = None
|
|
||||||
if config.true('_SOCKET_SEND'):
|
|
||||||
if settings.get('SESSION_SOCKET_PATH') != None:
|
|
||||||
sender = SocketSender(settings)
|
|
||||||
|
|
||||||
tx_iter = iter(processor)
|
|
||||||
out = Outputter(mode)
|
|
||||||
while True:
|
|
||||||
tx = None
|
|
||||||
try:
|
|
||||||
tx_bytes = next(tx_iter)
|
|
||||||
except StopIteration:
|
|
||||||
break
|
|
||||||
tx_hex = tx_bytes.hex()
|
|
||||||
if sender != None:
|
|
||||||
r = None
|
|
||||||
try:
|
|
||||||
r = sender.send(tx_hex)
|
|
||||||
except FileNotFoundError as e:
|
|
||||||
sys.stderr.write('send to socket {} failed: {}\n'.format(sender.path, e))
|
|
||||||
sys.exit(1)
|
|
||||||
logg.info('sent {} result {}'.format(tx_hex, r))
|
|
||||||
print(out.do(tx_hex))
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
||||||
@ -1,102 +0,0 @@
|
|||||||
# standard imports
|
|
||||||
import os
|
|
||||||
import logging
|
|
||||||
|
|
||||||
# external imports
|
|
||||||
import chainlib.eth.cli
|
|
||||||
from chaind.setup import Environment
|
|
||||||
from chaind.filter import StateFilter
|
|
||||||
from chainlib.eth.block import block_latest
|
|
||||||
from hexathon import strip_0x
|
|
||||||
from chainsyncer.store.fs import SyncFsStore
|
|
||||||
from chainsyncer.driver.chain_interface import ChainInterfaceDriver
|
|
||||||
from chainsyncer.error import SyncDone
|
|
||||||
from chainlib.eth.cli.arg import (
|
|
||||||
Arg,
|
|
||||||
ArgFlag,
|
|
||||||
process_args,
|
|
||||||
)
|
|
||||||
from chainlib.eth.cli.config import (
|
|
||||||
Config,
|
|
||||||
process_config,
|
|
||||||
)
|
|
||||||
from chainsyncer.cli.arg import (
|
|
||||||
apply_arg as apply_arg_sync,
|
|
||||||
apply_flag as apply_flag_sync,
|
|
||||||
)
|
|
||||||
from chainsyncer.data import config_dir as chainsyncer_config_dir
|
|
||||||
from chaind.data import config_dir as chaind_config_dir
|
|
||||||
from chaind.cli.arg import (
|
|
||||||
apply_arg,
|
|
||||||
apply_flag,
|
|
||||||
)
|
|
||||||
from chainlib.eth.cli.log import process_log
|
|
||||||
from chaind.settings import ChaindSettings
|
|
||||||
from chaind.cli.config import process_config as process_config_local
|
|
||||||
from chainsyncer.cli.config import process_config as process_config_syncer
|
|
||||||
|
|
||||||
# local imports
|
|
||||||
from chaind.eth.cache import EthCacheTx
|
|
||||||
from chaind.eth.settings import (
|
|
||||||
process_settings,
|
|
||||||
process_sync,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
logg = logging.getLogger()
|
|
||||||
|
|
||||||
script_dir = os.path.dirname(os.path.realpath(__file__))
|
|
||||||
config_dir = os.path.join(script_dir, '..', 'data', 'config')
|
|
||||||
|
|
||||||
env = Environment(domain='eth', env=os.environ)
|
|
||||||
|
|
||||||
arg_flags = ArgFlag()
|
|
||||||
arg_flags = apply_flag_sync(arg_flags)
|
|
||||||
arg_flags = apply_flag(arg_flags)
|
|
||||||
|
|
||||||
arg = Arg(arg_flags)
|
|
||||||
arg = apply_arg_sync(arg)
|
|
||||||
arg = apply_arg(arg)
|
|
||||||
|
|
||||||
flags = arg_flags.STD_BASE | arg_flags.CHAIN_SPEC | arg_flags.PROVIDER | arg_flags.SEQ | arg_flags.STATE
|
|
||||||
flags = arg_flags.more(flags, arg_flags.SYNC_RANGE_EXT)
|
|
||||||
flags = arg_flags.more(flags, arg_flags.CHAIND_BASE)
|
|
||||||
|
|
||||||
argparser = chainlib.eth.cli.ArgumentParser()
|
|
||||||
argparser = process_args(argparser, arg, flags)
|
|
||||||
args = argparser.parse_args()
|
|
||||||
|
|
||||||
logg = process_log(args, logg)
|
|
||||||
|
|
||||||
config = Config()
|
|
||||||
config.add_schema_dir(chainsyncer_config_dir)
|
|
||||||
config.add_schema_dir(chaind_config_dir)
|
|
||||||
config = process_config(config, arg, args, flags)
|
|
||||||
config = process_config_local(config, arg, args, flags)
|
|
||||||
config = process_config_syncer(config, arg, args, flags)
|
|
||||||
config.add('eth', 'CHAIND_ENGINE', False)
|
|
||||||
config.add('sync', 'CHAIND_COMPONENT', False)
|
|
||||||
logg.debug('config loaded:\n{}'.format(config))
|
|
||||||
|
|
||||||
settings = ChaindSettings(include_sync=True)
|
|
||||||
settings = process_settings(settings, config)
|
|
||||||
settings = process_sync(settings, config)
|
|
||||||
logg.debug('settings loaded:\n{}'.format(settings))
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
fltr = StateFilter(settings.get('CHAIN_SPEC'), settings.dir_for('queue'), EthCacheTx)
|
|
||||||
sync_store = SyncFsStore(settings.get('SESSION_DATA_PATH'), session_id=settings.get('SESSION_ID'))
|
|
||||||
sync_store.register(fltr)
|
|
||||||
|
|
||||||
logg.debug('session block offset {}'.format(settings.get('SYNCER_OFFSET')))
|
|
||||||
|
|
||||||
drv = ChainInterfaceDriver(sync_store, settings.get('SYNCER_INTERFACE'), offset=settings.get('SYNCER_OFFSET'), target=settings.get('SYNCER_LIMIT'))
|
|
||||||
try:
|
|
||||||
drv.run(settings.get('CONN'))
|
|
||||||
except SyncDone as e:
|
|
||||||
logg.info('sync done: {}'.format(e))
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
||||||
@ -1,32 +0,0 @@
|
|||||||
# external imports
|
|
||||||
from chainlib.eth.connection import EthHTTPConnection
|
|
||||||
from chainlib.eth.settings import process_settings as base_process_settings
|
|
||||||
from chaind.eth.chain import EthChainInterface
|
|
||||||
from chaind.settings import *
|
|
||||||
from chainsyncer.settings import process_sync_range
|
|
||||||
|
|
||||||
|
|
||||||
def process_common(settings, config):
|
|
||||||
rpc_provider = config.get('RPC_PROVIDER')
|
|
||||||
if rpc_provider == None:
|
|
||||||
rpc_provider = 'http://localhost:8545'
|
|
||||||
conn = EthHTTPConnection(url=rpc_provider, chain_spec=settings.get('CHAIN_SPEC'))
|
|
||||||
settings.set('RPC', conn)
|
|
||||||
return settings
|
|
||||||
|
|
||||||
|
|
||||||
def process_sync(settings, config):
|
|
||||||
settings.set('SYNCER_INTERFACE', EthChainInterface())
|
|
||||||
settings = process_sync_range(settings, config)
|
|
||||||
return settings
|
|
||||||
|
|
||||||
|
|
||||||
def process_settings(settings, config):
|
|
||||||
settings = base_process_settings(settings, config)
|
|
||||||
settings = process_common(settings, config)
|
|
||||||
settings = process_backend(settings, config)
|
|
||||||
settings = process_session(settings, config)
|
|
||||||
settings = process_socket(settings, config)
|
|
||||||
settings = process_token(settings, config)
|
|
||||||
|
|
||||||
return settings
|
|
||||||
@ -1 +0,0 @@
|
|||||||
from .base import *
|
|
||||||
@ -1,56 +0,0 @@
|
|||||||
# standard imports
|
|
||||||
import logging
|
|
||||||
|
|
||||||
# external imports
|
|
||||||
from funga.eth.transaction import EIP155Transaction
|
|
||||||
from hexathon import strip_0x
|
|
||||||
|
|
||||||
logg = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
class BaseTokenResolver:
|
|
||||||
|
|
||||||
def __init__(self, chain_spec, sender, signer, gas_oracle, nonce_oracle, advance_nonce=False):
|
|
||||||
self.chain_spec = chain_spec
|
|
||||||
self.chain_id = chain_spec.chain_id()
|
|
||||||
self.signer = signer
|
|
||||||
self.sender = sender
|
|
||||||
self.gas_oracle = gas_oracle
|
|
||||||
self.nonce_oracle = nonce_oracle
|
|
||||||
self.factory = None
|
|
||||||
self.gas_limit_start = None
|
|
||||||
self.gas_price_start = None
|
|
||||||
if advance_nonce:
|
|
||||||
self.nonce_getter = self.nonce_oracle.next_nonce
|
|
||||||
else:
|
|
||||||
self.nonce_getter = self.nonce_oracle.get_nonce
|
|
||||||
|
|
||||||
|
|
||||||
def reset(self):
|
|
||||||
gas_data = self.gas_oracle.get_gas()
|
|
||||||
self.gas_price_start = gas_data[0]
|
|
||||||
self.gas_limit_start = gas_data[1]
|
|
||||||
|
|
||||||
|
|
||||||
def get_values(self, gas_value, value, executable_address=None):
|
|
||||||
nonce = self.nonce_getter()
|
|
||||||
|
|
||||||
if executable_address == None:
|
|
||||||
return (value, 0, nonce)
|
|
||||||
|
|
||||||
try:
|
|
||||||
value = int(value)
|
|
||||||
except ValueError:
|
|
||||||
value = int(strip_0x(value), 16)
|
|
||||||
|
|
||||||
try:
|
|
||||||
gas_value = int(gas_value)
|
|
||||||
except ValueError:
|
|
||||||
gas_value = int(strip_0x(gas_value), 16)
|
|
||||||
|
|
||||||
return (gas_value, value, nonce,)
|
|
||||||
|
|
||||||
|
|
||||||
def sign(self, tx):
|
|
||||||
tx_o = EIP155Transaction(tx, tx['nonce'], self.chain_id)
|
|
||||||
tx_bytes = self.signer.sign_transaction_to_wire(tx_o)
|
|
||||||
return tx_bytes
|
|
||||||
@ -1,26 +0,0 @@
|
|||||||
# external imports
|
|
||||||
from eth_erc20 import ERC20
|
|
||||||
from chainlib.eth.tx import TxFormat
|
|
||||||
|
|
||||||
# local imports
|
|
||||||
from chaind.eth.token import BaseTokenResolver
|
|
||||||
|
|
||||||
|
|
||||||
class TokenResolver(BaseTokenResolver):
|
|
||||||
|
|
||||||
def __init__(self, chain_spec, sender, signer, gas_oracle, nonce_oracle):
|
|
||||||
super(TokenResolver, self).__init__(chain_spec, sender, signer, gas_oracle, nonce_oracle)
|
|
||||||
self.factory = ERC20(self.chain_spec, signer=self.signer, gas_oracle=self.gas_oracle, nonce_oracle=self.nonce_oracle)
|
|
||||||
|
|
||||||
|
|
||||||
def create(self, conn, recipient, gas_value, data=None, token_value=0, executable_address=None, passphrase=None):
|
|
||||||
|
|
||||||
if executable_address == None:
|
|
||||||
raise ValueError('executable address required')
|
|
||||||
|
|
||||||
(gas_value, token_value, nonce) = self.get_values(gas_value, token_value, executable_address=executable_address)
|
|
||||||
|
|
||||||
tx = self.factory.transfer(executable_address, self.sender, recipient, token_value, tx_format=TxFormat.DICT)
|
|
||||||
tx['value'] = gas_value
|
|
||||||
|
|
||||||
return tx
|
|
||||||
@ -1,30 +0,0 @@
|
|||||||
# external imports
|
|
||||||
from chainlib.eth.gas import Gas
|
|
||||||
from hexathon import strip_0x
|
|
||||||
|
|
||||||
# local imports
|
|
||||||
from chaind.eth.token import BaseTokenResolver
|
|
||||||
|
|
||||||
|
|
||||||
class GasTokenResolver(BaseTokenResolver):
|
|
||||||
|
|
||||||
def __init__(self, chain_spec, sender, signer, gas_oracle, nonce_oracle):
|
|
||||||
super(GasTokenResolver, self).__init__(chain_spec, sender, signer, gas_oracle, nonce_oracle, advance_nonce=True)
|
|
||||||
self.factory = Gas(self.chain_spec, signer=self.signer, gas_oracle=self.gas_oracle, nonce_oracle=self.nonce_oracle)
|
|
||||||
|
|
||||||
|
|
||||||
def create(self, conn, recipient, gas_value, data=None, token_value=0, executable_address=None, passphrase=None):
|
|
||||||
|
|
||||||
(gas_value, token_value, nonce) = self.get_values(gas_value, token_value, executable_address=executable_address)
|
|
||||||
|
|
||||||
tx = {
|
|
||||||
'from': self.sender,
|
|
||||||
'to': recipient,
|
|
||||||
'value': gas_value,
|
|
||||||
'data': data,
|
|
||||||
'nonce': nonce,
|
|
||||||
'gasPrice': self.gas_price_start,
|
|
||||||
'gas': self.gas_limit_start,
|
|
||||||
}
|
|
||||||
|
|
||||||
return tx
|
|
||||||
@ -1,114 +0,0 @@
|
|||||||
# standard imports
|
|
||||||
import logging
|
|
||||||
|
|
||||||
# external imports
|
|
||||||
from chaind.error import TxSourceError
|
|
||||||
from chainlib.eth.address import (
|
|
||||||
is_checksum_address,
|
|
||||||
to_checksum_address,
|
|
||||||
)
|
|
||||||
from chainlib.eth.tx import unpack
|
|
||||||
from chainlib.eth.gas import Gas
|
|
||||||
from hexathon import (
|
|
||||||
add_0x,
|
|
||||||
strip_0x,
|
|
||||||
)
|
|
||||||
from funga.eth.transaction import EIP155Transaction
|
|
||||||
|
|
||||||
logg = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
class Processor:
|
|
||||||
|
|
||||||
def __init__(self, resolver, source, use_checksum=True):
|
|
||||||
self.resolver = resolver
|
|
||||||
self.source = source
|
|
||||||
self.processor = []
|
|
||||||
self.safe = use_checksum
|
|
||||||
self.conn = None
|
|
||||||
|
|
||||||
|
|
||||||
def add_processor(self, processor):
|
|
||||||
self.processor.append(processor)
|
|
||||||
|
|
||||||
|
|
||||||
def load(self, conn, process=True):
|
|
||||||
self.conn = conn
|
|
||||||
for processor in self.processor:
|
|
||||||
self.content = processor.load(self.source)
|
|
||||||
if self.content != None:
|
|
||||||
if process:
|
|
||||||
try:
|
|
||||||
self.process()
|
|
||||||
except Exception as e:
|
|
||||||
raise TxSourceError('invalid source contents: {}'.format(str(e)))
|
|
||||||
return self.content
|
|
||||||
raise TxSourceError('unparseable source')
|
|
||||||
|
|
||||||
|
|
||||||
# 0: recipient
|
|
||||||
# 1: amount
|
|
||||||
# 2: token identifier (optional, when not specified network gas token will be used)
|
|
||||||
# 3: gas amount (optional)
|
|
||||||
def process(self):
|
|
||||||
txs = []
|
|
||||||
for i, r in enumerate(self.content):
|
|
||||||
logg.debug('processing {}'.format(r))
|
|
||||||
address = r[0]
|
|
||||||
if self.safe:
|
|
||||||
if not is_checksum_address(address):
|
|
||||||
raise ValueError('invalid checksum address {} in record {}'.format(address, i))
|
|
||||||
else:
|
|
||||||
address = to_checksum_address(address)
|
|
||||||
|
|
||||||
self.content[i][0] = add_0x(address)
|
|
||||||
try:
|
|
||||||
self.content[i][1] = int(r[1])
|
|
||||||
except ValueError:
|
|
||||||
self.content[i][1] = int(strip_0x(r[1]), 16)
|
|
||||||
native_token_value = 0
|
|
||||||
|
|
||||||
if len(self.content[i]) == 3:
|
|
||||||
self.content[i].append(native_token_value)
|
|
||||||
|
|
||||||
|
|
||||||
def __iter__(self):
|
|
||||||
self.resolver.reset()
|
|
||||||
self.cursor = 0
|
|
||||||
return self
|
|
||||||
|
|
||||||
|
|
||||||
def __next__(self):
|
|
||||||
if self.cursor == len(self.content):
|
|
||||||
raise StopIteration()
|
|
||||||
|
|
||||||
r = self.content[self.cursor]
|
|
||||||
|
|
||||||
value = r[1]
|
|
||||||
gas_value = 0
|
|
||||||
try:
|
|
||||||
gas_value = r[3]
|
|
||||||
except IndexError:
|
|
||||||
pass
|
|
||||||
logg.debug('gasvalue {}'.format(gas_value))
|
|
||||||
data = '0x'
|
|
||||||
|
|
||||||
executable_address = None
|
|
||||||
try:
|
|
||||||
executable_address = r[2]
|
|
||||||
except IndexError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
tx = self.resolver.create(self.conn, r[0], gas_value, data=data, token_value=value, executable_address=executable_address)
|
|
||||||
v = self.resolver.sign(tx)
|
|
||||||
|
|
||||||
self.cursor += 1
|
|
||||||
|
|
||||||
return v
|
|
||||||
|
|
||||||
|
|
||||||
def __str__(self):
|
|
||||||
names = []
|
|
||||||
for s in self.processor:
|
|
||||||
names.append(str(s))
|
|
||||||
return ','.join(names)
|
|
||||||
@ -2,7 +2,6 @@
|
|||||||
from chainlib.interface import ChainInterface
|
from chainlib.interface import ChainInterface
|
||||||
from chainlib.eth.block import (
|
from chainlib.eth.block import (
|
||||||
block_by_number,
|
block_by_number,
|
||||||
block_latest,
|
|
||||||
Block,
|
Block,
|
||||||
)
|
)
|
||||||
from chainlib.eth.tx import (
|
from chainlib.eth.tx import (
|
||||||
@ -10,6 +9,7 @@ from chainlib.eth.tx import (
|
|||||||
Tx,
|
Tx,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
class EthChainInterface(ChainInterface):
|
class EthChainInterface(ChainInterface):
|
||||||
|
|
||||||
def __init__(self):
|
def __init__(self):
|
||||||
@ -17,4 +17,3 @@ class EthChainInterface(ChainInterface):
|
|||||||
self._block_from_src = Block.from_src
|
self._block_from_src = Block.from_src
|
||||||
self._tx_receipt = receipt
|
self._tx_receipt = receipt
|
||||||
self._src_normalize = Tx.src_normalize
|
self._src_normalize = Tx.src_normalize
|
||||||
self._block_latest = block_latest
|
|
||||||
162
chaind_eth/cli/process.py
Normal file
162
chaind_eth/cli/process.py
Normal file
@ -0,0 +1,162 @@
|
|||||||
|
# standard imports
|
||||||
|
import logging
|
||||||
|
|
||||||
|
# external imports
|
||||||
|
from chaind.error import TxSourceError
|
||||||
|
from chainlib.eth.address import is_checksum_address
|
||||||
|
from chainlib.eth.tx import unpack
|
||||||
|
from chainlib.eth.gas import Gas
|
||||||
|
from hexathon import (
|
||||||
|
add_0x,
|
||||||
|
strip_0x,
|
||||||
|
)
|
||||||
|
from crypto_dev_signer.eth.transaction import EIP155Transaction
|
||||||
|
from eth_erc20 import ERC20
|
||||||
|
|
||||||
|
logg = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class Processor:
|
||||||
|
|
||||||
|
def __init__(self, sender, signer, source, chain_spec, gas_oracle, nonce_oracle, resolver=None):
|
||||||
|
self.sender = sender
|
||||||
|
self.signer = signer
|
||||||
|
self.source = source
|
||||||
|
self.processor = []
|
||||||
|
self.content = []
|
||||||
|
self.token = []
|
||||||
|
self.token_resolver = resolver
|
||||||
|
self.cursor = 0
|
||||||
|
self.gas_oracle = gas_oracle
|
||||||
|
self.nonce_oracle = nonce_oracle
|
||||||
|
self.nonce_start = None
|
||||||
|
self.gas_limit_start = None
|
||||||
|
self.gas_price_start = None
|
||||||
|
self.chain_spec = chain_spec
|
||||||
|
self.chain_id = chain_spec.chain_id()
|
||||||
|
|
||||||
|
|
||||||
|
def add_processor(self, processor):
|
||||||
|
self.processor.append(processor)
|
||||||
|
|
||||||
|
|
||||||
|
def load(self, process=True):
|
||||||
|
for processor in self.processor:
|
||||||
|
self.content = processor.load(self.source)
|
||||||
|
if self.content != None:
|
||||||
|
if process:
|
||||||
|
try:
|
||||||
|
self.process()
|
||||||
|
except Exception as e:
|
||||||
|
raise TxSourceError('invalid source contents: {}'.format(str(e)))
|
||||||
|
return self.content
|
||||||
|
raise TxSourceError('unparseable source')
|
||||||
|
|
||||||
|
|
||||||
|
# 0: recipient
|
||||||
|
# 1: amount
|
||||||
|
# 2: token identifier (optional, when not specified network gas token will be used)
|
||||||
|
# 3: gas amount (optional)
|
||||||
|
def process(self):
|
||||||
|
txs = []
|
||||||
|
for i, r in enumerate(self.content):
|
||||||
|
logg.debug('processing {}'.format(r))
|
||||||
|
if not is_checksum_address(r[0]):
|
||||||
|
raise ValueError('invalid checksum address {} in record {}'.format(r[0], i))
|
||||||
|
self.content[i][0] = add_0x(r[0])
|
||||||
|
try:
|
||||||
|
self.content[i][1] = int(r[1])
|
||||||
|
except ValueError:
|
||||||
|
self.content[i][1] = int(strip_0x(r[1]), 16)
|
||||||
|
native_token_value = 0
|
||||||
|
if self.token_resolver == None:
|
||||||
|
self.token.append(None)
|
||||||
|
else:
|
||||||
|
#self.content[i][2] = self.token_resolver.lookup(k)
|
||||||
|
token = self.token_resolver.lookup(r[2])
|
||||||
|
self.token.append(token)
|
||||||
|
|
||||||
|
if len(self.content[i]) == 3:
|
||||||
|
self.content[i].append(native_token_value)
|
||||||
|
|
||||||
|
|
||||||
|
def __iter__(self):
|
||||||
|
gas_data = self.gas_oracle.get_gas()
|
||||||
|
self.gas_price_start = gas_data[0]
|
||||||
|
self.gas_limit_start = gas_data[1]
|
||||||
|
self.cursor = 0
|
||||||
|
return self
|
||||||
|
|
||||||
|
|
||||||
|
def __next__(self):
|
||||||
|
if self.cursor == len(self.content):
|
||||||
|
raise StopIteration()
|
||||||
|
|
||||||
|
nonce = self.nonce_oracle.next_nonce()
|
||||||
|
|
||||||
|
token_factory = None
|
||||||
|
|
||||||
|
r = self.content[self.cursor]
|
||||||
|
token = self.token[self.cursor]
|
||||||
|
if token == None:
|
||||||
|
token_factory = Gas(self.chain_spec, signer=self.signer, gas_oracle=self.gas_oracle, nonce_oracle=self.nonce_oracle)
|
||||||
|
else:
|
||||||
|
token_factory = ERC20(self.chain_spec, signer=self.signer, gas_oracle=self.gas_oracle, nonce_oracle=self.nonce_oracle)
|
||||||
|
|
||||||
|
value = 0
|
||||||
|
gas_value = 0
|
||||||
|
data = '0x'
|
||||||
|
debug_destination = (r[2], token)
|
||||||
|
if debug_destination[1] == None:
|
||||||
|
debug_destination = (None, 'network gas token')
|
||||||
|
if isinstance(token_factory, ERC20):
|
||||||
|
(tx_hash_hex, o) = token_factory.transfer(token, self.sender, r[0], r[1])
|
||||||
|
logg.debug('tx {}'.format(o))
|
||||||
|
# TODO: allow chainlib to return data args only (TxFormat)
|
||||||
|
tx = unpack(bytes.fromhex(strip_0x(o['params'][0])), self.chain_spec)
|
||||||
|
data = tx['data']
|
||||||
|
try:
|
||||||
|
value = int(r[1])
|
||||||
|
except ValueError:
|
||||||
|
value = int(strip_0x(r[1]), 16)
|
||||||
|
try:
|
||||||
|
gas_value = int(r[3])
|
||||||
|
except:
|
||||||
|
gas_value = int(strip_0x(r[3]), 16)
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
value = int(r[1])
|
||||||
|
except ValueError:
|
||||||
|
value = int(strip_0x(r[1]), 16)
|
||||||
|
gas_value = value
|
||||||
|
|
||||||
|
logg.debug('token factory {} resolved sender {} recipient {} gas value {} token value {} token {}'.format(
|
||||||
|
str(token_factory),
|
||||||
|
self.sender,
|
||||||
|
r[0],
|
||||||
|
gas_value,
|
||||||
|
value,
|
||||||
|
debug_destination,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
tx = {
|
||||||
|
'from': self.sender,
|
||||||
|
'to': r[0],
|
||||||
|
'value': gas_value,
|
||||||
|
'data': data,
|
||||||
|
'nonce': nonce,
|
||||||
|
'gasPrice': self.gas_price_start,
|
||||||
|
'gas': self.gas_limit_start,
|
||||||
|
}
|
||||||
|
tx_o = EIP155Transaction(tx, nonce, self.chain_id)
|
||||||
|
tx_bytes = self.signer.sign_transaction_to_wire(tx_o)
|
||||||
|
self.cursor += 1
|
||||||
|
return tx_bytes
|
||||||
|
|
||||||
|
|
||||||
|
def __str__(self):
|
||||||
|
names = []
|
||||||
|
for s in self.processor:
|
||||||
|
names.append(str(s))
|
||||||
|
return ','.join(names)
|
||||||
86
chaind_eth/cli/resolver.py
Normal file
86
chaind_eth/cli/resolver.py
Normal file
@ -0,0 +1,86 @@
|
|||||||
|
# standard imports
|
||||||
|
import logging
|
||||||
|
|
||||||
|
# external imports
|
||||||
|
from chainlib.eth.constant import ZERO_ADDRESS
|
||||||
|
from chainlib.eth.address import is_checksum_address
|
||||||
|
from hexathon import strip_0x
|
||||||
|
from eth_token_index.index import TokenUniqueSymbolIndex
|
||||||
|
|
||||||
|
logg = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class LookNoop:
|
||||||
|
|
||||||
|
def __init__(self, check=True):
|
||||||
|
self.check = check
|
||||||
|
|
||||||
|
|
||||||
|
def get(self, k, rpc=None):
|
||||||
|
if not self.check:
|
||||||
|
address_bytes = bytes.fromhex(strip_0x(k))
|
||||||
|
if len(address_bytes) != 20:
|
||||||
|
raise ValueError('{} is not a valid address'.format(k))
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
if not is_checksum_address(k):
|
||||||
|
raise ValueError('not valid checksum address {}'.format(k))
|
||||||
|
except ValueError:
|
||||||
|
raise ValueError('not valid checksum address {}'.format(k))
|
||||||
|
return strip_0x(k)
|
||||||
|
|
||||||
|
|
||||||
|
def __str__(self):
|
||||||
|
return 'checksum address shortcircuit'
|
||||||
|
|
||||||
|
|
||||||
|
class TokenIndexLookup(TokenUniqueSymbolIndex):
|
||||||
|
|
||||||
|
|
||||||
|
def __init__(self, chain_spec, signer, gas_oracle, nonce_oracle, address, sender_address=ZERO_ADDRESS):
|
||||||
|
super(TokenIndexLookup, self).__init__(chain_spec, signer=signer, gas_oracle=gas_oracle, nonce_oracle=nonce_oracle)
|
||||||
|
self.local_address = address
|
||||||
|
self.sender_address = sender_address
|
||||||
|
|
||||||
|
|
||||||
|
def get(self, k, rpc=None):
|
||||||
|
o = self.address_of(self.local_address, k, sender_address=self.sender_address)
|
||||||
|
r = rpc.do(o)
|
||||||
|
address = self.parse_address_of(r)
|
||||||
|
if address != ZERO_ADDRESS:
|
||||||
|
return address
|
||||||
|
raise FileNotFoundError(address)
|
||||||
|
|
||||||
|
|
||||||
|
def __str__(self):
|
||||||
|
return 'token symbol index'
|
||||||
|
|
||||||
|
|
||||||
|
class DefaultResolver:
|
||||||
|
|
||||||
|
def __init__(self, chain_spec, rpc, sender_address=ZERO_ADDRESS):
|
||||||
|
self.chain_spec = chain_spec
|
||||||
|
self.rpc = rpc
|
||||||
|
self.lookups = []
|
||||||
|
self.lookup_pointers = []
|
||||||
|
self.cursor = 0
|
||||||
|
self.sender_address = sender_address
|
||||||
|
|
||||||
|
|
||||||
|
def add_lookup(self, lookup, reverse):
|
||||||
|
self.lookups.append(lookup)
|
||||||
|
self.lookup_pointers.append(reverse)
|
||||||
|
|
||||||
|
|
||||||
|
def lookup(self, k):
|
||||||
|
if k == '' or k == None:
|
||||||
|
return None
|
||||||
|
for lookup in self.lookups:
|
||||||
|
try:
|
||||||
|
address = lookup.get(k, rpc=self.rpc)
|
||||||
|
logg.debug('resolved token {} to {} with lookup {}'.format(k, address, lookup))
|
||||||
|
return address
|
||||||
|
except Exception as e:
|
||||||
|
logg.debug('lookup {} failed for {}: {}'.format(lookup, k, e))
|
||||||
|
|
||||||
|
raise FileNotFoundError(k)
|
||||||
81
chaind_eth/cli/retry.py
Normal file
81
chaind_eth/cli/retry.py
Normal file
@ -0,0 +1,81 @@
|
|||||||
|
# standard imports
|
||||||
|
import logging
|
||||||
|
|
||||||
|
# external imports
|
||||||
|
from chainlib.eth.gas import price
|
||||||
|
from chainlib.eth.tx import unpack
|
||||||
|
from chaind.error import TxSourceError
|
||||||
|
from crypto_dev_signer.eth.transaction import EIP155Transaction
|
||||||
|
from chainlib.eth.gas import Gas
|
||||||
|
from hexathon import (
|
||||||
|
add_0x,
|
||||||
|
strip_0x,
|
||||||
|
)
|
||||||
|
|
||||||
|
# local imports
|
||||||
|
from chaind_eth.cli.tx import TxProcessor
|
||||||
|
|
||||||
|
logg = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
DEFAULT_GAS_FACTOR = 1.1
|
||||||
|
|
||||||
|
|
||||||
|
class Retrier:
|
||||||
|
|
||||||
|
def __init__(self, sender, signer, source, chain_spec, gas_oracle, gas_factor=DEFAULT_GAS_FACTOR):
|
||||||
|
self.sender = sender
|
||||||
|
self.signer = signer
|
||||||
|
self.source = source
|
||||||
|
self.raw_content = []
|
||||||
|
self.content = []
|
||||||
|
self.cursor = 0
|
||||||
|
self.gas_oracle = gas_oracle
|
||||||
|
self.gas_factor = gas_factor
|
||||||
|
self.chain_spec = chain_spec
|
||||||
|
self.chain_id = chain_spec.chain_id()
|
||||||
|
self.processor = [TxProcessor()]
|
||||||
|
|
||||||
|
|
||||||
|
def load(self, process=True):
|
||||||
|
for processor in self.processor:
|
||||||
|
self.raw_content = processor.load(self.source)
|
||||||
|
if self.raw_content != None:
|
||||||
|
if process:
|
||||||
|
#try:
|
||||||
|
self.process()
|
||||||
|
#except Exception as e:
|
||||||
|
# raise TxSourceError('invalid source contents: {}'.format(str(e)))
|
||||||
|
return self.content
|
||||||
|
raise TxSourceError('unparseable source')
|
||||||
|
|
||||||
|
|
||||||
|
def process(self):
|
||||||
|
gas_data = self.gas_oracle.get_gas()
|
||||||
|
gas_price = gas_data[0]
|
||||||
|
for tx in self.raw_content:
|
||||||
|
tx_bytes = bytes.fromhex(strip_0x(tx))
|
||||||
|
tx = unpack(tx_bytes, self.chain_spec)
|
||||||
|
tx_gas_price_old = int(tx['gasPrice'])
|
||||||
|
if tx_gas_price_old < gas_price:
|
||||||
|
tx['gasPrice'] = gas_price
|
||||||
|
else:
|
||||||
|
tx['gasPrice'] = int(tx_gas_price_old * self.gas_factor)
|
||||||
|
if tx_gas_price_old == tx['gasPrice']:
|
||||||
|
tx['gasPrice'] += 1
|
||||||
|
tx_obj = EIP155Transaction(tx, tx['nonce'], self.chain_id)
|
||||||
|
new_tx_bytes = self.signer.sign_transaction_to_wire(tx_obj)
|
||||||
|
logg.debug('add tx {} with gas price changed from {} to {}: {}'.format(tx['hash'], tx_gas_price_old, tx['gasPrice'], new_tx_bytes.hex()))
|
||||||
|
self.content.append(new_tx_bytes)
|
||||||
|
|
||||||
|
|
||||||
|
def __iter__(self):
|
||||||
|
self.cursor = 0
|
||||||
|
return self
|
||||||
|
|
||||||
|
|
||||||
|
def __next__(self):
|
||||||
|
if self.cursor == len(self.content):
|
||||||
|
raise StopIteration()
|
||||||
|
tx = self.content[self.cursor]
|
||||||
|
self.cursor += 1
|
||||||
|
return tx
|
||||||
23
chaind_eth/cli/tx.py
Normal file
23
chaind_eth/cli/tx.py
Normal file
@ -0,0 +1,23 @@
|
|||||||
|
# standard imports
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logg = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
class TxProcessor:
|
||||||
|
|
||||||
|
def load(self, s):
|
||||||
|
contents = []
|
||||||
|
f = None
|
||||||
|
try:
|
||||||
|
f = open(s, 'r')
|
||||||
|
except FileNotFoundError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
contents = f.readlines()
|
||||||
|
f.close()
|
||||||
|
for i in range(len(contents)):
|
||||||
|
contents[i] = contents[i].rstrip()
|
||||||
|
return contents
|
||||||
|
|
||||||
|
def __str__(self):
|
||||||
|
return 'tx processor'
|
||||||
16
chaind_eth/data/config/config.ini
Normal file
16
chaind_eth/data/config/config.ini
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
[session]
|
||||||
|
socket_path =
|
||||||
|
runtime_dir =
|
||||||
|
id =
|
||||||
|
data_dir =
|
||||||
|
dispatch_delay = 4.0
|
||||||
|
|
||||||
|
[database]
|
||||||
|
engine =
|
||||||
|
name = chaind
|
||||||
|
driver =
|
||||||
|
user =
|
||||||
|
password =
|
||||||
|
host =
|
||||||
|
port =
|
||||||
|
debug = 0
|
||||||
4
chaind_eth/data/config/syncer/config.ini
Normal file
4
chaind_eth/data/config/syncer/config.ini
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
[syncer]
|
||||||
|
history_start = 0
|
||||||
|
skip_history = 0
|
||||||
|
loop_interval = 1
|
||||||
72
chaind_eth/dispatch.py
Normal file
72
chaind_eth/dispatch.py
Normal file
@ -0,0 +1,72 @@
|
|||||||
|
# standard imports
|
||||||
|
import logging
|
||||||
|
|
||||||
|
# external imports
|
||||||
|
from chainlib.eth.address import to_checksum_address
|
||||||
|
from chainlib.eth.tx import unpack
|
||||||
|
from chainlib.error import JSONRPCException
|
||||||
|
from chainqueue.enum import StatusBits
|
||||||
|
from chainqueue.sql.query import count_tx
|
||||||
|
from hexathon import strip_0x
|
||||||
|
from chainqueue.encode import TxNormalizer
|
||||||
|
|
||||||
|
#logg = logging.getLogger(__name__)
|
||||||
|
logg = logging.getLogger()
|
||||||
|
|
||||||
|
|
||||||
|
class Dispatcher:
|
||||||
|
|
||||||
|
status_inflight_mask = StatusBits.IN_NETWORK | StatusBits.FINAL
|
||||||
|
status_inflight_mask_match = StatusBits.IN_NETWORK
|
||||||
|
|
||||||
|
def __init__(self, chain_spec, adapter, limit=100):
|
||||||
|
self.address_counts = {}
|
||||||
|
self.chain_spec = chain_spec
|
||||||
|
self.adapter = adapter
|
||||||
|
self.limit = limit
|
||||||
|
self.tx_normalizer = TxNormalizer()
|
||||||
|
|
||||||
|
|
||||||
|
def __init_count(self, address, session):
|
||||||
|
c = self.address_counts.get(address)
|
||||||
|
if c == None:
|
||||||
|
c = self.limit - count_tx(self.chain_spec, address, self.status_inflight_mask, self.status_inflight_mask_match, session=session)
|
||||||
|
if c < 0:
|
||||||
|
c = 0
|
||||||
|
self.address_counts[address] = c
|
||||||
|
return c
|
||||||
|
|
||||||
|
|
||||||
|
def get_count(self, address, session):
|
||||||
|
address = self.tx_normalizer.wallet_address(address)
|
||||||
|
return self.__init_count(address, session)
|
||||||
|
|
||||||
|
|
||||||
|
def inc_count(self, address, session):
|
||||||
|
address = self.tx_normalizer.wallet_address(address)
|
||||||
|
self.__init_count(address, session)
|
||||||
|
self.address_counts[address] -= 1
|
||||||
|
|
||||||
|
|
||||||
|
def process(self, rpc, session):
|
||||||
|
c = 0
|
||||||
|
txs = self.adapter.upcoming(self.chain_spec, session=session)
|
||||||
|
for k in txs.keys():
|
||||||
|
signed_tx_bytes = bytes.fromhex(strip_0x(txs[k]))
|
||||||
|
tx_obj = unpack(signed_tx_bytes, self.chain_spec)
|
||||||
|
sender = to_checksum_address(tx_obj['from'])
|
||||||
|
address_count = self.get_count(sender, session)
|
||||||
|
if address_count == 0:
|
||||||
|
logg.debug('too many inflight txs for {}, skipping {}'.format(sender, k))
|
||||||
|
continue
|
||||||
|
logg.debug('processing tx {} {}'.format(k, txs[k]))
|
||||||
|
r = 0
|
||||||
|
try:
|
||||||
|
r = self.adapter.dispatch(self.chain_spec, rpc, k, txs[k], session)
|
||||||
|
except JSONRPCException as e:
|
||||||
|
logg.error('dispatch failed for {}: {}'.format(k, e))
|
||||||
|
continue
|
||||||
|
if r == 0:
|
||||||
|
self.inc_count(sender, session)
|
||||||
|
c += 1
|
||||||
|
return c
|
||||||
27
chaind_eth/filter.py
Normal file
27
chaind_eth/filter.py
Normal file
@ -0,0 +1,27 @@
|
|||||||
|
# standard imports
|
||||||
|
import logging
|
||||||
|
|
||||||
|
# external imports
|
||||||
|
from chainlib.status import Status
|
||||||
|
from chainqueue.sql.query import get_tx
|
||||||
|
from chainqueue.error import NotLocalTxError
|
||||||
|
from chainqueue.sql.state import set_final
|
||||||
|
|
||||||
|
logg = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class StateFilter:
|
||||||
|
|
||||||
|
def __init__(self, chain_spec):
|
||||||
|
self.chain_spec = chain_spec
|
||||||
|
|
||||||
|
|
||||||
|
def filter(self, conn, block, tx, session=None):
|
||||||
|
otx = None
|
||||||
|
try:
|
||||||
|
otx = get_tx(self.chain_spec, tx.hash, session=session)
|
||||||
|
except NotLocalTxError:
|
||||||
|
return False
|
||||||
|
logg.info('finalizing local tx {} with status {}'.format(tx.hash, tx.status))
|
||||||
|
status = tx.status != Status.SUCCESS
|
||||||
|
set_final(self.chain_spec, tx.hash, block=block.number, tx_index=tx.index, fail=status, session=session)
|
||||||
@ -120,6 +120,7 @@ chain_spec = ChainSpec.from_chain_str(config.get('CHAIN_SPEC'))
|
|||||||
rpc = chainlib.eth.cli.Rpc()
|
rpc = chainlib.eth.cli.Rpc()
|
||||||
conn = rpc.connect_by_config(config)
|
conn = rpc.connect_by_config(config)
|
||||||
|
|
||||||
|
logg.debug('error {}'.format(rpc.error_parser))
|
||||||
dsn = dsn_from_config(config)
|
dsn = dsn_from_config(config)
|
||||||
backend = SQLBackend(dsn, error_parser=rpc.error_parser, debug=config.true('DATABASE_DEBUG'))
|
backend = SQLBackend(dsn, error_parser=rpc.error_parser, debug=config.true('DATABASE_DEBUG'))
|
||||||
session_index_backend = SessionIndex(config.get('SESSION_ID'))
|
session_index_backend = SessionIndex(config.get('SESSION_ID'))
|
||||||
@ -135,6 +136,7 @@ def process_outgoing(chain_spec, adapter, rpc, limit=100):
|
|||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
|
havesends = 0
|
||||||
while True:
|
while True:
|
||||||
srvs = None
|
srvs = None
|
||||||
try:
|
try:
|
||||||
@ -75,7 +75,7 @@ chain_spec = ChainSpec.from_chain_str(config.get('CHAIN_SPEC'))
|
|||||||
dsn = dsn_from_config(config)
|
dsn = dsn_from_config(config)
|
||||||
logg.debug('dns {}'.format(dsn))
|
logg.debug('dns {}'.format(dsn))
|
||||||
SQLBackend.setup(dsn, debug=config.true('DATABASE_DEBUG'))
|
SQLBackend.setup(dsn, debug=config.true('DATABASE_DEBUG'))
|
||||||
rpc = EthHTTPConnection(url=config.get('RPC_PROVIDER'), chain_spec=chain_spec)
|
rpc = EthHTTPConnection(url=config.get('RPC_HTTP_PROVIDER'), chain_spec=chain_spec)
|
||||||
|
|
||||||
def register_filter_tags(filters, session):
|
def register_filter_tags(filters, session):
|
||||||
for f in filters:
|
for f in filters:
|
||||||
44
chainqueue/adapters/eth.py
Normal file
44
chainqueue/adapters/eth.py
Normal file
@ -0,0 +1,44 @@
|
|||||||
|
# standard imports
|
||||||
|
import logging
|
||||||
|
|
||||||
|
# external imports
|
||||||
|
from chainlib.eth.constant import ZERO_ADDRESS
|
||||||
|
from chainlib.eth.tx import (
|
||||||
|
unpack,
|
||||||
|
raw,
|
||||||
|
)
|
||||||
|
from hexathon import (
|
||||||
|
add_0x,
|
||||||
|
strip_0x,
|
||||||
|
)
|
||||||
|
|
||||||
|
# local imports
|
||||||
|
from chainqueue.adapters.sessionindex import SessionIndexAdapter
|
||||||
|
|
||||||
|
logg = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class EthAdapter(SessionIndexAdapter):
|
||||||
|
|
||||||
|
|
||||||
|
def translate(self, bytecode, chain_spec):
|
||||||
|
logg.debug('bytecode {}'.format(bytecode))
|
||||||
|
tx = unpack(bytecode, chain_spec)
|
||||||
|
tx['source_token'] = ZERO_ADDRESS
|
||||||
|
tx['destination_token'] = ZERO_ADDRESS
|
||||||
|
tx['from_value'] = tx['value']
|
||||||
|
tx['to_value'] = tx['value']
|
||||||
|
return tx
|
||||||
|
|
||||||
|
|
||||||
|
def dispatch(self, chain_spec, rpc, tx_hash, signed_tx, session=None):
|
||||||
|
o = raw(signed_tx)
|
||||||
|
r = self.backend.dispatch(chain_spec, rpc, tx_hash, o)
|
||||||
|
return r
|
||||||
|
|
||||||
|
|
||||||
|
# def cache(self, chain_spec):
|
||||||
|
# session = self.backend.create_session()
|
||||||
|
# r = self.backend.create(chain_spec, tx['nonce'], tx['from'], tx['hash'], add_0x(bytecode.hex()), session=session)
|
||||||
|
# session.close()
|
||||||
|
|
||||||
@ -1 +0,0 @@
|
|||||||
eth-erc20~=0.3.2
|
|
||||||
@ -1,5 +1,4 @@
|
|||||||
chaind~=0.3.0
|
chaind<=0.0.3,>=0.0.3a6
|
||||||
hexathon~=0.1.7
|
hexathon~=0.0.1a8
|
||||||
chainlib-eth~=0.3.0
|
chainlib-eth<=0.1.0,>=0.0.10a10
|
||||||
pyxdg~=0.27
|
eth-token-index<=0.3.0,>=0.2.4a1
|
||||||
funga-eth~=0.6.1
|
|
||||||
|
|||||||
@ -3,7 +3,7 @@
|
|||||||
set -e
|
set -e
|
||||||
set -x
|
set -x
|
||||||
set -a
|
set -a
|
||||||
export PYTHONPATH=${PYTHONPATH:-.}
|
export PYTHONPATH=${PYTHONPATH:.}
|
||||||
for f in `ls tests/*.py`; do
|
for f in `ls tests/*.py`; do
|
||||||
python $f
|
python $f
|
||||||
done
|
done
|
||||||
|
|||||||
33
setup.cfg
33
setup.cfg
@ -1,40 +1,41 @@
|
|||||||
[metadata]
|
[metadata]
|
||||||
name = chaind-eth
|
name = chaind-eth
|
||||||
version = 0.3.3
|
version = 0.0.3a5
|
||||||
description = Queue server for ethereum
|
description = Queue server for ethereum
|
||||||
author = Louis Holbrook
|
author = Louis Holbrook
|
||||||
author_email = dev@holbrook.no
|
author_email = dev@holbrook.no
|
||||||
url = https://git.defalsify.org/chaind-eth.git
|
url = https://gitlab.com/chaintools/chainqueue
|
||||||
keywords =
|
keywords =
|
||||||
dlt
|
cic
|
||||||
blockchain
|
|
||||||
cryptocurrency
|
cryptocurrency
|
||||||
ethereum
|
ethereum
|
||||||
|
solidarity
|
||||||
|
mutual_credit
|
||||||
classifiers =
|
classifiers =
|
||||||
Programming Language :: Python :: 3
|
Programming Language :: Python :: 3
|
||||||
Operating System :: OS Independent
|
Operating System :: OS Independent
|
||||||
Development Status :: 3 - Alpha
|
Development Status :: 3 - Alpha
|
||||||
Environment :: Console
|
Environment :: Console
|
||||||
Intended Audience :: Developers
|
Intended Audience :: Developers
|
||||||
License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)
|
License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
|
||||||
Topic :: Internet
|
Topic :: Internet
|
||||||
# Topic :: Blockchain :: EVM
|
# Topic :: Blockchain :: EVM
|
||||||
license = OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)
|
license = GPL3
|
||||||
licence_files =
|
licence_files =
|
||||||
LICENSE
|
LICENSE.txt
|
||||||
|
|
||||||
[options]
|
[options]
|
||||||
python_requires = >= 3.7
|
python_requires = >= 3.6
|
||||||
include_package_data = True
|
include_package_data = True
|
||||||
packages =
|
packages =
|
||||||
chaind.eth
|
chaind_eth
|
||||||
chaind.eth.runnable
|
chaind_eth.runnable
|
||||||
chaind.eth.cli
|
chaind_eth.cli
|
||||||
chaind.eth.token
|
chainqueue.adapters
|
||||||
|
|
||||||
[options.entry_points]
|
[options.entry_points]
|
||||||
console_scripts =
|
console_scripts =
|
||||||
chaind-eth-queuer = chaind.eth.runnable.queuer:main
|
chaind-eth-server = chaind_eth.runnable.server:main
|
||||||
chaind-eth-syncer = chaind.eth.runnable.syncer:main
|
chaind-eth-syncer = chaind_eth.runnable.syncer:main
|
||||||
chaind-eth-send = chaind.eth.runnable.send:main
|
chaind-eth-send = chaind_eth.runnable.send:main
|
||||||
#chaind-eth-resend = chaind_eth.runnable.resend:main
|
chaind-eth-resend = chaind_eth.runnable.resend:main
|
||||||
|
|||||||
15
setup.py
15
setup.py
@ -21,16 +21,6 @@ while True:
|
|||||||
test_requirements.append(l.rstrip())
|
test_requirements.append(l.rstrip())
|
||||||
f.close()
|
f.close()
|
||||||
|
|
||||||
erc20_requirements = []
|
|
||||||
f = open('erc20_requirements.txt', 'r')
|
|
||||||
while True:
|
|
||||||
l = f.readline()
|
|
||||||
if l == '':
|
|
||||||
break
|
|
||||||
erc20_requirements.append(l.rstrip())
|
|
||||||
f.close()
|
|
||||||
|
|
||||||
|
|
||||||
postgres_requirements = [
|
postgres_requirements = [
|
||||||
'psycopg2==2.8.6',
|
'psycopg2==2.8.6',
|
||||||
] + requirements
|
] + requirements
|
||||||
@ -40,8 +30,7 @@ setup(
|
|||||||
install_requires=requirements,
|
install_requires=requirements,
|
||||||
tests_require=test_requirements,
|
tests_require=test_requirements,
|
||||||
extras_require={
|
extras_require={
|
||||||
#'postgres': postgres_requirements,
|
'postgres': postgres_requirements,
|
||||||
#'sqlite': sqlite_requirements,
|
'sqlite': sqlite_requirements,
|
||||||
'erc20': erc20_requirements,
|
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|||||||
@ -1,10 +1,10 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=Chainqueue transaction dispatch interface
|
Description=Chainqueue transaction dispatch interface
|
||||||
#Requires=chaind-eth-sync@.service
|
Requires=chaind-eth-sync@.service
|
||||||
#After=chaind-eth-sync@.service
|
After=chaind-eth-sync@.service
|
||||||
|
|
||||||
[Service]
|
[Service]
|
||||||
Environment="SESSION_ID=%i"
|
Environment="SESSION_ID=%i"
|
||||||
Environment="SESSION_RUNTIME_DIR=/run/user/%U/chaind/eth"
|
Environment="SESSION_RUNTIME_DIR=/run/user/%U/chaind/eth"
|
||||||
ExecStart=%h/.local/bin/chaind-eth-tasker -v --session-id %i
|
ExecStart=%h/.local/bin/chaind-eth-server -v --session-id %i
|
||||||
Restart=on-failure
|
Restart=on-failure
|
||||||
|
|||||||
@ -4,5 +4,5 @@ Description=Ethereum chaind syncer
|
|||||||
[Service]
|
[Service]
|
||||||
Environment="SESSION_ID=%i"
|
Environment="SESSION_ID=%i"
|
||||||
Environment="SESSION_RUNTIME_DIR=/run/user/%U/chaind/eth"
|
Environment="SESSION_RUNTIME_DIR=/run/user/%U/chaind/eth"
|
||||||
ExecStart=%h/.local/bin/chaind-eth-syncer -v --session-id %i --head
|
ExecStart=%h/.local/bin/chaind-eth-syncer -v --session-id %i --skip-history
|
||||||
Restart=on-failure
|
Restart=on-failure
|
||||||
|
|||||||
@ -2,4 +2,3 @@ eth_tester==0.5.0b3
|
|||||||
py-evm==0.3.0a20
|
py-evm==0.3.0a20
|
||||||
rlp==2.0.1
|
rlp==2.0.1
|
||||||
coverage==5.5
|
coverage==5.5
|
||||||
jsonrpc_std~=0.1.0
|
|
||||||
|
|||||||
51
tests/chaind_eth_base.py
Normal file
51
tests/chaind_eth_base.py
Normal file
@ -0,0 +1,51 @@
|
|||||||
|
# standard imports
|
||||||
|
import os
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
# external imports
|
||||||
|
from chainsyncer.unittest.db import ChainSyncerDb
|
||||||
|
from chainqueue.unittest.db import ChainQueueDb
|
||||||
|
from chainlib.eth.unittest.ethtester import EthTesterCase
|
||||||
|
from chainqueue.adapters.eth import EthAdapter
|
||||||
|
from chainqueue.unittest.db import (
|
||||||
|
db_config,
|
||||||
|
dsn_from_config,
|
||||||
|
)
|
||||||
|
from chainqueue.sql.backend import SQLBackend
|
||||||
|
from chainlib.eth.address import to_checksum_address
|
||||||
|
from hexathon import add_0x
|
||||||
|
|
||||||
|
# local imports
|
||||||
|
from chaind_eth.chain import EthChainInterface
|
||||||
|
|
||||||
|
class TestBase(EthTesterCase):
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
super(TestBase, self).setUp()
|
||||||
|
self.db_chainsyncer = ChainSyncerDb(debug=bool(os.environ.get('DATABASE_DEBUG')))
|
||||||
|
self.session_chainsyncer = self.db_chainsyncer.bind_session()
|
||||||
|
|
||||||
|
self.db_chainqueue = ChainQueueDb(debug=bool(os.environ.get('DATABASE_DEBUG')))
|
||||||
|
self.session_chainqueue = self.db_chainqueue.bind_session()
|
||||||
|
|
||||||
|
self.interface = EthChainInterface()
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
self.session_chainsyncer.commit()
|
||||||
|
self.db_chainsyncer.release_session(self.session_chainsyncer)
|
||||||
|
self.session_chainqueue.commit()
|
||||||
|
self.db_chainqueue.release_session(self.session_chainqueue)
|
||||||
|
super(TestBase, self).tearDown()
|
||||||
|
|
||||||
|
|
||||||
|
class TestSQLBase(TestBase):
|
||||||
|
|
||||||
|
example_tx_hex = 'f8650d8405f5e10082520894ee38d3a40e177608d41978778206831f60dd0fa88204008077a040adee2ad0a0e566bced4b76a8899549e86719eb8866b87674b6fdc88479c201a030b3ca061bb330f4d78bc9cb8144c8e570339496f56b7809387de2ffeaa585d5'
|
||||||
|
example_tx = bytes.fromhex(example_tx_hex)
|
||||||
|
example_tx_sender = add_0x(to_checksum_address('eb3907ecad74a0013c259d5874ae7f22dcbcc95c'))
|
||||||
|
dsn = dsn_from_config(db_config)
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
super(TestSQLBase, self).setUp()
|
||||||
|
self.backend = SQLBackend(self.dsn, debug=bool(os.environ.get('DATABASE_DEBUG')))
|
||||||
|
self.adapter = EthAdapter(self.backend)
|
||||||
26
tests/test_adapter.py
Normal file
26
tests/test_adapter.py
Normal file
@ -0,0 +1,26 @@
|
|||||||
|
# stanndard imports
|
||||||
|
import logging
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
# external imports
|
||||||
|
from chainlib.chain import ChainSpec
|
||||||
|
|
||||||
|
# test imports
|
||||||
|
from tests.chaind_eth_base import TestSQLBase
|
||||||
|
|
||||||
|
logging.basicConfig(level=logging.DEBUG)
|
||||||
|
|
||||||
|
|
||||||
|
class TestAdapter(TestSQLBase):
|
||||||
|
|
||||||
|
def test_eth_adapter_translate(self):
|
||||||
|
self.adapter.translate(self.example_tx, self.chain_spec)
|
||||||
|
# succesful decode means translate is working, no further checks needed
|
||||||
|
|
||||||
|
|
||||||
|
def test_eth_adapter_add(self):
|
||||||
|
self.adapter.add(self.example_tx_hex, self.chain_spec, session=self.session_chainqueue)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
unittest.main()
|
||||||
38
tests/test_chain.py
Normal file
38
tests/test_chain.py
Normal file
@ -0,0 +1,38 @@
|
|||||||
|
# standard imports
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
# external imports
|
||||||
|
from chainlib.eth.gas import (
|
||||||
|
RPCGasOracle,
|
||||||
|
Gas,
|
||||||
|
)
|
||||||
|
from chainlib.eth.nonce import (
|
||||||
|
RPCNonceOracle,
|
||||||
|
)
|
||||||
|
from chainlib.eth.tx import (
|
||||||
|
TxFormat,
|
||||||
|
unpack,
|
||||||
|
)
|
||||||
|
from hexathon import (
|
||||||
|
strip_0x,
|
||||||
|
)
|
||||||
|
|
||||||
|
# test imports
|
||||||
|
from tests.chaind_eth_base import TestBase
|
||||||
|
|
||||||
|
class TestChain(TestBase):
|
||||||
|
|
||||||
|
def test_chain_interface(self):
|
||||||
|
gas_oracle = RPCGasOracle(conn=self.rpc)
|
||||||
|
nonce_oracle = RPCNonceOracle(self.accounts[0], conn=self.rpc)
|
||||||
|
c = Gas(self.chain_spec, signer=self.signer, gas_oracle=gas_oracle, nonce_oracle=nonce_oracle)
|
||||||
|
(tx_hash, tx_raw_rlp_signed) = c.create(self.accounts[0], self.accounts[1], 1024, tx_format=TxFormat.RLP_SIGNED)
|
||||||
|
|
||||||
|
tx_raw_rlp_signed_bytes = bytes.fromhex(strip_0x(tx_raw_rlp_signed))
|
||||||
|
tx_src = unpack(tx_raw_rlp_signed_bytes, self.chain_spec)
|
||||||
|
tx_src = self.interface.src_normalize(tx_src)
|
||||||
|
assert tx_src['gas_price'] == tx_src['gasPrice']
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
unittest.main()
|
||||||
52
tests/test_dispatch.py
Normal file
52
tests/test_dispatch.py
Normal file
@ -0,0 +1,52 @@
|
|||||||
|
# stanndard imports
|
||||||
|
import logging
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
# external imports
|
||||||
|
from hexathon import strip_0x
|
||||||
|
from chainlib.eth.tx import (
|
||||||
|
unpack,
|
||||||
|
TxFormat,
|
||||||
|
)
|
||||||
|
from chainqueue.sql.query import get_tx
|
||||||
|
from chainqueue.enum import StatusBits
|
||||||
|
from chainlib.eth.gas import (
|
||||||
|
RPCGasOracle,
|
||||||
|
Gas,
|
||||||
|
)
|
||||||
|
from chainlib.eth.nonce import (
|
||||||
|
RPCNonceOracle,
|
||||||
|
)
|
||||||
|
|
||||||
|
# local imports
|
||||||
|
from chaind_eth.dispatch import Dispatcher
|
||||||
|
|
||||||
|
# test imports
|
||||||
|
from tests.chaind_eth_base import TestSQLBase
|
||||||
|
|
||||||
|
logging.basicConfig(level=logging.DEBUG)
|
||||||
|
|
||||||
|
|
||||||
|
class TestDispatcher(TestSQLBase):
|
||||||
|
|
||||||
|
def test_dispatch_process(self):
|
||||||
|
gas_oracle = RPCGasOracle(conn=self.rpc)
|
||||||
|
nonce_oracle = RPCNonceOracle(self.accounts[0], conn=self.rpc)
|
||||||
|
c = Gas(self.chain_spec, signer=self.signer, gas_oracle=gas_oracle, nonce_oracle=nonce_oracle)
|
||||||
|
(tx_hash, tx_raw_rlp_signed) = c.create(self.accounts[0], self.accounts[1], 1024, tx_format=TxFormat.RLP_SIGNED)
|
||||||
|
|
||||||
|
tx_raw_rlp_signed_bytes = bytes.fromhex(strip_0x(tx_raw_rlp_signed))
|
||||||
|
dispatcher = Dispatcher(self.chain_spec, self.adapter, 1)
|
||||||
|
self.adapter.add(tx_raw_rlp_signed, self.chain_spec, session=self.session_chainqueue)
|
||||||
|
#self.assertEqual(dispatcher.get_count(self.example_tx_sender, self.session_chainqueue), 1)
|
||||||
|
self.assertEqual(dispatcher.get_count(self.accounts[0], self.session_chainqueue), 1)
|
||||||
|
|
||||||
|
dispatcher.process(self.rpc, self.session_chainqueue)
|
||||||
|
tx_obj = unpack(tx_raw_rlp_signed_bytes, self.chain_spec)
|
||||||
|
o = get_tx(self.chain_spec, tx_obj['hash'], session=self.session_chainqueue)
|
||||||
|
self.assertGreater(o['status'] & StatusBits.IN_NETWORK, 0)
|
||||||
|
self.assertEqual(dispatcher.get_count(self.accounts[0], self.session_chainqueue), 0)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
unittest.main()
|
||||||
101
tests/test_filter.py
Normal file
101
tests/test_filter.py
Normal file
@ -0,0 +1,101 @@
|
|||||||
|
# standard imports
|
||||||
|
import unittest
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
|
||||||
|
# external imports
|
||||||
|
from potaahto.symbols import snake_and_camel
|
||||||
|
from hexathon import (
|
||||||
|
strip_0x,
|
||||||
|
)
|
||||||
|
from chainqueue.adapters.eth import EthAdapter
|
||||||
|
from chainqueue.unittest.db import (
|
||||||
|
db_config,
|
||||||
|
dsn_from_config,
|
||||||
|
)
|
||||||
|
from chainqueue.sql.backend import SQLBackend
|
||||||
|
from chainqueue.enum import is_alive
|
||||||
|
from chainqueue.sql.query import get_tx
|
||||||
|
from chainlib.eth.gas import (
|
||||||
|
RPCGasOracle,
|
||||||
|
Gas,
|
||||||
|
)
|
||||||
|
from chainlib.eth.nonce import (
|
||||||
|
RPCNonceOracle,
|
||||||
|
)
|
||||||
|
from chainlib.eth.tx import (
|
||||||
|
TxFormat,
|
||||||
|
raw,
|
||||||
|
unpack,
|
||||||
|
receipt,
|
||||||
|
Tx,
|
||||||
|
)
|
||||||
|
from chainlib.eth.block import (
|
||||||
|
block_by_hash,
|
||||||
|
Block,
|
||||||
|
)
|
||||||
|
from chainqueue.sql.state import (
|
||||||
|
set_sent,
|
||||||
|
set_reserved,
|
||||||
|
set_ready,
|
||||||
|
)
|
||||||
|
|
||||||
|
# local imports
|
||||||
|
from chaind_eth.filter import StateFilter
|
||||||
|
from chaind_eth.chain import EthChainInterface
|
||||||
|
|
||||||
|
# test imports
|
||||||
|
from tests.chaind_eth_base import TestBase
|
||||||
|
|
||||||
|
logging.basicConfig(level=logging.DEBUG)
|
||||||
|
logg = logging.getLogger()
|
||||||
|
|
||||||
|
|
||||||
|
class TestFilter(TestBase):
|
||||||
|
|
||||||
|
def test_filter(self):
|
||||||
|
gas_oracle = RPCGasOracle(conn=self.rpc)
|
||||||
|
nonce_oracle = RPCNonceOracle(self.accounts[0], conn=self.rpc)
|
||||||
|
c = Gas(self.chain_spec, signer=self.signer, gas_oracle=gas_oracle, nonce_oracle=nonce_oracle)
|
||||||
|
(tx_hash, tx_raw_rlp_signed) = c.create(self.accounts[0], self.accounts[1], 1024, tx_format=TxFormat.RLP_SIGNED)
|
||||||
|
o = raw(tx_raw_rlp_signed)
|
||||||
|
self.rpc.do(o)
|
||||||
|
|
||||||
|
#o = receipt(tx_hash)
|
||||||
|
o = self.interface.tx_receipt(tx_hash)
|
||||||
|
rcpt = self.rpc.do(o)
|
||||||
|
|
||||||
|
#o = block_by_hash(rcpt['block_hash'])
|
||||||
|
o = self.interface.block_by_number(rcpt['block_number'])
|
||||||
|
block_src = self.rpc.do(o)
|
||||||
|
#block = Block(block_src)
|
||||||
|
block = self.interface.block_from_src(block_src)
|
||||||
|
|
||||||
|
dsn = dsn_from_config(db_config)
|
||||||
|
backend = SQLBackend(dsn, debug=bool(os.environ.get('DATABASE_DEBUG')))
|
||||||
|
adapter = EthAdapter(backend)
|
||||||
|
|
||||||
|
tx_raw_rlp_signed_bytes = bytes.fromhex(strip_0x(tx_raw_rlp_signed))
|
||||||
|
#adapter.add(tx_raw_rlp_signed_bytes, self.chain_spec, session=self.session_chainqueue)
|
||||||
|
adapter.add(tx_raw_rlp_signed, self.chain_spec, session=self.session_chainqueue)
|
||||||
|
|
||||||
|
set_ready(self.chain_spec, tx_hash, session=self.session_chainqueue)
|
||||||
|
set_reserved(self.chain_spec, tx_hash, session=self.session_chainqueue)
|
||||||
|
set_sent(self.chain_spec, tx_hash, session=self.session_chainqueue)
|
||||||
|
|
||||||
|
tx_src = unpack(tx_raw_rlp_signed_bytes, self.chain_spec)
|
||||||
|
tx_src = self.interface.src_normalize(tx_src)
|
||||||
|
tx = Tx(tx_src, block=block, rcpt=rcpt)
|
||||||
|
|
||||||
|
tx_repr = get_tx(self.chain_spec, tx_hash, session=self.session_chainqueue)
|
||||||
|
assert is_alive(tx_repr['status'])
|
||||||
|
|
||||||
|
fltr = StateFilter(self.chain_spec)
|
||||||
|
fltr.filter(self.rpc, block, tx, session=self.session_chainqueue)
|
||||||
|
|
||||||
|
tx_repr = get_tx(self.chain_spec, tx_hash, session=self.session_chainqueue)
|
||||||
|
assert not is_alive(tx_repr['status'])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
unittest.main()
|
||||||
15
tests/test_helo.py
Normal file
15
tests/test_helo.py
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
# standard imports
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
# test imports
|
||||||
|
from tests.chaind_eth_base import TestBase
|
||||||
|
|
||||||
|
|
||||||
|
class TestHelo(TestBase):
|
||||||
|
|
||||||
|
def test_helo(self):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
unittest.main()
|
||||||
@ -1,73 +0,0 @@
|
|||||||
# standard imports
|
|
||||||
import os
|
|
||||||
import tempfile
|
|
||||||
import unittest
|
|
||||||
import shutil
|
|
||||||
import logging
|
|
||||||
import hashlib
|
|
||||||
|
|
||||||
# external imports
|
|
||||||
from chainlib.chain import ChainSpec
|
|
||||||
from chainqueue.cache import CacheTokenTx
|
|
||||||
from chainlib.error import RPCException
|
|
||||||
from chainlib.status import Status as TxStatus
|
|
||||||
from chaind.unittest.fs import TestChaindFsBase
|
|
||||||
from chaind.driver import QueueDriver
|
|
||||||
from chaind.filter import StateFilter
|
|
||||||
from chainlib.eth.gas import Gas
|
|
||||||
from jsonrpc_std.parse import jsonrpc_validate_dict
|
|
||||||
from hexathon import strip_0x
|
|
||||||
|
|
||||||
# local imports
|
|
||||||
from chaind.eth.cache import EthCacheTx
|
|
||||||
from chaind.eth.dispatch import EthDispatcher
|
|
||||||
|
|
||||||
logging.basicConfig(level=logging.DEBUG)
|
|
||||||
logg = logging.getLogger()
|
|
||||||
|
|
||||||
|
|
||||||
class MockConn:
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
self.fails = []
|
|
||||||
self.last = None
|
|
||||||
|
|
||||||
|
|
||||||
def add_fail(self, v):
|
|
||||||
self.fails.append(v)
|
|
||||||
|
|
||||||
|
|
||||||
def do(self, v):
|
|
||||||
if v in self.fails:
|
|
||||||
raise RuntimeError(v)
|
|
||||||
v = jsonrpc_validate_dict(v)
|
|
||||||
if v['method'] != 'eth_sendRawTransaction':
|
|
||||||
raise ValueError('unsupported method {}'.format(v['method']))
|
|
||||||
self.last = v['params'][0]
|
|
||||||
|
|
||||||
|
|
||||||
class TestEthChaindFs(TestChaindFsBase):
|
|
||||||
|
|
||||||
def setUp(self):
|
|
||||||
self.cache_adapter = EthCacheTx
|
|
||||||
self.conn = MockConn()
|
|
||||||
self.dispatcher = EthDispatcher(self.conn)
|
|
||||||
super(TestEthChaindFs, self).setUp()
|
|
||||||
|
|
||||||
|
|
||||||
def test_deserialize(self):
|
|
||||||
data = "f8610d2a82520894eb3907ecad74a0013c259d5874ae7f22dcbcc95c8204008078a0ddbebd76701f6531e5ea42599f890268716e2bb38e3e125874f47595c2338049a00f5648d17b20efac8cb7ff275a510ebef6815e1599e29067821372b83eb1d28c" # valid RLP example data
|
|
||||||
hsh = self.adapter.put(data)
|
|
||||||
v = self.adapter.get(hsh)
|
|
||||||
self.assertEqual(data, v)
|
|
||||||
|
|
||||||
|
|
||||||
def test_dispatch(self):
|
|
||||||
data = "f8610d2a82520894eb3907ecad74a0013c259d5874ae7f22dcbcc95c8204008078a0ddbebd76701f6531e5ea42599f890268716e2bb38e3e125874f47595c2338049a00f5648d17b20efac8cb7ff275a510ebef6815e1599e29067821372b83eb1d28c" # valid RLP example data
|
|
||||||
hsh = self.adapter.put(data)
|
|
||||||
self.adapter.dispatch(hsh)
|
|
||||||
self.assertEqual(strip_0x(self.conn.last), strip_0x(data))
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
unittest.main()
|
|
||||||
Loading…
Reference in New Issue
Block a user