adding cic-eth as sub dir

This commit is contained in:
2021-02-01 09:12:51 -08:00
parent ed3991e997
commit a4587deac5
317 changed files with 819441 additions and 0 deletions

View File

@@ -0,0 +1,5 @@
all: html
.PHONY: html
html:
makeinfo --html --output html index.texi

View File

@@ -0,0 +1,34 @@
@node Python API
@chapter Python API
@section Installation
A @code{setuptools} file is provided to install @code{cic-eth} as a normal python package.
The API will send tasks to the celery and immediately return task @code{uuid}. If an HTTP callback is provided, the result of the operation will be sent to it as a @code{POST} request with @code{application/json} content-type, with the following payload as pseudo-schema:
@verbatim
{
'root': <string: uuid of topmost task in the chain>,
'status': <integer: anything but 0 is error>,
'message': <contextual data payload>,
'result': <return value of last subtask if success, or uuid of last subtask on failure>
}
@end verbatim
@subsection API calls
Please refer to inline code docstrings in @file{cic_eth/api.py} for details on API function signatures and behaviors.
@itemize
@item
@code{create_account(...)} - Create a new account and add the private key to the keystore
@item
@code{balance(...)} - Query the token balance of an address
@item
@code{transfer(...)} - Transfer tokens between addresses
@item
@code{convert(...)} - Convert between tokens for same address
@item
@code{convert_transfer(...)} - Convert tokens and send destination token to a specified address
@end itemize

View File

@@ -0,0 +1,28 @@
\input texinfo
@settitle Grassroots Economics CIC ETH 0.4.1
@copying
Released 2020 under GPL3
@end copying
@titlepage
@title Community Inclusion Currency Network
@author Louis Holbrook
@end titlepage
@c
@contents
@ifnottex
@node Top
@top Grassroots Economics CIC ETH
This document describes microservices that send transactions to and process events from an EVM blockchain.
@end ifnottex
@include overview.texi
@include setup.texi
@include run.texi
@include api.texi

View File

@@ -0,0 +1,42 @@
@node Overview
@chapter Overview
This is the core component for the CIC token network system. It provides a task worker infrastructure to interact with the Ethereum blockchain, aswell as daemons to monitor transactions.
@section Reading this document
Any paths in this document are relative to the root of the cic-eth git repository.
@section Contents
@itemize @bullet
@item Python API that dispatches celery tasks for high-level actions
@item Daemon for monitoring the state of and retrying outgoing transactions
@item Daemon for monitoring new transactions of interest confirmed onchain
@end itemize
@section Dependencies
Python dependencies can be acquired by running the usual python setup and/or using the provided requiremens.txt file.
The application has the following non-python dependencies:
@itemize @bullet
@item
@strong{PostgreSQL}: Database backend.
@item
@strong{Redis}: Broker and results for Celery workers.
@item
@strong{Bancor contracts}: A clone of the Bancor contact repository
@item
@strong{Ethereum node}: A synced ethereum node with keystore
@end itemize
@section Caveats
The application @emph{must not} use private keys for which transactions may be generated from other sources.
It application retrieves the transaction nonce for every transaction directly from the network. Any transaction submitted to it are passed on to the network with sequential nonces. The caller code needs to keep this in mind when considering whether to allow the account holder to submit additional transaction tasks before the current has been completed.
@section Known issues
The author has not yet figured out how to get @code{pytest-celery} to work with queues. Currently the functional tests hang when trying to run them.

View File

@@ -0,0 +1,34 @@
@node Running the application
@chapter Running the application
@section Celery worker
The helper script @file{scripts/cic-eth-tasker.py} will connect to redis and the ethereum node and spawn a worker for all celery tasks.
@section Transaction controller
The celery task chains the process outgoing transactions only send the transactions to the network, after which their responsibility ends. And further processing is handled by @file{scripts/cic-eth-manager.py}.
@subsection Monitor transaction state
Records of transactions are added to the database, and the daemon looks for confirmations in block for these transaction, updating their state accordingly. The resulting state may @emph{only} by either @strong{SUCCESS} or @strong{REVERT}. A transaction is any of these states is finalized and won't be retried.
@subsection Perform transfer after convert
Handles the compound workflow where sender with token @code{A} wishes to send tokens to recipient with token @code{B}.
The task worker only submits the convert transaction, because the actual amount of received tokens is not known before the convert takes place. The full amount of recevied tokens will be transferred after the successful convert.
Convert-then-transfer tasks are recorded separately, and if the controller receive a confirmed convert transaction that has a local record, it submits a new transfer task to the task workers.
@subsection Retry after timeout
@strong{Not yet implemented!}.
If a transaction lingers too long without being confirmed, the controller will attempt to re-submit the task and all its dependent tasks with a higher gas price.
@section Transaction monitor
@file{scripts/cic-eth-track.py} looks for token transfer or convert transaction for all known tokens on the network, and stores them in a local database cache.

View File

@@ -0,0 +1,98 @@
@node Setup
@chapter Setup
@section Prerequisites
@itemize
@item A running ETH provider with:
@itemize
@item Deployed bancor contracts
@item Sufficient balances for system accounts
@end itemize
@item A running ETH signer provider, that can handle the following 'web3' JSON-RPC calls:
@itemize
@item @command{personal_newAccount}
@item @command{eth_signTransaction}
@end itemize
@end itemize
@section Database migration
Execute @code{alembic upgrade head} in @file{cic_eth/db/migrations} to generate the schema in the database.
@section Configuration settings
@file{cic-eth} uses configuration files in @code{'ini'} format. All names are translated to upper case letters, and all values are concatenated to sections with a @key{_}. Example:
@verbatim
[foo]
bar_baz = 42
@end verbatim
The above entry becomes @code{FOO_BAR_BAZ} with value @code{42}
@subsection Database
@table @code
@item DATABASE_NAME
@item DATABASE_USER
@item DATABASE_PASSWORD
@item DATABASE_HOST
@item DATABASE_PORT
Postgres database settings
@end table
@subsection Redis
@table @code
@item REDIS_BROKER_URL
Redis settings
@end table
@subsection SSL
@table @code
@item SSL_ENABLE_CLIENT
Use SSL client certification for outgoing web requests (boolean)
@item SSL_CERT_FILE
Absolute path to client certificate file
@item SSL_KEY_FILE
Absolute path to client key file
@item SSL_PASSWORD
Password to unlock client key
@item SSL_CA_FILE
The client certificate's authority chain
@end table
@subsection Ethereum
@table @code
@item ETH_PROVIDER
URL to JSON-RPC API provider
@item ETH_GAS_PROVIDER_ADDRESS
Address providing gas to fund transactions for accounts in keystore
@end table
@subsection Bancor
@table @code
@item BANCOR_REGISTRY_ADDRESS
Contract address for the ContractRegistry contract
@item BANCOR_DIR
Absolute path to root of the Bancor Solidity contract GIT repository
@end table
@subsection Syncer
@table @code
@item SYNCER_LOOP_DELAY
Delay in seconds to wait between polls when no data has been found
@end table
@subsection Signer
@table @code
@item SIGNER_SECRET (development only)
Key used by keystore to encrypt secrets
@item SIGNER_PASSWORD
Password for additional private key encryption on keystore
@end table

View File

@@ -0,0 +1,26 @@
@chapter Transactions
@section States
@subsection Send transaction to network
@multitable @columnfractions .35 .15 .15 .35
@headitem Item @tab Exception @tab Action
@item Chain node unreachable
@tab ChainUnavailableError
@tab Retry: unlimited, backoff
@item Cannot save tx state to database
@tab DatabaseUnavailableError
@tab Crash: Notify by email
@item Signer unreachable
@tab SignerUnavailableError
@tab Retry: unlimited, backoff
@item Unknown address
@tab SignerAccountUnknownError
@tab Fail