Merge branch 'master' of gitlab.com:grassrootseconomics/cic-internal-integration into philip/multi-token-v1

.gitlab/issue_templates/bug.md

@@ -1,44 +0,0 @@
-<!---
-Please read this!
-
-Before opening a new issue, make sure to search for keywords in the issues
-filtered by the "bug" label:
-
-- https://gitlab.com/groups/grassrootseconomics/-/issues?scope=all&state=all&label_name[]=bug
-
-and verify the issue you're about to submit isn't a duplicate.
---->
-
-### Summary
-
-<!-- Summarize the bug encountered concisely. -->
-
-### Steps to reproduce
-
-<!-- Describe how one can reproduce the issue - this is very important. Please use an ordered list. -->
-
-### Example Project
-
-<!-- If possible, please create an example project here on GitLab.com that exhibits the problematic 
-behavior, and link to it here in the bug report. If you are using an older version of GitLab, this 
-will also determine whether the bug is fixed in a more recent version. -->
-
-### What is the current *bug* behavior?
-
-<!-- Describe what actually happens. -->
-
-### What is the expected *correct* behavior?
-
-<!-- Describe what you should see instead. -->
-
-### Relevant logs and/or screenshots
-
-<!-- Paste any relevant logs - please use code blocks (```) to format console output, logs, and code
- as it's tough to read otherwise. -->
-
-
-### Possible fixes
-
-<!-- If you can, link to the line of code that might be responsible for the problem. -->
-
-/label ~"bug"

CODE_OF_CONDUCT.md

@@ -1,6 +1,4 @@
-# Code of Conduct
-
-By contributing toward this project you agree to the following code of conduct
+# Contributor Covenant Code of Conduct
 
 ## Our Pledge
 
@@ -16,21 +14,21 @@ appearance, race, religion, or sexual identity and orientation.
 Examples of behavior that contributes to creating a positive environment
 include:
 
-- Using welcoming and inclusive language
-- Being respectful of differing viewpoints and experiences
-- Gracefully accepting constructive criticism
-- Focusing on what is best for the community
-- Showing empathy towards other community members
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
 
 Examples of unacceptable behavior by participants include:
 
-- The use of sexualized language or imagery and unwelcome sexual attention or
+* The use of sexualized language or imagery and unwelcome sexual attention or
   advances
-- Trolling, insulting/derogatory comments, and personal or political attacks
-- Public or private harassment
-- Publishing others' private information, such as a physical or electronic
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
   address, without explicit permission
-- Other conduct which could reasonably be considered inappropriate in a
+* Other conduct which could reasonably be considered inappropriate in a
   professional setting
 
 ## Our Responsibilities
@@ -57,7 +55,7 @@ a project may be further defined and clarified by project maintainers.
 ## Enforcement
 
 Instances of abusive, harassing, or otherwise unacceptable behavior may be
-reported by contacting the project team at info@grassecon.org . All
+reported by contacting the project team at [INSERT EMAIL ADDRESS]. All
 complaints will be reviewed and investigated and will result in a response that
 is deemed necessary and appropriate to the circumstances. The project team is
 obligated to maintain confidentiality with regard to the reporter of an incident.
@@ -77,8 +75,9 @@ available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.ht
 For answers to common questions about this code of conduct, see
 https://www.contributor-covenant.org/faq
 
+
+
+
 ...Try to keep in mind the immortal
 words of Bill and Ted, "Be excellent to each other."
 
-These documents are licensed under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) https://creativecommons.org/licenses/by-sa/4.0/
-And further any software Grassrotos Economics supports is under licenced under GNU GENERAL PUBLIC LICENSE Version 3 https://www.gnu.org/licenses/

CONTRIBUTING.md

@@ -1,66 +1,16 @@
-# Contribution Guidelines
+Hello and welcome to the CIC Stack repository. Targeted for use with the ethereum virtual machine and a ussd capable telecom provider. 
 
-**Karibu sana!** Hello and welcome to the Grassroots Economics [project](https://gitlab.com/grassrootseconomics). This guide will help you understand the overall
-organization of the CIC project to get you started with contributing to the project. You'll be able to pick up issues, write code to fix them, and get your work reviewed and merged.
+__To request a change to the code please fork this repository and sumbit a merge request.__
 
-## Table of Contents
+__If there is a Grassroots Economics Kanban Issue please include that in our MR it will help us track contributions. Karibu sana!__
 
-- [Prerequisites](#Prerequisites)
-- [Scope of contributions](#Scope-of-contributions)
-- [Issues](#Issues)
-  - [Bugs](#Bugs)
-  - [Good first issues](#Good-first-issues)
-- [Git guidelines](#Git-guidelines)
-- [Merge request processs](#Merge-request-processs)
-- [Environment setup](#Environment-setup)
-- [Code style](#Code-style)
-- [Building-and-testing](#Building-and-testing)
+__Visit the Development Kanban board here: https://gitlab.com/grassrootseconomics/cic-internal-integration/-/boards/2419764__
 
-## Prerequisites
+__Ask a question in our dev chat:__
 
-Before contributing to the CIC Stack, ensure you have:
+[Mattermost](https://chat.grassrootseconomics.net/cic/channels/dev)
 
-- Read and accepted to abide by the Code of Conduct
-<!-- TODO: Add link to CoC -->
-- Read and accepted the Developers Certificate of Origin
-<!-- TODO: Add link to DCO -->
+[Discord](https://discord.gg/XWunwAsX)
 
-## Scope of contributions
+[Matrix, IRC soon?]
 
-We generally welcome all sorts or contributions. No contribution is too small. We gladly accept contributions such as:
-
-- Documentation improvements including inline documentation or even minor typos
-- Answering questions in issue discussions and pull requests
-- Fixing known bugs on our issue tracker
-- Reporting unknown bugs
-<!-- TODO: Add resposible disclosure guidelines -->
-- Securty issues through responsible disclosure
-
-## Issues
-
-Use the existing issue template and issue tags to correctly catagorize and issue when creating issues. Make sure to search for duplicated before creating a new issue.
-
-### Bugs
-
-If a bug is encountered and you can reproduce it:
-
-- Define its priority with an issue tag
-- Describe the steps to reproduce it while providing the version of the bugged service/library and your environment setup
-
-### Good first issues
-
-These are pre-tagged issues which are friendly (low barrier to entry and clearly defined tasks) for new contributors to get aquinted to the project.
-
-## Git guideleines
-
-<!-- TODO: Review after finalzing branching or forking model -->
-
-## Merge request process
-
-<!-- TODO: Review after core_contrib -->
-
-## Environment setup
-
-## Code style
-
-## Building and testing

README.md

@@ -1,24 +1,41 @@
-## Community Inclusion Currency Stack (CIC Stack)
+# cic-internal-integration
 
-A custodial wallet and blockchain bidirectional interface engine for community inclusion currencies. Check the [CIC Stack summary](https://docs.grassecon.org/software) page for more information.
+## Getting started 
 
-### Contributing
+This repo uses docker-compose and docker buildkit. Set the following environment variables to get started:
 
-See our [contribution guidelines](https://docs.grassecon.org/community/contrib/) for more details on:
 
-- Environment setup
-- Source Code Management (SCM) guidelines
-- Filing issues
+```
+export COMPOSE_DOCKER_CLI_BUILD=1
+export DOCKER_BUILDKIT=1
+```
 
-#### Code of Conduct
+start services, database, redis and local ethereum node
+```
+docker-compose up -d
+```
 
-This project is released with a [Code of Conduct](https://docs.grassecon.org/community/conduct/). By participating in this project you agree to abide by its terms.
+Run app/contract-migration to deploy contracts
+```
+RUN_MASK=3 docker-compose up contract-migration
+```
 
-### Community
+stop cluster
+```
+docker-compose down
+```
 
-- [Mattermost](https://chat.grassrootseconomics.net/cic/channels/dev)
-- [Discord](https://discord.gg/ud32KMgH76)
+stop cluster and delete data
+```
+docker-compose down -v --remove-orphans
+```
 
-### License
+rebuild an images
+```
+docker-compose up --build <service_name>
+```
 
-Licensed under [GNU AGPLv3](https://gitlab.com/grassrootseconomics/cic-internal-integration/-/blob/master/LICENSE).
+to delete the buildkit cache
+```
+docker builder prune --filter type=exec.cachemount
+```

apps/cic-cache/cic_cache/cli/arg.py

@@ -14,7 +14,7 @@ class ArgumentParser(BaseArgumentParser):
         if local_arg_flags & CICFlag.CELERY:
             self.add_argument('-q', '--celery-queue', dest='celery_queue', type=str, default='cic-cache', help='Task queue')
         if local_arg_flags & CICFlag.SYNCER:
-            self.add_argument('--offset', type=int, help='Start block height for initial history sync')
+            self.add_argument('--offset', type=int, default=0, help='Start block height for initial history sync')
             self.add_argument('--no-history', action='store_true', dest='no_history', help='Skip initial history sync')
         if local_arg_flags & CICFlag.CHAIN:
             self.add_argument('-r', '--registry-address', type=str, dest='registry_address', help='CIC registry contract address')

apps/cic-cache/cic_cache/runnable/daemons/tracker.py

@@ -95,10 +95,10 @@ def main():
     syncer_backends = SQLBackend.resume(chain_spec, block_offset)
 
     if len(syncer_backends) == 0:
-        initial_block_start = int(config.get('SYNCER_OFFSET'))
-        initial_block_offset = int(block_offset)
+        initial_block_start = config.get('SYNCER_OFFSET')
+        initial_block_offset = block_offset
         if config.get('SYNCER_NO_HISTORY'):
-            initial_block_start = initial_block_offset
+            initial_block_start = block_offset
             initial_block_offset += 1
         syncer_backends.append(SQLBackend.initial(chain_spec, initial_block_offset, start_block_height=initial_block_start))
         logg.info('found no backends to resume, adding initial sync from history start {} end {}'.format(initial_block_start, initial_block_offset))

apps/cic-cache/docker/Dockerfile

@@ -4,7 +4,7 @@ FROM $DOCKER_REGISTRY/cic-base-images:python-3.8.6-dev-e8eb2ee2
 
 COPY requirements.txt .
 
-ARG EXTRA_PIP_INDEX_URL="https://pip.grassrootseconomics.net"
+ARG EXTRA_PIP_INDEX_URL="https://pip.grassrootseconomics.net:8433"
 ARG EXTRA_PIP_ARGS=""
 ARG PIP_INDEX_URL="https://pypi.org/simple"
 

apps/cic-cache/docker/db.sh

@@ -2,5 +2,5 @@
 
 set -e
 >&2 echo executing database migration
-python scripts/migrate.py --migrations-dir /usr/local/share/cic-cache/alembic -vv
+python scripts/migrate.py -c /usr/local/etc/cic-cache --migrations-dir /usr/local/share/cic-cache/alembic -vv
 set +e

apps/cic-cache/docker/run_tests.sh

@@ -2,7 +2,7 @@
 
 set -e
 
-pip install --extra-index-url https://pip.grassrootseconomics.net \
+pip install --extra-index-url https://pip.grassrootseconomics.net:8433 \
 --extra-index-url https://gitlab.com/api/v4/projects/27624814/packages/pypi/simple \
 -r test_requirements.txt 
 

apps/cic-eth/cic_eth/api/admin.py

@@ -123,7 +123,7 @@ class AdminApi:
         return s_lock.apply_async()
 
 
-    def tag_account(self, chain_spec, tag, address):
+    def tag_account(self, tag, address_hex, chain_spec):
         """Persistently associate an address with a plaintext tag.
 
         Some tags are known by the system and is used to resolve addresses to use for certain transactions. 
@@ -138,7 +138,7 @@ class AdminApi:
             'cic_eth.eth.account.set_role',
             [
                 tag,
-                address,
+                address_hex,
                 chain_spec.asdict(),
                 ],
             queue=self.queue,
@@ -146,30 +146,6 @@ class AdminApi:
         return s_tag.apply_async()
 
 
-    def get_tag_account(self, chain_spec, tag=None, address=None):
-        if address != None:
-            s_tag = celery.signature(
-                'cic_eth.eth.account.role',
-                [
-                    address,
-                    chain_spec.asdict(),
-                    ],
-                queue=self.queue,
-                )
-
-        else:
-            s_tag = celery.signature(
-                'cic_eth.eth.account.role_account',
-                [
-                    tag,
-                    chain_spec.asdict(),
-                ],
-                queue=self.queue,
-                )
-
-        return s_tag.apply_async()
-
-
     def have_account(self, address_hex, chain_spec):
         s_have = celery.signature(
             'cic_eth.eth.account.have',
@@ -527,7 +503,7 @@ class AdminApi:
                     queue=self.queue,
                     )
                 t = s.apply_async()
-                role = t.get()[0][1]
+                role = t.get()
                 if role != None:
                     tx['sender_description'] = role
 
@@ -580,7 +556,7 @@ class AdminApi:
                     queue=self.queue,
                     )
                 t = s.apply_async()
-                role = t.get()[0][1]
+                role = t.get()
                 if role != None:
                     tx['recipient_description'] = role
 

apps/cic-eth/cic_eth/api/api_task.py

@@ -515,7 +515,7 @@ class Api(ApiBase):
         :param password: Password to encode the password with in the backend (careful, you will have to remember it)
         :type password: str
         :param register: Register the new account in accounts index backend
-        :type register: bool
+        :type password: bool
         :returns: uuid of root task
         :rtype: celery.Task
         """

apps/cic-eth/cic_eth/check/gas.py

@@ -12,9 +12,8 @@ from cic_eth.db.models.base import SessionBase
 from cic_eth.db.enum import LockEnum
 from cic_eth.error import LockedError
 from cic_eth.admin.ctrl import check_lock
-from cic_eth.eth.gas import have_gas_minimum
 
-logg = logging.getLogger(__name__)
+logg = logging.getLogger().getChild(__name__)
 
 
 def health(*args, **kwargs):
@@ -32,15 +31,18 @@ def health(*args, **kwargs):
         return True
 
     gas_provider = AccountRole.get_address('GAS_GIFTER', session=session)
-    min_gas = int(config.get('ETH_GAS_HOLDER_MINIMUM_UNITS')) * int(config.get('ETH_GAS_GIFTER_REFILL_BUFFER'))
-    if config.get('ETH_MIN_FEE_PRICE'):
-        min_gas *= int(config.get('ETH_MIN_FEE_PRICE'))
-
-    r = have_gas_minimum(chain_spec, gas_provider, min_gas, session=session)
-
     session.close()
-    
-    if not r:
-        logg.error('EEK! gas gifter has balance {}, below minimum {}'.format(r, min_gas))
 
-    return r
+    rpc = RPCConnection.connect(chain_spec, 'default')
+    o = balance(gas_provider)
+    r = rpc.do(o)
+    try: 
+        r = int(r, 16)
+    except TypeError:
+        r = int(r)
+    gas_min = int(config.get('ETH_GAS_GIFTER_MINIMUM_BALANCE'))
+    if r < gas_min:
+        logg.error('EEK! gas gifter has balance {}, below minimum {}'.format(r, gas_min))
+        return False
+
+    return True

apps/cic-eth/cic_eth/check/start.py

@@ -1,18 +0,0 @@
-# external imports
-from chainlib.chain import ChainSpec
-
-# local imports
-from cic_eth.admin.ctrl import check_lock
-from cic_eth.enum import LockEnum
-from cic_eth.error import LockedError
-
-
-def health(*args, **kwargs):
-    config = kwargs['config']
-    chain_spec = ChainSpec.from_chain_str(config.get('CHAIN_SPEC'))
-
-    try:
-        check_lock(None, chain_spec.asdict(), LockEnum.START)
-    except LockedError as e:
-        return False
-    return True

apps/cic-eth/cic_eth/cli/arg.py

@@ -16,22 +16,16 @@ class ArgumentParser(BaseArgumentParser):
             self.add_argument('--redis-port', dest='redis_port', type=int, help='redis host to use for task submission')
             self.add_argument('--redis-db', dest='redis_db', type=int, help='redis db to use')
         if local_arg_flags & CICFlag.REDIS_CALLBACK:
-            self.add_argument('--redis-host-callback', dest='redis_host_callback', type=str, help='redis host to use for callback (defaults to redis host)')
-            self.add_argument('--redis-port-callback', dest='redis_port_callback', type=int, help='redis port to use for callback (defaults to redis port)')
+            self.add_argument('--redis-host-callback', dest='redis_host_callback', default='localhost', type=str, help='redis host to use for callback')
+            self.add_argument('--redis-port-callback', dest='redis_port_callback', default=6379, type=int, help='redis port to use for callback')
             self.add_argument('--redis-timeout', default=20.0, type=float, help='Redis callback timeout')
         if local_arg_flags & CICFlag.CELERY:
-            self.add_argument('--celery-scheme', type=str, help='Celery broker scheme (defaults to "redis")')
-            self.add_argument('--celery-host', type=str, help='Celery broker host (defaults to redis host)')
-            self.add_argument('--celery-port', type=str, help='Celery broker port (defaults to redis port)')
-            self.add_argument('--celery-db', type=int, help='Celery broker db (defaults to redis db)')
-            self.add_argument('--celery-result-scheme', type=str, help='Celery result backend scheme (defaults to celery broker scheme)')
-            self.add_argument('--celery-result-host', type=str, help='Celery result backend host (defaults to celery broker host)')
-            self.add_argument('--celery-result-port', type=str, help='Celery result backend port (defaults to celery broker port)')
-            self.add_argument('--celery-result-db', type=int, help='Celery result backend db (defaults to celery broker db)')
-            self.add_argument('--celery-no-result', action='store_true', help='Disable the Celery results backend')
             self.add_argument('-q', '--celery-queue', dest='celery_queue', type=str, default='cic-eth', help='Task queue')
         if local_arg_flags & CICFlag.SYNCER:
-            self.add_argument('--offset', type=int, help='Start block height for initial history sync')
+            self.add_argument('--offset', type=int, default=0, help='Start block height for initial history sync')
             self.add_argument('--no-history', action='store_true', dest='no_history', help='Skip initial history sync')
         if local_arg_flags & CICFlag.CHAIN:
             self.add_argument('-r', '--registry-address', type=str, dest='registry_address', help='CIC registry contract address')
+
+
+

apps/cic-eth/cic_eth/cli/base.py

@@ -24,8 +24,8 @@ class CICFlag(enum.IntEnum):
     # sync - nibble 4
     SYNCER = 4096
 
-argflag_local_base = argflag_std_base | Flag.CHAIN_SPEC
-argflag_local_task = CICFlag.CELERY
+
+argflag_local_task = CICFlag.CELERY 
 argflag_local_taskcallback = argflag_local_task | CICFlag.REDIS | CICFlag.REDIS_CALLBACK
 argflag_local_chain = CICFlag.CHAIN
 argflag_local_sync = CICFlag.SYNCER | CICFlag.CHAIN

apps/cic-eth/cic_eth/cli/config.py

@@ -1,18 +1,12 @@
 # standard imports
 import os
 import logging
-import urllib.parse
-import copy
 
 # external imports
 from chainlib.eth.cli import (
         Config as BaseConfig,
         Flag,
         )
-from urlybird.merge import (
-        urlhostmerge,
-        urlmerge,
-        )
 
 # local imports
 from .base import CICFlag
@@ -46,7 +40,6 @@ class Config(BaseConfig):
         if local_arg_flags & CICFlag.CHAIN:
             local_args_override['CIC_REGISTRY_ADDRESS'] = getattr(args, 'registry_address')
 
-        
         if local_arg_flags & CICFlag.CELERY:
             local_args_override['CELERY_QUEUE'] = getattr(args, 'celery_queue')
 
@@ -56,61 +49,15 @@ class Config(BaseConfig):
 
         config.dict_override(local_args_override, 'local cli args')
 
-        local_celery_args_override = {}
-        if local_arg_flags & CICFlag.CELERY:
-            hostport = urlhostmerge(
-                    None,
-                    config.get('REDIS_HOST'),
-                    config.get('REDIS_PORT'),
-                    )
-            redis_url = (
-                    'redis',
-                    hostport,
-                    getattr(args, 'redis_db', None),
-                    )
-            celery_config_url = urllib.parse.urlsplit(config.get('CELERY_BROKER_URL'))
-            hostport = urlhostmerge(
-                    celery_config_url[1],
-                    getattr(args, 'celery_host', None),
-                    getattr(args, 'celery_port', None),
-                    )
-            celery_arg_url = (
-                    getattr(args, 'celery_scheme', None),
-                    hostport,
-                    getattr(args, 'celery_db', None),
-                    )
-            celery_url = urlmerge(redis_url, celery_config_url, celery_arg_url)
-            celery_url_string = urllib.parse.urlunsplit(celery_url)
-            local_celery_args_override['CELERY_BROKER_URL'] = celery_url_string
-            if not getattr(args, 'celery_no_result'):
-                local_celery_args_override['CELERY_RESULT_URL'] = config.get('CELERY_RESULT_URL')
-                if local_celery_args_override['CELERY_RESULT_URL'] == None:
-                    local_celery_args_override['CELERY_RESULT_URL'] = local_celery_args_override['CELERY_BROKER_URL']
-                celery_config_url = urllib.parse.urlsplit(local_celery_args_override['CELERY_RESULT_URL'])
-                hostport = urlhostmerge(
-                        celery_config_url[1],
-                        getattr(args, 'celery_result_host', None),
-                        getattr(args, 'celery_result_port', None),
-                        )
-                celery_arg_url = (
-                        getattr(args, 'celery_result_scheme', None),
-                        hostport,
-                        getattr(args, 'celery_result_db', None),
-                        )
-                celery_url = urlmerge(celery_config_url, celery_arg_url)
-                logg.debug('celery url {} {}'.format(celery_config_url, celery_url))
-                celery_url_string = urllib.parse.urlunsplit(celery_url)
-                local_celery_args_override['CELERY_RESULT_URL'] = celery_url_string
-            config.add(config.true('CELERY_DEBUG'), 'CELERY_DEBUG', exists_ok=True)
-
-        config.dict_override(local_celery_args_override, 'local celery cli args')
-
         if local_arg_flags & CICFlag.REDIS_CALLBACK:
-            redis_host_callback = getattr(args, 'redis_host_callback', config.get('REDIS_HOST'))
-            redis_port_callback = getattr(args, 'redis_port_callback', config.get('REDIS_PORT'))
-            config.add(redis_host_callback, '_REDIS_HOST_CALLBACK')
-            config.add(redis_port_callback, '_REDIS_PORT_CALLBACK')
+            config.add(getattr(args, 'redis_host_callback'), '_REDIS_HOST_CALLBACK')
+            config.add(getattr(args, 'redis_port_callback'), '_REDIS_PORT_CALLBACK')
+
+        if local_arg_flags & CICFlag.CELERY:
+            config.add(config.true('CELERY_DEBUG'), 'CELERY_DEBUG', exists_ok=True)
 
         logg.debug('config loaded:\n{}'.format(config))
 
         return config
+
+

apps/cic-eth/cic_eth/data/config/celery.ini

@@ -1,5 +1,5 @@
 [celery]
-broker_url = 
+broker_url = redis://localhost:6379
 result_url = 
 queue = cic-eth
 debug = 0

apps/cic-eth/cic_eth/data/config/cic.ini

@@ -2,5 +2,5 @@
 registry_address = 
 trust_address =
 default_token_symbol =
-health_modules = cic_eth.check.db,cic_eth.check.redis,cic_eth.check.signer,cic_eth.check.gas,cic_eth.check.start
+health_modules = cic_eth.check.db,cic_eth.check.redis,cic_eth.check.signer,cic_eth.check.gas
 run_dir = /run

apps/cic-eth/cic_eth/data/config/eth.ini

@@ -1,6 +1,2 @@
 [eth]
-gas_holder_minimum_units = 180000
-gas_holder_refill_units = 15
-gas_holder_refill_threshold = 3
-gas_gifter_refill_buffer = 3
-min_fee_price = 1
+gas_gifter_minimum_balance = 10000000000000000000000

apps/cic-eth/cic_eth/db/migrations/default/versions/75d4767b3031_lock.py

@@ -23,7 +23,7 @@ def upgrade():
     op.create_table(
             'lock',
             sa.Column('id', sa.Integer, primary_key=True),
-            sa.Column("address", sa.String, nullable=True),
+            sa.Column("address", sa.String(42), nullable=True),
             sa.Column('blockchain', sa.String),
             sa.Column("flags", sa.BIGINT(), nullable=False, default=0),
             sa.Column("date_created", sa.DateTime, nullable=False, default=datetime.datetime.utcnow),

apps/cic-eth/cic_eth/db/migrations/default/versions/c91cafc3e0c1_add_gas_cache.py

@@ -1,31 +0,0 @@
-"""Add gas cache
-
-Revision ID: c91cafc3e0c1
-Revises: aee12aeb47ec
-Create Date: 2021-10-28 20:45:34.239865
-
-"""
-from alembic import op
-import sqlalchemy as sa
-
-
-# revision identifiers, used by Alembic.
-revision = 'c91cafc3e0c1'
-down_revision = 'aee12aeb47ec'
-branch_labels = None
-depends_on = None
-
-
-def upgrade():
-    op.create_table(
-            'gas_cache',
-            sa.Column('id', sa.Integer, primary_key=True),
-            sa.Column("address", sa.String, nullable=False),
-            sa.Column("tx_hash", sa.String, nullable=True),
-            sa.Column("method", sa.String, nullable=True),
-            sa.Column("value", sa.BIGINT(), nullable=False),
-            )
-
-
-def downgrade():
-    op.drop_table('gas_cache')

apps/cic-eth/cic_eth/db/models/gas_cache.py

@@ -1,27 +0,0 @@
-# standard imports
-import logging
-
-# external imports
-from sqlalchemy import Column, String, NUMERIC
-
-# local imports
-from .base import SessionBase
-
-logg = logging.getLogger(__name__)
-
-
-class GasCache(SessionBase):
-    """Provides gas budget cache for token operations
-    """
-    __tablename__ = 'gas_cache'
-
-    address = Column(String())
-    tx_hash = Column(String())
-    method = Column(String())
-    value = Column(NUMERIC())
-
-    def __init__(self, address, method, value, tx_hash):
-        self.address = address
-        self.tx_hash = tx_hash
-        self.method = method
-        self.value = value

apps/cic-eth/cic_eth/db/models/nonce.py

@@ -12,7 +12,7 @@ from cic_eth.error import (
         IntegrityError,
         )
 
-logg = logging.getLogger(__name__)
+logg = logging.getLogger()
 
 
 class Nonce(SessionBase):
@@ -21,7 +21,7 @@ class Nonce(SessionBase):
     __tablename__ = 'nonce'
 
     nonce = Column(Integer)
-    address_hex = Column(String(40))
+    address_hex = Column(String(42))
 
 
     @staticmethod

apps/cic-eth/cic_eth/db/models/role.py

@@ -24,22 +24,8 @@ class AccountRole(SessionBase):
     tag = Column(Text)
     address_hex = Column(String(42))
 
-
-    @staticmethod
-    def all(session=None):
-        session = SessionBase.bind_session(session)
-
-        pairs = []
-
-        q = session.query(AccountRole.tag, AccountRole.address_hex)
-        for r in q.all():
-            pairs.append((r[1], r[0]),)
-
-        SessionBase.release_session(session)
-        
-        return pairs
-
-
+    
+    # TODO: 
     @staticmethod
     def get_address(tag, session):
         """Get Ethereum address matching the given tag

apps/cic-eth/cic_eth/enum.py

@@ -69,12 +69,9 @@ class StatusEnum(enum.IntEnum):
 class LockEnum(enum.IntEnum):
     """
     STICKY: When set, reset is not possible
-    INIT: When set, startup is possible without second level sanity checks (e.g. gas gifter balance)
-    START: When set, startup is not possible, regardless of state
     CREATE: Disable creation of accounts
     SEND: Disable sending to network
     QUEUE: Disable queueing new or modified transactions
-    QUERY: Disable all queue state and transaction queries
     """
     STICKY=1
     INIT=2
@@ -82,8 +79,7 @@ class LockEnum(enum.IntEnum):
     SEND=8
     QUEUE=16
     QUERY=32
-    START=int(0x80000000)
-    ALL=int(0x7ffffffe)
+    ALL=int(0xfffffffffffffffe)
 
 
 def status_str(v, bits_only=False):

apps/cic-eth/cic_eth/error.py

@@ -64,10 +64,8 @@ class LockedError(Exception):
 
 class SeppukuError(Exception):
     """Exception base class for all errors that should cause system shutdown
+
     """
-    def __init__(self, message, lockdown=False):
-        self.message = message
-        self.lockdown = lockdown
 
 
 class SignerError(SeppukuError):

apps/cic-eth/cic_eth/eth/account.py

@@ -136,7 +136,7 @@ def register(self, account_address, chain_spec_dict, writer_address=None):
     # Generate and sign transaction
     rpc_signer = RPCConnection.connect(chain_spec, 'signer')
     nonce_oracle = CustodialTaskNonceOracle(writer_address, self.request.root_id, session=session) #, default_nonce)
-    gas_oracle = self.create_gas_oracle(rpc, code_callback=AccountRegistry.gas)
+    gas_oracle = self.create_gas_oracle(rpc, AccountRegistry.gas)
     account_registry = AccountsIndex(chain_spec, signer=rpc_signer, nonce_oracle=nonce_oracle, gas_oracle=gas_oracle)
     (tx_hash_hex, tx_signed_raw_hex) = account_registry.add(account_registry_address, writer_address, account_address, tx_format=TxFormat.RLP_SIGNED)
     rpc_signer.disconnect()
@@ -192,7 +192,7 @@ def gift(self, account_address, chain_spec_dict):
     # Generate and sign transaction
     rpc_signer = RPCConnection.connect(chain_spec, 'signer')
     nonce_oracle = CustodialTaskNonceOracle(account_address, self.request.root_id, session=session) #, default_nonce)
-    gas_oracle = self.create_gas_oracle(rpc, code_callback=MinterFaucet.gas)
+    gas_oracle = self.create_gas_oracle(rpc, MinterFaucet.gas)
     faucet = Faucet(chain_spec, signer=rpc_signer, nonce_oracle=nonce_oracle, gas_oracle=gas_oracle)
     (tx_hash_hex, tx_signed_raw_hex) = faucet.give_to(faucet_address, account_address, account_address, tx_format=TxFormat.RLP_SIGNED)
     rpc_signer.disconnect()
@@ -266,46 +266,19 @@ def set_role(self, tag, address, chain_spec_dict):
 
 @celery_app.task(bind=True, base=BaseTask)
 def role(self, address, chain_spec_dict):
-    """Return account role for address and/or role
+    """Return account role for address
 
     :param account: Account to check
     :type account: str, 0x-hex
-    :param chain_spec_dict: Chain spec dict representation
-    :type chain_spec_dict: dict
+    :param chain_str: Chain spec string representation
+    :type chain_str: str
     :returns: Account, or None if not exists
     :rtype: Varies
     """
     session = self.create_session()
     role_tag =  AccountRole.role_for(address, session=session)
     session.close()
-    return [(address, role_tag,)]
-
-
-@celery_app.task(bind=True, base=BaseTask)
-def role_account(self, role_tag, chain_spec_dict):
-    """Return address for role.
-
-    If the role parameter is None, will return addresses for all roles.
-
-    :param role_tag: Role to match
-    :type role_tag: str
-    :param chain_spec_dict: Chain spec dict representation
-    :type chain_spec_dict: dict
-    :returns: List with a single account/tag pair for a single tag, or a list of account and tag pairs for all tags
-    :rtype: list
-    """
-    session = self.create_session()
-
-    pairs = None
-    if role_tag != None:
-        addr = AccountRole.get_address(role_tag, session=session)
-        pairs = [(addr, role_tag,)]
-    else:
-        pairs = AccountRole.all(session=session)
-
-    session.close()
-
-    return pairs
+    return role_tag
 
 
 @celery_app.task(bind=True, base=CriticalSQLAlchemyTask)

apps/cic-eth/cic_eth/eth/erc20.py

@@ -10,9 +10,6 @@ from chainlib.eth.tx import (
         TxFormat,
         unpack,
         )
-from chainlib.eth.contract import (
-        ABIContractEncoder,
-        )
 from cic_eth_registry import CICRegistry
 from cic_eth_registry.erc20 import ERC20Token
 from hexathon import (
@@ -34,8 +31,10 @@ from cic_eth.error import (
         YouAreBrokeError,
         )
 from cic_eth.queue.tx import register_tx
-from cic_eth.eth.gas import create_check_gas_task
-from cic_eth.eth.util import CacheGasOracle
+from cic_eth.eth.gas import (
+        create_check_gas_task,
+        MaxGasOracle,
+        )
 from cic_eth.ext.address import translate_address
 from cic_eth.task import (
         CriticalSQLAlchemyTask,
@@ -155,12 +154,8 @@ def transfer_from(self, tokens, holder_address, receiver_address, value, chain_s
     rpc_signer = RPCConnection.connect(chain_spec, 'signer')
 
     session = self.create_session()
-
     nonce_oracle = CustodialTaskNonceOracle(holder_address, self.request.root_id, session=session)
-    enc = ABIContractEncoder()
-    enc.method('transferFrom')
-    method = enc.get()
-    gas_oracle = self.create_gas_oracle(rpc, t['address'], method=method, session=session, min_price=self.min_fee_price)
+    gas_oracle = self.create_gas_oracle(rpc, MaxGasOracle.gas)
     c = ERC20(chain_spec, signer=rpc_signer, gas_oracle=gas_oracle, nonce_oracle=nonce_oracle)
     try:
         (tx_hash_hex, tx_signed_raw_hex) = c.transfer_from(t['address'], spender_address, holder_address, receiver_address, value, tx_format=TxFormat.RLP_SIGNED)
@@ -230,12 +225,8 @@ def transfer(self, tokens, holder_address, receiver_address, value, chain_spec_d
     rpc_signer = RPCConnection.connect(chain_spec, 'signer')
 
     session = self.create_session()
-
-    enc = ABIContractEncoder()
-    enc.method('transfer')
-    method = enc.get()
-    gas_oracle = self.create_gas_oracle(rpc, t['address'], method=method, session=session, min_price=self.min_fee_price)
     nonce_oracle = CustodialTaskNonceOracle(holder_address, self.request.root_id, session=session)
+    gas_oracle = self.create_gas_oracle(rpc, MaxGasOracle.gas)
     c = ERC20(chain_spec, signer=rpc_signer, gas_oracle=gas_oracle, nonce_oracle=nonce_oracle)
     try:
         (tx_hash_hex, tx_signed_raw_hex) = c.transfer(t['address'], holder_address, receiver_address, value, tx_format=TxFormat.RLP_SIGNED)
@@ -303,12 +294,8 @@ def approve(self, tokens, holder_address, spender_address, value, chain_spec_dic
     rpc_signer = RPCConnection.connect(chain_spec, 'signer')
 
     session = self.create_session()
-
     nonce_oracle = CustodialTaskNonceOracle(holder_address, self.request.root_id, session=session)
-    enc = ABIContractEncoder()
-    enc.method('approve')
-    method = enc.get()
-    gas_oracle = self.create_gas_oracle(rpc, t['address'], method=method, session=session)
+    gas_oracle = self.create_gas_oracle(rpc, MaxGasOracle.gas)
     c = ERC20(chain_spec, signer=rpc_signer, gas_oracle=gas_oracle, nonce_oracle=nonce_oracle)
     try:
         (tx_hash_hex, tx_signed_raw_hex) = c.approve(t['address'], holder_address, spender_address, value, tx_format=TxFormat.RLP_SIGNED)
@@ -395,8 +382,6 @@ def cache_transfer_data(
     sender_address = tx_normalize.wallet_address(tx['from'])
     recipient_address = tx_normalize.wallet_address(tx_data[0])
     token_value = tx_data[1]
-    source_token_address = tx_normalize.executable_address(tx['to'])
-    destination_token_address = source_token_address
 
 
     session = SessionBase.create_session()
@@ -404,8 +389,8 @@ def cache_transfer_data(
             'hash': tx_hash_hex,
             'from': sender_address,
             'to': recipient_address,
-            'source_token': source_token_address,
-            'destination_token': destination_token_address,
+            'source_token': tx['to'],
+            'destination_token': tx['to'],
             'from_value': token_value,
             'to_value': token_value,
             }
@@ -437,16 +422,14 @@ def cache_transfer_from_data(
     spender_address = tx_data[0]
     recipient_address = tx_data[1]
     token_value = tx_data[2]
-    source_token_address = tx_normalize.executable_address(tx['to'])
-    destination_token_address = source_token_address
 
     session = SessionBase.create_session()
     tx_dict = {
             'hash': tx_hash_hex,
             'from': tx['from'],
             'to': recipient_address,
-            'source_token': source_token_address,
-            'destination_token': destination_token_address,
+            'source_token': tx['to'],
+            'destination_token': tx['to'],
             'from_value': token_value,
             'to_value': token_value,
             }
@@ -478,16 +461,14 @@ def cache_approve_data(
     sender_address = tx_normalize.wallet_address(tx['from'])
     recipient_address = tx_normalize.wallet_address(tx_data[0])
     token_value = tx_data[1]
-    source_token_address = tx_normalize.executable_address(tx['to'])
-    destination_token_address = source_token_address
 
     session = SessionBase.create_session()
     tx_dict = {
             'hash': tx_hash_hex,
             'from': sender_address,
             'to': recipient_address,
-            'source_token': source_token_address,
-            'destination_token': destination_token_address,
+            'source_token': tx['to'],
+            'destination_token': tx['to'],
             'from_value': token_value,
             'to_value': token_value,
             }

apps/cic-eth/cic_eth/eth/gas.py

@@ -41,7 +41,6 @@ from chainqueue.db.models.tx import TxCache
 from chainqueue.db.models.otx import Otx
 
 # local imports
-from cic_eth.db.models.gas_cache import GasCache
 from cic_eth.db.models.role import AccountRole
 from cic_eth.db.models.base import SessionBase
 from cic_eth.error import (
@@ -66,56 +65,17 @@ from cic_eth.encode import (
         ZERO_ADDRESS_NORMAL,
         unpack_normal,
         )
-from cic_eth.error import SeppukuError
-from cic_eth.eth.util import MAXIMUM_FEE_UNITS
 
 celery_app = celery.current_app
 logg = logging.getLogger()
 
 
+MAXIMUM_FEE_UNITS = 8000000
 
-@celery_app.task(base=CriticalSQLAlchemyTask)
-def apply_gas_value_cache(address, method, value, tx_hash):
-    return apply_gas_value_cache_local(address, method, value, tx_hash)
+class MaxGasOracle:
 
-
-def apply_gas_value_cache_local(address, method, value, tx_hash, session=None):
-    address = tx_normalize.executable_address(address)
-    tx_hash = tx_normalize.tx_hash(tx_hash)
-    value = int(value)
-
-    session = SessionBase.bind_session(session)
-    q = session.query(GasCache)
-    q = q.filter(GasCache.address==address)
-    q = q.filter(GasCache.method==method)
-    o = q.first()
-
-    if o == None:
-        o = GasCache(address, method, value, tx_hash)
-    elif tx.gas_used > o.value:
-        o.value = value
-        o.tx_hash = strip_0x(tx_hash)
-
-    session.add(o)
-    session.commit()
-
-    SessionBase.release_session(session)
-
-
-def have_gas_minimum(chain_spec, address, min_gas, session=None, rpc=None):
-    if rpc == None:
-        rpc = RPCConnection.connect(chain_spec, 'default')
-    o = balance(add_0x(address))
-    r = rpc.do(o)
-    try: 
-        r = int(r)
-    except ValueError:
-        r = strip_0x(r)
-        r = int(r, 16)
-    logg.debug('have gas minimum {} have gas {} minimum is {}'.format(address, r, min_gas))
-    if r < min_gas:
-        return False
-    return True
+    def gas(code=None):
+        return MAXIMUM_FEE_UNITS
 
 
 def create_check_gas_task(tx_signed_raws_hex, chain_spec, holder_address, gas=None, tx_hashes_hex=None, queue=None):
@@ -397,13 +357,6 @@ def refill_gas(self, recipient_address, chain_spec_dict):
     # set up evm RPC connection
     rpc = RPCConnection.connect(chain_spec, 'default')
 
-    # check the gas balance of the gifter
-    if not have_gas_minimum(chain_spec, gas_provider, self.safe_gas_refill_amount):
-        raise SeppukuError('Noooooooooooo; gas gifter {} is broke!'.format(gas_provider))
-
-    if not have_gas_minimum(chain_spec, gas_provider, self.safe_gas_gifter_balance):
-        logg.error('Gas gifter {} gas balance is below the safe level to operate!'.format(gas_provider))
-    
     # set up transaction builder
     nonce_oracle = CustodialTaskNonceOracle(gas_provider, self.request.root_id, session=session)
     gas_oracle = self.create_gas_oracle(rpc)

apps/cic-eth/cic_eth/eth/util.py

@@ -1,54 +0,0 @@
-# standard imports
-import logging
-
-# external imports
-from chainlib.eth.gas import RPCGasOracle
-from hexathon import strip_0x
-
-# local imports
-from cic_eth.db.models.gas_cache import GasCache
-from cic_eth.encode import tx_normalize
-from cic_eth.db.models.base import SessionBase
-
-MAXIMUM_FEE_UNITS = 8000000
-
-logg = logging.getLogger(__name__)
-
-
-class MaxGasOracle(RPCGasOracle):
-
-    def get_fee_units(self, code=None):
-        return MAXIMUM_FEE_UNITS
-
-
-class CacheGasOracle(MaxGasOracle):
-    """Returns a previously recorded value for fee unit expenditure for a contract call, if it exists. Otherwise returns max units.
-
-    :todo: instead of max units, connect a pluggable gas heuristics engine.
-    """
-
-    def __init__(self, conn, address, method=None, session=None, min_price=None, id_generator=None):
-        super(CacheGasOracle, self).__init__(conn, code_callback=self.get_fee_units, min_price=min_price, id_generator=id_generator)
-        self.value = None
-        self.address = address
-        self.method = method
-
-        address = tx_normalize.executable_address(address)
-        session = SessionBase.bind_session(session)
-        q = session.query(GasCache)
-        q = q.filter(GasCache.address==address)
-        if method != None:
-            method = strip_0x(method)
-            q = q.filter(GasCache.method==method)
-        o = q.first()
-        if o != None:
-            self.value = int(o.value)
-
-        SessionBase.release_session(session)
-
-
-    def get_fee_units(self, code=None):
-        if self.value != None:
-            logg.debug('found stored gas unit value {} for address {} method {}'.format(self.value, self.address, self.method))
-            return self.value
-        return super(CacheGasOracle, self).get_fee_units(code=code)

apps/cic-eth/cic_eth/pytest/fixtures_config.py

@@ -8,14 +8,15 @@ import confini
 
 script_dir = os.path.dirname(os.path.realpath(__file__))
 root_dir = os.path.dirname(os.path.dirname(script_dir))
-config_dir = os.path.join(root_dir, 'cic_eth', 'data', 'config')
 logg = logging.getLogger(__name__)
 
 
 @pytest.fixture(scope='session')
 def load_config():
-    override_config_dir = os.path.join(root_dir, 'config', 'test')
-    conf = confini.Config(config_dir, 'CICTEST', override_dirs=[override_config_dir])
+    config_dir = os.environ.get('CONFINI_DIR')
+    if config_dir == None:
+        config_dir = os.path.join(root_dir, 'config/test')
+    conf = confini.Config(config_dir, 'CICTEST')
     conf.process()
     logg.debug('config {}'.format(conf))
     return conf

apps/cic-eth/cic_eth/queue/balance.py

@@ -72,7 +72,7 @@ def __balance_incoming_compatible(token_address, receiver_address):
     status_compare = dead()
     q = q.filter(Otx.status.op('&')(status_compare)==0)
     # TODO: this can change the result for the recipient if tx is later obsoleted and resubmission is delayed. 
-    #q = q.filter(Otx.status.op('&')(StatusBits.IN_NETWORK)==StatusBits.IN_NETWORK)
+    q = q.filter(Otx.status.op('&')(StatusBits.IN_NETWORK)==StatusBits.IN_NETWORK)
     q = q.filter(TxCache.destination_token_address==token_address)
     delta = 0
     for r in q.all():

apps/cic-eth/cic_eth/runnable/daemons/filters/__init__.py

@@ -3,4 +3,3 @@ from .tx import TxFilter
 from .gas import GasFilter
 from .register import RegistrationFilter
 from .transferauth import TransferAuthFilter
-from .token import TokenFilter

apps/cic-eth/cic_eth/runnable/daemons/filters/token.py

@@ -1,63 +0,0 @@
-# standard imports
-import logging
-
-# external imports
-from eth_erc20 import ERC20
-from chainlib.eth.contract import (
-        ABIContractEncoder,
-        ABIContractType,
-        )
-from chainlib.eth.constant import ZERO_ADDRESS
-from chainlib.eth.address import is_same_address
-from chainlib.eth.error import RequestMismatchException
-from cic_eth_registry import CICRegistry
-from cic_eth_registry.erc20 import ERC20Token
-from eth_token_index import TokenUniqueSymbolIndex
-import celery
-
-# local imports
-from .base import SyncFilter
-
-logg = logging.getLogger(__name__)
-
-
-class TokenFilter(SyncFilter):
-
-    def __init__(self, chain_spec, queue, call_address=ZERO_ADDRESS):
-        self.queue = queue
-        self.chain_spec = chain_spec
-        self.caller_address = call_address
-
-
-    def filter(self, conn, block, tx, db_session=None):
-        if not tx.payload:
-            return (None, None)
-
-        try:
-            r = ERC20.parse_transfer_request(tx.payload)
-        except RequestMismatchException:
-            return (None, None)
-
-        token_address = tx.inputs[0]
-        token = ERC20Token(self.chain_spec, conn, token_address)
-
-        registry = CICRegistry(self.chain_spec, conn)
-        r = registry.by_name(token.symbol, sender_address=self.caller_address)
-        if is_same_address(r, ZERO_ADDRESS):
-            return None
-
-        enc = ABIContractEncoder()
-        enc.method('transfer')
-        method = enc.get()
-
-        s = celery.signature(
-                'cic_eth.eth.gas.apply_gas_value_cache',
-                [
-                    token_address,
-                    method,
-                    tx.gas_used,
-                    tx.hash,
-                    ],
-                queue=self.queue,
-                )
-        return s.apply_async()

apps/cic-eth/cic_eth/runnable/daemons/tasker.py

@@ -67,10 +67,7 @@ from cic_eth.registry import (
         connect_declarator,
         connect_token_registry,
         )
-from cic_eth.task import (
-        BaseTask,
-        CriticalWeb3Task,
-        )
+from cic_eth.task import BaseTask
 
 logging.basicConfig(level=logging.WARNING)
 logg = logging.getLogger()
@@ -79,18 +76,18 @@ arg_flags = cic_eth.cli.argflag_std_read
 local_arg_flags = cic_eth.cli.argflag_local_task
 argparser = cic_eth.cli.ArgumentParser(arg_flags)
 argparser.process_local_flags(local_arg_flags)
+#argparser.add_argument('--default-token-symbol', dest='default_token_symbol', type=str, help='Symbol of default token to use')
 argparser.add_argument('--trace-queue-status', default=None, dest='trace_queue_status', action='store_true', help='set to perist all queue entry status changes to storage')
 argparser.add_argument('--aux-all', action='store_true', help='include tasks from all submodules from the aux module path')
-argparser.add_argument('--min-fee-price', dest='min_fee_price', type=int, help='set minimum fee price for transactions, in wei')
 argparser.add_argument('--aux', action='append', type=str, default=[], help='add single submodule from the aux module path')
 args = argparser.parse_args()
 
 # process config
 extra_args = {
+#    'default_token_symbol': 'CIC_DEFAULT_TOKEN_SYMBOL',
     'aux_all': None,
     'aux': None,
     'trace_queue_status': 'TASKS_TRACE_QUEUE_STATUS',
-    'min_fee_price': 'ETH_MIN_FEE_PRICE',
         }
 config = cic_eth.cli.Config.from_args(args, arg_flags, local_arg_flags)
 
@@ -218,7 +215,6 @@ def main():
     argv.append('-n')
     argv.append(config.get('CELERY_QUEUE'))
 
-    # TODO: More elegant way of setting queue-wide settings
     BaseTask.default_token_symbol = default_token_symbol
     BaseTask.default_token_address = default_token_address
     default_token = ERC20Token(chain_spec, conn, add_0x(BaseTask.default_token_address))
@@ -226,14 +222,6 @@ def main():
     BaseTask.default_token_decimals = default_token.decimals
     BaseTask.default_token_name = default_token.name
     BaseTask.trusted_addresses = trusted_addresses
-    CriticalWeb3Task.safe_gas_refill_amount = int(config.get('ETH_GAS_HOLDER_MINIMUM_UNITS')) * int(config.get('ETH_GAS_HOLDER_REFILL_UNITS'))
-    CriticalWeb3Task.safe_gas_threshold_amount = int(config.get('ETH_GAS_HOLDER_MINIMUM_UNITS')) * int(config.get('ETH_GAS_HOLDER_REFILL_THRESHOLD'))
-    CriticalWeb3Task.safe_gas_gifter_balance = int(config.get('ETH_GAS_HOLDER_MINIMUM_UNITS')) * int(config.get('ETH_GAS_GIFTER_REFILL_BUFFER'))
-    if config.get('ETH_MIN_FEE_PRICE'):
-        BaseTask.min_fee_price = int(config.get('ETH_MIN_FEE_PRICE'))
-        CriticalWeb3Task.safe_gas_threshold_amount *= BaseTask.min_fee_price
-        CriticalWeb3Task.safe_gas_refill_amount *= BaseTask.min_fee_price
-        CriticalWeb3Task.safe_gas_gifter_balance *= BaseTask.min_fee_price
 
     BaseTask.run_dir = config.get('CIC_RUN_DIR')
     logg.info('default token set to {} {}'.format(BaseTask.default_token_symbol, BaseTask.default_token_address))

apps/cic-eth/cic_eth/runnable/daemons/tracker.py

@@ -36,7 +36,6 @@ from cic_eth.runnable.daemons.filters import (
         TxFilter,
         RegistrationFilter,
         TransferAuthFilter,
-        TokenFilter,
         )
 from cic_eth.stat import init_chain_stat
 from cic_eth.registry import (
@@ -100,10 +99,10 @@ def main():
     syncer_backends = SQLBackend.resume(chain_spec, block_offset)
 
     if len(syncer_backends) == 0:
-        initial_block_start = int(config.get('SYNCER_OFFSET'))
-        initial_block_offset = int(block_offset)
+        initial_block_start = config.get('SYNCER_OFFSET')
+        initial_block_offset = block_offset
         if config.true('SYNCER_NO_HISTORY'):
-            initial_block_start = initial_block_offset
+            initial_block_start = block_offset
             initial_block_offset += 1
         syncer_backends.append(SQLBackend.initial(chain_spec, initial_block_offset, start_block_height=initial_block_start))
         logg.info('found no backends to resume, adding initial sync from history start {} end {}'.format(initial_block_start, initial_block_offset))
@@ -155,8 +154,6 @@ def main():
 
     gas_filter = GasFilter(chain_spec, config.get('CELERY_QUEUE'))
 
-    token_gas_cache_filter = TokenFilter(chain_spec, config.get('CELERY_QUEUE'))
-
     #transfer_auth_filter = TransferAuthFilter(registry, chain_spec, config.get('_CELERY_QUEUE'))
 
     i = 0
@@ -166,7 +163,6 @@ def main():
         syncer.add_filter(registration_filter)
         # TODO: the two following filter functions break the filter loop if return uuid. Pro: less code executed. Con: Possibly unintuitive flow break
         syncer.add_filter(tx_filter)
-        syncer.add_filter(token_gas_cache_filter)
         #syncer.add_filter(transfer_auth_filter)
         for cf in callback_filters:
             syncer.add_filter(cf)

apps/cic-eth/cic_eth/runnable/tag.py

@@ -8,7 +8,6 @@ import re
 # external imports
 import cic_eth.cli
 from chainlib.chain import ChainSpec
-from chainlib.eth.address import is_address
 from xdg.BaseDirectory import xdg_config_home
 
 # local imports
@@ -22,18 +21,12 @@ logg = logging.getLogger()
 arg_flags = cic_eth.cli.argflag_std_base | cic_eth.cli.Flag.UNSAFE | cic_eth.cli.Flag.CHAIN_SPEC
 local_arg_flags = cic_eth.cli.argflag_local_taskcallback
 argparser = cic_eth.cli.ArgumentParser(arg_flags)
-argparser.add_argument('--set', action='store_true', help='sets the given tag')
-argparser.add_argument('--tag', type=str, help='operate on the given tag')
-argparser.add_positional('address', required=False, type=str, help='address associated with tag')
+argparser.add_positional('tag', type=str, help='address tag')
+argparser.add_positional('address', type=str, help='address')
 argparser.process_local_flags(local_arg_flags)
 args = argparser.parse_args()
 
-extra_args = {
-        'set': None,
-        'tag': None,
-        'address': None,
-        }
-config = cic_eth.cli.Config.from_args(args, arg_flags, local_arg_flags, extra_args=extra_args)
+config = cic_eth.cli.Config.from_args(args, arg_flags, local_arg_flags)
 
 celery_app = cic_eth.cli.CeleryApp.from_config(config)
 
@@ -46,17 +39,7 @@ api = AdminApi(None)
 
 
 def main():
-    if config.get('_ADDRESS') != None and not is_address(config.get('_ADDRESS')):
-        sys.stderr.write('Invalid address {}'.format(config.get('_ADDRESS')))
-        sys.exit(1)
-
-    if config.get('_SET'):
-        admin_api.tag_account(chain_spec, config.get('_TAG'), config.get('_ADDRESS'))
-    else:
-        t = admin_api.get_tag_account(chain_spec, tag=config.get('_TAG'), address=config.get('_ADDRESS'))
-        r = t.get()
-        for v in r:
-            sys.stdout.write('{}\t{}\n'.format(v[1], v[0]))
+    admin_api.tag_account(args.tag, args.address, chain_spec)
 
 
 if __name__ == '__main__':

apps/cic-eth/cic_eth/runnable/transfer.py

@@ -18,7 +18,7 @@ from cic_eth.api import Api
 logging.basicConfig(level=logging.WARNING)
 logg = logging.getLogger('create_account_script')
 
-arg_flags = cic_eth.cli.argflag_local_base
+arg_flags = cic_eth.cli.argflag_std_base
 local_arg_flags = cic_eth.cli.argflag_local_taskcallback
 argparser = cic_eth.cli.ArgumentParser(arg_flags)
 argparser.add_argument('--token-symbol', dest='token_symbol', type=str, help='Token symbol')

apps/cic-eth/cic_eth/runnable/view.py

@@ -16,14 +16,9 @@ import confini
 import celery
 from chainlib.chain import ChainSpec
 from chainlib.eth.connection import EthHTTPConnection
-from hexathon import (
-        add_0x,
-        strip_0x,
-        uniform as hex_uniform,
-        )
+from hexathon import add_0x
 
 # local imports
-import cic_eth.cli
 from cic_eth.api.admin import AdminApi
 from cic_eth.db.enum import (
     StatusEnum,
@@ -36,35 +31,59 @@ logging.basicConfig(level=logging.WARNING)
 logg = logging.getLogger()
 
 default_format = 'terminal'
+default_config_dir = os.environ.get('CONFINI_DIR', '/usr/local/etc/cic')
 
-arg_flags = cic_eth.cli.argflag_std_base
-local_arg_flags = cic_eth.cli.argflag_local_taskcallback
-argparser = cic_eth.cli.ArgumentParser(arg_flags)
+argparser = argparse.ArgumentParser()
+argparser.add_argument('-p', '--provider', dest='p', default='http://localhost:8545', type=str, help='Web3 provider url (http only)')
+argparser.add_argument('-r', '--registry-address', dest='r', type=str, help='CIC registry address')
 argparser.add_argument('-f', '--format', dest='f', default=default_format, type=str, help='Output format')
+argparser.add_argument('--status-raw', dest='status_raw', action='store_true', help='Output status bit enum names only')
+argparser.add_argument('-c', type=str, default=default_config_dir, help='config root to use')
+argparser.add_argument('-i', '--chain-spec', dest='i', type=str, help='chain spec')
+argparser.add_argument('-q', type=str, default='cic-eth', help='celery queue to submit transaction tasks to')
+argparser.add_argument('--env-prefix', default=os.environ.get('CONFINI_ENV_PREFIX'), dest='env_prefix', type=str, help='environment prefix for variables to overwrite configuration')
+argparser.add_argument('-v', action='store_true', help='Be verbose')
+argparser.add_argument('-vv', help='be more verbose', action='store_true')
 argparser.add_argument('query', type=str, help='Transaction, transaction hash, account or "lock"')
-argparser.process_local_flags(local_arg_flags)
 args = argparser.parse_args()
 
+if args.v == True:
+    logging.getLogger().setLevel(logging.INFO)
+elif args.vv == True:
+    logging.getLogger().setLevel(logging.DEBUG)
 
-extra_args = {
-    'f': '_FORMAT',
-    'query': '_QUERY',
+
+config_dir = os.path.join(args.c)
+os.makedirs(config_dir, 0o777, True)
+config = confini.Config(config_dir, args.env_prefix)
+config.process()
+args_override = {
+        'ETH_PROVIDER': getattr(args, 'p'),
+        'CIC_CHAIN_SPEC': getattr(args, 'i'),
+        'CIC_REGISTRY_ADDRESS': getattr(args, 'r'),
         }
-config = cic_eth.cli.Config.from_args(args, arg_flags, local_arg_flags, extra_args=extra_args)
+# override args
+config.dict_override(args_override, 'cli args')
+config.censor('PASSWORD', 'DATABASE')
+config.censor('PASSWORD', 'SSL')
+logg.debug('config loaded from {}:\n{}'.format(config_dir, config))
 
-celery_app = cic_eth.cli.CeleryApp.from_config(config)
-queue = config.get('CELERY_QUEUE')
+try:
+    config.add(add_0x(args.query), '_QUERY', True)
+except:
+    config.add(args.query, '_QUERY', True)
 
-chain_spec = ChainSpec.from_chain_str(config.get('CHAIN_SPEC'))
+celery_app = celery.Celery(broker=config.get('CELERY_BROKER_URL'), backend=config.get('CELERY_RESULT_URL'))
 
-# connect to celery
-celery_app = cic_eth.cli.CeleryApp.from_config(config)
+queue = args.q
 
-# set up rpc
-rpc = cic_eth.cli.RPC.from_config(config) #, use_signer=True)
-conn = rpc.get_default()
+chain_spec = ChainSpec.from_chain_str(config.get('CIC_CHAIN_SPEC'))
 
-admin_api = AdminApi(conn)
+rpc = EthHTTPConnection(args.p)
+
+#registry_address = config.get('CIC_REGISTRY_ADDRESS')
+
+admin_api = AdminApi(rpc)
 
 t = admin_api.registry()
 registry_address = t.get()
@@ -94,7 +113,7 @@ def render_tx(o, **kwargs):
     
     for v in o.get('status_log', []):
         d = datetime.datetime.fromisoformat(v[0])
-        e = status_str(v[1], config.get('_RAW'))
+        e = status_str(v[1], args.status_raw)
         content += '{}: {}\n'.format(d, e)
 
     return content
@@ -135,24 +154,20 @@ def render_lock(o, **kwargs):
 def main():
     txs  = []
     renderer = render_tx
+    if len(config.get('_QUERY')) > 66:
+        #registry = connect_registry(rpc, chain_spec, registry_address)
+        #admin_api.tx(chain_spec, tx_raw=config.get('_QUERY'), registry=registry, renderer=renderer)
+        admin_api.tx(chain_spec, tx_raw=config.get('_QUERY'), renderer=renderer)
+    elif len(config.get('_QUERY')) > 42:
+        #registry = connect_registry(rpc, chain_spec, registry_address)
+        #admin_api.tx(chain_spec, tx_hash=config.get('_QUERY'), registry=registry, renderer=renderer)
+        admin_api.tx(chain_spec, tx_hash=config.get('_QUERY'), renderer=renderer)
 
-    query = config.get('_QUERY')
-    try:
-        query = hex_uniform(strip_0x(query))
-    except TypeError:
-        pass
-    except ValueError:
-        pass
-
-    if len(query) > 64:
-        admin_api.tx(chain_spec, tx_raw=query, renderer=renderer)
-    elif len(query) > 40:
-        admin_api.tx(chain_spec, tx_hash=query, renderer=renderer)
-
-    elif len(query) == 40:
-        txs = admin_api.account(chain_spec, query, include_recipient=False, renderer=render_account)
+    elif len(config.get('_QUERY')) == 42:
+        #registry = connect_registry(rpc, chain_spec, registry_address)
+        txs = admin_api.account(chain_spec, config.get('_QUERY'), include_recipient=False, renderer=render_account)
         renderer = render_account
-    elif len(query) >= 4 and query[:4] == 'lock':
+    elif len(config.get('_QUERY')) >= 4 and config.get('_QUERY')[:4] == 'lock':
         t = admin_api.get_lock()
         txs = t.get()
         renderer = render_lock
@@ -160,7 +175,7 @@ def main():
             r = renderer(txs)
             sys.stdout.write(r + '\n')
     else:
-        raise ValueError('cannot parse argument {}'.format(query))
+        raise ValueError('cannot parse argument {}'.format(config.get('_QUERY')))
                    
 
 if __name__ == '__main__':

apps/cic-eth/cic_eth/task.py

@@ -17,7 +17,6 @@ from cic_eth_registry.error import UnknownContractError
 # local imports
 from cic_eth.error import SeppukuError
 from cic_eth.db.models.base import SessionBase
-from cic_eth.eth.util import CacheGasOracle
 
 #logg = logging.getLogger().getChild(__name__)
 logg = logging.getLogger()
@@ -30,32 +29,14 @@ class BaseTask(celery.Task):
     session_func = SessionBase.create_session
     call_address = ZERO_ADDRESS
     trusted_addresses = []
-    min_fee_price = 1
+    create_nonce_oracle = RPCNonceOracle
+    create_gas_oracle = RPCGasOracle
     default_token_address = None
     default_token_symbol = None
     default_token_name = None
     default_token_decimals = None
     run_dir = '/run'
 
-
-    def create_gas_oracle(self, conn, address=None, *args, **kwargs):
-        if address == None:
-            return RPCGasOracle(
-                conn,
-                code_callback=kwargs.get('code_callback'),
-                min_price=self.min_fee_price,
-                id_generator=kwargs.get('id_generator'),
-                )
-
-        return CacheGasOracle(
-                conn,
-                address,
-                method=kwargs.get('method'),
-                min_price=self.min_fee_price,
-                id_generator=kwargs.get('id_generator'),
-                )
-
-
     def create_session(self):
         return BaseTask.session_func()
 
@@ -97,18 +78,19 @@ class CriticalWeb3Task(CriticalTask):
     autoretry_for = (
         ConnectionError,
         )
-    safe_gas_threshold_amount = 60000 * 3
+    safe_gas_threshold_amount = 2000000000 * 60000 * 3
     safe_gas_refill_amount = safe_gas_threshold_amount * 5 
-    safe_gas_gifter_balance = safe_gas_threshold_amount * 5 * 100
 
 
-class CriticalSQLAlchemyAndWeb3Task(CriticalWeb3Task):
+class CriticalSQLAlchemyAndWeb3Task(CriticalTask):
     autoretry_for = (
         sqlalchemy.exc.DatabaseError,
         sqlalchemy.exc.TimeoutError,
         ConnectionError,
         sqlalchemy.exc.ResourceClosedError,
         )
+    safe_gas_threshold_amount = 2000000000 * 60000 * 3
+    safe_gas_refill_amount = safe_gas_threshold_amount * 5 
 
 
 class CriticalSQLAlchemyAndSignerTask(CriticalTask):
@@ -118,11 +100,14 @@ class CriticalSQLAlchemyAndSignerTask(CriticalTask):
         sqlalchemy.exc.ResourceClosedError,
         ) 
 
-class CriticalWeb3AndSignerTask(CriticalWeb3Task):
+class CriticalWeb3AndSignerTask(CriticalTask):
     autoretry_for = (
         ConnectionError,
         )
-    
+    safe_gas_threshold_amount = 2000000000 * 60000 * 3
+    safe_gas_refill_amount = safe_gas_threshold_amount * 5 
+
+
 @celery_app.task()
 def check_health(self):
     pass

apps/cic-eth/config/test/accounts.ini

@@ -0,0 +1,2 @@
+[accounts]
+writer_address = 

apps/cic-eth/config/test/bancor.ini

@@ -0,0 +1,2 @@
+[bancor]
+dir = tests/testdata/bancor

apps/cic-eth/config/test/celery.ini

@@ -1,3 +1,5 @@
 [celery]
 broker_url = filesystem://
 result_url = filesystem://
+#broker_url = redis://
+#result_url = redis://

apps/cic-eth/config/test/chain.ini

@@ -0,0 +1,2 @@
+[chain]
+spec =

apps/cic-eth/config/test/cic.ini

@@ -0,0 +1,4 @@
+[cic]
+registry_address = 
+chain_spec = 
+trust_address =

apps/cic-eth/config/test/dispatcher.ini

@@ -0,0 +1,2 @@
+[dispatcher]
+loop_interval = 0.1

apps/cic-eth/config/test/eth.ini

@@ -0,0 +1,8 @@
+[eth]
+#ws_provider = ws://localhost:8546
+#ttp_provider = http://localhost:8545
+provider = http://localhost:8545
+gas_provider_address = 
+#chain_id = 
+abi_dir = 
+faucet_giver_address = 

apps/cic-eth/config/test/signer.ini

@@ -1,2 +1,5 @@
 [signer]
-provider = /run/crypto-dev-signer/jsonrpc.ipc
+socket_path = /run/crypto-dev-signer/jsonrpc.ipc
+secret = deedbeef
+database_name = signer_test
+dev_keys_path = 

apps/cic-eth/config/test/ssl.ini

@@ -0,0 +1,6 @@
+[SSL]
+enable_client = false
+cert_file = 
+key_file =
+password = 
+ca_file =

apps/cic-eth/config/test/syncer.ini

@@ -0,0 +1,2 @@
+[SYNCER]
+loop_interval = 1

apps/cic-eth/docker/Dockerfile

@@ -7,7 +7,7 @@ FROM $DOCKER_REGISTRY/cic-base-images:python-3.8.6-dev-e8eb2ee2
 # TODO can we take all the requirements out of setup.py and just do a pip install -r requirements.txt && python setup.py
 #COPY cic-eth/requirements.txt .
 
-ARG EXTRA_PIP_INDEX_URL=https://pip.grassrootseconomics.net
+ARG EXTRA_PIP_INDEX_URL=https://pip.grassrootseconomics.net:8433
 ARG EXTRA_PIP_ARGS=""
 ARG PIP_INDEX_URL=https://pypi.org/simple
 
@@ -15,7 +15,7 @@ RUN --mount=type=cache,mode=0755,target=/root/.cache/pip \
     pip install --index-url $PIP_INDEX_URL \
     --pre \
     --extra-index-url $EXTRA_PIP_INDEX_URL $EXTRA_PIP_ARGS \
-    cic-eth-aux-erc20-demurrage-token~=0.0.2a7
+        cic-eth-aux-erc20-demurrage-token~=0.0.2a7
 
 
 COPY *requirements.txt ./
@@ -26,7 +26,7 @@ RUN --mount=type=cache,mode=0755,target=/root/.cache/pip \
     -r requirements.txt \
     -r services_requirements.txt \
     -r admin_requirements.txt 
-
+  
 COPY . .
 RUN python setup.py install
 

apps/cic-eth/docker/run_tests.sh

@@ -2,7 +2,7 @@
 
 set -e
 
-pip install --extra-index-url https://pip.grassrootseconomics.net --extra-index-url https://gitlab.com/api/v4/projects/27624814/packages/pypi/simple \
+pip install --extra-index-url https://pip.grassrootseconomics.net:8433 --extra-index-url https://gitlab.com/api/v4/projects/27624814/packages/pypi/simple \
 -r admin_requirements.txt \
 -r services_requirements.txt \
 -r test_requirements.txt  

apps/cic-eth/requirements.txt

@@ -1,4 +1,4 @@
 celery==4.4.7
-chainlib-eth>=0.0.10a20,<0.1.0
+chainlib-eth>=0.0.10a16,<0.1.0
 semver==2.13.0
-urlybird~=0.0.1a2
+crypto-dev-signer>=0.4.15rc2,<0.5.0

apps/cic-eth/setup.cfg

@@ -1,7 +1,7 @@
 [metadata]
 name = cic-eth
 #version = attr: cic_eth.version.__version_string__
-version = 0.12.5a2
+version = 0.12.4a13
 description = CIC Network Ethereum interaction 
 author = Louis Holbrook
 author_email = dev@holbrook.no

apps/cic-eth/tests/filters/test_token_filter.py

@@ -1,97 +0,0 @@
-# external imports
-from eth_erc20 import ERC20
-from chainlib.connection import RPCConnection
-from chainlib.eth.nonce import RPCNonceOracle
-from chainlib.eth.gas import (
-        Gas,
-        OverrideGasOracle,
-        )
-from chainlib.eth.tx import (
-        TxFormat,
-        receipt,
-        raw,
-        unpack,
-        Tx,
-        )
-from chainlib.eth.block import (
-        Block,
-        block_latest,
-        block_by_number,
-        )
-from chainlib.eth.address import is_same_address
-from chainlib.eth.contract import ABIContractEncoder
-from hexathon import strip_0x
-from eth_token_index import TokenUniqueSymbolIndex
-
-# local imports
-from cic_eth.runnable.daemons.filters.token import TokenFilter
-from cic_eth.db.models.gas_cache import GasCache
-from cic_eth.db.models.base import SessionBase
-
-
-def test_filter_gas(
-        default_chain_spec,
-        init_database,
-        eth_rpc,
-        eth_signer,
-        contract_roles,
-        agent_roles,
-        token_roles,
-        foo_token,
-        token_registry,
-        register_lookups,
-        celery_session_worker,
-        cic_registry,
-    ):
-
-    rpc = RPCConnection.connect(default_chain_spec, 'default')
-    nonce_oracle = RPCNonceOracle(token_roles['FOO_TOKEN_OWNER'], eth_rpc)
-    gas_oracle = OverrideGasOracle(price=1000000000, limit=1000000)
-    c = ERC20(default_chain_spec, signer=eth_signer, nonce_oracle=nonce_oracle, gas_oracle=gas_oracle)
-    (tx_hash_hex, tx_signed_raw_hex) = c.transfer(foo_token, token_roles['FOO_TOKEN_OWNER'], agent_roles['ALICE'], 100, tx_format=TxFormat.RLP_SIGNED)
-    o = raw(tx_signed_raw_hex)
-    eth_rpc.do(o)
-    o = receipt(tx_hash_hex)
-    rcpt = eth_rpc.do(o)
-    assert rcpt['status'] == 1
-
-    fltr = TokenFilter(default_chain_spec, queue=None, call_address=agent_roles['ALICE'])
-
-    o = block_latest()
-    r = eth_rpc.do(o)
-    o = block_by_number(r, include_tx=False)
-    r = eth_rpc.do(o)
-    block = Block(r)
-    block.txs = [tx_hash_hex]
-
-    tx_signed_raw_bytes = bytes.fromhex(strip_0x(tx_signed_raw_hex))
-    tx_src = unpack(tx_signed_raw_bytes, default_chain_spec)
-    tx = Tx(tx_src, block=block)
-    tx.apply_receipt(rcpt)
-    t = fltr.filter(eth_rpc, block, tx, db_session=init_database)
-    assert t == None
-
-    nonce_oracle = RPCNonceOracle(contract_roles['CONTRACT_DEPLOYER'], eth_rpc)
-    c = TokenUniqueSymbolIndex(default_chain_spec, signer=eth_signer, nonce_oracle=nonce_oracle)
-    (tx_hash_hex_register, o) = c.register(token_registry, contract_roles['CONTRACT_DEPLOYER'], foo_token)
-    eth_rpc.do(o)
-    o = receipt(tx_hash_hex)
-    r = eth_rpc.do(o)
-    assert r['status'] == 1
-
-    t = fltr.filter(eth_rpc, block, tx, db_session=init_database)
-    r = t.get_leaf()
-    assert t.successful()
-
-    q = init_database.query(GasCache)
-    q = q.filter(GasCache.tx_hash==strip_0x(tx_hash_hex))
-    o = q.first()
-
-    assert is_same_address(o.address, strip_0x(foo_token))
-    assert o.value > 0
-
-    enc = ABIContractEncoder()
-    enc.method('transfer')
-    method = enc.get()
-
-    assert o.method == method

apps/cic-eth/tests/run_tests.sh

@@ -2,7 +2,7 @@
 
 set -e
 
-pip install --extra-index-url https://pip.grassrootseconomics.net --extra-index-url https://gitlab.com/api/v4/projects/27624814/packages/pypi/simple
+pip install --extra-index-url https://pip.grassrootseconomics.net:8433 --extra-index-url https://gitlab.com/api/v4/projects/27624814/packages/pypi/simple
 -r admin_requirements.txt
 -r services_requirements.txt
 -r test_requirements.txt 

apps/cic-eth/tests/task/api/test_admin.py

@@ -103,11 +103,11 @@ def test_tag_account(
 
     api = AdminApi(eth_rpc, queue=None)
 
-    t = api.tag_account(default_chain_spec, 'foo', agent_roles['ALICE'])
+    t = api.tag_account('foo', agent_roles['ALICE'], default_chain_spec)
     t.get()
-    t = api.tag_account(default_chain_spec, 'bar', agent_roles['BOB'])
+    t = api.tag_account('bar', agent_roles['BOB'], default_chain_spec)
     t.get()
-    t = api.tag_account(default_chain_spec, 'bar', agent_roles['CAROL'])
+    t = api.tag_account('bar', agent_roles['CAROL'], default_chain_spec)
     t.get()
 
     assert AccountRole.get_address('foo', init_database) == tx_normalize.wallet_address(agent_roles['ALICE'])

apps/cic-eth/tests/task/test_task_account.py

@@ -141,57 +141,9 @@ def test_role_task(
             )
     t = s.apply_async()
     r = t.get()
-    assert r[0][0] == address
-    assert r[0][1] == 'foo'
+    assert r == 'foo'
 
 
-def test_get_role_task(
-        init_database,
-        celery_session_worker,
-        default_chain_spec,
-        ):
-    address_foo = '0x' + os.urandom(20).hex()
-    role_foo = AccountRole.set('foo', address_foo)
-    init_database.add(role_foo)
-
-    address_bar = '0x' + os.urandom(20).hex()
-    role_bar = AccountRole.set('bar', address_bar)
-    init_database.add(role_bar)
-
-    init_database.commit()
-
-    s = celery.signature(
-            'cic_eth.eth.account.role_account',
-            [
-                'bar',
-                default_chain_spec.asdict(),
-                ],
-            queue=None,
-            )
-    t = s.apply_async()
-    r = t.get()
-    assert r[0][0] == address_bar
-    assert r[0][1] == 'bar'
-
-    s = celery.signature(
-            'cic_eth.eth.account.role_account',
-            [
-                None,
-                default_chain_spec.asdict(),
-                ],
-            queue=None,
-            )
-    t = s.apply_async()
-    r = t.get()
-    x_tags = ['foo', 'bar']
-    x_addrs = [address_foo, address_bar]
-
-    for v in r:
-        x_addrs.remove(v[0])
-        x_tags.remove(v[1])
-    
-    assert len(x_tags) == 0
-    assert len(x_addrs) == 0
 
 def test_gift(
     init_database,

apps/cic-notify/docker/Dockerfile

@@ -6,7 +6,7 @@ FROM $DOCKER_REGISTRY/cic-base-images:python-3.8.6-dev-e8eb2ee2
 RUN apt-get install libffi-dev -y
 
 
-ARG EXTRA_PIP_INDEX_URL=https://pip.grassrootseconomics.net
+ARG EXTRA_PIP_INDEX_URL=https://pip.grassrootseconomics.net:8433
 ARG EXTRA_PIP_ARGS=""
 ARG PIP_INDEX_URL=https://pypi.org/simple
 

apps/cic-notify/docker/run_tests.sh

@@ -2,7 +2,7 @@
 
 set -e
 
-pip install --extra-index-url https://pip.grassrootseconomics.net \
+pip install --extra-index-url https://pip.grassrootseconomics.net:8433 \
 --extra-index-url https://gitlab.com/api/v4/projects/27624814/packages/pypi/simple \
 -r test_requirements.txt 
 

apps/cic-signer/Dockerfile

@@ -8,7 +8,7 @@ RUN apt-get install libffi-dev -y
 
 COPY requirements.txt . 
 
-ARG EXTRA_PIP_INDEX_URL="https://pip.grassrootseconomics.net"
+ARG EXTRA_PIP_INDEX_URL="https://pip.grassrootseconomics.net:8433"
 ARG EXTRA_PIP_ARGS=""
 ARG PIP_INDEX_URL="https://pypi.org/simple"
 RUN --mount=type=cache,mode=0755,target=/root/.cache/pip \

apps/cic-signer/cic_signer/data/config/database.ini

@@ -1,10 +0,0 @@
-[database]
-engine = postgres
-driver = psycopg2
-host = localhost
-port = 5432
-name = cic_signer
-user =
-password =
-debug = 0
-pool_size = 0

apps/cic-signer/cic_signer/data/config/signer.ini

@@ -1,3 +0,0 @@
-[signer]
-provider =
-secret =

apps/cic-signer/requirements.txt

@@ -1,2 +1 @@
 funga-eth[sql]>=0.5.1a1,<0.6.0
-chainlib-eth>=0.0.10a18

apps/cic-signer/scripts/sweep.py

@@ -1,128 +0,0 @@
-# standard imports
-import os
-import logging
-import uuid
-import random
-import sys
-
-# external imports
-from chainlib.chain import ChainSpec
-from chainlib.eth.constant import ZERO_ADDRESS
-from chainlib.eth.gas import (
-        balance,
-        Gas,
-        )
-from hexathon import (
-        add_0x,
-        strip_0x,
-        )
-from chainlib.eth.connection import EthHTTPSignerConnection
-from funga.eth.signer import EIP155Signer
-from funga.eth.keystore.sql import SQLKeystore
-from chainlib.cli.wallet import Wallet
-from chainlib.eth.address import AddressChecksum
-from chainlib.eth.nonce import RPCNonceOracle
-from chainlib.eth.gas import OverrideGasOracle
-from chainlib.eth.address import (
-        is_checksum_address,
-        to_checksum_address,
-        )
-from chainlib.eth.tx import (
-        TxFormat,
-        )
-import chainlib.eth.cli
-
-       
-script_dir = os.path.dirname(os.path.realpath(__file__))
-config_dir = os.path.join(script_dir, '..', 'cic_signer', 'data', 'config')
-
-logging.basicConfig(level=logging.WARNING)
-logg = logging.getLogger()
-
-arg_flags = chainlib.eth.cli.argflag_std_write | chainlib.eth.cli.Flag.WALLET
-argparser = chainlib.eth.cli.ArgumentParser(arg_flags)
-args = argparser.parse_args()
-
-config = chainlib.eth.cli.Config.from_args(args, arg_flags, base_config_dir=config_dir)
-
-# set up rpc
-chain_spec = ChainSpec.from_chain_str(config.get('CHAIN_SPEC'))
-
-
-# connect to database
-dsn = 'postgresql://{}:{}@{}:{}/{}'.format(
-        config.get('DATABASE_USER'),
-        config.get('DATABASE_PASSWORD'),
-        config.get('DATABASE_HOST'),
-        config.get('DATABASE_PORT'),
-        config.get('DATABASE_NAME'),    
-    )
-logg.info('using dsn {}'.format(dsn))
-
-keystore = SQLKeystore(dsn, symmetric_key=bytes.fromhex(config.get('SIGNER_SECRET')))
-wallet = Wallet(EIP155Signer, keystore=keystore, checksummer=AddressChecksum)
-
-rpc = chainlib.eth.cli.Rpc(wallet=wallet)
-conn = rpc.connect_by_config(config)
-
-wallet.init()
-
-def main():
-    if config.get('_RECIPIENT') == None:
-        sys.stderr.write('Missing sink address\n')
-        sys.exit(1)
-
-    sink_address = config.get('_RECIPIENT')
-    if config.get('_UNSAFE'):
-        sink_address = to_checksum_address(sink_address)
-    if not is_checksum_address(sink_address):
-        sys.stderr.write('Invalid sink address {}\n'.format(sink_address))
-        sys.exit(1)
-
-    if (config.get('_RPC_SEND')):
-        verify_string = random.randbytes(4).hex()
-        verify_string_check = input("\033[;31m*** WARNING! WARNING! WARNING! ***\033[;39m\nThis action will transfer all remaining gas from all accounts in custodial care to account {}. To confirm, please enter the string: {}\n".format(config.get('_RECIPIENT'), verify_string))
-        if verify_string != verify_string_check:
-            sys.stderr.write('Verify string mismatch. Aborting!\n')
-            sys.exit(1)
-
-    signer = rpc.get_signer()
-
-    gas_oracle = rpc.get_gas_oracle()
-    gas_pair = gas_oracle.get_fee()
-    gas_price = gas_pair[0]
-    gas_limit = 21000
-    gas_cost = gas_price * gas_limit
-    gas_oracle = OverrideGasOracle(price=gas_price, limit=gas_limit)
-    logg.info('using gas price {}'.format(gas_price))
-
-    for r in keystore.list():
-        account = r[0]
-
-        o = balance(add_0x(account))
-        r = conn.do(o)
-        account_balance = 0
-        try:
-            r = strip_0x(r)
-            account_balance = int(r, 16)
-        except ValueError:
-            account_balance = int(r)
-
-        transfer_amount = account_balance - gas_cost
-        if transfer_amount <= 0:
-            logg.warning('address {} has balance {} which is less than gas cost {}, skipping'.format(account, account_balance, gas_cost))
-            continue
-
-        nonce_oracle = RPCNonceOracle(account, conn)
-        c = Gas(chain_spec, gas_oracle=gas_oracle, nonce_oracle=nonce_oracle, signer=signer)
-        tx_hash_hex = None
-        if (config.get('_RPC_SEND')):
-            (tx_hash_hex, o) = c.create(account, config.get('_RECIPIENT'), transfer_amount)
-            r = conn.do(o)
-        else:
-            (tx_hash_hex, o) = c.create(account, config.get('_RECIPIENT'), transfer_amount, tx_format=TxFormat.RLP_SIGNED)
-
-        logg.info('address {} balance {} net transfer {} tx {}'.format(account, account_balance, transfer_amount, tx_hash_hex))
-
-if __name__ == '__main__':
-    main()

apps/cic-ussd/cic_ussd/db/migrations/default/versions/f289e8510444_.py

@@ -24,8 +24,6 @@ def upgrade():
                     sa.Column('preferred_language', sa.String(), nullable=True),
                     sa.Column('password_hash', sa.String(), nullable=True),
                     sa.Column('failed_pin_attempts', sa.Integer(), nullable=False),
-                    sa.Column('guardians', sa.String(), nullable=True),
-                    sa.Column('guardian_quora', sa.Integer(), nullable=False),
                     sa.Column('status', sa.Integer(), nullable=False),
                     sa.Column('created', sa.DateTime(), nullable=False),
                     sa.Column('updated', sa.DateTime(), nullable=False),

apps/cic-ussd/cic_ussd/db/models/account.py

@@ -4,8 +4,6 @@ import json
 # external imports
 from cic_eth.api import Api
 from cic_types.condiments import MetadataPointer
-from sqlalchemy import Column, Integer, String
-from sqlalchemy.orm.session import Session
 
 # local imports
 from cic_ussd.account.metadata import get_cached_preferred_language, parse_account_metadata
@@ -14,9 +12,8 @@ from cic_ussd.db.enum import AccountStatus
 from cic_ussd.db.models.base import SessionBase
 from cic_ussd.db.models.task_tracker import TaskTracker
 from cic_ussd.encoder import check_password_hash, create_password_hash
-from cic_ussd.phone_number import Support
-
-support_phone = Support.phone_number
+from sqlalchemy import Column, Integer, String
+from sqlalchemy.orm.session import Session
 
 
 class Account(SessionBase):
@@ -32,16 +29,12 @@ class Account(SessionBase):
     failed_pin_attempts = Column(Integer)
     status = Column(Integer)
     preferred_language = Column(String)
-    guardians = Column(String)
-    guardian_quora = Column(Integer)
 
     def __init__(self, blockchain_address, phone_number):
         self.blockchain_address = blockchain_address
         self.phone_number = phone_number
         self.password_hash = None
         self.failed_pin_attempts = 0
-        # self.guardians = f'{support_phone}' if support_phone else None
-        self.guardian_quora = 1
         self.status = AccountStatus.PENDING.value
 
     def __repr__(self):
@@ -52,28 +45,6 @@ class Account(SessionBase):
         self.failed_pin_attempts = 0
         self.status = AccountStatus.ACTIVE.value
 
-    def add_guardian(self, phone_number: str):
-        set_guardians = phone_number
-        if self.guardians:
-            set_guardians = self.guardians.split(',')
-            set_guardians.append(phone_number)
-            ','.join(set_guardians)
-        self.guardians = set_guardians
-
-    def remove_guardian(self, phone_number: str):
-        set_guardians = self.guardians.split(',')
-        set_guardians.remove(phone_number)
-        if len(set_guardians) > 1:
-            self.guardians = ','.join(set_guardians)
-        else:
-            self.guardians = set_guardians[0]
-
-    def get_guardians(self) -> list:
-        return self.guardians.split(',') if self.guardians else []
-
-    def set_guardian_quora(self, quora: int):
-        self.guardian_quora = quora
-
     def create_password(self, password):
         """This method takes a password value and hashes the value before assigning it to the corresponding
         `hashed_password` attribute in the user record.

apps/cic-ussd/cic_ussd/db/ussd_menu.json

@@ -317,102 +317,6 @@
             "display_key": "ussd.kenya.exit_successful_token_selection",
             "name": "exit_successful_token_selection",
             "parent": null
-        },
-        "54": {
-            "description": "Pin management menu for operations related to an account's pin.",
-            "display_key": "ussd.kenya.pin_management",
-            "name": "pin_management",
-            "parent": "start"
-        },
-        "55": {
-            "description": "Phone number entry for account whose pin is being reset.",
-            "display_key": "ussd.kenya.reset_guarded_pin",
-            "name": "reset_guarded_pin",
-            "parent": "pin_management"
-        },
-        "56": {
-            "description": "Pin entry for initiating request to reset an account's pin.",
-            "display_key": "ussd.kenya.reset_guarded_pin_authorization",
-            "name": "reset_guarded_pin_authorization",
-            "parent": "pin_management"
-        },
-        "57": {
-            "description": "Exit menu following successful pin reset initiation.",
-            "display_key": "ussd.kenya.exit_pin_reset_initiated_success",
-            "name": "exit_pin_reset_initiated_success",
-            "parent": "pin_management"
-        },
-        "58": {
-            "description": "Exit menu in the event that an account is not a set guardian.",
-            "display_key": "ussd.kenya.exit_not_authorized_for_pin_reset",
-            "name": "exit_not_authorized_for_pin_reset",
-            "parent": "pin_management"
-        },
-        "59": {
-            "description": "Pin guard menu for handling guardianship operations.",
-            "display_key": "ussd.kenya.guard_pin",
-            "name": "guard_pin",
-            "parent": "pin_management"
-        },
-        "60": {
-            "description": "Pin entry to display a list of set guardians.",
-            "display_key": "ussd.kenya.guardian_list_pin_authorization",
-            "name": "guardian_list_pin_authorization",
-            "parent": "guard_pin"
-        },
-        "61": {
-            "description": "Menu to display list of set guardians.",
-            "display_key": "ussd.kenya.guardian_list",
-            "name": "guardian_list",
-            "parent": "guard_pin"
-        },
-        "62": {
-            "description": "Phone number entry to add an account as a guardian to reset pin.",
-            "display_key": "ussd.kenya.add_guardian",
-            "name": "add_guardian",
-            "parent": "guard_pin"
-        },
-        "63": {
-            "description": "Pin entry to confirm addition of an account as a guardian.",
-            "display_key": "ussd.kenya.add_guardian_pin_authorization",
-            "name": "add_guardian_pin_authorization",
-            "parent": "guard_pin"
-        },
-        "64": {
-            "description": "Exit menu when an account is successfully added as pin reset guardian.",
-            "display_key": "ussd.kenya.exit_guardian_addition_success",
-            "name": "exit_guardian_addition_success",
-            "parent": "guard_pin"
-        },
-        "65": {
-            "description": "Phone number entry to remove an account as a guardian to reset pin.",
-            "display_key": "ussd.kenya.remove_guardian",
-            "name": "remove_guardian",
-            "parent": "guard_pin"
-        },
-        "66": {
-            "description": "Pin entry to confirm removal of an account as a guardian.",
-            "display_key": "ussd.kenya.remove_guardian_pin_authorization",
-            "name": "remove_guardian_pin_authorization",
-            "parent": "guard_pin"
-        },
-        "67": {
-            "description": "Exit menu when an account is successfully removed as pin reset guardian.",
-            "display_key": "ussd.kenya.exit_guardian_removal_success",
-            "name": "exit_guardian_removal_success",
-            "parent": "guard_pin"
-        },
-        "68": {
-            "description": "Exit menu when invalid phone number entry for guardian addition. ",
-            "display_key": "ussd.kenya.exit_invalid_guardian_addition",
-            "name": "exit_invalid_guardian_addition",
-            "parent": "guard_pin"
-        },
-        "69": {
-            "description": "Exit menu when invalid phone number entry for guardian removal. ",
-            "display_key": "ussd.kenya.exit_invalid_guardian_removal",
-            "name": "exit_invalid_guardian_removal",
-            "parent": "guard_pin"
         }
     }
 }

apps/cic-ussd/cic_ussd/phone_number.py

@@ -5,6 +5,7 @@ from typing import Optional
 import phonenumbers
 
 # local imports
+from cic_ussd.db.models.account import Account
 
 
 class E164Format:

apps/cic-ussd/cic_ussd/processor/menu.py

@@ -121,27 +121,6 @@ class MenuProcessor:
                 self.display_key, preferred_language, last_transaction_set=last_transaction_set
             )
 
-    def add_guardian_pin_authorization(self):
-        guardian_information = self.guardian_metadata()
-        return self.pin_authorization(guardian_information=guardian_information)
-
-    def guardian_list(self):
-        preferred_language = get_cached_preferred_language(self.account.blockchain_address)
-        if not preferred_language:
-            preferred_language = i18n.config.get('fallback')
-        set_guardians = self.account.get_guardians()
-        if set_guardians:
-            guardians_list = ''
-            guardians_list_header = translation_for('helpers.guardians_list_header', preferred_language)
-            for phone_number in set_guardians:
-                guardian = Account.get_by_phone_number(phone_number, self.session)
-                guardian_information = guardian.standard_metadata_id()
-                guardians_list += f'{guardian_information}\n'
-            guardians_list = guardians_list_header + '\n' + guardians_list
-        else:
-            guardians_list = translation_for('helpers.no_guardians_list', preferred_language)
-        return translation_for(self.display_key, preferred_language, guardians_list=guardians_list)
-
     def account_tokens(self) -> str:
         cached_token_data_list = get_cached_token_data_list(self.account.blockchain_address)
         token_data_list = parse_token_list(cached_token_data_list)
@@ -228,20 +207,6 @@ class MenuProcessor:
             f'{self.display_key}.retry', preferred_language, retry_pin_entry=retry_pin_entry
         )
 
-    def guarded_account_metadata(self):
-        guarded_account_phone_number = self.ussd_session.get('data').get('guarded_account_phone_number')
-        guarded_account = Account.get_by_phone_number(guarded_account_phone_number, self.session)
-        return guarded_account.standard_metadata_id()
-
-    def guardian_metadata(self):
-        guardian_phone_number = self.ussd_session.get('data').get('guardian_phone_number')
-        guardian = Account.get_by_phone_number(guardian_phone_number, self.session)
-        return guardian.standard_metadata_id()
-
-    def reset_guarded_pin_authorization(self):
-        guarded_account_information = self.guarded_account_metadata()
-        return self.pin_authorization(guarded_account_information=guarded_account_information)
-
     def start_menu(self):
         """
         :return:
@@ -312,47 +277,6 @@ class MenuProcessor:
             sender_information=tx_sender_information
         )
 
-    def exit_guardian_addition_success(self) -> str:
-        guardian_information = self.guardian_metadata()
-        preferred_language = get_cached_preferred_language(self.account.blockchain_address)
-        if not preferred_language:
-            preferred_language = i18n.config.get('fallback')
-        return translation_for(self.display_key,
-                               preferred_language,
-                               guardian_information=guardian_information)
-
-    def exit_guardian_removal_success(self):
-        guardian_information = self.guardian_metadata()
-        preferred_language = get_cached_preferred_language(self.account.blockchain_address)
-        if not preferred_language:
-            preferred_language = i18n.config.get('fallback')
-        return translation_for(self.display_key,
-                               preferred_language,
-                               guardian_information=guardian_information)
-
-    def exit_invalid_guardian_addition(self):
-        failure_reason = self.ussd_session.get('data').get('failure_reason')
-        preferred_language = get_cached_preferred_language(self.account.blockchain_address)
-        if not preferred_language:
-            preferred_language = i18n.config.get('fallback')
-        return translation_for(self.display_key, preferred_language, error_exit=failure_reason)
-
-    def exit_invalid_guardian_removal(self):
-        failure_reason = self.ussd_session.get('data').get('failure_reason')
-        preferred_language = get_cached_preferred_language(self.account.blockchain_address)
-        if not preferred_language:
-            preferred_language = i18n.config.get('fallback')
-        return translation_for(self.display_key, preferred_language, error_exit=failure_reason)
-
-    def exit_pin_reset_initiated_success(self):
-        guarded_account_information = self.guarded_account_metadata()
-        preferred_language = get_cached_preferred_language(self.account.blockchain_address)
-        if not preferred_language:
-            preferred_language = i18n.config.get('fallback')
-        return translation_for(self.display_key,
-                               preferred_language,
-                               guarded_account_information=guarded_account_information)
-
     def exit_insufficient_balance(self):
         """
         :return:
@@ -455,41 +379,18 @@ def response(account: Account, display_key: str, menu_name: str, session: Sessio
         return menu_processor.transaction_pin_authorization()
 
     if menu_name == 'token_selection_pin_authorization':
+        logg.debug(f'RESPONSE IS: {menu_processor.token_selection_pin_authorization()}')
         return menu_processor.token_selection_pin_authorization()
 
     if menu_name == 'exit_insufficient_balance':
         return menu_processor.exit_insufficient_balance()
 
-    if menu_name == 'exit_invalid_guardian_addition':
-        return menu_processor.exit_invalid_guardian_addition()
-
-    if menu_name == 'exit_invalid_guardian_removal':
-        return menu_processor.exit_invalid_guardian_removal()
-
     if menu_name == 'exit_successful_transaction':
         return menu_processor.exit_successful_transaction()
 
-    if menu_name == 'exit_guardian_addition_success':
-        return menu_processor.exit_guardian_addition_success()
-
-    if menu_name == 'exit_guardian_removal_success':
-        return menu_processor.exit_guardian_removal_success()
-
-    if menu_name == 'exit_pin_reset_initiated_success':
-        return menu_processor.exit_pin_reset_initiated_success()
-
     if menu_name == 'account_balances':
         return menu_processor.account_balances()
 
-    if menu_name == 'guardian_list':
-        return menu_processor.guardian_list()
-
-    if menu_name == 'add_guardian_pin_authorization':
-        return menu_processor.add_guardian_pin_authorization()
-
-    if menu_name == 'reset_guarded_pin_authorization':
-        return menu_processor.reset_guarded_pin_authorization()
-
     if 'pin_authorization' in menu_name:
         return menu_processor.pin_authorization()
 

apps/cic-ussd/cic_ussd/state_machine/logic/menu.py

@@ -81,18 +81,6 @@ def menu_six_selected(state_machine_data: Tuple[str, dict, Account, Session]) ->
     return user_input == '6'
 
 
-def menu_nine_selected(state_machine_data: Tuple[str, dict, Account, Session]) -> bool:
-    """
-    This function checks that user input matches a string with value '6'
-    :param state_machine_data: A tuple containing user input, a ussd session and user object.
-    :type state_machine_data: tuple
-    :return: A user input's match with '6'
-    :rtype: bool
-    """
-    user_input, ussd_session, account, session = state_machine_data
-    return user_input == '9'
-
-
 def menu_zero_zero_selected(state_machine_data: Tuple[str, dict, Account, Session]) -> bool:
     """
     This function checks that user input matches a string with value '00'

apps/cic-ussd/cic_ussd/state_machine/logic/pin_guard.py

@@ -1,200 +0,0 @@
-# standard imports
-import logging
-from typing import Tuple
-
-# external imports
-import celery
-import i18n
-from phonenumbers.phonenumberutil import NumberParseException
-from sqlalchemy.orm.session import Session
-
-# local imports
-from cic_ussd.account.metadata import get_cached_preferred_language
-from cic_ussd.db.models.account import Account
-from cic_ussd.db.models.base import SessionBase
-from cic_ussd.phone_number import process_phone_number, E164Format
-from cic_ussd.session.ussd_session import save_session_data
-from cic_ussd.translation import translation_for
-
-
-logg = logging.getLogger(__file__)
-
-
-def save_guardian_to_session_data(state_machine_data: Tuple[str, dict, Account, Session]):
-    """
-    :param state_machine_data:
-    :type state_machine_data:
-    :return:
-    :rtype:
-    """
-    user_input, ussd_session, account, session = state_machine_data
-    session_data = ussd_session.get('data') or {}
-    guardian_phone_number = process_phone_number(phone_number=user_input, region=E164Format.region)
-    session_data['guardian_phone_number'] = guardian_phone_number
-    save_session_data('cic-ussd', session, session_data, ussd_session)
-
-
-def save_guarded_account_session_data(state_machine_data: Tuple[str, dict, Account, Session]):
-    """
-    :param state_machine_data:
-    :type state_machine_data:
-    :return:
-    :rtype:
-    """
-    user_input, ussd_session, account, session = state_machine_data
-    session_data = ussd_session.get('data') or {}
-    guarded_account_phone_number = process_phone_number(phone_number=user_input, region=E164Format.region)
-    session_data['guarded_account_phone_number'] = guarded_account_phone_number
-    save_session_data('cic-ussd', session, session_data, ussd_session)
-
-
-def retrieve_person_metadata(state_machine_data: Tuple[str, dict, Account, Session]):
-    """
-    :param state_machine_data:
-    :type state_machine_data:
-    :return:
-    :rtype:
-    """
-    user_input, ussd_session, account, session = state_machine_data
-    guardian_phone_number = process_phone_number(user_input, E164Format.region)
-    guardian = Account.get_by_phone_number(guardian_phone_number, session)
-    blockchain_address = guardian.blockchain_address
-    s_query_person_metadata = celery.signature(
-        'cic_ussd.tasks.metadata.query_person_metadata', [blockchain_address], queue='cic-ussd')
-    s_query_person_metadata.apply_async()
-
-
-def is_valid_guardian_addition(state_machine_data: Tuple[str, dict, Account, Session]):
-    """
-    :param state_machine_data:
-    :type state_machine_data:
-    :return:
-    :rtype:
-    """
-    user_input, ussd_session, account, session = state_machine_data
-    try:
-        phone_number = process_phone_number(user_input, E164Format.region)
-    except NumberParseException:
-        phone_number = None
-
-    preferred_language = get_cached_preferred_language(account.blockchain_address)
-    if not preferred_language:
-        preferred_language = i18n.config.get('fallback')
-
-    is_valid_account = Account.get_by_phone_number(phone_number, session) is not None
-    is_initiator = phone_number == account.phone_number
-    is_existent_guardian = phone_number in account.get_guardians()
-
-    failure_reason = ''
-    if not is_valid_account:
-        failure_reason = translation_for('helpers.error.no_matching_account', preferred_language)
-
-    if is_initiator:
-        failure_reason = translation_for('helpers.error.is_initiator', preferred_language)
-
-    if is_existent_guardian:
-        failure_reason = translation_for('helpers.error.is_existent_guardian', preferred_language)
-
-    if failure_reason:
-        session_data = ussd_session.get('data') or {}
-        session_data['failure_reason'] = failure_reason
-        save_session_data('cic-ussd', session, session_data, ussd_session)
-
-    return phone_number is not None and is_valid_account and not is_existent_guardian and not is_initiator
-
-
-def add_pin_guardian(state_machine_data: Tuple[str, dict, Account, Session]):
-    """
-    :param state_machine_data:
-    :type state_machine_data:
-    :return:
-    :rtype:
-    """
-    user_input, ussd_session, account, session = state_machine_data
-    guardian_phone_number = ussd_session.get('data').get('guardian_phone_number')
-    account.add_guardian(guardian_phone_number)
-    session.add(account)
-    session.flush()
-    SessionBase.release_session(session=session)
-
-
-def is_set_pin_guardian(account: Account, checked_number: str, preferred_language: str, session: Session, ussd_session: dict):
-    """"""
-    failure_reason = ''
-    set_guardians = []
-    if account:
-        set_guardians = account.get_guardians()
-    else:
-        failure_reason = translation_for('helpers.error.no_matching_account', preferred_language)
-
-    is_set_guardian = checked_number in set_guardians
-    is_initiator = checked_number == account.phone_number
-
-    if not is_set_guardian:
-        failure_reason = translation_for('helpers.error.is_not_existent_guardian', preferred_language)
-
-    if is_initiator:
-        failure_reason = translation_for('helpers.error.is_initiator', preferred_language)
-
-    if failure_reason:
-        session_data = ussd_session.get('data') or {}
-        session_data['failure_reason'] = failure_reason
-        save_session_data('cic-ussd', session, session_data, ussd_session)
-
-    return is_set_guardian and not is_initiator
-
-
-def is_dialers_pin_guardian(state_machine_data: Tuple[str, dict, Account, Session]):
-    user_input, ussd_session, account, session = state_machine_data
-    phone_number = process_phone_number(phone_number=user_input, region=E164Format.region)
-    preferred_language = get_cached_preferred_language(account.blockchain_address)
-    if not preferred_language:
-        preferred_language = i18n.config.get('fallback')
-    return is_set_pin_guardian(account, phone_number, preferred_language, session, ussd_session)
-
-
-def is_others_pin_guardian(state_machine_data: Tuple[str, dict, Account, Session]):
-    user_input, ussd_session, account, session = state_machine_data
-    preferred_language = get_cached_preferred_language(account.blockchain_address)
-    phone_number = process_phone_number(phone_number=user_input, region=E164Format.region)
-    guarded_account = Account.get_by_phone_number(phone_number, session)
-    if not preferred_language:
-        preferred_language = i18n.config.get('fallback')
-    return is_set_pin_guardian(guarded_account, account.phone_number, preferred_language, session, ussd_session)
-
-
-def remove_pin_guardian(state_machine_data: Tuple[str, dict, Account, Session]):
-    """
-    :param state_machine_data:
-    :type state_machine_data:
-    :return:
-    :rtype:
-    """
-    user_input, ussd_session, account, session = state_machine_data
-    guardian_phone_number = ussd_session.get('data').get('guardian_phone_number')
-    account.remove_guardian(guardian_phone_number)
-    session.add(account)
-    session.flush()
-    SessionBase.release_session(session=session)
-
-
-def initiate_pin_reset(state_machine_data: Tuple[str, dict, Account, Session]):
-    """
-    :param state_machine_data:
-    :type state_machine_data:
-    :return:
-    :rtype:
-    """
-    user_input, ussd_session, account, session = state_machine_data
-    session_data = ussd_session.get('data')
-    quorum_count = session_data['quorum_count'] if session_data.get('quorum_count') else 0
-    quorum_count += 1
-    session_data['quorum_count'] = quorum_count
-    save_session_data('cic-ussd', session, session_data, ussd_session)
-    guarded_account_phone_number = session_data.get('guarded_account_phone_number')
-    guarded_account = Account.get_by_phone_number(guarded_account_phone_number, session)
-    if quorum_count >= guarded_account.guardian_quora:
-        guarded_account.reset_pin(session)
-        logg.debug(f'Reset initiated for: {guarded_account.phone_number}')
-        session_data['quorum_count'] = 0
-        save_session_data('cic-ussd', session, session_data, ussd_session)

apps/cic-ussd/docker/Dockerfile

@@ -10,7 +10,7 @@ RUN mkdir -vp pgp/keys
 RUN mkdir -vp cic-ussd
 RUN mkdir -vp data
 
-ARG EXTRA_PIP_INDEX_URL=https://pip.grassrootseconomics.net
+ARG EXTRA_PIP_INDEX_URL=https://pip.grassrootseconomics.net:8433
 ARG EXTRA_PIP_ARGS=""
 ARG PIP_INDEX_URL=https://pypi.org/simple
 
@@ -18,7 +18,7 @@ RUN --mount=type=cache,mode=0755,target=/root/.cache/pip \
     pip install --index-url $PIP_INDEX_URL \
     --pre \
     --extra-index-url $EXTRA_PIP_INDEX_URL $EXTRA_PIP_ARGS \
-    cic-eth-aux-erc20-demurrage-token~=0.0.2a7
+        cic-eth-aux-erc20-demurrage-token~=0.0.2a7
 
 
 COPY *requirements.txt ./

apps/cic-ussd/docker/run_tests.sh

@@ -3,7 +3,7 @@
 set -e
 
 
-pip install --extra-index-url https://pip.grassrootseconomics.net \
+pip install --extra-index-url https://pip.grassrootseconomics.net:8433 \
 --extra-index-url https://gitlab.com/api/v4/projects/27624814/packages/pypi/simple \
 -r test_requirements.txt 
 

apps/cic-ussd/states/exit_states.json

@@ -11,7 +11,5 @@
   "account_creation_prompt",
   "exit_successful_transaction",
   "exit_insufficient_balance",
-  "exit_invalid_guardian_addition",
-  "exit_invalid_guardian_removal",
   "complete"
 ]

apps/cic-ussd/states/pin_management_states.json

@@ -1,16 +0,0 @@
-[
-  "pin_management",
-  "reset_guarded_pin",
-  "reset_guarded_pin_authorization",
-  "exit_pin_reset_initiated_success",
-  "exit_not_authorized_for_pin_reset",
-  "guard_pin",
-  "guardian_list_pin_authorization",
-  "guardian_list",
-  "add_guardian",
-  "add_guardian_pin_authorization",
-  "exit_guardian_addition_success",
-  "remove_guardian",
-  "remove_guardian_pin_authorization",
-  "exit_guardian_removal_success"
-]

apps/cic-ussd/transitions/account_management_transitions.json

@@ -50,7 +50,7 @@
   {
     "trigger": "scan_data",
     "source": "account_management",
-    "dest": "pin_management",
+    "dest": "enter_current_pin",
     "conditions": "cic_ussd.state_machine.logic.menu.menu_five_selected"
   },
   {

apps/cic-ussd/transitions/pin_guard_transitions.json

@@ -1,119 +0,0 @@
-[
-  {
-    "trigger": "scan_data",
-    "source": "guard_pin",
-    "dest": "guardian_list_pin_authorization",
-    "conditions": "cic_ussd.state_machine.logic.menu.menu_one_selected"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "guardian_list_pin_authorization",
-    "dest": "guardian_list",
-    "conditions": "cic_ussd.state_machine.logic.pin.is_authorized_pin"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "guardian_list_pin_authorization",
-    "dest": "exit_pin_blocked",
-    "conditions": "cic_ussd.state_machine.logic.pin.is_blocked_pin"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "guard_pin",
-    "dest": "add_guardian",
-    "conditions": "cic_ussd.state_machine.logic.menu.menu_two_selected"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "add_guardian",
-    "dest": "add_guardian_pin_authorization",
-    "after": [
-      "cic_ussd.state_machine.logic.pin_guard.save_guardian_to_session_data",
-      "cic_ussd.state_machine.logic.pin_guard.retrieve_person_metadata"
-    ],
-    "conditions": "cic_ussd.state_machine.logic.pin_guard.is_valid_guardian_addition"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "add_guardian",
-    "dest": "exit_invalid_guardian_addition",
-    "unless": "cic_ussd.state_machine.logic.pin_guard.is_valid_guardian_addition"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "add_guardian_pin_authorization",
-    "dest": "exit_guardian_addition_success",
-    "after": "cic_ussd.state_machine.logic.pin_guard.add_pin_guardian",
-    "conditions": "cic_ussd.state_machine.logic.pin.is_authorized_pin"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "add_guardian_pin_authorization",
-    "dest": "exit_pin_blocked",
-    "conditions": "cic_ussd.state_machine.logic.pin.is_locked_account"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "guard_pin",
-    "dest": "remove_guardian",
-    "conditions": "cic_ussd.state_machine.logic.menu.menu_three_selected"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "remove_guardian",
-    "dest": "remove_guardian_pin_authorization",
-    "after": [
-      "cic_ussd.state_machine.logic.pin_guard.save_guardian_to_session_data",
-      "cic_ussd.state_machine.logic.pin_guard.retrieve_person_metadata"
-    ],
-    "conditions": "cic_ussd.state_machine.logic.pin_guard.is_dialers_pin_guardian"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "remove_guardian",
-    "dest": "exit_invalid_guardian_removal",
-    "unless": "cic_ussd.state_machine.logic.pin_guard.is_dialers_pin_guardian"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "remove_guardian_pin_authorization",
-    "dest": "exit_guardian_removal_success",
-    "after": "cic_ussd.state_machine.logic.pin_guard.remove_pin_guardian",
-    "conditions": "cic_ussd.state_machine.logic.pin.is_authorized_pin"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "remove_guardian_pin_authorization",
-    "dest": "exit_pin_blocked",
-    "conditions": "cic_ussd.state_machine.logic.pin.is_locked_account"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "exit_guardian_removal_success",
-    "dest": "exit",
-    "conditions": "cic_ussd.state_machine.logic.menu.menu_nine_selected"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "exit_invalid_guardian_addition",
-    "dest": "exit",
-    "conditions": "cic_ussd.state_machine.logic.menu.menu_nine_selected"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "exit_invalid_guardian_removal",
-    "dest": "exit",
-    "conditions": "cic_ussd.state_machine.logic.menu.menu_nine_selected"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "guardian_list",
-    "dest": "exit",
-    "conditions": "cic_ussd.state_machine.logic.menu.menu_nine_selected"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "guard_pin",
-    "dest": "exit_invalid_menu_option"
-  }
-]

apps/cic-ussd/transitions/pin_management_transitions.json

@@ -1,20 +0,0 @@
-[
-  {
-    "trigger": "scan_data",
-    "source": "pin_management",
-    "dest": "enter_current_pin",
-    "conditions": "cic_ussd.state_machine.logic.menu.menu_one_selected"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "pin_management",
-    "dest": "reset_guarded_pin",
-    "conditions": "cic_ussd.state_machine.logic.menu.menu_two_selected"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "pin_management",
-    "dest": "guard_pin",
-    "conditions": "cic_ussd.state_machine.logic.menu.menu_three_selected"
-  }
-]

apps/cic-ussd/transitions/pin_reset_transitions.json

@@ -1,43 +0,0 @@
-[
-  {
-    "trigger": "scan_data",
-    "source": "reset_guarded_pin",
-    "dest": "reset_guarded_pin_authorization",
-    "after": [
-      "cic_ussd.state_machine.logic.pin_guard.save_guarded_account_session_data",
-      "cic_ussd.state_machine.logic.pin_guard.retrieve_person_metadata"
-    ],
-    "conditions": "cic_ussd.state_machine.logic.pin_guard.is_others_pin_guardian"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "reset_guarded_pin_authorization",
-    "dest": "exit_pin_reset_initiated_success",
-    "after": "cic_ussd.state_machine.logic.pin_guard.initiate_pin_reset",
-    "conditions": "cic_ussd.state_machine.logic.pin.is_authorized_pin"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "exit_pin_reset_initiated_success",
-    "dest": "exit",
-    "conditions": "cic_ussd.state_machine.logic.menu.menu_nine_selected"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "reset_guarded_pin_authorization",
-    "dest": "exit_pin_blocked",
-    "conditions": "cic_ussd.state_machine.logic.pin.is_locked_account"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "reset_guarded_pin",
-    "dest": "exit_not_authorized_for_pin_reset",
-    "unless": "cic_ussd.state_machine.logic.pin_guard.is_others_pin_guardian"
-  },
-  {
-    "trigger": "scan_data",
-    "source": "exit_not_authorized_for_pin_reset",
-    "dest": "exit",
-    "conditions": "cic_ussd.state_machine.logic.menu.menu_nine_selected"
-  }
-]

apps/cic-ussd/var/lib/locale/helpers.en.yml

@@ -18,19 +18,4 @@ en:
   sent: |-
     Sent
   to: |-
-    To
-  guardians_list_header: |-
-    Walinzi uliowaongeza ni:
-  no_guardians_list: |-
-    No guardians set
-  error:
-    no_phone_number_provided: |-
-      No phone number was provided.
-    no_matching_account: |-
-      The number provided is not registered.
-    is_initiator: |-
-      Phone number cannot be your own.
-    is_existent_guardian: |-
-      This phone number is is already added as a guardian.
-    is_not_existent_guardian: |-
-      Phone number not set as PIN reset guardian.
+    To

apps/cic-ussd/var/lib/locale/helpers.sw.yml

@@ -18,19 +18,4 @@ sw:
   sent: |-
     Ulituma
   to: |-
-    Kwa
-  guardians_list_header: |-
-    Your set guardians are:
-  no_guardians_list: |-
-    Hamna walinzi walioongezwa
-  error:
-    no_phone_number_provided: |-
-      Namabari ya simu haijawekwa.
-    no_matching_account: |-
-      Nambari uliyoweka haijasajiliwa.
-    is_initiator: |-
-      Nambari yafaa kuwa tofauti na yako.
-    is_existent_guardian: |-
-      Namabari hii tayari imeongezwa kama mlinzi wa nambari ya siri.
-    is_not_existent_guardian: |-
-      Nambari hii haijaongezwa kama mlinzi wa nambari ya siri.
+    Kwa

apps/cic-ussd/var/lib/locale/ussd.en.yml

@@ -70,7 +70,7 @@ en:
       2. Change language
       3. Check balance
       4. Check statement
-      5. PIN options
+      5. Change PIN
       0. Back
     metadata_management: |-
       CON My profile
@@ -96,13 +96,6 @@ en:
       0. Back
     retry_pin_entry: |-
       CON Incorrect PIN entered, please try again. You have %{remaining_attempts} attempts remaining.
-      0. Back
-    pin_management: |-
-      CON Pin options
-      1. Change PIN
-      2. Reset PIN
-      3. Guard PIN
-      0. Back
     enter_current_pin:
       first: |-
         CON Enter current PIN.
@@ -115,73 +108,6 @@ en:
     new_pin_confirmation: |-
       CON Enter your new four number PIN again
       0. Back
-    reset_guarded_pin: |-
-      CON Enter phone number you are the guardian to reset their pin
-      0. Back
-    reset_guarded_pin_authorization:
-      first: |-
-        CON Enter YOUR pin to confirm %{guarded_account_information}'s reset
-        0. Back
-      retry: |-
-        %{retry_pin_entry}
-    exit_pin_reset_initiated_success: |-
-      CON Success: You have initiated a PIN reset for %{guarded_account_information}
-      0. Back
-      9. Exit
-    exit_not_authorized_for_pin_reset: |-
-      CON Failure: You are not authorized to reset that PIN. You must be a guardian!
-      0. Back
-      9. Exit
-    guard_pin: |-
-      CON Pin guard
-      1. View guardians
-      2. Add guardian
-      3. Remove guardian
-      0. Back
-    guardian_list_pin_authorization:
-      first: |-
-        CON Enter your pin to view set guardians
-        0. Back
-      retry: |-
-        %{retry_pin_entry}
-    guardian_list: |-
-      CON %{guardians_list}
-      0. Back
-      9. Exit
-    add_guardian: |-
-      CON Enter phone number to add as pin reset guardian
-      0. Back
-    add_guardian_pin_authorization:
-      first: |-
-        CON Enter your pin to add %{guardian_information} as your PIN reset guardian
-        0. Back
-      retry: |-
-        %{retry_pin_entry}
-    exit_guardian_addition_success: |-
-      CON Success: %{guardian_information} can now reset your PIN
-      0. Back
-      9. Exit
-    exit_invalid_guardian_addition: |-
-      CON %{error_exit}
-      0. Back
-      9. Exit
-    remove_guardian: |-
-      CON Enter phone number to revoke guardianship:
-      0. Back
-    remove_guardian_pin_authorization:
-      first: |-
-        CON Enter your pin to remove %{guardian_information} as your PIN reset guardian
-        0. Back
-      retry: |-
-        %{retry_pin_entry}
-    exit_guardian_removal_success: |-
-      CON Success: %{guardian_information} PIN reset guardianship is revoked
-      0. Back
-      9. Exit
-    exit_invalid_guardian_removal: |-
-      CON %{error_exit}
-      0. Back
-      9. Exit
     transaction_pin_authorization:
       first: |-
         CON %{recipient_information} will receive %{transaction_amount} %{token_symbol} from %{sender_information}.

apps/cic-ussd/var/lib/locale/ussd.sw.yml

@@ -69,7 +69,7 @@ sw:
       2. Chagua lugha utakayotumia
       3. Angalia salio
       4. Angalia taarifa ya matumizi
-      5. Mipangilio ya nambari ya siri
+      5. Badilisha nambari ya siri
       0. Nyuma
     metadata_management: |-
       CON Wasifu wangu
@@ -95,13 +95,6 @@ sw:
       0. Nyuma
     retry_pin_entry: |-
       CON Nambari uliyoweka si sahihi, jaribu tena. Una majaribio %{remaining_attempts} yaliyobaki.
-      0. Nyuma
-    pin_management: |-
-      CON Pin options
-      1. Badilisha nambari yangu ya siri
-      2. Tuma ombili la kubadilisha nambari ya siri
-      3. Linda nambari ya siri
-      0. Nyuma
     enter_current_pin:
       first: |-
         CON Weka nambari ya siri.
@@ -114,73 +107,6 @@ sw:
     new_pin_confirmation: |-
       CON Weka nambari yako ya siri tena
       0. Nyuma
-    reset_guarded_pin: |-
-      CON Weka nambari ya simu ili kutuma ombi la kubalisha nambari ya siri.
-      0. Nyuma
-    reset_guarded_pin_authorization:
-      first: |-
-        CON Weka nambari YAKO ya siri ili kudhibitisha ombi la kubadilisha nambari ya siri ya %{guarded_account_information}.
-        0. Nyuma
-      retry: |-
-        %{retry_pin_entry}
-    exit_pin_reset_initiated_success: |-
-      CON Ombi lako la kubadili nambari ya siri ya %{guarded_account_information} limetumwa.
-      0. Nyuma
-      9. Ondoka
-    exit_not_authorized_for_pin_reset: |-
-      CON Huruhusiwi kutuma ombi la kubadilisha nambari ya siri.
-      0. Nyuma
-      9. Ondoka
-    guard_pin: |-
-      CON Linda nambari ya siri
-      1. Walinzi wa namabari ya siri
-      2. Ongeza mlinzi
-      3. Ondoa mlinzi
-      0. Nyuma
-    guardian_list_pin_authorization:
-      first: |-
-        CON Weka nambari yako ya siri ili kuona walinzi uliowaongeza
-        0. Nyuma
-      retry: |-
-        %{retry_pin_entry}
-    guardian_list: |-
-      CON %{guardians_list}
-      0. Nyuma
-      9. Ondoka
-    add_guardian: |-
-      CON Weka nambari ya simu ili kuongeza mlinzi
-      0. Nyuma
-    add_guardian_pin_authorization:
-      first: |-
-        CON Weka nambari YAKO ya siri ili kumwongeza %{guardian_information} kama mlinzi
-        0. Nyuma
-      retry: |-
-        %{retry_pin_entry}
-    exit_guardian_addition_success: |-
-      CON Ombi lako la kumwongeza: %{guardian_information} kama mlinzi limefanikiwa
-      0. Nyuma
-      9. Ondoka
-    exit_invalid_guardian_addition: |-
-      CON %{error_exit}
-      0. Nyuma
-      9. Ondoka
-    remove_guardian: |-
-      CON Weka nambari ya simu ili kuondoa mlinzi
-      0. Nyuma
-    remove_guardian_pin_authorization:
-      first: |-
-        CON Weka nambari YAKO ya siri ili kumwondoa %{guardian_information} kama mlinzi
-        0. Nyuma
-      retry: |-
-        %{retry_pin_entry}
-    exit_guardian_removal_success: |-
-      CON Ombi lako la kumwondoa: %{guardian_information} kama mlinzi limefanikiwa
-      0. Nyuma
-      9. Ondoka
-    exit_invalid_guardian_removal: |-
-      CON %{error_exit}
-      0. Nyuma
-      9. Ondoka
     transaction_pin_authorization:
       first: |-
         CON %{recipient_information} atapokea %{transaction_amount} %{token_symbol} kutoka kwa %{sender_information}.

apps/contract-migration/3_deploy_token.sh

@@ -152,12 +152,6 @@ else
 	deploy_minter_${TOKEN_MINTER_MODE} $TOKEN_ADDRESS
 fi
 
->&2 echo -e "\033[;96mTransfer a single token to self to poke the gas cacher\033[;39m"
-advance_nonce
-debug_rpc
-r=`erc20-transfer $DEV_WAIT_FLAG --nonce $nonce $fee_price_arg -p $RPC_PROVIDER -y $WALLET_KEY_FILE -i $CHAIN_SPEC -u $DEV_DEBUG_FLAG -s -e $TOKEN_ADDRESS -a $DEV_ETH_ACCOUNT_CONTRACT_DEPLOYER 1`
-add_pending_tx_hash $r
-
 check_wait 3
 
 >&2 echo -e "\033[;96mWriting token metadata and proofs\033[;39m"

apps/contract-migration/4_init_custodial.sh

@@ -18,7 +18,6 @@ fi
 must_address "$CIC_REGISTRY_ADDRESS" "registry"
 must_eth_rpc
 
-
 # get required addresses from registries
 token_index_address=`eth-contract-registry-list -u -i $CHAIN_SPEC -p $RPC_PROVIDER -e $CIC_REGISTRY_ADDRESS $DEV_DEBUG_FLAG --raw TokenRegistry`
 accounts_index_address=`eth-contract-registry-list -u -i $CHAIN_SPEC -p $RPC_PROVIDER -e $CIC_REGISTRY_ADDRESS $DEV_DEBUG_FLAG --raw AccountRegistry`
@@ -29,11 +28,11 @@ REDIS_HOST_CALLBACK=${REDIS_HOST_CALLBACK:-$REDIS_HOST}
 REDIS_PORT_CALLBACK=${REDIS_PORT_CALLBACK:-$REDIS_PORT}
 >&2 echo -e "\033[;96mcreate account for gas gifter\033[;39m"
 gas_gifter=`cic-eth-create --redis-timeout 120 $DEV_DEBUG_FLAG --redis-host-callback $REDIS_HOST_CALLBACK --redis-port-callback $REDIS_PORT_CALLBACK --no-register`
-cic-eth-tag -i $CHAIN_SPEC --set --tag GAS_GIFTER $gas_gifter
+cic-eth-tag -i $CHAIN_SPEC GAS_GIFTER $gas_gifter
 
 >&2 echo -e "\033[;96mcreate account for accounts index writer\033[;39m"
 accounts_index_writer=`cic-eth-create --redis-timeout 120 $DEV_DEBUG_FLAG --redis-host-callback $REDIS_HOST_CALLBACK --redis-port-callback $REDIS_PORT_CALLBACK --no-register`
-cic-eth-tag -i $CHAIN_SPEC --set --tag ACCOUNT_REGISTRY_WRITER $accounts_index_writer
+cic-eth-tag -i $CHAIN_SPEC ACCOUNT_REGISTRY_WRITER $accounts_index_writer
 
 
 # Assign system writer for accounts index

apps/contract-migration/README.md

@@ -27,7 +27,6 @@ In the current version of the scripts, two token types may be deployed; [`giftab
 This step may be run multiple times, as long as the token symbol is different from all previously deployed tokens.
 
 
-
 ## 4. Initialize custodial engine.
 
 Adds system accounts to the custodial engine, and unlocks the initialization seal. After this step, the custodial system is ready to use.
@@ -36,12 +35,10 @@ Adds system accounts to the custodial engine, and unlocks the initialization sea
 ## Services dependency graph
 
 1. evm
-2. bootstrap runlevel 1 - deploy global contracts (RUN_MASK=1 docker-compose up bootstrap)
-3. bootstrap runlevel 2 - deploy instance contracts (RUN_MASK=2 docker-compose up bootstrap)
-4. bootstrap runlevel 4 - deploy token (RUN_MASK=4 docker-compose up bootstrap)
+2. bootstrap runlevel 1
+3. bootstrap runlevel 2
+4. bootstrap runlevel 3
 5. redis
 6. postgres
-7. cic-eth-tasker 
-8. boostrap runlevel 8 - deploy custodial contracts (RUN_MASK=8 docker-compose up bootstrap)
-9. boostrap runlevel 16 - data seeding for development (RUN_MASK=16 docker-compose up bootstrap)
-10. bring up the remainig services (docker-compose up -d)
+7. cic-eth-tasker
+8. boostrap runlevel 4

apps/contract-migration/config.sh

@@ -1,7 +1,6 @@
 #!/bin/bash
 
 set -a
-set -e
 
 if [ -z $DEV_DATA_DIR ]; then
 	export DEV_DATA_DIR=`mktemp -d`
@@ -34,7 +33,6 @@ else
 	fi
 	rm $bash_debug_flag -f ${DEV_DATA_DIR}/env_reset
 	rm $bash_debug_flag -f $noncefile
-	export SYNCER_OFFSET=`eth-info --raw block`
 	confini-dump --schema-dir ./config --prefix export > ${DEV_DATA_DIR}/env_reset
 fi
 
@@ -57,5 +55,4 @@ fi
 # Migration variable processing
 confini-dump --schema-dir ./config > ${DEV_DATA_DIR}/env_reset
 
-set +e
 set +a

apps/contract-migration/config/config.ini

@@ -25,6 +25,3 @@ port =
 [cic]
 registry_address =
 trust_address =
-
-[syncer]
-offset =

apps/contract-migration/docker/Dockerfile

@@ -19,7 +19,7 @@ COPY requirements.txt .
 #RUN apt-get install libffi-dev
 
 ARG pip_index_url=https://pypi.org/simple
-ARG EXTRA_PIP_INDEX_URL="https://pip.grassrootseconomics.net"
+ARG EXTRA_PIP_INDEX_URL="https://pip.grassrootseconomics.net:8433"
 ARG EXTRA_PIP_ARGS=""
 ARG PIP_INDEX_URL="https://pypi.org/simple"
 ARG pip_trusted_host=pypi.org

apps/contract-migration/requirements.txt

@@ -1,5 +1,5 @@
-cic-eth[tools]==0.12.5a2
-chainlib-eth>=0.0.10a17,<0.1.0
+cic-eth[tools]==0.12.4a13
+chainlib-eth>=0.0.10a15,<0.1.0
 eth-erc20>=0.1.2a3,<0.2.0
 erc20-demurrage-token>=0.0.5a2,<0.1.0
 eth-address-index>=0.2.4a1,<0.3.0

apps/data-seeding/README.md

@@ -6,22 +6,24 @@ This folder contains tools to generate and import test data.
 
 Three sets of tools are available, sorted by respective subdirectories.
 
-- **eth**: Import using sovereign wallets.
-- **cic_eth**: Import using the `cic_eth` custodial engine.
-- **cic_ussd**: Import using the `cic_ussd` interface (backed by `cic_eth`)
+* **eth**: Import using sovereign wallets.
+* **cic_eth**: Import using the `cic_eth` custodial engine.
+* **cic_ussd**: Import using the `cic_ussd` interface (backed by `cic_eth`)
 
 Each of the modules include two main scripts:
 
-- **import_users.py**: Registers all created accounts in the network
-- **import_balance.py**: Transfer an opening balance using an external keystore wallet
+* **import_users.py**: Registers all created accounts in the network
+* **import_balance.py**: Transfer an opening balance using an external keystore wallet
 
 The balance script will sync with the blockchain, processing transactions and triggering actions when it finds. In its current version it does not keep track of any other state, so it will run indefinitly and needs You the Human to decide when it has done what it needs to do.
 
+
 In addition the following common tools are available:
 
-- **create_import_users.py**: User creation script
-- **verify.py**: Import verification script
-- **cic_meta**: Metadata imports
+* **create_import_users.py**: User creation script
+* **verify.py**: Import verification script
+* **cic_meta**: Metadata imports
+
 
 ## REQUIREMENTS
 
@@ -34,12 +36,14 @@ source .venv/bin/activate
 
 Install all requirements from the `requirements.txt` file:
 
-`pip install --extra-index-url https://pip.grassrootseconomics.net -r requirements.txt`
+`pip install --extra-index-url https://pip.grassrootseconomics.net:8433 -r requirements.txt`
+
 
 If you are importing metadata, also do ye olde:
 
 `npm install`
 
+
 ## HOW TO USE
 
 ### Step 1 - Data creation
@@ -56,6 +60,7 @@ If you want to use a `import_balance.py` script to add to the user's balance fro
 
 `python create_import_users.py --gift-threshold <max_units_to_send> [--dir <datadir>] <number_of_users>`
 
+
 ### Step 2 - Services
 
 Unless you know what you are doing, start with a clean slate, and execute (in the repository root):
@@ -67,46 +72,46 @@ Then go through, in sequence:
 #### Base requirements
 
 If you are importing using `eth` and _not_ importing metadata, then the only service you need running in the cluster is:
-
-- eth
+* eth
 
 In all other cases you will _also_ need:
+* postgres
+* redis
 
-- postgres
-- redis
 
 #### EVM provisions
 
-This step is needed in _all_ cases.
+This step is needed in *all* cases.
 
 `RUN_MASK=1 docker-compose up contract-migration`
 
 After this step is run, you can find top-level ethereum addresses (like the cic registry address, which you will need below) in `<repository_root>/service-configs/.env`
 
+
 #### Custodial provisions
 
 This step is _only_ needed if you are importing using `cic_eth` or `cic_ussd`
 
 `RUN_MASK=2 docker-compose up contract-migration`
 
+
 #### Custodial services
 
 If importing using `cic_eth` or `cic_ussd` also run:
-
-- cic-eth-tasker
-- cic-eth-dispatcher
-- cic-eth-tracker
-- cic-eth-retrier
+* cic-eth-tasker
+* cic-eth-dispatcher
+* cic-eth-tracker
+* cic-eth-retrier
 
 If importing using `cic_ussd` also run:
-
-- cic-user-tasker
-- cic-user-ussd-server
-- cic-notify-tasker
+* cic-user-tasker
+* cic-user-ussd-server
+* cic-notify-tasker
 
 If metadata is to be imported, also run:
+* cic-meta-server
+
 
-- cic-meta-server
 
 ### Step 3 - User imports
 
@@ -120,7 +125,10 @@ All external balance transactions are saved in raw wire format in `<datadir>/txs
 
 If the contract migrations have been executed with the default "giftable" token contract, then the `token_symbol` in the `import_balance` scripts should be set to `GFT`.
 
-#### Alternative 1 - Sovereign wallet import - `eth`
+
+
+#### Alternative 1 - Sovereign wallet import - `eth` 
+
 
 First, make a note of the **block height** before running anything:
 
@@ -134,7 +142,9 @@ After the script completes, keystore files for all generated accouts will be fou
 
 Then run:
 
-`python eth/import_balance.py -v -r <cic_registry_address> -p <eth_provider> --token-symbol <token_symbol> --offset <block_height_at_start> -y ../keystore/UTC--2021-01-08T17-18-44.521011372Z--eb3907ecad74a0013c259d5874ae7f22dcbcc95c <datadir>`
+`python eth/import_balance.py -v -r <cic_registry_address> -p <eth_provider> --token-symbol <token_symbol> --offset <block_height_at_start> -y ../keystore/UTC--2021-01-08T17-18-44.521011372Z--eb3907ecad74a0013c259d5874ae7f22dcbcc95c <datadir>` 
+
+
 
 #### Alternative 2 - Custodial engine import - `cic_eth`
 
@@ -148,6 +158,7 @@ In another terminal:
 
 The `redis_hostname_in_docker` value is the hostname required to reach the redis server from within the docker cluster, and should be `redis` if you left the docker-compose unchanged. The `import_users` script will receive the address of each newly created custodial account on a redis subscription fed by a callback task in the `cic_eth` account creation task chain.
 
+
 #### Alternative 3 - USSD import - `cic_ussd`
 
 If you have previously run the `cic_ussd` import incompletely, it could be a good idea to purge the queue. If you have left docker-compose unchanged, `redis_url` should be `redis://localhost:63379`.
@@ -166,10 +177,13 @@ In the event that you are running the command in a local environment you may wan
 
 `python cic_ussd/import_users.py -v --ussd-host <user_ussd_server_host> --ussd-port <user_ussd_server_port> --ussd-no-ssl -c config out`
 
+
+
 ### Step 4 - Metadata import (optional)
 
 The metadata import scripts can be run at any time after step 1 has been completed.
 
+
 #### Importing user metadata
 
 To import the main user metadata structs, run:
@@ -180,18 +194,22 @@ Monitors a folder for output from the `import_users.py` script, adding the metad
 
 If _number of users_ is omitted the script will run until manually interrupted.
 
+
+
 #### Importing phone pointer
 
 `node cic_meta/import_meta_phone.js <datadir> <number_of_users>`
 
 If you imported using `cic_ussd`, the phone pointer is _already added_ and this script will do nothing.
 
+
 ### Importing preferences metadata
 
 `node cic_meta/import_meta_preferences.js <datadir> <number_of_users>`
 
 If you used the `cic_ussd/import_user.py` script to import your users, preferences metadata is generated and will be imported.
 
+
 ##### Importing pins and ussd data (optional)
 
 Once the user imports are complete the next step should be importing the user's pins and auxiliary ussd data. This can be done in 3 steps:
@@ -210,26 +228,26 @@ Once the creation of the pins file is complete, proceed to import the pins and u
 `python cic_ussd/import_pins.py -c config -v pinsdir <path to pin export dir tree>`
 
 - To import ussd data:
-  `python cic_ussd/import_ussd_data.py -c config -v userdir <path to the users export dir tree>`
+`python cic_ussd/import_ussd_data.py -c config -v userdir <path to the users export dir tree>`
 
 The balance script is a celery task worker, and will not exit by itself in its current version. However, after it's done doing its job, you will find "reached nonce ... exiting" among the last lines of the log.
 
 The connection parameters for the `cic-ussd-server` is currently _hardcoded_ in the `import_users.py` script file.
 
+
 ### Step 5 - Verify
 
-`python verify.py -v -c config -r <cic_registry_address> -p <eth_provider> --token-symbol <token_symbol> <datadir>`
+`python verify.py -v -c config -r <cic_registry_address> -p <eth_provider> --token-symbol <token_symbol> <datadir>` 
 
 Included checks:
-
-- Private key is in cic-eth keystore
-- Address is in accounts index
-- Address has gas balance
-- Address has triggered the token faucet
-- Address has token balance matching the gift threshold
-- Personal metadata can be retrieved and has exact match
-- Phone pointer metadata can be retrieved and matches address
-- USSD menu response is initial state after registration
+* Private key is in cic-eth keystore
+* Address is in accounts index
+* Address has gas balance
+* Address has triggered the token faucet
+* Address has token balance matching the gift threshold 
+* Personal metadata can be retrieved and has exact match
+* Phone pointer metadata can be retrieved and matches address
+* USSD menu response is initial state after registration
 
 Checks can be selectively included and excluded. See `--help` for details.
 
@@ -237,6 +255,7 @@ Will output one line for each check, with name of check and number of errors fou
 
 Should exit with code 0 if all input data is found in the respective services.
 
+
 ## KNOWN ISSUES
 
 - If the faucet disbursement is set to a non-zero amount, the balances will be off. The verify script needs to be improved to check the faucet amount.

apps/data-seeding/cic_eth/import_users.py

@@ -40,7 +40,7 @@ argparser = argparse.ArgumentParser()
 argparser.add_argument('-c', type=str, help='config override directory')
 argparser.add_argument('-i', '--chain-spec', dest='i', type=str, help='Chain specification string')
 argparser.add_argument('-f', action='store_true', help='force clear previous state')
-argparser.add_argument('--old-chain-spec', type=str, dest='old_chain_spec', default='evm:foo:1:oldchain', help='chain spec')
+argparser.add_argument('--old-chain-spec', type=str, dest='old_chain_spec', default='evm:oldchain:1', help='chain spec')
 argparser.add_argument('--redis-host', dest='redis_host', type=str, help='redis host to use for task submission')
 argparser.add_argument('--redis-port', dest='redis_port', type=int, help='redis host to use for task submission')
 argparser.add_argument('--redis-db', dest='redis_db', type=int, help='redis db to use for task submission and callback')
@@ -227,10 +227,9 @@ if __name__ == '__main__':
                     )
             os.makedirs(os.path.dirname(filepath), exist_ok=True)
            
-            sub_old_chain_str = '{}:{}'.format(old_chain_spec.network_id(), old_chain_spec.common_name())
-            logg.debug('u id {}'.format(u.identities))
+            sub_old_chain_str = '{}:{}'.format(old_chain_spec.common_name(), old_chain_spec.network_id())
             f = open(filepath, 'w')
-            k = u.identities['evm'][old_chain_spec.fork()][sub_old_chain_str][0]
+            k = u.identities['evm'][sub_old_chain_str][0]
             tag_data = {'tags': user_tags[strip_0x(k)]}
             f.write(json.dumps(tag_data))
             f.close()

apps/data-seeding/cic_ussd/import_balance.py

@@ -65,8 +65,7 @@ args_override = {
     'REDIS_DB': getattr(args, 'redis_db'),
     'META_HOST': getattr(args, 'meta_host'),
     'META_PORT': getattr(args, 'meta_port'),
-    'WALLET_KEY_FILE': getattr(args, 'y'),
-    'TOKEN_SYMBOL': getattr(args, 'token_symbol'),
+    'WALLET_KEY_FILE': getattr(args, 'y')
 }
 config.dict_override(args_override, 'cli flag')
 config.censor('PASSWORD', 'DATABASE')
@@ -111,7 +110,7 @@ def main():
                                                     config.get('CIC_REGISTRY_ADDRESS'),
                                                     signer_address,
                                                     signer)
-    ImportTask.balance_processor.init(config.get('TOKEN_SYMBOL'))
+    ImportTask.balance_processor.init(args.token_symbol)
     balances = {}
     accuracy = 10 ** 6
     count = 0

apps/data-seeding/config/token.ini

@@ -1,2 +0,0 @@
-[token]
-symbol =

apps/data-seeding/config/traffic.ini

@@ -1,4 +0,0 @@
-[traffic]
-#local.noop_traffic = 2
-#local.account = 2
-local.transfer = 2

apps/data-seeding/docker/Dockerfile

@@ -1,39 +1,29 @@
-ARG DOCKER_REGISTRY="registry.gitlab.com/grassrootseconomics"
-
-FROM $DOCKER_REGISTRY/cic-base-images:python-3.8.6-dev-e8eb2ee2
+# syntax = docker/dockerfile:1.2
+FROM registry.gitlab.com/grassrootseconomics/cic-base-images:python-3.8.6-dev-5ab8bf45
 
 WORKDIR /root
 
 RUN mkdir -vp /usr/local/etc/cic
 
-ARG NPM_REPOSITORY=${NPM_REPOSITORY:-https://registry.npmjs.org}
-RUN npm config set snyk=false
-#RUN npm config set registry={NPM_REPOSITORY}
-RUN npm config set registry=${NPM_REPOSITORY}
-
-# copy the dependencies
-COPY package.json package-lock.json ./
-RUN --mount=type=cache,mode=0755,target=/root/.npm \
-    npm set cache /root/.npm && \
-    npm cache verify && \
-    npm ci --verbose
+COPY package.json \
+     package-lock.json \
+     ./
 
 
-#RUN npm ci --production --verbose
+RUN npm ci --production 
 #RUN --mount=type=cache,mode=0755,target=/root/node_modules npm install
 
 COPY common/ cic_ussd/common/
 COPY requirements.txt .
 COPY config/ config
 
-ARG EXTRA_PIP_INDEX_URL=https://pip.grassrootseconomics.net
+ARG EXTRA_PIP_INDEX_URL=https://pip.grassrootseconomics.net:8433
 ARG EXTRA_PIP_ARGS=""
 ARG PIP_INDEX_URL=https://pypi.org/simple
 
 RUN  pip install --index-url $PIP_INDEX_URL  \
-     --pre \
-     --extra-index-url $EXTRA_PIP_INDEX_URL $EXTRA_PIP_ARGS \
-     -r requirements.txt
+	  --extra-index-url $EXTRA_PIP_INDEX_URL $EXTRA_PIP_ARGS \
+    -r requirements.txt
 
 COPY . . 
 

apps/data-seeding/import_ussd.sh

@@ -1,11 +1,5 @@
 #!/bin/bash
 
-. /tmp/cic/config/env_reset
-
-OUT_DIR=out
-CONFIG_DIR=config
-confini-dump --schema-dir $CONFIG_DIR
-
 if [[ -d "$OUT_DIR" ]]
 then
   echo -e "\033[;96mfound existing IMPORT DIR cleaning up...\033[;96m"
@@ -25,14 +19,14 @@ then
 fi
 
 echo -e "\033[;96mCreating seed data...\033[;96m"
-python create_import_users.py -vv  -c "$CONFIG_DIR" --dir "$OUT_DIR" "$NUMBER_OF_USERS"
+python create_import_users.py -vv  -c "$CONFIG" --dir "$OUT_DIR" "$NUMBER_OF_USERS"
 wait $!
 
 echo -e "\033[;96mCheck for running celery workers ...\033[;96m"
 if [ -f ./cic-ussd-import.pid ];
 then
   echo -e "\033[;96mFound a running worker. Killing ...\033[;96m"
-  kill -TERM $(<cic-ussd-import.pid)
+  kill -9 $(<cic-ussd-import.pid)
 fi
 
 echo -e "\033[;96mPurge tasks from celery worker\033[;96m"
@@ -43,11 +37,11 @@ if [ "$INCLUDE_BALANCES" != "y" ]
 then
   echo -e "\033[;96mRunning worker without opening balance transactions\033[;96m"
   TARGET_TX_COUNT=$NUMBER_OF_USERS
-  nohup python cic_ussd/import_balance.py -vv -c "$CONFIG_DIR" -p "$ETH_PROVIDER" -r "$CIC_REGISTRY_ADDRESS" --token-symbol "$TOKEN_SYMBOL" -y "$WALLET_KEY_FILE" "$OUT_DIR" > nohup.out 2> nohup.err < /dev/null &
+  nohup python cic_ussd/import_balance.py -vv -c "$CONFIG" -p "$ETH_PROVIDER" -r "$CIC_REGISTRY_ADDRESS" --token-symbol "$TOKEN_SYMBOL" -y "$WALLET_KEY_FILE" "$OUT_DIR" > nohup.out 2> nohup.err < /dev/null &
 else
   echo -e "\033[;96mRunning worker with opening balance transactions\033[;96m"
   TARGET_TX_COUNT=$((NUMBER_OF_USERS*2))
-  nohup python cic_ussd/import_balance.py -vv -c "$CONFIG_DIR" -p "$ETH_PROVIDER" -r "$CIC_REGISTRY_ADDRESS" --include-balances --token-symbol "$TOKEN_SYMBOL" -y "$WALLET_KEY_FILE" "$OUT_DIR" &
+  nohup python cic_ussd/import_balance.py -vv -c "$CONFIG" -p "$ETH_PROVIDER" -r "$CIC_REGISTRY_ADDRESS" --include-balances --token-symbol "$TOKEN_SYMBOL" -y "$WALLET_KEY_FILE" "$OUT_DIR" &
 fi
 
 echo -e "\033[;96mTarget count set to ${TARGET_TX_COUNT}"
@@ -59,12 +53,12 @@ done
 IMPORT_BALANCE_JOB=$(<cic-import-ussd.pid)
 
 echo -e "\033[;96mStart import users job\033[;96m"
-if [ ! -z "$USSD_SSL" ]
+if [ "$USSD_SSL" == "y" ]
 then
   echo -e "\033[;96mTargeting secure ussd-user server\033[;96m"
-  python cic_ussd/import_users.py -vv -f -c "$CONFIG_DIR" --ussd-host "$USSD_HOST" --ussd-port "$USSD_PORT" "$OUT_DIR"
+  python cic_ussd/import_users.py -vv -f -c "$CONFIG" --ussd-host "$USSD_HOST" --ussd-port "$USSD_PORT" "$OUT_DIR"
 else
-  python cic_ussd/import_users.py -vv -f -c "$CONFIG_DIR" --ussd-host "$USSD_HOST" --ussd-port "$USSD_PORT" --ussd-no-ssl "$OUT_DIR"
+  python cic_ussd/import_users.py -vv -f -c "$CONFIG" --ussd-host "$USSD_HOST" --ussd-port "$USSD_PORT" --ussd-no-ssl "$OUT_DIR"
 fi
 
 echo -e "\033[;96mWaiting for import balance job to complete ...\033[;96m"
@@ -72,13 +66,13 @@ tail --pid="$IMPORT_BALANCE_JOB" -f /dev/null
 set -e
 
 echo -e "\033[;96mImporting pins\033[;96m"
-python cic_ussd/import_pins.py -c "$CONFIG_DIR" -vv "$OUT_DIR"
+python cic_ussd/import_pins.py -c "$CONFIG" -vv "$OUT_DIR"
 set +e
 wait $!
 set -e
 
 echo -e "\033[;96mImporting ussd data\033[;96m"
-python cic_ussd/import_ussd_data.py -c "$CONFIG_DIR" -vv "$OUT_DIR"
+python cic_ussd/import_ussd_data.py -c "$CONFIG" -vv "$OUT_DIR"
 set +e
 wait $!
 
@@ -88,9 +82,8 @@ node cic_meta/import_meta.js "$OUT_DIR" "$NUMBER_OF_USERS"
 echo -e "\033[;96mImport preferences metadata\033[;96m"
 node cic_meta/import_meta_preferences.js "$OUT_DIR" "$NUMBER_OF_USERS"
 
-CIC_NOTIFY_DATABASE=postgres://$DATABASE_USER:$DATABASE_PASSWORD@$DATABASE_HOST:$DATABASE_PORT/$DATABASE_NAME_NOTIFY
+CIC_NOTIFY_DATABASE=postgres://$DATABASE_USER:$DATABASE_PASSWORD@$DATABASE_HOST:$DATABASE_PORT/$NOTIFY_DATABASE_NAME
 NOTIFICATION_COUNT=$(psql -qtA "$CIC_NOTIFY_DATABASE" -c 'SELECT COUNT(message) FROM notification WHERE message IS NOT NULL')
-# TODO: add max wait
 while (("$NOTIFICATION_COUNT" < "$TARGET_TX_COUNT" ))
 do
   NOTIFICATION_COUNT=$(psql -qtA "$CIC_NOTIFY_DATABASE" -c 'SELECT COUNT(message) FROM notification WHERE message IS NOT NULL')

apps/data-seeding/requirements.txt

@@ -1,10 +1,10 @@
 sarafu-faucet~=0.0.7a2
 cic-eth[tools]~=0.12.4a13
-cic-types~=0.2.1a2
+cic-types~=0.2.0a6
 funga>=0.5.1a1,<=0.5.15
 faker==4.17.1
 chainsyncer~=0.0.7a3
-chainlib-eth~=0.0.10a18
+chainlib-eth~=0.0.10a10
 eth-address-index~=0.2.4a1
 eth-contract-registry~=0.6.3a3
 eth-accounts-index~=0.1.2a3

apps/data-seeding/verify.py

@@ -60,7 +60,7 @@ eth_tests = [
         ]
 
 phone_tests = [
-#        'ussd',
+        'ussd',
         'ussd_pins'
         ]