{"action":"created","issue":{"url":"https://api.github.com/repos/golemfactory/concent/issues/479","repository_url":"https://api.github.com/repos/golemfactory/concent","labels_url":"https://api.github.com/repos/golemfactory/concent/issues/479/labels{/name}","comments_url":"https://api.github.com/repos/golemfactory/concent/issues/479/comments","events_url":"https://api.github.com/repos/golemfactory/concent/issues/479/events","html_url":"https://github.com/golemfactory/concent/issues/479","id":331169671,"node_id":"MDU6SXNzdWUzMzExNjk2NzE=","number":479,"title":"[WIP][Blueprint] Bankster","user":{"login":"cameel","id":137030,"node_id":"MDQ6VXNlcjEzNzAzMA==","avatar_url":"https://avatars3.githubusercontent.com/u/137030?v=4","gravatar_id":"","url":"https://api.github.com/users/cameel","html_url":"https://github.com/cameel","followers_url":"https://api.github.com/users/cameel/followers","following_url":"https://api.github.com/users/cameel/following{/other_user}","gists_url":"https://api.github.com/users/cameel/gists{/gist_id}","starred_url":"https://api.github.com/users/cameel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cameel/subscriptions","organizations_url":"https://api.github.com/users/cameel/orgs","repos_url":"https://api.github.com/users/cameel/repos","events_url":"https://api.github.com/users/cameel/events{/privacy}","received_events_url":"https://api.github.com/users/cameel/received_events","type":"User","site_admin":false},"labels":[{"id":822561969,"node_id":"MDU6TGFiZWw4MjI1NjE5Njk=","url":"https://api.github.com/repos/golemfactory/concent/labels/blueprint","name":"blueprint","color":"0052cc","default":false}],"state":"open","locked":false,"assignee":{"login":"cameel","id":137030,"node_id":"MDQ6VXNlcjEzNzAzMA==","avatar_url":"https://avatars3.githubusercontent.com/u/137030?v=4","gravatar_id":"","url":"https://api.github.com/users/cameel","html_url":"https://github.com/cameel","followers_url":"https://api.github.com/users/cameel/followers","following_url":"https://api.github.com/users/cameel/following{/other_user}","gists_url":"https://api.github.com/users/cameel/gists{/gist_id}","starred_url":"https://api.github.com/users/cameel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cameel/subscriptions","organizations_url":"https://api.github.com/users/cameel/orgs","repos_url":"https://api.github.com/users/cameel/repos","events_url":"https://api.github.com/users/cameel/events{/privacy}","received_events_url":"https://api.github.com/users/cameel/received_events","type":"User","site_admin":false},"assignees":[{"login":"cameel","id":137030,"node_id":"MDQ6VXNlcjEzNzAzMA==","avatar_url":"https://avatars3.githubusercontent.com/u/137030?v=4","gravatar_id":"","url":"https://api.github.com/users/cameel","html_url":"https://github.com/cameel","followers_url":"https://api.github.com/users/cameel/followers","following_url":"https://api.github.com/users/cameel/following{/other_user}","gists_url":"https://api.github.com/users/cameel/gists{/gist_id}","starred_url":"https://api.github.com/users/cameel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cameel/subscriptions","organizations_url":"https://api.github.com/users/cameel/orgs","repos_url":"https://api.github.com/users/cameel/repos","events_url":"https://api.github.com/users/cameel/events{/privacy}","received_events_url":"https://api.github.com/users/cameel/received_events","type":"User","site_admin":false}],"milestone":null,"comments":3,"created_at":"2018-06-11T12:32:55Z","updated_at":"2018-06-20T15:19:07Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"## Bankster\r\n\r\nBankster is an intermediate layer between Concent Core and the Ethereum Client (Geth).\r\nAll communication with the Ethereum Client is performed via Golem's Smart Contracts Interface (SCI) which is used as a library.\r\n\r\n### Components and communication\r\n![bankster-components](https://user-images.githubusercontent.com/137030/41231712-4e124bfa-6d84-11e8-8be2-3f085ff1d3dc.png)\r\n\r\n### API\r\n\r\n#### `POST bankster/claim-deposit/` endpoint\r\n- **Input format**: JSON\r\n- **Output format**: JSON\r\n\r\nThe purpose of this endpoint is to check whether the clients participating in a use case have enough funds in their deposits to cover all the costs associated with the use case in the pessimistic scenario and to mark those funds as locked until they get paid out.\r\n\r\nConcent core communicates with this endpoint at the beginning of any use case that may require payment from deposit within a single subtask.\r\nCurrently those are `ForcedAcceptance` and `AdditionalVerification`.\r\n`ForcedPayment` use case operates on more than one subtask and for that reason requires a separate endpoint.\r\n\r\nThe funds are locked in anticipation of a future payment but it does not necessarily mean that exactly that amount will be paid out or even that it will actually be paid.\r\nThe actual amount paid will depend on the outcome of the use case.\r\n\r\n##### Input\r\n| Parameter                       | Type                  | Optional | Remarks                                                                                                                 |\r\n|---------------------------------|-----------------------|----------|-------------------------------------------------------------------------------------------------------------------------|\r\n| `subtask_id`                    | string                | no       | ID of the subtask. It's not a foreign key to `Subtask` because in some use cases (e.g. payment use case) we do not store the subtask details.\r\n| `concent_use_case`              | `ConcentUseCase`      | no       | Use case in which Concent claim the deposit.\r\n| `requestor_public_key`          | string                | no       | Public key of the requestor. Comes from `TaskToCompute`.\r\n| `provider_public_key`           | string                | no       | Public key of the provider. Comes from `TaskToCompute`.\r\n| `requestor_ethereum_public_key` | string                | no       | Address of the Ethereum account belonging to the requestor. Cannot be the same as `provider_ethereum_public_key`. Comes from `TaskToCompute`.\r\n| `provider_ethereum_public_key`  | string                | no       | Address of the Ethereum account belonging to the provider. Cannot be the same as `requestor_ethereum_public_key`. Comes from `TaskToCompute`.\r\n| `subtask_cost`                  | decimal               | no       | The cost of performing the work on the subtask that the requestor has agreed to pay the provider. Comes from `TaskToCompute.amount`. Must be greater than zero.\r\n\r\n##### Output\r\n| Parameter                       | Type                  | Optional | Remarks                                                                                                                 |\r\n|---------------------------------|-----------------------|----------|-------------------------------------------------------------------------------------------------------------------------|\r\n| `claim_against_requestor`       | int                   | no       | The database ID of the `DepositClaim` object that was created to lock a part of requestor's deposit.\r\n| `claim_against_provider`        | int                   | yes      | The database ID of the `DepositClaim` object that was created to lock a part of provider's deposit. May be empty if nothing is claimed from the provider.\r\n\r\n##### Sequence of operation\r\n1. **Claim calculation**: Bankster determines the amount that needs to be claimed from each account:\r\n    - In `ForcedAcceptance` use case:\r\n        - Requestor may have to pay `subtask_cost`.\r\n        - Provider does not pay anything.\r\n    - In `AdditionalVerification` use case:\r\n        - Requestor may have to pay `subtask_cost`.\r\n        - Provider needs to pay verification cost (which is constant and determined by Concent's settings).\r\n2. **Initialization**:\r\n    - Bankster creates `Client` and `DepositAccount` objects for the requestor and also for the provider if there's a non-zero claim against his account.\r\n        - The transaction is committed at this point to prevent these objects from being rolled back in case of failure.\r\n3. **Deposit check**:\r\n    - Bankster asks SCI about the amount of funds available in requestor's deposit.\r\n    - If the amount claimed from provider's deposit is non-zero, Bankster asks SCI about the amount of funds available in his deposit.\r\n4. **Claim freeze**:\r\n    - Bankster puts a database locks on all `DepositAccount` objects that will be used as payers in newly created `DatabaseClaim`s.\r\n        - If the objects are already locked, waits and retries a few times and eventually fails.\r\n    - The freeze is done after the deposit check to avoid holding the lock for too long.\r\n        Deposit check requires one or more HTTP requests and is a relatively slow operation.\r\n5. **Requestor's deposit check**:\r\n    - Bankster sums the `amount`s of all existing `DepositClaim`s where the requestor is the payer.\r\n    - **Claims against requestor's deposit can be paid partially** because the service has already been performed by the provider and giving him something is better than giving nothing.\r\n        If the existing claims against requestor's deposit are greater or equal to his current deposit, we can't add a new claim.\r\n        - Bankster responds with `claim_against_requestor` and `claim_against_requestor` being both `None`.\r\n    - Otherwise Bankster continues.\r\n6. **Provider's deposit check**: If the amount claimed from his deposit is non-zero.\r\n    - Bankster sums the `amount`s of all existing `DepositClaim`s where the provider is the payer.\r\n    - **Claims against provider's deposit must be paid in full** because they're payments for using Concent and we did not perform the service yet so we can just refuse.\r\n      If the total of existing claims **and** the current claim is greater or equal to the current deposit, we can't add a new claim.\r\n        - Bankster removes the `DepositClaim` just created aginst requestor's deposit.\r\n        - Bankster responds with `claim_against_requestor` and `claim_against_requestor` being both `None`.\r\n    - Otherwise Bankster continues.\r\n7. **Deposit lock**: For both provider and requestor, if the amount claimed from the deposit is non-zero:\r\n    - Bankster creates a `DepositClaim` object\r\n        - `payee_ethereum_public_key`: payee can be either the provider (if the requetor is the payer) or Concent (if the provider is the payer).\r\n            The address of Concent's account either comes from settings or is automatically inserted by the contract.\r\n        - `amount`: The full amount claimed.\r\n            Even if there's not enough deposit, we don't reduce the claim until the moment when we actually create a transaction.\r\n            The situation may change in the meantime - some claims may be canceled or the client may increase (or reduce) the deposit.\r\n        - `tx_hash`: We're not creating a transaction yet so this field is left empty.\r\n8. **Unfreeze and response**: If everything goes well, Bankster sends a HTTP 200 response.\r\n    - IDs of all the created `DepositClaim`s are included in the response.\r\n        At least one claim must have been created.\r\n    - Database locks on `EthereumAccount`s are released.\r\n\r\n##### Expected Concent Core behavior\r\n- Concent Core contacts this endpoint at the beginning of a use case, after validating client's message and determining that the service can be performed.\r\n- Concent Core can access `DepositClaim` objects created by Bankster.\r\n- Concent Core is responsible for removing `DepositClaim` objects if it fails at any point after receiving a positive response from Bankster.\r\n    - Bankster removes `DepositClaim` objects only after they're successfully paid for.\r\n- Concent Core is responsible for removing `DepositClaim` objects if the outcome of the use case indicates that they don't need to be paid (e.g. if additional verification confirms requestor's decision to reject the result).\r\n- `Client` and `DepositAccount` objects are never removed.\r\n    They stay in the database even if the operation fails and even after all the `DepositClaim`s that refer to them are removed.\r\n\r\n#### `POST bankster/trigger-payments/` endpoint\r\n- **Input format**: JSON\r\n- **Output format**: JSON\r\n\r\n##### Input\r\n| Parameter                     | Type                  | Optional | Remarks                                                                                                                 |\r\n|-------------------------------|-----------------------|----------|-------------------------------------------------------------------------------------------------------------------------|\r\n| `deposit_claim_ids`           | list of int           | no       | The list of database IDs of the created `DepositClaim` objects for which payment transactions should be submitted.\r\n\r\n##### Output\r\n| Parameter                     | Type                  | Optional | Remarks                                                                                                                 |\r\n|-------------------------------|-----------------------|----------|-------------------------------------------------------------------------------------------------------------------------|\r\n| `success`                     | list bool             | no       | A value indicating for each transaction whether it has been successfully submitted. The order matches the order of `deposit_claim_ids` in the input.\r\n\r\n#### `POST bankster/settle-overdue-acceptances/` endpoint\r\n- **Input format**: JSON\r\n- **Output format**: JSON\r\n\r\n##### Input\r\n| Parameter                     | Type                        | Optional | Remarks                                                                                                                 |\r\n|-------------------------------|-----------------------------|----------|-------------------------------------------------------------------------------------------------------------------------|\r\n| `payer_ethereum_public_key`   | string                      | no       | Address of the Ethereum account belonging to the client (requestor) who is supposed to pay the claim. Cannot be the same as `payee_ethereum_public_key`.\r\n| `payee_ethereum_public_key`   | string                      | no       | Address of the Ethereum account belonging to the client (provider) who is supposed to receive the claim. Cannot be the same as `payer_ethereum_public_key`.\r\n| `acceptances`                 | list of `SubtaskAcceptance` | no       | List of dicts describing each acceptance.\r\n\r\n###### `SubtaskAcceptance`\r\n| Field                         | Type                        | Optional | Remarks                                                                                                                 |\r\n|-------------------------------|-----------------------------|----------|-------------------------------------------------------------------------------------------------------------------------|\r\n| `subtask_id`                  | string                      | no       | ID of the subtask.\r\n| `payment_ts`                  | timestamp                   | no       | `payment_ts` timestamp from the acceptance message.\r\n| `amount`                      | decimal                     | no       | Amount to be paid.\r\n\r\n##### Output\r\nEmpty\r\n\r\n### Database\r\n\r\n#### `DepositClaim` model\r\n- **Database**: control\r\n\r\n| Column name                   | Type                  | Optional | Remarks                                                                                                                 |\r\n|-------------------------------|-----------------------|----------|-------------------------------------------------------------------------------------------------------------------------|\r\n| `subtask_id`                  | string                | no       | ID of the subtask. It's not a foreign key to `Subtask` because in some use cases (e.g. payment use case) we do not store the subtask details.\r\n| `payer_deposit_account`       | `DepositAccount`      | no       | The deposit account belonging to the client who is supposed to pay the claim. `payer_deposit_account.ethereum_public_key` cannot be the same as `payee_ethereum_public_key`.\r\n| `payee_ethereum_public_key`   | string                | no       | Address of the Ethereum account belonging to the entity (requestor, provider or Concent) who is supposed to receive the claim. Cannot be the same as `payer_deposit_account.ethereum_public_key`.\r\n| `concent_use_case`            | `ConcentUseCase`      | no       | Use case in which Concent claim the deposit.\r\n| `amount`                      | decimal               | no       | The amount claimed. Must be greater than zero.\r\n| `tx_hash`                     | string                | yes      | The hash of the Ethereum transaction that will cover the claim. Empty if the transaction has not been created yet.\r\n| `created_at`                  | timestamp             | no       | The creation time of the object.\r\n| `updated_at`                  | timestamp             | no       | Time of the last modification of any field in the object.\r\n\r\n#### `DepositAccount` model\r\n- **Database**: control\r\n\r\nThe main purpose of this object is to allow us to put a database lock on all `DepositClaim`s belonging to a specific payer, though it may get another purpose in the future.\r\nPutting locks directly on `DepositClaim` would not work when there are no claims yet.\r\n\r\nWe never create `DepositAccount` for the payee because the payee may be Concent itself.\r\n\r\n| Column name                   | Type                  | Optional | Remarks                                                                                                                 |\r\n|-------------------------------|-----------------------|----------|-------------------------------------------------------------------------------------------------------------------------|\r\n| `client`                      | `Client`              | no       | `Client` who owns the deposit on this account.\r\n| `ethereum_public_key`         | string                | no       | Address of the Ethereum account belonging to the client who owns the deposit account.\r\n\r\n#### ConcentUseCase enum\r\n- `ForcedAcceptance`\r\n- `AdditionalVerification`\r\n- `ForcedPayment`"},"comment":{"url":"https://api.github.com/repos/golemfactory/concent/issues/comments/398789730","html_url":"https://github.com/golemfactory/concent/issues/479#issuecomment-398789730","issue_url":"https://api.github.com/repos/golemfactory/concent/issues/479","id":398789730,"node_id":"MDEyOklzc3VlQ29tbWVudDM5ODc4OTczMA==","user":{"login":"cameel","id":137030,"node_id":"MDQ6VXNlcjEzNzAzMA==","avatar_url":"https://avatars3.githubusercontent.com/u/137030?v=4","gravatar_id":"","url":"https://api.github.com/users/cameel","html_url":"https://github.com/cameel","followers_url":"https://api.github.com/users/cameel/followers","following_url":"https://api.github.com/users/cameel/following{/other_user}","gists_url":"https://api.github.com/users/cameel/gists{/gist_id}","starred_url":"https://api.github.com/users/cameel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cameel/subscriptions","organizations_url":"https://api.github.com/users/cameel/orgs","repos_url":"https://api.github.com/users/cameel/repos","events_url":"https://api.github.com/users/cameel/events{/privacy}","received_events_url":"https://api.github.com/users/cameel/received_events","type":"User","site_admin":false},"created_at":"2018-06-20T15:19:07Z","updated_at":"2018-06-20T15:19:07Z","author_association":"CONTRIBUTOR","body":"I don't see why.\r\n\r\nLet's say you do `POST bankster/trigger-payments/` with `deposit_claim_ids=[1, 2, 3, 4]`. You must have four `DepositClaim` objects in your database and both Bankster and Concent Core can access them. First for example with `id=1` and `subtask_id='aaa'`. Second with `id=2` and `subtask_id='bbb'` and so on. You get `success=[True, False, True, True]` in response. So you know that deposit `2` associated with subtask `'bbb'` is the one that failed.\r\n\r\nIf you want to process the response you just go over both lists:\r\n``` python\r\nfor deposit_claim, success in zip(deposit_claims, success_list):\r\n    if not success:\r\n        print(f\"Failed to pay for subtask {desposit_claim.subtask_id}\")\r\n```"}}
{"actor":{"display_login":"cameel"}}