AaronFiore/dfi-docs
1
0
Fork 0
forked from EvergreenCrypto/dfi-docs
Code Activity
Files
d2c26e2674b19f949ad7b18c9b5fe477fa093f4c
dfi-docs/markdown/How-do-I-use-it.md

400 lines
22 KiB
Markdown
Raw Blame History

[//]: # (docker-finance | modern accounting for the power-user)
[//]: # ()
[//]: # (Copyright [C] 2021-2026 Aaron Fiore [Founder, Evergreen Crypto LLC])
[//]: # ()
[//]: # (This program is free software: you can redistribute it and/or modify)
[//]: # (it under the terms of the GNU General Public License as published by)
[//]: # (the Free Software Foundation, either version 3 of the License, or)
[//]: # ([at your option] any later version.)
[//]: # ()
[//]: # (This program is distributed in the hope that it will be useful,)
[//]: # (but WITHOUT ANY WARRANTY; without even the implied warranty of)
[//]: # (MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the)
[//]: # (GNU General Public License for more details.)
[//]: # ()
[//]: # (You should have received a copy of the GNU General Public License)
[//]: # (along with this program. If not, see <https://www.gnu.org/licenses/>.)
# [<img src="../assets/branding/png/dfi.png" height=10% width=10%/>](https://gitea.evergreencrypto.co/EvergreenCrypto/docker-finance "docker-finance")
- **[How do I use it?](#how-do-i-use-it)**
* [Mostly-Unified CLI](#mostly-unified-cli)
* [Flow Layout](#flow-layout)
* [Plugins](#plugins)
* [Caveats & Oddities](#caveats--oddities)
* [FAQ](#faq)
## How do I use it?
### Mostly-Unified CLI
You'll only need the single alias `dfi` when using docker-finance on your client (host) and/or within your container.
However, this alias can be broken down into the following:
1. The client script which handles the client-side (host) system: `docker.bash` (alias `dfi`)
2. The container script which handles the container-side system: `finance.bash` (alias `dfi`)
These two scripts can be rationalized by the following format:
<script> <super/sub> <command> [args]
For example, these [Screenshots](What-does-it-do.md#screenshots) describe a setup with mockup data where the client (host) user named `business`, along with container `$DOCKER_FINANCE_USER` named `business`, engage in client/container activity. The container profile named `testprofile` and its subprofile named `testuser` can be described as `<super/sub>` portion of the format (`testprofile/testuser`).
It should be noted that, for your convenience:
- command arguments [args] can be arranged in any order
- commandline completion is available for all `dfi <super/sub> command [args]` (save your fingers and tab away!)
- For client (host), an image must first be built for completion to be available
---
#### Client (Host) Command Format
The client (host) command format consists of:
docker.bash <platform/username:tag> <command> [args]
Where:
- `docker.bash` is located in `${DOCKER_FINANCE_CLIENT_REPO}/client`
- `<platform/username:tag>`
- `platform` is the image platform (archlinux, ubuntu)
- `username` is the client (host) username with read/write permissions to the container (see [Configuration Files](How-do-I-get-started.md#configuration-files))
- `tag` is a custom tag you can use to delineate any number of images that you wish you create (`latest`, `dev`, etc.)
- `<command>` is the command to pass to `docker.bash`
- `[args]` are the (optional) arguments to pass to `<command>`
If the [Installation](How-do-I-get-started.md#installation) was successful, `docker.bash` is aliased to `docker-finance` and `dfi` (either can be used).
For a complete list of commands and usage help (`<platform/user:tag>` *not* required):
```bash
dfi help
```
To view the help usage of a specific command, for example; the `edit` command (`<platform/user:tag>` *is* required):
```bash
dfi archlinux/${USER}:default edit help
```
---
#### Container Command Format
The container command format consists of:
finance.bash <profile/subprofile> <command> [args]
Where:
- `finance.bash` is located in `${DOCKER_FINANCE_CLIENT_REPO}/container`
- `<profile/subprofile>`
- `profile` is the profile, as defined during [Environment Generation](How-do-I-get-started.md#environment-generation)
- `subprofile` is the subprofile (user), as defined during [Environment Generation](How-do-I-get-started.md#environment-generation)
- `<command>` is the command to pass to `finance.bash`
- `[args]` are the (optional) arguments to pass to `<command>`
By default, `finance.bash` is aliased to `finance` and `dfi` (either can be used).
For a complete list of commands (`<profile/subprofile>` *not* required):
```bash
dfi help
```
To view the help usage of a specific command, for example; the `fetch` command (`<profile/subprofile>` *is* required):
Assuming `<profile/subprofile>` is `testprofile/testuser`:
```bash
dfi testprofile/testuser fetch help
```
Or, use a subprofile alias, as described in [Subscript](How-do-I-get-started.md#subscript):
```bash
testuser_fetch help
```
### Flow Layout
A primary read through of [hledger](https://hledger.org) and [hledger-flow](https://github.com/apauley/hledger-flow/blob/master/README.org) documentation should bring you up to speed on most of the essentials.
As for docker-finance specifics, you can create a *test profile* during [Environment Generation](How-do-I-get-started.md#environment-generation) to see what your flow's layout should look like.
> Note: be sure to select 'y' when asked if this will be a development profile, and then go on to create account(s).
Once inside the container, assuming you created a profile named `testprofile` and subprofile named `testuser`, issue the following commands:
```bash
finance testprofile/testuser fetch all=price api=mobula year=all && finance testprofile/testuser import year=2018
```
> Note: for this *test profile* with developer mockups, you **MUST** import from `2018` as there are accounts that begin from that year
After experimenting with a *test profile*, you can re-run `gen` again to create a regular profile.
#### Profiles
All profiles/subprofiles are installed into the parent directory `${DOCKER_FINANCE_CONTAINER_FLOW}/profiles`.
Peeking inside `${DOCKER_FINANCE_CONTAINER_FLOW}/profiles/profile/subprofile`, you'll see the following:
- `all-years.journal` and `directives.journal`
- These top-level journals are generated by `hledger-flow`. Ignore these and use the container `edit` command for all journal editing.
- `conf.d`
- Location of all docker-finance configuration files (see `edit help` for details).
- `import`
- Location of all CSV data and *real* journals. This is where you'll place CSV files and custom account/subaccount changes (see `edit help` for details).
- `prices`
- Location of all market price data, by year, as acquired by `fetch price` (see `fetch help` for details).
- `reports`
- Location of all generated reports, by year, as generated by `reports` (see `reports help` for details).
- `taxes`
- Location of all generated taxes, by year, as generated by `taxes` (see `taxes help` for details).
Note:
- For manual CSV downloads, place you CSV file into your `${DOCKER_FINANCE_CONTAINER_FLOW}/profiles/profile/subprofile/import/subprofile/account/subaccount/1-in/year` directory (replacing `year` with the year of data that the file/data represents). See `import help` for details.
- When you want to edit custom settings for an account or a subaccount, use the container `edit` command. See container's `edit help` for details.
#### Times
All `times` related files will reside in `${DOCKER_FINANCE_CONTAINER_FLOW}/times` (this includes the timewarrior database).
See the container `times help` command for details.
### Plugins
Plugins (pluggables) allow you to leverage `dfi` client/container APIs, libraries and environments to meet your unique needs.
Checkout this two-part client/container set of [Bitcoin plugins](#plugins-bitcoin) to see how all APIs/libraries/environments can work together in unison.
For more information and other examples, see some of the various existing plugins (pluggables) and respective help usage, i.e.;
- Client-side (host): `dfi <platform/user:tag> plugins help`
- Container-side: `dfi <profile/subprofile> plugins help`
- Within `root` interpreter: `dfi::help()`
#### Plugins: Layout
Plugins are categorical:
1. Repository (`repo`) plugins
- These plugins remain within the repository and will require a pull request for any changes to be made to them.
2. End-user (`custom`) plugins
- These plugins remain on your client (host) and are bind-mounted to your container; allowing you to drop-in any code that you write while keeping them local and private.
Within these categories are subcategories where plugins exist either client-side (host) or container-side; meaning, they rely upon (or operate within) their respective client/container APIs/libraries/environments:
1. `client` | *Tends to operate only client-side (host) but can also utilize a container*
- `docker` | *Operates **only** from the `bash` shell*
* Called client-side with `dfi <platform/user:tag> plugins`
* Can be any language so long as:
- The file is executable by the shell
- The plugin reads the shell environment
- The plugin initializes the respective library (`lib_docker`)
2. `container` | *Operates **only** within the container*
- `finance` | *Operates **only** from the `bash` shell*
* Called container-side with `dfi <profile/subprofile> plugins`
* Can be any language so long as:
- The file is executable by the shell
- The plugin reads the shell environment
- The plugin initializes the respective library (`lib_finance`)
- `root` | *Operates **only** within the `root` interpreter*
* Called container-side by either two different ways:
- Within a running `dfi <profile/subprofile> root` instance:
* `dfi::plugin::load("repo/a_repo_plugin/plugin.cc")`
* `dfi::plugin::load("custom/my_custom_plugin/plugin.cc")`
- With the `dfi <profile/subprofile> root plugins` command
* Use tab completion to see available plugins (pluggables)
* Can only be languages supported by the interpreter (C/C++)
To mirror these categories, a client-side `custom` plugin directory layout is generated upon `dfi <platform/user:tag> gen`. This layout consists of:
1. `${DOCKER_FINANCE_CLIENT_PLUGINS}/client/docker`
- This path remains client-side only (not bind-mounted)
- This layout mirrors `repo` plugins [`client/plugins`](https://gitea.evergreencrypto.co/EvergreenCrypto/docker-finance/src/branch/master/client/plugins)
2. `${DOCKER_FINANCE_CLIENT_PLUGINS}/container/{finance,root}`
- The container directory is bind-mounted to `DOCKER_FINANCE_CONTAINER_PLUGINS`
- This layout mirrors `repo` plugins [`container/plugins`](https://gitea.evergreencrypto.co/EvergreenCrypto/docker-finance/src/branch/master/container/plugins)
**WARNING: don't change the above expected layout!** However, you can add subdirectories, e.g.;
- `${DOCKER_FINANCE_CLIENT_PLUGINS}/client/docker/my_docker_plugins/plugin.bash`
- `${DOCKER_FINANCE_CLIENT_PLUGINS}/container/finance/my_finance_plugins/plugin.bash`
- `${DOCKER_FINANCE_CLIENT_PLUGINS}/container/root/my_plugin/my_plugin.cc`
* NOTE: `root` pluggable auto-(un)loading requires a parent directory as the callable namespace (and more)
- See docs for details: `dfi dev-tools/${USER}:default doxygen gen`
#### Plugins: Bitcoin
`dfi`'s bitcoin plugin is a two-part client/container set of plugins that gives you direct access to bitcoin's libbitcoinkernel (and related headers/symbols).
The following demo assumes that you'll be using a fresh setup and that you've at least satisfied the required dependencies in [Installation](How-do-I-get-started.md#installation) (Docker Engine/Compose/Buildx, Bash, Git).
If you're a first-time user and/or developer who simply wants a quickstart, run the following before proceeding:
```bash
git clone --depth=1 https://gitea.com/EvergreenCrypto/docker-finance docker-finance/repo
./docker-finance/repo/client/install.bash && source ~/.bashrc
dfi archlinux/${USER}:default gen all=all profile=testprofile/testuser confirm=no dev=on
```
##### Plugins: Bitcoin: Client
Here, we prepare client-side dependencies and build everything needed for the container-side plugin:
<video src="../assets/examples/webm/client-plugins-bitcoin.webm" controls></video>
Shell 1:
```bash
# NOTE: editing will only be required once (unless you `gen type=build` in the future)
dfi archlinux/${USER}:default edit type=build
dfi archlinux/${USER}:default build type=default
```
Shell 2:
```bash
dfi archlinux/${USER}:default up
```
Shell 1:
```bash
dfi archlinux/${USER}:default plugins repo/bitcoin.bash get
dfi archlinux/${USER}:default plugins repo/bitcoin.bash build
```
##### Plugins: Bitcoin: Container
Here, we see the multiple ways the container-side plugin can be loaded and also test its functionality:
<video src="../assets/examples/webm/container-root-plugins-bitcoin_cli.webm" controls></video>
Shell 2 (or open a new shell into container, as seen in the demo):
```bash
dfi testprofile/testuser root
```
Within `root` interpreter:
```cpp
// NOTE:
// - The demo shows `btck` tab completion (which can't be put here)
// - semicolons are not needed, since the following is executed per line
GetRandHash()
dfi::plugin::load("repo/bitcoin/bitcoin.cc")
GetRandHash()
dfi::macro::load("repo/test/unit.C", "Random*")
.quit
```
Shell 2:
```bash
BENCHMARK_FILTER="^Random" dfi testprofile/testuser root plugins/repo/bitcoin/bitcoin.cc 'dfi::macro::load(\"repo/test/benchmark.C\"); dfi::common::exit(0);'
dfi testprofile/testuser root plugins/repo/bitcoin/bitcoin.cc 'dfi::macro::load(\"repo/web/server.C\")'
```
##### Plugins: Bitcoin: Web browser
Here, we see a real-world visualization of what the container-side plugin can produce. In this example, with the plugin previously loaded (as seen above), we sample bitcoin's RNG:
<video src="../assets/examples/webm/container-root-plugins-bitcoin_web.webm" controls></video>
- Open browser to `http://127.0.0.1:8080`
* Default port can be changed with client-side command: `dfi archlinux/${USER}:default edit type=env`
- Click `rng_sample` -> Enter sample amount
- Click `reload`
### Caveats & Oddities
#### Caveats & Oddities: Flow
Your `flow` directory will contain a symlink called `src` which links to code that processes your data. **Do not delete this symlink**.
##### Caveats & Oddities: Flow: Prices
Before you try to infer market prices, be sure to fetch prices *before* you do your first import (or first import of the year). If you do not fetch, the prices journal will **not** be included within the import and, if you have a previous year of prices, **you will unwittingly infer against that previous year instead of your expected year!**
##### Caveats & Oddities: Flow: Accounts: Trezor
In the "Trezor Suite" app, change your wallet name to your subaccount(s). For example, to delineate between your Trezor One from several Trezor T devices, and to delineate between their separate wallets *within* every device, follow these steps:
Example, using your #2 Trezor T device and one of its BTC "storage" wallets:
1. Change wallet name in app to `t-2:storage-1` as it's your Trezor T device #2, 1st bitcoin wallet named `storage-1` (versus your 2nd bitcoin wallet named `storage-2`, etc.)
2. Export the CSV file to the appropriate directory. It will be in the format of `t_2_storage_1_20230629T230013.csv` (timestamp will be different)
3. Rename the file to `t-2:storage-1_BTC.csv` (be sure to append the currency ticker to the file. So, `_BTC` if bitcoin or `_LTC` if litecoin, etc).
> Note: see Trezor `mockup` data within the docker-finance repository for a working example.
docker-finance relies on `Amount unit` within the file for the actual symbol/currency so, this file naming convention serves at least two purposes:
1. This allows you to maintain device continuity by reusing wallet names for different currencies.
2. This allows you to export, in the future, to the correct file from the associated hardware wallet because each hardware wallet exports its own unique CSV.
##### Caveats & Oddities: Flow: Taxes
- If you have a wallets designated for `SPEND`ing/`GIFT`ing or `INCOME`, you can use custom rules to mark all outgoing/incoming transactions as such (ex., using tags `taxed_as:SPEND`/`taxed_as:GIFT`/`taxed_as:INCOME`/etc.). See implementation for details.
- **WARNING**: *all* `GIFTIN` cost-basis must be manually entered from the corresponding `GIFT` results/calculations (as gifted from another).
- For blockchain-related transactions, you can easily add cost-basis of a gift received (`GIFTIN`) by TXID in your custom rules
- Example: despite Electrum providing `fiat_value`, you'll need to manually enter in your custom rules the correct `GIFT` value (if divergent)
### FAQ
> Q. Are there two versions of the `dfi` script? Or a single script that gathers all functionality under various subcommands?
The `dfi` command has different functionality depending on its [context](#mostly-unified-cli). The client-side (host) command only operates client-side (but can also operate *on* a container) while the container-side `dfi` command only operates within the container.
This pairing is part of that "translucent" quality of the system where the "same" command behaves differently, depending on the context. It allows for a fluid transition between the dual worlds of the docker-finance system. A good visual example of how the two can work together can be seen in this [demo](#plugins-bitcoin).
> Q. Is this dual world of docker images essential to docker-finance? Could it work with just 1 docker image, or 0 images?
The "dual world" aspect only describes the client (host) and container system. Currently, there are 2 `finance` images ([only 1 is actively maintained (`archlinux`)](https://gitea.evergreencrypto.co/EvergreenCrypto/docker-finance/issues/239)) and 1 optional `dev-tools` (developer) image.
The client (host) is *not* an image: these terms only describe the supported host (currently Linux). You can run `dfi` on an Ubuntu host, a Debian host, an Arch Linux host etc. When you see `archlinux` in the documentation, that only describes what the `finance` image is based (*not* what the client (host) is).
As for 0 image, well that would defeat the purpose! :) I think the world ran fine before Docker but the benefits of Docker, at least in this case, far outweigh the cost of the overhead.
> Q. What's the essential need for docker here? Is it just a packaging/distribution choice?
`dfi` is an all-in-one accounting system; a privacy-aware, security-aware, modern-finance-aware accounting system; a system of operating within an operating system; a financial framework for power-users.
But why does `dfi` exist? Perhaps it's easier to ask the following:
1. How can I manage my own accounting locally without the need for a 3rd party but with the features provided by 3rd parties?
2. How can I use [hledger](https://hledger.org/) to account for transactions fetched by [ccxt](https://github.com/ccxt/ccxt/) then generate privacy-aware tax reports for a [cost-basis calculator](https://bitcoin.tax/) while performing local analysis using [root](https://root.cern/) and simultaneously track the [time](https://taskwarrior.org/) it takes to complete all of this then bill a client with a [custom billing invoice/email plugin](#plugins); all with a single command issued from the commandline to a single locally administered system?
3. How can the previous question be applied to multiple client profiles who's accounting must be technologically, mathematically, financially and ethically separated; while also enjoying the benefits of the all-in-one system?
4. How can the previous question's execution be generic enough and templated enough to work for all other `dfi` project end-users without every end-user needing to maintain their own dependencies, their own locations for all relevant data or even their own functional needs (e.g., "I don't need `root` but I need to `fetch` my data and use `hledger`") but also simultaneously allow them the flexibility to do so (should they choose)?
5. How does one provide a baseline accounting system and [workflow](How-do-I-use-it.md#user-content-flow-layout) for users to work with that also allows them the freedom and flexibility of Docker, while also allowing the API contracts required by the aforementioned dependencies to communicate with each other across multiple languages and environments, in order to produce useful data aggregation, bookkeeping and analysis?
6. How does one stay in line with the spirit of file-based accounting for all of the aforementioned while also avoiding a traditional DBMS and the language/UI overhead that's usually involved?
7. How does one manage the dependencies and upstream requirements of these many different languages and environments without putting that burden on an end-user other than them running a single `update` command?
8. How does one do all of this without compiling any code? Or rather; no compiling outside of a JIT compiler or interpreter outside of a commonly used native Linux shell?
9. How are all of the aforementioned dependencies contained within a single system, without needing to manage packages across multiple Linux distributions?
10. How does one do all of this with only 2 runtime dependencies for the end-user? Meaning, end-users only need to concern themselves with having `bash` and `docker` installed on their client (host) in order to run?
I believe `dfi` answers all of these questions; satifies all of these requirements and more.
> Q. Why not just provide some Dockerfiles and compose files and be done with it?
The docker-finance system is not that simple, but that simplicity is offered to end-users. You can work with your own custom Dockerfile and docker-compose.yml while still working within the system: simply run the client-side `dfi` command with `edit type=build,compose`.
This style of extensibility allows you to add or remove what you need without needing to submit a pull request (or if your need is entirely personal/unique and shouldn't be applied to other `dfi` end-users).
> Q. Can I use my own custom container-side `dfi` implementation?
Yes! Simply edit your client-side (host) environment with `edit type=env` and update `DOCKER_FINANCE_CONTAINER_CMD` to point to your own command.
[//]: # (vim: sw=2 sts=2 si ai et)
View Git Blame Copy Permalink