dfi-docs/markdown/How-do-I-use-it.md

• How do I use it?
  • Mostly-Unified CLI
  • Flow Layout
  • Plugins
  • Caveats & Oddities
  • FAQ

How do I use it?

Mostly-Unified CLI

You'll only need the single alias dfi when using docker-finance on your client (host) and/or within your container.

However, this alias can be broken down into the following:

1. The client script which handles the client-side (host) system: docker.bash (alias dfi)
2. The container script which handles the container-side system: finance.bash (alias dfi)

These two scripts can be rationalized by the following format:

<script> <super/sub> <command> [args]

For example, these Screenshots describe a setup with mockup data where the client (host) user named business, along with container $DOCKER_FINANCE_USER named business, engage in client/container activity. The container profile named testprofile and its subprofile named testuser can be described as <super/sub> portion of the format (testprofile/testuser).

It should be noted that, for your convenience:

• command arguments [args] can be arranged in any order
• commandline completion is available for all dfi <super/sub> command [args] (save your fingers and tab away!)
  • For client (host), an image must first be built for completion to be available

---

Client (Host) Command Format

The client (host) command format consists of:

docker.bash <platform/username:tag> <command> [args]

Where:

• docker.bash is located in ${DOCKER_FINANCE_CLIENT_REPO}/client
• <platform/username:tag>
  • platform is the image platform (archlinux, ubuntu)
  • username is the client (host) username with read/write permissions to the container (see Configuration Files)
  • tag is a custom tag you can use to delineate any number of images that you wish you create (latest, dev, etc.)
• <command> is the command to pass to docker.bash
• [args] are the (optional) arguments to pass to <command>

If the Installation was successful, docker.bash is aliased to docker-finance and dfi (either can be used).

For a complete list of commands and usage help (<platform/user:tag> not required):

dfi help

To view the help usage of a specific command, for example; the edit command (<platform/user:tag> is required):

dfi archlinux/${USER}:default edit help

---

Container Command Format

The container command format consists of:

finance.bash <profile/subprofile> <command> [args]

Where:

• finance.bash is located in ${DOCKER_FINANCE_CLIENT_REPO}/container
• <profile/subprofile>
  • profile is the profile, as defined during Environment Generation
  • subprofile is the subprofile (user), as defined during Environment Generation
• <command> is the command to pass to finance.bash
• [args] are the (optional) arguments to pass to <command>

By default, finance.bash is aliased to finance and dfi (either can be used).

For a complete list of commands (<profile/subprofile> not required):

dfi help

To view the help usage of a specific command, for example; the fetch command (<profile/subprofile> is required):

Assuming <profile/subprofile> is testprofile/testuser:

dfi testprofile/testuser fetch help

Or, use a subprofile alias, as described in Subscript:

testuser_fetch help

Flow Layout

A primary read through of hledger and hledger-flow documentation should bring you up to speed on most of the essentials.

As for docker-finance specifics, you can create a test profile during Environment Generation to see what your flow's layout should look like.

Note: be sure to select 'y' when asked if this will be a development profile, and then go on to create account(s).

Once inside the container, assuming you created a profile named testprofile and subprofile named testuser, issue the following commands:

finance testprofile/testuser fetch all=price api=mobula year=all && finance testprofile/testuser import year=2018

Note: for this test profile with developer mockups, you MUST import from 2018 as there are accounts that begin from that year

After experimenting with a test profile, you can re-run gen again to create a regular profile.

Profiles

All profiles/subprofiles are installed into the parent directory ${DOCKER_FINANCE_CONTAINER_FLOW}/profiles.

Peeking inside ${DOCKER_FINANCE_CONTAINER_FLOW}/profiles/profile/subprofile, you'll see the following:

• all-years.journal and directives.journal

  • These top-level journals are generated by hledger-flow. Ignore these and use the container edit command for all journal editing.

• conf.d

  • Location of all docker-finance configuration files (see edit help for details).

• import

  • Location of all CSV data and real journals. This is where you'll place CSV files and custom account/subaccount changes (see edit help for details).

• prices

  • Location of all market price data, by year, as acquired by fetch price (see fetch help for details).

• reports

  • Location of all generated reports, by year, as generated by reports (see reports help for details).

• taxes

  • Location of all generated taxes, by year, as generated by taxes (see taxes help for details).

Note:

• For manual CSV downloads, place you CSV file into your ${DOCKER_FINANCE_CONTAINER_FLOW}/profiles/profile/subprofile/import/subprofile/account/subaccount/1-in/year directory (replacing year with the year of data that the file/data represents). See import help for details.
• When you want to edit custom settings for an account or a subaccount, use the container edit command. See container's edit help for details.

Times

All times related files will reside in ${DOCKER_FINANCE_CONTAINER_FLOW}/times (this includes the timewarrior database).

See the container times help command for details.

Plugins

Plugins (pluggables) allow you to leverage dfi client/container APIs, libraries and environments to meet your unique needs.

Checkout this two-part client/container set of Bitcoin plugins to see how all APIs/libraries/environments can work together in unison.

For more information and other examples, see some of the various existing plugins (pluggables) and respective help usage, i.e.;

• Client-side (host): dfi <platform/user:tag> plugins help
• Container-side: dfi <profile/subprofile> plugins help
• Within root interpreter: dfi::help()

Plugins: Layout

Plugins are categorical:

1. Repository (repo) plugins

   • These plugins remain within the repository and will require a pull request for any changes to be made to them.

2. End-user (custom) plugins

   • These plugins remain on your client (host) and are bind-mounted to your container; allowing you to drop-in any code that you write while keeping them local and private.

Within these categories are subcategories where plugins exist either client-side (host) or container-side; meaning, they rely upon (or operate within) their respective client/container APIs/libraries/environments:

1. client | Tends to operate only client-side (host) but can also utilize a container
   • docker | Operates only from the bash shell
     • Called client-side with dfi <platform/user:tag> plugins
     • Can be any language so long as:
       • The file is executable by the shell
       • The plugin reads the shell environment
       • The plugin initializes the respective library (lib_docker)
2. container | Operates only within the container
   • finance | Operates only from the bash shell
     • Called container-side with dfi <profile/subprofile> plugins
     • Can be any language so long as:
       • The file is executable by the shell
       • The plugin reads the shell environment
       • The plugin initializes the respective library (lib_finance)
   • root | Operates only within the root interpreter
     • Called container-side by either two different ways:
       • Within a running dfi <profile/subprofile> root instance:
         • dfi::plugin::load("repo/a_repo_plugin/plugin.cc")
         • dfi::plugin::load("custom/my_custom_plugin/plugin.cc")
       • With the dfi <profile/subprofile> root plugins command
         • Use tab completion to see available plugins (pluggables)
     • Can only be languages supported by the interpreter (C/C++)

To mirror these categories, a client-side custom plugin directory layout is generated upon dfi <platform/user:tag> gen. This layout consists of:

1. ${DOCKER_FINANCE_CLIENT_PLUGINS}/client/docker
   • This path remains client-side only (not bind-mounted)
   • This layout mirrors repo plugins client/plugins
2. ${DOCKER_FINANCE_CLIENT_PLUGINS}/container/{finance,root}
   • The container directory is bind-mounted to DOCKER_FINANCE_CONTAINER_PLUGINS
   • This layout mirrors repo plugins container/plugins

WARNING: don't change the above expected layout! However, you can add subdirectories, e.g.;

• ${DOCKER_FINANCE_CLIENT_PLUGINS}/client/docker/my_docker_plugins/plugin.bash
• ${DOCKER_FINANCE_CLIENT_PLUGINS}/container/finance/my_finance_plugins/plugin.bash
• ${DOCKER_FINANCE_CLIENT_PLUGINS}/container/root/my_plugin/my_plugin.cc
  • NOTE: root pluggable auto-(un)loading requires a parent directory as the callable namespace (and more)
    • See docs for details: dfi dev-tools/${USER}:default doxygen gen

Plugins: Bitcoin

dfi's bitcoin plugin is a two-part client/container set of plugins that gives you direct access to bitcoin's libbitcoinkernel (and related headers/symbols).

The following demo assumes that you'll be using a fresh setup and that you've at least satisfied the required dependencies in Installation (Docker Engine/Compose/Buildx, Bash, Git).

If you're a first-time user and/or developer who simply wants a quickstart, run the following before proceeding:

git clone --depth=1 https://gitea.com/EvergreenCrypto/docker-finance docker-finance/repo
./docker-finance/repo/client/install.bash && source ~/.bashrc
dfi archlinux/${USER}:default gen all=all profile=testprofile/testuser confirm=no dev=on

Plugins: Bitcoin: Client

Here, we prepare client-side dependencies and build everything needed for the container-side plugin:

Shell 1:

# NOTE: editing will only be required once (unless you `gen type=build` in the future)
dfi archlinux/${USER}:default edit type=build
dfi archlinux/${USER}:default build type=default

Shell 2:

dfi archlinux/${USER}:default up

Shell 1:

dfi archlinux/${USER}:default plugins repo/bitcoin.bash get
dfi archlinux/${USER}:default plugins repo/bitcoin.bash build

Plugins: Bitcoin: Container

Here, we see the multiple ways the container-side plugin can be loaded and also test its functionality:

Shell 2 (or open a new shell into container, as seen in the demo):

dfi testprofile/testuser root

Within root interpreter:

// NOTE:
//   - The demo shows `btck` tab completion (which can't be put here)
//   - semicolons are not needed, since the following is executed per line
GetRandHash()
dfi::plugin::load("repo/bitcoin/bitcoin.cc")
GetRandHash()
dfi::macro::load("repo/test/unit.C", "Random*")
.quit

Shell 2:

BENCHMARK_FILTER="^Random" dfi testprofile/testuser root plugins/repo/bitcoin/bitcoin.cc 'dfi::macro::load(\"repo/test/benchmark.C\"); dfi::common::exit(0);'
dfi testprofile/testuser root plugins/repo/bitcoin/bitcoin.cc 'dfi::macro::load(\"repo/web/server.C\")'

Plugins: Bitcoin: Web browser

Here, we see a real-world visualization of what the container-side plugin can produce. In this example, with the plugin previously loaded (as seen above), we sample bitcoin's RNG:

• Open browser to http://127.0.0.1:8080
  • Default port can be changed with client-side command: dfi archlinux/${USER}:default edit type=env
• Click rng_sample -> Enter sample amount
• Click reload

Caveats & Oddities

Caveats & Oddities: Flow

Your flow directory will contain a symlink called src which links to code that processes your data. Do not delete this symlink.

Caveats & Oddities: Flow: Prices

Before you try to infer market prices, be sure to fetch prices before you do your first import (or first import of the year). If you do not fetch, the prices journal will not be included within the import and, if you have a previous year of prices, you will unwittingly infer against that previous year instead of your expected year!

Caveats & Oddities: Flow: Accounts: Trezor

In the "Trezor Suite" app, change your wallet name to your subaccount(s). For example, to delineate between your Trezor One from several Trezor T devices, and to delineate between their separate wallets within every device, follow these steps:

Example, using your #2 Trezor T device and one of its BTC "storage" wallets:

1. Change wallet name in app to t-2:storage-1 as it's your Trezor T device #2, 1st bitcoin wallet named storage-1 (versus your 2nd bitcoin wallet named storage-2, etc.)
2. Export the CSV file to the appropriate directory. It will be in the format of t_2_storage_1_20230629T230013.csv (timestamp will be different)
3. Rename the file to t-2:storage-1_BTC.csv (be sure to append the currency ticker to the file. So, _BTC if bitcoin or _LTC if litecoin, etc).

Note: see Trezor mockup data within the docker-finance repository for a working example.

docker-finance relies on Amount unit within the file for the actual symbol/currency so, this file naming convention serves at least two purposes:

1. This allows you to maintain device continuity by reusing wallet names for different currencies.
2. This allows you to export, in the future, to the correct file from the associated hardware wallet because each hardware wallet exports its own unique CSV.

Caveats & Oddities: Flow: Taxes

• If you have a wallets designated for SPENDing/GIFTing or INCOME, you can use custom rules to mark all outgoing/incoming transactions as such (ex., using tags taxed_as:SPEND/taxed_as:GIFT/taxed_as:INCOME/etc.). See implementation for details.
• WARNING: all GIFTIN cost-basis must be manually entered from the corresponding GIFT results/calculations (as gifted from another).
  • For blockchain-related transactions, you can easily add cost-basis of a gift received (GIFTIN) by TXID in your custom rules
    • Example: despite Electrum providing fiat_value, you'll need to manually enter in your custom rules the correct GIFT value (if divergent)

FAQ

Q. Are there two versions of the dfi script? Or a single script that gathers all functionality under various subcommands?

The dfi command has different functionality depending on its context. The client-side (host) command only operates client-side (but can also operate on a container) while the container-side dfi command only operates within the container.

This pairing is part of that "translucent" quality of the system where the "same" command behaves differently, depending on the context. It allows for a fluid transition between the dual worlds of the docker-finance system. A good visual example of how the two can work together can be seen in this demo.

Q. Is this dual world of docker images essential to docker-finance? Could it work with just 1 docker image, or 0 images?

The "dual world" aspect only describes the client (host) and container system. Currently, there are 2 finance images (only 1 is actively maintained (archlinux)) and 1 optional dev-tools (developer) image.

The client (host) is not an image: these terms only describe the supported host (currently Linux). You can run dfi on an Ubuntu host, a Debian host, an Arch Linux host etc. When you see archlinux in the documentation, that only describes what the finance image is based (not what the client (host) is).

As for 0 image, well that would defeat the purpose! :) I think the world ran fine before Docker but the benefits of Docker, at least in this case, far outweigh the cost of the overhead.

Q. What's the essential need for docker here? Is it just a packaging/distribution choice?

dfi is an all-in-one accounting system; a privacy-aware, security-aware, modern-finance-aware accounting system; a system of operating within an operating system; a financial framework for power-users.

But why does dfi exist? Perhaps it's easier to ask the following:

1. How can I manage my own accounting locally without the need for a 3rd party but with the features provided by 3rd parties?
2. How can I use hledger to account for transactions fetched by ccxt then generate privacy-aware tax reports for a cost-basis calculator while performing local analysis using root and simultaneously track the time it takes to complete all of this then bill a client with a custom billing invoice/email plugin; all with a single command issued from the commandline to a single locally administered system?
3. How can the previous question be applied to multiple client profiles who's accounting must be technologically, mathematically, financially and ethically separated; while also enjoying the benefits of the all-in-one system?
4. How can the previous question's execution be generic enough and templated enough to work for all other dfi project end-users without every end-user needing to maintain their own dependencies, their own locations for all relevant data or even their own functional needs (e.g., "I don't need root but I need to fetch my data and use hledger") but also simultaneously allow them the flexibility to do so (should they choose)?
5. How does one provide a baseline accounting system and workflow for users to work with that also allows them the freedom and flexibility of Docker, while also allowing the API contracts required by the aforementioned dependencies to communicate with each other across multiple languages and environments, in order to produce useful data aggregation, bookkeeping and analysis?
6. How does one stay in line with the spirit of file-based accounting for all of the aforementioned while also avoiding a traditional DBMS and the language/UI overhead that's usually involved?
7. How does one manage the dependencies and upstream requirements of these many different languages and environments without putting that burden on an end-user other than them running a single update command?
8. How does one do all of this without compiling any code? Or rather; no compiling outside of a JIT compiler or interpreter outside of a commonly used native Linux shell?
9. How are all of the aforementioned dependencies contained within a single system, without needing to manage packages across multiple Linux distributions?
10. How does one do all of this with only 2 runtime dependencies for the end-user? Meaning, end-users only need to concern themselves with having bash and docker installed on their client (host) in order to run?

I believe dfi answers all of these questions; satifies all of these requirements and more.

Q. Why not just provide some Dockerfiles and compose files and be done with it?

The docker-finance system is not that simple, but that simplicity is offered to end-users. You can work with your own custom Dockerfile and docker-compose.yml while still working within the system: simply run the client-side dfi command with edit type=build,compose.

This style of extensibility allows you to add or remove what you need without needing to submit a pull request (or if your need is entirely personal/unique and shouldn't be applied to other dfi end-users).

Q. Can I use my own custom container-side dfi implementation?

Yes! Simply edit your client-side (host) environment with edit type=env and update DOCKER_FINANCE_CONTAINER_CMD to point to your own command.