A new config API is introduced, and ~all of the codebase is adapted to it.
The old API is kept but mainly only for dynamic usage where its extra flexibility is needed.
Using examples, the old config API looked this:
```
>>> config.get("request_expiry", 86400)
604800
>>> config.set_key("request_expiry", 86400)
>>>
```
The new config API instead:
```
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS
604800
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS = 86400
>>>
```
The old API operated on arbitrary string keys, the new one uses
a static ~enum-like list of variables.
With the new API:
- there is a single centralised list of config variables, as opposed to
these being scattered all over
- no more duplication of default values (in the getters)
- there is now some (minimal for now) type-validation/conversion for
the config values
closes https://github.com/spesmilo/electrum/pull/5640
closes https://github.com/spesmilo/electrum/pull/5649
Note: there is yet a third API added here, for certain niche/abstract use-cases,
where we need a reference to the config variable itself.
It should only be used when needed:
```
>>> var = config.cv.WALLET_PAYREQ_EXPIRY_SECONDS
>>> var
<ConfigVarWithConfig key='request_expiry'>
>>> var.get()
604800
>>> var.set(3600)
>>> var.get_default_value()
86400
>>> var.is_set()
True
>>> var.is_modifiable()
True
```
Previously if a local tx got broadcast, it was still displayed as local
in the history until it got mined (or some other action forced a full refresh).
- they are now distinguished from local by the status text "in %d blocks"
- this status text needs updating occasionally: on new blocks,
and some lnwatcher interactions, hence new event: "adb_set_future_tx"
- wallet.add_input_info() previously had a fallback to download parent
prev txs from the network (after a lookup in wallet.db failed).
wallet.add_input_info() is not async, so the network request cannot
be done cleanly there and was really just a hack.
- tx.add_info_from_wallet() calls wallet.add_input_info() on each txin,
in which case these network requests were done sequentially, not concurrently
- the network part of wallet.add_input_info() is now split out into new method:
txin.add_info_from_network()
- in addition to tx.add_info_from_wallet(), there is now also tx.add_info_from_network()
- callers of old tx.add_info_from_wallet() should now called either
- tx.add_info_from_wallet(), then tx.add_info_from_network(), preferably in that order
- tx.add_info_from_wallet() alone is sufficient if the tx is complete,
or typically when not in a signing context
- callers of wallet.bump_fee and wallet.dscancel are now expected to have already
called tx.add_info_from_network(), as it cannot be done in a non-async context
(but for the common case of all-inputs-are-ismine, bump_fee/dscancel should work regardless)
- PartialTxInput.utxo was moved to the baseclass, TxInput.utxo
fixes https://github.com/spesmilo/electrum/issues/8240#8240 was triggering an AssertionError in wallet.get_invoice_status,
as code there was assuming conf >= 0. To trigger, force-close
a LN channel, and while the sweep is waiting on the CSV, try to
make a payment in the Send tab to the ismine change address used
for the sweep in the future_tx. (order of events can also be reversed)
- add a new event, 'adb_removed_tx'
- new wallet method: get_tx_parents
- number of parents is shown in coins tab
- detailed list of parents is shown in dialog
ShortIDs were originally designed for lightning channels, and are now
understood by some block explorers.
This allows to remove one column in the UTXO tab (height is redundant).
In the transaction dialog, the space saving ensures that all inputs fit
into one line (it was not the case previously with p2wsh addresses).
For clarity and consistency, the ShortID is displayed for both inputs
and outputs in the transaction dialog.
- AddressSynchronizer no longer has its own state re up_to_date,
it defers to Synchronizer/Verifier instead
- Synchronizer is now tracking address sync states throughout their lifecycle:
and Synchronizer.is_up_to_date() checks all states
- Synchronizer.add_queue (internal) is removed as it was redundant
- should fix wallet.is_up_to_date flickering during sync due to race
related:
dc6c4814061c20a29a22
This reverts commit dc6c481406 as it introduced its own issue:
while add_address was running on one thread, synchronizer._reset could be running on another,
and by the time the "enqueue" coro would run, it would use a new add_queue and
addr would not be in requested_addrs anymore...
```
I/w | wallet.Standard_Wallet.[test_segwit_2] | starting taskgroup.
I | lnworker.LNWallet.[test_segwit_2] | starting taskgroup.
E/i | interface.[testnet.qtornado.com:51002] | Exception in run: KeyError('tb1q3wmgf8n5eettnj50pzgnfrrpdpjmwn37x7nzsc5780kk4je9v4hspym8mu')
Traceback (most recent call last):
File ".../electrum/electrum/util.py", line 1243, in wrapper
return await func(*args, **kwargs)
File ".../electrum/electrum/interface.py", line 506, in wrapper_func
return await func(self, *args, **kwargs)
File ".../electrum/electrum/interface.py", line 529, in run
await self.open_session(ssl_context)
File ".../electrum/electrum/interface.py", line 679, in open_session
async with self.taskgroup as group:
File ".../aiorpcX/aiorpcx/curio.py", line 304, in __aexit__
await self.join()
File ".../electrum/electrum/util.py", line 1339, in join
task.result()
File ".../electrum/electrum/synchronizer.py", line 80, in _run_tasks
async with taskgroup as group:
File ".../aiorpcX/aiorpcx/curio.py", line 304, in __aexit__
await self.join()
File ".../electrum/electrum/util.py", line 1339, in join
task.result()
File ".../electrum/electrum/synchronizer.py", line 127, in subscribe_to_address
self.requested_addrs.remove(addr)
KeyError: 'tb1q3wmgf8n5eettnj50pzgnfrrpdpjmwn37x7nzsc5780kk4je9v4hspym8mu'
```
AddressSynchronizer.add_address called synchronizer.add, which would only
schedule adding the addr to the Synchronizer in the next event loop iter.
If during that time, the synchronizer called adb.set_up_to_date(True),
the wallet would falsely believe and advertise itself as up_to_date
(as the wallet would see wallet.synchronize not creating new addresses,
and adb (via synchronizer) telling it is up_to_date).
Moments later, the synchronizer._add_address is finally executed and
up_to_date=False propagates out synchronizer->adb->wallet.
- revert the logic of do_breach_remedy to what it was
before 0ca3d66d15,
but now calling self.maybe_redeem unconditionally.
- replace mempool transactions only if the fee increases
- do not notify the GUI if a local tx is replaced
- delete labels when replacing
The wallet needs its own up_to_date logic:
- the adb being up_to_date means all its addresses are synced
- but an HD wallet might decide to roll the gap limit and generate new addresses
- the adb does not know about this...
- the wallet should be considered *not* up_to_date
- relatedly, it is now the wallet that decides to reset the network request counters
- note that wallet.main() was never cleaned up previously.
- now wallet gets its own taskgroup, which is cleaned up in wallet.stop.
Follow-up to adb refactor: 121d8732f1
- separate AddressSynchronizer from Wallet and LNWatcher
- the AddressSynchronizer class is referred to as 'adb' (address database)
- Use callbacks to replace overloaded methods
It does not make sense to count change outputs in our unconfirmed balance,
because our balance will not be negatively affected if the transaction does
not get confirmed.
It is also incorrect to add signed values of get_addr_balance in order to compute
the balance over a domain. For example, this leads to incoming and outgoing
transactions cancelling out in our total unconfirmed balance.
This commit looks at the coins that are spent by a transaction. If those
coins belong to us and are confirmed, we do not count the transaction outputs
in our unconfirmed balance.
As a result, get_balance always returns positive values for unconfirmed balance.
see https://github.com/spesmilo/electrum/issues/5376
crash report for electrum 4.2.1:
```
Traceback (most recent call last):
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/python-installs/Electrum/kivy/base.py", line 347, in mainloop
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/python-installs/Electrum/kivy/base.py", line 391, in idle
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/python-installs/Electrum/kivy/base.py", line 342, in dispatch_input
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/python-installs/Electrum/kivy/base.py", line 308, in post_dispatch_input
File "kivy/_event.pyx", line 724, in kivy._event.EventDispatcher.dispatch
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/python-installs/Electrum/kivy/uix/behaviors/button.py", line 179, in on_touch_up
File "kivy/_event.pyx", line 720, in kivy._event.EventDispatcher.dispatch
File "kivy/_event.pyx", line 1263, in kivy._event.EventObservers.dispatch
File "kivy/_event.pyx", line 1147, in kivy._event.EventObservers._dispatch
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/python-installs/Electrum/kivy/lang/builder.py", line 57, in custom_callback
File "<string>", line 42, in <module>
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/gui/kivy/uix/dialogs/wallets.py", line 70, in cb
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/gui/kivy/main_window.py", line 710, in load_wallet_by_name
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/gui/kivy/main_window.py", line 728, in on_open_wallet
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/gui/kivy/main_window.py", line 691, in on_wizard_success
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/wallet.py", line 3267, in __new__
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/wallet.py", line 3111, in __init__
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/wallet.py", line 2904, in __init__
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/wallet.py", line 288, in __init__
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/address_synchronizer.py", line 97, in __init__
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/wallet.py", line 402, in load_and_cleanup
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/address_synchronizer.py", line 106, in load_and_cleanup
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/util.py", line 439, in <lambda>
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/util.py", line 435, in do_profile
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/address_synchronizer.py", line 432, in load_local_history
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/address_synchronizer.py", line 533, in _add_tx_to_local_history
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/address_synchronizer.py", line 548, in _mark_address_history_changed
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/other_builds/python3/arm64-v8a__ndk_target_21/python3/Lib/asyncio/locks.py", line 260, in __init__
File "/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/other_builds/python3/arm64-v8a__ndk_target_21/python3/Lib/asyncio/events.py", line 639, in get_event_loop
RuntimeError: There is no current event loop in thread 'GUI'.
```
After some consideration I am fairly certain there is no need to take
wallet.lock in `is_up_to_date()`. Any caller that might want some kind
of guarantees re the value returned by is_up_to_date() would need to
enforce them itself by e.g. taking wallet.lock around its critical code
block. That is, even if is_up_to_date() itself takes the lock, between
the call returning and the caller reading the value there could still
have been a race.
Also, the GUI was directly accessing the field already.
aiorpcx 0.20 changed the behaviour/API of TaskGroups.
When used as a context manager, TaskGroups no longer propagate
exceptions raised by their tasks. Instead, the calling code has
to explicitly check the results of tasks and decide whether to re-raise
any exceptions.
This is a significant change, and so this commit introduces "OldTaskGroup",
which should behave as the TaskGroup class of old aiorpcx. All existing
usages of TaskGroup are replaced with OldTaskGroup.
closes https://github.com/spesmilo/electrum/issues/7446
