This patch changes the CallbackManager to use WeakMethods (weakrefs) to
break the ref cycle and allow the GC to clean up the wallet objects.
unregister_callbacks() will also get called automatically, from
EventListener.__del__, to clean up the CallbackManager.
I also added a few unit tests for this.
fixes https://github.com/spesmilo/electrum/issues/10427
-----
original problem:
In many subclasses of `EventListener`, such as `Abstract_Wallet`, `LNWatcher`,
`LNPeerManager`, we call `register_callbacks()` in `__init__`.
`unregister_callbacks()` is usually called in the `stop()` method.
Example - consider the wallet object:
- `Abstract_Wallet.__init__()` calls `register_callbacks()`
- there is a `start_network()` method
- there is a `stop()` method, which calls `unregister_callbacks()`
- typically the wallet API user only calls `stop()` if they also called
`start_network()`.
This means the callbacks are often left registered, leading to the wallet
objects not getting GC-ed. The GC won't clean them up as
`util.callback_mgr.callbacks` stores strong refs to instance methods
of `Abstract_Wallet`, hence strong refs to the `Abstract_Wallet` objects.
An annoying example is `daemon.check_password_for_directory`, which
potentially creates wallet objects for all wallet files in the datadir.
It simply constructs the wallets, does not call `start_network()` and
neither does it call `stop()`.
- to gracefully take duplicate calls of register_callbacks(): should be idempotent now
- as a side-effect, the order of the callbacks is changed and not guaranteed
- not like anyone should have been relying on it before though
ec65c53 replaces the usage of `PeerInTest` with `Peer` in
test_lnpeer.py.
PeerInTests sets `Peer.DELAY_INC_MSG_PROCESSING_SLEEP` to 0 so all
incoming messages get processed immediately. Because `Peer` instead of
`TestInPeer` was used the delay caused `test_reestablish_with_old_state`
to fail regularly because bob receives the old channel state and kills
the OldTaskGroup of the unittest with GracefulDisconnect before Alice
processed the answer of Bob and is still in ChannelState.REESTABLISHING.
```
FAILED tests/test_lnpeer.py::TestPeerDirect::test_reestablish_with_old_state - AssertionError: <PeerState.REESTABLISHING: 1> != <PeerState.BAD: 3>
```
- LNWallet no longer "is-an" LNWorker, instead LNWallet "has-an" LNWorker
- the motivation is to make the unit tests nicer, and allow writing unit tests for more things
- I hope this makes it possible to e.g. test lnsweep in the unit tests
- some stuff we would previously have to write a regtest for, maybe we can write a unit test for, now
- in unit tests, MockLNWallet now
- inherits LNWallet
- the Wallet is no longer being mocked
Lightning addresses with 'lightning:' do occur in the wild and make
sense (how else would e.g. the smartphone know to open a lightning
wallet instead of the e-mail client). So we should allow this.
When creating a new wallet in a Electrum instance with existing wallets
this change forces the user to reuse a password of any existing wallet
if `SimpleConfig.WALLET_USE_SINGLE_PASSWORD` is True.
This prevents the amount of different passwords from increasing and
guides the user towards a single wallet password (the intended default).
The new test
TestPeerForwarding::test_dont_settle_htlcs_receiver_and_forwarder covers
the receiver as well as the forwarder. So this unittest becomes obsolete
as it only tests the receiver.
- whitespaces are safe to remove from strings, and is convenient if we do this for users
- bytes-like inputs should be left alone: individual bytes that look like whitespaces can appear in them anywhere
- even stripping the leading/trailing whitespaces is not safe to do: the first byte of the nVersion or the last byte of the nLocktime might look like whitespace too!
- instead, leading/trailing whitespaces can be stripped closer to where they are input, e.g. in the GUI
- e.g. ".txn" files that we ourselves create contain a complete tx as a hex string, with a trailing final newline in the file
- instead of reading that as bytes, we can read it as text
- ".psbt" files OTOH are binary
Removes all whitespace characters from a raw transaction string.
This is useful for example when loading raw transactions from text input
as it happens that there are some newline characters in the text.
I noticed this when copying-pasting from a timelock recovery pdf.
We tried to delete incoming channels that didn't get funded after
lnutil.CHANNEL_OPENING_TIMEOUT, however an assert prevented this:
```
3.63 | E | lnwatcher.LNWatcher.[default_wallet-LNW] | Exception in check_onchain_situation: AssertionError()
Traceback (most recent call last):
File "/home/user/code/electrum-fork/electrum/util.py", line 1233, in wrapper
return await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/code/electrum-fork/electrum/lnwatcher.py", line 117, in check_onchain_situation
await self.update_channel_state(
...<5 lines>...
keep_watching=keep_watching)
File "/home/user/code/electrum-fork/electrum/lnwatcher.py", line 135, in update_channel_state
chan.update_onchain_state(
~~~~~~~~~~~~~~~~~~~~~~~~~^
funding_txid=funding_txid,
^^^^^^^^^^^^^^^^^^^^^^^^^^
...<2 lines>...
closing_height=closing_height,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
keep_watching=keep_watching)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/code/electrum-fork/electrum/lnchannel.py", line 341, in update_onchain_state
self.update_unfunded_state()
~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/user/code/electrum-fork/electrum/lnchannel.py", line 382, in update_unfunded_state
self.lnworker.remove_channel(self.channel_id)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/user/code/electrum-fork/electrum/lnworker.py", line 3244, in remove_channel
assert chan.can_be_deleted()
~~~~~~~~~~~~~~~~~~~^^
AssertionError
```
Adds the invoice features to the `PaymentInfo` class so we can check if
the sender respects our requested features (e.g. if they tried to send
mpp if we requested no mpp).
Allows storing two different payment info of the same payment hash by
including the direction into the db key.
We create and store PaymentInfo for sending attempts and for requests (receiving),
if we try to pay ourself (e.g. through a channel rebalance) the checks
in `save_payment_info` would prevent this and throw an exception.
By storing the PaymentInfos of outgoing and incoming payments separately in
the db this collision is avoided and it makes it easier to reason about
which PaymentInfo belongs where.
I was unable to do a "Max" amount submarine swap because the
`fee_estimate` method used by `LNWallet.num_sats_can_send()` uses a
hardcoded `fee_proportional_millionths` to estimate the fee for the
lightning payment.
When the actual fee determined later is higher
than the estimated fee the payment fails as the channel is unable to add
the htlc sum including the real fees as the amount exceeds the balance of
the channel.
Using the fees the maximum fees user has configured and estimate the
potential fee as inverse of PaymentFeeBudget is more
reliable/conservative as we definitely aren't going to pay more fees
than this amount.
Splits `LNWallet.dont_settle_htlcs` into `LNWallet.dont_settle_htlcs`
and `LNWallet.dont_expire_htlcs`.
Registering a payment hash in dont_settle_htlcs will prevent it from
getting fulfilled if we have the preimage stored. The preimage will not
be released before the the payment hash gets removed from
dont_settle_htlcs. Htlcs can still get expired as usual or failed if no
preimage is known.
This is only used by Just-in-time channel openings.
Registering a payment hash in dont_expire_htlcs allows to overwrite the
minimum final cltv delta value after which htlcs would usually get
expired. This allows to delay expiry of htlcs or, if the value in the
dont_settle_htlcs dict is None, completely prevent expiry and let the
htlc get expired onchain.
Splitting this up in two different dicts makes it more explicit and
easier to reason about what they are actually doing.
Please enter the commit message for your changes. Lines starting
Adds test_payment_bundle_with_hold_invoice to simulate the use of a
payment bundle in which one invoice of the bundle needs to trigger a hold invoice
callback (similar to submarine swaps).
Also modifies the test helper _test_simple_payment() to compare the
results of all payment attempts instead of just returning after the
first (of multiple) payments raises its result causing the test to miss
if all payments were successful or not.
Add test `test_hold_invoice_set_doesnt_get_expired` to test_lnpeer to
ensure a mpp set on which a hold invoice callback doesn't get expired
automatically if the cltv_abs falls below MIN_FINAL_CLTV_DELTA_ACCEPTED
as these sets should only get failed if the htlcs are safe to fail by
the target of the hold invoice callback (e.g. swap got refunded
successfully).
