1
0
Commit Graph

141 Commits

Author SHA1 Message Date
Oren
2fb0dd066f Timelock Recovery Extension (#9589)
* Timelock Recovery Extension

* Timelock Recovery Extension tests

* Use fee_policy instead of fee_est

Following 3f327eea07

* making tx with base_tx

Following ab14c3e138

* move plugin metadata from __init__.py to manifest.json

* removing json large indentation

* timelock recovery icon

* timelock recovery plugin: fix typos

* timelock recovery plugin: use menu instead of status bar.

The status bar should be used for displaying status. For example,
hardware wallet plugins use it because their connection status is
changing and needs to be displayed.

* timelock recovery plugin: ask for password only once

* timelock recovery plugin: ask whether to create cancellation tx in the initial window

* remove unnecessary code.

(calling run_hook from a plugin does not make sense)

* show alert and cancellation address at the end.

skip unnecessary dialog

* timelock recovery plugin: do not show transactions one by one.

Set the fee policy in the first dialog, and use the same fee
policy for all tx. We could add 3 sliders to this dialog, if
different fees are needed, but I think this really isn't
really necessary.

* simplify default_wallet for tests

All the lightning-related stuff is irrelevant for
this plugin.

Also use a different destination address
for the test recovery-plan (an address
that does not belong to the same wallet).

* Fee selection should be above fee calculation

also show fee calculation result with "fee: " label.

* hide Sign and Broadcast buttons during view

* recalculate cancellation transaction

The checkbox could be clicked after the fee rate
has been set. Calling update_transactions() may seem
inefficient, but it's the simplest way to avoid such edge-cases.

Also set the context's cancellation transaction to None when the
checkbox is unset.

* use context.cancellation_tx instead of checkbox value

context.cancellation_tx will be None iff the checkbox was unset

* hide cancellation address if not used

* init monospace font correctly

* timelock recovery plugin: add input info at signing time.

Fixes trezor exception: 'Missing previous tx'

* timelock recovery: remove unused parameters

* avoid saving the tx in a separate var

fixing the assertions

* avoid caching recovery & cancellation inputs

* timelock recovery: separate help window from agreement.

move agreement at the end of the flow, rephrase it

* do not cache alert_tx_outputs

* do not crash when not enough funds

not enough funds can happen
when multiple addresses are specified
in payto_e, with an amount larger
than the wallet has - so we set
the payto_e color to red.

It can also happen when the user
selects a really high fee, but this
is not common in a "recovery"
wallet with significant funds.

* If files not saved - ask before closing

* move the checkbox above the save buttons

people read the text from top to
bottom and may not understand
why the buttons are disabled

---------

Co-authored-by: f321x <f321x@tutamail.com>
Co-authored-by: ThomasV <thomasv@electrum.org>
2025-04-22 10:02:01 +02:00
f321x
8d79c58c5e Stop including all invoice r_tags in legacy trampoline onion
This change modifies create_trampoline_onion to only include as many
available r_tags as there is space left in the trampoline onion payload.

Previously we tried to include all passed invoice r_tags of legacy
trampoline payments into the payload which caused an user facing
exception and payment failure as the onion can only store a max of 400
bytes.
A single, single hop r_tag is around 52 bytes and the payload
without r_tags is already at ~280 bytes. So usually there is enough
space for 2 r_tags.
The implementation shuffles the r_tags on each call
so the payment will try different route hints on the attempts (fee level
increase or user retry).

I have logged the following byte sizes of the trampoline onion with a 2
trampoline onion hop and changing amounts of r_tags:

3 rtags:
payload size [0]: 113 (hop size: 81)
payload size [1]: 440 (hop size: 295) ( 52 bytes/rtag )
payload size [2]: 550 (hop size: 78)

2 rtags:
payload size [0]: 113 (hop size: 81)
payload size [1]: 386 (hop size: 241) ( 52 bytes/rtag )
payload size [2]: 496 (hop size: 78)

1 rtag:
payload size [0]: 113 (hop size: 81)
payload size [1]: 334 (hop size: 189) ( 52 bytes/rtag )
payload size [2]: 444 (hop size: 78)

0 rtags:
payload size [0]: 113 (hop size: 81)
payload size [1]: 282 (hop size: 137)
payload size [2]: 392 (hop size: 78)

As can be seen in the data, using 2 trampoline hops there is not enough
space for even a single r_tag which is why this option is being removed
too.
2025-04-14 19:20:11 +02:00
ThomasV
bcb6df72a7 Merge pull request #9729 from spesmilo/recursive_config
recursive config file
2025-04-11 20:02:25 +02:00
SomberNight
bd0085e680 tests: commands: add test_setconfig_none 2025-04-11 17:17:53 +00:00
ThomasV
8f3490c87e recursive config file
move plugin variables into sub dictionaries of user config
2025-04-11 19:06:48 +02:00
SomberNight
5c17f452b1 tests: util.custom_task_factory: fix res warn "coro was never awaited"
follow-up 70d1e1170e

```
=============================== warnings summary ===============================
tests/test_util.py::TestUtil::test_custom_task_factory
  /tmp/cirrus-ci-build/tests/test_util.py:504: RuntimeWarning: coroutine 'TestUtil.test_custom_task_factory.<locals>.foo' was never awaited
    self.assertEqual(foo().__qualname__, task.get_coro().__qualname__)
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

```
2025-04-10 13:10:58 +00:00
ThomasV
1494cca486 Merge pull request #9681 from f321x/txbatcherror
wallet: make txbatcher wait for network connection
2025-04-09 12:44:57 +02:00
SomberNight
70d1e1170e asyncio: clarify strong refs for run_coroutine_threadsafe
We added some code in 0b3a283586
to explicitly hold strong refs for all tasks/futures. At the time I was uncertain if that also solves
GC issues with asyncio.run_coroutine_threadsafe.
ref https://github.com/spesmilo/electrum/pull/9608#issuecomment-2703681663

Looks like it does. run_coroutine_threadsafe *is* going through the custom task factory.
See the unit test.
The somewhat confusing thing is that we need a few event loop iterations for the task factory to run,
due to how run_coroutine_threadsafe is implemented. And also, the task that we will hold as strong ref
in the global set is not the concurrent.futures.Future that run_coroutine_threadsafe returns.

So this commit simply "fixes" the unit test so that it showcases this, and removes related, older, plumbing
from util.py that we now know is no longer needed because of this.
2025-04-08 18:54:58 +00:00
f321x
42bd334f5b add coinchooser test with zero fee estimation 2025-04-07 17:14:25 +02:00
f321x
03018fa218 make txbatcher wait for network connection 2025-04-04 11:43:24 +02:00
ThomasV
001a1152fd add unit tests for ln_utxo_reserve 2025-04-03 14:39:05 +02:00
SomberNight
5c233ac325 ci: enable more flake8 stuff
```
$ export ELECTRUM_LINTERS=E9,E101,E129,E273,E274,E703,E71,E722,F5,F6,F7,F8,W191,W29,B
$ export ELECTRUM_LINTERS_IGNORE=B007,B009,B010,B019,B036,F541,F841
$ flake8 . --count --select="$ELECTRUM_LINTERS" --ignore="$ELECTRUM_LINTERS_IGNORE" --show-source --statistics --exclude "*_pb2.py,electrum/_vendor/"
./electrum/commands.py:98:1: F811 redefinition of unused 'format_satoshis' from line 48
def format_satoshis(x):
^
./electrum/commands.py:437:9: F811 redefinition of unused 'Mnemonic' from line 62
        from .mnemonic import Mnemonic
        ^
./electrum/gui/qt/wizard/wallet.py:37:5: F811 redefinition of unused 'Daemon' from line 14
    from electrum.daemon import Daemon
    ^
./electrum/lntransport.py:14:1: F811 redefinition of unused 'Optional' from line 12
from typing import NamedTuple, List, Tuple, Mapping, Optional, TYPE_CHECKING, Union, Dict, Set, Sequence
^
./electrum/lntransport.py:14:1: F811 redefinition of unused 'TYPE_CHECKING' from line 12
from typing import NamedTuple, List, Tuple, Mapping, Optional, TYPE_CHECKING, Union, Dict, Set, Sequence
^
./electrum/plugin.py:966:13: F811 redefinition of unused 'hid' from line 593
            import hid
            ^
./electrum/plugin.py:1040:13: F811 redefinition of unused 'hid' from line 593
            import hid
            ^
./electrum/util.py:44:1: F811 redefinition of unused 'json' from line 26
import json
^
./electrum/util.py:46:1: F811 redefinition of unused 'NamedTuple' from line 29
from typing import NamedTuple, Optional
^
./electrum/util.py:46:1: F811 redefinition of unused 'Optional' from line 29
from typing import NamedTuple, Optional
^
./electrum/util.py:1456:56: F811 redefinition of unused 'traceback' from line 34
        async def __aexit__(self, exc_type, exc_value, traceback):
                                                       ^
./electrum/wallet_db.py:536:9: F811 redefinition of unused 'LOCAL' from line 46
        LOCAL = 1
        ^
./electrum/wallet_db.py:537:9: F811 redefinition of unused 'REMOTE' from line 46
        REMOTE = -1
        ^
./tests/test_bitcoin.py:28:1: F811 redefinition of unused 'bitcoin' from line 9
from electrum import crypto, constants, bitcoin
^
./tests/test_txbatcher.py:11:1: F811 redefinition of unused 'Transaction' from line 7
from electrum.transaction import Transaction, PartialTxInput, PartialTxOutput, TxOutpoint
^
./tests/test_wallet_vertical.py:20:1: F811 redefinition of unused 'Transaction' from line 10
from electrum.transaction import Transaction, PartialTxOutput, tx_from_any, Sighash
^
16    F811 redefinition of unused 'format_satoshis' from line 48
16

```
2025-04-02 16:21:59 +00:00
SomberNight
5f551183ae tests/test_txbatcher.py: fix race in network.next_tx()
With `txbatcher.SLEEP_INTERVAL = 0.01`, on my laptop, the batcher called try_broadcasting() even before next_tx() effectively awaited _tx_event.
This resulted in the test failing due to timeout.

Basically, if the txbatcher.SLEEP_INTERVAL was too low, the 2-4 event loop iterations needed to await _tx_event.wait() took too long.
(note that the exact number of event loop iterations needed depends on the python version and the OS)
2025-04-01 15:54:05 +00:00
SomberNight
2807be08c8 tests/test_txbatcher.py: de-duplicate "create wallet" logic 2025-04-01 15:00:23 +00:00
f321x
0b19b660c5 don't use fallback feerates in lightning by default 2025-04-01 14:12:02 +02:00
ThomasV
e67cb560f6 Commands: add option documentation in docstring.
This allows plugins to document the commands they add.

The docstring is parsed as a regular expression:

    arg:<type>:<name>:<description>\n

Types are defined in commands.arg_types.

Note that this commit removes support for single letter
shortcuts in command options.

If a command is not properly documented, a warning is issued
with print(), because no logger is available at this point.
2025-03-19 13:03:38 +01:00
ThomasV
d8964a00e7 config vars for plugins 2025-03-19 11:59:05 +01:00
ThomasV
26910ef81d Merge pull request #9620 from accumulator/lightning_pass_invoice_not_bolt11
refactor lnworker.pay_invoice to accept Invoice object instead of bolt11 string
2025-03-18 20:09:01 +01:00
f321x
246f03fe20 allow all plugins to be either zip or directory based 2025-03-17 16:27:33 +01:00
ThomasV
81d4e90f66 Merge branch 'master' into 140325-force-close-exception 2025-03-14 19:38:13 +01:00
SomberNight
977d8b1dd6 wallet: kill create_transaction 2025-03-14 17:19:41 +00:00
f321x
5edbf923cd fix sweeping anchor outputs with multiple change addresses option enabled, don't consider tx inputs sufficient value if there are no outputs so change outpu gets added 2025-03-14 18:15:04 +01:00
SomberNight
3c3778db9c wallet: towards killing create_transaction: rm "sign" arg 2025-03-14 16:44:46 +00:00
ThomasV
fbebe7de1a Make lnwatcher not async
This fixes offline history not having the proper labels
2025-03-14 11:09:11 +01:00
SomberNight
8ccd31fe49 wallet: set_frozen_state_of_coins to handle freeze=None
Internally whether a coin is frozen is tri-state:
- forced-True, set by the user
- forced-False, set by the user
- unset/default: is_frozen_coin() can decide whether the coin should be frozen

This patch lets set_frozen_state_of_coins() undo a previous explicit setting of True/False,
by calling it with a value of None.
Note: there is still no way in the GUI to undo an explicit setting of True/False.
2025-03-13 18:32:58 +00:00
ThomasV
bdb7a82220 batch payment manager:
The class TxBatcher handles the creation, broadcast and replacement
of replaceable transactions. Callers (LNWatcher, SwapManager) use
methods add_payment_output and add_sweep_info. Transactions
created by TxBatcher may combine sweeps and outgoing payments.

Transactions created by TxBatcher will have their fee bumped
automatically (this was only the case for sweeps before).

TxBatcher manages several TxBatches. TxBatches are created
dynamically when needed.

The GUI does not touch txbatcher transactions:
  - wallet.get_candidates_for_batching excludes txbatcher
    transactions
  - RBF dialogs do not work with txbatcher transactions

wallet:
  - instead of reading config variables, make_unsigned_transaction
    takes new parameters: base_tx, send_change_to_lighting

tests:
  - unit tests in test_txbatcher.py (replaces test_sswaps.py)
  - force all regtests to use MPP, so that we sweep transactions
    with several HTLCs. This forces the payment manager to aggregate
    first-stage HTLC tx inputs. second-stage are not batched for now.
2025-03-13 10:17:10 +01:00
ThomasV
798df671ea If we have proposed htlcs in a channel that was force-closed,
call lnworker.htlc_failed once the htlc_tx is deeply mined.

In the case of a forwarding, this will fail incoming htlcs.
(fixes #8547)
2025-03-12 20:11:11 +01:00
SomberNight
0d120145d5 tests/regtest.py: trivial clean-up
from dde7fd09ac
2025-03-10 14:54:50 +00:00
ThomasV
707d87a0d4 Revert "regtests: in py ctx, refer to configvars directly, not as str literals"
This reverts commit dde7fd09ac.
2025-03-10 12:01:41 +01:00
Sander van Grieken
6fdb6c93f7 refactor lnworker.pay_invoice to accept Invoice object instead of bolt11 string
rename lnworker._check_invoice to lnworker._check_bolt11_invoice
2025-03-09 14:47:34 +01:00
Sander van Grieken
6331ac0f85 lnutil: imports, whitespace, remove unused code 2025-03-09 12:30:02 +01:00
SomberNight
dde7fd09ac regtests: in py ctx, refer to configvars directly, not as str literals
though in the bash script, we cannot do this
2025-03-07 17:26:12 +00:00
SomberNight
a7c634b1ab regtests: extend "extract_preimage" to cover both types of extracts 2025-03-07 17:06:55 +00:00
SomberNight
c146321924 regtests: e-x: update COIN name
ref ca59151d73
2025-03-07 15:40:12 +00:00
SomberNight
6ea6c80f21 regtests: split up long line on e-x invocation 2025-03-07 15:40:05 +00:00
ThomasV
8011ae00ab Merge pull request #9586 from f321x/jit-block-preimage
Add mechanism to block htlcs from settling back
2025-03-07 12:04:39 +01:00
f321x
109ad2b3da add mechanism to block htlcs from settling back 2025-03-07 12:03:59 +01:00
ghost43
32b0e62cdc Merge pull request #9608 from SomberNight/202503_asyncio_gc
asyncio: hold our own strong refs for tasks and futures
2025-03-06 16:07:55 +00:00
ThomasV
074ede8f12 make_unsigned_transaction: pass merge_duplicate_outputs as
a parameter, because it must not be used in some contexts
(e.g. tx batcher).

This makes config.MERGE_DUPLICATE_OUTPUTS a GUI option
2025-03-06 09:22:12 +01:00
SomberNight
0b3a283586 asyncio: hold our own strong refs for tasks and futures
see https://docs.python.org/3.13/library/asyncio-task.html#asyncio.create_task :

> Important
>
> Save a reference to the result of this function, to avoid a task
> disappearing mid-execution. The event loop only keeps weak references
> to tasks. A task that isn’t referenced elsewhere may get garbage
> collected at any time, even before it’s done. For reliable
> “fire-and-forget” background tasks, gather them in a collection

ref https://github.com/python/cpython/issues/91887
ref https://github.com/beeware/toga/pull/2814
2025-03-05 17:06:04 +00:00
SomberNight
2aa427b4bf Merge branch 'pr/9507': qt: refactor NetworkChoiceLayout to ProxyWidget+ServerWidget
ref https://github.com/spesmilo/electrum/pull/9507
2025-03-05 15:01:05 +00:00
ThomasV
ab14c3e138 tx batching in GUI:
- discard config.WALLET_BATCH_RBF
 - allow the user to choose base_tx from a list of batching
   candidates in ConfirmTxDialog
2025-03-05 14:07:02 +01:00
ThomasV
840243e029 separate fee policy from config
- Wallet.make_unsigned_transaction takes a FeePolicy parameter
 - fee sliders act on a FeePolicy instead of config
 - different fee policies may be used for different purposes
 - do not detect dust outputs in lnsweep, delegate that to lnwatcher
2025-03-05 10:29:26 +01:00
SomberNight
ffc84ab7d3 tests: fix warning in test_lntransport.py::TestLNTransport::test_loop
$ python3 -X dev -m pytest tests/test_lntransport.py

```
=============================== warnings summary ===============================
tests/test_lntransport.py::TestLNTransport::test_loop
  /usr/local/lib/python3.11/asyncio/streams.py:414: ResourceWarning: unclosed <StreamWriter transport=<_SelectorSocketTransport closing fd=15 read=idle write=<idle, bufsize=0>> reader=<StreamReader transport=<_SelectorSocketTransport closing fd=15 read=idle write=<idle, bufsize=0>>>>
    warnings.warn(f"unclosed {self!r}", ResourceWarning)
```
2025-03-04 15:49:22 +00:00
Sander van Grieken
fea598cfbe network: create ProxySettings class replacing dict and encapsulating proxy related funcs,
allow enable/disable proxy without nuking proxy mode, host and port (explicit enable_proxy config setting),
move tor probe from frontend to backend code, add probe buttons for Qt and QML
2025-03-04 14:23:33 +01:00
f321x
d348da811a introduce gossip query handling and forwarding 2025-03-04 13:10:37 +01:00
SomberNight
f445a476f9 config: raise on setting mistyped or non-existent ConfigVar 2025-03-03 17:41:23 +00:00
ThomasV
3d2531cb93 reintroduce separate request types for lightning and onchain
cmdline: add_request has a --lightning option
2025-02-25 11:27:32 +01:00
ThomasV
ea4adbb4d6 Merge pull request #9573 from f321x/simplify_subswap_fee
Simplify submarine swap onchain fee model to single base fee 'mining_fee'
2025-02-20 18:42:56 +01:00
ThomasV
7eec7e6a20 increase sleep time in TestOnionMessageManager.test_forward
Co-authored-by: ghost43 <somber.night@protonmail.com>
2025-02-20 18:05:21 +01:00