channel_db.load_data() is slow, slowing down startup time (when trampoline is disabled).
util.list_enabled_bits() is one of the main contributors to the slowness, called by validate_features().
One could argue that we could even simply *not* call validate_features for gossip messages as part of load_data,
as they have already been validated before storing them in the db. However re-validating them there is a good
clean-up/sanity check IMO. Note that what is considered "valid" can change over time, so just because validate_features
passed when we originally received and stored a gossip message, it might no longer be valid a year later if the bolts change.
This caching decreases the time needed for load_data on two different machines / gossip dbs as below:
47 sec -> 10 sec
18 sec -> 6 sec
If instead of caching, I just rm the validate_features() calls, the benchmarks are almost identical, within noise.
That is, the cache looks really effective.
(the rest of the slowness is mostly due to lnmsg.decode_msg)
```
>>> lnutil.validate_features.cache_info()
CacheInfo(hits=172674, misses=287, maxsize=1000, currsize=277)
```
-----
We could alternatively directly cache util.list_enabled_bits (instead of validate_features).
That would be a bit slower and might end up using a lot more memory in some cases I think, but maybe conceptually would be cleaner.
Also note that if validate_features() raises an exception, that is not cached.
```
$ export ELECTRUM_LINTERS=E9,E101,E129,E273,E274,E703,E71,E722,F5,F6,F7,F8,W191,W29,B
$ export ELECTRUM_LINTERS_IGNORE=B007,B009,B010,B019,B036,F541,F841
$ flake8 . --count --select="$ELECTRUM_LINTERS" --ignore="$ELECTRUM_LINTERS_IGNORE" --show-source --statistics --exclude "*_pb2.py,electrum/_vendor/"
./electrum/commands.py:98:1: F811 redefinition of unused 'format_satoshis' from line 48
def format_satoshis(x):
^
./electrum/commands.py:437:9: F811 redefinition of unused 'Mnemonic' from line 62
from .mnemonic import Mnemonic
^
./electrum/gui/qt/wizard/wallet.py:37:5: F811 redefinition of unused 'Daemon' from line 14
from electrum.daemon import Daemon
^
./electrum/lntransport.py:14:1: F811 redefinition of unused 'Optional' from line 12
from typing import NamedTuple, List, Tuple, Mapping, Optional, TYPE_CHECKING, Union, Dict, Set, Sequence
^
./electrum/lntransport.py:14:1: F811 redefinition of unused 'TYPE_CHECKING' from line 12
from typing import NamedTuple, List, Tuple, Mapping, Optional, TYPE_CHECKING, Union, Dict, Set, Sequence
^
./electrum/plugin.py:966:13: F811 redefinition of unused 'hid' from line 593
import hid
^
./electrum/plugin.py:1040:13: F811 redefinition of unused 'hid' from line 593
import hid
^
./electrum/util.py:44:1: F811 redefinition of unused 'json' from line 26
import json
^
./electrum/util.py:46:1: F811 redefinition of unused 'NamedTuple' from line 29
from typing import NamedTuple, Optional
^
./electrum/util.py:46:1: F811 redefinition of unused 'Optional' from line 29
from typing import NamedTuple, Optional
^
./electrum/util.py:1456:56: F811 redefinition of unused 'traceback' from line 34
async def __aexit__(self, exc_type, exc_value, traceback):
^
./electrum/wallet_db.py:536:9: F811 redefinition of unused 'LOCAL' from line 46
LOCAL = 1
^
./electrum/wallet_db.py:537:9: F811 redefinition of unused 'REMOTE' from line 46
REMOTE = -1
^
./tests/test_bitcoin.py:28:1: F811 redefinition of unused 'bitcoin' from line 9
from electrum import crypto, constants, bitcoin
^
./tests/test_txbatcher.py:11:1: F811 redefinition of unused 'Transaction' from line 7
from electrum.transaction import Transaction, PartialTxInput, PartialTxOutput, TxOutpoint
^
./tests/test_wallet_vertical.py:20:1: F811 redefinition of unused 'Transaction' from line 10
from electrum.transaction import Transaction, PartialTxOutput, tx_from_any, Sighash
^
16 F811 redefinition of unused 'format_satoshis' from line 48
16
```
```
./electrum/commands.py:144:9: F824 `global known_commands` is unused: name is never assigned in scope
global known_commands
^
./electrum/commands.py:1916:9: F824 `global known_commands` is unused: name is never assigned in scope
global known_commands
^
./electrum/gui/qt/main_window.py:2405:13: F824 `nonlocal done` is unused: name is never assigned in scope
nonlocal done
^
./electrum/i18n.py:52:5: F824 `global language` is unused: name is never assigned in scope
global language
^
./electrum/plugin.py:189:9: F824 `global _root_permission_cache` is unused: name is never assigned in scope
global _root_permission_cache
^
5 F824 `global known_commands` is unused: name is never assigned in scope
5
```
With `txbatcher.SLEEP_INTERVAL = 0.01`, on my laptop, the batcher called try_broadcasting() even before next_tx() effectively awaited _tx_event.
This resulted in the test failing due to timeout.
Basically, if the txbatcher.SLEEP_INTERVAL was too low, the 2-4 event loop iterations needed to await _tx_event.wait() took too long.
(note that the exact number of event loop iterations needed depends on the python version and the OS)
it was annoying that the checkbox got included in the submitted issue
The "Description" text also gets included in the submitted issue, which is also annoying,
but I could not figure out how to remove that.
