note: Would it be ok to log potentially secret (semi-sensitive) data?
We take care not to log onchain private keys as they are extremely sensitive,
but what about logging a LN transport message that might contain channel secrets?
Decided not to, for now.
Sometimes the trampoline node fails to send a payment and returns
TEMPORARY_CHANNEL_FAILURE, while it succeeds with a higher trampoline
fee. My guess is that the initial fee was not sufficient to try
all routes, and that a higher fee allows to use extra routes. I
suppose it would make more sense if the trampoline node returned
TRAMPOLINE_FEE_INSUFFICIENT in that case.
wallet.lock -> wallet.transaction_lock order must be respected.
deadlock example:
```
# ThreadID: 19100
File: ...\Python\Python310\lib\threading.py", line 966, in _bootstrap
self._bootstrap_inner()
File: ...\Python\Python310\lib\threading.py", line 1009, in _bootstrap_inner
self.run()
File: ...\Python\Python310\lib\threading.py", line 946, in run
self._target(*self._args, **self._kwargs)
File: "...\electrum\electrum\util.py", line 1552, in run_event_loop
loop.run_until_complete(stopping_fut)
File: ...\Python\Python310\lib\asyncio\base_events.py", line 633, in run_until_complete
self.run_forever()
File: ...\Python\Python310\lib\asyncio\windows_events.py", line 321, in run_forever
super().run_forever()
File: ...\Python\Python310\lib\asyncio\base_events.py", line 600, in run_forever
self._run_once()
File: ...\Python\Python310\lib\asyncio\base_events.py", line 1896, in _run_once
handle._run()
File: ...\Python\Python310\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File: "...\electrum\electrum\wallet.py", line 494, in on_event_adb_added_verified_tx
self._update_invoices_and_reqs_touched_by_tx(tx_hash)
File: "...\electrum\electrum\wallet.py", line 2454, in _update_invoices_and_reqs_touched_by_tx
status = self.get_invoice_status(request)
File: "...\electrum\electrum\wallet.py", line 2333, in get_invoice_status
paid, conf = self.is_onchain_invoice_paid(invoice)
File: "...\electrum\electrum\wallet.py", line 1120, in is_onchain_invoice_paid
is_paid, conf_needed, relevant_txs = self._is_onchain_invoice_paid(invoice)
File: "...\electrum\electrum\wallet.py", line 1095, in _is_onchain_invoice_paid
with self.lock, self.transaction_lock:
File: "...\electrum\electrum\address_synchronizer.py", line 70, in acquire
return self._lock.acquire(*args, **kwargs)
# ThreadID: 20040
File: "C:\Program Files\JetBrains\PyCharm Community Edition 2021.3.3\plugins\python-ce\helpers\pycharm\_jb_pytest_runner.py", line 51, in <module>
sys.exit(pytest.main(args, plugins_to_load + [Plugin]))
File: "...\Python\Python310\site-packages\_pytest\config\__init__.py", line 164, in main
ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
File: ...\Python\Python310\lib\site-packages\pluggy\_hooks.py", line 265, in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
File: ...\Python\Python310\lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File: ...\Python\Python310\lib\site-packages\pluggy\_callers.py", line 39, in _multicall
res = hook_impl.function(*args)
File: "...\Python\Python310\site-packages\_pytest\main.py", line 315, in pytest_cmdline_main
return wrap_session(config, _main)
File: "...\Python\Python310\site-packages\_pytest\main.py", line 268, in wrap_session
session.exitstatus = doit(config, session) or 0
File: "...\Python\Python310\site-packages\_pytest\main.py", line 322, in _main
config.hook.pytest_runtestloop(session=session)
File: ...\Python\Python310\lib\site-packages\pluggy\_hooks.py", line 265, in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
File: ...\Python\Python310\lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File: ...\Python\Python310\lib\site-packages\pluggy\_callers.py", line 39, in _multicall
res = hook_impl.function(*args)
File: "...\Python\Python310\site-packages\_pytest\main.py", line 347, in pytest_runtestloop
item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
File: ...\Python\Python310\lib\site-packages\pluggy\_hooks.py", line 265, in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
File: ...\Python\Python310\lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File: ...\Python\Python310\lib\site-packages\pluggy\_callers.py", line 39, in _multicall
res = hook_impl.function(*args)
File: "...\Python\Python310\site-packages\_pytest\runner.py", line 111, in pytest_runtest_protocol
runtestprotocol(item, nextitem=nextitem)
File: "...\Python\Python310\site-packages\_pytest\runner.py", line 130, in runtestprotocol
reports.append(call_and_report(item, "call", log))
File: "...\Python\Python310\site-packages\_pytest\runner.py", line 219, in call_and_report
call = call_runtest_hook(item, when, **kwds)
File: "...\Python\Python310\site-packages\_pytest\runner.py", line 258, in call_runtest_hook
return CallInfo.from_call(
File: "...\Python\Python310\site-packages\_pytest\runner.py", line 338, in from_call
result: Optional[TResult] = func()
File: "...\Python\Python310\site-packages\_pytest\runner.py", line 259, in <lambda>
lambda: ihook(item=item, **kwds), when=when, reraise=reraise
File: ...\Python\Python310\lib\site-packages\pluggy\_hooks.py", line 265, in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
File: ...\Python\Python310\lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File: ...\Python\Python310\lib\site-packages\pluggy\_callers.py", line 39, in _multicall
res = hook_impl.function(*args)
File: "...\Python\Python310\site-packages\_pytest\runner.py", line 166, in pytest_runtest_call
item.runtest()
File: "...\Python\Python310\site-packages\_pytest\unittest.py", line 327, in runtest
self._testcase(result=self) # type: ignore[arg-type]
File: ...\Python\Python310\lib\unittest\case.py", line 650, in __call__
return self.run(*args, **kwds)
File: ...\Python\Python310\lib\unittest\case.py", line 591, in run
self._callTestMethod(testMethod)
File: ...\Python\Python310\lib\unittest\case.py", line 549, in _callTestMethod
method()
File: "...\electrum\electrum\tests\test_invoices.py", line 120, in test_wallet_without_ln_creates_payreq_and_gets_paid_onchain
self.assertEqual(PR_PAID, wallet1.get_invoice_status(pr))
File: "...\electrum\electrum\wallet.py", line 2333, in get_invoice_status
paid, conf = self.is_onchain_invoice_paid(invoice)
File: "...\electrum\electrum\wallet.py", line 1120, in is_onchain_invoice_paid
is_paid, conf_needed, relevant_txs = self._is_onchain_invoice_paid(invoice)
File: "...\electrum\electrum\wallet.py", line 1095, in _is_onchain_invoice_paid
with self.lock, self.transaction_lock:
File: "...\electrum\electrum\address_synchronizer.py", line 70, in acquire
return self._lock.acquire(*args, **kwargs)
```
With a PyCharm debugger attached, sometimes the python process is so
CPU-starved for me that create_and_start_event_loop() returned
before the event loop actually started, resulting in weird errors.
I guess this could happen even without a debugger attached on a
sufficiently slow CPU.
```
...\electrum\electrum\wallet.py:3580: in restore_wallet_from_text
wallet = Wallet(db, storage, config=config)
...\electrum\electrum\wallet.py:3501: in __new__
wallet = WalletClass(db, storage, config=config)
...\electrum\electrum\wallet.py:3345: in __init__
Deterministic_Wallet.__init__(self, db, storage, config=config)
...\electrum\electrum\wallet.py:3135: in __init__
self.synchronize()
...\electrum\electrum\wallet.py:3283: in synchronize
count += self.synchronize_sequence(False)
...\electrum\electrum\wallet.py:3267: in synchronize_sequence
self.create_new_address(for_change)
...\electrum\electrum\wallet.py:3254: in create_new_address
self.adb.add_address(address)
...\electrum\electrum\address_synchronizer.py:213: in add_address
self.up_to_date_changed()
...\electrum\electrum\address_synchronizer.py:680: in up_to_date_changed
util.trigger_callback('adb_set_up_to_date', self)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <electrum.util.CallbackManager object at 0x000002B1788AD6F0>
event = 'adb_set_up_to_date'
args = (<electrum.address_synchronizer.AddressSynchronizer object at 0x000002B17A687670>,)
def trigger_callback(self, event, *args):
"""Trigger a callback with given arguments.
Can be called from any thread. The callback itself will get scheduled
on the event loop.
"""
if self.asyncio_loop is None:
self.asyncio_loop = get_asyncio_loop()
> assert self.asyncio_loop.is_running(), "event loop not running"
E AssertionError: event loop not running
...\electrum\electrum\util.py:1734: AssertionError
```
In the past we decided not to put a timestamp into the stderr logs
to have shorter log lines (to save column width in a terminal).
However over time I at least have found that it would be valuable
to have timestamps also in the stderr - e.g. when users provide logs.
Often I am only interested in the time taken between logged events,
so as a compromise to still save some length, I propose adding relative
timestamps (relative to process startup time).
Compare these log lines from the file logger:
```
20220816T120601.882003Z | INFO | gui.qt.ElectrumGui | starting Qt main loop
20220816T120601.905619Z | INFO | gui.qt.history_list.HistoryModel | refreshing... reason: update_tabs
20220816T120601.911908Z | DEBUG | util.profiler | Abstract_Wallet.get_full_history 0.0059 sec
20220816T120602.095670Z | INFO | interface.[testnet.hsmiths.com:53012] | connection established. version: ['ElectrumX 1.16.0', '1.4']
```
With these from the existing stderr logger:
```
I/w | wallet.Standard_Wallet.[test_segwit_3] | set_up_to_date: True
I/i | interface.[testnet.aranguren.org:51002] | set blockchain with height 2343721
D | util.profiler | ElectrumWindow.load_wallet 0.0778 sec
I | gui.qt.ElectrumGui | starting Qt main loop
```
With these re what I propose for the stderr logger:
```
3.20 | D | util.profiler | Abstract_Wallet.get_full_history 0.0029 sec
5.70 | I | i/interface.[testnet1.bauerj.eu:50002] | disconnecting due to: ConnectError(ConnectionRefusedError(22, 'The remote computer refused the network connection', None, 1225, None))
38.63 | I | w/wallet.Standard_Wallet.[9dk] | starting taskgroup.
38.84 | D | util.profiler | WalletDB._write 0.0059 sec
62.96 | I | i/interface.[blockstream.info:993] | set blockchain with height 2343722
150.65 | I | exchange_rate.CoinGecko | getting fx quotes for EUR
```
As suggested by SomberNight in PR #8091, the difference is that this
commit handles currencies in case-insensitive manner.
Co-authored-by: ghost43 <somber.night@protonmail.com>
Which ccy rates are available depends on the configured
exchange (config key use_exchange) and the configured currency (config
key currency). Only for some exchanges, the fx.ccy-BTC fx rate is
available (depends on the ExchangeBase.get_rates implementation).
As they say, for hodlers 1 BTC = 1 BTC.
There are many cryptocurrencies available in CoinGecko and some other
exchange rate providers. If the user wants to use a cryptocurrency as
a display currency, the precisions used to be 2. This patch adds
precisions of some cryptocurrencies.
