c9536180c5
closes https://github.com/spesmilo/electrum/issues/8403
> In Python 3.10 that worked fine, however in Python 3.11 large integer check https://github.com/python/cpython/issues/95778, so now this throws an error.
Apparently this change was deemed a security fix and was backported to all supported branches of CPython (going back to 3.7). i.e. it affects ~all versions of python (if sufficiently updated with bugfix patches), not just 3.11
> Some offending node aliases:
> ```
> ergvein-fiatchannels
> test-mainnet
> arakis
> ```
The features bits set by some of these nodes:
```
(1, 7, 8, 11, 13, 14, 17, 19, 23, 27, 45, 32973, 52973)
(1, 7, 8, 11, 13, 14, 17, 19, 23, 27, 39, 45, 55, 32973, 52973)
```
> P.S. I see there are a lot of nodes with 253 bytes in their feature vectors. Any idea why that could happen?
Note that the valid [merged-into-spec features](50b2df24a2/09-features.md) currently only go as high as ~51.
However the spec does not specify how to choose feature bits for experimental stuff, so I guess some people are using values in the 50k range. The only limit imposed by the spec on the length of the features bitvector is an implicit one due to the max message size: every msg must be smaller than 65KB, and the features bitvector needs to fit inside the init message, hence it can be up to ~524K bits.
(note that the features are not stored in a sparse representation in the init message and in gossip messages, so if many nodes set such high feature bits, that would noticably impact the size of the gossip).
-----
Anyway, our current implementation of LnFeatures is subclassing IntFlag, and it looks like it does not work well for such large integers. I've managed to make IntFlags reasonably in python 3.11 by overriding __str__ and __repr__ (note that in cpython it is apparently only the base2<->base10 conversions that are slow, power-of-2 conversions are fast, so we can e.g. use `hex()`). However in python 3.10 and older, enum.py itself seems really slow for bigints, e.g. enum._decompose in python 3.10.
Try e.g. this script, which is instant in py3.11 but takes minutes in py3.10:
```py
from enum import IntFlag
class c(IntFlag):
known_flag_1 = 1 << 0
known_flag_2 = 1 << 1
known_flag_3 = 1 << 2
if hasattr(IntFlag, "_numeric_repr_"): # python 3.11+
_numeric_repr_ = hex
def __repr__(self):
return f"<{self._name_}: {hex(self._value_)}>"
def __str__(self):
return hex(self._value_)
a = c(2**70000-1)
q1 = repr(a)
q2 = str(a)
```
AFAICT we have two options: either we rewrite LnFeatures so that it does not use IntFlag (and enum.py), or, for the short term as workaround, we could just reject very large feature bits.
For now, I've opted to the latter, rejecting feature bits over 10k.
(note that another option is bumping the min required python to 3.11, in which case with the overrides added in this commit the performance looks perfectly fine)
Electrum - Lightweight Bitcoin client
Licence: MIT Licence
Author: Thomas Voegtlin
Language: Python (>= 3.8)
Homepage: https://electrum.org/
Getting started
(If you've come here looking to simply run Electrum, you may download it here.)
Electrum itself is pure Python, and so are most of the required dependencies, but not everything. The following sections describe how to run from source, but here is a TL;DR:
$ sudo apt-get install libsecp256k1-dev
$ python3 -m pip install --user ".[gui,crypto]"
Not pure-python dependencies
If you want to use the Qt interface, install the Qt dependencies:
$ sudo apt-get install python3-pyqt5
For elliptic curve operations, libsecp256k1 is a required dependency:
$ sudo apt-get install libsecp256k1-dev
Alternatively, when running from a cloned repository, a script is provided to build libsecp256k1 yourself:
$ sudo apt-get install automake libtool
$ ./contrib/make_libsecp256k1.sh
Due to the need for fast symmetric ciphers, cryptography is required. Install from your package manager (or from pip):
$ sudo apt-get install python3-cryptography
If you would like hardware wallet support, see this.
Running from tar.gz
If you downloaded the official package (tar.gz), you can run Electrum from its root directory without installing it on your system; all the pure python dependencies are included in the 'packages' directory. To run Electrum from its root directory, just do:
$ ./run_electrum
You can also install Electrum on your system, by running this command:
$ sudo apt-get install python3-setuptools python3-pip
$ python3 -m pip install --user .
This will download and install the Python dependencies used by
Electrum instead of using the 'packages' directory.
It will also place an executable named electrum in ~/.local/bin,
so make sure that is on your PATH variable.
Development version (git clone)
(For OS-specific instructions, see here for Windows, and for macOS)
Check out the code from GitHub:
$ git clone https://github.com/spesmilo/electrum.git
$ cd electrum
$ git submodule update --init
Run install (this should install dependencies):
$ python3 -m pip install --user -e .
Create translations (optional):
$ sudo apt-get install python3-requests gettext qttools5-dev-tools
$ ./contrib/pull_locale
Finally, to start Electrum:
$ ./run_electrum
Run tests
Run unit tests with pytest:
$ pytest electrum/tests -v
To run a single file, specify it directly like this:
$ pytest electrum/tests/test_bitcoin.py -v
Creating Binaries
Contributing
Any help testing the software, reporting or fixing bugs, reviewing pull requests and recent changes, writing tests, or helping with outstanding issues is very welcome. Implementing new features, or improving/refactoring the codebase, is of course also welcome, but to avoid wasted effort, especially for larger changes, we encourage discussing these on the issue tracker or IRC first.
Besides GitHub,
most communication about Electrum development happens on IRC, in the
#electrum channel on Libera Chat. The easiest way to participate on IRC is
with the web client, web.libera.chat.