follow-up 713a20e309https://github.com/spesmilo/electrum/pull/9833
On Windows, above commit broke dns_hacks.py with dnspython==2.0.0 and 2.1.0.
Newer dnspython works.
Root cause not immediately obvious. Probably not worth debugging, I will just bump the required version instead.
With dnspython==2.0.0, the log gets spammed and dns fails:
```
$ python3 -m pip install --user "dnspython==2.0.0"
10.59 | E | asyncio | Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
Traceback (most recent call last):
File "...\Python310\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "...\Python310\lib\asyncio\proactor_events.py", line 158, in _call_connection_lost
self._protocol.connection_lost(exc)
File "...\Python310\site-packages\dns\_asyncio_backend.py", line 38, in connection_lost
self.recvfrom.set_exception(exc)
asyncio.exceptions.InvalidStateError: invalid state
```
With dnspython==2.1.0, no more log spam but all dns resolutions time out:
```
$ python3 -m pip install --user "dnspython==2.1.0"
33.29 | I | dns_hacks | dnspython failed to resolve dns (AAAA) for 'testnet.qtornado.com' with error: Timeout('The DNS operation timed out after 31.591506242752075 seconds')
33.29 | I | dns_hacks | dnspython failed to resolve dns (AAAA) for 'api.coingecko.com' with error: Timeout('The DNS operation timed out after 31.590490579605103 seconds')
33.29 | I | dns_hacks | dnspython failed to resolve dns (A) for 'testnet.qtornado.com' with error: Timeout('The DNS operation timed out after 31.591506242752075 seconds')
33.29 | I | dns_hacks | dnspython failed to resolve dns (A) for 'api.coingecko.com' with error: Timeout('The DNS operation timed out after 31.590490579605103 seconds')
33.35 | I | dns_hacks | dnspython failed to resolve dns (AAAA) for 'blockstream.info' with error: Timeout('The DNS operation timed out after 31.59534502029419 seconds')
33.35 | I | dns_hacks | dnspython failed to resolve dns (A) for 'blockstream.info' with error: Timeout('The DNS operation timed out after 31.594367265701294 seconds')
33.38 | I | dns_hacks | dnspython failed to resolve dns (AAAA) for 'electrum.blockstream.info' with error: Timeout('The DNS operation timed out after 31.602211713790894 seconds')
33.38 | I | dns_hacks | dnspython failed to resolve dns (A) for 'electrum.blockstream.info' with error: Timeout('The DNS operation timed out after 31.60122585296631 seconds')
```
This removes support for Ledger HW.1 and "Nano" (non-S) devices.
These were manufactured/sold around 2015-2016, and are long unsupported by the upstream vendor.
We previously added a deprecation warning to the GUI [0] released in 4.3.3 (2023-01-02), to warn owners of these devices.
This PR now fully removes support.
As a consequence, the unmaintained btchip-python dependency can now be removed, which solves [1].
[0]: 9b82eb6d06
[1]: https://github.com/spesmilo/electrum/issues/9370#issuecomment-2593675364
note: these sdists are now reproducibly built.
Ideally the person updating the pinned hash should be different from the one who uploaded the dep to PyPI and reproduce and verify the hash. This prevents hiding a backdoor in just the sdist uploaded to PyPI.
but people can try to reproduce the PyPI sdists at any time and ring alarm bells if they can't
- `bitstring` started depending on `bitarray` in version 4.1 [0]
- that would mean one additional dependency for us (from yet another maintainer), which is not even pure python
- we only use bitstring for bolt11-parsing
- hence this PR rewrites the bolt11-parsing and removes `bitstring` as dependency
- note: I benchmarked lndecode using [1], and the new code performs better,
taking around 80% time needed for old code (when using bitstring 3.1.9, pure python).
Though the variance is quite large in both cases.
[0]: 95ee533ee4/release_notes.txt (L108)
[1]: d7597d96d0
- partial writes are append only.
- StoredDict objects will append partial writes to the wallet
file when items are added, replaced, removed.
- Lists in the wallet file that have not been registered
as StoredObject are converted to StoredList, which
overloads append() and remove(). Those methods too will
append partial writes to the wallet file.
- Unlike the old jsonpatch branch, this branch does not support
file encryption. Encrypted files always fully rewritten, even
if the change before encryption is a partial write.
- @amount_msat.validator prevents the creation of invoices with e.g. too large amounts
- however the qml gui is mutating invoices by directly setting the `amount_msat` field,
and it looks like attrs validators only run during init.
We can use `on_setattr` (introduced in attrs==20.1.0).
- a wallet db upgrade is added to rm existing insane invoices
- btw the qml gui was already doing its own input validation on the textedit
(see qeconfig.btcAmountRegex). however that only limits the input to not have more
chars than what is needed to represent 21M BTC (e.g. you can still enter 99M BTC,
which the invoice logic does not tolerate later on - but is normally caught).
fixes https://github.com/spesmilo/electrum/issues/8582
6.2.0 was released to put a minimum requirement on hidapi 0.14.0,
which includes the fix for this issue:
https://github.com/libusb/hidapi/issues/531
That bug caused hidapi on macOS 13.3 to report 0 as the interface
number for all hid devices, which led to the bitbox02 multi edition being listed
twice instead of once - once for the main HW wallet interface and once erroneously
For the U2F interface (which should not be listed).
or more like four:
```
$ pipdeptree
hatchling==1.11.1
- editables [required: >=0.3, installed: 0.3]
- packaging [required: >=21.3, installed: 21.3]
- pyparsing [required: >=2.0.2,!=3.0.5, installed: 3.0.9]
- pathspec [required: >=0.10.1, installed: 0.10.1]
- pluggy [required: >=1.0.0, installed: 1.0.0]
- tomli [required: >=1.2.2, installed: 2.0.1]
```
Let's monitor how the situation evolves, and whether other packages start requiring hatchling,
but for now I am not going to add four new packages into the trusted base set...
Pinning colorama to an older version for now.
related:
https://github.com/tartley/colorama/pull/338https://github.com/tartley/colorama/issues/349
The current system python in the docker baseimage is py3.9, while the
targetpython and hostpython built by p4a are py3.8, and this was causing
linker issues in the pyqt5 build.
It is also cleaner IMHO to have p4a handle what is needed for a recipe
instead of assuming it is available in the system. (and I think this is
how other existing recipes work)
fixes https://github.com/spesmilo/electrum/issues/8016
protobuf 4.x introduced breaking changes compared to 3.20.
To adapt, we would have to regenerate paymentrequest_pb2.py, using protoc>=3.19.0, however ubuntu does not have new enough protoc.
Also, unsure if newly generated paymentrequest_pb2.py would be supported by older versions of protobuf.
Best to just wait for things to settle.
```
electrum/__init__.py:20: in <module>
from .wallet import Wallet
electrum/wallet.py:70: in <module>
from . import transaction, bitcoin, coinchooser, paymentrequest, ecc, bip32
electrum/paymentrequest.py:37: in <module>
from . import paymentrequest_pb2 as pb2
electrum/paymentrequest_pb2.py:36: in <module>
_descriptor.FieldDescriptor(
.tox/py3/lib/python3.9/site-packages/google/protobuf/descriptor.py:560: in __new__
_message.Message._CheckCalledFromGeneratedFile()
E TypeError: Descriptors cannot not be created directly.
E If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E If you cannot immediately regenerate your protos, some other possible workarounds are:
E 1. Downgrade the protobuf package to 3.20.x or lower.
E 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E
E More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
```
ceaae1b6a3/CHANGES.txt (L47)
This allows more freedom than using releases from PyPI.
(atm there is no released version that fixes https://github.com/pyinstaller/pyinstaller/pull/6701 )
Also, we now build the pyinstaller bootloader, just like in the windows build:
one fewer binary blob to trust.
see https://github.com/spesmilo/electrum/pull/7721#issuecomment-1072669548
-----
pyinstaller 4.2 failed (during its runtime) to create exes (worked with cpython 3.9.10, but not with cpython 3.9.11):
```
80572 INFO: Processing module hooks...
80573 INFO: Loading module hook 'hook-certifi.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
80618 INFO: Loading module hook 'hook-cryptography.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
82879 INFO: Loading module hook 'hook-dns.rdata.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
84147 INFO: Loading module hook 'hook-mnemonic.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
84207 INFO: Loading module hook 'hook-pycparser.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
84212 INFO: Loading module hook 'hook-usb1.py' from 'C:\\python3\\lib\\site-packages\\usb1\\__pyinstaller'...
84215 INFO: --- libusb1 pyinstaller hook ---
84226 INFO: Added libusb binaries: [('C:\\python3\\lib\\site-packages\\usb1\\libusb-1.0.dll', 'usb1')]
84228 INFO: Loading module hook 'hook-difflib.py' from 'C:\\python3\\lib\\site-packages\\PyInstaller\\hooks'...
84237 INFO: Excluding import of doctest from module difflib
84237 INFO: Loading module hook 'hook-distutils.py' from 'C:\\python3\\lib\\site-packages\\PyInstaller\\hooks'...
Unable to find "C:\python3\Include\pyconfig.h" when adding binary and data files.This would mean your Python installation doesn't
come with proper library files. This usually happens by missing development
package, or unsuitable build parameters of Python installation.
* On Debian/Ubuntu, you would need to install Python development packages
* apt-get install python3-dev
* apt-get install python-dev
* If you're building Python by yourself, please rebuild your Python with
`--enable-shared` (or, `--enable-framework` on Darwin)
🗯 ERROR: build-electrum-git failed
```
Looks like this might be fixed in pyinstaller 4.3 (we are using 4.2):
https://github.com/pyinstaller/pyinstaller/pull/5218
-----
While trying to update pyinstaller to have that fix, several issues found with versions 4.3-4.10,
now reported upstream and already fixed:
https://github.com/pyinstaller/pyinstaller/issues/6338https://github.com/pyinstaller/pyinstaller/issues/6339https://github.com/pyinstaller/pyinstaller/issues/6686
So the pyinstaller commit pinned here is from the stable "v4" branch, some commits after the 4.10 tag,
which has fixes for the above issues.
