Ah ok, I give up for now... the prev does not really work.
The prior commit works on Windows but not on macOS.
On Windows, it would package all plugins as code and only as code.
On MacOS however, it would not package any plugins at all. And with this commit,
where I mark the plugins folder to be packaged as *data*, it packages all plugins as *both* code and data.
Not sure why.
Let's just package all plugins as both code and data, and ignore the code instances explicitly...
The hw_wallet and jade plugins were being included twice in the pyinstaller binaries.
They were apparently the only two plugins being picked as "modules" (a.pure),
which by default are included as compiled-bytecode.
In addition, we included "electrum/plugins" as data (a.data), which meant all
plugins in source form.
Instead of hacking around to fix the specific issue, this attempts a larger clean-up.
(=plugin name collision)
This assert is currently failing in pyinstaller builds
(probably have been since it was added in 552bfb589a).
Looks like the pyinstaller binaries include two copies of the hw_wallet and the jade plugins.
This likely has been the case for years, we just never noticed. -- but it started triggering the new assert.
```
2.78 | I | plugin | iter_modules=[
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='audio_modem', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='bitbox02', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='coldcard', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='cosigner_pool', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='digitalbitbox', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='hw_wallet', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='jade', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='keepkey', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='labels', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='ledger', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='payserver', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='revealer', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='safe_t', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='swapserver', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='trezor', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='trustedcoin', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='virtualkeyboard', ispkg=True),
ModuleInfo(module_finder=<pyimod02_importers.FrozenImporter object at 0x015FDFB8>, name='hw_wallet', ispkg=True),
ModuleInfo(module_finder=<pyimod02_importers.FrozenImporter object at 0x015FDFB8>, name='jade', ispkg=True)]
```
Not sure why this a problem in the mac build but not in the windows build...
Also not sure why this only came up now -- was working before.
(I changed a few things in prev commits re the build scripts but I can't see the cause)
```
34827 INFO: Processing module hooks...
34828 INFO: Loading module hook 'hook-cryptography.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
35932 INFO: Loading module hook 'hook-mnemonic.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
35937 INFO: Loading module hook 'hook-certifi.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
35939 INFO: Loading module hook 'hook-dns.rdata.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
36502 INFO: Loading module hook 'hook-usb1.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/usb1/__pyinstaller'...
36503 INFO: --- libusb1 pyinstaller hook ---
36504 INFO: Added libusb binaries: []
36505 INFO: Loading module hook 'hook-sqlite3.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
36932 INFO: Loading module hook 'hook-pkg_resources.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
38303 INFO: Loading module hook 'hook-PyQt5.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
38584 WARNING: Hidden import "sip" not found!
38586 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
38587 INFO: Loading module hook 'hook-lib2to3.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
38678 INFO: Loading module hook 'hook-_tkinter.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
39005 INFO: checking Tree
39006 INFO: Building Tree because Tree-00.toc is non existent
39006 INFO: Building Tree Tree-00.toc
39021 INFO: checking Tree
39022 INFO: Building Tree because Tree-01.toc is non existent
39022 INFO: Building Tree Tree-01.toc
39157 INFO: checking Tree
39158 INFO: Building Tree because Tree-02.toc is non existent
39158 INFO: Building Tree Tree-02.toc
39165 INFO: Loading module hook 'hook-encodings.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
39568 INFO: Loading module hook 'hook-distutils.util.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
39586 INFO: Loading module hook 'hook-platform.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
39587 INFO: Loading module hook 'hook-PyQt5.QtGui.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
39734 INFO: Loading module hook 'hook-PyQt5.QtPrintSupport.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
39984 INFO: Loading module hook 'hook-PyQt5.QtMultimedia.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
40164 INFO: Loading module hook 'hook-packaging.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
40165 INFO: Loading module hook 'hook-pickle.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
40187 INFO: Loading module hook 'hook-heapq.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
40192 INFO: Loading module hook 'hook-difflib.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
40196 INFO: Loading module hook 'hook-PyQt5.QtWidgets.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
40361 INFO: Loading module hook 'hook-multiprocessing.util.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
40382 INFO: Loading module hook 'hook-sysconfig.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
40421 INFO: Loading module hook 'hook-PyQt5.QtNetwork.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
40545 INFO: Loading module hook 'hook-xml.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
40691 INFO: Loading module hook 'hook-distutils.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
40692 INFO: Loading module hook 'hook-PyQt5.QtCore.py' from '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks'...
40903 INFO: Looking for ctypes DLLs
40987 WARNING: Library Cfgmgr32 required via ctypes not found
40992 WARNING: Library setupapi required via ctypes not found
40994 WARNING: Library Advapi32 required via ctypes not found
40996 WARNING: Ignoring /System/Library/Frameworks/IOKit.framework/IOKit imported from /Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/serial/tools/list_ports_osx.py - only basenames are supported with ctypes imports!
40996 WARNING: Ignoring /System/Library/Frameworks/CoreFoundation.framework/CoreFoundation imported from /Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/serial/tools/list_ports_osx.py - only basenames are supported with ctypes imports!
41028 INFO: Analyzing run-time hooks ...
41041 INFO: Including run-time hook '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pyqt5.py'
41043 INFO: Including run-time hook '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py'
41047 INFO: Including run-time hook '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks/rthooks/pyi_rth_inspect.py'
41049 INFO: Including run-time hook '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks/rthooks/pyi_rth_subprocess.py'
41051 INFO: Including run-time hook '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
41055 INFO: Including run-time hook '/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgres.py'
41140 INFO: Looking for dynamic libraries
719 INFO: gettext setting initial language to None
Traceback (most recent call last):
File "/Users/vagrant/electrum/contrib/osx/build-venv/bin/pyinstaller", line 8, in <module>
sys.exit(run())
File "/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/__main__.py", line 178, in run
run_build(pyi_config, spec_file, **vars(args))
File "/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/__main__.py", line 59, in run_build
PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
File "/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/building/build_main.py", line 934, in main
build(specfile, distpath, workpath, clean_build)
File "/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/building/build_main.py", line 856, in build
exec(code, spec_namespace)
File "contrib/osx/osx.spec", line 48, in <module>
a = Analysis([electrum+ MAIN_SCRIPT,
File "/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/building/build_main.py", line 381, in __init__
self.__postinit__()
File "/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/building/datastruct.py", line 173, in __postinit__
self.assemble()
File "/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/building/build_main.py", line 660, in assemble
isolated.call(find_binary_dependencies, list(self.binaries), self.binding_redirects, collected_packages)
File "/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/isolated/_parent.py", line 238, in call
return isolated.call(function, *args, **kwargs)
File "/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/isolated/_parent.py", line 176, in call
raise RuntimeError(f"Child process call to {function.__name__}() failed with:\n" + output)
RuntimeError: Child process call to find_binary_dependencies() failed with:
File "/Users/vagrant/electrum/electrum/gui/qml/__init__.py", line 8, in <module>
import PyQt6
ModuleNotFoundError: No module named 'PyQt6'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/isolated/_child.py", line 63, in run_next_command
output = function(*args, **kwargs)
File "/Users/vagrant/electrum/contrib/osx/build-venv/lib/python3.10/site-packages/PyInstaller/building/build_main.py", line 157, in find_binary_dependencies
__import__(package)
File "/Users/vagrant/electrum/electrum/gui/qml/__init__.py", line 10, in <module>
sys.exit("Error: Could not import PyQt6. On Linux systems, you may try 'sudo apt-get install python3-pyqt6'")
SystemExit: Error: Could not import PyQt6. On Linux systems, you may try 'sudo apt-get install python3-pyqt6'
🗯 ERROR: Could not build binary
```
```
602 WARNING: Failed to collect submodules for 'pkg_resources._vendor.pyparsing.diagram' because importing 'pkg_resources._vendor.pyparsing.diagram' raised: ModuleNotFoundError: No module named 'railroad'
libsecp256k1 library failed to load. exceptions: [FileNotFoundError("Could not find module 'C:\\python3\\lib\\site-packages\\electrum\\libsecp256k1-2.dll' (or one of its dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'C:\\python3\\lib\\site-packages\\electrum\\libsecp256k1-1.dll'(or one of its dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'C:\\python3\\lib\\site-packages\\electrum\\libsecp256k1-0.dll' (or one of its dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'libsecp256k1-2.dll' (or one ofits dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'libsecp256k1-1.dll' (or one of its dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'libsecp256k1-0.dll' (or one of its dependencies). Try using the full path with constructor syntax.")]
libsecp256k1 library failed to load. exceptions: [FileNotFoundError("Could not find module 'C:\\python3\\lib\\site-packages\\electrum\\libsecp256k1-2.dll' (or one of its dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'C:\\python3\\lib\\site-packages\\electrum\\libsecp256k1-1.dll'(or one of its dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'C:\\python3\\lib\\site-packages\\electrum\\libsecp256k1-0.dll' (or one of its dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'libsecp256k1-2.dll' (or one ofits dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'libsecp256k1-1.dll' (or one of its dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'libsecp256k1-0.dll' (or one of its dependencies). Try using the full path with constructor syntax.")]
libsecp256k1 library failed to load. exceptions: [FileNotFoundError("Could not find module 'C:\\python3\\lib\\site-packages\\electrum\\libsecp256k1-2.dll' (or one of its dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'C:\\python3\\lib\\site-packages\\electrum\\libsecp256k1-1.dll'(or one of its dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'C:\\python3\\lib\\site-packages\\electrum\\libsecp256k1-0.dll' (or one of its dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'libsecp256k1-2.dll' (or one ofits dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'libsecp256k1-1.dll' (or one of its dependencies). Try using the full path with constructor syntax."), FileNotFoundError("Could not find module 'libsecp256k1-0.dll' (or one of its dependencies). Try using the full path with constructor syntax.")]
5921 WARNING: collect_data_files - skipping data collection for module 'electrum.plugins' as it is not a package.
```
pyinstaller tries to import electrum and its different submodules at build-time
during the "Analysis" phase. sys._GUI_QT_VERSION was not getting set there,
and the resulting exception was blocking pyinstaller from discovering that
gui/common_qt is being used.
at runtime:
```
$ ./dist/Electrum.app/Contents/MacOS/run_electrum
1.53 | E | daemon.Daemon | GUI raised exception: Exception('Error loading trustedcoin plugin: ModuleNotFoundError("No module named \'electrum.gui.common_qt\'")'). shutting down.
1.53 | E | __main__ | daemon.run_gui errored
Traceback (most recent call last):
File "electrum/plugin.py", line 135, in load_plugin
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/Users/vagrant/electrum/dist/Electrum.app/Contents/MacOS/electrum/plugins/trustedcoin/qt.py", line 51, in <module>
from .common_qt import TrustedcoinPluginQObject
File "/Users/vagrant/electrum/dist/Electrum.app/Contents/MacOS/electrum/plugins/trustedcoin/common_qt.py", line 16, in <module>
from electrum.gui.common_qt.plugins import PluginQObject
ModuleNotFoundError: No module named 'electrum.gui.common_qt'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "run_electrum", line 456, in handle_cmd
d.run_gui()
File "electrum/daemon.py", line 617, in run_gui
self.gui_object = gui.ElectrumGui(config=self.config, daemon=self, plugins=self._plugins)
File "electrum/util.py", line 473, in do_profile
o = func(*args, **kw_args)
File "electrum/gui/qt/__init__.py", line 153, in __init__
self.plugins.load_plugin('trustedcoin')
File "electrum/plugin.py", line 138, in load_plugin
raise Exception(f"Error loading {name} plugin: {repr(e)}") from e
Exception: Error loading trustedcoin plugin: ModuleNotFoundError("No module named 'electrum.gui.common_qt'")
```
qml: use backend sighash check and add user confirmation path
qt: use backend sighash check and add user confirmation path
qml: include get_warning_for_risk_of_burning_coins_as_fees test in txdetails
qt: add warning icon to sighash warning
add sighash and fee checks to wallet.sign_transaction, making all warnings fatal unless ignore_warnings is set to True
tests: test sign_transaction on both code paths with ignore_warnings True and False,
raise specific exceptions TransactionPotentiallyDangerousException and TransactionDangerousException
fixes https://github.com/spesmilo/electrum/issues/3315
The cause was that tracebacks look different whether stack items have source text available.
When using the pyinstaller windows binary, there is no source text available.
Example when running from source:
```
>>> a
Traceback (most recent call last):
File "...\electrum\gui\qt\console.py", line 256, in exec_command
result = eval(command, self.namespace, self.namespace)
File "<string>", line 1, in <module>
NameError: name 'a' is not defined
```
Example for pyinstaller windows binary:
```
>>> a
Traceback (most recent call last):
File "electrum\gui\qt\console.py", line 256, in exec_command
File "<string>", line 1, in <module>
NameError: name 'a' is not defined
```
ThomasV reported contrib/release.sh generating .aab artifacts.
aabs are shady business... best to avoid handing privkeys to big G (or any 3rd party!).
This must be a consequence of the recent p4a rebase,
e.g. 04bb63bbf0
gui/qt/rbf_dialog.py (old) lines 57-64 were implementing logic that should not be part of GUI code.
Case in point, gui/qml/qetxfinalizer.py (old) lines 511-513 duplicated half of that logic but not the other half.
That logic is now moved to wallet.get_bumpfee_strategies_for_tx().
More context: a user on irc got confused when using the qml gui. They were sending "max" and wanted to bump_fee.
The qml gui selected the "preserve_payment" strategy by default, using which there was no solution, and the user
did not notice that the strategy can be altered (via the "method" dropdown). The qt gui had logic to select
"decrease_payment" by default in such a case (which does find a solution to bump) but this logic was not
duplicated in the qml gui.
Instead of duplicating the logic, this commit moves it to shared lib code.
avoid overloading invoiceParser with new payment identifiers while potentially long-running
resolve/finalize steps are ongoing, show errors inresolve/finalize steps to user.
- all forwarding types use the same flow
- forwarding callback returns a htlc_key or None
- forwarding info is persisted in lnworker:
- ongoing_forwardings
- downstream to upstream htlc_key
- htlc_key -> error_bytes
