Correction to comment in prev commit (and removing it here):
spec.loader.exec_module does not spawn new threads, it simply
executes the module in the current thread.
I got confused but turns out "load_plugin" itself is sometimes
not called from the main thread. Specifically (e.g.), since the recent
wizard rewrite, in the qt gui, the wizard loads the hww plugins
from a new thread.
This now better explains the macos hww crashes: they had started
appearing because we upgraded hidapi (which made it more sensitive
to having to import from main thread) AND scanning(->importing) from
the wizard no longer happened on the main thread after the rewrite.
Plugins should be thread-safe in terms of where they are imported from.
Let's log the importer thread's name (added here), to help recognise
related threading issues.
Ah ok, I give up for now... the prev does not really work.
The prior commit works on Windows but not on macOS.
On Windows, it would package all plugins as code and only as code.
On MacOS however, it would not package any plugins at all. And with this commit,
where I mark the plugins folder to be packaged as *data*, it packages all plugins as *both* code and data.
Not sure why.
Let's just package all plugins as both code and data, and ignore the code instances explicitly...
(=plugin name collision)
This assert is currently failing in pyinstaller builds
(probably have been since it was added in 552bfb589a).
Looks like the pyinstaller binaries include two copies of the hw_wallet and the jade plugins.
This likely has been the case for years, we just never noticed. -- but it started triggering the new assert.
```
2.78 | I | plugin | iter_modules=[
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='audio_modem', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='bitbox02', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='coldcard', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='cosigner_pool', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='digitalbitbox', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='hw_wallet', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='jade', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='keepkey', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='labels', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='ledger', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='payserver', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='revealer', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='safe_t', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='swapserver', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='trezor', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='trustedcoin', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='virtualkeyboard', ispkg=True),
ModuleInfo(module_finder=<pyimod02_importers.FrozenImporter object at 0x015FDFB8>, name='hw_wallet', ispkg=True),
ModuleInfo(module_finder=<pyimod02_importers.FrozenImporter object at 0x015FDFB8>, name='jade', ispkg=True)]
```
Plugins.stop() is now part of the normal shutdown flow (since 90f39bce88),
so this shaves up to 100 ms off that (avg: 50 ms).
It is also waited on in around ~60 unit tests: this saves 3 sec.
This method is often called when there is already an existing paired
client for the keystore, in which case we can avoid scan_devices() -
which would needlessly take several seconds.
Related to prev commit: multiple (compatible) keystores can now match
with the same HardwareClientBase, so we might as well also allow
reusing existing Clients for the wizard.
- the DeviceMgr no longer uses xpubs to keep track of paired hw devices
- instead, introduce keystore.pairing_code(), based on soft_device_id
- xpubs are now only used in a single place when the actual pairing happens
- motivation is to allow pairing a single device with multiple generic
output script descriptors, not just a single account-level xpub
- as a side-effect, we now allow pairing a device with multiple open
windows simultaneously (if keystores have the same root fingerprint
-- was already the case before if keystores had the same xpub)
* typecheck client selected by keystore - check if plugin types match
* add log msg + refactor if branching
* moved type check to own function; moved from 'client_for_keystore' to 'force_pair_xpub' and 'client_by_xpub'
* refactor
Co-authored-by: avirgovi <avirgovi@cisco.com>
ledger now gives an error if querying xpub while device is not (unlocked and in bitcoin app).
we do query the xpub however, to calc root fingerprint to be used as soft device id.
trace:
I | plugin.DeviceMgr | scanning devices...
D | util.profiler | DeviceMgr.scan_devices 3.4463
I | plugin.DeviceMgr | Registering <electrum.plugins.ledger.ledger.Ledger_Client object at 0x0000029DF6B08520>
E | gui.qt.installwizard.InstallWizard |
Traceback (most recent call last):
File "...\electrum\electrum\plugins\ledger\ledger.py", line 225, in checkDevice
self.perform_hw1_preflight()
File "...\electrum\electrum\plugin.py", line 362, in wrapper
return run_in_hwd_thread(partial(func, *args, **kwargs))
File "...\electrum\electrum\plugin.py", line 352, in run_in_hwd_thread
return func()
File "...\electrum\electrum\plugins\ledger\ledger.py", line 219, in perform_hw1_preflight
raise e
File "...\electrum\electrum\plugins\ledger\ledger.py", line 179, in perform_hw1_preflight
firmwareInfo = self.dongleObject.getFirmwareVersion()
File "...\Python38\site-packages\btchip\btchip.py", line 563, in getFirmwareVersion
response = self.dongle.exchange(bytearray(apdu))
File "...\Python38\site-packages\btchip\btchipComm.py", line 127, in exchange
raise BTChipException("Invalid status %04x" % sw, sw)
btchip.btchipException.BTChipException: Exception : Invalid status 6700
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "...\electrum\electrum\base_wizard.py", line 317, in _choose_hw_device
device_infos = devmgr.unpaired_device_infos(None, plugin, devices=scanned_devices,
File "...\electrum\electrum\plugin.py", line 612, in unpaired_device_infos
soft_device_id=client.get_soft_device_id(),
File "...\electrum\electrum\plugin.py", line 362, in wrapper
return run_in_hwd_thread(partial(func, *args, **kwargs))
File "...\electrum\electrum\plugin.py", line 355, in run_in_hwd_thread
return fut.result()
File "...\Python38\lib\concurrent\futures\_base.py", line 439, in result
return self.__get_result()
File "...\Python38\lib\concurrent\futures\_base.py", line 388, in __get_result
raise self._exception
File "...\Python38\lib\concurrent\futures\thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "...\electrum\electrum\plugins\ledger\ledger.py", line 91, in get_soft_device_id
self._soft_device_id = self.request_root_fingerprint_from_device()
File "...\electrum\electrum\plugin.py", line 362, in wrapper
return run_in_hwd_thread(partial(func, *args, **kwargs))
File "...\electrum\electrum\plugin.py", line 352, in run_in_hwd_thread
return func()
File "...\electrum\electrum\plugins\hw_wallet\plugin.py", line 259, in request_root_fingerprint_from_device
child_of_root_xpub = self.get_xpub("m/0'", xtype='standard')
File "...\electrum\electrum\plugin.py", line 362, in wrapper
return run_in_hwd_thread(partial(func, *args, **kwargs))
File "...\electrum\electrum\plugin.py", line 352, in run_in_hwd_thread
return func()
File "...\electrum\electrum\plugins\ledger\ledger.py", line 57, in catch_exception
return func(self, *args, **kwargs)
File "...\electrum\electrum\plugins\ledger\ledger.py", line 111, in get_xpub
self.checkDevice()
File "...\electrum\electrum\plugin.py", line 362, in wrapper
return run_in_hwd_thread(partial(func, *args, **kwargs))
File "...\electrum\electrum\plugin.py", line 352, in run_in_hwd_thread
return func()
File "...\electrum\electrum\plugins\ledger\ledger.py", line 228, in checkDevice
raise UserFacingException(_("Device not in Bitcoin mode")) from e
electrum.util.UserFacingException: Device not in Bitcoin mode
W | gui.qt.installwizard.InstallWizard | error getting device infos for ledger: Device not in Bitcoin mode
shesek reported on IRC:
> the button widget for opening plugins configuration gets cached in `settings_widgets`
> even after the plugin is disabled and re-enabled, which causes it to call `settings_dialog()`
> on the previous plugin instance that got unloaded instead of the new one.
