AaronFiore/electrum
1
0
Fork 0
Code Activity
18,905 Commits 1 Branch 181 Tags
5cb844fc7c4ad89efbc5dd8ec0dffcd61fd729fe
Commit Graph

144 Commits

Author SHA1 Message Date
SomberNight
c52a29fc22 build: appimage: fix trezor plugin for new trezorlib 
- similar to prev commit that fixes it for the pyinstaller builds
- note that .dist-info/RECORD files are still not reproducible for many packages :((((

```
$ cd dist/
$ ./electrum-*-x86_64.AppImage1 --appimage-extract
$ mv squashfs-root/ squashfs-root1/
$ ./electrum-*-x86_64.AppImage2 --appimage-extract
$ mv squashfs-root/ squashfs-root2/
$ $(cd squashfs-root1; find -type f -exec sha256sum '{}' \; > ./../sha256sum1)
$ $(cd squashfs-root2; find -type f -exec sha256sum '{}' \; > ./../sha256sum2)
$ diff sha256sum1 sha256sum2 > d
$ cat d
507c507
< 269a133bd0d3c85265219c14154323ed934ac36ef9b389da609e43788de2bbcd  ./usr/lib/python3.12/site-packages/cffi-1.17.1.dist-info/RECORD
---
> d2c08987f207eed8e90476c576eeff67c0809d1bfd35234e583a2dc6895c5ff1  ./usr/lib/python3.12/site-packages/cffi-1.17.1.dist-info/RECORD
518c518
< d08bd76099191932f3cd289c19427ccab51d9c00a6c2e5126c57f83a84ec2246  ./usr/lib/python3.12/site-packages/aiohttp-3.12.4.dist-info/RECORD
---
> 281c5c7e2b28e56134fa8690a2a446e3ebc515f8cb3705cd5d06e83c53812fab  ./usr/lib/python3.12/site-packages/aiohttp-3.12.4.dist-info/RECORD
630c630
< 33665bc46c14e2c0f409ef2627170b093f6d7741e9516928928acbeeb717f155  ./usr/lib/python3.12/site-packages/propcache-0.3.1.dist-info/RECORD
---
> 4fddb29289418edb3ec392a09b8efa826333a60cf508055cac809375e497174e  ./usr/lib/python3.12/site-packages/propcache-0.3.1.dist-info/RECORD
1497c1497
< f80c1b47d15a7262ab2b81bcfc1f69816961b9f6b0fd3775ee5b5eb40d4b7a5b  ./usr/lib/python3.12/site-packages/pyqt6_sip-13.10.2.dist-info/RECORD
---
> 1b8fb8ea4d50d401cd4f6689531ab4922cb064b9caaec1aae6bd885c3a387eca  ./usr/lib/python3.12/site-packages/pyqt6_sip-13.10.2.dist-info/RECORD
3108c3108
< ca06bbe1dba05cd647e89411dd301fad15f42d0ff53ac2c509391cf0303c15c4  ./usr/lib/python3.12/site-packages/multidict-6.4.4.dist-info/RECORD
---
> 400741699b95d3c6bde3c64c7a4bb04b26cbdaee437a17b398d202794a1bd4cd  ./usr/lib/python3.12/site-packages/multidict-6.4.4.dist-info/RECORD
3233c3233
< f67e8f271725676df296fab7fab72099f3b853905f9ff677b5de261a52412411  ./usr/lib/python3.12/site-packages/hidapi-0.14.0.post4.dist-info/RECORD
---
> f4d43d5e4c3c2aa3e69fa40c7307b23b82ce4c5e4d5fb041a75b11ddc2ed7423  ./usr/lib/python3.12/site-packages/hidapi-0.14.0.post4.dist-info/RECORD
3271c3271
< aaa92bb84d0f56e82832903f1109dd4362c09ea4af0fefe3d0c95509bc0920ed  ./usr/lib/python3.12/site-packages/cbor2-5.6.5.dist-info/RECORD
---
> 8a21af0fadf22898266581df9987c97ee737a670bef9d71d5f53427619466b88  ./usr/lib/python3.12/site-packages/cbor2-5.6.5.dist-info/RECORD
```

-----

Example RECORD files:

```
$ diff squashfs-root1/./usr/lib/python3.12/site-packages/aiohttp-3.12.4.dist-info/RECORD squashfs-root2/./usr/lib/python3.12/site-packages/aiohttp-3.12.4.dist-info/RECORD
64c64
< aiohttp/_http_parser.cpython-312-x86_64-linux-gnu.so,sha256=SjcWt8faDDTOxBHetnDdlQU31sPj0Suc8pwY1_UnKkE,2868384
---
> aiohttp/_http_parser.cpython-312-x86_64-linux-gnu.so,sha256=xdf5Uy_NnFLmSmqNmeY7wytgwFdY7pHxZLOFFfl2Aq8,2868384
66c66
< aiohttp/_http_writer.cpython-312-x86_64-linux-gnu.so,sha256=g6PDJVuICGYq8rywuI6XFWLP6tw1KLWLpwSvJODWo18,510280
---
> aiohttp/_http_writer.cpython-312-x86_64-linux-gnu.so,sha256=U_Y0F0oYClrGRIKO_agXtAohjJ_jqfG375vESBZ56CY,510280
80c80
< aiohttp/_websocket/mask.cpython-312-x86_64-linux-gnu.so,sha256=u0PluBaJntMBktyD1AYv267FvbUO6gpj-Nt7erLj3Qg,257384
---
> aiohttp/_websocket/mask.cpython-312-x86_64-linux-gnu.so,sha256=tEnvrEzSPEtaFI8V_Ey08zy4SBmDhnm-QnxTxgDSYM4,257384
85c85
< aiohttp/_websocket/reader_c.cpython-312-x86_64-linux-gnu.so,sha256=wfcclW8jYOTVeLDRPO87dPvOBrtXiqxBsnEbP0yyGpI,1816496
---
> aiohttp/_websocket/reader_c.cpython-312-x86_64-linux-gnu.so,sha256=jfey8U3MSIp04r0bG_a1TFkIKSzMQ-FLSAWD3nY4wgg,1816496
```

```
$ diff squashfs-root1/./usr/lib/python3.12/site-packages/hidapi-0.14.0.post4.dist-info/RECORD squashfs-root2/./usr/lib/python3.12/site-packages/hidapi-0.14.0.post4.dist-info/RECORD
1c1
< hid.cpython-312-x86_64-linux-gnu.so,sha256=JefmLwo-XKZZmno3PBdmHnREed0Yw7IaPYCI2k1UmbM,1152024
---
> hid.cpython-312-x86_64-linux-gnu.so,sha256=38htTAcIC4nnTPIIjD0jkdtdwtGyWlBIhQwQ-g0K0iI,1152024
12c12
< hidraw.cpython-312-x86_64-linux-gnu.so,sha256=dJ4kVI0jXUj_Ba2cMcAL7Wjy8fkO21VPbhAmR-5n30Y,1156768
---
> hidraw.cpython-312-x86_64-linux-gnu.so,sha256=7908Jgbg7A-3JRtfKabPr_WJkMIjYgd5pX78_Yajo54,1156768
```

-----

error at runtime:
```
 22.51 | E | plugins.trezor.trezor | error importing trezor plugin deps
Traceback (most recent call last):
  File "/tmp/.mount_electrteDkNu/usr/lib/python3.12/importlib/metadata/__init__.py", line 397, in from_name
    return next(cls.discover(name=name))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/.mount_electrteDkNu/usr/lib/python3.12/site-packages/electrum/plugins/trezor/trezor.py", line 29, in <module>
    from .clientbase import TrezorClientBase, RecoveryDeviceInputMethod
  File "/tmp/.mount_electrteDkNu/usr/lib/python3.12/site-packages/electrum/plugins/trezor/clientbase.py", line 18, in <module>
    import trezorlib.device
  File "/tmp/.mount_electrteDkNu/usr/lib/python3.12/site-packages/trezorlib/device.py", line 27, in <module>
    from slip10 import SLIP10
  File "/tmp/.mount_electrteDkNu/usr/lib/python3.12/site-packages/slip10/__init__.py", line 6, in <module>
    __version__ = importlib.metadata.version(__package__ or __name__)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/.mount_electrteDkNu/usr/lib/python3.12/importlib/metadata/__init__.py", line 889, in version
    return distribution(distribution_name).version
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/.mount_electrteDkNu/usr/lib/python3.12/importlib/metadata/__init__.py", line 862, in distribution
    return Distribution.from_name(distribution_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/.mount_electrteDkNu/usr/lib/python3.12/importlib/metadata/__init__.py", line 399, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for slip10

```
 2025-06-05 18:46:12 +00:00
SomberNight
d17c4beee9 appimage build: add comment we should update to new appimagetool toolchain 2025-06-05 16:52:06 +00:00
SomberNight
fa271e3958 build: bump python versions in binaries 
note: 3.12 just transitioned to security-only status,
so can't bump win/mac binaries without switching to 3.13
(as we don't compile our own cpython for those)
 2025-06-05 16:51:56 +00:00
SomberNight
0c42dfc5f1 build: rerun freeze_packages 2025-06-05 16:50:55 +00:00
SomberNight
608fedd7cd build: libsecp: rm hardcoded ABI version 2025-06-05 16:50:51 +00:00
SomberNight
bf0ad20c60 build: bump python versions in binaries 2025-06-05 16:50:35 +00:00
SomberNight
37ca5f7eff build: appimage: bump base from debian buster(2019) to bullseye(2021) 2025-06-05 16:50:31 +00:00
SomberNight
31b176169a contrib: mv locale-related scripts to contrib/locale/ 2025-04-14 17:18:37 +00:00
SomberNight
1144d9b8ea build: add script build_cleanlocale.sh 2025-04-14 17:18:34 +00:00
SomberNight
3567a4cfb0 mv git submodule electrum-locale from contrib to electrum/locale 
- this merges `contrib/deterministic-build/locale` and `electrum/locale`
- it is now once again possible have translations when running from a local git clone
    - which was already possible in the past before crowdin removed their unauthenticated APIs
        - see https://github.com/spesmilo/electrum/issues/9531
    - however, the translations available are the often-old frozen strings from electrum-locale
        - while previously one could just download the latest strings from crowdin
 2025-04-14 17:18:30 +00:00
SomberNight
b70da1d3b9 build: set YARL_NO_EXTENSIONS for all builds 
due to cython version mismatch -- easier to just disable the compilation for now
 2025-04-01 16:59:26 +00:00
SomberNight
f35437f03c build: set ELECTRUM_ECC_DONT_COMPILE=1, instead manually build lib 
Haven't checked if electrum-ecc compiles libsecp reproducibly.
For now let's just keep the old flow.
(but if we spent time on making that compilation reproducible,
the appimage and the macos builds could use it directly)
 2024-10-10 15:46:18 +00:00
SomberNight
276488e3d0 binaries: document min requirements for target systems 2024-10-04 14:02:51 +00:00
SomberNight
cfe8502f96 qt desktop gui: upgrade qt5->qt6 
closes https://github.com/spesmilo/electrum/issues/8007
 2024-09-18 15:48:38 +00:00
SomberNight
0d0aa0f04a build: appimage: dl xcb-util as debian src pkg instead of upstream git 
The upstream git repo has been unreachable for at least days.
 2024-09-10 15:45:07 +00:00
SomberNight
9508f76b7f appimage build: bump python version (3.10.13->3.11.9) 2024-04-18 17:18:44 +00:00
SomberNight
5369be66da appimage build: build zbar ourselves 
to be able to package new version, which has security fixes
 2024-01-10 20:48:27 +00:00
SomberNight
615cb66692 appimage build: bump python version (3.10.11->3.10.13) 2023-12-24 05:15:39 +00:00
SomberNight
1ff4130804 contrib/docker_notes.md: add notes re debian apt mirror, and envvars 
related https://github.com/spesmilo/electrum/issues/8496
 2023-06-22 15:45:30 +00:00
SomberNight
16c3c3c4c0 appimage build: bump python version (3.9.15->3.10.11) 2023-06-02 17:17:36 +00:00
SomberNight
cede16a522 libsecp256k1: update hardcoded .so lib name in binaries 
follow-up 2a2b683d23

TODO: maybe we should drop the version number in the lib name we bundle...
 2023-04-21 14:42:51 +00:00
SomberNight
16fcfe34a7 build: rm "contrib" from debian apt sources lists 2023-04-18 16:09:50 +00:00
SomberNight
f25e384654 build: fail if not inside git clone 
related: https://github.com/spesmilo/electrum/issues/8284
 2023-03-28 22:35:19 +00:00
SomberNight
c9b6a6c01e build: fix repro builds where host userid != 1000 
- repro builds to use fixed uid=1000 inside the container
  - in case the file permissions leak into the binaries, they are still reproducible
  - chown 1000:1000 fresh_clone
- repro builds to create fresh_clone dir outside git clone
  - otherwise the local dev build would still interact with the fresh_clone dir
    - due to e.g. recursive "find -exec touch",
    - and even the "docker build" cmd itself would try to stat/read it
      - see https://github.com/docker/for-linux/issues/380
  - and "rm -rf fresh_clone" needs sudo if the host uid is not 1000
  - this way the local dev build does not need sudo

to recap:
- local dev builds use the host userid inside the container, directly operate on the project dir
  - does not need sudo
- repro builds create a fresh git clone, chown it to 1000, and use userid=1000 inside the container
  - if the host userid is 1000, does not need sudo
  - otherwise, needs sudo

closes https://github.com/spesmilo/electrum/issues/8261
 2023-03-20 23:32:59 +00:00
SomberNight
6e472efd5f build: follow-up prev: only use host userid for local dev builds 
reproducibility probably needs a hardcoded userid

Also, move the UID arg later in the dockerfiles, for better caching.
(if local dev build and repro build set different UIDs, the build caches
will diverge at that step)
 2023-03-20 02:06:54 +00:00
Sander van Grieken
ab073827cf build: use uid of user building the build containers 2023-03-20 02:06:50 +00:00
SomberNight
08ae0a73b2 build: add separate .dockerignore files 
The .dockerignore symlink in the project root dir is only picked up by the android build.
The android build has the project root as its build context for "docker build" --
the other builds have their own subdirectories as build context, e.g. contrib/build-linux/appimage.
 2023-03-20 02:05:40 +00:00
SomberNight
d8abab34d8 build: rm "non-free" from debian apt sources lists 
was not needed, and better to avoid :)
 2023-03-18 06:22:38 +00:00
Sander van Grieken
65abb90049 build: update build container base image versions, update apt sources to current 2023-02-27 11:16:04 +01:00
ghost43
0eea47c78d libsecp256k1: update hardcoded .so lib name in binaries (#8186) 
follow-up 7d83335e34
 2023-02-04 01:36:19 +00:00
SomberNight
65ae281180 build: don't hardcode num worker threads "make -j4" 2022-12-03 22:50:18 +00:00
Sander van Grieken
c0772019f1 rerun freeze_container_distro.sh and update docker base images for debian buster and bullseye 2022-11-06 10:42:39 +01:00
SomberNight
1e404f4e30 binaries: update python and openssl 
note: python 3.9.x is now in source-only mode, so could not update to latest...
it is time to investigate upgrading to python 3.10 in the win and mac binaries
 2022-11-05 14:41:23 +00:00
SomberNight
6f311c8930 appimage build: better caching, both locally and on CI 2022-11-04 15:49:16 +00:00
SomberNight
34488478e6 appimage build: build and include libxcb-util1 
fixes https://github.com/spesmilo/electrum/issues/8011

> Upgraded from Appimage 4.1.5 to 4.3.2 on MX-Linux 19.
> The new version fails to start with:
>
> ```
> $ electrum
> qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
> This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may > fix this problem.
>
> Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.
>
> Aborted (core dumped)
> ```

debian 10 only distributes libxcb-util0.
debian 11 distributes libxcb-util1.
our base image atm is debian 10, so we build the package from source.
 2022-10-11 18:24:02 +00:00
SomberNight
b5900eae98 contrib: reformat most shell scripts 
Mostly just indentations.
For consistency, to conform to .editorconfig.
 2022-09-27 14:55:22 +00:00
SomberNight
0abecffe6d mac build: force using source dist for most of our python dependencies 
We compile from tar.gz, instead of using pre-built binary wheels from PyPI.
(or if the dep is pure-python, use tar.gz instead of "source-only" wheel)

-----
Some unorganised things below for future reference.

```
$ dsymutil -dump-debug-map dist1/hid.cpython-39-darwin.so
warning: (x86_64) /private/var/folders/1n/zc14m3td0rg4nt0ftklmm7z00000gn/T/pip-install-bm88zvc1/hidapi_cd307bc31ab34252b77d11d6d7212fc5/build/temp.macosx-10.9-x86_64-3.9/hid.o unable to open object file: No such file or directory
warning: (x86_64) /private/var/folders/1n/zc14m3td0rg4nt0ftklmm7z00000gn/T/pip-install-bm88zvc1/hidapi_cd307bc31ab34252b77d11d6d7212fc5/build/temp.macosx-10.9-x86_64-3.9/hidapi/mac/hid.o unable to open object file: No such file or directory
---
triple:          'x86_64-apple-darwin'
binary-path:     'dist1/hid.cpython-39-darwin.so'
...
```

```
$ nm -pa dist1/hid.cpython-39-darwin.so
```

- https://stackoverflow.com/questions/10044697/where-how-does-apples-gcc-store-dwarf-inside-an-executable
- https://github.com/pypa/pip/issues/6505
- https://github.com/pypa/pip/issues/7808#issuecomment-770275723
- https://github.com/NixOS/nixpkgs/pull/91272
- https://github.com/cython/cython/pull/1576
- 9d2ba1611b/Cython/Compiler/ModuleNode.py (L913)
 2022-09-26 12:16:02 +00:00
SomberNight
4f574afe5a Merge remote-tracking branch 'remotes/spesmilo/appimage-debian-base' 
Change Docker base images from Ubuntu to Debian, and use `snapshot.debian.org` as apt source list.
Ubuntu occasionally removes version-pinned packages from apt (see #7484), which
- breaks historical reproducible builds
- introduces maintenance burden as we have to update the version pins
Hopefully this change fixes both issues.

merges https://github.com/spesmilo/electrum/pull/7926
closes https://github.com/spesmilo/electrum/issues/7484
 2022-08-22 13:10:25 +00:00
SomberNight
019d213325 Revert "mac build: force using source dist for most of our python dependencies" 
This reverts commit 0c2a885c66.

Reverting for now due to reproducibility issues:
```
error: /Users/vagrant/Downloads/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate: can't open file: /tmp/electrum_compare_dmg/signed_app/Electrum.app/Contents/MacOS/libffi.8.dylib (No such file or directory)
```
`Electrum.app/Contents/MacOS/libffi.8.dylib` is included when building on one machine, but missing on the other...
Will have to investigate later.

related https://github.com/spesmilo/electrum/pull/7918
 2022-08-17 17:07:22 +00:00
SomberNight
c2b5e3ec15 build: use build-locale.sh in all build scripts 2022-08-16 19:06:47 +00:00
Sander van Grieken
f63df43614 wine: rename freeze_appimage_distro.sh to freeze_containers_distro.sh and 
adapt build-wine container to Debian+snapshots base
 2022-08-11 12:36:48 +02:00
Sander van Grieken
a1975e2449 appimage: add freeze_appimage_distro.sh and adapt Dockerfile to use debian buster base 2022-08-11 12:15:22 +02:00
SomberNight
7e2fab809d appimage build: better parameterise python version 2022-08-10 21:03:55 +02:00
SomberNight
0c2a885c66 mac build: force using source dist for most of our python dependencies 
We compile from tar.gz, instead of using pre-built binary wheels from PyPI.
(or if the dep is pure-python, use tar.gz instead of "source-only" wheel)

-----
Some unorganised things below for future reference.

```
$ dsymutil -dump-debug-map dist1/hid.cpython-39-darwin.so
warning: (x86_64) /private/var/folders/1n/zc14m3td0rg4nt0ftklmm7z00000gn/T/pip-install-bm88zvc1/hidapi_cd307bc31ab34252b77d11d6d7212fc5/build/temp.macosx-10.9-x86_64-3.9/hid.o unable to open object file: No such file or directory
warning: (x86_64) /private/var/folders/1n/zc14m3td0rg4nt0ftklmm7z00000gn/T/pip-install-bm88zvc1/hidapi_cd307bc31ab34252b77d11d6d7212fc5/build/temp.macosx-10.9-x86_64-3.9/hidapi/mac/hid.o unable to open object file: No such file or directory
---
triple:          'x86_64-apple-darwin'
binary-path:     'dist1/hid.cpython-39-darwin.so'
...
```

```
$ nm -pa dist1/hid.cpython-39-darwin.so
```

- https://stackoverflow.com/questions/10044697/where-how-does-apples-gcc-store-dwarf-inside-an-executable
- https://github.com/pypa/pip/issues/6505
- https://github.com/pypa/pip/issues/7808#issuecomment-770275723
- https://github.com/NixOS/nixpkgs/pull/91272
- https://github.com/cython/cython/pull/1576
- 9d2ba1611b/Cython/Compiler/ModuleNode.py (L913)
 2022-08-06 07:52:03 +02:00
gruve-p
bc5b5eb0b1 appimage: Bump git 
https://launchpad.net/ubuntu/+source/git/1:2.17.1-1ubuntu0.12
 2022-07-20 15:45:01 +02:00
gruve-p
e50db4bf2b appimage: update libssl-dev libssl1.1 openssl 
Windows image: update gnupg2 dirmngr
 2022-07-09 14:41:41 +02:00
gruve-p
29ab90e1b1 appimage: update libssl-dev libssl1.1 openssl (#7869) 2022-06-24 05:05:09 +00:00
SomberNight
1fedb23946 appimage: update package (libdbus-1-3) in dockerfile 2022-05-11 18:08:36 +02:00
gruve-p
eb66ed8e4b appimage: update libssl-dev libssl1.0.0 openssl libsqlite3-dev (#7798) 2022-05-05 14:57:35 +00:00
ghost43
c45170acfb Merge pull request #7785 from SomberNight/202204_bump_appimage 
build appimage: bump pkg2appimage version
 2022-04-26 18:29:10 +00:00