- repro builds to use fixed uid=1000 inside the container
- in case the file permissions leak into the binaries, they are still reproducible
- chown 1000:1000 fresh_clone
- repro builds to create fresh_clone dir outside git clone
- otherwise the local dev build would still interact with the fresh_clone dir
- due to e.g. recursive "find -exec touch",
- and even the "docker build" cmd itself would try to stat/read it
- see https://github.com/docker/for-linux/issues/380
- and "rm -rf fresh_clone" needs sudo if the host uid is not 1000
- this way the local dev build does not need sudo
to recap:
- local dev builds use the host userid inside the container, directly operate on the project dir
- does not need sudo
- repro builds create a fresh git clone, chown it to 1000, and use userid=1000 inside the container
- if the host userid is 1000, does not need sudo
- otherwise, needs sudo
closes https://github.com/spesmilo/electrum/issues/8261
reproducibility probably needs a hardcoded userid
Also, move the UID arg later in the dockerfiles, for better caching.
(if local dev build and repro build set different UIDs, the build caches
will diverge at that step)
The .dockerignore symlink in the project root dir is only picked up by the android build.
The android build has the project root as its build context for "docker build" --
the other builds have their own subdirectories as build context, e.g. contrib/build-linux/appimage.
note: python 3.9.x is now in source-only mode, so could not update to latest...
it is time to investigate upgrading to python 3.10 in the win and mac binaries
fixes https://github.com/spesmilo/electrum/issues/8011
> Upgraded from Appimage 4.1.5 to 4.3.2 on MX-Linux 19.
> The new version fails to start with:
>
> ```
> $ electrum
> qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
> This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may > fix this problem.
>
> Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.
>
> Aborted (core dumped)
> ```
debian 10 only distributes libxcb-util0.
debian 11 distributes libxcb-util1.
our base image atm is debian 10, so we build the package from source.
Change Docker base images from Ubuntu to Debian, and use `snapshot.debian.org` as apt source list.
Ubuntu occasionally removes version-pinned packages from apt (see #7484), which
- breaks historical reproducible builds
- introduces maintenance burden as we have to update the version pins
Hopefully this change fixes both issues.
merges https://github.com/spesmilo/electrum/pull/7926
closes https://github.com/spesmilo/electrum/issues/7484
This reverts commit 0c2a885c66.
Reverting for now due to reproducibility issues:
```
error: /Users/vagrant/Downloads/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate: can't open file: /tmp/electrum_compare_dmg/signed_app/Electrum.app/Contents/MacOS/libffi.8.dylib (No such file or directory)
```
`Electrum.app/Contents/MacOS/libffi.8.dylib` is included when building on one machine, but missing on the other...
Will have to investigate later.
related https://github.com/spesmilo/electrum/pull/7918
We now distribute two tarballs, a "normal" one (the default, recommended for users),
and a new strictly source-only one (for Linux distro packagers).
The normal tarball, in addition to including everything from
the source-only one, also includes:
- compiled (`.mo`) locale files (in addition to source `.po` locale files)
- compiled (`_pb2.py`) protobuf files (in addition to source `.proto` files)
- the `packages/` folder containing source-only pure-python runtime dependencies
merges https://github.com/spesmilo/electrum/pull/7594
with follow-ups
4.1.5->4.2.1, the appimage grew 54M->66M. This change shrinks it back to 58M.
```
$ ls -lah
total 224M
drwxrwxr-x 5 user user 4.0K Mar 27 18:18 .
drwxrwxr-x 16 user user 4.0K Mar 23 16:02 ..
-rwxrw-r-- 1 user user 48M Dec 18 2020 electrum-4.0.9-x86_64.AppImage
-rwxrw-r-- 1 user user 54M Jan 19 14:25 electrum-4.1.5-x86_64.AppImage
-rwxr-xr-x 1 user user 58M Mar 27 18:12 electrum-4.2.1-dirty-x86_64.AppImage
-rwxrw-r-- 1 user user 66M Mar 27 15:00 electrum-4.2.1-x86_64.AppImage
```
I've used the great `ncdu` tool to investigate file sizes.
```
$ du squashfs-root-415/usr/lib/python3.7/ --max-depth=1 | sort -nr | head -n8
154608 squashfs-root-415/usr/lib/python3.7/
138864 squashfs-root-415/usr/lib/python3.7/site-packages
4720 squashfs-root-415/usr/lib/python3.7/lib-dynload
1744 squashfs-root-415/usr/lib/python3.7/encodings
664 squashfs-root-415/usr/lib/python3.7/pydoc_data
460 squashfs-root-415/usr/lib/python3.7/distutils
460 squashfs-root-415/usr/lib/python3.7/asyncio
436 squashfs-root-415/usr/lib/python3.7/email
$ du squashfs-root-421/usr/lib/python3.9/ --max-depth=1 | sort -nr | head -n8
194088 squashfs-root-421/usr/lib/python3.9/
143512 squashfs-root-421/usr/lib/python3.9/site-packages
33824 squashfs-root-421/usr/lib/python3.9/config-3.9-x86_64-linux-gnu
5244 squashfs-root-421/usr/lib/python3.9/lib-dynload
1720 squashfs-root-421/usr/lib/python3.9/encodings
696 squashfs-root-421/usr/lib/python3.9/pydoc_data
520 squashfs-root-421/usr/lib/python3.9/asyncio
464 squashfs-root-421/usr/lib/python3.9/distutils
```
We should delete `usr/lib/python3.9/config-3.9-x86_64-linux-gnu/` (which is 33M unpacked)
With py3.7 (electrum 4.1.5), this folder was named `config-3.7m-x86_64-linux-gnu`,
presumably because the default config to compile py3.7 was `--with-pymalloc`,
but maybe it is not for py3.9... ? not sure. (see https://peps.python.org/pep-3149/ )
This is bumping the python versions bundled inside our binaries.
For macOS and AppImage, from 3.9.10 to 3.9.11.
For Android, from 3.8.12 to 3.8.13.
Windows is left untouched as I am having issues with the wine build when using 3.9.11.
(see https://github.com/spesmilo/electrum/pull/7721#issuecomment-1071901116 )
