Extend "create new user to avoid using root" copypasta to support being run as root
(ARG UID=0).
When running build.sh directly on a cirrus CI runner, we are running as root.
from https://stackoverflow.com/a/69088164
I am trying to run the build.sh scripts directly on a cirrus CI runner, and "docker run -it" is erroring due to lack of a tty.
Updates the appimage build scripts to use the newer
https://github.com/AppImage/appimagetool tool to bundle the appimage
instead of the discontinued https://github.com/AppImage/AppImageKit.
To prevent the new appimagetool from downloading a random "latest"
appimage runtime (`type2-runtime`) binary this PR also adds
functionality to clone and build
https://github.com/AppImage/type2-runtime from source. This is done
using the build scripts provided in the `type2-runtime` repository,
however the Dockerfile they use for building is replaced by a copy with
pinned package versions to prevent issues with reproducibility.
This should fix the issue of missing libfuse2 which users of the appimage
have on "modern" distributions.
The new `type2-runtime` is statically linked and includes the required
dependencies now instead of relying on the host to provide it.
note: 3.12 just transitioned to security-only status,
so can't bump win/mac binaries without switching to 3.13
(as we don't compile our own cpython for those)
- this merges `contrib/deterministic-build/locale` and `electrum/locale`
- it is now once again possible have translations when running from a local git clone
- which was already possible in the past before crowdin removed their unauthenticated APIs
- see https://github.com/spesmilo/electrum/issues/9531
- however, the translations available are the often-old frozen strings from electrum-locale
- while previously one could just download the latest strings from crowdin
- debian 11 only has python 3.9, deb12 has py3.11
- pip install pip is no longer needed, atm apt has new enough pip
- and on deb12, started getting "error: externally-managed-environment"
- faketime does not seem to work properly on debian 12
(getting reproducibility issues for the tarball)
- so instead we untar, fix the timestamps manually, and re-tar
Haven't checked if electrum-ecc compiles libsecp reproducibly.
For now let's just keep the old flow.
(but if we spent time on making that compilation reproducible,
the appimage and the macos builds could use it directly)
- repro builds to use fixed uid=1000 inside the container
- in case the file permissions leak into the binaries, they are still reproducible
- chown 1000:1000 fresh_clone
- repro builds to create fresh_clone dir outside git clone
- otherwise the local dev build would still interact with the fresh_clone dir
- due to e.g. recursive "find -exec touch",
- and even the "docker build" cmd itself would try to stat/read it
- see https://github.com/docker/for-linux/issues/380
- and "rm -rf fresh_clone" needs sudo if the host uid is not 1000
- this way the local dev build does not need sudo
to recap:
- local dev builds use the host userid inside the container, directly operate on the project dir
- does not need sudo
- repro builds create a fresh git clone, chown it to 1000, and use userid=1000 inside the container
- if the host userid is 1000, does not need sudo
- otherwise, needs sudo
closes https://github.com/spesmilo/electrum/issues/8261
reproducibility probably needs a hardcoded userid
Also, move the UID arg later in the dockerfiles, for better caching.
(if local dev build and repro build set different UIDs, the build caches
will diverge at that step)
The .dockerignore symlink in the project root dir is only picked up by the android build.
The android build has the project root as its build context for "docker build" --
the other builds have their own subdirectories as build context, e.g. contrib/build-linux/appimage.
note: python 3.9.x is now in source-only mode, so could not update to latest...
it is time to investigate upgrading to python 3.10 in the win and mac binaries
fixes https://github.com/spesmilo/electrum/issues/8011
> Upgraded from Appimage 4.1.5 to 4.3.2 on MX-Linux 19.
> The new version fails to start with:
>
> ```
> $ electrum
> qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
> This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may > fix this problem.
>
> Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.
>
> Aborted (core dumped)
> ```
debian 10 only distributes libxcb-util0.
debian 11 distributes libxcb-util1.
our base image atm is debian 10, so we build the package from source.
Change Docker base images from Ubuntu to Debian, and use `snapshot.debian.org` as apt source list.
Ubuntu occasionally removes version-pinned packages from apt (see #7484), which
- breaks historical reproducible builds
- introduces maintenance burden as we have to update the version pins
Hopefully this change fixes both issues.
merges https://github.com/spesmilo/electrum/pull/7926
closes https://github.com/spesmilo/electrum/issues/7484
