1
0
Commit Graph

87 Commits

Author SHA1 Message Date
SomberNight
9d46fe775a build: try to consolidate instructions and decr codedupe in release.sh 2021-06-18 17:10:15 +02:00
SomberNight
3df1c3bbf7 build: rename some scripts
also, merge sdist/build.sh and sdist/make_tgz into sdist/make_sdist.sh
2021-06-18 16:29:03 +02:00
SomberNight
0842f4977b build: fresh_clone: turns out we can just clone from a local dir :P
this saves us some network requests, and reduces trust in github
2021-06-18 16:28:58 +02:00
ghost43
c43896fe6a Merge pull request #7263 from SomberNight/202104_android
reproducible build for Android apk
2021-06-10 13:19:41 +00:00
SomberNight
3c9346cce2 android build: attempt at reproducible builds 2021-06-09 18:34:09 +02:00
Thorsten Hempel
b990ff78a5 appimage: update libc6-dev package (#7299) 2021-06-04 08:36:23 +00:00
ThomasV
7c3f1d9e65 appimage: update libudev-dev package 2021-04-17 12:16:54 +02:00
SomberNight
4e91555d56 appimage build: bundle dependencies of Qt xcb plugin
fixes #7198

We recently bumped the bundled version of PyQt (5.14.2->5.15.4).
It seems the new version has a lot more dynamic dependencies.

From https://doc.qt.io/qt-5/linux-requirements.html :
> From Qt 5.15 onwards, Qt does require libxcb 1.11. Also, the -qt-xcb
> configure option got removed that was bundling some of the libs below.

We are using the prebuilt wheels for PyQt5 from PyPI. Presumably those
had previously been built using the -qt-xcb option.
2021-04-12 18:21:57 +02:00
SomberNight
c23fca37d4 build: clarify which locale files are included
related: 2cb4e56be3
2021-03-31 17:00:39 +02:00
Axel Gembe
6e1caf7e88 AppImage build: Include libxcb into the image
Ubuntu 20.04 needs `libxcb-xinerama.so.0`
and Debian 10 needs `libxcb-util.so.1` for the Qt `xcb` plugin.

-----

ported from 380f04a805

see https://github.com/Electron-Cash/Electron-Cash/issues/2196

-----

In particular, see this comment:

>> I confirmed that both Ubuntu 20.04 and Debian 10.8 clean install have an issue with the 4.2.4 AppImage.
>
> Any idea what broke it? I assume it used to work.

Yes, it did work in 4.2.3 which used PyQt5 5.13.2. I just had a look at the xcb plugin and it definitely has less dependencies:
```
ago@ubuntu2004vm:~/src/Electron-Cash/dist$ ldd squashfs-root/usr/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqxcb.so|grep xcb
        libX11-xcb.so.1 => /lib/x86_64-linux-gnu/libX11-xcb.so.1 (0x00007f0ec9d07000)
        libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f0ec9cdd000)
        libxcb-xkb.so.1 => /lib/x86_64-linux-gnu/libxcb-xkb.so.1 (0x00007f0ec6ead000)
```

compared to PyQt5 5.15.2 in 4.2.4:
```
ago@ubuntu2004vm:~/src/Electron-Cash/dist$ ldd squashfs-root/usr/lib/python3.8/site-packages/PyQt5/Qt/plugins/platforms/libqxcb.so|grep xcb
        libX11-xcb.so.1 => /lib/x86_64-linux-gnu/libX11-xcb.so.1 (0x00007fcb5715f000)
        libxcb-icccm.so.4 => /lib/x86_64-linux-gnu/libxcb-icccm.so.4 (0x00007fcb57158000)
        libxcb-image.so.0 => /lib/x86_64-linux-gnu/libxcb-image.so.0 (0x00007fcb56f51000)
        libxcb-shm.so.0 => /lib/x86_64-linux-gnu/libxcb-shm.so.0 (0x00007fcb56f4c000)
        libxcb-util.so.1 => /lib/x86_64-linux-gnu/libxcb-util.so.1 (0x00007fcb56d46000)
        libxcb-keysyms.so.1 => /lib/x86_64-linux-gnu/libxcb-keysyms.so.1 (0x00007fcb56d41000)
        libxcb-randr.so.0 => /lib/x86_64-linux-gnu/libxcb-randr.so.0 (0x00007fcb56d2e000)
        libxcb-render-util.so.0 => /lib/x86_64-linux-gnu/libxcb-render-util.so.0 (0x00007fcb56d27000)
        libxcb-render.so.0 => /lib/x86_64-linux-gnu/libxcb-render.so.0 (0x00007fcb56d16000)
        libxcb-shape.so.0 => /lib/x86_64-linux-gnu/libxcb-shape.so.0 (0x00007fcb56d11000)
        libxcb-sync.so.1 => /lib/x86_64-linux-gnu/libxcb-sync.so.1 (0x00007fcb56d07000)
        libxcb-xfixes.so.0 => /lib/x86_64-linux-gnu/libxcb-xfixes.so.0 (0x00007fcb56cfd000)
        libxcb-xinerama.so.0 => /lib/x86_64-linux-gnu/libxcb-xinerama.so.0 (0x00007fcb56cf8000)
        libxcb-xkb.so.1 => /lib/x86_64-linux-gnu/libxcb-xkb.so.1 (0x00007fcb56cda000)
        libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007fcb56cae000)
```
2021-03-27 21:41:09 +01:00
SomberNight
472d112433 sdist build: (minor) exclude more stuff from tar.gz, rm git clean log
The `git clean --dry-run` became redundant with "fresh clone" builds.
The new exclude in MANIFEST.in should not affect production builds (also due to fresh clone),
but they are nice when building from dirty local dir.
2021-03-27 03:39:10 +01:00
SomberNight
eaffced6dd binaries: bump python version 2021-03-13 16:52:19 +01:00
SomberNight
6e6e956de4 appimage build: adapt to new PyQt (5.15.3+) layout 2021-03-13 16:52:15 +01:00
gruve-p
abc155b688 Appimage build: update git=1:2.7.4-0ubuntu1.10 15b5d9d (#7091) 2021-03-09 23:35:02 +00:00
SomberNight
dded25f398 windows build: cache pip downloads 2021-03-07 22:12:26 +01:00
ThomasV
9b7f9219a3 fix travis builds (update openssl) 2021-02-19 15:49:37 +01:00
SomberNight
3d640dfc1f windows binaries: bump python version (3.7.9->3.8.7)
Had to also bump ubuntu version 18.04->20.04,
as was getting errors running the self-compiled pyinstaller otherwise (weird...):

```
from .utils.git import get_repo_revision
ModuleNotFoundError: No module named 'PyInstaller.utils'
```
(similar to https://github.com/pyinstaller/pyinstaller/issues/4403 )

-----

Note re appimage: when trying to compile python 3.8.x on ubuntu 16.04, I am getting:

./Modules/posixmodule.c: In function ‘os_copy_file_range_impl’:
./Modules/posixmodule.c:10351:15: error: implicit declaration of function ‘copy_file_range’ [-Werror=implicit-function-declaration]
         ret = copy_file_range(src, p_offset_src, dst, p_offset_dst, count, flags);

This is because ubuntu 16.04 has too old glibc.
2021-01-12 02:28:35 +01:00
SomberNight
19f806ddf4 build: don't allow setuptools to sneakily install build-time deps
see https://pip.pypa.io/en/stable/reference/pip_install/#controlling-setup-requires
> Setuptools offers the setup_requires setup() keyword for specifying
> dependencies that need to be present in order for the setup.py
> script to run. Internally, Setuptools uses easy_install to
> fulfill these dependencies.
> pip has no way to control how these dependencies are located.
> None of the package index options have an effect.

With these changes, we will now instead hard fail if this were to happen.

related: https://github.com/spesmilo/electrum/issues/5859#issuecomment-743621898
2020-12-12 02:52:38 +01:00
SomberNight
4ca2a5cf3e appimage build: build most of our python dependencies from source
instead of using pre-built binary wheels from PyPI
2020-12-09 16:38:03 +01:00
SomberNight
37a124fa1c appimage: update package in dockerfile 2020-12-09 15:00:42 +01:00
SomberNight
a4e342ac58 requirements: rename some files 2020-11-14 04:30:48 +01:00
Jin Eguchi
56f380a62c appimage: update openssl & libudev-dev (#6599) 2020-09-17 14:39:20 +00:00
SomberNight
9204102663 binaries: pip install build requirements first
I no longer trust pip to install packages from a requirements.txt file in the correct order.
For reproducibility, let's install pip/setuptools/wheels/cython first.

see https://github.com/pypa/pip/issues/2362#issuecomment-418423458
see #5859 and #6382
2020-09-08 16:44:35 +02:00
SomberNight
4f46741c52 binaries: bump python version (3.7.7->3.7.9) 2020-09-08 16:44:24 +02:00
SomberNight
995250948a appimage build: pin glibc version in docker image, for reproducibility
fixes #6357
2020-07-08 23:54:54 +02:00
SomberNight
3c6b049f9a appimage: update package in dockerfile 2020-06-17 00:33:36 +02:00
SomberNight
f5f3394552 git sanity: enforce "git checkout commithash" actually pulls commit
If there is a collision between a branch name and a commit hash, git
will choose the branch, even if the full 40-hex-long commit hash is
given. GitHub disallows branches/tags with such a name but git itself
does not. By adding the `^{commit}` syntax sugar after a ref name,
we can tell git that we want the commit hash to be preferred,
and hence we don't need to trust GitHub (only git).

see https://security.stackexchange.com/questions/225411/
2020-06-16 19:55:17 +02:00
SomberNight
dacc61a41d sdist build: update message about reproducibility 2020-06-13 03:12:33 +02:00
SomberNight
c5c8ea15bb sdist build: stop making .zip distributables as they are not deterministic
see https://bugs.python.org/issue40963
2020-06-12 19:48:33 +02:00
SomberNight
901a900ec5 sdist build: when building docker image, no interactive prompts!
see https://askubuntu.com/questions/909277/
2020-06-12 19:48:29 +02:00
SomberNight
a06f5da7c2 sdist build: bump base image to ubuntu 20.04 2020-06-12 19:48:26 +02:00
SomberNight
891390f9a1 sdist build: umask should be specified for git clone
(not nice to change umask of host :/)
2020-06-12 19:48:23 +02:00
SomberNight
24a007840f sdist build: use modern pip
the one in apt refused to install certain package versions (that were pinned by hash!!)
and installed different versions instead... e.g.:

Collecting wheel==0.34.2 (from -r /opt/electrum/contrib/build-linux/sdist/../../../contrib/deterministic-build/requirements.txt (line 112))
  Downloading 521c6dc7fe/wheel-0.34.2.tar.gz (58kB)
    100% |████████████████████████████████| 61kB 3.8MB/s
  Requested wheel==0.34.2 from 521c6dc7fe/wheel-0.34.2.tar.gz (sha256)=8788e9155fe14f54164c1b9eb0a319d98ef02c160725587ad60f14ddc57b6f96 (from -r /opt/electrum/contrib/build-linux/sdist/../../../contrib/deterministic-build/requirements.txt (line 112)), but installing version 0.30.0
2020-06-12 19:48:19 +02:00
SomberNight
e12bc4817a attempt at reproducible tarballs (sdist) 2020-06-12 19:48:15 +02:00
SomberNight
b0230f6a4b build: fix win/appimage binaries following jsonrpc dep-removal
follow-up #6220
2020-06-11 05:37:52 +02:00
SomberNight
2dfef9dde6 appimage: update package in dockerfile
Ubuntu no longer serves old version
2020-06-11 02:52:09 +02:00
Jin Eguchi
6339afee3b appimage: update openssl (#6186) 2020-05-29 00:30:20 +00:00
SomberNight
587f8df8ad binaries: update base docker image for wine/appimage 2020-05-14 20:24:17 +02:00
SomberNight
7143e9199f binaries: bump python version (3.7.6->3.7.7) 2020-05-14 20:24:07 +02:00
ThomasV
594f13b6f7 appimage: update libudev-dev in Dockerfile 2020-05-13 10:55:08 +02:00
Jin Eguchi
1846154ca3 build: update git in dockerfiles (#6107) 2020-04-21 22:48:01 +00:00
SomberNight
c2d6a902dd build: update some packages in dockerfiles
Ubuntu no longer serves old version
2020-04-15 18:06:59 +02:00
SomberNight
e5e512df8c appimage: update package in dockerfile
Ubuntu no longer serves old version
2020-03-10 18:20:46 +01:00
SomberNight
4cec098d2d build: create a standalone build script for libsecp256k1
heavily based on Electron-Cash/Electron-Cash@eda015908e
2020-02-11 16:48:24 +01:00
Jin Eguchi
4313bde4c2 appimage: update libudev-dev (#5936) 2020-02-07 11:41:04 +00:00
wakiyamap
1237134339 Fix travis appimage 2020-01-29 12:55:23 +09:00
SomberNight
a5cd34dc08 follow-up prev (oops, only committed part of the changes) 2020-01-22 18:26:29 +01:00
Axel Gembe
d3385e49bb Build: Install libxkbcommon-x11 in AppImage
Newer distributions do not install libxkbcommon-x11 by default
anymore and Qt depends on it.

-----

taken from ca3e4501cd
2020-01-22 18:16:53 +01:00
Axel Gembe
4406eebbfe Build: Uninstall Cython from AppImage
Cython is not needed at runtime.

-----

taken from c64910055d

related #5859
2020-01-22 12:27:17 +01:00
SomberNight
80025a3af4 requirements-hw: re-add Cython
this reverts ec496a8222
Cython must be pinned down for reproducible builds
related #5859
2020-01-22 12:08:30 +01:00