1
0
Commit Graph

831 Commits

Author SHA1 Message Date
SomberNight
c2b5e3ec15 build: use build-locale.sh in all build scripts 2022-08-16 19:06:47 +00:00
SomberNight
d0de44a794 build: also build a "source-only" linux-packager-friendly tarball
We now distribute two tarballs, a "normal" one (the default, recommended for users),
and a new strictly source-only one (for Linux distro packagers).
The normal tarball, in addition to including everything from
the source-only one, also includes:
- compiled (`.mo`) locale files (in addition to source `.po` locale files)
- compiled (`_pb2.py`) protobuf files (in addition to source `.proto` files)
- the `packages/` folder containing source-only pure-python runtime dependencies

merges https://github.com/spesmilo/electrum/pull/7594
with follow-ups
2022-08-16 17:28:41 +00:00
SomberNight
7d36f2ba3a build: follow-up source-only tarballs 2022-08-16 17:23:08 +00:00
yanmaani
9a37184a57 build: create source-only tarball in release.sh 2022-08-15 12:00:00 +00:00
SomberNight
7e2fab809d appimage build: better parameterise python version 2022-08-10 21:03:55 +02:00
SomberNight
0c2a885c66 mac build: force using source dist for most of our python dependencies
We compile from tar.gz, instead of using pre-built binary wheels from PyPI.
(or if the dep is pure-python, use tar.gz instead of "source-only" wheel)

-----
Some unorganised things below for future reference.

```
$ dsymutil -dump-debug-map dist1/hid.cpython-39-darwin.so
warning: (x86_64) /private/var/folders/1n/zc14m3td0rg4nt0ftklmm7z00000gn/T/pip-install-bm88zvc1/hidapi_cd307bc31ab34252b77d11d6d7212fc5/build/temp.macosx-10.9-x86_64-3.9/hid.o unable to open object file: No such file or directory
warning: (x86_64) /private/var/folders/1n/zc14m3td0rg4nt0ftklmm7z00000gn/T/pip-install-bm88zvc1/hidapi_cd307bc31ab34252b77d11d6d7212fc5/build/temp.macosx-10.9-x86_64-3.9/hidapi/mac/hid.o unable to open object file: No such file or directory
---
triple:          'x86_64-apple-darwin'
binary-path:     'dist1/hid.cpython-39-darwin.so'
...
```

```
$ nm -pa dist1/hid.cpython-39-darwin.so
```

- https://stackoverflow.com/questions/10044697/where-how-does-apples-gcc-store-dwarf-inside-an-executable
- https://github.com/pypa/pip/issues/6505
- https://github.com/pypa/pip/issues/7808#issuecomment-770275723
- https://github.com/NixOS/nixpkgs/pull/91272
- https://github.com/cython/cython/pull/1576
- 9d2ba1611b/Cython/Compiler/ModuleNode.py (L913)
2022-08-06 07:52:03 +02:00
SomberNight
87904e2c7e win build: force using source dist for some of our python dependencies
Instead of using pre-built binary wheels from PyPI.
We should tighten this more (re other requirements-*.txt files),
but there is no C compiler available inside the wine environment atm...
2022-08-06 06:43:31 +02:00
SomberNight
caa6983f81 contrib/{add_cosigner,make_download}: cd to project root
to make it agnostic of cwd
2022-08-06 01:23:43 +02:00
SomberNight
bf4482ce1e (trivial) replace URL in comment to include git hash instead of branch 2022-08-05 22:57:24 +02:00
SomberNight
787dd69282 win build: clear $CONTRIB_WINE/dist between builds 2022-08-05 17:53:20 +02:00
SomberNight
150ebe116a mac build: maybe fix reproducibility fail (_cbor.cpython-39-darwin.so)
vagrants-iMac:electrum vagrant$ ./contrib/osx/compare_dmg dist/electrum-4.3.0-ghost43.dmg /Users/vagrant/Desktop/electrum-4.3.0-thomas1.dmg
[...]
Extracting signatures from release app...
Created mac_extracted_sigs.tar.gz
Applying extracted signatures to unsigned app...
Done. .app with sigs applied is at: /tmp/electrum_compare_dmg/signed_app
++ diff -qr /tmp/electrum_compare_dmg/signed_app /tmp/electrum_compare_dmg/dmg2
+ diff='Files /tmp/electrum_compare_dmg/signed_app/Electrum.app/Contents/MacOS/cbor/_cbor.cpython-39-darwin.so and /tmp/electrum_compare_dmg/dmg2/Electrum.app/Contents/MacOS/cbor/_cbor.cpython-39-darwin.so differ'
+ diff='diff errored'
+ set +x
diff errored
DMGs do *not* match.
failure

user@user-VirtualBox:~/wspace/tmp$ vbindiff comp/signed_app/_cbor.cpython-39-darwin.so comp/dmg2/_cbor.cpython-39-darwin.so

comp/signed_app/_cbor.cpython-39-darwin.so
0000 6AC0: 00 5F 50 79 49 6E 69 74  5F 5F 63 62 6F 72 2E 6D  ._PyInit __cbor.m
0000 6AD0: 6F 64 65 66 00 5F 43 62  6F 72 4D 65 74 68 6F 64  odef._Cb orMethod
0000 6AE0: 73 00 2F 70 72 69 76 61  74 65 2F 76 61 72 2F 66  s./priva te/var/f
0000 6AF0: 6F 6C 64 65 72 73 2F 35  36 2F 64 38 36 70 35 39  olders/5 6/d86p59
0000 6B00: 37 31 31 67 7A 63 62 38  73 31 71 37 31 36 78 31  711gzcb8 s1q716x1
0000 6B10: 6C 63 30 30 30 30 67 6E  2F 54 2F 70 69 70 2D 69  lc0000gn /T/pip-i
0000 6B20: 6E 73 74 61 6C 6C 2D 36  6D 69 36 68 6C 75 65 2F  nstall-6 mi6hlue/
comp/dmg2/_cbor.cpython-39-darwin.so
0000 6AC0: 00 5F 50 79 49 6E 69 74  5F 5F 63 62 6F 72 2E 6D  ._PyInit __cbor.m
0000 6AD0: 6F 64 65 66 00 5F 43 62  6F 72 4D 65 74 68 6F 64  odef._Cb orMethod
0000 6AE0: 73 00 2F 70 72 69 76 61  74 65 2F 76 61 72 2F 66  s./priva te/var/f
0000 6AF0: 6F 6C 64 65 72 73 2F 37  68 2F 70 33 30 7A 5F 74  olders/7 h/p30z_t
0000 6B00: 79 31 35 30 31 32 70 66  5F 33 64 79 78 62 73 39  y15012pf _3dyxbs9
0000 6B10: 33 34 30 30 30 30 67 6E  2F 54 2F 70 69 70 2D 69  340000gn /T/pip-i
0000 6B20: 6E 73 74 61 6C 6C 2D 30  68 64 39 63 35 6D 65 2F  nstall-0 hd9c5me/

related: https://github.com/pypa/pip/issues/6505
2022-08-05 15:57:10 +02:00
ThomasV
4fcc5a580a prepare release 4.3.0: update version.py and locale 2022-08-05 14:05:49 +02:00
SomberNight
752b37a03b dependencies: bundle older hidapi in binaries
closes https://github.com/spesmilo/electrum/issues/7738
2022-08-04 21:47:50 +02:00
gruve-p
bc5b5eb0b1 appimage: Bump git
https://launchpad.net/ubuntu/+source/git/1:2.17.1-1ubuntu0.12
2022-07-20 15:45:01 +02:00
gruve-p
e50db4bf2b appimage: update libssl-dev libssl1.1 openssl
Windows image: update gnupg2 dirmngr
2022-07-09 14:41:41 +02:00
SomberNight
57b17d13ac android build: make building testnet apk easier
Just uncomment line 53 in make_apk, and you get a testnet apk that can co-exist with your mainnet install.
No need to do a clean rebuild of ".buildozer/" either.
2022-07-09 04:47:37 +02:00
SomberNight
e72f575eea android build: factor out package name ("org.electrum.electrum")
to make it easier to change, to make it easy to have a co-existing mainnet and testnet install
(or two mainnet installs, etc)
2022-07-09 04:47:32 +02:00
SomberNight
d79da7a248 android build: (qml) pin new transitive dependencies for reproducibility 2022-07-08 16:37:05 +02:00
SomberNight
bc88e1c328 android build: (qml) bump p4a commit
to include single new commit c6e39ae1fb
2022-07-08 16:37:01 +02:00
SomberNight
3d0baf8d00 android build: restore prev "make theming" behaviour
- note: "make theming" is kivy-specific, and not needed for the qml gui apk
- note: "make theming" does not run automatically as part of the build scripts,
        although it used to in the past. For reproducible builds, the
        "electrum/gui/kivy/theming/atlas" git submodule contains the build
        artefacts. However, the user is supposed to manually run "make theming"
        when changing the atlas/images.
2022-07-08 13:35:28 +02:00
Sander van Grieken
c656b02319 update tip pthon-for-android qt5-wip 2022-07-07 18:30:16 +02:00
Sander van Grieken
3868878be4 filter out more unneeded files for packaging 2022-07-07 18:29:01 +02:00
Sander van Grieken
758a30462e implement QR code scanning 2022-07-07 18:29:01 +02:00
Sander van Grieken
492f246b9a qml: add QR code imageprovider using qrcode/PIL
adds buildozer 'pillow' recipe to requirements

add initial PoC on qml receive tab
2022-07-07 18:28:00 +02:00
Sander van Grieken
a2fac2e3e3 buildozer: exclude env dir, don't include vs, fs extensions 2022-07-07 18:28:00 +02:00
ThomasV
08154da3b6 add command line parameters to android/build.sh, and use separate .buildozer dirs for kivy and qt5 2022-07-07 18:28:00 +02:00
ThomasV
634a647fb1 android: parameterize GUI framework 2022-07-07 18:28:00 +02:00
ThomasV
49a67a5cd2 build android apk with qt5
update requirements file for building PyQt5
2022-07-07 18:28:00 +02:00
yanmaani
04b81d6b82 contrib: add generate_payreqpb2.sh script 2022-07-04 12:00:00 +00:00
yanmaani
2da1859110 build: support OMIT_UNCLEAN_FILES in sdist/build.sh 2022-07-04 12:00:00 +00:00
yanmaani
a7e17ae4ee build: add OMIT_UNCLEAN_FILES option to make_sdist.sh 2022-07-04 12:00:00 +00:00
yanmaani
45b8f11c68 build: refactor out locale generation into build_locale.sh 2022-07-04 12:00:00 +00:00
SomberNight
0fca35fa40 android build: increase buildozer log level (except on cirrus ci) 2022-07-01 18:52:30 +02:00
SomberNight
bb0db0e3c0 build: incl "frozenlist" dep as pure-python, without C stuff
While attempting to reproducibly build the qml android apk, one of the differences
was due to the "frozenlist" dependency (pulled in by aiohttp) - the compiled C parts
were not deterministic. By setting this env var, we can opt-out [0] of all the C
accelerated parts and just use the pure-python implementation. We are already doing
the same for other aiohttp-related packages anyway.

[0]: c2794cac12/setup.py (L7)
2022-07-01 18:35:42 +02:00
Sander van Grieken
8d57129ad6 p4a: update commit ref to head of qt5-wip (fixes cpu count used for build) 2022-06-30 21:23:41 +02:00
gruve-p
29ab90e1b1 appimage: update libssl-dev libssl1.1 openssl (#7869) 2022-06-24 05:05:09 +00:00
SomberNight
941db4214c README: add Windows- & macOS-specific "how to run from source" readmes
note: unclear where to put these files... `contrib/build-wine/` and `contrib/osx/`
contain binary-building-related files. maybe we could have a `doc/` folder
if the need for more similar files arise.
atm there are these two, plus maybe `contrib/docker_notes.md`.
2022-06-09 17:40:57 +02:00
SomberNight
52b73880f9 requirements: pin protobuf "<4"
protobuf 4.x introduced breaking changes compared to 3.20.
To adapt, we would have to regenerate paymentrequest_pb2.py, using protoc>=3.19.0, however ubuntu does not have new enough protoc.
Also, unsure if newly generated paymentrequest_pb2.py would be supported by older versions of protobuf.
Best to just wait for things to settle.

```
electrum/__init__.py:20: in <module>
    from .wallet import Wallet
electrum/wallet.py:70: in <module>
    from . import transaction, bitcoin, coinchooser, paymentrequest, ecc, bip32
electrum/paymentrequest.py:37: in <module>
    from . import paymentrequest_pb2 as pb2
electrum/paymentrequest_pb2.py:36: in <module>
    _descriptor.FieldDescriptor(
.tox/py3/lib/python3.9/site-packages/google/protobuf/descriptor.py:560: in __new__
    _message.Message._CheckCalledFromGeneratedFile()
E   TypeError: Descriptors cannot not be created directly.
E   If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E   If you cannot immediately regenerate your protos, some other possible workarounds are:
E    1. Downgrade the protobuf package to 3.20.x or lower.
E    2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E
E   More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
```

ceaae1b6a3/CHANGES.txt (L47)
2022-05-27 18:38:26 +02:00
SomberNight
8e9274bde4 mac build: README: add more details about installing Xcode 2022-05-27 17:51:53 +02:00
SomberNight
ecf4e3db48 mac build: compare_dmg script: if binaries mismatch, print the diff 2022-05-27 17:48:18 +02:00
ThomasV
3ce42edbab update locale submodule 2022-05-27 12:17:41 +02:00
ThomasV
7cf495d1e0 Windows build scripts: Read codesign passphrase. Lazy check if unsigned binaries are there. 2022-05-27 11:07:32 +02:00
SomberNight
2362d6d010 follow-up "remove email plugin"
follow-up c67c4e95dc
2022-05-24 18:33:19 +02:00
Jamie C. Driver
15f108aaa5 Add support for the Blockstream Jade hww 2022-05-20 16:03:46 +01:00
SomberNight
1fedb23946 appimage: update package (libdbus-1-3) in dockerfile 2022-05-11 18:08:36 +02:00
gruve-p
eb66ed8e4b appimage: update libssl-dev libssl1.0.0 openssl libsqlite3-dev (#7798) 2022-05-05 14:57:35 +00:00
ghost43
c45170acfb Merge pull request #7785 from SomberNight/202204_bump_appimage
build appimage: bump pkg2appimage version
2022-04-26 18:29:10 +00:00
Michel van Kessel
54f5ad46b3 appimage: update package (git) in dockerfile (#7786)
https://launchpad.net/ubuntu/+source/git/1:2.17.1-1ubuntu0.11
2022-04-26 18:28:29 +00:00
SomberNight
6633ef4c35 build appimage: bump pkg2appimage version
should fix https://github.com/spesmilo/electrum/issues/7784
by including commit 1e3ecde6b9
2022-04-25 21:04:46 +02:00
gruve-p
75044f334b appimage: update git 2022-04-14 22:53:46 +02:00