AaronFiore/electrum
1
0
Fork 0
Code Activity

Merge pull request #8738 from SomberNight/202312_mac_build

Browse Source 
mac build: migrate to new VM (macOS 10.14 -> 11)
This commit is contained in:
ThomasV
2023-12-12 12:08:40 +01:00
committed by GitHub
parent dd3fc041f9 c98830d091
commit c1b359d43f
3 changed files with 56 additions and 93 deletions
Download Patch File Download Diff File Expand all files Collapse all files

88
contrib/osx/README.md
View File

@@ -11,7 +11,7 @@ This guide explains how to build Electrum binaries for macOS systems.
This needs to be done on a system running macOS or OS X. This needs to be done on a system running macOS or OS X.
The script is only tested on Intel-based Macs, and the binary built The script is only tested on Intel-based (x86_64) Macs, and the binary built
targets `x86_64` currently. targets `x86_64` currently.
Notes about compatibility with different macOS versions: Notes about compatibility with different macOS versions:
@@ -22,40 +22,50 @@ Notes about compatibility with different macOS versions:
imposes a minimum supported macOS version. imposes a minimum supported macOS version.
- If you want to build binaries that conform to the macOS "Gatekeeper", so as to - If you want to build binaries that conform to the macOS "Gatekeeper", so as to
minimise the warnings users get, the binaries need to be codesigned with a minimise the warnings users get, the binaries need to be codesigned with a
certificate issued by Apple, and starting with macOS 10.15 the binaries also certificate issued by Apple, and starting with macOS 10.15 (targets) the binaries also
need to be notarized by Apple's central server. The catch is that to be able to build need to be notarized by Apple's central server. To be able to build
binaries that Apple will notarise (due to the requirements on the binaries themselves, binaries that Apple will notarize (due to the requirements on the binaries themselves,
e.g. hardened runtime) the build machine needs at least macOS 10.14. e.g. hardened runtime) the build machine needs at least macOS 10.14.
See [#6128](https://github.com/spesmilo/electrum/issues/6128). See [#6128](https://github.com/spesmilo/electrum/issues/6128).
- There are two tools that can be used to notarize a binary, both part of Xcode:
the old `altool` and the newer `notarytool`. `altool`
[was deprecated](https://developer.apple.com/news/?id=y5mjxqmn) by Apple.
`notarytool` requires Xcode 13+, and that in turn requires macOS 11.3+.
We currently build the release binaries on macOS 10.14.6, and these seem to run on We currently build the release binaries on macOS 11.7.10, and these seem to run on
10.13 or newer. 10.14 or newer. (note: 10.13 might also work, haven't tested)
Before starting, you should install [`brew`](https://brew.sh/).
#### Notes about reproducibility #### Notes about reproducibility
- We recommend creating a VM with a macOS guest, e.g. using VirtualBox, - We recommend creating a VM with a macOS guest, e.g. using VirtualBox,
and building there. and building there.
- The guest should run macOS 10.14.6 (that specific version). - The guest should run macOS 11.7.10 (that specific version).
- The unix username should be `vagrant`, and `electrum` should be cloned directly - The unix username should be `vagrant`, and `electrum` should be cloned directly
to the user's home dir: `/Users/vagrant/electrum`. to the user's home dir: `/Users/vagrant/electrum`.
- Builders need to use the same version of Xcode; and note that - Builders need to use the same version of Xcode; and note that
full Xcode and Xcode commandline tools differ! full Xcode and Xcode commandline tools differ!
- Xcode CLI tools are sufficient for everything, except it is missing `altool`, We use the Xcode CLI tools as installed by brew. (version 13.2)
which is needed for the release-manager to notarise the `.dmg`.
- so full Xcode is needed, to have `altool`.
- however, brew now consider macOS 10.14 too old, and for some reason it
requires Xcode CLI tools. (`Error: Xcode alone is not sufficient on Mojave.`)
So, we need *both* full Xcode and Xcode CLI tools. Both with version 11.3.1. Sanity checks:
The two Xcodes should be located exactly as follows:
``` ```
$ xcode-select -p $ xcode-select -p
/Users/vagrant/Downloads/Xcode.app/Contents/Developer /Library/Developer/CommandLineTools
$ xcrun --show-sdk-path $ xcrun --show-sdk-path
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
$ pkgutil --pkg-info=com.apple.pkg.CLTools_Executables
package-id: com.apple.pkg.CLTools_Executables
version: 13.2.0.0.1.1638488800
volume: /
location: /
install-time: XXXXXXXXXX
groups: com.apple.FindSystemFiles.pkg-group
$ gcc --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 13.0.0 (clang-1300.0.29.30)
Target: x86_64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
``` ```
- Installing extraneous brew packages can result in build differences. - Installing extraneous brew packages can result in build differences.
For example, pyinstaller seems to pick up and bundle brew-installed `libffi`. For example, pyinstaller seems to pick up and bundle brew-installed `libffi`.
@@ -64,41 +74,12 @@ Before starting, you should install [`brew`](https://brew.sh/).
(or run e.g. `git clean -ffxd` to rm all local changes). (or run e.g. `git clean -ffxd` to rm all local changes).
#### 1. Get Xcode #### 1. Install brew
Notarizing the application requires full Xcode Install [`brew`](https://brew.sh/).
(not just command line tools as that is missing `altool`).
Get it from [here](https://developer.apple.com/download/more/). Let brew install the Xcode CLI tools.
Unfortunately, you need an "Apple ID" account.
(note: the last Xcode that runs on macOS 10.14.6 is Xcode 11.3.1)
Install full Xcode:
```
$ shasum -a 256 "$HOME/Downloads/Xcode_11.3.1.xip"
9a92379b90734a9068832f858d594d3c3a30a7ddc3bdb6da49c738aed9ad34b5 /Users/vagrant/Downloads/Xcode_11.3.1.xip
$ xip -x "$HOME/Downloads/Xcode_11.3.1.xip"
$ sudo xcode-select -s "$HOME/Downloads/Xcode.app/Contents/Developer/"
$ # agree with licence
$ sudo xcodebuild -license
```
(note: unsure if needed:)
```
$ # try this to install additional component:
$ sudo xcodebuild -runFirstLaunch
```
Install Xcode CLI tools:
```
$ sudo rm -rf /Library/Developer/CommandLineTools
$ shasum -a 256 "$HOME/Downloads/Command_Line_Tools_for_Xcode_11.3.1.dmg"
1c4b477285641cca5313f456b712bf726aca8db77f38793420e1d451588673f9 /Users/vagrant/Downloads/Command_Line_Tools_for_Xcode_11.3.1.dmg
$ hdiutil attach "$HOME/Downloads/Command_Line_Tools_for_Xcode_11.3.1.dmg"
$ sudo installer -package "/Volumes/Command Line Developer Tools/Command Line Tools.pkg" -target /
$ hdiutil detach "/Volumes/Command Line Developer Tools"
```
#### 2. Build Electrum #### 2. Build Electrum
@@ -107,14 +88,19 @@ $ hdiutil detach "/Volumes/Command Line Developer Tools"
This creates both a folder named Electrum.app and the .dmg file. This creates both a folder named Electrum.app and the .dmg file.
If you want the binaries codesigned for MacOS and notarised by Apple's central server, ##### 2.1. For release binaries, here be dragons
If you want the binaries codesigned for macOS and notarised by Apple's central server,
provide these env vars to the `make_osx.sh` script: provide these env vars to the `make_osx.sh` script:
CODESIGN_CERT="Developer ID Application: Electrum Technologies GmbH (L6P37P7P56)" \ CODESIGN_CERT="Developer ID Application: Electrum Technologies GmbH (L6P37P7P56)" \
APPLE_TEAM_ID="L6P37P7P56" \
APPLE_ID_USER="me@email.com" \ APPLE_ID_USER="me@email.com" \
APPLE_ID_PASSWORD="1234" \ APPLE_ID_PASSWORD="1234" \
./contrib/osx/make_osx.sh ./contrib/osx/make_osx.sh
(note: `APPLE_ID_PASSWORD` is an app-specific password, *not* the account password)
## Verifying reproducibility and comparing against official binary ## Verifying reproducibility and comparing against official binary

4
contrib/osx/make_osx.sh
View File

@@ -101,6 +101,10 @@ source $VENV_DIR/bin/activate
# see additional "strip" pass on built files later in the file. # see additional "strip" pass on built files later in the file.
export CFLAGS="-g0" export CFLAGS="-g0"
# Do not build universal binaries. The default on macos 11+ and xcode 12+ is "-arch arm64 -arch x86_64"
# but with that e.g. "hid.cpython-310-darwin.so" is not reproducible as built by clang.
export ARCHFLAGS="-arch x86_64"
info "Installing build dependencies" info "Installing build dependencies"
# note: re pip installing from PyPI, # note: re pip installing from PyPI,
# we prefer compiling C extensions ourselves, instead of using binary wheels, # we prefer compiling C extensions ourselves, instead of using binary wheels,

57
contrib/osx/notarize_app.sh
View File

@@ -7,7 +7,7 @@ if [ -z "$1" ]; then
exit 1 exit 1
fi fi
if [ -z "$APPLE_ID_USER" ] || [ -z "$APPLE_ID_PASSWORD" ]; then if [ -z "$APPLE_ID_USER" ] || [ -z "$APPLE_ID_PASSWORD" ] || [ -z "$APPLE_TEAM_ID" ]; then
echo "You need to set your Apple ID credentials with \$APPLE_ID_USER and \$APPLE_ID_PASSWORD." echo "You need to set your Apple ID credentials with \$APPLE_ID_USER and \$APPLE_ID_PASSWORD."
exit 1 exit 1
fi fi
@@ -23,12 +23,14 @@ ditto -c -k --rsrc --keepParent "$APP_BUNDLE" "${APP_BUNDLE}.zip"
# Submit for notarization # Submit for notarization
echo "Submitting $APP_BUNDLE for notarization..." echo "Submitting $APP_BUNDLE for notarization..."
RESULT=$(xcrun altool --notarize-app --type osx \ RESULT=$(xcrun notarytool submit \
--file "${APP_BUNDLE}.zip" \ --team-id "$APPLE_TEAM_ID" \
--primary-bundle-id org.electrum.electrum \ --apple-id "$APPLE_ID_USER" \
--username $APPLE_ID_USER \ --password "$APPLE_ID_PASSWORD" \
--password @env:APPLE_ID_PASSWORD \ --output-format plist \
--output-format xml --wait \
--timeout 10m \
"${APP_BUNDLE}.zip"
) )
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
@@ -37,46 +39,17 @@ if [ $? -ne 0 ]; then
exit 1 exit 1
fi fi
REQUEST_UUID=$(echo "$RESULT" | xpath \ STATUS=$(echo "$RESULT" | xpath -e \
"//key[normalize-space(text()) = 'RequestUUID']/following-sibling::string[1]/text()" 2>/dev/null "//key[normalize-space(text()) = 'status']/following-sibling::string[1]/text()" 2> /dev/null)
)
if [ -z "$REQUEST_UUID" ]; then if [ "$STATUS" = "Accepted" ]; then
echo "Submitting $APP_BUNDLE failed:" echo "Notarization of $APP_BUNDLE succeeded!"
else
echo "Notarization of $APP_BUNDLE failed:"
echo "$RESULT" echo "$RESULT"
exit 1 exit 1
fi fi
echo "$(echo "$RESULT" | xpath \
"//key[normalize-space(text()) = 'success-message']/following-sibling::string[1]/text()" 2> /dev/null)"
# Poll for notarization status
echo "Submitted notarization request $REQUEST_UUID, waiting for response..."
sleep 60
while :
do
RESULT=$(xcrun altool --notarization-info "$REQUEST_UUID" \
--username "$APPLE_ID_USER" \
--password @env:APPLE_ID_PASSWORD \
--output-format xml
)
STATUS=$(echo "$RESULT" | xpath \
"//key[normalize-space(text()) = 'Status']/following-sibling::string[1]/text()" 2>/dev/null
)
if [ "$STATUS" = "success" ]; then
echo "Notarization of $APP_BUNDLE succeeded!"
break
elif [ "$STATUS" = "in progress" ]; then
echo "Notarization in progress..."
sleep 20
else
echo "Notarization of $APP_BUNDLE failed:"
echo "$RESULT"
exit 1
fi
done
# Staple the notary ticket # Staple the notary ticket
xcrun stapler staple "$APP_BUNDLE" xcrun stapler staple "$APP_BUNDLE"