mirror
/
Signal-Android
kopia lustrzana https://github.com/ryukoposting/Signal-Android
1
0
Forkuj 0
Kod Zgłoszenia Packages Projekty Wydania Wiki Aktywność

Improve the reproducible build process. 

* Moved stuff into it's own `reproducible-builds` directory.
* Improved reproducible build by using a debian snapshot and more clearly listing dependencies.
* Removed signing from assembleReelase.
* Updated README.
fork-5.53.8
Greyson Parrelli 2020-10-12 19:32:58 -04:00
rodzic de7f103130
commit 83e63ff854
10 zmienionych plików z 116 dodań i 132 usunięć
Pokaż statystyki Ściągnij plik aktualizacji Ściągnij plik porównania Expand all files Collapse all files

1
.gitignore vendored
Wyświetl plik

@ -23,6 +23,5 @@ ffpr
test/androidTestEspresso/res/values/arrays.xml
obj/
jni/libspeex/.deps/
*.sh
pkcs11.password
dev.keystore

25
Dockerfile
Wyświetl plik

@ -1,25 +0,0 @@
FROM ubuntu:17.10
RUN dpkg --add-architecture i386 && \
apt-get update -y && \
apt-get install -y software-properties-common && \
apt-get update -y && \
apt-get install -y libc6:i386=2.26-0ubuntu2.1 libncurses5:i386=6.0+20160625-1ubuntu1 libstdc++6:i386=7.2.0-8ubuntu3.2 lib32z1=1:1.2.11.dfsg-0ubuntu2 wget openjdk-8-jdk=8u171-b11-0ubuntu0.17.10.1 git unzip opensc pcscd && \
rm -rf /var/lib/apt/lists/* && \
apt-get autoremove -y && \
apt-get clean
ENV ANDROID_SDK_FILENAME android-sdk_r24.4.1-linux.tgz
ENV ANDROID_SDK_URL https://dl.google.com/android/${ANDROID_SDK_FILENAME}
ENV ANDROID_API_LEVELS android-28
ENV ANDROID_BUILD_TOOLS_VERSION 28.0.3
ENV ANDROID_HOME /usr/local/android-sdk-linux
ENV PATH ${PATH}:${ANDROID_HOME}/tools:${ANDROID_HOME}/platform-tools
RUN cd /usr/local/ && \
wget -q ${ANDROID_SDK_URL} && \
tar -xzf ${ANDROID_SDK_FILENAME} && \
rm ${ANDROID_SDK_FILENAME}
RUN echo y | android update sdk --no-ui -a --filter ${ANDROID_API_LEVELS}
RUN echo y | android update sdk --no-ui -a --filter extra-android-m2repository,extra-android-support,extra-google-google_play_services,extra-google-m2repository
RUN echo y | android update sdk --no-ui -a --filter tools,platform-tools,build-tools-${ANDROID_BUILD_TOOLS_VERSION}
RUN rm -rf ${ANDROID_HOME}/tools && unzip ${ANDROID_HOME}/temp/*.zip -d ${ANDROID_HOME}

10
app/build.gradle
Wyświetl plik

@ -454,16 +454,6 @@ task signProductionWebsiteRelease {
}
}
tasks.whenTaskAdded { task ->
if (task.name.equals("assemblePlayRelease")) {
task.finalizedBy signProductionPlayRelease
}
if (task.name.equals("assembleWebsiteRelease")) {
task.finalizedBy signProductionWebsiteRelease
}
}
def getLastCommitTimestamp() {
new ByteArrayOutputStream().withStream { os ->
def result = exec {

28
reproducible-builds/Dockerfile 100644
Wyświetl plik

@ -0,0 +1,28 @@
FROM debian:stretch
COPY docker/ docker/
COPY docker/apt.conf docker/sources.list /etc/apt/
RUN dpkg --add-architecture i386
RUN apt-get update -y && apt-get install -y apt-utils
RUN apt-get update -y && apt-get install -y $(cat docker/dependencies.txt)
RUN docker/print-versions.sh docker/dependencies.txt
ENV ANDROID_SDK_FILENAME android-sdk_r24.4.1-linux.tgz
ENV ANDROID_API_LEVELS android-28
ENV ANDROID_BUILD_TOOLS_VERSION 28.0.3
ENV ANDROID_HOME /usr/local/android-sdk-linux
ENV PATH ${PATH}:${ANDROID_HOME}/tools:${ANDROID_HOME}/platform-tools
RUN cd /usr/local/ && \
wget -q "https://dl.google.com/android/${ANDROID_SDK_FILENAME}" && \
tar -xzf ${ANDROID_SDK_FILENAME} && \
rm ${ANDROID_SDK_FILENAME}
RUN echo y | android update sdk --no-ui -a --filter ${ANDROID_API_LEVELS}
RUN echo y | android update sdk --no-ui -a --filter extra-android-m2repository,extra-android-support,extra-google-google_play_services,extra-google-m2repository
RUN echo y | android update sdk --no-ui -a --filter tools,platform-tools,build-tools-${ANDROID_BUILD_TOOLS_VERSION}
RUN rm -rf ${ANDROID_HOME}/tools

160
ReproducibleBuilds.md → reproducible-builds/README.md
Wyświetl plik

@ -3,30 +3,33 @@
## TL;DR
You can just use these [instructions](https://signal.org/blog/reproducible-android/) from the official announcement at Open Whisper Systems's blog:
```
```bash
# Clone the Signal Android source repository
$ git clone https://github.com/signalapp/Signal-Android.git && cd Signal-Android
git clone https://github.com/signalapp/Signal-Android.git && cd Signal-Android
# Check out the release tag for the version you'd like to compare
$ git checkout v[the version number]
git checkout v[the version number]
# Build the Docker image
cd reproducible-builds
docker build -t signal-android .
# Go back up to the root of the project
cd ..
# Build using the Docker environment
$ docker run --rm -v $(pwd):/project -w /project whispersystems/signal-android:1.3 ./gradlew clean assembleRelease
docker run --rm -v $(pwd):/project -w /project signal-android ./gradlew clean assembleRelease
# Verify the APKs
$ python3 apkdiff/apkdiff.py build/outputs/apks/project-release-unsigned.apk path/to/SignalFromPlay.apk
python3 apkdiff/apkdiff.py build/outputs/apks/project-release-unsigned.apk path/to/SignalFromPlay.apk
```
Note that the instructions above use a pre-built Signal Docker image from [Docker Hub](https://hub.docker.com/u/whispersystems/). If you wish to compile the image yourself, continue reading the longer version below.
***
## Introduction
Since version 3.15.0 Signal for Android has supported reproducible builds. This is achieved by replicating the build environment as a Docker image. You'll need to build the image, run a container instance of it, compile Signal inside the container and finally compare the resulted APK to the APK that is distributed in the Google Play Store.
Since version 3.15.0 Signal for Android has supported reproducible builds. The instructions were then updated for version 5.0.0. This is achieved by replicating the build environment as a Docker image. You'll need to build the image, run a container instance of it, compile Signal inside the container and finally compare the resulted APK to the APK that is distributed in the Google Play Store.
The command line parts in this guide are written for Linux but with some little modifications you can adapt them to macOS (OS X) and Windows. In the following sections we will use `3.15.2` as an example Signal version. You'll just need to replace all occurrences of `3.15.2` with the version number you are about to verify.
@ -34,24 +37,19 @@ The command line parts in this guide are written for Linux but with some little
## Setting up directories
First let's create a new directory for this whole reproducible builds project. In your home folder (`~`), create a new directory called `reproducible-signal`.
```
user@host:$ mkdir ~/reproducible-signal
```bash
mkdir ~/reproducible-signal
```
Next create another directory inside `reproducible-signal` called `apk-from-google-play-store`.
```
user@host:$ mkdir ~/reproducible-signal/apk-from-google-play-store
```bash
mkdir ~/reproducible-signal/apk-from-google-play-store
```
We will use this directory to share APKs between the host OS and the Docker container.
Finally create one more directory inside `reproducible-signal` called `image-build-context`.
```
user@host:$ mkdir ~/reproducible-signal/image-build-context
```
This directory will be used later to build our Docker image.
## Getting the Google Play Store version of Signal APK
@ -61,41 +59,44 @@ First make sure that the Signal version you want to verify is installed on your
Plug your device to your computer and run this command to pull the APK from the device:
```
user@host:$ adb pull $(adb shell pm path org.thoughtcrime.securesms | grep /base.apk | awk -F':' '{print $2}') ~/reproducible-signal/apk-from-google-play-store/Signal-$(adb shell dumpsys package org.thoughtcrime.securesms | grep versionName | awk -F'=' '{print $2}').apk
```bash
adb pull $(adb shell pm path org.thoughtcrime.securesms | grep /base.apk | awk -F':' '{print $2}') ~/reproducible-signal/apk-from-google-play-store/Signal-$(adb shell dumpsys package org.thoughtcrime.securesms | grep versionName | awk -F'=' '{print $2}').apk
```
This will pull a file into `~/reproducible-signal/apk-from-google-play-store/` with the name `Signal-<version>.apk`
Alternatively, you can do this step-by-step:
```bash
adb shell pm path org.thoughtcrime.securesms
```
user@host:$ adb shell pm path org.thoughtcrime.securesms
```
This will output something like:
```
```bash
package:/data/app/org.thoughtcrime.securesms-aWRzcGlzcG9wZA==/base.apk
```
The output will tell you where the Signal APK is located in your device. (In this example the path is `/data/app/org.thoughtcrime.securesms-aWRzcGlzcG9wZA==/base.apk`)
Now using this information, pull the APK from your device to the `reproducible-signal/apk-from-google-play-store` directory you created before:
```
user@host:$ adb pull \
/data/app/org.thoughtcrime.securesms-aWRzcGlzcG9wZA==/base.apk \
~/reproducible-signal/apk-from-google-play-store/Signal-3.15.2.apk
```bash
adb pull \
/data/app/org.thoughtcrime.securesms-aWRzcGlzcG9wZA==/base.apk \
~/reproducible-signal/apk-from-google-play-store/Signal-3.15.2.apk
```
We will use this APK in the final part when we compare it with the self-built APK from GitHub.
## Identifying the ABI
Since v4.37.0, the APKs have been split by ABI, the CPU architecture of the target device. Google play will serve the correct one to you for your device.
Since v4.37.0, the APKs have been split by ABI, the CPU architecture of the target device. Google Play will serve the correct one to you for your device.
To identify which ABIs the google play APK supports, we can look inside the APK, which is just a zip file:
```
user@host:$ unzip -l ~/reproducible-signal/apk-from-google-play-store/Signal-*.apk | grep lib/
```bash
unzip -l ~/reproducible-signal/apk-from-google-play-store/Signal-*.apk | grep lib/
```
Example:
@ -135,41 +136,28 @@ In the following sections we will assume that your Docker installation works wit
## Building a Docker image for Signal
#### Grabbing the `Dockerfile`
First you will need the `Dockerfile` for Signal Android. It comes bundled with Signal's source code. The `Dockerfile` contains instructions on how to automatically build a Docker image for Signal. You just need to run it and it builds itself.
Download the `Dockerfile` to the `image-build-context` directory.
First, you need to pull down the source for Signal-Android, which contains everything you need to build the project, including the `Dockerfile`. The `Dockerfile` contains instructions on how to automatically build a Docker image for Signal. It's located in the `reproducible-builds` directory of the repository. To get it, clone the project:
```
user@host:$ wget -O ~/reproducible-signal/image-build-context/Dockerfile_v3.15.2 \
https://raw.githubusercontent.com/signalapp/Signal-Android/v3.15.2/Dockerfile
git clone https://github.com/signalapp/Signal-Android.git signal-source
```
Note that the `Dockerfile` is specific to the Signal version you want to compare to. Again you have to adjust the URL above to match the right version. (Though sometimes the file might not be up to date, see the [Troubleshooting section](#troubleshooting))
#### Building the image
Now we have everything we need to build the Docker image for Signal. Go to the `image-build-context` directory:
Then, checkout the specific version you're trying to build:
```
user@host:$ cd ~/reproducible-signal/image-build-context
git checkout --quiet v5.0.0
```
And list the contents.
Then, to build it, go into the `reproducible-builds` directory:
```
user@host:$ ls
```
The output should look like this:
```
Dockerfile_v3.15.2
cd ~/reproducible-signal/signal-source/reproducible-builds
```
Now in this directory build the image using `Dockerfile_v3.15.2`:
...and run the docker build command:
```
user@host:$ docker build --file Dockerfile_v3.15.2 --tag signal-android .
docker build -t signal-android .
```
(Note that there is a dot at the end of that command!)
@ -181,61 +169,39 @@ Wait a few years for the build to finish... :construction_worker:
:calendar: :sleeping:
After the build has finished, you may wish to list all your Docker images to see that it's really there:
```
user@host:$ docker images
docker images
```
Output should look something like this:
```
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
signal-android latest c6b84450b896 46 seconds ago 2.94 GB
ubuntu 14.04.3 8693db7e8a00 9 weeks ago 187.9 MB
```
## Compiling Signal inside a container
Next we will run a container of the image we just built, grab Signal's source code and compile Signal.
Next we compile Signal.
First go to the `reproducible-signal` directory:
First go to the directory where the source code is: `reproducible-signal/signal-source`:
```
user@host:$ cd ~/reproducible-signal/
cd ~/reproducible-signal/signal-source
```
To run a new ephemeral container with an interactive terminal session execute the following long command:
```
user@host:$ docker run \
--name signal \
--rm \
--interactive \
--tty \
--volume $(pwd)/apk-from-google-play-store:/signal-build/apk-from-google-play-store \
--workdir /signal-build \
signal-android
```
Now you are inside the container.
To build with the docker image you just built (`signal-android`), run:
Grab Signal's source code from GitHub and go to the repository directory:
```
root@container:# git clone https://github.com/signalapp/Signal-Android.git
root@container:# cd Signal-Android
docker run --rm -v $(pwd):/project -w /project signal-android ./gradlew clean assemblePlayRelease
```
Before you can compile, you **must** ensure that you are at the right commit. In other words you **must** checkout the version you wish to verify (here we are verifying 3.15.2):
```
root@container:# git checkout --quiet v3.15.2
```
Now you may compile the release APK by running:
```
root@container:# ./gradlew clean assemblePlayRelease --exclude-task signProductionPlayRelease
```
This will take a few minutes :sleeping:
#### Checking if the APKs match
After the build has completed successfully we can finally compare if the APKs match. For the comparison we need of course the Google Play Store version of Signal APK which you copied to the `apk-from-google-play-store` directory in the beginning of this guide. Because we used that directory as a `--volume` parameter for our container, we can see all the files in that directory within our container.
### Checking if the APKs match
So now we can compare the APKs using the `apkdiff.py` tool.
@ -247,17 +213,20 @@ See [Identifying the ABI](#identifying-the-abi) above if you don't know the ABI
Once you have determined the ABI, add an `abi` environment variable. For example, suppose we determine that `armeabi-v7a` is the ABI google play has served:
```
root@container:# export abi=armeabi-v7a
```bash
export abi=armeabi-v7a
```
And the diff script to compare:
```
root@container:# python3 apkdiff/apkdiff.py \
build/outputs/apk/play/release/*play-$abi-release-unsigned*.apk \
../apk-from-google-play-store/Signal-3.15.2.apk
And run the diff script to compare (updating the filenames for your specific version):
```bash
python3 reproducible-builds/apkdiff/apkdiff.py \
build/outputs/apk/play/release/*play-$abi-release-unsigned*.apk \
../apk-from-google-play-store/Signal-5.0.0.apk
```
Output:
```
APKs match!
```
@ -277,7 +246,6 @@ If the build environment (i.e. `Dockerfile`) has not changed, you don't need to
If you cannot get things to work, please do not open an issue or comment on an existing issue at GitHub. Instead, ask for help at https://community.signalusers.org/c/development
Some common issues why things may not work:
- some pinned packages in the `Dockerfile` are not available anymore and building of the Docker image fails
- the Android packages in the Docker image are outdated and compiling Signal fails
- you built the Docker image with a wrong version of the `Dockerfile`
- you didn't checkout the correct Signal version tag with Git before compiling

0
apkdiff/apkdiff.py → reproducible-builds/apkdiff/apkdiff.py
Wyświetl plik

6
reproducible-builds/docker/apt.conf 100644
Wyświetl plik

@ -0,0 +1,6 @@
Acquire::Check-Valid-Until "false";
Acquire::Languages "none";
Binary::apt-get::Acquire::AllowInsecureRepositories "false";
APT::Install-Recommends "false";
APT::Immediate-Configure "false";

8
reproducible-builds/docker/dependencies.txt 100644
Wyświetl plik

@ -0,0 +1,8 @@
libc6:i386=2.31-2
libncurses5:i386=6.2-1
libstdc++6:i386=10.2.0-3
lib32z1=1:1.2.11.dfsg-2
wget=1.20.3-1+b3
openjdk-8-jdk=8u265-b01-0+deb9u1
git=1:2.28.0-1
unzip=6.0-25

7
reproducible-builds/docker/print-versions.sh 100755
Wyświetl plik

@ -0,0 +1,7 @@
#!/bin/bash
while read dep; do
dep_name=$(echo $dep | cut -f1 -d '=')
version=$(dpkg -s $dep_name | grep 'Version: ' | cut -f2 -d ' ')
echo "$dep_name=$version"
done < $1

3
reproducible-builds/docker/sources.list 100644
Wyświetl plik

@ -0,0 +1,3 @@
deb http://httpredir.debian.org/debian/ stretch main
deb http://security.debian.org/ stretch/updates main
deb http://snapshot.debian.org/archive/debian/20200731T211026Z/ unstable main