blob: 869b713796f00c032ca3ab65e05bf40fff36665c [file] [log] [blame]
Doug Zongkereef39442009-04-02 12:14:19 -07001#!/usr/bin/env python
2#
3# Copyright (C) 2008 The Android Open Source Project
4#
5# Licensed under the Apache License, Version 2.0 (the "License");
6# you may not use this file except in compliance with the License.
7# You may obtain a copy of the License at
8#
9# http://www.apache.org/licenses/LICENSE-2.0
10#
11# Unless required by applicable law or agreed to in writing, software
12# distributed under the License is distributed on an "AS IS" BASIS,
13# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14# See the License for the specific language governing permissions and
15# limitations under the License.
16
17"""
Tao Bao30df8b42018-04-23 15:32:53 -070018Given a target-files zipfile, produces an OTA package that installs that build.
19An incremental OTA is produced if -i is given, otherwise a full OTA is produced.
Doug Zongkereef39442009-04-02 12:14:19 -070020
Tao Bao30df8b42018-04-23 15:32:53 -070021Usage: ota_from_target_files [options] input_target_files output_ota_package
Doug Zongkereef39442009-04-02 12:14:19 -070022
Tao Bao30df8b42018-04-23 15:32:53 -070023Common options that apply to both of non-A/B and A/B OTAs
24
25 --downgrade
26 Intentionally generate an incremental OTA that updates from a newer build
Tao Baofaa8e0b2018-04-12 14:31:43 -070027 to an older one (e.g. downgrading from P preview back to O MR1).
28 "ota-downgrade=yes" will be set in the package metadata file. A data wipe
29 will always be enforced when using this flag, so "ota-wipe=yes" will also
30 be included in the metadata file. The update-binary in the source build
31 will be used in the OTA package, unless --binary flag is specified. Please
32 also check the comment for --override_timestamp below.
Tao Bao30df8b42018-04-23 15:32:53 -070033
34 -i (--incremental_from) <file>
35 Generate an incremental OTA using the given target-files zip as the
36 starting build.
37
38 -k (--package_key) <key>
39 Key to use to sign the package (default is the value of
40 default_system_dev_certificate from the input target-files's
Tao Bao59cf0c52019-06-25 10:04:24 -070041 META/misc_info.txt, or "build/make/target/product/security/testkey" if
42 that value is not specified).
Doug Zongkerafb32ea2011-09-22 10:28:04 -070043
44 For incremental OTAs, the default value is based on the source
45 target-file, not the target build.
Doug Zongkereef39442009-04-02 12:14:19 -070046
Tao Bao30df8b42018-04-23 15:32:53 -070047 --override_timestamp
48 Intentionally generate an incremental OTA that updates from a newer build
Tao Baofaa8e0b2018-04-12 14:31:43 -070049 to an older one (based on timestamp comparison), by setting the downgrade
50 flag in the package metadata. This differs from --downgrade flag, as we
51 don't enforce a data wipe with this flag. Because we know for sure this is
52 NOT an actual downgrade case, but two builds happen to be cut in a reverse
53 order (e.g. from two branches). A legit use case is that we cut a new
54 build C (after having A and B), but want to enfore an update path of A ->
55 C -> B. Specifying --downgrade may not help since that would enforce a
56 data wipe for C -> B update.
57
58 We used to set a fake timestamp in the package metadata for this flow. But
59 now we consolidate the two cases (i.e. an actual downgrade, or a downgrade
60 based on timestamp) with the same "ota-downgrade=yes" flag, with the
61 difference being whether "ota-wipe=yes" is set.
Doug Zongkereef39442009-04-02 12:14:19 -070062
Tao Bao30df8b42018-04-23 15:32:53 -070063 --wipe_user_data
64 Generate an OTA package that will wipe the user data partition when
65 installed.
66
Yifan Hong50e79542018-11-08 17:44:12 -080067 --retrofit_dynamic_partitions
68 Generates an OTA package that updates a device to support dynamic
69 partitions (default False). This flag is implied when generating
70 an incremental OTA where the base build does not support dynamic
71 partitions but the target build does. For A/B, when this flag is set,
72 --skip_postinstall is implied.
73
xunchangabfa2652019-02-19 16:27:10 -080074 --skip_compatibility_check
Yifan Hong9276cf02019-08-21 16:37:04 -070075 Skip checking compatibility of the input target files package.
xunchangabfa2652019-02-19 16:27:10 -080076
xunchang1cfe2512019-02-19 14:14:48 -080077 --output_metadata_path
78 Write a copy of the metadata to a separate file. Therefore, users can
79 read the post build fingerprint without extracting the OTA package.
80
Yifan Hong65afc072020-04-17 10:08:10 -070081 --force_non_ab
82 This flag can only be set on an A/B device that also supports non-A/B
83 updates. Implies --two_step.
84 If set, generate that non-A/B update package.
85 If not set, generates A/B package for A/B device and non-A/B package for
86 non-A/B device.
87
Hongguang Chen49ab1b902020-10-19 14:15:43 -070088 -o (--oem_settings) <main_file[,additional_files...]>
89 Comma separated list of files used to specify the expected OEM-specific
90 properties on the OEM partition of the intended device. Multiple expected
91 values can be used by providing multiple files. Only the first dict will
92 be used to compute fingerprint, while the rest will be used to assert
93 OEM-specific properties.
94
Tao Bao30df8b42018-04-23 15:32:53 -070095Non-A/B OTA specific options
96
97 -b (--binary) <file>
98 Use the given binary as the update-binary in the output package, instead
99 of the binary in the build's target_files. Use for development only.
100
101 --block
102 Generate a block-based OTA for non-A/B device. We have deprecated the
103 support for file-based OTA since O. Block-based OTA will be used by
104 default for all non-A/B devices. Keeping this flag here to not break
105 existing callers.
106
107 -e (--extra_script) <file>
108 Insert the contents of file at the end of the update script.
Tao Bao43078aa2015-04-21 14:32:35 -0700109
leozwangaa6c1a12015-08-14 10:57:58 -0700110 --full_bootloader
111 Similar to --full_radio. When generating an incremental OTA, always
112 include a full copy of bootloader image.
113
Tao Bao30df8b42018-04-23 15:32:53 -0700114 --full_radio
115 When generating an incremental OTA, always include a full copy of radio
116 image. This option is only meaningful when -i is specified, because a full
117 radio is always included in a full OTA if applicable.
Michael Runge63f01de2014-10-28 19:24:19 -0700118
Tao Bao30df8b42018-04-23 15:32:53 -0700119 --log_diff <file>
120 Generate a log file that shows the differences in the source and target
121 builds for an incremental package. This option is only meaningful when -i
122 is specified.
123
Tao Bao8608cde2016-02-25 19:49:55 -0800124 --oem_no_mount
Tao Bao30df8b42018-04-23 15:32:53 -0700125 For devices with OEM-specific properties but without an OEM partition, do
126 not mount the OEM partition in the updater-script. This should be very
127 rarely used, since it's expected to have a dedicated OEM partition for
128 OEM-specific properties. Only meaningful when -o is specified.
Tao Bao8608cde2016-02-25 19:49:55 -0800129
Tao Bao30df8b42018-04-23 15:32:53 -0700130 --stash_threshold <float>
131 Specify the threshold that will be used to compute the maximum allowed
132 stash size (defaults to 0.8).
Doug Zongkerdbfaae52009-04-21 17:12:54 -0700133
Tao Bao30df8b42018-04-23 15:32:53 -0700134 -t (--worker_threads) <int>
135 Specify the number of worker-threads that will be used when generating
136 patches for incremental updates (defaults to 3).
Tao Bao3e6161a2017-02-28 11:48:48 -0800137
Tao Bao30df8b42018-04-23 15:32:53 -0700138 --verify
139 Verify the checksums of the updated system and vendor (if any) partitions.
140 Non-A/B incremental OTAs only.
Doug Zongker1c390a22009-05-14 19:06:36 -0700141
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800142 -2 (--two_step)
Tao Bao30df8b42018-04-23 15:32:53 -0700143 Generate a 'two-step' OTA package, where recovery is updated first, so
144 that any changes made to the system partition are done using the new
145 recovery (new kernel, etc.).
146
147A/B OTA specific options
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800148
Tianjie Xu1b079832019-08-28 12:19:23 -0700149 --disable_fec_computation
150 Disable the on device FEC data computation for incremental updates.
151
Tao Baof7140c02018-01-30 17:09:24 -0800152 --include_secondary
153 Additionally include the payload for secondary slot images (default:
154 False). Only meaningful when generating A/B OTAs.
155
156 By default, an A/B OTA package doesn't contain the images for the
157 secondary slot (e.g. system_other.img). Specifying this flag allows
158 generating a separate payload that will install secondary slot images.
159
160 Such a package needs to be applied in a two-stage manner, with a reboot
161 in-between. During the first stage, the updater applies the primary
162 payload only. Upon finishing, it reboots the device into the newly updated
163 slot. It then continues to install the secondary payload to the inactive
164 slot, but without switching the active slot at the end (needs the matching
165 support in update_engine, i.e. SWITCH_SLOT_ON_REBOOT flag).
166
167 Due to the special install procedure, the secondary payload will be always
168 generated as a full payload.
169
Tao Baodea0f8b2016-06-20 17:55:06 -0700170 --payload_signer <signer>
171 Specify the signer when signing the payload and metadata for A/B OTAs.
172 By default (i.e. without this flag), it calls 'openssl pkeyutl' to sign
173 with the package private key. If the private key cannot be accessed
174 directly, a payload signer that knows how to do that should be specified.
175 The signer will be supplied with "-inkey <path_to_key>",
176 "-in <input_file>" and "-out <output_file>" parameters.
Baligh Uddin2abbbd02016-06-22 12:14:16 -0700177
178 --payload_signer_args <args>
179 Specify the arguments needed for payload signer.
Tao Bao15a146a2018-02-21 16:06:59 -0800180
Tianjie Xu21e6deb2019-10-07 18:01:00 -0700181 --payload_signer_maximum_signature_size <signature_size>
182 The maximum signature size (in bytes) that would be generated by the given
183 payload signer. Only meaningful when custom payload signer is specified
184 via '--payload_signer'.
185 If the signer uses a RSA key, this should be the number of bytes to
186 represent the modulus. If it uses an EC key, this is the size of a
187 DER-encoded ECDSA signature.
188
xunchang376cc7c2019-04-08 23:04:58 -0700189 --payload_signer_key_size <key_size>
Tianjie Xu21e6deb2019-10-07 18:01:00 -0700190 Deprecated. Use the '--payload_signer_maximum_signature_size' instead.
xunchang376cc7c2019-04-08 23:04:58 -0700191
Tianjied6867162020-05-10 14:30:13 -0700192 --boot_variable_file <path>
193 A file that contains the possible values of ro.boot.* properties. It's
194 used to calculate the possible runtime fingerprints when some
195 ro.product.* properties are overridden by the 'import' statement.
196 The file expects one property per line, and each line has the following
197 format: 'prop_name=value1,value2'. e.g. 'ro.boot.product.sku=std,pro'
198
Tao Bao15a146a2018-02-21 16:06:59 -0800199 --skip_postinstall
200 Skip the postinstall hooks when generating an A/B OTA package (default:
201 False). Note that this discards ALL the hooks, including non-optional
202 ones. Should only be used if caller knows it's safe to do so (e.g. all the
203 postinstall work is to dexopt apps and a data wipe will happen immediately
204 after). Only meaningful when generating A/B OTAs.
Yifan Hong38ab4d82020-06-18 15:19:56 -0700205
206 --partial "<PARTITION> [<PARTITION>[...]]"
207 Generate partial updates, overriding ab_partitions list with the given
208 list.
Hongguang Chen49ab1b902020-10-19 14:15:43 -0700209
210 --custom_image <custom_partition=custom_image>
211 Use the specified custom_image to update custom_partition when generating
212 an A/B OTA package. e.g. "--custom_image oem=oem.img --custom_image
213 cus=cus_test.img"
Doug Zongkereef39442009-04-02 12:14:19 -0700214"""
215
Tao Bao89fbb0f2017-01-10 10:47:58 -0800216from __future__ import print_function
217
Tao Bao32fcdab2018-10-12 10:30:39 -0700218import logging
Doug Zongkerfc44a512014-08-26 13:10:25 -0700219import multiprocessing
Tao Bao2dd1c482017-02-03 16:49:39 -0800220import os.path
Baligh Uddin2abbbd02016-06-22 12:14:16 -0700221import shlex
Tao Bao15a146a2018-02-21 16:06:59 -0800222import shutil
Tao Bao85f16982018-03-08 16:28:33 -0800223import struct
Tao Bao481bab82017-12-21 11:23:09 -0800224import sys
Doug Zongkereef39442009-04-02 12:14:19 -0700225import zipfile
226
227import common
Kelvin Zhang2e417382020-08-20 11:33:11 -0400228import ota_utils
Kelvin Zhang0876c412020-06-23 15:06:58 -0400229import target_files_diff
Kelvin Zhangcff4d762020-07-29 16:37:51 -0400230from check_target_files_vintf import CheckVintfIfTrebleEnabled
231from non_ab_ota import GenerateNonAbOtaPackage
232from ota_utils import (UNZIP_PATTERN, FinalizeMetadata, GetPackageMetadata,
233 PropertyFiles)
Kelvin Zhang0876c412020-06-23 15:06:58 -0400234
Tao Bao481bab82017-12-21 11:23:09 -0800235if sys.hexversion < 0x02070000:
236 print("Python 2.7 or newer is required.", file=sys.stderr)
237 sys.exit(1)
238
Tao Bao32fcdab2018-10-12 10:30:39 -0700239logger = logging.getLogger(__name__)
Tao Bao481bab82017-12-21 11:23:09 -0800240
Kelvin Zhang2e417382020-08-20 11:33:11 -0400241OPTIONS = ota_utils.OPTIONS
Michael Runge63f01de2014-10-28 19:24:19 -0700242OPTIONS.verify = False
Doug Zongkereef39442009-04-02 12:14:19 -0700243OPTIONS.patch_threshold = 0.95
Doug Zongkerdbfaae52009-04-21 17:12:54 -0700244OPTIONS.wipe_user_data = False
Doug Zongker1c390a22009-05-14 19:06:36 -0700245OPTIONS.extra_script = None
Doug Zongkerfc44a512014-08-26 13:10:25 -0700246OPTIONS.worker_threads = multiprocessing.cpu_count() // 2
247if OPTIONS.worker_threads == 0:
248 OPTIONS.worker_threads = 1
Doug Zongker9b23f2c2013-11-25 14:44:12 -0800249OPTIONS.two_step = False
Tao Baof7140c02018-01-30 17:09:24 -0800250OPTIONS.include_secondary = False
Tao Bao457cbf62017-03-06 09:56:01 -0800251OPTIONS.block_based = True
Doug Zongker25568482014-03-03 10:21:27 -0800252OPTIONS.updater_binary = None
Tianjie Xu9afb2212020-05-10 21:48:15 +0000253OPTIONS.oem_dicts = None
Michael Runge6e836112014-04-15 17:40:21 -0700254OPTIONS.oem_source = None
Tao Bao8608cde2016-02-25 19:49:55 -0800255OPTIONS.oem_no_mount = False
Tao Bao43078aa2015-04-21 14:32:35 -0700256OPTIONS.full_radio = False
leozwangaa6c1a12015-08-14 10:57:58 -0700257OPTIONS.full_bootloader = False
Tao Baod47d8e12015-05-21 14:09:49 -0700258# Stash size cannot exceed cache_size * threshold.
259OPTIONS.cache_size = None
260OPTIONS.stash_threshold = 0.8
Tao Baod62c6032015-11-30 09:40:20 -0800261OPTIONS.log_diff = None
Tao Baodea0f8b2016-06-20 17:55:06 -0700262OPTIONS.payload_signer = None
Baligh Uddin2abbbd02016-06-22 12:14:16 -0700263OPTIONS.payload_signer_args = []
Tianjie Xu21e6deb2019-10-07 18:01:00 -0700264OPTIONS.payload_signer_maximum_signature_size = None
Tao Bao5f8ff932017-03-21 22:35:00 -0700265OPTIONS.extracted_input = None
Tao Bao15a146a2018-02-21 16:06:59 -0800266OPTIONS.skip_postinstall = False
xunchangabfa2652019-02-19 16:27:10 -0800267OPTIONS.skip_compatibility_check = False
Tianjie Xu1b079832019-08-28 12:19:23 -0700268OPTIONS.disable_fec_computation = False
Yifan Hong38ab4d82020-06-18 15:19:56 -0700269OPTIONS.partial = None
Hongguang Chen49ab1b902020-10-19 14:15:43 -0700270OPTIONS.custom_images = {}
Tao Bao8dcf7382015-05-21 14:09:49 -0700271
Tao Bao15a146a2018-02-21 16:06:59 -0800272POSTINSTALL_CONFIG = 'META/postinstall_config.txt'
Yifan Hong50e79542018-11-08 17:44:12 -0800273DYNAMIC_PARTITION_INFO = 'META/dynamic_partitions_info.txt'
Yifan Hongb433eba2019-03-06 12:42:53 -0800274AB_PARTITIONS = 'META/ab_partitions.txt'
Kelvin Zhangcff4d762020-07-29 16:37:51 -0400275
Tao Baof0c4aa22018-04-30 20:29:30 -0700276# Files to be unzipped for target diffing purpose.
277TARGET_DIFFING_UNZIP_PATTERN = ['BOOT', 'RECOVERY', 'SYSTEM/*', 'VENDOR/*',
Yifan Hongcfb917a2020-05-07 14:58:20 -0700278 'PRODUCT/*', 'SYSTEM_EXT/*', 'ODM/*',
Yifan Hongf496f1b2020-07-15 16:52:59 -0700279 'VENDOR_DLKM/*', 'ODM_DLKM/*']
Yifan Hongb433eba2019-03-06 12:42:53 -0800280RETROFIT_DAP_UNZIP_PATTERN = ['OTA/super_*.img', AB_PARTITIONS]
Tao Bao3e759462019-09-17 22:43:11 -0700281
282# Images to be excluded from secondary payload. We essentially only keep
283# 'system_other' and bootloader partitions.
284SECONDARY_PAYLOAD_SKIPPED_IMAGES = [
Yifan Hongc08cbf02020-09-15 19:07:39 +0000285 'boot', 'dtbo', 'modem', 'odm', 'odm_dlkm', 'product', 'radio', 'recovery',
Tianjiec3850642020-05-13 14:47:31 -0700286 'system_ext', 'vbmeta', 'vbmeta_system', 'vbmeta_vendor', 'vendor',
Yifan Hongf496f1b2020-07-15 16:52:59 -0700287 'vendor_boot']
Tao Bao6b0b2f92017-03-05 11:38:11 -0800288
Tao Bao2dd1c482017-02-03 16:49:39 -0800289
Tao Baofabe0832018-01-17 15:52:28 -0800290class PayloadSigner(object):
291 """A class that wraps the payload signing works.
292
293 When generating a Payload, hashes of the payload and metadata files will be
294 signed with the device key, either by calling an external payload signer or
295 by calling openssl with the package key. This class provides a unified
296 interface, so that callers can just call PayloadSigner.Sign().
297
298 If an external payload signer has been specified (OPTIONS.payload_signer), it
299 calls the signer with the provided args (OPTIONS.payload_signer_args). Note
300 that the signing key should be provided as part of the payload_signer_args.
301 Otherwise without an external signer, it uses the package key
302 (OPTIONS.package_key) and calls openssl for the signing works.
303 """
304
305 def __init__(self):
306 if OPTIONS.payload_signer is None:
307 # Prepare the payload signing key.
308 private_key = OPTIONS.package_key + OPTIONS.private_key_suffix
309 pw = OPTIONS.key_passwords[OPTIONS.package_key]
310
311 cmd = ["openssl", "pkcs8", "-in", private_key, "-inform", "DER"]
312 cmd.extend(["-passin", "pass:" + pw] if pw else ["-nocrypt"])
313 signing_key = common.MakeTempFile(prefix="key-", suffix=".key")
314 cmd.extend(["-out", signing_key])
Tao Baobec89c12018-10-15 11:53:28 -0700315 common.RunAndCheckOutput(cmd, verbose=False)
Tao Baofabe0832018-01-17 15:52:28 -0800316
317 self.signer = "openssl"
318 self.signer_args = ["pkeyutl", "-sign", "-inkey", signing_key,
319 "-pkeyopt", "digest:sha256"]
Tianjie Xu21e6deb2019-10-07 18:01:00 -0700320 self.maximum_signature_size = self._GetMaximumSignatureSizeInBytes(
321 signing_key)
Tao Baofabe0832018-01-17 15:52:28 -0800322 else:
323 self.signer = OPTIONS.payload_signer
324 self.signer_args = OPTIONS.payload_signer_args
Tianjie Xu21e6deb2019-10-07 18:01:00 -0700325 if OPTIONS.payload_signer_maximum_signature_size:
326 self.maximum_signature_size = int(
327 OPTIONS.payload_signer_maximum_signature_size)
xunchang376cc7c2019-04-08 23:04:58 -0700328 else:
Tianjie Xu21e6deb2019-10-07 18:01:00 -0700329 # The legacy config uses RSA2048 keys.
330 logger.warning("The maximum signature size for payload signer is not"
331 " set, default to 256 bytes.")
332 self.maximum_signature_size = 256
xunchang376cc7c2019-04-08 23:04:58 -0700333
334 @staticmethod
Tianjie Xu21e6deb2019-10-07 18:01:00 -0700335 def _GetMaximumSignatureSizeInBytes(signing_key):
336 out_signature_size_file = common.MakeTempFile("signature_size")
337 cmd = ["delta_generator", "--out_maximum_signature_size_file={}".format(
338 out_signature_size_file), "--private_key={}".format(signing_key)]
339 common.RunAndCheckOutput(cmd)
340 with open(out_signature_size_file) as f:
341 signature_size = f.read().rstrip()
Luca Stefani88e1a142020-03-27 14:05:12 +0100342 logger.info("%s outputs the maximum signature size: %s", cmd[0],
Tianjie Xu21e6deb2019-10-07 18:01:00 -0700343 signature_size)
344 return int(signature_size)
Tao Baofabe0832018-01-17 15:52:28 -0800345
346 def Sign(self, in_file):
347 """Signs the given input file. Returns the output filename."""
348 out_file = common.MakeTempFile(prefix="signed-", suffix=".bin")
349 cmd = [self.signer] + self.signer_args + ['-in', in_file, '-out', out_file]
Tao Bao718faed2019-08-02 13:24:19 -0700350 common.RunAndCheckOutput(cmd)
Tao Baofabe0832018-01-17 15:52:28 -0800351 return out_file
352
353
Tao Bao40b18822018-01-30 18:19:04 -0800354class Payload(object):
355 """Manages the creation and the signing of an A/B OTA Payload."""
356
357 PAYLOAD_BIN = 'payload.bin'
358 PAYLOAD_PROPERTIES_TXT = 'payload_properties.txt'
Tao Baof7140c02018-01-30 17:09:24 -0800359 SECONDARY_PAYLOAD_BIN = 'secondary/payload.bin'
360 SECONDARY_PAYLOAD_PROPERTIES_TXT = 'secondary/payload_properties.txt'
Tao Bao40b18822018-01-30 18:19:04 -0800361
Tao Bao667ff572018-02-10 00:02:40 -0800362 def __init__(self, secondary=False):
363 """Initializes a Payload instance.
364
365 Args:
366 secondary: Whether it's generating a secondary payload (default: False).
367 """
Tao Bao40b18822018-01-30 18:19:04 -0800368 self.payload_file = None
369 self.payload_properties = None
Tao Bao667ff572018-02-10 00:02:40 -0800370 self.secondary = secondary
Tao Bao40b18822018-01-30 18:19:04 -0800371
Tao Baof0c4aa22018-04-30 20:29:30 -0700372 def _Run(self, cmd): # pylint: disable=no-self-use
Tao Bao718faed2019-08-02 13:24:19 -0700373 # Don't pipe (buffer) the output if verbose is set. Let
374 # brillo_update_payload write to stdout/stderr directly, so its progress can
375 # be monitored.
376 if OPTIONS.verbose:
377 common.RunAndCheckOutput(cmd, stdout=None, stderr=None)
378 else:
379 common.RunAndCheckOutput(cmd)
380
Tao Bao40b18822018-01-30 18:19:04 -0800381 def Generate(self, target_file, source_file=None, additional_args=None):
382 """Generates a payload from the given target-files zip(s).
383
384 Args:
385 target_file: The filename of the target build target-files zip.
386 source_file: The filename of the source build target-files zip; or None if
387 generating a full OTA.
388 additional_args: A list of additional args that should be passed to
389 brillo_update_payload script; or None.
390 """
391 if additional_args is None:
392 additional_args = []
393
394 payload_file = common.MakeTempFile(prefix="payload-", suffix=".bin")
395 cmd = ["brillo_update_payload", "generate",
396 "--payload", payload_file,
397 "--target_image", target_file]
398 if source_file is not None:
399 cmd.extend(["--source_image", source_file])
Tianjie Xu1b079832019-08-28 12:19:23 -0700400 if OPTIONS.disable_fec_computation:
401 cmd.extend(["--disable_fec_computation", "true"])
Tao Bao40b18822018-01-30 18:19:04 -0800402 cmd.extend(additional_args)
Tao Bao718faed2019-08-02 13:24:19 -0700403 self._Run(cmd)
Tao Bao40b18822018-01-30 18:19:04 -0800404
405 self.payload_file = payload_file
406 self.payload_properties = None
407
408 def Sign(self, payload_signer):
409 """Generates and signs the hashes of the payload and metadata.
410
411 Args:
412 payload_signer: A PayloadSigner() instance that serves the signing work.
413
414 Raises:
415 AssertionError: On any failure when calling brillo_update_payload script.
416 """
417 assert isinstance(payload_signer, PayloadSigner)
418
419 # 1. Generate hashes of the payload and metadata files.
420 payload_sig_file = common.MakeTempFile(prefix="sig-", suffix=".bin")
421 metadata_sig_file = common.MakeTempFile(prefix="sig-", suffix=".bin")
422 cmd = ["brillo_update_payload", "hash",
423 "--unsigned_payload", self.payload_file,
Tianjie Xu21e6deb2019-10-07 18:01:00 -0700424 "--signature_size", str(payload_signer.maximum_signature_size),
Tao Bao40b18822018-01-30 18:19:04 -0800425 "--metadata_hash_file", metadata_sig_file,
426 "--payload_hash_file", payload_sig_file]
Tao Bao718faed2019-08-02 13:24:19 -0700427 self._Run(cmd)
Tao Bao40b18822018-01-30 18:19:04 -0800428
429 # 2. Sign the hashes.
430 signed_payload_sig_file = payload_signer.Sign(payload_sig_file)
431 signed_metadata_sig_file = payload_signer.Sign(metadata_sig_file)
432
433 # 3. Insert the signatures back into the payload file.
434 signed_payload_file = common.MakeTempFile(prefix="signed-payload-",
435 suffix=".bin")
436 cmd = ["brillo_update_payload", "sign",
437 "--unsigned_payload", self.payload_file,
438 "--payload", signed_payload_file,
Tianjie Xu21e6deb2019-10-07 18:01:00 -0700439 "--signature_size", str(payload_signer.maximum_signature_size),
Tao Bao40b18822018-01-30 18:19:04 -0800440 "--metadata_signature_file", signed_metadata_sig_file,
441 "--payload_signature_file", signed_payload_sig_file]
Tao Bao718faed2019-08-02 13:24:19 -0700442 self._Run(cmd)
Tao Bao40b18822018-01-30 18:19:04 -0800443
444 # 4. Dump the signed payload properties.
445 properties_file = common.MakeTempFile(prefix="payload-properties-",
446 suffix=".txt")
447 cmd = ["brillo_update_payload", "properties",
448 "--payload", signed_payload_file,
449 "--properties_file", properties_file]
Tao Bao718faed2019-08-02 13:24:19 -0700450 self._Run(cmd)
Tao Bao40b18822018-01-30 18:19:04 -0800451
Tao Bao667ff572018-02-10 00:02:40 -0800452 if self.secondary:
453 with open(properties_file, "a") as f:
454 f.write("SWITCH_SLOT_ON_REBOOT=0\n")
455
Tao Bao40b18822018-01-30 18:19:04 -0800456 if OPTIONS.wipe_user_data:
457 with open(properties_file, "a") as f:
458 f.write("POWERWASH=1\n")
459
460 self.payload_file = signed_payload_file
461 self.payload_properties = properties_file
462
Tao Bao667ff572018-02-10 00:02:40 -0800463 def WriteToZip(self, output_zip):
Tao Bao40b18822018-01-30 18:19:04 -0800464 """Writes the payload to the given zip.
465
466 Args:
467 output_zip: The output ZipFile instance.
468 """
469 assert self.payload_file is not None
470 assert self.payload_properties is not None
471
Tao Bao667ff572018-02-10 00:02:40 -0800472 if self.secondary:
Tao Baof7140c02018-01-30 17:09:24 -0800473 payload_arcname = Payload.SECONDARY_PAYLOAD_BIN
474 payload_properties_arcname = Payload.SECONDARY_PAYLOAD_PROPERTIES_TXT
475 else:
476 payload_arcname = Payload.PAYLOAD_BIN
477 payload_properties_arcname = Payload.PAYLOAD_PROPERTIES_TXT
478
Tao Bao40b18822018-01-30 18:19:04 -0800479 # Add the signed payload file and properties into the zip. In order to
480 # support streaming, we pack them as ZIP_STORED. So these entries can be
481 # read directly with the offset and length pairs.
Tao Baof7140c02018-01-30 17:09:24 -0800482 common.ZipWrite(output_zip, self.payload_file, arcname=payload_arcname,
Tao Bao40b18822018-01-30 18:19:04 -0800483 compress_type=zipfile.ZIP_STORED)
484 common.ZipWrite(output_zip, self.payload_properties,
Tao Baof7140c02018-01-30 17:09:24 -0800485 arcname=payload_properties_arcname,
Tao Bao40b18822018-01-30 18:19:04 -0800486 compress_type=zipfile.ZIP_STORED)
487
488
Tao Bao481bab82017-12-21 11:23:09 -0800489def _LoadOemDicts(oem_source):
Alain Vongsouvanh7f804ba2017-02-16 13:06:55 -0800490 """Returns the list of loaded OEM properties dict."""
Tao Bao481bab82017-12-21 11:23:09 -0800491 if not oem_source:
492 return None
493
Alain Vongsouvanh7f804ba2017-02-16 13:06:55 -0800494 oem_dicts = []
Tao Bao481bab82017-12-21 11:23:09 -0800495 for oem_file in oem_source:
496 with open(oem_file) as fp:
497 oem_dicts.append(common.LoadDictionaryFromLines(fp.readlines()))
Alain Vongsouvanh7f804ba2017-02-16 13:06:55 -0800498 return oem_dicts
Doug Zongkereef39442009-04-02 12:14:19 -0700499
Doug Zongkereef39442009-04-02 12:14:19 -0700500
Tao Baod3fc38a2018-03-08 16:09:01 -0800501class StreamingPropertyFiles(PropertyFiles):
502 """A subclass for computing the property-files for streaming A/B OTAs."""
503
504 def __init__(self):
505 super(StreamingPropertyFiles, self).__init__()
506 self.name = 'ota-streaming-property-files'
507 self.required = (
508 # payload.bin and payload_properties.txt must exist.
509 'payload.bin',
510 'payload_properties.txt',
511 )
512 self.optional = (
Tianjie Xu4c05f4a2018-09-14 16:24:41 -0700513 # care_map is available only if dm-verity is enabled.
514 'care_map.pb',
Tao Baod3fc38a2018-03-08 16:09:01 -0800515 'care_map.txt',
516 # compatibility.zip is available only if target supports Treble.
517 'compatibility.zip',
518 )
519
520
Tao Bao85f16982018-03-08 16:28:33 -0800521class AbOtaPropertyFiles(StreamingPropertyFiles):
522 """The property-files for A/B OTA that includes payload_metadata.bin info.
523
524 Since P, we expose one more token (aka property-file), in addition to the ones
525 for streaming A/B OTA, for a virtual entry of 'payload_metadata.bin'.
526 'payload_metadata.bin' is the header part of a payload ('payload.bin'), which
527 doesn't exist as a separate ZIP entry, but can be used to verify if the
528 payload can be applied on the given device.
529
530 For backward compatibility, we keep both of the 'ota-streaming-property-files'
531 and the newly added 'ota-property-files' in P. The new token will only be
532 available in 'ota-property-files'.
533 """
534
535 def __init__(self):
536 super(AbOtaPropertyFiles, self).__init__()
537 self.name = 'ota-property-files'
538
539 def _GetPrecomputed(self, input_zip):
540 offset, size = self._GetPayloadMetadataOffsetAndSize(input_zip)
541 return ['payload_metadata.bin:{}:{}'.format(offset, size)]
542
543 @staticmethod
544 def _GetPayloadMetadataOffsetAndSize(input_zip):
545 """Computes the offset and size of the payload metadata for a given package.
546
547 (From system/update_engine/update_metadata.proto)
548 A delta update file contains all the deltas needed to update a system from
549 one specific version to another specific version. The update format is
550 represented by this struct pseudocode:
551
552 struct delta_update_file {
553 char magic[4] = "CrAU";
554 uint64 file_format_version;
555 uint64 manifest_size; // Size of protobuf DeltaArchiveManifest
556
557 // Only present if format_version > 1:
558 uint32 metadata_signature_size;
559
560 // The Bzip2 compressed DeltaArchiveManifest
561 char manifest[metadata_signature_size];
562
563 // The signature of the metadata (from the beginning of the payload up to
564 // this location, not including the signature itself). This is a
565 // serialized Signatures message.
566 char medatada_signature_message[metadata_signature_size];
567
568 // Data blobs for files, no specific format. The specific offset
569 // and length of each data blob is recorded in the DeltaArchiveManifest.
570 struct {
571 char data[];
572 } blobs[];
573
574 // These two are not signed:
575 uint64 payload_signatures_message_size;
576 char payload_signatures_message[];
577 };
578
579 'payload-metadata.bin' contains all the bytes from the beginning of the
580 payload, till the end of 'medatada_signature_message'.
581 """
582 payload_info = input_zip.getinfo('payload.bin')
Shashikant Baviskar338856f2018-04-12 12:11:22 +0900583 payload_offset = payload_info.header_offset
584 payload_offset += zipfile.sizeFileHeader
585 payload_offset += len(payload_info.extra) + len(payload_info.filename)
Tao Bao85f16982018-03-08 16:28:33 -0800586 payload_size = payload_info.file_size
587
Tao Bao59cf0c52019-06-25 10:04:24 -0700588 with input_zip.open('payload.bin') as payload_fp:
Tao Bao85f16982018-03-08 16:28:33 -0800589 header_bin = payload_fp.read(24)
590
591 # network byte order (big-endian)
592 header = struct.unpack("!IQQL", header_bin)
593
594 # 'CrAU'
595 magic = header[0]
596 assert magic == 0x43724155, "Invalid magic: {:x}".format(magic)
597
598 manifest_size = header[2]
599 metadata_signature_size = header[3]
600 metadata_total = 24 + manifest_size + metadata_signature_size
601 assert metadata_total < payload_size
602
603 return (payload_offset, metadata_total)
604
605
Yifan Hong38ab4d82020-06-18 15:19:56 -0700606def UpdatesInfoForSpecialUpdates(content, partitions_filter,
607 delete_keys=None):
608 """ Updates info file for secondary payload generation, partial update, etc.
609
610 Scan each line in the info file, and remove the unwanted partitions from
611 the dynamic partition list in the related properties. e.g.
612 "super_google_dynamic_partitions_partition_list=system vendor product"
613 will become "super_google_dynamic_partitions_partition_list=system".
614
615 Args:
616 content: The content of the input info file. e.g. misc_info.txt.
617 partitions_filter: A function to filter the desired partitions from a given
618 list
619 delete_keys: A list of keys to delete in the info file
620
621 Returns:
622 A string of the updated info content.
623 """
624
625 output_list = []
626 # The suffix in partition_list variables that follows the name of the
627 # partition group.
628 list_suffix = 'partition_list'
629 for line in content.splitlines():
630 if line.startswith('#') or '=' not in line:
631 output_list.append(line)
632 continue
633 key, value = line.strip().split('=', 1)
634
635 if delete_keys and key in delete_keys:
636 pass
637 elif key.endswith(list_suffix):
638 partitions = value.split()
639 # TODO for partial update, partitions in the same group must be all
640 # updated or all omitted
641 partitions = filter(partitions_filter, partitions)
642 output_list.append('{}={}'.format(key, ' '.join(partitions)))
643 else:
644 output_list.append(line)
645 return '\n'.join(output_list)
646
647
Tao Bao15a146a2018-02-21 16:06:59 -0800648def GetTargetFilesZipForSecondaryImages(input_file, skip_postinstall=False):
Tao Baof7140c02018-01-30 17:09:24 -0800649 """Returns a target-files.zip file for generating secondary payload.
650
651 Although the original target-files.zip already contains secondary slot
652 images (i.e. IMAGES/system_other.img), we need to rename the files to the
653 ones without _other suffix. Note that we cannot instead modify the names in
654 META/ab_partitions.txt, because there are no matching partitions on device.
655
656 For the partitions that don't have secondary images, the ones for primary
657 slot will be used. This is to ensure that we always have valid boot, vbmeta,
658 bootloader images in the inactive slot.
659
660 Args:
661 input_file: The input target-files.zip file.
Tao Bao15a146a2018-02-21 16:06:59 -0800662 skip_postinstall: Whether to skip copying the postinstall config file.
Tao Baof7140c02018-01-30 17:09:24 -0800663
664 Returns:
665 The filename of the target-files.zip for generating secondary payload.
666 """
Tianjie Xu1c808002019-09-11 00:29:26 -0700667
668 def GetInfoForSecondaryImages(info_file):
Yifan Hong38ab4d82020-06-18 15:19:56 -0700669 """Updates info file for secondary payload generation."""
Tianjie Xu1c808002019-09-11 00:29:26 -0700670 with open(info_file) as f:
Yifan Hong38ab4d82020-06-18 15:19:56 -0700671 content = f.read()
672 # Remove virtual_ab flag from secondary payload so that OTA client
673 # don't use snapshots for secondary update
674 delete_keys = ['virtual_ab', "virtual_ab_retrofit"]
675 return UpdatesInfoForSpecialUpdates(
676 content, lambda p: p not in SECONDARY_PAYLOAD_SKIPPED_IMAGES,
677 delete_keys)
Tianjie Xu1c808002019-09-11 00:29:26 -0700678
Tao Baof7140c02018-01-30 17:09:24 -0800679 target_file = common.MakeTempFile(prefix="targetfiles-", suffix=".zip")
680 target_zip = zipfile.ZipFile(target_file, 'w', allowZip64=True)
681
Kelvin Zhang928c2342020-09-22 16:15:57 -0400682 with zipfile.ZipFile(input_file, 'r', allowZip64=True) as input_zip:
Tao Baodba59ee2018-01-09 13:21:02 -0800683 infolist = input_zip.infolist()
Tao Bao12489802018-07-12 14:47:38 -0700684
Tao Bao0ff15de2019-03-20 11:26:06 -0700685 input_tmp = common.UnzipTemp(input_file, UNZIP_PATTERN)
Tao Baodba59ee2018-01-09 13:21:02 -0800686 for info in infolist:
Tao Baof7140c02018-01-30 17:09:24 -0800687 unzipped_file = os.path.join(input_tmp, *info.filename.split('/'))
688 if info.filename == 'IMAGES/system_other.img':
689 common.ZipWrite(target_zip, unzipped_file, arcname='IMAGES/system.img')
690
691 # Primary images and friends need to be skipped explicitly.
692 elif info.filename in ('IMAGES/system.img',
693 'IMAGES/system.map'):
694 pass
Tao Bao3e759462019-09-17 22:43:11 -0700695
696 # Copy images that are not in SECONDARY_PAYLOAD_SKIPPED_IMAGES.
697 elif info.filename.startswith(('IMAGES/', 'RADIO/')):
698 image_name = os.path.basename(info.filename)
699 if image_name not in ['{}.img'.format(partition) for partition in
700 SECONDARY_PAYLOAD_SKIPPED_IMAGES]:
701 common.ZipWrite(target_zip, unzipped_file, arcname=info.filename)
Tao Baof7140c02018-01-30 17:09:24 -0800702
Tao Bao15a146a2018-02-21 16:06:59 -0800703 # Skip copying the postinstall config if requested.
704 elif skip_postinstall and info.filename == POSTINSTALL_CONFIG:
705 pass
706
Tianjie Xu1c808002019-09-11 00:29:26 -0700707 elif info.filename.startswith('META/'):
708 # Remove the unnecessary partitions for secondary images from the
709 # ab_partitions file.
710 if info.filename == AB_PARTITIONS:
711 with open(unzipped_file) as f:
712 partition_list = f.read().splitlines()
713 partition_list = [partition for partition in partition_list if partition
Tao Bao3e759462019-09-17 22:43:11 -0700714 and partition not in SECONDARY_PAYLOAD_SKIPPED_IMAGES]
Kelvin Zhang0876c412020-06-23 15:06:58 -0400715 common.ZipWriteStr(target_zip, info.filename,
716 '\n'.join(partition_list))
Tianjie Xu1c808002019-09-11 00:29:26 -0700717 # Remove the unnecessary partitions from the dynamic partitions list.
718 elif (info.filename == 'META/misc_info.txt' or
719 info.filename == DYNAMIC_PARTITION_INFO):
720 modified_info = GetInfoForSecondaryImages(unzipped_file)
721 common.ZipWriteStr(target_zip, info.filename, modified_info)
722 else:
723 common.ZipWrite(target_zip, unzipped_file, arcname=info.filename)
Tao Baof7140c02018-01-30 17:09:24 -0800724
Tao Baof7140c02018-01-30 17:09:24 -0800725 common.ZipClose(target_zip)
726
727 return target_file
728
729
Tao Bao15a146a2018-02-21 16:06:59 -0800730def GetTargetFilesZipWithoutPostinstallConfig(input_file):
731 """Returns a target-files.zip that's not containing postinstall_config.txt.
732
733 This allows brillo_update_payload script to skip writing all the postinstall
734 hooks in the generated payload. The input target-files.zip file will be
735 duplicated, with 'META/postinstall_config.txt' skipped. If input_file doesn't
736 contain the postinstall_config.txt entry, the input file will be returned.
737
738 Args:
739 input_file: The input target-files.zip filename.
740
741 Returns:
742 The filename of target-files.zip that doesn't contain postinstall config.
743 """
744 # We should only make a copy if postinstall_config entry exists.
Kelvin Zhang928c2342020-09-22 16:15:57 -0400745 with zipfile.ZipFile(input_file, 'r', allowZip64=True) as input_zip:
Tao Bao15a146a2018-02-21 16:06:59 -0800746 if POSTINSTALL_CONFIG not in input_zip.namelist():
747 return input_file
748
749 target_file = common.MakeTempFile(prefix="targetfiles-", suffix=".zip")
750 shutil.copyfile(input_file, target_file)
751 common.ZipDelete(target_file, POSTINSTALL_CONFIG)
752 return target_file
753
Kelvin Zhanga59bb272020-10-30 12:52:25 -0400754def ParseInfoDict(target_file_path):
755 with zipfile.ZipFile(target_file_path, 'r', allowZip64=True) as zfp:
756 return common.LoadInfoDict(zfp)
Tao Bao15a146a2018-02-21 16:06:59 -0800757
Yifan Hong38ab4d82020-06-18 15:19:56 -0700758def GetTargetFilesZipForPartialUpdates(input_file, ab_partitions):
759 """Returns a target-files.zip for partial ota update package generation.
760
761 This function modifies ab_partitions list with the desired partitions before
762 calling the brillo_update_payload script. It also cleans up the reference to
763 the excluded partitions in the info file, e.g misc_info.txt.
764
765 Args:
766 input_file: The input target-files.zip filename.
767 ab_partitions: A list of partitions to include in the partial update
768
769 Returns:
770 The filename of target-files.zip used for partial ota update.
771 """
772
773 def AddImageForPartition(partition_name):
774 """Add the archive name for a given partition to the copy list."""
775 for prefix in ['IMAGES', 'RADIO']:
776 image_path = '{}/{}.img'.format(prefix, partition_name)
777 if image_path in namelist:
778 copy_entries.append(image_path)
779 map_path = '{}/{}.map'.format(prefix, partition_name)
780 if map_path in namelist:
781 copy_entries.append(map_path)
782 return
783
784 raise ValueError("Cannot find {} in input zipfile".format(partition_name))
785
786 with zipfile.ZipFile(input_file, allowZip64=True) as input_zip:
Kelvin Zhanga59bb272020-10-30 12:52:25 -0400787 original_ab_partitions = input_zip.read(
788 AB_PARTITIONS).decode().splitlines()
Yifan Hong38ab4d82020-06-18 15:19:56 -0700789 namelist = input_zip.namelist()
790
791 unrecognized_partitions = [partition for partition in ab_partitions if
792 partition not in original_ab_partitions]
793 if unrecognized_partitions:
794 raise ValueError("Unrecognized partitions when generating partial updates",
795 unrecognized_partitions)
796
797 logger.info("Generating partial updates for %s", ab_partitions)
798
799 copy_entries = ['META/update_engine_config.txt']
800 for partition_name in ab_partitions:
801 AddImageForPartition(partition_name)
802
803 # Use zip2zip to avoid extracting the zipfile.
804 partial_target_file = common.MakeTempFile(suffix='.zip')
805 cmd = ['zip2zip', '-i', input_file, '-o', partial_target_file]
806 cmd.extend(['{}:{}'.format(name, name) for name in copy_entries])
807 common.RunAndCheckOutput(cmd)
808
809 partial_target_zip = zipfile.ZipFile(partial_target_file, 'a',
810 allowZip64=True)
811 with zipfile.ZipFile(input_file, allowZip64=True) as input_zip:
812 common.ZipWriteStr(partial_target_zip, 'META/ab_partitions.txt',
813 '\n'.join(ab_partitions))
814 for info_file in ['META/misc_info.txt', DYNAMIC_PARTITION_INFO]:
815 if info_file not in input_zip.namelist():
816 logger.warning('Cannot find %s in input zipfile', info_file)
817 continue
818 content = input_zip.read(info_file).decode()
819 modified_info = UpdatesInfoForSpecialUpdates(
820 content, lambda p: p in ab_partitions)
821 common.ZipWriteStr(partial_target_zip, info_file, modified_info)
822
823 # TODO(xunchang) handle 'META/care_map.pb', 'META/postinstall_config.txt'
824 common.ZipClose(partial_target_zip)
825
826 return partial_target_file
827
828
Yifan Hong50e79542018-11-08 17:44:12 -0800829def GetTargetFilesZipForRetrofitDynamicPartitions(input_file,
Yifan Hongb433eba2019-03-06 12:42:53 -0800830 super_block_devices,
831 dynamic_partition_list):
Yifan Hong50e79542018-11-08 17:44:12 -0800832 """Returns a target-files.zip for retrofitting dynamic partitions.
833
834 This allows brillo_update_payload to generate an OTA based on the exact
835 bits on the block devices. Postinstall is disabled.
836
837 Args:
838 input_file: The input target-files.zip filename.
839 super_block_devices: The list of super block devices
Yifan Hongb433eba2019-03-06 12:42:53 -0800840 dynamic_partition_list: The list of dynamic partitions
Yifan Hong50e79542018-11-08 17:44:12 -0800841
842 Returns:
843 The filename of target-files.zip with *.img replaced with super_*.img for
844 each block device in super_block_devices.
845 """
846 assert super_block_devices, "No super_block_devices are specified."
847
848 replace = {'OTA/super_{}.img'.format(dev): 'IMAGES/{}.img'.format(dev)
Tao Bao03fecb62018-11-28 10:59:23 -0800849 for dev in super_block_devices}
Yifan Hong50e79542018-11-08 17:44:12 -0800850
851 target_file = common.MakeTempFile(prefix="targetfiles-", suffix=".zip")
852 shutil.copyfile(input_file, target_file)
853
Kelvin Zhang928c2342020-09-22 16:15:57 -0400854 with zipfile.ZipFile(input_file, allowZip64=True) as input_zip:
Yifan Hong50e79542018-11-08 17:44:12 -0800855 namelist = input_zip.namelist()
856
Yifan Hongb433eba2019-03-06 12:42:53 -0800857 input_tmp = common.UnzipTemp(input_file, RETROFIT_DAP_UNZIP_PATTERN)
858
859 # Remove partitions from META/ab_partitions.txt that is in
860 # dynamic_partition_list but not in super_block_devices so that
861 # brillo_update_payload won't generate update for those logical partitions.
862 ab_partitions_file = os.path.join(input_tmp, *AB_PARTITIONS.split('/'))
863 with open(ab_partitions_file) as f:
864 ab_partitions_lines = f.readlines()
865 ab_partitions = [line.strip() for line in ab_partitions_lines]
866 # Assert that all super_block_devices are in ab_partitions
867 super_device_not_updated = [partition for partition in super_block_devices
868 if partition not in ab_partitions]
869 assert not super_device_not_updated, \
870 "{} is in super_block_devices but not in {}".format(
871 super_device_not_updated, AB_PARTITIONS)
872 # ab_partitions -= (dynamic_partition_list - super_block_devices)
Kelvin Zhang0876c412020-06-23 15:06:58 -0400873 new_ab_partitions = common.MakeTempFile(
874 prefix="ab_partitions", suffix=".txt")
Yifan Hongb433eba2019-03-06 12:42:53 -0800875 with open(new_ab_partitions, 'w') as f:
876 for partition in ab_partitions:
877 if (partition in dynamic_partition_list and
Kelvin Zhanga59bb272020-10-30 12:52:25 -0400878 partition not in super_block_devices):
Tao Bao59cf0c52019-06-25 10:04:24 -0700879 logger.info("Dropping %s from ab_partitions.txt", partition)
880 continue
Yifan Hongb433eba2019-03-06 12:42:53 -0800881 f.write(partition + "\n")
882 to_delete = [AB_PARTITIONS]
883
Yifan Hong50e79542018-11-08 17:44:12 -0800884 # Always skip postinstall for a retrofit update.
Yifan Hongb433eba2019-03-06 12:42:53 -0800885 to_delete += [POSTINSTALL_CONFIG]
Yifan Hong50e79542018-11-08 17:44:12 -0800886
887 # Delete dynamic_partitions_info.txt so that brillo_update_payload thinks this
888 # is a regular update on devices without dynamic partitions support.
889 to_delete += [DYNAMIC_PARTITION_INFO]
890
Tao Bao03fecb62018-11-28 10:59:23 -0800891 # Remove the existing partition images as well as the map files.
Tao Bao59cf0c52019-06-25 10:04:24 -0700892 to_delete += list(replace.values())
Tao Bao03fecb62018-11-28 10:59:23 -0800893 to_delete += ['IMAGES/{}.map'.format(dev) for dev in super_block_devices]
Yifan Hong50e79542018-11-08 17:44:12 -0800894
895 common.ZipDelete(target_file, to_delete)
896
Yifan Hong50e79542018-11-08 17:44:12 -0800897 target_zip = zipfile.ZipFile(target_file, 'a', allowZip64=True)
898
899 # Write super_{foo}.img as {foo}.img.
900 for src, dst in replace.items():
901 assert src in namelist, \
Tao Bao59cf0c52019-06-25 10:04:24 -0700902 'Missing {} in {}; {} cannot be written'.format(src, input_file, dst)
Yifan Hong50e79542018-11-08 17:44:12 -0800903 unzipped_file = os.path.join(input_tmp, *src.split('/'))
904 common.ZipWrite(target_zip, unzipped_file, arcname=dst)
905
Yifan Hongb433eba2019-03-06 12:42:53 -0800906 # Write new ab_partitions.txt file
907 common.ZipWrite(target_zip, new_ab_partitions, arcname=AB_PARTITIONS)
908
Yifan Hong50e79542018-11-08 17:44:12 -0800909 common.ZipClose(target_zip)
910
911 return target_file
912
Kelvin Zhanga59bb272020-10-30 12:52:25 -0400913
Hongguang Chen49ab1b902020-10-19 14:15:43 -0700914def GetTargetFilesZipForCustomImagesUpdates(input_file, custom_images):
915 """Returns a target-files.zip for custom partitions update.
916
917 This function modifies ab_partitions list with the desired custom partitions
918 and puts the custom images into the target target-files.zip.
919
920 Args:
921 input_file: The input target-files.zip filename.
922 custom_images: A map of custom partitions and custom images.
923
924 Returns:
925 The filename of a target-files.zip which has renamed the custom images in
926 the IMAGS/ to their partition names.
927 """
928 # Use zip2zip to avoid extracting the zipfile.
929 target_file = common.MakeTempFile(prefix="targetfiles-", suffix=".zip")
930 cmd = ['zip2zip', '-i', input_file, '-o', target_file]
931
932 with zipfile.ZipFile(input_file, allowZip64=True) as input_zip:
933 namelist = input_zip.namelist()
934
935 # Write {custom_image}.img as {custom_partition}.img.
936 for custom_partition, custom_image in custom_images.items():
937 default_custom_image = '{}.img'.format(custom_partition)
938 if default_custom_image != custom_image:
939 logger.info("Update custom partition '%s' with '%s'",
940 custom_partition, custom_image)
941 # Default custom image need to be deleted first.
942 namelist.remove('IMAGES/{}'.format(default_custom_image))
943 # IMAGES/{custom_image}.img:IMAGES/{custom_partition}.img.
944 cmd.extend(['IMAGES/{}:IMAGES/{}'.format(custom_image,
945 default_custom_image)])
946
947 cmd.extend(['{}:{}'.format(name, name) for name in namelist])
948 common.RunAndCheckOutput(cmd)
949
950 return target_file
Yifan Hong50e79542018-11-08 17:44:12 -0800951
Kelvin Zhanga59bb272020-10-30 12:52:25 -0400952def GeneratePartitionTimestampFlags(partition_state):
953 partition_timestamps = [
954 part.partition_name + ":" + part.version
955 for part in partition_state]
956 return ["--partition_timestamps", ",".join(partition_timestamps)]
957
Kelvin Zhang80195722020-11-04 14:38:34 -0500958def GeneratePartitionTimestampFlagsDowngrade(pre_partition_state, post_partition_state):
959 assert pre_partition_state is not None
960 partition_timestamps = {}
961 for part in pre_partition_state:
962 partition_timestamps[part.partition_name] = part.version
963 for part in post_partition_state:
964 partition_timestamps[part.partition_name] = \
965 max(part.version, partition_timestamps[part.partition_name])
966 return [
967 "--partition_timestamps",
968 ",".join([key + ":" + val for (key, val) in partition_timestamps.items()])
969 ]
970
Tao Baof0c4aa22018-04-30 20:29:30 -0700971def GenerateAbOtaPackage(target_file, output_file, source_file=None):
Tao Baofe5b69a2018-03-02 09:47:43 -0800972 """Generates an Android OTA package that has A/B update payload."""
Tao Baodea0f8b2016-06-20 17:55:06 -0700973 # Stage the output zip package for package signing.
Tao Bao491d7e22018-02-21 13:17:22 -0800974 if not OPTIONS.no_signing:
975 staging_file = common.MakeTempFile(suffix='.zip')
976 else:
977 staging_file = output_file
Tao Baoa652c002018-03-01 19:31:38 -0800978 output_zip = zipfile.ZipFile(staging_file, "w",
Kelvin Zhang928c2342020-09-22 16:15:57 -0400979 compression=zipfile.ZIP_DEFLATED, allowZip64=True)
Tao Baoc098e9e2016-01-07 13:03:56 -0800980
Tao Bao481bab82017-12-21 11:23:09 -0800981 if source_file is not None:
Kelvin Zhang39aea442020-08-17 11:04:25 -0400982 assert "ab_partitions" in OPTIONS.source_info_dict, \
983 "META/ab_partitions.txt is required for ab_update."
984 assert "ab_partitions" in OPTIONS.target_info_dict, \
985 "META/ab_partitions.txt is required for ab_update."
Tao Bao1c320f82019-10-04 23:25:12 -0700986 target_info = common.BuildInfo(OPTIONS.target_info_dict, OPTIONS.oem_dicts)
987 source_info = common.BuildInfo(OPTIONS.source_info_dict, OPTIONS.oem_dicts)
Kelvin Zhang31233e52020-11-03 13:42:46 -0500988 vendor_prop = source_info.info_dict.get("vendor.build.prop")
989 if vendor_prop and \
990 vendor_prop.GetProp("ro.virtual_ab.compression.enabled") == "true":
991 # TODO(zhangkelvin) Remove this once FEC on VABC is supported
992 logger.info("Virtual AB Compression enabled, disabling FEC")
993 OPTIONS.disable_fec_computation = True
Tao Bao481bab82017-12-21 11:23:09 -0800994 else:
Kelvin Zhang39aea442020-08-17 11:04:25 -0400995 assert "ab_partitions" in OPTIONS.info_dict, \
996 "META/ab_partitions.txt is required for ab_update."
Tao Bao1c320f82019-10-04 23:25:12 -0700997 target_info = common.BuildInfo(OPTIONS.info_dict, OPTIONS.oem_dicts)
Tao Bao481bab82017-12-21 11:23:09 -0800998 source_info = None
Tao Baoc098e9e2016-01-07 13:03:56 -0800999
Yifan Hong38ab4d82020-06-18 15:19:56 -07001000 additional_args = []
1001
Hongguang Chen49ab1b902020-10-19 14:15:43 -07001002 # Prepare custom images.
1003 if OPTIONS.custom_images:
1004 target_file = GetTargetFilesZipForCustomImagesUpdates(
1005 target_file, OPTIONS.custom_images)
1006
Yifan Hong50e79542018-11-08 17:44:12 -08001007 if OPTIONS.retrofit_dynamic_partitions:
1008 target_file = GetTargetFilesZipForRetrofitDynamicPartitions(
Yifan Hongb433eba2019-03-06 12:42:53 -08001009 target_file, target_info.get("super_block_devices").strip().split(),
1010 target_info.get("dynamic_partition_list").strip().split())
Yifan Hong38ab4d82020-06-18 15:19:56 -07001011 elif OPTIONS.partial:
1012 target_file = GetTargetFilesZipForPartialUpdates(target_file,
1013 OPTIONS.partial)
1014 additional_args += ["--is_partial_update", "true"]
Yifan Hong50e79542018-11-08 17:44:12 -08001015 elif OPTIONS.skip_postinstall:
Tao Bao15a146a2018-02-21 16:06:59 -08001016 target_file = GetTargetFilesZipWithoutPostinstallConfig(target_file)
Kelvin Zhang39aea442020-08-17 11:04:25 -04001017 # Target_file may have been modified, reparse ab_partitions
1018 with zipfile.ZipFile(target_file, allowZip64=True) as zfp:
1019 target_info.info_dict['ab_partitions'] = zfp.read(
Kelvin Zhang31233e52020-11-03 13:42:46 -05001020 AB_PARTITIONS).decode().strip().split("\n")
Tao Bao15a146a2018-02-21 16:06:59 -08001021
Kelvin Zhang39aea442020-08-17 11:04:25 -04001022 # Metadata to comply with Android OTA package format.
1023 metadata = GetPackageMetadata(target_info, source_info)
Tao Bao40b18822018-01-30 18:19:04 -08001024 # Generate payload.
1025 payload = Payload()
1026
Kelvin Zhanga59bb272020-10-30 12:52:25 -04001027 partition_timestamps_flags = []
Tao Bao40b18822018-01-30 18:19:04 -08001028 # Enforce a max timestamp this payload can be applied on top of.
Tao Baoff1b86e2017-10-03 14:17:57 -07001029 if OPTIONS.downgrade:
Tao Bao2a12ed72018-01-22 11:35:00 -08001030 max_timestamp = source_info.GetBuildProp("ro.build.date.utc")
Kelvin Zhang80195722020-11-04 14:38:34 -05001031 partition_timestamps_flags = GeneratePartitionTimestampFlagsDowngrade(
1032 metadata.precondition.partition_state,
1033 metadata.postcondition.partition_state
1034 )
Tao Baoff1b86e2017-10-03 14:17:57 -07001035 else:
Tianjiea2076132020-08-19 17:25:32 -07001036 max_timestamp = str(metadata.postcondition.timestamp)
Kelvin Zhanga59bb272020-10-30 12:52:25 -04001037 partition_timestamps_flags = GeneratePartitionTimestampFlags(
1038 metadata.postcondition.partition_state)
Tao Baoc098e9e2016-01-07 13:03:56 -08001039
Kelvin Zhanga59bb272020-10-30 12:52:25 -04001040 additional_args += ["--max_timestamp", max_timestamp]
1041
1042 payload.Generate(
1043 target_file,
1044 source_file,
1045 additional_args + partition_timestamps_flags
1046 )
Tao Baoc098e9e2016-01-07 13:03:56 -08001047
Tao Bao40b18822018-01-30 18:19:04 -08001048 # Sign the payload.
Tao Baof7140c02018-01-30 17:09:24 -08001049 payload_signer = PayloadSigner()
1050 payload.Sign(payload_signer)
Tao Baoc098e9e2016-01-07 13:03:56 -08001051
Tao Bao40b18822018-01-30 18:19:04 -08001052 # Write the payload into output zip.
1053 payload.WriteToZip(output_zip)
Tao Baoc098e9e2016-01-07 13:03:56 -08001054
Tao Baof7140c02018-01-30 17:09:24 -08001055 # Generate and include the secondary payload that installs secondary images
1056 # (e.g. system_other.img).
1057 if OPTIONS.include_secondary:
1058 # We always include a full payload for the secondary slot, even when
1059 # building an incremental OTA. See the comments for "--include_secondary".
Tao Bao15a146a2018-02-21 16:06:59 -08001060 secondary_target_file = GetTargetFilesZipForSecondaryImages(
1061 target_file, OPTIONS.skip_postinstall)
Tao Bao667ff572018-02-10 00:02:40 -08001062 secondary_payload = Payload(secondary=True)
Kelvin Zhanga59bb272020-10-30 12:52:25 -04001063 assert not OPTIONS.downgrade
1064 partition_timestamps_flags = GeneratePartitionTimestampFlags(
1065 [part
1066 for part in metadata.postcondition.partition_state
1067 if part.partition_name not in SECONDARY_PAYLOAD_SKIPPED_IMAGES]
1068 )
Tao Baodb1fe412018-02-09 23:15:05 -08001069 secondary_payload.Generate(secondary_target_file,
Kelvin Zhanga59bb272020-10-30 12:52:25 -04001070 additional_args=["--max_timestamp",
1071 max_timestamp]+partition_timestamps_flags)
Tao Baof7140c02018-01-30 17:09:24 -08001072 secondary_payload.Sign(payload_signer)
Tao Bao667ff572018-02-10 00:02:40 -08001073 secondary_payload.WriteToZip(output_zip)
Tao Baof7140c02018-01-30 17:09:24 -08001074
Tianjie Xucfa86222016-03-07 16:31:19 -08001075 # If dm-verity is supported for the device, copy contents of care_map
1076 # into A/B OTA package.
Kelvin Zhang928c2342020-09-22 16:15:57 -04001077 target_zip = zipfile.ZipFile(target_file, "r", allowZip64=True)
Tao Bao481bab82017-12-21 11:23:09 -08001078 if (target_info.get("verity") == "true" or
Kelvin Zhang39aea442020-08-17 11:04:25 -04001079 target_info.get("avb_enable") == "true"):
Tianjie Xu4c05f4a2018-09-14 16:24:41 -07001080 care_map_list = [x for x in ["care_map.pb", "care_map.txt"] if
1081 "META/" + x in target_zip.namelist()]
1082
1083 # Adds care_map if either the protobuf format or the plain text one exists.
1084 if care_map_list:
1085 care_map_name = care_map_list[0]
1086 care_map_data = target_zip.read("META/" + care_map_name)
1087 # In order to support streaming, care_map needs to be packed as
Tao Bao40b18822018-01-30 18:19:04 -08001088 # ZIP_STORED.
Tianjie Xu4c05f4a2018-09-14 16:24:41 -07001089 common.ZipWriteStr(output_zip, care_map_name, care_map_data,
Tao Bao481bab82017-12-21 11:23:09 -08001090 compress_type=zipfile.ZIP_STORED)
Tianjie Xucfa86222016-03-07 16:31:19 -08001091 else:
Tao Bao32fcdab2018-10-12 10:30:39 -07001092 logger.warning("Cannot find care map file in target_file package")
Tao Bao21803d32017-04-19 10:16:09 -07001093
Tao Bao21803d32017-04-19 10:16:09 -07001094 common.ZipClose(target_zip)
Tianjie Xucfa86222016-03-07 16:31:19 -08001095
Yifan Hong9276cf02019-08-21 16:37:04 -07001096 CheckVintfIfTrebleEnabled(target_file, target_info)
1097
Tao Baofe5b69a2018-03-02 09:47:43 -08001098 # We haven't written the metadata entry yet, which will be handled in
1099 # FinalizeMetadata().
Tao Baoc96316c2017-01-24 22:10:49 -08001100 common.ZipClose(output_zip)
1101
Tao Bao85f16982018-03-08 16:28:33 -08001102 # AbOtaPropertyFiles intends to replace StreamingPropertyFiles, as it covers
1103 # all the info of the latter. However, system updaters and OTA servers need to
1104 # take time to switch to the new flag. We keep both of the flags for
1105 # P-timeframe, and will remove StreamingPropertyFiles in later release.
Tao Baod3fc38a2018-03-08 16:09:01 -08001106 needed_property_files = (
Tao Bao85f16982018-03-08 16:28:33 -08001107 AbOtaPropertyFiles(),
Tao Baod3fc38a2018-03-08 16:09:01 -08001108 StreamingPropertyFiles(),
1109 )
1110 FinalizeMetadata(metadata, staging_file, output_file, needed_property_files)
Tao Baoc96316c2017-01-24 22:10:49 -08001111
Tao Baoc098e9e2016-01-07 13:03:56 -08001112
Doug Zongkereef39442009-04-02 12:14:19 -07001113def main(argv):
1114
1115 def option_handler(o, a):
Tao Bao4b76a0e2017-10-31 12:13:33 -07001116 if o in ("-k", "--package_key"):
Doug Zongkereef39442009-04-02 12:14:19 -07001117 OPTIONS.package_key = a
Doug Zongkereef39442009-04-02 12:14:19 -07001118 elif o in ("-i", "--incremental_from"):
1119 OPTIONS.incremental_source = a
Tao Bao43078aa2015-04-21 14:32:35 -07001120 elif o == "--full_radio":
1121 OPTIONS.full_radio = True
leozwangaa6c1a12015-08-14 10:57:58 -07001122 elif o == "--full_bootloader":
1123 OPTIONS.full_bootloader = True
Tao Bao337633f2017-12-06 15:20:19 -08001124 elif o == "--wipe_user_data":
Doug Zongkerdbfaae52009-04-21 17:12:54 -07001125 OPTIONS.wipe_user_data = True
Tao Bao5d182562016-02-23 11:38:39 -08001126 elif o == "--downgrade":
1127 OPTIONS.downgrade = True
1128 OPTIONS.wipe_user_data = True
Tao Bao3e6161a2017-02-28 11:48:48 -08001129 elif o == "--override_timestamp":
Tao Baofaa8e0b2018-04-12 14:31:43 -07001130 OPTIONS.downgrade = True
Michael Runge6e836112014-04-15 17:40:21 -07001131 elif o in ("-o", "--oem_settings"):
Alain Vongsouvanh7f804ba2017-02-16 13:06:55 -08001132 OPTIONS.oem_source = a.split(',')
Tao Bao8608cde2016-02-25 19:49:55 -08001133 elif o == "--oem_no_mount":
1134 OPTIONS.oem_no_mount = True
Doug Zongker1c390a22009-05-14 19:06:36 -07001135 elif o in ("-e", "--extra_script"):
1136 OPTIONS.extra_script = a
Martin Blumenstingl374e1142014-05-31 20:42:55 +02001137 elif o in ("-t", "--worker_threads"):
1138 if a.isdigit():
1139 OPTIONS.worker_threads = int(a)
1140 else:
1141 raise ValueError("Cannot parse value %r for option %r - only "
1142 "integers are allowed." % (a, o))
Doug Zongker9b23f2c2013-11-25 14:44:12 -08001143 elif o in ("-2", "--two_step"):
1144 OPTIONS.two_step = True
Tao Baof7140c02018-01-30 17:09:24 -08001145 elif o == "--include_secondary":
1146 OPTIONS.include_secondary = True
Doug Zongker26e66192014-02-20 13:22:07 -08001147 elif o == "--no_signing":
Takeshi Kanemotoe153b342013-11-14 17:20:50 +09001148 OPTIONS.no_signing = True
Dan Albert8b72aef2015-03-23 19:13:21 -07001149 elif o == "--verify":
Michael Runge63f01de2014-10-28 19:24:19 -07001150 OPTIONS.verify = True
Doug Zongker26e66192014-02-20 13:22:07 -08001151 elif o == "--block":
1152 OPTIONS.block_based = True
Doug Zongker25568482014-03-03 10:21:27 -08001153 elif o in ("-b", "--binary"):
1154 OPTIONS.updater_binary = a
Tao Bao8dcf7382015-05-21 14:09:49 -07001155 elif o == "--stash_threshold":
1156 try:
1157 OPTIONS.stash_threshold = float(a)
1158 except ValueError:
1159 raise ValueError("Cannot parse value %r for option %r - expecting "
1160 "a float" % (a, o))
Tao Baod62c6032015-11-30 09:40:20 -08001161 elif o == "--log_diff":
1162 OPTIONS.log_diff = a
Tao Baodea0f8b2016-06-20 17:55:06 -07001163 elif o == "--payload_signer":
1164 OPTIONS.payload_signer = a
Baligh Uddin2abbbd02016-06-22 12:14:16 -07001165 elif o == "--payload_signer_args":
1166 OPTIONS.payload_signer_args = shlex.split(a)
Tianjie Xu21e6deb2019-10-07 18:01:00 -07001167 elif o == "--payload_signer_maximum_signature_size":
1168 OPTIONS.payload_signer_maximum_signature_size = a
xunchang376cc7c2019-04-08 23:04:58 -07001169 elif o == "--payload_signer_key_size":
Tianjie Xu21e6deb2019-10-07 18:01:00 -07001170 # TODO(Xunchang) remove this option after cleaning up the callers.
1171 logger.warning("The option '--payload_signer_key_size' is deprecated."
1172 " Use '--payload_signer_maximum_signature_size' instead.")
1173 OPTIONS.payload_signer_maximum_signature_size = a
Dan Willemsencea5cd22017-03-21 14:44:27 -07001174 elif o == "--extracted_input_target_files":
1175 OPTIONS.extracted_input = a
Tao Bao15a146a2018-02-21 16:06:59 -08001176 elif o == "--skip_postinstall":
1177 OPTIONS.skip_postinstall = True
Yifan Hong50e79542018-11-08 17:44:12 -08001178 elif o == "--retrofit_dynamic_partitions":
1179 OPTIONS.retrofit_dynamic_partitions = True
xunchangabfa2652019-02-19 16:27:10 -08001180 elif o == "--skip_compatibility_check":
1181 OPTIONS.skip_compatibility_check = True
xunchang1cfe2512019-02-19 14:14:48 -08001182 elif o == "--output_metadata_path":
1183 OPTIONS.output_metadata_path = a
Tianjie Xu1b079832019-08-28 12:19:23 -07001184 elif o == "--disable_fec_computation":
1185 OPTIONS.disable_fec_computation = True
Yifan Hong65afc072020-04-17 10:08:10 -07001186 elif o == "--force_non_ab":
1187 OPTIONS.force_non_ab = True
Tianjied6867162020-05-10 14:30:13 -07001188 elif o == "--boot_variable_file":
1189 OPTIONS.boot_variable_file = a
Yifan Hong38ab4d82020-06-18 15:19:56 -07001190 elif o == "--partial":
1191 partitions = a.split()
1192 if not partitions:
1193 raise ValueError("Cannot parse partitions in {}".format(a))
1194 OPTIONS.partial = partitions
Hongguang Chen49ab1b902020-10-19 14:15:43 -07001195 elif o == "--custom_image":
1196 custom_partition, custom_image = a.split("=")
1197 OPTIONS.custom_images[custom_partition] = custom_image
Doug Zongkereef39442009-04-02 12:14:19 -07001198 else:
1199 return False
Doug Zongkerdbfaae52009-04-21 17:12:54 -07001200 return True
Doug Zongkereef39442009-04-02 12:14:19 -07001201
1202 args = common.ParseOptions(argv, __doc__,
Tao Bao337633f2017-12-06 15:20:19 -08001203 extra_opts="b:k:i:d:e:t:2o:",
Dan Albert8b72aef2015-03-23 19:13:21 -07001204 extra_long_opts=[
Dan Albert8b72aef2015-03-23 19:13:21 -07001205 "package_key=",
1206 "incremental_from=",
Tao Bao43078aa2015-04-21 14:32:35 -07001207 "full_radio",
leozwangaa6c1a12015-08-14 10:57:58 -07001208 "full_bootloader",
Dan Albert8b72aef2015-03-23 19:13:21 -07001209 "wipe_user_data",
Tao Bao5d182562016-02-23 11:38:39 -08001210 "downgrade",
Tao Bao3e6161a2017-02-28 11:48:48 -08001211 "override_timestamp",
Dan Albert8b72aef2015-03-23 19:13:21 -07001212 "extra_script=",
1213 "worker_threads=",
Dan Albert8b72aef2015-03-23 19:13:21 -07001214 "two_step",
Tao Baof7140c02018-01-30 17:09:24 -08001215 "include_secondary",
Dan Albert8b72aef2015-03-23 19:13:21 -07001216 "no_signing",
1217 "block",
1218 "binary=",
1219 "oem_settings=",
Tao Bao8608cde2016-02-25 19:49:55 -08001220 "oem_no_mount",
Dan Albert8b72aef2015-03-23 19:13:21 -07001221 "verify",
Tao Bao8dcf7382015-05-21 14:09:49 -07001222 "stash_threshold=",
Tao Baod62c6032015-11-30 09:40:20 -08001223 "log_diff=",
Tao Baodea0f8b2016-06-20 17:55:06 -07001224 "payload_signer=",
Baligh Uddin2abbbd02016-06-22 12:14:16 -07001225 "payload_signer_args=",
Tianjie Xu21e6deb2019-10-07 18:01:00 -07001226 "payload_signer_maximum_signature_size=",
xunchang376cc7c2019-04-08 23:04:58 -07001227 "payload_signer_key_size=",
Dan Willemsencea5cd22017-03-21 14:44:27 -07001228 "extracted_input_target_files=",
Tao Bao15a146a2018-02-21 16:06:59 -08001229 "skip_postinstall",
Yifan Hong50e79542018-11-08 17:44:12 -08001230 "retrofit_dynamic_partitions",
xunchangabfa2652019-02-19 16:27:10 -08001231 "skip_compatibility_check",
xunchang1cfe2512019-02-19 14:14:48 -08001232 "output_metadata_path=",
Tianjie Xu1b079832019-08-28 12:19:23 -07001233 "disable_fec_computation",
Yifan Hong65afc072020-04-17 10:08:10 -07001234 "force_non_ab",
Tianjied6867162020-05-10 14:30:13 -07001235 "boot_variable_file=",
Yifan Hong38ab4d82020-06-18 15:19:56 -07001236 "partial=",
Hongguang Chen49ab1b902020-10-19 14:15:43 -07001237 "custom_image=",
Dan Albert8b72aef2015-03-23 19:13:21 -07001238 ], extra_option_handler=option_handler)
Doug Zongkereef39442009-04-02 12:14:19 -07001239
1240 if len(args) != 2:
1241 common.Usage(__doc__)
1242 sys.exit(1)
1243
Tao Bao32fcdab2018-10-12 10:30:39 -07001244 common.InitLogging()
1245
Tao Bao2db13852018-01-08 22:28:57 -08001246 # Load the build info dicts from the zip directly or the extracted input
1247 # directory. We don't need to unzip the entire target-files zips, because they
1248 # won't be needed for A/B OTAs (brillo_update_payload does that on its own).
1249 # When loading the info dicts, we don't need to provide the second parameter
1250 # to common.LoadInfoDict(). Specifying the second parameter allows replacing
1251 # some properties with their actual paths, such as 'selinux_fc',
1252 # 'ramdisk_dir', which won't be used during OTA generation.
Dan Willemsencea5cd22017-03-21 14:44:27 -07001253 if OPTIONS.extracted_input is not None:
Tao Bao2db13852018-01-08 22:28:57 -08001254 OPTIONS.info_dict = common.LoadInfoDict(OPTIONS.extracted_input)
Dan Willemsencea5cd22017-03-21 14:44:27 -07001255 else:
Kelvin Zhanga59bb272020-10-30 12:52:25 -04001256 OPTIONS.info_dict = ParseInfoDict(args[0])
Kelvin Zhang80195722020-11-04 14:38:34 -05001257 if OPTIONS.partial:
1258 OPTIONS.info_dict['ab_partitions'] = \
1259 list(
1260 set(OPTIONS.info_dict['ab_partitions']) & set(OPTIONS.partial)
1261 )
1262
1263 if OPTIONS.downgrade:
1264 # We should only allow downgrading incrementals (as opposed to full).
1265 # Otherwise the device may go back from arbitrary build with this full
1266 # OTA package.
1267 if OPTIONS.incremental_source is None:
1268 raise ValueError("Cannot generate downgradable full OTAs")
1269
Tao Baoc098e9e2016-01-07 13:03:56 -08001270
Yifan Hong38ab4d82020-06-18 15:19:56 -07001271 # TODO(xunchang) for retrofit and partial updates, maybe we should rebuild the
1272 # target-file and reload the info_dict. So the info will be consistent with
1273 # the modified target-file.
1274
Tao Bao32fcdab2018-10-12 10:30:39 -07001275 logger.info("--- target info ---")
1276 common.DumpInfoDict(OPTIONS.info_dict)
Tao Bao2db13852018-01-08 22:28:57 -08001277
Kelvin Zhang80195722020-11-04 14:38:34 -05001278
Tao Bao2db13852018-01-08 22:28:57 -08001279 # Load the source build dict if applicable.
1280 if OPTIONS.incremental_source is not None:
1281 OPTIONS.target_info_dict = OPTIONS.info_dict
Kelvin Zhanga59bb272020-10-30 12:52:25 -04001282 OPTIONS.source_info_dict = ParseInfoDict(OPTIONS.incremental_source)
Tao Bao2db13852018-01-08 22:28:57 -08001283
Tao Bao32fcdab2018-10-12 10:30:39 -07001284 logger.info("--- source info ---")
1285 common.DumpInfoDict(OPTIONS.source_info_dict)
Tao Bao2db13852018-01-08 22:28:57 -08001286
1287 # Load OEM dicts if provided.
Tao Bao481bab82017-12-21 11:23:09 -08001288 OPTIONS.oem_dicts = _LoadOemDicts(OPTIONS.oem_source)
1289
Yifan Hong50e79542018-11-08 17:44:12 -08001290 # Assume retrofitting dynamic partitions when base build does not set
Yifan Hong50611032018-11-20 14:27:38 -08001291 # use_dynamic_partitions but target build does.
Yifan Hong50e79542018-11-08 17:44:12 -08001292 if (OPTIONS.source_info_dict and
Yifan Hong50611032018-11-20 14:27:38 -08001293 OPTIONS.source_info_dict.get("use_dynamic_partitions") != "true" and
Kelvin Zhang39aea442020-08-17 11:04:25 -04001294 OPTIONS.target_info_dict.get("use_dynamic_partitions") == "true"):
Yifan Hong50e79542018-11-08 17:44:12 -08001295 if OPTIONS.target_info_dict.get("dynamic_partition_retrofit") != "true":
1296 raise common.ExternalError(
1297 "Expect to generate incremental OTA for retrofitting dynamic "
1298 "partitions, but dynamic_partition_retrofit is not set in target "
1299 "build.")
1300 logger.info("Implicitly generating retrofit incremental OTA.")
1301 OPTIONS.retrofit_dynamic_partitions = True
1302
1303 # Skip postinstall for retrofitting dynamic partitions.
1304 if OPTIONS.retrofit_dynamic_partitions:
1305 OPTIONS.skip_postinstall = True
1306
Tao Baoc098e9e2016-01-07 13:03:56 -08001307 ab_update = OPTIONS.info_dict.get("ab_update") == "true"
Yifan Hong65afc072020-04-17 10:08:10 -07001308 allow_non_ab = OPTIONS.info_dict.get("allow_non_ab") == "true"
1309 if OPTIONS.force_non_ab:
1310 assert allow_non_ab, "--force_non_ab only allowed on devices that supports non-A/B"
1311 assert ab_update, "--force_non_ab only allowed on A/B devices"
1312
1313 generate_ab = not OPTIONS.force_non_ab and ab_update
Tao Baoc098e9e2016-01-07 13:03:56 -08001314
Christian Oderf63e2cd2017-05-01 22:30:15 +02001315 # Use the default key to sign the package if not specified with package_key.
1316 # package_keys are needed on ab_updates, so always define them if an
Yifan Hong65afc072020-04-17 10:08:10 -07001317 # A/B update is getting created.
1318 if not OPTIONS.no_signing or generate_ab:
Christian Oderf63e2cd2017-05-01 22:30:15 +02001319 if OPTIONS.package_key is None:
1320 OPTIONS.package_key = OPTIONS.info_dict.get(
1321 "default_system_dev_certificate",
Dan Willemsen0ab1be62019-04-09 21:35:37 -07001322 "build/make/target/product/security/testkey")
Christian Oderf63e2cd2017-05-01 22:30:15 +02001323 # Get signing keys
1324 OPTIONS.key_passwords = common.GetKeyPasswords([OPTIONS.package_key])
1325
Yifan Hong65afc072020-04-17 10:08:10 -07001326 if generate_ab:
Tao Baof0c4aa22018-04-30 20:29:30 -07001327 GenerateAbOtaPackage(
Tao Baoc098e9e2016-01-07 13:03:56 -08001328 target_file=args[0],
1329 output_file=args[1],
1330 source_file=OPTIONS.incremental_source)
1331
Dan Willemsencea5cd22017-03-21 14:44:27 -07001332 else:
Tao Baof0c4aa22018-04-30 20:29:30 -07001333 GenerateNonAbOtaPackage(
1334 target_file=args[0],
1335 output_file=args[1],
1336 source_file=OPTIONS.incremental_source)
Doug Zongkerfdd8e692009-08-03 17:27:48 -07001337
Tao Baof0c4aa22018-04-30 20:29:30 -07001338 # Post OTA generation works.
1339 if OPTIONS.incremental_source is not None and OPTIONS.log_diff:
1340 logger.info("Generating diff logs...")
1341 logger.info("Unzipping target-files for diffing...")
1342 target_dir = common.UnzipTemp(args[0], TARGET_DIFFING_UNZIP_PATTERN)
1343 source_dir = common.UnzipTemp(
1344 OPTIONS.incremental_source, TARGET_DIFFING_UNZIP_PATTERN)
Doug Zongkereb0a78a2014-01-27 10:01:06 -08001345
Tao Baof0c4aa22018-04-30 20:29:30 -07001346 with open(OPTIONS.log_diff, 'w') as out_file:
Tao Baof0c4aa22018-04-30 20:29:30 -07001347 target_files_diff.recursiveDiff(
1348 '', source_dir, target_dir, out_file)
Doug Zongker62d4f182014-08-04 16:06:43 -07001349
Tao Bao32fcdab2018-10-12 10:30:39 -07001350 logger.info("done.")
Doug Zongkereef39442009-04-02 12:14:19 -07001351
1352
1353if __name__ == '__main__':
1354 try:
Ying Wang7e6d4e42010-12-13 16:25:36 -08001355 common.CloseInheritedPipes()
Doug Zongkereef39442009-04-02 12:14:19 -07001356 main(sys.argv[1:])
Tao Bao32fcdab2018-10-12 10:30:39 -07001357 except common.ExternalError:
1358 logger.exception("\n ERROR:\n")
Doug Zongkereef39442009-04-02 12:14:19 -07001359 sys.exit(1)
Doug Zongkerfc44a512014-08-26 13:10:25 -07001360 finally:
1361 common.Cleanup()