Commit graph

600 commits

Author SHA1 Message Date
Dmitry Muhomor
9293b9d453 Revert "lower lynx SPL to 2023-09-01"
This reverts commit c7dafe9a0b.
2023-10-04 13:37:31 -04:00
Dmitry Muhomor
6ac4ae9191 Revert "lower 5th gen Pixel SPL to 2023-09-01"
This reverts commit 464ac24d8e.
2023-10-04 13:37:31 -04:00
Dmitry Muhomor
dc0acf27a1 collect-state: add an option to automatically make prep OS build 2023-10-04 10:13:37 -04:00
Dmitry Muhomor
a306cd2272 collect-state: fix the output file name format 2023-10-04 10:13:37 -04:00
Dmitry Muhomor
e2d0305460 download: add an option to unpack factory images 2023-10-04 10:13:37 -04:00
Dmitry Muhomor
d70d036a53 add index of Pixel Android 14 QPR1 Beta 1 builds 2023-10-04 10:13:37 -04:00
Dmitry Muhomor
814ce97b60 update vendor specs for devices with newly lowered SPL 2023-10-03 03:03:38 -04:00
Dmitry Muhomor
c7dafe9a0b lower lynx SPL to 2023-09-01 2023-10-03 02:47:09 -04:00
Dmitry Muhomor
464ac24d8e lower 5th gen Pixel SPL to 2023-09-01 2023-10-03 02:47:09 -04:00
Dmitry Muhomor
ec80c66a15 23-09: update build-id-to-tag index 2023-09-18 16:11:45 -04:00
Dmitry Muhomor
3ad4fbf6f0 correctly set prev_build_id property for sunfish and felix
This property is useful for doing manual comparisons.
2023-09-18 15:59:28 -04:00
Daniel Micay
b9a09595ed update vendor specs for September 2023 Android 13 release 2023-09-18 15:59:20 -04:00
Dmitry Muhomor
4bd8839b7d update configs for September 2023 Android 13 release 2023-09-18 14:20:48 -04:00
Dmitry Muhomor
852f8be352 add index of Pixel Android 14 Beta 5.3 builds 2023-09-18 14:20:48 -04:00
Dmitry Muhomor
1ceb322463 add command for fetching info about stock OS kernel from its repo 2023-09-09 02:43:06 -04:00
Dmitry Muhomor
cfbfaf7010 make kernel repo URL part of device config 2023-09-09 02:43:06 -04:00
Dmitry Muhomor
e21ac2fa2a add docs/usage.md 2023-08-31 16:03:43 -04:00
Dmitry Muhomor
0691a32fa6 generate-all: pass extra commands to lunch/choosecombo in envsetup.sh
Requires the corresponding patch in build/make repo.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
c8f93555c0 filter out several unused overlays without parsing them
Vendor module trees remain the same.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
5f80831d88 update vendor module specs after resource overlay changes
Notable changes:
- multiple previously-unparseable overlays are added for felix and tangorpro
- fingerprint setup strings are no longer cut-off at first line
- udfps_lottie_translate_y dimension is now parsed correctly
- "Face unlock"-related strings are removed ("Face unlock" is not shipped)
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
29cd63fdda update overlay filters for felix and tangorpro
Previously-removed overlays no longer break the resource parser.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
c036c79a5e filter out HbmSVManager overlay on all devices
HbmSVManager ("high brightness mode manager") itself is not shipped, even though this overlay
no longer breaks resource parser.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
55c6f0db6b rewrite aapt2-based overlay parser
Previous parser relied on text output of `aapt2/ dump xmltree` and `aapt2 dump resources` commands,
which is not meant for parsing.

This led to multiple parsing bugs: cutoff multiline strings, floating point parsing issues,
quoting issues, non-English locale issues etc.

Rewritten parser instead relies on protobuf aapt2 output, which allows to resolve most of these
issues.

A simple patch to aapt2 in frameworks/base is required to slightly expand its protobuf output.

Other notable changes:
- support for the "fraction" resource type
- parsing is now parallelized at APK level
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
2543191d23 add ts-proto output for Resources.proto from aapt2 2023-08-31 16:03:43 -04:00
Dmitry Muhomor
386f1710fe add ts-proto dependency
It allows to use aapt2 resource protobuf types.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
adec586797 add protobufjs dependency
It's needed for parsing protobuf aapt2 output.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
e46c929e60 remove @types/hasha from devDependencies
It's not neeeded, hasha includes types already.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
456fddae1b remove product/etc/CarrierSettings/ from dep_files
Inline carrier settings extraction is no longer used.

This directory is not filtered out elsewhere, it will still be included in full (non-prep) vendor
modules. However, collected system state files that were generated prior to this commit need to be
regenerated, since they list these files as included in AOSP due to the way dep_files directive
works.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
86c7472066 do not use BUILD_BROKEN_ELF_PREBUILT_PRODUCT_COPY_FILES 2023-08-31 16:03:43 -04:00
Dmitry Muhomor
7403e95179 add vendor-{specs,skels} for all supported devices 2023-08-31 16:03:43 -04:00
Dmitry Muhomor
cb4a7e34b9 update gitignore
android-info.txt files are about to be added to vendor-skels/.

/dl is useful to have as a symlink.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
3416e4a554 generate-all: verify the result against a reference FileTreeSpec
This commit adds an extra step to generate-all:
- if --updateSpec flag is specified, FileTreeSpec of the generated vendor module is written out
to vendor-specs/$VENDOR/$DEVICE.yaml file ("reference FileTreeSpec").
- if --updateSpec is not specified, FileTreeSpec of the generated vendor module is compared against
the reference FileTreeSpec. If there are any differences, generated vendor module is deleted and an
exception is thrown.

Also, when --updateSpec flag is used, text files inside vendor module are written out to
vendor-skels/$VENDOR/$DEVICE directory.

Contents of vendor-skels/ are useful for keeping track of vendor module changes via git and for
showing diffs in case text files in the generated module don't match the reference ones.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
73c1f665f3 add code for generating and comparing FileTreeSpecs 2023-08-31 16:03:43 -04:00
Dmitry Muhomor
c84d027c23 update links to adevtool GitHub 2023-08-31 16:03:43 -04:00
Dmitry Muhomor
bcc612c622 sort items in generated overlay resource files
This makes them easier to read.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
5c9e695863 reorder PRODUCT_{PACKAGES,COPY_FILES} in device makefile
These directives are usually very large and are less interesting than PRODUCT_<partition>_PROPERTIES,
PRODUCT_OVERRIDE_FINGERPRINT variables. Move them to the bottom to improve readability.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
2aaa970886 add show-status command
It shows BUILD_ID, AOSP tag, platform security patch level and stock OS image availability for
each of specified devices.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
3bf5977e81 add command for making "BUILD_ID to AOSP tag" map 2023-08-31 16:03:43 -04:00
Dmitry Muhomor
dcc8339fd7 update generate-all command
- pass device config with the standard -d flag instead of through argument
- default to using factory and OTA images for current build ID when --stockSrc is not specified
- add extract_android_ota_payload.py as the final step, with an option to skip it
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
12e586385f update generate-prep command
- pass device config with the standard -d flag instead of through argument. This allows to specify
multiple devices without using DeviceList config file
- default to using factory image for current build ID when --stockSrc is not specified
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
97b724790b improve handling of device factory images
- support unpacking factory images without root by using debugfs rdump (ext4) and
fsck.erofs --extract (erofs)
- support reusing unpacked factory images between adevtool runs to reduce latency
- unpack factory images for multiple devices in parallel. Extraction of files from partition images
is parallelized too
- extract partition images from factory image inner zip directly, without extracting it, since this
inner zip is uncompressed
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
9750841f3d collect-state: support specifying device names with DeviceList(s)
For example, `collect-state -d pixel-gen{6,7} barbet`
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
40354ec422 collect-state: set default output path to vendor/state 2023-08-31 16:03:43 -04:00
Dmitry Muhomor
2a3fba5d2e generate-all: set default of customSrc flag to vendor/state 2023-08-31 16:03:43 -04:00
Dmitry Muhomor
22b2d700db add helper functions for async spawning of processes 2023-08-31 16:03:43 -04:00
Dmitry Muhomor
9be2a36aa5 add hasha dependency
This is a simple wrapper for built-in node.js hashing APIs.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
40abb08329 add yauzl-promise dependency
This is a zip unpacking library.

unzipit library that is already present doesn't support extracting from inner factory image zips
without unpacking them and extracts files to memory first, which consumes huge amount of memory
when unpacking factory images, especially in parallel.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
8d8604dbe7 check PLATFORM_SECURITY_PATCH value in product Makefile
On devices that no longer receive vendor updates (e.g. Pixel 4 / 4 XL), PLATFORM_SECURITY_PATCH
should be overriden to correctly indicate that.
2023-08-31 16:03:43 -04:00
Dmitry Muhomor
70129836eb check in product Makefile that BUILD_ID is set correctly 2023-08-31 16:03:43 -04:00
Dmitry Muhomor
0b0b76ef70 do not use bright green color for action spinner text
It has bad readability when using dark-on-light terminal color schemes.
2023-08-31 16:03:43 -04:00