mirror of
https://github.com/awalsh128/cache-apt-pkgs-action.git
synced 2026-11-14 05:46:50 +00:00
a84878c774
* ci: bundle the action with rollup and upload it for integration jobs The integration jobs download an action-bundle-<sha> artifact that nothing built or uploaded, so all of them failed on their first step. v2 also had no bundle: action.yml runs dist/index.js, and an action has no node_modules at runtime. - Add rollup.config.mjs, which bundles the tsc output (dist/src) into a self-contained dist/index.js. npm run build now runs it. - Fix the setup action, which never built: with if_no_artifact_found: ignore the download step always succeeds, so check found_artifact. Download cached builds into dist/, and overwrite the build artifact since several jobs in a run can build it. - Upload action.yml and the bundle as action-bundle-<sha> from setup-and-cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chor: remove stale file * fix: write logs under the home cache directory and await artifact upload Instruments opened its log files at the relative path cache-apt-pkgs/ before that directory existed, so the action crashed at startup with ENOENT. It also pointed artifact uploads there while Cache uses ~/cache-apt-pkgs. Resolve both with a shared resolveCacheDir helper and create the directory before opening the logs. The artifact upload was also never awaited, so a failed upload became an unhandled rejection. Await it and report failures as a warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: ship ts-apt's apt-fast script with the bundle On runners with aria2 installed, ts-apt uses apt-fast and spawns scripts/dist/apt-fast.sh relative to its own module. Bundling moved that module, so the action failed with "spawn .../scripts/dist/apt-fast.sh ENOENT". Write the bundle to dist/action/index.js and copy ts-apt's scripts/dist to dist/scripts/dist, which is where ts-apt looks from there. Point action.yml at the new entrypoint and include both in the action bundle artifact. Workflow artifacts drop file modes, so the action marks the script executable at startup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build: pin ts-apt to e627c3b for install timeout and dpkg-query fixes ts-apt killed every apt command after 30 seconds, so installs failed with exit code 124, and listInstalledFiles passed name=version to dpkg-query -L, which rejects it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: make diagnostic artifact names unique per step Every job in a run uploaded capa_artifacts-<run id>, so only the first upload succeeded. Add the run attempt, job and step IDs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run cache restore jobs after the jobs that create the cache The restore and add-package jobs all only needed testing, so they ran in parallel with standard_workflow_install and raced on the same cache key. Restore the ordering v1 uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: authenticate the action pin check to avoid API rate limits The check called the GitHub API anonymously and hit the rate limit for the shared runner IP. The script uses GITHUB_TOKEN when it's set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build: pin ts-apt to 342c7d0 for native arch and stdin input Adds PackageManager.getNativeArchitecture and CommandOptions.input, used by the version-list outputs and archive creation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: report requested and dependency package versions separately package-version-list held every installed package and both outputs were identical. Outputs also carried arch suffixes inconsistently and followed install order. - package-version-list lists only requested packages; all-package-version-list lists every installed package. - Entries are name=version, with :arch only for packages installed for a non-native architecture (e.g. libc6:i386), so native output matches v1. - Entries are sorted with PackageName.compareTo. - Requested packages are matched by name, and by arch when both specify one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: archive and restore packages with system tar under sudo The in-process tar library ran as the runner user, so archiving failed on root-only files such as /etc/default/cacerts from ca-certificates-java (regression #72), and extraction could skip root-owned paths without failing. - Create and extract with sudo tar; the file list is piped NUL-separated on stdin so large packages don't exceed the argument size limit. - Keep directory symlinks such as /lib on extract. - Hand archives back to the runner user for the cache save. - Disable the command timeout for tar and maintainer scripts. - Record the installed (versioned) entries in manifest_main.json; it held unversioned names with no files. - Return every installed package on a cache hit so outputs include deps. - Split runInstallScripts into per-script runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: update package expectations for Ubuntu 24.04 libasound2 is libasound2t64 on 24.04 (regression #81), and gir1.2-gdkpixbuf-2.0 moved to 2.42.10+dfsg-3ubuntu3.3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: add ts-apt boundary rules to AGENTS.md APT and dpkg logic belongs in ts-apt; ts-apt changes are pushed before the action bumps its pin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: pin version-list jobs to ubuntu-24.04 and fix spaces-in-paths tests - list_versions / list_all_versions: run on ubuntu-24.04. The expected versions are from 24.04, but ubuntu-latest now resolves newer ones (e.g. rolldice=1.16-1build4, xdot=1.6-1). - regression_76: cache libopenal-data ("Default HRTF.mhr") instead of Intel oneAPI. Drops the Intel repo and the unused opencl and libsundials-dev installs (188 MB, 155 packages), which made the job take 75+ minutes when the Ubuntu mirror was slow. Cap at 15 minutes. - regression_81: list alsa-ucm-conf explicitly so the test caches a package with spaces in its file paths. Same fixes as #235 and #236 on master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
187 lines
10 KiB
Markdown
187 lines
10 KiB
Markdown
<img src="logo.png" alt="Description" height="80" align="center" style="padding-right: 1em;"> <span style="font-size: 2em; font-weight: bold;">Cache APT Packages Action v2</span>
|
|
|
|
[](https://github.com/awalsh128/fluentcpp/blob/master/LICENSE)
|
|
|
|
|
|
[](https://codecov.io/gh/awalsh128/cache-apt-pkgs-action)
|
|
|
|
This action allows caching of Advanced Package Tool (APT) package dependencies to improve workflow execution time instead of installing the packages on every run.
|
|
|
|
For more information on changes in `v2` see the [Version 2 FAQ](V2_FAQ.MD)
|
|
|
|
> [!TIP]
|
|
> If you find this project useful, please consider supporting it [as a sponsor and show some ❤️ for open source maintainers](https://github.com/sponsors/awalsh128).
|
|
|
|
> [!NOTE]
|
|
> Looking for co-maintainers to help review changes, and investigate issues. I haven't had as much time to stay on top of this action as I would like to and want to make sure it is still responsive and reliable for the community. If you are interested, please reach out.
|
|
|
|
## Documentation
|
|
|
|
This action is a composition of [actions/cache](https://github.com/actions/cache/) and the [ts-apt](http://github.com/awalsh128/ts-apt) library. Some actions require additional APT based packages to be installed in order for other steps to be executed. Packages can be installed when ran but can consume much of the execution workflow time.
|
|
|
|
## Usage
|
|
|
|
### Pre-requisites
|
|
|
|
Create a workflow `.yml` file in your repositories `.github/workflows` directory. An [example workflow](#example-workflow) is available below. For more information, reference the GitHub Help Documentation for [Creating a workflow file](https://help.github.com/en/articles/configuring-a-workflow#creating-a-workflow-file).
|
|
|
|
### Versions
|
|
|
|
There are three kinds of version labels you can use.
|
|
|
|
- `@latest` - This will give you the latest release of **v1, not v2**.
|
|
- `@latest-v2` - This will give you the latest release of **v2**.
|
|
- `@v#` - Major only will give you the latest release for that major version only (e.g. `v1`).
|
|
- Branch
|
|
- version 1.x
|
|
- `@master` - Most recent manual and automated tested code. Possibly unstable since it is pre-release (publishes as `1.x.x`).
|
|
- `@staging` - Most recent automated tested code and can sometimes contain experimental features (no publish).
|
|
- version 2.x
|
|
- `@main-2` - Live and stable version, pre-production before action release (publishes as `2.x.x`).
|
|
- `@staging-2` - Possibly unstable and can sometimes contain experimental features (publishes as `2.x.x-rc#`).
|
|
|
|
### Inputs
|
|
|
|
- `packages` - Space delimited list of packages to install.
|
|
- `version` - Version of cache to load. Each version will have its own cache. Note, all characters except spaces are allowed.
|
|
- `execute_install_scripts` - Execute Debian package pre and post install script upon restore. See [Caveats / Non-file Dependencies](#non-file-dependencies) for more information.
|
|
- `empty_packages_behavior` - Desired behavior when the given `packages` is empty. `'error'` (default), `'warn'` or `'ignore'`.
|
|
|
|
### Outputs
|
|
|
|
- `cache-hit` - A boolean value to indicate a cache was found for the packages requested.
|
|
- `package-version-list` - The main requested packages and versions that are installed. Represented as a sorted comma delimited list with equals delimit on the package version (i.e. \<package1>=\<version1>,\<package2>=\<version2>,...). Packages installed for a non-native architecture include it (i.e. \<package>:\<arch>=\<version>).
|
|
- `all-package-version-list` - All the pulled in packages and versions, including dependencies, that are installed. Represented as a sorted comma delimited list with equals delimit on the package version (i.e. \<package1>=\<version1>,\<package2>=\<version2>,...). Packages installed for a non-native architecture include it (i.e. \<package>:\<arch>=\<version>).
|
|
|
|
### Security Compliance
|
|
|
|
This action runs as a JavaScript GitHub Action on the `node24` runtime and does not require consumers to run `npm install` in their workflow in order to use it. The implementation dependency surface is limited to the action runtime packages declared in `package.json`: `@actions/cache`, `@actions/core`, `tar`, `winston`, and `ts-apt`.
|
|
|
|
For workflows with stricter compliance requirements, the main security characteristics are:
|
|
|
|
- `packages` should be treated as an allowlisted input in your workflow. Prefer explicit package names and versions where reproducibility matters.
|
|
- `version` can be used as a cache namespace so you can intentionally rotate caches when package policy or runner baselines change.
|
|
- `empty_packages_behavior` can be left at the default `error` to fail closed when an expected package list is missing.
|
|
- `execute_install_scripts` is disabled by default. Enable it only when required, because Debian maintainer scripts execute arbitrary package-provided shell logic during restore.
|
|
- `debug` is disabled by default. Enable it only for investigation and review logs before sharing them outside your organization.
|
|
|
|
#### Features
|
|
|
|
Security-relevant action features:
|
|
|
|
- Package inputs are normalized before cache lookup, which reduces accidental cache divergence from ordering, commas, backslashes, or duplicate whitespace in the package list.
|
|
- Unpinned package names are resolved to concrete package versions before the cache key is generated, which improves traceability of what was actually cached for a run.
|
|
- Cache keys are derived from the normalized package set, the user-provided `version`, and the runner architecture, which helps isolate caches across package changes and incompatible platforms.
|
|
- The action rejects invalid boolean inputs and rejects `version` values containing spaces, reducing ambiguous workflow configuration.
|
|
- When creating archives, the action records installed package manifests and only archives existing files and symlinks from installed packages, plus maintainer scripts when present.
|
|
- The `package-version-list` and `all-package-version-list` outputs can be captured by downstream workflow steps for audit logs, attestation inputs, or compliance reporting.
|
|
|
|
#### Usage Recommendations
|
|
|
|
For GitHub Actions workflow hardening, prefer the following controls around this action:
|
|
|
|
- Pin this action to a major version you trust, or to a full commit SHA for stricter supply chain control.
|
|
- Use the minimum required workflow `permissions` instead of broad defaults.
|
|
- Run on GitHub-hosted or otherwise trusted runners with controlled APT sources.
|
|
- Limit who can modify workflow files and package lists through branch protection and pull request review.
|
|
- Review whether cached APT contents are acceptable for your repository's cache retention policy and data handling requirements.
|
|
|
|
Example hardened usage:
|
|
|
|
```yaml
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
build:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: awalsh128/cache-apt-pkgs-action@v2
|
|
with:
|
|
packages: curl=8.5.0-2ubuntu10.6 jq=1.7.1-3build1
|
|
version: ubuntu-24.04-v1
|
|
empty_packages_behavior: error
|
|
execute_install_scripts: false
|
|
```
|
|
|
|
### Cache scopes
|
|
|
|
The cache is scoped to the packages given and the branch. The default branch cache is available to other branches.
|
|
|
|
### Example workflow
|
|
|
|
This was a motivating use case for creating this action.
|
|
|
|
```yaml
|
|
name: Create Documentation
|
|
on: push
|
|
jobs:
|
|
build_and_deploy_docs:
|
|
runs-on: ubuntu-latest
|
|
name: Build Doxygen documentation and deploy
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: awalsh128/cache-apt-pkgs-action@latest
|
|
with:
|
|
packages: dia doxygen doxygen-doc doxygen-gui doxygen-latex graphviz mscgen
|
|
version: 1.0
|
|
|
|
- name: Build
|
|
run: |
|
|
cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}
|
|
cmake --build ${{github.workspace}}/build --config ${{env.BUILD_TYPE}}
|
|
|
|
- name: Deploy
|
|
uses: JamesIves/github-pages-deploy-action@4.1.5
|
|
with:
|
|
branch: gh-pages
|
|
folder: ${{github.workspace}}/build/website
|
|
```
|
|
|
|
```yaml
|
|
---
|
|
install_doxygen_deps:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: awalsh128/cache-apt-pkgs-action@latest
|
|
with:
|
|
packages: dia doxygen doxygen-doc doxygen-gui doxygen-latex graphviz mscgen
|
|
version: 1.0
|
|
```
|
|
|
|
## Caveats
|
|
|
|
### Non-file Dependencies
|
|
|
|
This action is based on the principle that most packages can be cached as a fileset. There are situations though where this is not enough.
|
|
|
|
- Pre and post installation scripts needs to be ran from `/var/lib/dpkg/info/{package name}.[preinst, postinst]`.
|
|
- The Debian package database needs to be queried for scripts above (i.e. `dpkg-query`).
|
|
|
|
The `execute_install_scripts` argument can be used to attempt to execute the install scripts but they are no guaranteed to resolve the issue.
|
|
|
|
```yaml
|
|
- uses: awalsh128/cache-apt-pkgs-action@latest
|
|
with:
|
|
packages: mypackage
|
|
version: 1.0
|
|
execute_install_scripts: true
|
|
```
|
|
|
|
If this does not solve your issue, you will need to run `apt-get install` as a separate step for that particular package unfortunately.
|
|
|
|
```yaml
|
|
run: apt-get install mypackage
|
|
shell: bash
|
|
```
|
|
|
|
Please reach out if you have found a workaround for your scenario and it can be generalized. There is only so much this action can do and can't get into the area of reverse engineering Debian package manager. It would be beyond the scope of this action and may result in a lot of extended support and brittleness. Also, it would be better to contribute to Debian packager instead at that point.
|
|
|
|
For more context and information see [issue #57](https://github.com/awalsh128/cache-apt-pkgs-action/issues/57#issuecomment-1321024283) which contains the investigation and conclusion.
|
|
|
|
### Cache Limits
|
|
|
|
A repository can have up to 5GB of caches. Once the 5GB limit is reached, older caches will be evicted based on when the cache was last accessed. Caches that are not accessed within the last week will also be evicted. To get more information on how to access and manage your actions's caches, see [GitHub Actions / Using workflows / Cache dependencies](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#viewing-cache-entries).
|