Compare commits

...

18 Commits

Author SHA1 Message Date
bircni
68547886a5 feat: wait for healthy services and fill the job context (#1107)
Service containers were started and then left alone, so a job's first step could run while a database was still starting up. The runner now waits for every service whose image or `options` declare a healthcheck, as GitHub does. An unhealthy service fails the job with its container log, one that never becomes healthy fails it after `container.service_ready_timeout` (default `5m`, negative disables the wait), and one that exits without a healthcheck only gets its log and a warning.

The started containers also fill the `job` context, whose fields existed but were never populated: `job.container.{id,network}` and `job.services.<id>.{id,network,ports}`. `ports` is keyed by the plain container port, so `job.services.postgres.ports['5432']` resolves to the host port Docker picked.

---------

Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1107
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-08-05 19:46:43 +00:00
bircni
8700adc933 feat: config command to edit config files (#1140)
Changing a setting after `generate-config` meant hand-editing YAML, which is awkward in provisioning scripts. `config` now edits an existing file in place:

```bash
./gitea-runner -c config.yaml config set runner.capacity 4
./gitea-runner -c config.yaml config set runner.envs.MY_VAR value
./gitea-runner -c config.yaml config add runner.labels 'ubuntu:docker://node:22'
./gitea-runner -c config.yaml config remove runner.labels 'ubuntu:docker://node:22'
./gitea-runner -c config.yaml config get runner.labels
```

Edits go through the YAML node tree, so comments, key order and the blank lines between top-level sections survive — a test asserts that appending a label to `config.example.yaml` changes nothing but the added line. Keys are resolved by reflecting over the `Config` struct's yaml tags, so an unknown key or a value of the wrong type is rejected before the file is touched. The write is atomic and keeps the file's symlink, owner, mode and line endings.

---------

Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1140
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-08-05 16:48:25 +00:00
bircni
b70ff6893a feat: gate set-env/add-path and render annotation locations (#1109)
`::set-env::` and `::add-path::` let a step rewrite the environment of every later step from its own output, which the runner honoured silently. They are now refused, as GitHub has done since 2020, and `ACTIONS_ALLOW_UNSECURE_COMMANDS` opts back in per step or job. Support for that variable is new here too, and is the only opt-in, matching GitHub rather than adding a runner config key on top.

Annotations keep their source location: Gitea has no annotation store and its web UI strips command properties, so `::error file=main.go,line=12::msg` is rendered as `::error::main.go:12: msg`.

`DEVELOPMENT.md` writes down the log line encoding rules this relies on.

---------

Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1109
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-08-05 16:43:17 +00:00
silverwind
3618385b28 fix: serve the whole results service from the cache server (#1141)
`ACTIONS_RESULTS_URL` names one origin serving every `github.actions.results.api.v1` service. Gitea serves the artifact half and this runner the cache half, so announcing `ACTIONS_CACHE_SERVICE_V2` while that URL pointed at Gitea was a promise the environment could not keep, and `docker buildx` posted its cache calls at Gitea and got a 404.

The cache server now forwards the artifact half to the instance each job registers with, so it is the whole results service and jobs are pointed at it. The announcement follows, and the bundle patch follows the cache URL instead.

Also fixes three things no JavaScript client reached: camelCase in the v2 responses where the Go clients read proto names, the missing `x-ms-request-id` on blob uploads that panics buildkit, and `cache.external_server` passed through without the trailing slash the v1 client concatenates onto.

Tests run the real actions against the services they look for: `actions/cache` over both API versions, the artifact actions up and back down through the forwarding, and `setup-node`. The regression itself is covered by asserting that whatever a job is handed as `ACTIONS_RESULTS_URL` answers a cache service call.

Fixes https://gitea.com/gitea/runner/issues/1139

Reviewed-on: https://gitea.com/gitea/runner/pulls/1141
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
2026-08-03 16:41:53 +00:00
Renovate Bot
55a625f733 chore(deps): update dependencies (#1138)
Co-authored-by: Renovate Bot <renovate-bot@gitea.com>
2026-08-03 09:22:17 +00:00
silverwind
47366f8f34 fix: coerce every expression value kind to a string and drop the format() panic (#1135)
`coerceToString` covered only a few kinds and returned the unconverted `reflect.Value` for the rest, so callers rendered contexts as `<*model.GithubContext Value>` and mangled sized integers and floats the same way. It now returns a string, which makes that placeholder unrepresentable, and is exported as `CoerceToString` so Gitea can drop its own diverging copy. It also takes an already reflected value, so internal callers need no conversion and no guard against the zero `Value`.

`format()` panicked whenever a lone `}` was followed by anything other than another `}`. Only a trailing `}` reached the existing unmatched-brace check, so an expression such as `format('a}b')`, which any workflow can write, took the process down instead. It now returns that same error.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1135
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-08-02 22:18:20 +00:00
silverwind
b7aeda6e7f docs: sync AGENTS.md with gitea/gitea (#1134)
Carries over the applicable rules from https://gitea.com/gitea/gitea `AGENTS.md`, including the Conventional Commits and `make lint-go-windows` requirements this repo already enforces but never documented for agents. Swaps `Co-Authored-By` for the `Assisted-by` trailer.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1134
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-08-02 20:29:08 +00:00
silverwind
aced51b4d5 chore: drop Masterminds/semver and pkg/errors, re-sync the docker/cli copies (#1132)
1. Drop `Masterminds/semver`, the Docker API version check needs dotted-numeric comparison and moby's client ships `versions.GreaterThanOrEqualTo`.
1. A malformed API version now reports as unsupported instead of panicking.
1. Drop `pkg/errors`, nothing used stack traces, `Wrap` or `Cause`.
1. Drop its depguard rule, and the `io/ioutil` one that only masked staticcheck's `SA1019`.
1. Re-sync `act/container/docker_cli.go` and `act/container/docker_cli_test.go`, copies of docker/cli's `opts.go` and `opts_test.go`, from a March and a 2022 commit to the version go.mod pins.
1. Record the local deviations in each header.
1. Fix `invalidParameter`, which lacked `Unwrap`, hiding the wrapped cause from `errors.Is` and `errors.As`.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1132
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-08-02 17:36:31 +00:00
silverwind
34bfa19150 fix: resolve symlinked container paths before building tar entries (#1130)
Docker 29.7 extracts copied archives through `os.Root`, which refuses to follow a symlink to an absolute target, so copying into `/var/run/act` fails with `path escapes from parent` on the many images that link `/var/run` to `/run`. The daemon now resolves every path component for us before any tar entry name is built, and the destination is created with one directory entry per missing component, which no daemon version rejects.

Verified against real daemons (29.4.0, 29.5.0, 29.5.1, 29.5.3, 29.6.2, 29.7.0-rc.1) with `debian:bookworm` (absolute symlink) and `alpine:3` (relative symlink), and against `moby/go-archive` v0.2.0 through the pending fix branch.

1. Fixes https://gitea.com/gitea/runner/issues/1128
1. Upstream bug: https://github.com/moby/moby/issues/53258
1. Supersedes the no-op change in https://gitea.com/gitea/runner/pulls/1129, which cannot help since the daemon strips leading slashes itself

Reviewed-on: https://gitea.com/gitea/runner/pulls/1130
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-07-31 16:09:27 +00:00
bircni
96d9f491db fix: strip leading slash from mkdir tarball in CopyTarStream (#1129)
`CopyTarStream` creates the destination directory by extracting a one-entry tarball with `DestinationPath: "/"`, but named that entry with the absolute `destPath`. Docker Engine 29.5+ tightened path validation on the copy API and rejects absolute entry names against a `/` destination with `statat var/run/act/actions/<sha>: path escapes from parent`, so the action directory never reached the job container and `actions/checkout` failed during "Set up job". Stripping the leading slash makes the entry relative, matching what the sibling `copyDir` already does and the upstream fix in nektos/act v0.2.89. Adds a regression test asserting the mkdir tarball entry is relative.

 Fixes #1128

Reviewed-on: https://gitea.com/gitea/runner/pulls/1129
Reviewed-by: Zettat123 <39446+zettat123@noreply.gitea.com>
2026-07-31 15:11:44 +00:00
bircni
14ec00b66e fix!: strip host-escape container options when privileged mode is disabled (#1058)
Workflow-controlled `jobs.<job>.container.options` were merged directly into the
Docker `HostConfig`. When the runner's privileged mode is disabled, only
`Privileged` was forced to `false` — host namespace flags, capability expansion,
security-profile overrides, and device/runtime access from the workflow YAML
survived into the final `HostConfig`. A workflow author could therefore enter
host PID/IPC namespaces and execute commands as root on the runner host:

```yaml
container:
  image: ubuntu:22.04
  options: >-
    --pid=host --ipc=host --cap-add=ALL
    --security-opt seccomp=unconfined --security-opt apparmor=unconfined
```

## Fix

`mergeContainerConfigs()` now strips the dangerous options-derived `HostConfig`
fields before merging when privileged mode is off: `PidMode`, `IpcMode`,
`UTSMode`, `CgroupnsMode`, `UsernsMode`, `CapAdd`, `SecurityOpt`, `Devices`,
`DeviceCgroupRules`, `DeviceRequests`, `VolumesFrom`, `Runtime`, `CgroupParent`,
and `Sysctls`. Each strip emits a warning, matching the existing
`--network ignored` handling. Options remain honored when privileged mode is
enabled, since the administrator has already opted into host access.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1058
Reviewed-by: Zettat123 <39446+zettat123@noreply.gitea.com>
2026-07-31 12:43:19 +00:00
bircni
68c6a5b4f1 feat: mask secrets that reach the log in an encoded form (#1108)
Only the verbatim value of a secret was masked, so a secret leaked through an action that serialized it stayed readable: `toJSON(secrets)` escapes it, an Authorization header carries it base64-encoded, a URL percent-encodes it. Each secret and `::add-mask::` value is now masked in those forms too, matching the value encoders of GitHub's runner. Encodings that leave the value unchanged are skipped, so a plain token still costs a single replacement pair. Includes regression tests.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1108
Reviewed-by: Zettat123 <39446+zettat123@noreply.gitea.com>
2026-07-31 12:35:10 +00:00
bircni
0cd0e52a24 fix!: guard against two runner processes sharing one runner file (#1099)
Starting two runner daemons with the same `.runner` file makes both present an
identical UUID+token, so Gitea treats them as one runner and they cancel each
other's jobs. This adds a non-blocking advisory lock on a sibling
`<runner-file>.lock`: the daemon (and `register`) acquire it at startup, and a
second process on the same host fails fast with a clear error instead of silently
interfering. The OS releases the lock when the process exits — including a hard
kill — so no stale lock is left behind. Legitimate multi-runner setups are
unaffected since each already uses its own `runner.file`.

Note: this covers the common single-host case; two hosts sharing a copied
`.runner` (e.g. over NFS) would still need server-side detection in Gitea.

---------

Co-authored-by: Zettat123 <zettat123@gmail.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1099
Reviewed-by: Zettat123 <39446+zettat123@noreply.gitea.com>
2026-07-31 12:15:04 +00:00
bircni
47d5b5ad03 feat!: add cache service v2, add toolkit patches (#1110)
Serves `github.actions.results.api.v1.CacheService` next to the v1 cache API, sharing its store, plus the subset of the Azure blob protocol the toolkit uploads with. On by default via `cache.v2`, and works with `external_server`.

Clients reach it through two edits in the action's own bundle: the GHES check is opened, and the cache service URL is taken from `ACTIONS_CACHE_URL`.

The same GHES check is what makes the stock `actions/upload-artifact` and `download-artifact` abort on Gitea. Opening it makes them work without the `gitea-upload-artifact` fork, from `upload-artifact@v4.4.0` on.

Verified against 118 real bundles, every major version of 16 actions: 92 patched, the rest deliberately left alone, and every patched bundle checked with `node --check`. Also end to end against pinned `actions/cache@v6.1.0` with an unreachable results URL, so only the patch can make the cache work.

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: bircni <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1110
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
2026-07-31 12:08:44 +00:00
Renovate Bot
2398d4a527 fix(deps): update module github.com/ulikunitz/xz to v0.5.15 [security] (#1127)
This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [github.com/ulikunitz/xz](https://github.com/ulikunitz/xz) | `v0.5.10` → `v0.5.15` | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fulikunitz%2fxz/v0.5.15?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fulikunitz%2fxz/v0.5.10/v0.5.15?slim=true) |

---

### github.com/ulikunitz/xz leaks memory when decoding a corrupted multiple LZMA archives
[CVE-2025-58058](https://nvd.nist.gov/vuln/detail/CVE-2025-58058) / [GHSA-jc7w-c686-c4v9](https://github.com/advisories/GHSA-jc7w-c686-c4v9) / [GO-2025-3922](https://pkg.go.dev/vuln/GO-2025-3922)

<details>
<summary>More information</summary>

#### Details
##### Summary

It is possible to put data in front of an LZMA-encoded byte stream without detecting the situation while reading the header. This can lead to increased memory consumption because the current implementation allocates the full decoding buffer directly after reading the header. The LZMA header doesn't include a magic number or has  a checksum to detect such an issue according to the [specification](https://github.com/jljusten/LZMA-SDK/blob/master/DOC/lzma-specification.txt).

Note that the code recognizes the issue later while reading the stream, but at this time the memory allocation has already been done.

##### Mitigations

The release v0.5.15 includes following mitigations:

- The ReaderConfig DictCap field is now interpreted as a limit for the dictionary size.
- The default is 2 Gigabytes - 1 byte (2^31-1 bytes).
- Users can check with the [Reader.Header] method what the actual values are in  their LZMA files and set a smaller limit using ReaderConfig.
- The dictionary size will not exceed the larger of the file size and the minimum dictionary size. This is another measure to prevent huge memory allocations for the dictionary.
- The code supports stream sizes only up to a pebibyte (1024^5).

Note that the original v0.5.14 version had a compiler error for 32 bit platforms, which has been fixed by v0.5.15.

##### Methods affected

Only software that uses [lzma.NewReader](https://pkg.go.dev/github.com/ulikunitz/xz/lzma#NewReader) or [lzma.ReaderConfig.NewReader](https://pkg.go.dev/github.com/ulikunitz/xz/lzma#ReaderConfig.NewReader) is affected. There is no issue for software using the xz functionality.

I thank  @&#8203;GregoryBuligin for his report, which is provided below.

##### Summary
When unpacking a large number of LZMA archives, even in a single goroutine, if the first byte of the archive file is 0 (a zero byte added to the beginning), an error __writeMatch: distance out of range__ occurs. Memory consumption spikes sharply, and the GC clearly cannot handle this situation.

##### Details
Judging by the error  __writeMatch: distance out of range__, the problems occur in the code around this function.
c8314b8f21/lzma/decoderdict.go (L81)

##### PoC
Run a function similar to this one in 1 or several goroutines on a multitude of LZMA archives that have a 0 (a zero byte) added to the beginning.
```
const ProjectLocalPath = "some/path"
const TmpDir = "tmp"

func UnpackLZMA(lzmaFile string) error {
	file, err := os.Open(lzmaFile)
	if err != nil {
		return err
	}
	defer file.Close()

	reader, err := lzma.NewReader(bufio.NewReader(file))
	if err != nil {
		return err
	}

	tmpFile, err := os.CreateTemp(TmpDir, TmpLZMAPrefix)
	if err != nil {
		return err
	}
	defer func() {
		tmpFile.Close()
		_ = os.Remove(tmpFile.Name())
	}()

	sha256Hasher := sha256.New()
	multiWriter := io.MultiWriter(tmpFile, sha256Hasher)

	if _, err = io.Copy(multiWriter, reader); err != nil {
		return err
	}

	unpackHash := hex.EncodeToString(sha256Hasher.Sum(nil))
	unpackDir := filepath.Join(
		ProjectLocalPath, unpackHash[:2],
	)
	_ = os.MkdirAll(unpackDir, DirPerm)

	unpackPath := filepath.Join(unpackDir, unpackHash)

	return os.Rename(tmpFile.Name(), unpackPath)
}
```

##### Impact
Servers with a small amount of RAM that download and unpack a large number of unverified LZMA archives

#### Severity
- CVSS Score: 5.3 / 10 (Medium)
- Vector String: `CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L`

#### References
- [https://github.com/ulikunitz/xz/security/advisories/GHSA-jc7w-c686-c4v9](https://github.com/ulikunitz/xz/security/advisories/GHSA-jc7w-c686-c4v9)
- [https://nvd.nist.gov/vuln/detail/CVE-2025-58058](https://nvd.nist.gov/vuln/detail/CVE-2025-58058)
- [88ddf1d0d9)
- [https://github.com/ulikunitz/xz](https://github.com/ulikunitz/xz)

This data is provided by [OSV](https://osv.dev/vulnerability/GHSA-jc7w-c686-c4v9) and the [GitHub Advisory Database](https://github.com/github/advisory-database) ([CC-BY 4.0](https://github.com/github/advisory-database/blob/main/LICENSE.md)).
</details>

---

### Memory leaks when decoding a corrupted multiple LZMA archives in github.com/ulikunitz/xz
[CVE-2025-58058](https://nvd.nist.gov/vuln/detail/CVE-2025-58058) / [GHSA-jc7w-c686-c4v9](https://github.com/advisories/GHSA-jc7w-c686-c4v9) / [GO-2025-3922](https://pkg.go.dev/vuln/GO-2025-3922)

<details>
<summary>More information</summary>

#### Details
Memory leaks when decoding a corrupted multiple LZMA archives in github.com/ulikunitz/xz

#### Severity
Unknown

#### References
- [https://github.com/ulikunitz/xz/security/advisories/GHSA-jc7w-c686-c4v9](https://github.com/ulikunitz/xz/security/advisories/GHSA-jc7w-c686-c4v9)
- [88ddf1d0d9)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2025-3922) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Release Notes

<details>
<summary>ulikunitz/xz (github.com/ulikunitz/xz)</summary>

### [`v0.5.15`](https://github.com/ulikunitz/xz/compare/v0.5.14...v0.5.15)

[Compare Source](https://github.com/ulikunitz/xz/compare/v0.5.14...v0.5.15)

### [`v0.5.14`](https://github.com/ulikunitz/xz/compare/v0.5.13...v0.5.14)

[Compare Source](https://github.com/ulikunitz/xz/compare/v0.5.13...v0.5.14)

### [`v0.5.13`](https://github.com/ulikunitz/xz/compare/v0.5.12...v0.5.13)

[Compare Source](https://github.com/ulikunitz/xz/compare/v0.5.12...v0.5.13)

### [`v0.5.12`](https://github.com/ulikunitz/xz/compare/v0.5.11...v0.5.12)

[Compare Source](https://github.com/ulikunitz/xz/compare/v0.5.11...v0.5.12)

### [`v0.5.11`](https://github.com/ulikunitz/xz/compare/v0.5.10...v0.5.11)

[Compare Source](https://github.com/ulikunitz/xz/compare/v0.5.10...v0.5.11)

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - ""
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xOTEuMiIsInVwZGF0ZWRJblZlciI6IjQzLjE5MS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://gitea.com/gitea/runner/pulls/1127
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: Renovate Bot <renovate-bot@gitea.com>
2026-07-30 17:05:53 +00:00
silverwind
b7a3bf98bc fix: stop leaking per-job docker networks (#1124)
Leaked per-job networks each hold a subnet of the daemon's address pool and nothing reclaimed them, so a host eventually fails every job with `all predefined address pools have been fully subnetted`. This is what CI hit in https://gitea.com/gitea/runner/actions/runs/742177.

- teardown no longer loses a container, and its network with it: a failed `ContainerRemove` was reported as success, a container whose id was never learned was skipped, and the daemon's own `AutoRemove` teardown was raced
- the idle cleanup reclaims what teardown cannot: networks carry `com.gitea.runner.uuid`, so a runner only touches its own, and a cutoff keeps a job starting during the pass out of scope
- pull failures reported mid-stream were discarded, surfacing later as a confusing `No such image`; they now propagate, and fall back to a local copy instead of failing the job
- `NetworkCreate` retries pool exhaustion briefly, then says which knobs to turn
- digest-pinned images are not re-pulled, and removal kills first so it never waits out Podman's stop timeout (measured 10.1s → 0.09s per container)

---------

Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1124
Reviewed-by: bircni <bircni@icloud.com>
2026-07-30 16:25:47 +00:00
silverwind
da4037899a chore: add renovate markers to Makefile tool packages (#1126)
The shared renovate config at https://gitea.com/gitea/renovate-config already updates `_PACKAGE` variables carrying a trailing `# renovate: datasource=` marker, so add them here.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1126
Co-authored-by: silverwind <me@silverwind.io>
2026-07-30 16:09:28 +00:00
silverwind
e4fe49dba4 chore: scope fmt-check to Go sources (#1125)
`fmt-check` runs `make fmt` and then diffs the whole working tree, so any unrelated uncommitted file makes it fail with `Please run 'make fmt' and commit the result` even when the formatter changed nothing.

Restrict the diff to the files the formatter can touch, matching what [gitea's Makefile](https://github.com/go-gitea/gitea/blob/main/Makefile) does.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1125
Reviewed-by: techknowlogick <9+techknowlogick@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-07-30 13:06:28 +00:00
82 changed files with 5936 additions and 778 deletions

View File

@@ -85,7 +85,7 @@ jobs:
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4 uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4
- name: Login to DockerHub - name: Login to DockerHub
uses: docker/login-action@abd2ef45e78c5afb21d64d4ca52ee8550d9572c7 # v4 uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
with: with:
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }} password: ${{ secrets.DOCKER_PASSWORD }}

View File

@@ -88,7 +88,7 @@ jobs:
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4 uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4
- name: Login to DockerHub - name: Login to DockerHub
uses: docker/login-action@abd2ef45e78c5afb21d64d4ca52ee8550d9572c7 # v4 uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
with: with:
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }} password: ${{ secrets.DOCKER_PASSWORD }}

1
.gitignore vendored
View File

@@ -2,6 +2,7 @@
.env .env
!/act/runner/testdata/secrets/.env !/act/runner/testdata/secrets/.env
.runner .runner
.runner.lock
coverage.txt coverage.txt
.tmp/ .tmp/
/config.yaml /config.yaml

View File

@@ -37,12 +37,8 @@ linters:
rules: rules:
main: main:
deny: deny:
- pkg: io/ioutil
desc: use os or io instead
- pkg: golang.org/x/exp - pkg: golang.org/x/exp
desc: it's experimental and unreliable desc: it's experimental and unreliable
- pkg: github.com/pkg/errors
desc: use builtin errors package instead
nolintlint: nolintlint:
allow-unused: false allow-unused: false
require-explanation: true require-explanation: true

View File

@@ -1,10 +1,19 @@
- Never assume, verify before claiming
- Use `make help` to find available development targets - Use `make help` to find available development targets
- Run `make fmt` to format `.go` files, and run `make lint-go` to lint them - PR descriptions: minimal, only what and why, no task lists or file listings
- Run `make tidy` after any `go.mod` changes - Reference issues and PRs by full URL, not by number
- Run single go unit tests with `go test -run '^TestName$' ./modulepath/` - Use Conventional Commits for commit messages and PR titles, plus the `enhance` type for user-facing enhancements
- Add the current year into the copyright header of new `.go` files - Add an `Assisted-by: AGENT_NAME:MODEL_VERSION` trailer to commit messages, never `Co-Authored-By` or `Signed-off-by`
- Ensure no trailing whitespace in edited files - Attribute agent authorship on one trailing line in issue and pull request comments, never as a PR description section
- Never force-push, amend, or squash unless asked. Use new commits and normal push for pull request updates - Never force-push, amend, or squash unless asked. Use new commits and normal push for pull request updates
- Preserve existing code comments, do not remove or rewrite comments that are still relevant - Comments: write almost none, short and preferably same-line, explaining why for a future reader. Never narrate code, the change or the prompt. Preserve existing ones that still apply
- Include authorship attribution in issue and pull request comments - Add the current year into the copyright header of new `.go` files
- Add `Co-Authored-By` lines to all commits, indicating name and model used - Read `DEVELOPMENT.md` for internals and conventions
- Ensure no trailing whitespace in edited files
- Run `make fmt` after `.go` edits, `make tidy` after `go.mod` edits, and `make checks` for the non-lint source checks
- Lint what changed with `make lint-go`, and `make lint-go-windows` for Windows and platform-split files
- Fix the cause rather than disabling a linter or weakening a test. Where unavoidable, use the narrowest scope with a trailing comment giving the reason
- Run single go tests with `go test -run '^TestName$' ./modulepath/`. `make test` self-skips the integration tests without docker or network, `make test-dind` runs the daemon-facing tests against the built dind image
- Write the fewest, fastest tests covering the behavior, extending an existing one where possible. Prefer unit tests where logic is testable in isolation
- Wait on a deterministic condition rather than `sleep`
- Update the files under `docs/` when behavior documented there changes

32
DEVELOPMENT.md Normal file
View File

@@ -0,0 +1,32 @@
# Development
## Job log line format
Gitea stores one log row per line and its web UI decodes the payload, so getting the encoding
wrong never fails a test here, it only shows up in the browser.
**A row cannot contain a real newline.** `FormatLog` rewrites `\n` to a literal backslash-n and
truncates at 64 KiB on a byte boundary.
**The payload of a line starting with a recognised prefix is decoded**, with the escape set
depending on the prefix:
| prefix | decodes |
| --- | --- |
| `##[error]` `##[warning]` `##[notice]` `##[debug]` `##[group]` `##[endgroup]` `##[add-matcher]` | `%25` `%0D` `%0A` `%3B` `%5D` |
| `::error::` `::warning::` `::notice::` `::debug::` (with or without ` key=value` properties), `::group::` `::endgroup::` `::add-matcher::` | `%25` `%0D` `%0A` |
| `##[command]` `[command]`, or no recognised prefix | nothing |
### Rules
- **Emitting a command line?** Escape the payload with `runner.EscapeCommandData`. One escaper
covers both forms: it escapes `%` first, so a literal `%3B` becomes `%253B` that the extra
`##[…]` rules cannot match, and a raw `;` or `]` is never decoded. It is also what makes
multi-line work, `\n` becomes `%0A` and the UI turns it back into a line break.
- **Forwarding a command from step output?** Leave the payload alone, it arrived escaped and is
decoded once. Decoding here double-decodes and destroys multi-line.
- **No prefix?** Do not escape, and split multi-line values into one row each.
- **Interpolating a secret?** Masking runs after escaping, so `AppendSecretMasker` registers the
encoded forms too.
- Command *properties* also escape `%3A` and `%2C`, which the UI never decodes, so the reporter
decodes exactly those two when folding a location into an annotation.

View File

@@ -4,9 +4,9 @@ DIST_DIRS := $(DIST)/binaries $(DIST)/release
GO ?= go GO ?= go
SHASUM ?= shasum -a 256 SHASUM ?= shasum -a 256
HAS_GO = $(shell hash $(GO) > /dev/null 2>&1 && echo "GO" || echo "NOGO" ) HAS_GO = $(shell hash $(GO) > /dev/null 2>&1 && echo "GO" || echo "NOGO" )
XGO_PACKAGE ?= src.techknowlogick.com/xgo@latest XGO_PACKAGE ?= src.techknowlogick.com/xgo@v1.9.0 # renovate: datasource=go
XGO_VERSION := go-1.26.x XGO_VERSION := go-1.26.x
GXZ_PACKAGE ?= github.com/ulikunitz/xz/cmd/gxz@v0.5.10 GXZ_PACKAGE ?= github.com/ulikunitz/xz/cmd/gxz@v0.5.16 # renovate: datasource=go
LINUX_ARCHS ?= linux/amd64,linux/arm64 LINUX_ARCHS ?= linux/amd64,linux/arm64
DARWIN_ARCHS ?= darwin-12/amd64,darwin-12/arm64 DARWIN_ARCHS ?= darwin-12/amd64,darwin-12/arm64
@@ -18,8 +18,8 @@ DOCKER_TAG ?= nightly
DOCKER_REF := $(DOCKER_IMAGE):$(DOCKER_TAG) DOCKER_REF := $(DOCKER_IMAGE):$(DOCKER_TAG)
DOCKER_ROOTLESS_REF := $(DOCKER_IMAGE):$(DOCKER_TAG)-dind-rootless DOCKER_ROOTLESS_REF := $(DOCKER_IMAGE):$(DOCKER_TAG)-dind-rootless
GOLANGCI_LINT_PACKAGE ?= github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.12.2 GOLANGCI_LINT_PACKAGE ?= github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.12.2 # renovate: datasource=go
GOVULNCHECK_PACKAGE ?= golang.org/x/vuln/cmd/govulncheck@v1.3.0 GOVULNCHECK_PACKAGE ?= golang.org/x/vuln/cmd/govulncheck@v1.6.0 # renovate: datasource=go
GOTEST_FLAGS ?= -race -timeout 20m -parallel 8 GOTEST_FLAGS ?= -race -timeout 20m -parallel 8
@@ -97,7 +97,7 @@ go-check:
.PHONY: fmt-check .PHONY: fmt-check
fmt-check: fmt fmt-check: fmt
@diff=$$(git diff --color=always); \ @diff=$$(git diff --color=always -- '*.go'); \
if [ -n "$$diff" ]; then \ if [ -n "$$diff" ]; then \
echo "Please run 'make fmt' and commit the result:"; \ echo "Please run 'make fmt' and commit the result:"; \
printf "%s" "$${diff}"; \ printf "%s" "$${diff}"; \

View File

@@ -132,9 +132,11 @@ Same idea as `dind`, but built on `docker:dind-rootless` so the bundled daemon a
The runner is configured with a YAML file. Generate a starting point (this matches what ships in the tree): The runner is configured with a YAML file. Generate a starting point (this matches what ships in the tree):
```bash ```bash
./gitea-runner generate-config > config.yaml ./gitea-runner config generate > config.yaml
``` ```
> The top-level `generate-config` command still does the same thing, but is deprecated in favour of `config generate`.
Pass it with `-c` / `--config` on any command that loads configuration (`register`, `daemon`, `cache-server`): Pass it with `-c` / `--config` on any command that loads configuration (`register`, `daemon`, `cache-server`):
```bash ```bash
@@ -143,7 +145,26 @@ Pass it with `-c` / `--config` on any command that loads configuration (`registe
./gitea-runner -c config.yaml cache-server ./gitea-runner -c config.yaml cache-server
``` ```
Every option is described in [config.example.yaml](internal/pkg/config/config.example.yaml) (the same content `generate-config` prints). Every option is described in [config.example.yaml](internal/pkg/config/config.example.yaml) (the same content `config generate` prints).
#### Editing a config file
`config` changes an existing file in place, keeping its comments and key order, which is handy in provisioning scripts:
```bash
./gitea-runner -c config.yaml config set runner.capacity 4
./gitea-runner -c config.yaml config set runner.timeout 90m # written as 1h30m0s
./gitea-runner -c config.yaml config set runner.envs.MY_VAR value
./gitea-runner -c config.yaml config add runner.labels 'ubuntu:docker://node:22'
./gitea-runner -c config.yaml config remove runner.labels 'ubuntu:docker://node:22'
./gitea-runner -c config.yaml config get runner.labels
```
`-c` is optional for these subcommands: without it they use `config.yaml` (or `config.yml`) from the working directory, falling back to the directory of the `gitea-runner` binary, and print which file they picked to stderr.
Keys are the dotted YAML path and are validated against the known options, so a typo is rejected instead of being written. `add` and `remove` only work on list options such as `runner.labels` and `container.valid_volumes`, and fail if the value is already present or missing. `set` replaces the whole list when given several values.
The file is re-encoded on every edit, so indentation is normalised to two spaces and blank lines inside a section are dropped.
#### Without a config file #### Without a config file
@@ -209,6 +230,26 @@ Whenever the resulting labels differ from the ones in the registration file, the
> **Note:** A runner that only exposes `host` labels still needs access to a Docker daemon (e.g. a mounted `/var/run/docker.sock`) whenever a job uses a `docker://` action or a service container. `host` labels only change where the job's own steps run; container-based steps and actions are still executed with Docker. > **Note:** A runner that only exposes `host` labels still needs access to a Docker daemon (e.g. a mounted `/var/run/docker.sock`) whenever a job uses a `docker://` action or a service container. `host` labels only change where the job's own steps run; container-based steps and actions are still executed with Docker.
#### Service containers
A job's `services` are started before its steps run. When a service's image or its `options` declare a healthcheck, the runner waits for it to report healthy, so a workflow does not have to poll for its own services:
```yaml
services:
postgres:
image: postgres:17
options: >-
--health-cmd pg_isready
--health-interval 5s
--health-retries 10
```
A service that reports unhealthy fails the job right away, with its container log. One that never becomes healthy fails it after `container.service_ready_timeout` (default `5m`, negative disables the wait). A service that exits without declaring a healthcheck only gets its log and a warning.
A job in a container reaches a service by its id on the job network, on the port the service listens on, for example `psql -h postgres -p 5432`. The started containers also fill the `job` context: `job.container.{id,network}` and `job.services.<id>.{id,network,ports}`, where `ports` maps a container port to the host port Docker published it on, for the services that publish one.
Unlike GitHub, a job whose steps run on the host (a `host` label without `container:`) starts no service containers, so `job.services` and `job.container` stay empty. Give such a job a `container:` when it needs services.
#### Proxy #### Proxy
Set these variables in the runner's environment, with systemd `Environment=`, `docker run -e`, or Kubernetes `env:`: Set these variables in the runner's environment, with systemd `Environment=`, `docker run -e`, or Kubernetes `env:`:
@@ -242,6 +283,17 @@ A password in a proxy URL is hidden in job logs. Any step can still read it, bec
Each runner starts its own cache server automatically. Cache entries are local to that runner — runners do not share a cache by default. Each runner starts its own cache server automatically. Cache entries are local to that runner — runners do not share a cache by default.
**Cache service v2**
`actions/cache@v4.2` and later can use the *cache service v2* API. The runner serves it from the same store as v1, on by default, and it works with `external_server`. Turn it off with:
```yaml
cache:
v2: false
```
Those actions refuse any host they do not take for GitHub. Rather than misreport the server URL, the runner edits that check out of the action's own bundle and keeps the untouched copy beside it; a bundle it does not recognise is left alone and keeps to v1. The same edit lets the stock `actions/upload-artifact` and `actions/download-artifact` work from `v4.4.0` on, without the `gitea-upload-artifact` fork.
**Shared cache across multiple runners** **Shared cache across multiple runners**
Run one dedicated `gitea-runner cache-server` that all runners point at. Run one dedicated `gitea-runner cache-server` that all runners point at.
@@ -283,11 +335,12 @@ Besides `GITEA_INSTANCE_URL` and `GITEA_RUNNER_REGISTRATION_TOKEN`, the image en
For a fuller container-oriented walkthrough, see [examples/docker](examples/docker/README.md). For a fuller container-oriented walkthrough, see [examples/docker](examples/docker/README.md).
When `container.bind_workdir` is enabled, stale task workspace directories can be cleaned while the runner is idle: While the runner is idle it cleans up after earlier jobs:
- directories older than `runner.workdir_cleanup_age` are removed (default: `24h`; set `0` to disable) - when `container.bind_workdir` is enabled, stale task workspace directories older than `runner.workdir_cleanup_age` are removed (default: `24h`; set `0` to disable)
- cleanup runs every `runner.idle_cleanup_interval` (default: `10m`; set `0` to disable)
- only purely numeric subdirectories under `container.workdir_parent` are treated as task workspaces and may be removed - only purely numeric subdirectories under `container.workdir_parent` are treated as task workspaces and may be removed
- cleanup assumes `container.workdir_parent` is not shared across multiple runners - cleanup assumes `container.workdir_parent` is not shared across multiple runners
- on runners that use docker, per-job networks left behind by jobs the runner did not live to tear down are removed, identified by the `com.gitea.runner.uuid` label carrying this runner's uuid
- cleanup runs every `runner.idle_cleanup_interval` (default: `10m`; set `0` to disable), and setting either knob to `0` disables all of the above
#### Post-task script (`runner.post_task_script`) #### Post-task script (`runner.post_task_script`)

View File

@@ -52,7 +52,13 @@ type credKey struct{}
// poison another repo's cache, even from inside a container that reaches the // poison another repo's cache, even from inside a container that reaches the
// cache server over the docker bridge network. // cache server over the docker bridge network.
type JobCredential struct { type JobCredential struct {
Repo string Repo string `json:"repo"`
// Results is the instance whose artifact service this server forwards for the job, and
// InsecureTLS how the runner reaches it; see results.go. The tags are the wire format a
// remote runner registers with.
Results string `json:"results"`
InsecureTLS bool `json:"insecure_tls"`
} }
// credEntry holds a registered job's credential along with an active // credEntry holds a registered job's credential along with an active
@@ -158,12 +164,14 @@ func StartHandler(dir, outboundIP string, port uint16, internalSecret string, lo
router.POST(apiPath+"/clean", h.bearerAuth(h.clean)) router.POST(apiPath+"/clean", h.bearerAuth(h.clean))
// Artifact GET is signed via query-string HMAC because @actions/cache // Artifact GET is signed via query-string HMAC because @actions/cache
// does not attach Authorization when downloading archiveLocation. // does not attach Authorization when downloading archiveLocation.
router.GET(apiPath+"/artifacts/:id", h.signedURLAuth(h.get)) router.GET(apiPath+"/artifacts/:id", h.signedAuth("", h.get))
// Control-plane: a remote runner registers/revokes per-job tokens so the // Control-plane: a remote runner registers/revokes per-job tokens so the
// cache API can authenticate them. Always wired so the routes exist; the // cache API can authenticate them. Always wired so the routes exist; the
// handlers themselves 401 when internalSecret is unset. // handlers themselves 401 when internalSecret is unset.
router.POST(internalPath+"/register", h.internalAuth(h.internalRegister)) router.POST(internalPath+"/register", h.internalAuth(h.internalRegister))
router.POST(internalPath+"/revoke", h.internalAuth(h.internalRevoke)) router.POST(internalPath+"/revoke", h.internalAuth(h.internalRevoke))
h.registerV2Routes(router)
router.NotFound = http.HandlerFunc(h.forwardOrNotFound)
h.router = router h.router = router
@@ -210,10 +218,11 @@ func (h *Handler) ExternalURL() string {
// is only accepted while the job is running. // is only accepted while the job is running.
// //
// Registrations are reference-counted: if a token is already registered, the // Registrations are reference-counted: if a token is already registered, the
// existing repo is kept and the refcount is incremented. The entry is // credential it was registered with is kept and the refcount is incremented.
// removed only when every revoker returned by RegisterJob has been called. // The entry is removed only when every revoker returned by RegisterJob has
// been called.
// This keeps a stray re-registration from silently revoking a live job. // This keeps a stray re-registration from silently revoking a live job.
func (h *Handler) RegisterJob(token, repo string) func() { func (h *Handler) RegisterJob(token string, cred JobCredential) func() {
if h == nil || token == "" { if h == nil || token == "" {
return func() {} return func() {}
} }
@@ -222,7 +231,7 @@ func (h *Handler) RegisterJob(token, repo string) func() {
existing.refs++ existing.refs++
} else { } else {
h.creds[token] = &credEntry{ h.creds[token] = &credEntry{
cred: JobCredential{Repo: repo}, cred: cred,
refs: 1, refs: 1,
} }
} }
@@ -339,7 +348,7 @@ func (h *Handler) find(w http.ResponseWriter, r *http.Request, _ httprouter.Para
} }
defer db.Close() defer db.Close()
cache, err := findCache(db, cred.Repo, keys, version) cache, err := h.lookupCache(db, cred.Repo, keys, version)
if err != nil { if err != nil {
h.responseJSON(w, r, 500, err) h.responseJSON(w, r, 500, err)
return return
@@ -348,15 +357,6 @@ func (h *Handler) find(w http.ResponseWriter, r *http.Request, _ httprouter.Para
h.responseJSON(w, r, 204) h.responseJSON(w, r, 204)
return return
} }
if ok, err := h.storage.Exist(cache.ID); err != nil {
h.responseJSON(w, r, 500, err)
return
} else if !ok {
_ = db.Delete(cache.ID, cache)
h.responseJSON(w, r, 204)
return
}
h.responseJSON(w, r, 200, map[string]any{ h.responseJSON(w, r, 200, map[string]any{
"result": "hit", "result": "hit",
"archiveLocation": h.signedArtifactURL(cache.ID, time.Now().Add(artifactURLTTL)), "archiveLocation": h.signedArtifactURL(cache.ID, time.Now().Add(artifactURLTTL)),
@@ -364,6 +364,25 @@ func (h *Handler) find(w http.ResponseWriter, r *http.Request, _ httprouter.Para
}) })
} }
// lookupCache returns the entry to restore for these keys, or (nil, nil) when there is none:
// either nothing matched, or the match had lost its blob to a prune, in which case the dangling
// entry is dropped on the way out.
func (h *Handler) lookupCache(db *bolthold.Store, repo string, keys []string, version string) (*Cache, error) {
cache, err := findCache(db, repo, keys, version)
if err != nil || cache == nil {
return nil, err
}
ok, err := h.storage.Exist(cache.ID)
if err != nil {
return nil, err
}
if !ok {
_ = db.Delete(cache.ID, cache)
return nil, nil //nolint:nilnil // absence is not an error here
}
return cache, nil
}
// POST /_apis/artifactcache/caches // POST /_apis/artifactcache/caches
func (h *Handler) reserve(w http.ResponseWriter, r *http.Request, _ httprouter.Params) { func (h *Handler) reserve(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
cred := credFromContext(r.Context()) cred := credFromContext(r.Context())
@@ -438,7 +457,7 @@ func (h *Handler) upload(w http.ResponseWriter, r *http.Request, params httprout
h.responseJSON(w, r, 500, err) h.responseJSON(w, r, 500, err)
return return
} }
h.useCache(id) _ = h.touchCache(uint64(id), false)
h.responseJSON(w, r, 200) h.responseJSON(w, r, 200)
} }
@@ -479,23 +498,7 @@ func (h *Handler) commit(w http.ResponseWriter, r *http.Request, params httprout
db.Close() db.Close()
size, err := h.storage.Commit(cache.ID, cache.Size) if err := h.commitCache(cache); err != nil {
if err != nil {
h.responseJSON(w, r, 500, err)
return
}
// write real size back to cache, it may be different from the current value when the request doesn't specify it.
cache.Size = size
db, err = h.openDB()
if err != nil {
h.responseJSON(w, r, 500, err)
return
}
defer db.Close()
cache.Complete = true
if err := db.Update(cache.ID, cache); err != nil {
h.responseJSON(w, r, 500, err) h.responseJSON(w, r, 500, err)
return return
} }
@@ -503,8 +506,28 @@ func (h *Handler) commit(w http.ResponseWriter, r *http.Request, params httprout
h.responseJSON(w, r, 200) h.responseJSON(w, r, 200)
} }
// commitCache assembles the uploaded parts and marks the entry complete. The caller must
// have closed its store first: Commit concatenates the whole archive and would otherwise
// hold bolt's exclusive file lock for the duration.
func (h *Handler) commitCache(cache *Cache) error {
written, err := h.storage.Commit(cache.ID, cache.Size)
if err != nil {
return err
}
// write real size back to cache, it may be different from the current value when the request doesn't specify it.
cache.Size = written
cache.Complete = true
db, err := h.openDB()
if err != nil {
return err
}
defer db.Close()
return db.Update(cache.ID, cache)
}
// GET /_apis/artifactcache/artifacts/:id // GET /_apis/artifactcache/artifacts/:id
// Authenticated via signed URL (see signedURLAuth), not bearer, because the // Authenticated via signed URL (see signedAuth), not bearer, because the
// @actions/cache toolkit downloads archiveLocation without Authorization. // @actions/cache toolkit downloads archiveLocation without Authorization.
// Repository scoping is already enforced at find() time; the signature binds // Repository scoping is already enforced at find() time; the signature binds
// the URL to the specific cache ID and an expiry. // the URL to the specific cache ID and an expiry.
@@ -514,7 +537,7 @@ func (h *Handler) get(w http.ResponseWriter, r *http.Request, params httprouter.
h.responseJSON(w, r, 400, err) h.responseJSON(w, r, 400, err)
return return
} }
h.useCache(id) _ = h.touchCache(uint64(id), false)
h.storage.Serve(w, r, uint64(id)) h.storage.Serve(w, r, uint64(id))
} }
@@ -548,7 +571,9 @@ func (h *Handler) bearerAuth(handler httprouter.Handle) httprouter.Handle {
} }
} }
func (h *Handler) signedURLAuth(handler httprouter.Handle) httprouter.Handle { // signedAuth authenticates a signed URL. purpose separates the flavours of URL the
// handler hands out, so one cannot be replayed as another; see computeSignature.
func (h *Handler) signedAuth(purpose string, handler httprouter.Handle) httprouter.Handle {
return func(w http.ResponseWriter, r *http.Request, params httprouter.Params) { return func(w http.ResponseWriter, r *http.Request, params httprouter.Params) {
h.logger.Debugf("%s %s", r.Method, r.URL.Path) h.logger.Debugf("%s %s", r.Method, r.URL.Path)
id, err := strconv.ParseInt(params.ByName("id"), 10, 64) id, err := strconv.ParseInt(params.ByName("id"), 10, 64)
@@ -571,7 +596,7 @@ func (h *Handler) signedURLAuth(handler httprouter.Handle) httprouter.Handle {
h.responseJSON(w, r, http.StatusUnauthorized, errors.New("signature expired")) h.responseJSON(w, r, http.StatusUnauthorized, errors.New("signature expired"))
return return
} }
expected := h.computeSignature(id, exp) expected := h.computeSignature(purpose, id, exp)
if !hmac.Equal([]byte(sig), []byte(expected)) { if !hmac.Equal([]byte(sig), []byte(expected)) {
h.responseJSON(w, r, http.StatusUnauthorized, errors.New("bad signature")) h.responseJSON(w, r, http.StatusUnauthorized, errors.New("bad signature"))
return return
@@ -602,7 +627,7 @@ func (h *Handler) internalAuth(handler httprouter.Handle) httprouter.Handle {
type internalRegisterBody struct { type internalRegisterBody struct {
Token string `json:"token"` Token string `json:"token"`
Repo string `json:"repo"` JobCredential
} }
type internalRevokeBody struct { type internalRevokeBody struct {
@@ -610,6 +635,15 @@ type internalRevokeBody struct {
} }
// POST /_internal/register // POST /_internal/register
// ResultsURL is what a job registered with cred should be given as ACTIONS_RESULTS_URL, or "" when
// the credential names no instance to forward the artifact half to.
func (h *Handler) ResultsURL(cred JobCredential) string {
if h == nil || cred.Results == "" {
return ""
}
return h.ExternalURL()
}
func (h *Handler) internalRegister(w http.ResponseWriter, r *http.Request, _ httprouter.Params) { func (h *Handler) internalRegister(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
var body internalRegisterBody var body internalRegisterBody
if err := json.NewDecoder(r.Body).Decode(&body); err != nil { if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
@@ -620,8 +654,9 @@ func (h *Handler) internalRegister(w http.ResponseWriter, r *http.Request, _ htt
h.responseJSON(w, r, http.StatusBadRequest, errors.New("token is required")) h.responseJSON(w, r, http.StatusBadRequest, errors.New("token is required"))
return return
} }
h.RegisterJob(body.Token, body.Repo) h.RegisterJob(body.Token, body.JobCredential)
h.responseJSON(w, r, http.StatusOK) // A server too old to forward answers without this, which is how the caller knows.
h.responseJSON(w, r, http.StatusOK, map[string]any{"results_url": h.ResultsURL(body.JobCredential)})
} }
// POST /_internal/revoke // POST /_internal/revoke
@@ -655,19 +690,26 @@ func credFromContext(ctx context.Context) JobCredential {
return JobCredential{} return JobCredential{}
} }
func (h *Handler) computeSignature(cacheID, exp int64) string { // computeSignature signs a URL for one cache entry and expiry. purpose is mixed into the
// message so a URL handed out for writing an entry cannot be replayed to read one, and the
// other way round. Downloads use the empty purpose, the message v1 has always signed.
func (h *Handler) computeSignature(purpose string, cacheID, exp int64) string {
mac := hmac.New(sha256.New, h.secret) mac := hmac.New(sha256.New, h.secret)
fmt.Fprintf(mac, "%d:%d", cacheID, exp) fmt.Fprintf(mac, "%s%d:%d", purpose, cacheID, exp)
return hex.EncodeToString(mac.Sum(nil)) return hex.EncodeToString(mac.Sum(nil))
} }
func (h *Handler) signedArtifactURL(cacheID uint64, exp time.Time) string { // signedURL builds a URL under path that signedAuth accepts for the same purpose.
func (h *Handler) signedURL(path, purpose string, cacheID uint64, exp time.Time) string {
expUnix := exp.Unix() expUnix := exp.Unix()
sig := h.computeSignature(int64(cacheID), expUnix)
q := url.Values{} q := url.Values{}
q.Set("exp", strconv.FormatInt(expUnix, 10)) q.Set("exp", strconv.FormatInt(expUnix, 10))
q.Set("sig", sig) q.Set("sig", h.computeSignature(purpose, int64(cacheID), expUnix))
return fmt.Sprintf("%s%s/artifacts/%d?%s", h.ExternalURL(), apiPath, cacheID, q.Encode()) return fmt.Sprintf("%s%s/%d?%s", h.ExternalURL(), path, cacheID, q.Encode())
}
func (h *Handler) signedArtifactURL(cacheID uint64, exp time.Time) string {
return h.signedURL(apiPath+"/artifacts", "", cacheID, exp)
} }
// if not found, return (nil, nil) instead of an error. // if not found, return (nil, nil) instead of an error.
@@ -675,16 +717,12 @@ func findCache(db *bolthold.Store, repo string, keys []string, version string) (
cache := &Cache{} cache := &Cache{}
for _, prefix := range keys { for _, prefix := range keys {
// if a key in the list matches exactly, don't return partial matches // if a key in the list matches exactly, don't return partial matches
if err := db.FindOne(cache, exact, err := findExactCache(db, repo, prefix, version, true)
bolthold.Where("Repo").Eq(repo).
And("Key").Eq(prefix).
And("Version").Eq(version).
And("Complete").Eq(true).
SortBy("CreatedAt").Reverse()); err == nil || !errors.Is(err, bolthold.ErrNotFound) {
if err != nil { if err != nil {
return nil, fmt.Errorf("find cache: %w", err) return nil, err
} }
return cache, nil if exact != nil {
return exact, nil
} }
prefixPattern := "^" + regexp.QuoteMeta(prefix) prefixPattern := "^" + regexp.QuoteMeta(prefix)
re, err := regexp.Compile(prefixPattern) re, err := regexp.Compile(prefixPattern)
@@ -707,6 +745,34 @@ func findCache(db *bolthold.Store, repo string, keys []string, version string) (
return nil, nil //nolint:nilnil // pre-existing issue from nektos/act return nil, nil //nolint:nilnil // pre-existing issue from nektos/act
} }
// findExactCache returns the entry for exactly this key and version, or (nil, nil) if there is
// none. Unlike findCache it never falls back to a prefix (restore-key) match, which is what both
// its callers need: a new key that is only a prefix of an existing key is not the same entry.
//
// A completed entry is the one to restore, sorted by when it was written. An incomplete one is a
// reservation being uploaded to, sorted by when it was last written to, because the upload route
// touches UsedAt on every part.
func findExactCache(db *bolthold.Store, repo, key, version string, complete bool) (*Cache, error) {
sortBy := "UsedAt"
if complete {
sortBy = "CreatedAt"
}
cache := &Cache{}
err := db.FindOne(cache,
bolthold.Where("Repo").Eq(repo).
And("Key").Eq(key).
And("Version").Eq(version).
And("Complete").Eq(complete).
SortBy(sortBy).Reverse())
if errors.Is(err, bolthold.ErrNotFound) {
return nil, nil //nolint:nilnil // absence is not an error here
}
if err != nil {
return nil, fmt.Errorf("find cache: %w", err)
}
return cache, nil
}
func insertCache(db *bolthold.Store, cache *Cache) error { func insertCache(db *bolthold.Store, cache *Cache) error {
if err := db.Insert(bolthold.NextSequence(), cache); err != nil { if err := db.Insert(bolthold.NextSequence(), cache); err != nil {
return fmt.Errorf("insert cache: %w", err) return fmt.Errorf("insert cache: %w", err)
@@ -718,18 +784,30 @@ func insertCache(db *bolthold.Store, cache *Cache) error {
return nil return nil
} }
func (h *Handler) useCache(id int64) { // touchCache stamps UsedAt so gcCache does not reap an entry mid-upload. With requireIncomplete
// it also refuses an entry that is already complete, which is what the v2 blob route needs: its
// upload URL outlives the finalize call, and overwriting a finished entry would leave the blob
// other jobs restore no longer matching its recorded size. An entry missing from the store is
// accepted, since the signature proves the id was handed out.
func (h *Handler) touchCache(id uint64, requireIncomplete bool) error {
db, err := h.openDB() db, err := h.openDB()
if err != nil { if err != nil {
return return err
} }
defer db.Close() defer db.Close()
cache := &Cache{} cache := &Cache{}
if err := db.Get(id, cache); err != nil { if err := db.Get(id, cache); err != nil {
return if errors.Is(err, bolthold.ErrNotFound) {
return nil
}
return err
}
if requireIncomplete && cache.Complete {
return fmt.Errorf("cache %d: already complete", id)
} }
cache.UsedAt = time.Now().Unix() cache.UsedAt = time.Now().Unix()
_ = db.Update(cache.ID, cache) return db.Update(cache.ID, cache)
} }
const ( const (

View File

@@ -52,7 +52,7 @@ func TestHandler(t *testing.T) {
dir := filepath.Join(t.TempDir(), "artifactcache") dir := filepath.Join(t.TempDir(), "artifactcache")
handler, err := StartHandler(dir, "", 0, "", nil) handler, err := StartHandler(dir, "", 0, "", nil)
require.NoError(t, err) require.NoError(t, err)
handler.RegisterJob(testToken, testRepo) handler.RegisterJob(testToken, JobCredential{Repo: testRepo})
base := fmt.Sprintf("%s%s", handler.ExternalURL(), apiPath) base := fmt.Sprintf("%s%s", handler.ExternalURL(), apiPath)
@@ -890,7 +890,7 @@ func TestHandler_UnregisterRevokes(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
defer handler.Close() defer handler.Close()
unregister := handler.RegisterJob("tmp-token", testRepo) unregister := handler.RegisterJob("tmp-token", JobCredential{Repo: testRepo})
base := handler.ExternalURL() + apiPath base := handler.ExternalURL() + apiPath
req, err := http.NewRequest(http.MethodGet, base+"/cache?keys=x&version=y", nil) req, err := http.NewRequest(http.MethodGet, base+"/cache?keys=x&version=y", nil)
@@ -920,8 +920,8 @@ func TestHandler_CrossRepoIsolation(t *testing.T) {
handler, err := StartHandler(dir, "", 0, "", nil) handler, err := StartHandler(dir, "", 0, "", nil)
require.NoError(t, err) require.NoError(t, err)
defer handler.Close() defer handler.Close()
handler.RegisterJob("token-a", "owner/repoA") handler.RegisterJob("token-a", JobCredential{Repo: "owner/repoA"})
handler.RegisterJob("token-b", "owner/repoB") handler.RegisterJob("token-b", JobCredential{Repo: "owner/repoB"})
base := handler.ExternalURL() + apiPath base := handler.ExternalURL() + apiPath
key := "shared-key" key := "shared-key"
@@ -986,7 +986,7 @@ func TestHandler_ArtifactSignature(t *testing.T) {
handler, err := StartHandler(dir, "", 0, "", nil) handler, err := StartHandler(dir, "", 0, "", nil)
require.NoError(t, err) require.NoError(t, err)
defer handler.Close() defer handler.Close()
handler.RegisterJob(testToken, testRepo) handler.RegisterJob(testToken, JobCredential{Repo: testRepo})
base := handler.ExternalURL() + apiPath base := handler.ExternalURL() + apiPath
@@ -1041,14 +1041,14 @@ func TestHandler_SecretPersistsAcrossRestarts(t *testing.T) {
first, err := StartHandler(dir, "127.0.0.1", 0, "", nil) first, err := StartHandler(dir, "127.0.0.1", 0, "", nil)
require.NoError(t, err) require.NoError(t, err)
exp := time.Now().Add(artifactURLTTL).Unix() exp := time.Now().Add(artifactURLTTL).Unix()
sig := first.computeSignature(42, exp) sig := first.computeSignature("", 42, exp)
require.NoError(t, first.Close()) require.NoError(t, first.Close())
second, err := StartHandler(dir, "127.0.0.1", 0, "", nil) second, err := StartHandler(dir, "127.0.0.1", 0, "", nil)
require.NoError(t, err) require.NoError(t, err)
defer second.Close() defer second.Close()
assert.Equal(t, sig, second.computeSignature(42, exp)) assert.Equal(t, sig, second.computeSignature("", 42, exp))
} }
// TestHandler_ArtifactSignatureDownload is a happy-path round trip that // TestHandler_ArtifactSignatureDownload is a happy-path round trip that
@@ -1059,7 +1059,7 @@ func TestHandler_ArtifactSignatureDownload(t *testing.T) {
handler, err := StartHandler(dir, "", 0, "", nil) handler, err := StartHandler(dir, "", 0, "", nil)
require.NoError(t, err) require.NoError(t, err)
defer handler.Close() defer handler.Close()
handler.RegisterJob(testToken, testRepo) handler.RegisterJob(testToken, JobCredential{Repo: testRepo})
base := handler.ExternalURL() + apiPath base := handler.ExternalURL() + apiPath
key := "download-key" key := "download-key"
@@ -1100,8 +1100,8 @@ func TestHandler_RegisterJob_RefCounted(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
defer handler.Close() defer handler.Close()
first := handler.RegisterJob("shared", testRepo) first := handler.RegisterJob("shared", JobCredential{Repo: testRepo})
second := handler.RegisterJob("shared", testRepo) second := handler.RegisterJob("shared", JobCredential{Repo: testRepo})
base := handler.ExternalURL() + apiPath base := handler.ExternalURL() + apiPath
probe := func() int { probe := func() int {
@@ -1131,8 +1131,8 @@ func TestHandler_GC_PerRepoDedup(t *testing.T) {
handler, err := StartHandler(dir, "", 0, "", nil) handler, err := StartHandler(dir, "", 0, "", nil)
require.NoError(t, err) require.NoError(t, err)
defer handler.Close() defer handler.Close()
handler.RegisterJob("tok-a", "owner/repoA") handler.RegisterJob("tok-a", JobCredential{Repo: "owner/repoA"})
handler.RegisterJob("tok-b", "owner/repoB") handler.RegisterJob("tok-b", JobCredential{Repo: "owner/repoB"})
key := "shared-dedup-key" key := "shared-dedup-key"
version := "c19da02a2bd7e77277f1ac29ab45c09b7d46a4ee758284e26bb3045ad11d9d20" version := "c19da02a2bd7e77277f1ac29ab45c09b7d46a4ee758284e26bb3045ad11d9d20"

View File

@@ -0,0 +1,273 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package artifactcache
import (
"cmp"
"encoding/json"
"encoding/xml"
"errors"
"fmt"
"io"
"net/http"
"strconv"
"strings"
"time"
"github.com/julienschmidt/httprouter"
)
// The cache service v2 API. A client on this version talks twirp to
// `github.actions.results.api.v1.CacheService` instead of the /_apis/artifactcache
// endpoints, and uploads the archive to the returned URL with the Azure blob protocol.
// Both API versions are served from the same store, so a repository keeps its cache
// when a workflow moves between action versions.
//
// Responses carry the proto field names, which is what Gitea's own results API emits and the only
// spelling the Go clients parse. The JavaScript toolkit accepts either.
const (
cacheServiceV2Path = "/twirp/github.actions.results.api.v1.CacheService"
// blobPath authenticates by signature, because the client uploads without an
// Authorization header. Downloads are handed the v1 artifact URL instead.
blobPath = apiPath + "/blobs"
// blobUploadPurpose keeps an upload URL from being replayed to read an entry.
blobUploadPurpose = "upload:"
blobUploadURLTTL = time.Hour
// twirpInternal is the only error code that is not the client's fault.
twirpInternal = "internal"
)
func (h *Handler) registerV2Routes(router *httprouter.Router) {
router.POST(cacheServiceV2Path+"/CreateCacheEntry", h.bearerAuth(h.v2CreateCacheEntry))
router.POST(cacheServiceV2Path+"/FinalizeCacheEntryUpload", h.bearerAuth(h.v2FinalizeCacheEntryUpload))
router.POST(cacheServiceV2Path+"/GetCacheEntryDownloadURL", h.bearerAuth(h.v2GetCacheEntryDownloadURL))
router.PUT(blobPath+"/:id", h.signedAuth(blobUploadPurpose, h.v2UploadBlob))
}
// An entry that already exists is reported as not ok, which is how the client learns to skip
// the upload.
func (h *Handler) v2CreateCacheEntry(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
cred := credFromContext(r.Context())
req, err := decodeTwirpRequest[v2CreateRequest](r)
if err != nil {
h.twirpError(w, r, "malformed_request", err)
return
}
if req.Key == "" || req.Version == "" {
h.twirpError(w, r, "invalid_argument", errors.New("key and version are required"))
return
}
db, err := h.openDB()
if err != nil {
h.twirpError(w, r, twirpInternal, err)
return
}
defer db.Close()
// An exact (key, version) match means the entry is already cached; the client then skips
// the upload. A prefix match must not count here, or a shorter key would be reported as
// existing and silently never saved.
if existing, err := findExactCache(db, cred.Repo, req.Key, req.Version, true); err != nil {
h.twirpError(w, r, twirpInternal, err)
return
} else if existing != nil {
h.twirpNotOK(w, r)
return
}
now := time.Now().Unix()
cache := &Cache{
Repo: cred.Repo,
Key: req.Key,
Version: req.Version,
Size: -1, // the size is only known at finalize time
CreatedAt: now,
UsedAt: now,
}
if err := insertCache(db, cache); err != nil {
h.twirpError(w, r, twirpInternal, err)
return
}
h.responseJSON(w, r, http.StatusOK, map[string]any{
"ok": true,
"signed_upload_url": h.signedURL(blobPath, blobUploadPurpose, cache.ID, time.Now().Add(blobUploadURLTTL)),
})
}
func (h *Handler) v2FinalizeCacheEntryUpload(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
cred := credFromContext(r.Context())
req, err := decodeTwirpRequest[v2FinalizeRequest](r)
if err != nil {
h.twirpError(w, r, "malformed_request", err)
return
}
db, err := h.openDB()
if err != nil {
h.twirpError(w, r, twirpInternal, err)
return
}
defer db.Close()
cache, err := findExactCache(db, cred.Repo, req.Key, req.Version, false)
if err != nil {
h.twirpError(w, r, twirpInternal, err)
return
}
if cache == nil {
h.twirpNotOK(w, r)
return
}
db.Close() // commitCache needs the store closed
cache.Size, _ = cmp.Or(req.SizeBytes, req.SizeBytesCamel).Int64()
if err := h.commitCache(cache); err != nil {
h.logger.Errorf("finalize cache %d (%s): %v", cache.ID, cache.Key, err)
h.twirpNotOK(w, r)
return
}
h.responseJSON(w, r, http.StatusOK, map[string]any{
"ok": true,
// int64 fields travel as strings in the proto JSON mapping.
"entry_id": strconv.FormatUint(cache.ID, 10),
})
}
func (h *Handler) v2GetCacheEntryDownloadURL(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
cred := credFromContext(r.Context())
req, err := decodeTwirpRequest[v2DownloadRequest](r)
if err != nil {
h.twirpError(w, r, "malformed_request", err)
return
}
db, err := h.openDB()
if err != nil {
h.twirpError(w, r, twirpInternal, err)
return
}
defer db.Close()
cache, err := h.lookupCache(db, cred.Repo, req.keys(), req.Version)
if err != nil {
h.twirpError(w, r, twirpInternal, err)
return
}
if cache == nil {
h.twirpNotOK(w, r)
return
}
h.responseJSON(w, r, http.StatusOK, map[string]any{
"ok": true,
"signed_download_url": h.signedArtifactURL(cache.ID, time.Now().Add(artifactURLTTL)),
"matched_key": cache.Key,
})
}
// The archive arrives over the subset of the Azure blob API the toolkit uses: a small
// cache is a single PUT, a large one is staged as blocks that a final block list puts
// in order.
func (h *Handler) v2UploadBlob(w http.ResponseWriter, r *http.Request, params httprouter.Params) {
id, err := strconv.ParseUint(params.ByName("id"), 10, 64)
if err != nil {
h.responseJSON(w, r, http.StatusBadRequest, err)
return
}
if err := h.touchCache(id, true); err != nil {
h.responseJSON(w, r, http.StatusBadRequest, err)
return
}
query := r.URL.Query()
switch strings.ToLower(query.Get("comp")) {
case "block":
blockID := query.Get("blockid")
if blockID == "" {
h.responseJSON(w, r, http.StatusBadRequest, errors.New("missing blockid"))
return
}
err = h.storage.WriteBlock(id, blockID, r.Body)
case "blocklist":
var list struct{ Latest []string }
if err := xml.NewDecoder(io.LimitReader(r.Body, 8<<20)).Decode(&list); err != nil {
h.responseJSON(w, r, http.StatusBadRequest, fmt.Errorf("malformed block list: %w", err))
return
}
err = h.storage.OrderBlocks(id, list.Latest)
default:
err = h.storage.Write(id, 0, r.Body)
}
if err != nil {
h.responseJSON(w, r, http.StatusInternalServerError, err)
return
}
// The Azure SDK client dereferences this without checking, so its absence panics the caller.
w.Header().Set("x-ms-request-id", strconv.FormatInt(time.Now().UnixNano(), 10))
w.WriteHeader(http.StatusCreated)
}
// twirpNotOK is the negative answer all three endpoints share: no such entry to restore, no
// reservation to finalize, or an entry that already exists and need not be uploaded again.
func (h *Handler) twirpNotOK(w http.ResponseWriter, r *http.Request) {
h.responseJSON(w, r, http.StatusOK, map[string]any{"ok": false})
}
// twirpError reports in the shape a twirp client expects, so the toolkit surfaces the message
// instead of a parse error.
func (h *Handler) twirpError(w http.ResponseWriter, r *http.Request, code string, err error) {
h.logger.Debugf("%s %s: %v", r.Method, r.URL.Path, err)
status := http.StatusBadRequest
if code == twirpInternal {
status = http.StatusInternalServerError
}
h.responseJSON(w, r, status, map[string]any{"code": code, "msg": err.Error()})
}
// The twirp request bodies. The toolkit's client serialises with useProtoFieldName, so the proto
// names are what arrive; the camelCase spellings of the same mapping are accepted too, as are
// int64s sent as a bare number rather than the string the mapping prescribes.
type (
v2CreateRequest struct {
Key string `json:"key"`
Version string `json:"version"`
}
v2FinalizeRequest struct {
Key string `json:"key"`
Version string `json:"version"`
SizeBytes json.Number `json:"size_bytes"`
SizeBytesCamel json.Number `json:"sizeBytes"`
}
v2DownloadRequest struct {
Key string `json:"key"`
Version string `json:"version"`
RestoreKeys []string `json:"restore_keys"`
RestoreKeysCamel []string `json:"restoreKeys"`
}
)
func (d v2DownloadRequest) keys() []string {
restoreKeys := d.RestoreKeys
if len(restoreKeys) == 0 {
restoreKeys = d.RestoreKeysCamel
}
return append([]string{d.Key}, restoreKeys...)
}
func decodeTwirpRequest[T any](r *http.Request) (T, error) {
var req T
err := json.NewDecoder(io.LimitReader(r.Body, 1<<20)).Decode(&req)
return req, err
}

View File

@@ -0,0 +1,241 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package artifactcache
import (
"bytes"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"path/filepath"
"strconv"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// v2Call posts a twirp request to the cache service and returns the decoded response.
// Field names are the proto ones, which is what the toolkit's client sends.
func v2Call(t *testing.T, handler *Handler, client *http.Client, method string, request any) map[string]any {
t.Helper()
body, err := json.Marshal(request)
require.NoError(t, err)
resp, err := client.Post(handler.ExternalURL()+cacheServiceV2Path+"/"+method, "application/json", bytes.NewReader(body))
require.NoError(t, err)
defer resp.Body.Close()
require.Equal(t, http.StatusOK, resp.StatusCode)
got := map[string]any{}
require.NoError(t, json.NewDecoder(resp.Body).Decode(&got))
return got
}
// putBlob uploads to a signed URL and returns the status, so a test can assert a refusal.
func putBlob(t *testing.T, url string, content []byte) int {
t.Helper()
req, err := http.NewRequestWithContext(t.Context(), http.MethodPut, url, bytes.NewReader(content))
require.NoError(t, err)
resp, err := http.DefaultClient.Do(req)
require.NoError(t, err)
defer resp.Body.Close()
if resp.StatusCode == http.StatusCreated {
// The Azure SDK client dereferences this header without checking, so a blob upload that
// omits it panics the caller rather than failing it.
require.NotEmpty(t, resp.Header.Get("x-ms-request-id"))
}
return resp.StatusCode
}
func getURL(t *testing.T, url string) []byte {
t.Helper()
req, err := http.NewRequestWithContext(t.Context(), http.MethodGet, url, nil)
require.NoError(t, err)
resp, err := http.DefaultClient.Do(req)
require.NoError(t, err)
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
return body
}
func startTestHandler(t *testing.T) *Handler {
t.Helper()
handler, err := StartHandler(filepath.Join(t.TempDir(), "artifactcache"), "127.0.0.1", 0, "", nil)
require.NoError(t, err)
t.Cleanup(func() { _ = handler.Close() })
handler.RegisterJob(testToken, JobCredential{Repo: testRepo})
return handler
}
// saveV2 runs the reserve/upload/finalize sequence and returns the finalize response along
// with the upload URL it used.
func saveV2(t *testing.T, handler *Handler, key, version string, content []byte) (finalized map[string]any, uploadURL string) {
t.Helper()
created := v2Call(t, handler, testClient, "CreateCacheEntry", map[string]any{"key": key, "version": version})
require.Equal(t, true, created["ok"])
uploadURL, _ = created["signed_upload_url"].(string)
require.NotEmpty(t, uploadURL)
require.Equal(t, http.StatusCreated, putBlob(t, uploadURL, content))
return v2Call(t, handler, testClient, "FinalizeCacheEntryUpload", map[string]any{
"key": key, "version": version,
"size_bytes": strconv.Itoa(len(content)),
}), uploadURL
}
// The whole round trip an actions/cache v2 client makes, plus the guarantees on the signed
// URLs it is handed: unsigned requests are refused, an upload URL cannot be replayed to read
// or to replace a finalized entry.
func TestCacheServiceV2RoundTrip(t *testing.T) {
handler := startTestHandler(t)
content := []byte("the cached archive")
unsigned := fmt.Sprintf("%s%s/1", handler.ExternalURL(), blobPath)
assert.Equal(t, http.StatusUnauthorized, putBlob(t, unsigned, content))
finalized, uploadURL := saveV2(t, handler, "deps-v1", "abc123", content)
require.Equal(t, true, finalized["ok"])
assert.NotEmpty(t, finalized["entry_id"])
// The upload URL outlives the finalize call, so replaying it must not poison the entry,
// and it is an upload URL only: nothing reads a blob back through it.
assert.Equal(t, http.StatusBadRequest, putBlob(t, uploadURL, []byte("poisoned")))
resp, err := http.Get(uploadURL) //nolint:noctx // the URL is the server under test
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusMethodNotAllowed, resp.StatusCode)
got := v2Call(t, handler, testClient, "GetCacheEntryDownloadURL", map[string]any{"key": "deps-v1", "version": "abc123"})
require.Equal(t, true, got["ok"])
assert.Equal(t, "deps-v1", got["matched_key"])
downloadURL, _ := got["signed_download_url"].(string)
require.NotEmpty(t, downloadURL)
assert.Equal(t, content, getURL(t, downloadURL))
}
// A large archive is staged as blocks and only put in order by the final block list, so
// blocks that arrive out of order must still be assembled the way the client asked.
func TestCacheServiceV2BlockUpload(t *testing.T) {
handler := startTestHandler(t)
created := v2Call(t, handler, testClient, "CreateCacheEntry", map[string]any{"key": "blocks", "version": "v1"})
uploadURL, _ := created["signed_upload_url"].(string)
require.NotEmpty(t, uploadURL)
blocks := map[string][]byte{}
var order []string
for i, part := range []string{"hello ", "world", "!"} {
blockID := base64.StdEncoding.EncodeToString(fmt.Appendf(nil, "block-%d", i))
blocks[blockID] = []byte(part)
order = append(order, blockID)
}
// Upload in an order that is not the block list order.
for _, blockID := range []string{order[2], order[0], order[1]} {
require.Equal(t, http.StatusCreated, putBlob(t, uploadURL+"&comp=block&blockid="+blockID, blocks[blockID]))
}
var list bytes.Buffer
list.WriteString(`<?xml version="1.0" encoding="utf-8"?><BlockList>`)
for _, blockID := range order {
fmt.Fprintf(&list, "<Latest>%s</Latest>", blockID)
}
list.WriteString(`</BlockList>`)
require.Equal(t, http.StatusCreated, putBlob(t, uploadURL+"&comp=blocklist", list.Bytes()))
finalized := v2Call(t, handler, testClient, "FinalizeCacheEntryUpload", map[string]any{
"key": "blocks", "version": "v1", "size_bytes": len("hello world!"),
})
require.Equal(t, true, finalized["ok"])
got := v2Call(t, handler, testClient, "GetCacheEntryDownloadURL", map[string]any{"key": "blocks", "version": "v1"})
require.Equal(t, true, got["ok"])
assert.Equal(t, "hello world!", string(getURL(t, got["signed_download_url"].(string))))
}
func TestCacheServiceV2Lookups(t *testing.T) {
handler := startTestHandler(t)
saved, _ := saveV2(t, handler, "deps-abc", "v1", []byte("x"))
require.Equal(t, true, saved["ok"])
t.Run("reports a miss for an unknown key", func(t *testing.T) {
got := v2Call(t, handler, testClient, "GetCacheEntryDownloadURL", map[string]any{"key": "nothing", "version": "v1"})
assert.Equal(t, false, got["ok"])
})
// The toolkit serialises with the proto field names; the camelCase spellings of the same
// proto JSON mapping are accepted alongside them.
for _, field := range []string{"restore_keys", "restoreKeys"} {
t.Run("restore keys match by prefix, spelled "+field, func(t *testing.T) {
got := v2Call(t, handler, testClient, "GetCacheEntryDownloadURL", map[string]any{
"key": "deps-zzz", field: []string{"deps-"}, "version": "v1",
})
require.Equal(t, true, got["ok"])
assert.Equal(t, "deps-abc", got["matched_key"])
})
}
t.Run("an existing entry is not reserved twice", func(t *testing.T) {
again := v2Call(t, handler, testClient, "CreateCacheEntry", map[string]any{"key": "deps-abc", "version": "v1"})
assert.Equal(t, false, again["ok"])
})
// A key that is only a prefix of an existing one is a different entry, so the
// reservation check must be exact and not a restore-key prefix match, or the shorter
// key would be reported as existing and silently never saved.
t.Run("a prefix of an existing key is still reserved", func(t *testing.T) {
reserved := v2Call(t, handler, testClient, "CreateCacheEntry", map[string]any{"key": "deps", "version": "v1"})
require.Equal(t, true, reserved["ok"])
assert.NotEmpty(t, reserved["signed_upload_url"])
})
t.Run("finalizing without a reservation is not ok", func(t *testing.T) {
got := v2Call(t, handler, testClient, "FinalizeCacheEntryUpload", map[string]any{
"key": "never-reserved", "version": "v1", "size_bytes": 1,
})
assert.Equal(t, false, got["ok"])
})
// The size the client declares is what Commit validates the assembled archive against.
t.Run("finalizing with the wrong size is not ok", func(t *testing.T) {
created := v2Call(t, handler, testClient, "CreateCacheEntry", map[string]any{"key": "wrong-size", "version": "v1"})
require.Equal(t, http.StatusCreated, putBlob(t, created["signed_upload_url"].(string), []byte("four")))
got := v2Call(t, handler, testClient, "FinalizeCacheEntryUpload", map[string]any{
"key": "wrong-size", "version": "v1", "size_bytes": 99,
})
assert.Equal(t, false, got["ok"])
})
// Both API versions are served from one store, so an entry written through v2 is a hit for
// a v1 client asking for the same key and version.
t.Run("a v1 client sees an entry written through v2", func(t *testing.T) {
resp, err := testClient.Get(fmt.Sprintf("%s%s/cache?keys=deps-abc&version=v1", handler.ExternalURL(), apiPath))
require.NoError(t, err)
defer resp.Body.Close()
require.Equal(t, http.StatusOK, resp.StatusCode)
got := map[string]any{}
require.NoError(t, json.NewDecoder(resp.Body).Decode(&got))
assert.Equal(t, "deps-abc", got["cacheKey"])
assert.NotEmpty(t, got["archiveLocation"])
})
// The cache of one repository must stay invisible to another, as it does for the v1 API.
t.Run("another repository sees nothing", func(t *testing.T) {
handler.RegisterJob("other-runtime-token", JobCredential{Repo: "other/repo"})
otherClient := &http.Client{Transport: &bearerTransport{token: "other-runtime-token"}}
got := v2Call(t, handler, otherClient, "GetCacheEntryDownloadURL", map[string]any{"key": "deps-abc", "version": "v1"})
assert.Equal(t, false, got["ok"])
})
}

View File

@@ -0,0 +1,59 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package artifactcache
import (
"crypto/tls"
"net/http"
"net/http/httputil"
"net/url"
"strings"
)
// The results service is one origin serving every github.actions.results.api.v1 service, and
// Gitea implements only the artifact half of it. Forwarding that half from here makes this origin
// the whole service, so ACTIONS_RESULTS_URL can point at it truthfully, which is what the clients
// this runner cannot patch need, docker buildx among them.
//
// The instance to forward to travels with the job registration rather than with configuration, so
// a cache server shared between runners serves each of their instances.
const artifactServicePath = "/twirp/github.actions.results.api.v1.ArtifactService/"
// forwardOrNotFound is the router's fallback: the artifact service of the instance the job
// registered with, and the 404 the router would have written otherwise.
func (h *Handler) forwardOrNotFound(w http.ResponseWriter, r *http.Request) {
cred, ok := h.lookupCredential(bearerToken(r))
if !ok || cred.Results == "" || !strings.HasPrefix(r.URL.Path, artifactServicePath) {
http.NotFound(w, r)
return
}
target, err := url.Parse(strings.TrimSuffix(cred.Results, "/"))
if err != nil {
h.logger.Errorf("artifact service forward to %q: %v", cred.Results, err)
w.WriteHeader(http.StatusBadGateway)
return
}
h.logger.Debugf("%s %s: forwarding to %s", r.Method, r.URL.Path, target)
proxy := &httputil.ReverseProxy{
Rewrite: func(r *httputil.ProxyRequest) {
r.SetURL(target)
// Gitea builds the URLs it hands back from this Host, and their scheme from the
// connection unless a forwarded header overrides it, so artifact bodies go to Gitea
// directly and never through here.
r.Out.Host = target.Host
},
ErrorHandler: func(w http.ResponseWriter, _ *http.Request, err error) {
h.logger.Warnf("artifact service forward to %s: %v", target, err)
w.WriteHeader(http.StatusBadGateway)
},
}
if cred.InsecureTLS {
proxy.Transport = insecureTransport
}
proxy.ServeHTTP(w, r)
}
// insecureTransport is shared, because a transport per request would pool no connections.
var insecureTransport = &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}} //nolint:gosec // the runner reaches its instance on the operator's say-so

View File

@@ -0,0 +1,56 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package artifactcache
import (
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// The artifact half is forwarded under the Host Gitea knows itself by, so the URLs it hands back
// still point at Gitea, and nothing else is proxied.
func TestFrontResultsService(t *testing.T) {
var gotHost, gotPath, gotProto string
gitea := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotHost, gotPath, gotProto = r.Host, r.URL.Path, r.Header.Get("X-Forwarded-Proto")
_, _ = io.WriteString(w, `{"ok":true}`)
}))
defer gitea.Close()
handler, err := StartHandler(t.TempDir(), "127.0.0.1", 0, "", nil)
require.NoError(t, err)
defer handler.Close()
const token = "forward-token"
client := &http.Client{Transport: &bearerTransport{token: token}}
post := func(path string) int {
req, err := http.NewRequestWithContext(t.Context(), http.MethodPost, handler.ExternalURL()+path, nil)
require.NoError(t, err)
resp, err := client.Do(req)
require.NoError(t, err)
resp.Body.Close()
return resp.StatusCode
}
assert.Equal(t, http.StatusNotFound, post(artifactServicePath+"CreateArtifact"),
"an unregistered token is forwarded nowhere")
defer handler.RegisterJob(token, JobCredential{Repo: "owner/repo", Results: gitea.URL})()
assert.Equal(t, http.StatusOK, post(artifactServicePath+"CreateArtifact"))
assert.Equal(t, strings.TrimPrefix(gitea.URL, "http://"), gotHost, "Gitea must see the host it mints its URLs from")
assert.Empty(t, gotProto, "a forwarded scheme would make an https Gitea mint http URLs")
assert.Equal(t, artifactServicePath+"CreateArtifact", gotPath)
gotPath = ""
assert.Equal(t, http.StatusNotFound, post("/twirp/github.actions.results.api.v1.OtherService/Do"))
assert.Equal(t, http.StatusNotFound, post("/api/v1/repos/owner/repo"))
assert.Empty(t, gotPath, "only the artifact service is forwarded")
}

View File

@@ -5,12 +5,15 @@
package artifactcache package artifactcache
import ( import (
"crypto/sha256"
"encoding/hex"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
"os" "os"
"path/filepath" "path/filepath"
"strconv" "strconv"
"strings"
) )
type Storage struct { type Storage struct {
@@ -37,7 +40,10 @@ func (s *Storage) Exist(id uint64) (bool, error) {
} }
func (s *Storage) Write(id uint64, offset int64, reader io.Reader) error { func (s *Storage) Write(id uint64, offset int64, reader io.Reader) error {
name := s.tempName(id, offset) return s.writeFile(s.tempName(id, offset), reader)
}
func (s *Storage) writeFile(name string, reader io.Reader) error {
if err := os.MkdirAll(filepath.Dir(name), 0o755); err != nil { if err := os.MkdirAll(filepath.Dir(name), 0o755); err != nil {
return err return err
} }
@@ -51,6 +57,26 @@ func (s *Storage) Write(id uint64, offset int64, reader io.Reader) error {
return err return err
} }
func (s *Storage) WriteBlock(id uint64, blockID string, reader io.Reader) error {
return s.writeFile(s.blockName(id, blockID), reader)
}
// OrderBlocks renames the staged blocks into the order the block list gives. A block the list
// does not name keeps its staged name, which is how Commit leaves it out, as Azure drops it. One
// rename pass is safe because a staged name always carries blockFilePrefix and a target name
// never does, so no rename can collide with a block not yet moved.
func (s *Storage) OrderBlocks(id uint64, blockIDs []string) error {
for i, blockID := range blockIDs {
if err := os.Rename(s.blockName(id, blockID), s.tempName(id, int64(i))); err != nil {
if os.IsNotExist(err) {
return fmt.Errorf("block %q of cache %d was never uploaded: %w", blockID, id, err)
}
return err
}
}
return nil
}
func (s *Storage) Commit(id uint64, size int64) (int64, error) { func (s *Storage) Commit(id uint64, size int64) (int64, error) {
defer func() { defer func() {
_ = os.RemoveAll(s.tempDir(id)) _ = os.RemoveAll(s.tempDir(id))
@@ -65,6 +91,31 @@ func (s *Storage) Commit(id uint64, size int64) (int64, error) {
if err := os.MkdirAll(filepath.Dir(name), 0o755); err != nil { if err := os.MkdirAll(filepath.Dir(name), 0o755); err != nil {
return 0, err return 0, err
} }
written, err := assemble(name, tempNames)
if err != nil {
return 0, err
}
// If size is less than 0, it means the size is unknown.
// We can't check the size of the file, just skip the check.
// It happens when the request comes from old versions of actions, like `actions/cache@v2`.
if size >= 0 && written != size {
_ = os.Remove(name)
return 0, fmt.Errorf("broken file: %v != %v", written, size)
}
return written, nil
}
// assemble concatenates the uploaded parts into name. A single part, which is what the v2 API
// produces below the client's block threshold, is already the whole archive and is moved.
func assemble(name string, tempNames []string) (int64, error) {
if len(tempNames) == 1 {
info, err := os.Stat(tempNames[0])
if err != nil {
return 0, err
}
return info.Size(), os.Rename(tempNames[0], name)
}
file, err := os.Create(name) file, err := os.Create(name)
if err != nil { if err != nil {
return 0, err return 0, err
@@ -84,16 +135,6 @@ func (s *Storage) Commit(id uint64, size int64) (int64, error) {
} }
written += n written += n
} }
// If size is less than 0, it means the size is unknown.
// We can't check the size of the file, just skip the check.
// It happens when the request comes from old versions of actions, like `actions/cache@v2`.
if size >= 0 && written != size {
_ = file.Close()
_ = os.Remove(name)
return 0, fmt.Errorf("broken file: %v != %v", written, size)
}
return written, nil return written, nil
} }
@@ -119,6 +160,17 @@ func (s *Storage) tempName(id uint64, offset int64) string {
return filepath.Join(s.tempDir(id), fmt.Sprintf("%016x", offset)) return filepath.Join(s.tempDir(id), fmt.Sprintf("%016x", offset))
} }
// blockFilePrefix marks a staged, not yet ordered block, so that tempNames can keep it out of
// Commit's name-ordered concatenation.
const blockFilePrefix = "block-"
func (s *Storage) blockName(id uint64, blockID string) string {
// The block id is client-chosen (base64), so it is hashed rather than trusted as a
// path element.
sum := sha256.Sum256([]byte(blockID))
return filepath.Join(s.tempDir(id), blockFilePrefix+hex.EncodeToString(sum[:]))
}
func (s *Storage) tempNames(id uint64) ([]string, error) { func (s *Storage) tempNames(id uint64) ([]string, error) {
dir := s.tempDir(id) dir := s.tempDir(id)
files, err := os.ReadDir(dir) files, err := os.ReadDir(dir)
@@ -127,7 +179,7 @@ func (s *Storage) tempNames(id uint64) ([]string, error) {
} }
var names []string var names []string
for _, v := range files { for _, v := range files {
if !v.IsDir() { if !v.IsDir() && !strings.HasPrefix(v.Name(), blockFilePrefix) {
names = append(names, filepath.Join(dir, v.Name())) names = append(names, filepath.Join(dir, v.Name()))
} }
} }

View File

@@ -6,12 +6,14 @@ package container
import ( import (
"context" "context"
"errors"
"fmt" "fmt"
"io" "io"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
"github.com/docker/go-connections/nat" "github.com/docker/go-connections/nat"
"github.com/moby/moby/api/types/container"
) )
// ExitCodeError reports a non-zero process exit code from a container command. // ExitCodeError reports a non-zero process exit code from a container command.
@@ -57,6 +59,32 @@ type FileEntry struct {
Body string Body string
} }
// Container and healthcheck states, as plain strings so a caller of Info needs no docker
// SDK of its own.
const (
StateRunning = string(container.StateRunning)
HealthNone = string(container.NoHealthcheck)
HealthStarting = string(container.Starting)
HealthHealthy = string(container.Healthy)
HealthUnhealthy = string(container.Unhealthy)
)
// ErrContainerNotFound reports a container the daemon no longer knows. Its text is a
// fragment, missingContainerError composes it into the message every operation shares.
var ErrContainerNotFound = errors.New("does not exist")
// Info is a snapshot of a container, as of one inspect.
type Info struct {
ID string
State string // the docker container state: "created", "running", "exited", ...
ExitCode int
Health string // one of the Health* constants
// HealthOutput is the last healthcheck probe's output.
HealthOutput string
Ports map[string]string // container port ("5432") to the host port it is published on
}
// Container for managing docker run containers // Container for managing docker run containers
type Container interface { type Container interface {
Create(capAdd, capDrop []string) common.Executor Create(capAdd, capDrop []string) common.Executor
@@ -65,6 +93,8 @@ type Container interface {
CopyTarStream(ctx context.Context, destPath string, tarStream io.Reader) error CopyTarStream(ctx context.Context, destPath string, tarStream io.Reader) error
CopyDir(destPath, srcPath string, useGitIgnore bool) common.Executor CopyDir(destPath, srcPath string, useGitIgnore bool) common.Executor
GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error)
Inspect(ctx context.Context) (*Info, error)
DumpLogs(ctx context.Context) error
Pull(forcePull bool) common.Executor Pull(forcePull bool) common.Executor
Start(attach bool) common.Executor Start(attach bool) common.Executor
Exec(command []string, env map[string]string, user, workdir string) common.Executor Exec(command []string, env map[string]string, user, workdir string) common.Executor
@@ -89,6 +119,7 @@ type NewDockerBuildExecutorInput struct {
type NewDockerNetworkCreateExecutorInput struct { type NewDockerNetworkCreateExecutorInput struct {
EnableIPv4 *bool EnableIPv4 *bool
EnableIPv6 *bool EnableIPv6 *bool
RunnerUUID string
} }
// NewDockerPullExecutorInput the input for the NewDockerPullExecutor function // NewDockerPullExecutorInput the input for the NewDockerPullExecutor function

View File

@@ -4,8 +4,10 @@
//go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd)) //go:build !(WITHOUT_DOCKER || !(linux || darwin || windows || netbsd))
// This file is exact copy of https://github.com/docker/cli/blob/9a471180cb7d39c236d090399a9d362c3f5a8ebd/cli/command/container/opts.go // This file is exact copy of https://github.com/docker/cli/blob/dfc4efb1e2ab8c06d70d2a1366ad448d2f917e90/cli/command/container/opts.go with:
// appended with license information. // * appended with license information
// * regexp and loader.ParseVolume in place of the import-restricted internal/lazyregexp and internal/volumespec
// * invalidParameter from the package's errors.go, and convertPortSet/convertPortMap for the callers in docker_run.go
// //
// docker/cli is licensed under the Apache License, Version 2.0. // docker/cli is licensed under the Apache License, Version 2.0.
// See DOCKER_LICENSE for the full license text. // See DOCKER_LICENSE for the full license text.
@@ -30,6 +32,7 @@ import (
"strings" "strings"
"time" "time"
cerrdefs "github.com/containerd/errdefs"
"github.com/docker/cli/cli/compose/loader" "github.com/docker/cli/cli/compose/loader"
"github.com/docker/cli/opts" "github.com/docker/cli/opts"
"github.com/docker/go-connections/nat" "github.com/docker/go-connections/nat"
@@ -380,7 +383,7 @@ func parse(flags *pflag.FlagSet, copts *containerOptions, serverOS string) (*con
var binds []string var binds []string
volumes := copts.volumes.GetMap() volumes := copts.volumes.GetMap()
// add any bind targets to the list of container volumes // add any bind targets to the list of container volumes
for bind := range copts.volumes.GetMap() { for bind := range volumes {
parsed, err := loader.ParseVolume(bind) parsed, err := loader.ParseVolume(bind)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -515,13 +518,13 @@ func parse(flags *pflag.FlagSet, copts *containerOptions, serverOS string) (*con
// collect all the environment variables for the container // collect all the environment variables for the container
envVariables, err := opts.ReadKVEnvStrings(copts.envFile.GetSlice(), copts.env.GetSlice()) envVariables, err := opts.ReadKVEnvStrings(copts.envFile.GetSlice(), copts.env.GetSlice())
if err != nil { if err != nil {
return nil, err return nil, fmt.Errorf("--env-file: %w", err)
} }
// collect all the labels for the container // collect all the labels for the container
labels, err := opts.ReadKVStrings(copts.labelsFile.GetSlice(), copts.labels.GetSlice()) labels, err := opts.ReadKVStrings(copts.labelsFile.GetSlice(), copts.labels.GetSlice())
if err != nil { if err != nil {
return nil, err return nil, fmt.Errorf("--label-file: %w", err)
} }
pidMode := container.PidMode(copts.pidMode) pidMode := container.PidMode(copts.pidMode)
@@ -1164,16 +1167,17 @@ func toNetipAddrSlice(ips []string) []netip.Addr {
} }
// invalidParameter wraps an error to indicate it was caused by invalid input. // invalidParameter wraps an error to indicate it was caused by invalid input.
// This is a local replacement for docker/docker/errdefs.InvalidParameter. // This is a copy of docker/cli's cli/command/container/errors.go, which is not importable.
type invalidParameterError struct{ error } type invalidParameterErr struct{ error }
func (e invalidParameterError) InvalidParameter() {} func (invalidParameterErr) InvalidParameter() {}
func (e invalidParameterErr) Unwrap() error { return e.error }
func invalidParameter(err error) error { func invalidParameter(err error) error {
if err == nil { if err == nil || cerrdefs.IsInvalidArgument(err) {
return nil return err
} }
return invalidParameterError{err} return invalidParameterErr{err}
} }
func convertPortSet(ports nat.PortSet) (network.PortSet, error) { func convertPortSet(ports nat.PortSet) (network.PortSet, error) {

View File

@@ -2,20 +2,22 @@
// Copyright 2022 The nektos/act Authors. All rights reserved. // Copyright 2022 The nektos/act Authors. All rights reserved.
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
// This file is exact copy of https://github.com/docker/cli/blob/9ac8584acfd501c3f4da0e845e3a40ed15c85041/cli/command/container/opts_test.go with: // This file is exact copy of https://github.com/docker/cli/blob/dfc4efb1e2ab8c06d70d2a1366ad448d2f917e90/cli/command/container/opts_test.go with:
// * appended with license information // * appended with license information
// * commented out case 'invalid-mixed-network-types' in test TestParseNetworkConfig // * added tests for the locally changed parseDevice, validateDevice and invalidParameter
// //
// docker/cli is licensed under the Apache License, Version 2.0. // docker/cli is licensed under the Apache License, Version 2.0.
// See DOCKER_LICENSE for the full license text. // See DOCKER_LICENSE for the full license text.
// //
//nolint:depguard,gocritic // verbatim copy from docker/cli tests //nolint:gocritic // verbatim copy from docker/cli tests
package container package container
import ( import (
"errors"
"fmt" "fmt"
"io" "io"
"net"
"net/netip" "net/netip"
"os" "os"
"runtime" "runtime"
@@ -23,18 +25,23 @@ import (
"testing" "testing"
"time" "time"
"github.com/docker/go-connections/nat"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts" "github.com/google/go-cmp/cmp/cmpopts"
"github.com/moby/moby/api/types/container" "github.com/moby/moby/api/types/container"
networktypes "github.com/moby/moby/api/types/network" networktypes "github.com/moby/moby/api/types/network"
"github.com/pkg/errors"
"github.com/spf13/pflag" "github.com/spf13/pflag"
"gotest.tools/v3/assert" "gotest.tools/v3/assert"
is "gotest.tools/v3/assert/cmp" is "gotest.tools/v3/assert/cmp"
"gotest.tools/v3/skip" "gotest.tools/v3/skip"
) )
func mustParseMAC(s string) networktypes.HardwareAddr {
mac, err := net.ParseMAC(s)
if err != nil {
panic(err)
}
return networktypes.HardwareAddr(mac)
}
func TestValidateAttach(t *testing.T) { func TestValidateAttach(t *testing.T) {
valid := []string{ valid := []string{
"stdin", "stdin",
@@ -64,12 +71,12 @@ func parseRun(args []string) (*container.Config, *container.HostConfig, *network
if err := flags.Parse(args); err != nil { if err := flags.Parse(args); err != nil {
return nil, nil, nil, err return nil, nil, nil, err
} }
// TODO: fix tests to accept ContainerConfig // TODO(dnephin): fix tests to accept ContainerConfig; see https://github.com/moby/moby/pull/31621
containerConfig, err := parse(flags, copts, runtime.GOOS) containerCfg, err := parse(flags, copts, runtime.GOOS)
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, nil, err
} }
return containerConfig.Config, containerConfig.HostConfig, containerConfig.NetworkingConfig, err return containerCfg.Config, containerCfg.HostConfig, containerCfg.NetworkingConfig, err
} }
func setupRunFlags() (*pflag.FlagSet, *containerOptions) { func setupRunFlags() (*pflag.FlagSet, *containerOptions) {
@@ -82,20 +89,81 @@ func setupRunFlags() (*pflag.FlagSet, *containerOptions) {
func mustParse(t *testing.T, args string) (*container.Config, *container.HostConfig, *networktypes.NetworkingConfig) { func mustParse(t *testing.T, args string) (*container.Config, *container.HostConfig, *networktypes.NetworkingConfig) {
t.Helper() t.Helper()
config, hostConfig, networkingConfig, err := parseRun(append(strings.Split(args, " "), "ubuntu", "bash")) config, hostConfig, nwConfig, err := parseRun(append(strings.Split(args, " "), "ubuntu", "bash"))
assert.NilError(t, err) assert.NilError(t, err)
return config, hostConfig, networkingConfig return config, hostConfig, nwConfig
} }
func TestParseRunLinks(t *testing.T) { func TestParseRunLinks(t *testing.T) {
if _, hostConfig, _ := mustParse(t, "--link a:b"); len(hostConfig.Links) == 0 || hostConfig.Links[0] != "a:b" { tests := []struct {
t.Fatalf("Error parsing links. Expected []string{\"a:b\"}, received: %v", hostConfig.Links) name string
input string
expHostConfigLinks []string
expNetConfigLinks map[string][]string
}{
// Default bridge - legacy links ...
{
name: "default/onelink",
input: "--link a:b",
expHostConfigLinks: []string{"a:b"},
expNetConfigLinks: map[string][]string{"default": nil},
},
{
name: "default/twolinks",
input: "--link a:b --link c:d",
expHostConfigLinks: []string{"a:b", "c:d"},
expNetConfigLinks: map[string][]string{"default": nil},
},
{
name: "bridge/onelink",
input: "--network bridge --link a:b",
expHostConfigLinks: []string{"a:b"},
// expNetConfigLinks - no EndpointsConfig is created for a single named network with no options set.
// See the "For backward compatibility" comment in parseNetworkOpts().
},
{
name: "default/nolinks",
expNetConfigLinks: map[string][]string{"default": nil},
},
// User-defined bridge - links become DNS aliases ...
{
name: "userdefnet/onelink",
input: "--network userdefnet --link a:b",
expHostConfigLinks: []string{"a:b"},
expNetConfigLinks: map[string][]string{"userdefnet": {"a:b"}},
},
{
name: "userdefnet/twolinks",
input: "--network userdefnet --link a:b --link c:d",
expHostConfigLinks: []string{"a:b", "c:d"},
expNetConfigLinks: map[string][]string{"userdefnet": {"a:b", "c:d"}},
},
{
name: "userdefnet/nolinks",
input: "--network userdefnet",
},
{
// Link options are applied to the first network (and there's no "advanced syntax"
// link key, like "--network name=userdefnet,link=a:b").
name: "links apply to the first network",
input: "--network userdefnet --link a:b --network bar --link c:d",
expHostConfigLinks: []string{"a:b", "c:d"},
expNetConfigLinks: map[string][]string{"userdefnet": {"a:b", "c:d"}, "bar": nil},
},
} }
if _, hostConfig, _ := mustParse(t, "--link a:b --link c:d"); len(hostConfig.Links) < 2 || hostConfig.Links[0] != "a:b" || hostConfig.Links[1] != "c:d" {
t.Fatalf("Error parsing links. Expected []string{\"a:b\", \"c:d\"}, received: %v", hostConfig.Links) for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, hostConfig, netConfig := mustParse(t, tc.input)
assert.Check(t, is.DeepEqual(hostConfig.Links, tc.expHostConfigLinks))
assert.Check(t, is.Len(netConfig.EndpointsConfig, len(tc.expNetConfigLinks)))
for netName, expLinks := range tc.expNetConfigLinks {
nc, ok := netConfig.EndpointsConfig[netName]
assert.Assert(t, ok)
assert.Check(t, is.DeepEqual(nc.Links, expLinks))
} }
if _, hostConfig, _ := mustParse(t, ""); len(hostConfig.Links) != 0 { })
t.Fatalf("Error parsing links. No link expected, received: %v", hostConfig.Links)
} }
} }
@@ -294,37 +362,7 @@ func compareRandomizedStrings(a, b, c, d string) error {
if a == d && b == c { if a == d && b == c {
return nil return nil
} }
return errors.Errorf("strings don't match") return errors.New("strings don't match")
}
func mustNetworkPort(t *testing.T, value string) networktypes.Port {
t.Helper()
port, err := networktypes.ParsePort(value)
if err != nil {
t.Fatalf("failed to parse network port %q: %v", value, err)
}
return port
}
func mustAddr(t *testing.T, value string) netip.Addr {
t.Helper()
addr, err := netip.ParseAddr(value)
if err != nil {
t.Fatalf("failed to parse address %q: %v", value, err)
}
return addr
}
func mustAddrs(t *testing.T, values ...string) []netip.Addr {
t.Helper()
addrs := make([]netip.Addr, 0, len(values))
for _, value := range values {
addrs = append(addrs, mustAddr(t, value))
}
return addrs
} }
// Simple parse with MacAddress validation // Simple parse with MacAddress validation
@@ -334,10 +372,11 @@ func TestParseWithMacAddress(t *testing.T) {
if _, _, _, err := parseRun([]string{invalidMacAddress, "img", "cmd"}); err != nil && err.Error() != "invalidMacAddress is not a valid mac address" { if _, _, _, err := parseRun([]string{invalidMacAddress, "img", "cmd"}); err != nil && err.Error() != "invalidMacAddress is not a valid mac address" {
t.Fatalf("Expected an error with %v mac-address, got %v", invalidMacAddress, err) t.Fatalf("Expected an error with %v mac-address, got %v", invalidMacAddress, err)
} }
_, hostConfig, networkingConfig := mustParse(t, validMacAddress) _, hostConfig, nwConfig := mustParse(t, validMacAddress)
endpoint := networkingConfig.EndpointsConfig[string(hostConfig.NetworkMode)] defaultNw := hostConfig.NetworkMode.NetworkName()
assert.Check(t, endpoint != nil) if nwConfig.EndpointsConfig[defaultNw].MacAddress.String() != "92:d0:c6:0a:29:33" {
assert.Equal(t, "92:d0:c6:0a:29:33", endpoint.MacAddress.String()) t.Fatalf("Expected the default endpoint to have the MacAddress '92:d0:c6:0a:29:33' set, got '%v'", nwConfig.EndpointsConfig[defaultNw].MacAddress)
}
} }
func TestRunFlagsParseWithMemory(t *testing.T) { func TestRunFlagsParseWithMemory(t *testing.T) {
@@ -408,93 +447,144 @@ func TestParseHostnameDomainname(t *testing.T) {
} }
func TestParseWithExpose(t *testing.T) { func TestParseWithExpose(t *testing.T) {
invalids := []string{ t.Run("invalid", func(t *testing.T) {
":", tests := map[string]string{
"8080:9090", ":": `invalid range format for --expose: invalid start port ':': invalid syntax`,
"/tcp", "8080:9090": `invalid range format for --expose: invalid start port '8080:9090': invalid syntax`,
"/udp", "/tcp": `invalid range format for --expose: invalid start port '': value is empty`,
"NaN/tcp", "/udp": `invalid range format for --expose: invalid start port '': value is empty`,
"NaN-NaN/tcp", "NaN/tcp": `invalid range format for --expose: invalid start port 'NaN': invalid syntax`,
"8080-NaN/tcp", "NaN-NaN/tcp": `invalid range format for --expose: invalid start port 'NaN': invalid syntax`,
"1234567890-8080/tcp", "8080-NaN/tcp": `invalid range format for --expose: invalid end port 'NaN': invalid syntax`,
"1234567890-8080/tcp": `invalid range format for --expose: invalid start port '1234567890': value out of range`,
} }
valids := map[string][]nat.Port{ for expose, expectedError := range tests {
"8080/tcp": {"8080/tcp"}, t.Run(expose, func(t *testing.T) {
"8080/udp": {"8080/udp"}, _, _, _, err := parseRun([]string{fmt.Sprintf("--expose=%v", expose), "img", "cmd"})
"8080/ncp": {"8080/ncp"}, assert.Error(t, err, expectedError)
"8080-8080/udp": {"8080/udp"}, })
"8080-8082/tcp": {"8080/tcp", "8081/tcp", "8082/tcp"},
} }
for _, expose := range invalids { })
if _, _, _, err := parseRun([]string{fmt.Sprintf("--expose=%v", expose), "img", "cmd"}); err == nil { t.Run("valid", func(t *testing.T) {
t.Fatalf("Expected error with '--expose=%v', got none", expose) tests := map[string][]networktypes.Port{
"8080/tcp": {networktypes.MustParsePort("8080/tcp")},
"8080/udp": {networktypes.MustParsePort("8080/udp")},
"8080/ncp": {networktypes.MustParsePort("8080/ncp")},
"8080-8080/udp": {networktypes.MustParsePort("8080/udp")},
"8080-8082/tcp": {networktypes.MustParsePort("8080/tcp"), networktypes.MustParsePort("8081/tcp"), networktypes.MustParsePort("8082/tcp")},
} }
} for expose, exposedPorts := range tests {
for expose, exposedPorts := range valids { t.Run(expose, func(t *testing.T) {
config, _, _, err := parseRun([]string{fmt.Sprintf("--expose=%v", expose), "img", "cmd"}) config, _, _, err := parseRun([]string{fmt.Sprintf("--expose=%v", expose), "img", "cmd"})
if err != nil { assert.NilError(t, err)
t.Fatal(err)
}
if len(config.ExposedPorts) != len(exposedPorts) {
t.Fatalf("Expected %v exposed port, got %v", len(exposedPorts), len(config.ExposedPorts))
}
for _, port := range exposedPorts { for _, port := range exposedPorts {
if _, ok := config.ExposedPorts[mustNetworkPort(t, string(port))]; !ok { _, ok := config.ExposedPorts[port]
t.Fatalf("Expected %v, got %v", exposedPorts, config.ExposedPorts) assert.Check(t, ok, "missing port %q in exposed ports: %#+v", port, config.ExposedPorts[port])
}
} }
})
} }
})
t.Run("merge with published", func(t *testing.T) {
// Merge with actual published port // Merge with actual published port
config, _, _, err := parseRun([]string{"--publish=80", "--expose=80-81/tcp", "img", "cmd"}) config, _, _, err := parseRun([]string{"--publish=80", "--expose=80-81/tcp", "img", "cmd"})
if err != nil { assert.NilError(t, err)
t.Fatal(err) assert.Check(t, is.Len(config.ExposedPorts, 2))
} ports := []networktypes.Port{networktypes.MustParsePort("80/tcp"), networktypes.MustParsePort("81/tcp")}
if len(config.ExposedPorts) != 2 {
t.Fatalf("Expected 2 exposed ports, got %v", config.ExposedPorts)
}
ports := []nat.Port{"80/tcp", "81/tcp"}
for _, port := range ports { for _, port := range ports {
if _, ok := config.ExposedPorts[mustNetworkPort(t, string(port))]; !ok { _, ok := config.ExposedPorts[port]
t.Fatalf("Expected %v, got %v", ports, config.ExposedPorts) assert.Check(t, ok, "missing port %q in exposed ports: %#+v", port, config.ExposedPorts[port])
}
} }
})
} }
func TestParseDevice(t *testing.T) { func TestParseDevice(t *testing.T) {
skip.If(t, runtime.GOOS != "linux") // Windows and macOS validate server-side skip.If(t, runtime.GOOS != "linux") // Windows and macOS validate server-side
valids := map[string]container.DeviceMapping{ testCases := []struct {
"/dev/snd": { devices []string
deviceMapping *container.DeviceMapping
deviceRequests []container.DeviceRequest
}{
{
devices: []string{"/dev/snd"},
deviceMapping: &container.DeviceMapping{
PathOnHost: "/dev/snd", PathOnHost: "/dev/snd",
PathInContainer: "/dev/snd", PathInContainer: "/dev/snd",
CgroupPermissions: "rwm", CgroupPermissions: "rwm",
}, },
"/dev/snd:rw": { },
{
devices: []string{"/dev/snd:rw"},
deviceMapping: &container.DeviceMapping{
PathOnHost: "/dev/snd", PathOnHost: "/dev/snd",
PathInContainer: "/dev/snd", PathInContainer: "/dev/snd",
CgroupPermissions: "rw", CgroupPermissions: "rw",
}, },
"/dev/snd:/something": { },
{
devices: []string{"/dev/snd:/something"},
deviceMapping: &container.DeviceMapping{
PathOnHost: "/dev/snd", PathOnHost: "/dev/snd",
PathInContainer: "/something", PathInContainer: "/something",
CgroupPermissions: "rwm", CgroupPermissions: "rwm",
}, },
"/dev/snd:/something:rw": { },
{
devices: []string{"/dev/snd:/something:rw"},
deviceMapping: &container.DeviceMapping{
PathOnHost: "/dev/snd", PathOnHost: "/dev/snd",
PathInContainer: "/something", PathInContainer: "/something",
CgroupPermissions: "rw", CgroupPermissions: "rw",
}, },
},
{
devices: []string{"vendor.com/class=name"},
deviceMapping: nil,
deviceRequests: []container.DeviceRequest{
{
Driver: "cdi",
DeviceIDs: []string{"vendor.com/class=name"},
},
},
},
{
devices: []string{"vendor.com/class=name", "/dev/snd:/something:rw"},
deviceMapping: &container.DeviceMapping{
PathOnHost: "/dev/snd",
PathInContainer: "/something",
CgroupPermissions: "rw",
},
deviceRequests: []container.DeviceRequest{
{
Driver: "cdi",
DeviceIDs: []string{"vendor.com/class=name"},
},
},
},
} }
for device, deviceMapping := range valids {
_, hostconfig, _, err := parseRun([]string{fmt.Sprintf("--device=%v", device), "img", "cmd"}) for _, tc := range testCases {
if err != nil { t.Run(fmt.Sprintf("%s", tc.devices), func(t *testing.T) {
t.Fatal(err) var args []string
for _, d := range tc.devices {
args = append(args, fmt.Sprintf("--device=%v", d))
} }
if len(hostconfig.Devices) != 1 { args = append(args, "img", "cmd")
t.Fatalf("Expected 1 devices, got %v", hostconfig.Devices)
_, hostconfig, _, err := parseRun(args)
assert.NilError(t, err)
if tc.deviceMapping != nil {
if assert.Check(t, is.Len(hostconfig.Devices, 1)) {
assert.Check(t, is.DeepEqual(*tc.deviceMapping, hostconfig.Devices[0]))
} }
if hostconfig.Devices[0] != deviceMapping { } else {
t.Fatalf("Expected %v, got %v", deviceMapping, hostconfig.Devices) assert.Check(t, is.Len(hostconfig.Devices, 0))
} }
assert.Check(t, is.DeepEqual(tc.deviceRequests, hostconfig.DeviceRequests))
})
} }
} }
@@ -576,20 +666,20 @@ func TestParseNetworkConfig(t *testing.T) {
name string name string
flags []string flags []string
expected map[string]*networktypes.EndpointSettings expected map[string]*networktypes.EndpointSettings
expectedCfg container.HostConfig expectedHostCfg container.HostConfig
expectedErr string expectedErr string
}{ }{
{ {
name: "single-network-legacy", name: "single-network-legacy",
flags: []string{"--network", "net1"}, flags: []string{"--network", "net1"},
expected: map[string]*networktypes.EndpointSettings{}, expected: map[string]*networktypes.EndpointSettings{},
expectedCfg: container.HostConfig{NetworkMode: "net1"}, expectedHostCfg: container.HostConfig{NetworkMode: "net1"},
}, },
{ {
name: "single-network-advanced", name: "single-network-advanced",
flags: []string{"--network", "name=net1"}, flags: []string{"--network", "name=net1"},
expected: map[string]*networktypes.EndpointSettings{}, expected: map[string]*networktypes.EndpointSettings{},
expectedCfg: container.HostConfig{NetworkMode: "net1"}, expectedHostCfg: container.HostConfig{NetworkMode: "net1"},
}, },
{ {
name: "single-network-legacy-with-options", name: "single-network-legacy-with-options",
@@ -607,15 +697,15 @@ func TestParseNetworkConfig(t *testing.T) {
expected: map[string]*networktypes.EndpointSettings{ expected: map[string]*networktypes.EndpointSettings{
"net1": { "net1": {
IPAMConfig: &networktypes.EndpointIPAMConfig{ IPAMConfig: &networktypes.EndpointIPAMConfig{
IPv4Address: mustAddr(t, "172.20.88.22"), IPv4Address: netip.MustParseAddr("172.20.88.22"),
IPv6Address: mustAddr(t, "2001:db8::8822"), IPv6Address: netip.MustParseAddr("2001:db8::8822"),
LinkLocalIPs: mustAddrs(t, "169.254.2.2", "fe80::169:254:2:2"), LinkLocalIPs: []netip.Addr{netip.MustParseAddr("169.254.2.2"), netip.MustParseAddr("fe80::169:254:2:2")},
}, },
Links: []string{"foo:bar", "bar:baz"}, Links: []string{"foo:bar", "bar:baz"},
Aliases: []string{"web1", "web2"}, Aliases: []string{"web1", "web2"},
}, },
}, },
expectedCfg: container.HostConfig{NetworkMode: "net1"}, expectedHostCfg: container.HostConfig{NetworkMode: "net1"},
}, },
{ {
name: "multiple-network-advanced-mixed", name: "multiple-network-advanced-mixed",
@@ -631,14 +721,15 @@ func TestParseNetworkConfig(t *testing.T) {
"--network-alias", "web2", "--network-alias", "web2",
"--network", "net2", "--network", "net2",
"--network", "name=net3,alias=web3,driver-opt=field3=value3,ip=172.20.88.22,ip6=2001:db8::8822", "--network", "name=net3,alias=web3,driver-opt=field3=value3,ip=172.20.88.22,ip6=2001:db8::8822",
"--network", "name=net4,mac-address=02:32:1c:23:00:04,link-local-ip=169.254.169.254",
}, },
expected: map[string]*networktypes.EndpointSettings{ expected: map[string]*networktypes.EndpointSettings{
"net1": { "net1": {
DriverOpts: map[string]string{"field1": "value1"}, DriverOpts: map[string]string{"field1": "value1"},
IPAMConfig: &networktypes.EndpointIPAMConfig{ IPAMConfig: &networktypes.EndpointIPAMConfig{
IPv4Address: mustAddr(t, "172.20.88.22"), IPv4Address: netip.MustParseAddr("172.20.88.22"),
IPv6Address: mustAddr(t, "2001:db8::8822"), IPv6Address: netip.MustParseAddr("2001:db8::8822"),
LinkLocalIPs: mustAddrs(t, "169.254.2.2", "fe80::169:254:2:2"), LinkLocalIPs: []netip.Addr{netip.MustParseAddr("169.254.2.2"), netip.MustParseAddr("fe80::169:254:2:2")},
}, },
Links: []string{"foo:bar", "bar:baz"}, Links: []string{"foo:bar", "bar:baz"},
Aliases: []string{"web1", "web2"}, Aliases: []string{"web1", "web2"},
@@ -647,17 +738,23 @@ func TestParseNetworkConfig(t *testing.T) {
"net3": { "net3": {
DriverOpts: map[string]string{"field3": "value3"}, DriverOpts: map[string]string{"field3": "value3"},
IPAMConfig: &networktypes.EndpointIPAMConfig{ IPAMConfig: &networktypes.EndpointIPAMConfig{
IPv4Address: mustAddr(t, "172.20.88.22"), IPv4Address: netip.MustParseAddr("172.20.88.22"),
IPv6Address: mustAddr(t, "2001:db8::8822"), IPv6Address: netip.MustParseAddr("2001:db8::8822"),
}, },
Aliases: []string{"web3"}, Aliases: []string{"web3"},
}, },
"net4": {
MacAddress: mustParseMAC("02:32:1c:23:00:04"),
IPAMConfig: &networktypes.EndpointIPAMConfig{
LinkLocalIPs: []netip.Addr{netip.MustParseAddr("169.254.169.254")},
}, },
expectedCfg: container.HostConfig{NetworkMode: "net1"}, },
},
expectedHostCfg: container.HostConfig{NetworkMode: "net1"},
}, },
{ {
name: "single-network-advanced-with-options", name: "single-network-advanced-with-options",
flags: []string{"--network", "name=net1,alias=web1,alias=web2,driver-opt=field1=value1,driver-opt=field2=value2,ip=172.20.88.22,ip6=2001:db8::8822"}, flags: []string{"--network", "name=net1,alias=web1,alias=web2,driver-opt=field1=value1,driver-opt=field2=value2,ip=172.20.88.22,ip6=2001:db8::8822,mac-address=02:32:1c:23:00:04"},
expected: map[string]*networktypes.EndpointSettings{ expected: map[string]*networktypes.EndpointSettings{
"net1": { "net1": {
DriverOpts: map[string]string{ DriverOpts: map[string]string{
@@ -665,19 +762,31 @@ func TestParseNetworkConfig(t *testing.T) {
"field2": "value2", "field2": "value2",
}, },
IPAMConfig: &networktypes.EndpointIPAMConfig{ IPAMConfig: &networktypes.EndpointIPAMConfig{
IPv4Address: mustAddr(t, "172.20.88.22"), IPv4Address: netip.MustParseAddr("172.20.88.22"),
IPv6Address: mustAddr(t, "2001:db8::8822"), IPv6Address: netip.MustParseAddr("2001:db8::8822"),
}, },
Aliases: []string{"web1", "web2"}, Aliases: []string{"web1", "web2"},
MacAddress: mustParseMAC("02:32:1c:23:00:04"),
}, },
}, },
expectedCfg: container.HostConfig{NetworkMode: "net1"}, expectedHostCfg: container.HostConfig{NetworkMode: "net1"},
}, },
{ {
name: "multiple-networks", name: "multiple-networks",
flags: []string{"--network", "net1", "--network", "name=net2"}, flags: []string{"--network", "net1", "--network", "name=net2"},
expected: map[string]*networktypes.EndpointSettings{"net1": {}, "net2": {}}, expected: map[string]*networktypes.EndpointSettings{"net1": {}, "net2": {}},
expectedCfg: container.HostConfig{NetworkMode: "net1"}, expectedHostCfg: container.HostConfig{NetworkMode: "net1"},
},
{
name: "advanced-options-with-standalone-mac-address-flag",
flags: []string{"--network=name=net1,alias=foobar", "--mac-address", "52:0f:f3:dc:50:10"},
expected: map[string]*networktypes.EndpointSettings{
"net1": {
Aliases: []string{"foobar"},
MacAddress: mustParseMAC("52:0f:f3:dc:50:10"),
},
},
expectedHostCfg: container.HostConfig{NetworkMode: "net1"},
}, },
{ {
name: "conflict-network", name: "conflict-network",
@@ -699,13 +808,26 @@ func TestParseNetworkConfig(t *testing.T) {
flags: []string{"--network", "name=net1,ip=172.20.88.22,ip6=2001:db8::8822", "--ip6", "2001:db8::8822"}, flags: []string{"--network", "name=net1,ip=172.20.88.22,ip6=2001:db8::8822", "--ip6", "2001:db8::8822"},
expectedErr: `conflicting options: cannot specify both --ip6 and per-network IPv6 address`, expectedErr: `conflicting options: cannot specify both --ip6 and per-network IPv6 address`,
}, },
// case is skipped as it fails w/o any change {
// name: "invalid-mixed-network-types",
//{ flags: []string{"--network", "name=host", "--network", "net1"},
// name: "invalid-mixed-network-types", expectedErr: `conflicting options: cannot attach both user-defined and non-user-defined network-modes`,
// flags: []string{"--network", "name=host", "--network", "net1"}, },
// expectedErr: `conflicting options: cannot attach both user-defined and non-user-defined network-modes`, {
//}, name: "conflict-options-link-local-ip",
flags: []string{"--network", "name=net1,link-local-ip=169.254.169.254", "--link-local-ip", "169.254.10.8"},
expectedErr: `conflicting options: cannot specify both --link-local-ip and per-network link-local IP addresses`,
},
{
name: "conflict-options-mac-address",
flags: []string{"--network", "name=net1,mac-address=02:32:1c:23:00:04", "--mac-address", "02:32:1c:23:00:04"},
expectedErr: `conflicting options: cannot specify both --mac-address and per-network MAC address`,
},
{
name: "invalid-mac-address",
flags: []string{"--network", "name=net1,mac-address=foobar"},
expectedErr: "foobar is not a valid mac address",
},
} }
for _, tc := range tests { for _, tc := range tests {
@@ -718,10 +840,8 @@ func TestParseNetworkConfig(t *testing.T) {
} }
assert.NilError(t, err) assert.NilError(t, err)
assert.DeepEqual(t, hConfig.NetworkMode, tc.expectedCfg.NetworkMode) assert.DeepEqual(t, hConfig.NetworkMode, tc.expectedHostCfg.NetworkMode)
if diff := cmp.Diff(tc.expected, nwConfig.EndpointsConfig, cmpopts.EquateComparable(netip.Addr{})); diff != "" { assert.DeepEqual(t, nwConfig.EndpointsConfig, tc.expected, cmpopts.EquateComparable(netip.Addr{}))
t.Fatalf("unexpected endpoints (-want +got):\n%s", diff)
}
}) })
} }
} }
@@ -770,42 +890,84 @@ func TestRunFlagsParseShmSize(t *testing.T) {
} }
func TestParseRestartPolicy(t *testing.T) { func TestParseRestartPolicy(t *testing.T) {
invalids := map[string]string{ tests := []struct {
"always:2:3": "invalid restart policy format: maximum retry count must be an integer", input string
"on-failure:invalid": "invalid restart policy format: maximum retry count must be an integer", expected container.RestartPolicy
} expectedErr string
valids := map[string]container.RestartPolicy{ }{
"": {}, {
"always": { input: "",
Name: "always",
MaximumRetryCount: 0,
}, },
"on-failure:1": { {
Name: "on-failure", input: "no",
expected: container.RestartPolicy{
Name: container.RestartPolicyDisabled,
},
},
{
input: ":1",
expectedErr: "invalid restart policy format: no policy provided before colon",
},
{
input: "always",
expected: container.RestartPolicy{
Name: container.RestartPolicyAlways,
},
},
{
input: "always:2:3",
expectedErr: "invalid restart policy format: maximum retry count must be an integer",
},
{
input: "on-failure:1",
expected: container.RestartPolicy{
Name: container.RestartPolicyOnFailure,
MaximumRetryCount: 1, MaximumRetryCount: 1,
}, },
},
{
input: "on-failure:invalid",
expectedErr: "invalid restart policy format: maximum retry count must be an integer",
},
{
input: "unless-stopped",
expected: container.RestartPolicy{
Name: container.RestartPolicyUnlessStopped,
},
},
{
input: "unless-stopped:invalid",
expectedErr: "invalid restart policy format: maximum retry count must be an integer",
},
// Unknown / invalid combinations: validation is handled by the daemon>
{
input: "anything:123",
expected: container.RestartPolicy{Name: "anything", MaximumRetryCount: 123},
},
{
input: "negative:-123",
expected: container.RestartPolicy{Name: "negative", MaximumRetryCount: -123},
},
} }
for restart, expectedError := range invalids { for _, tc := range tests {
if _, _, _, err := parseRun([]string{"--restart=" + restart, "img", "cmd"}); err == nil || err.Error() != expectedError { t.Run(tc.input, func(t *testing.T) {
t.Fatalf("Expected an error with message '%v' for %v, got %v", expectedError, restart, err) _, hostConfig, _, err := parseRun([]string{"--restart=" + tc.input, "img", "cmd"})
} if tc.expectedErr != "" {
} assert.Check(t, is.Error(err, tc.expectedErr))
for restart, expected := range valids { assert.Check(t, is.Nil(hostConfig))
_, hostconfig, _, err := parseRun([]string{fmt.Sprintf("--restart=%v", restart), "img", "cmd"}) } else {
if err != nil { assert.NilError(t, err)
t.Fatal(err) assert.Check(t, is.DeepEqual(hostConfig.RestartPolicy, tc.expected))
}
if hostconfig.RestartPolicy != expected {
t.Fatalf("Expected %v, got %v", expected, hostconfig.RestartPolicy)
} }
})
} }
} }
func TestParseRestartPolicyAutoRemove(t *testing.T) { func TestParseRestartPolicyAutoRemove(t *testing.T) {
_, _, _, err := parseRun([]string{"--rm", "--restart=always", "img", "cmd"}) //nolint:dogsled // verbatim copy from docker/cli tests _, _, _, err := parseRun([]string{"--rm", "--restart=always", "img", "cmd"}) //nolint:dogsled // verbatim copy from docker/cli tests
if err == nil { const expected = "conflicting options: cannot specify both --restart and --rm"
t.Fatal("Expected error for conflicting --restart and --rm, but got none") assert.Check(t, is.Error(err, expected))
}
} }
func TestParseHealth(t *testing.T) { func TestParseHealth(t *testing.T) {
@@ -841,8 +1003,8 @@ func TestParseHealth(t *testing.T) {
checkError("--no-healthcheck conflicts with --health-* options", checkError("--no-healthcheck conflicts with --health-* options",
"--no-healthcheck", "--health-cmd=/check.sh -q", "img", "cmd") "--no-healthcheck", "--health-cmd=/check.sh -q", "img", "cmd")
health = checkOk("--health-timeout=2s", "--health-retries=3", "--health-interval=4.5s", "--health-start-period=5s", "img", "cmd") health = checkOk("--health-timeout=2s", "--health-retries=3", "--health-interval=4.5s", "--health-start-period=5s", "--health-start-interval=1s", "img", "cmd")
if health.Timeout != 2*time.Second || health.Retries != 3 || health.Interval != 4500*time.Millisecond || health.StartPeriod != 5*time.Second { if health.Timeout != 2*time.Second || health.Retries != 3 || health.Interval != 4500*time.Millisecond || health.StartPeriod != 5*time.Second || health.StartInterval != 1*time.Second {
t.Fatalf("--health-*: got %#v", health) t.Fatalf("--health-*: got %#v", health)
} }
} }
@@ -863,13 +1025,13 @@ func TestParseLoggingOpts(t *testing.T) {
} }
func TestParseEnvfileVariables(t *testing.T) { //nolint:dupl // verbatim copy from docker/cli tests func TestParseEnvfileVariables(t *testing.T) { //nolint:dupl // verbatim copy from docker/cli tests
e := "open nonexistent: no such file or directory" expErr := "--env-file: open nonexistent: no such file or directory"
if runtime.GOOS == "windows" { if runtime.GOOS == "windows" {
e = "open nonexistent: The system cannot find the file specified." expErr = "--env-file: open nonexistent: The system cannot find the file specified."
} }
// env ko // env ko
if _, _, _, err := parseRun([]string{"--env-file=nonexistent", "img", "cmd"}); err == nil || err.Error() != e { if _, _, _, err := parseRun([]string{"--env-file=nonexistent", "img", "cmd"}); err == nil || err.Error() != expErr {
t.Fatalf("Expected an error with message '%s', got %v", e, err) t.Fatalf("Expected an error with message '%s', got %v", expErr, err)
} }
// env ok // env ok
config, _, _, err := parseRun([]string{"--env-file=testdata/valid.env", "img", "cmd"}) config, _, _, err := parseRun([]string{"--env-file=testdata/valid.env", "img", "cmd"})
@@ -905,7 +1067,7 @@ func TestParseEnvfileVariablesWithBOMUnicode(t *testing.T) {
} }
// UTF16 with BOM // UTF16 with BOM
e := "invalid env file" e := "invalid utf8 bytes at line"
if _, _, _, err := parseRun([]string{"--env-file=testdata/utf16.env", "img", "cmd"}); err == nil || !strings.Contains(err.Error(), e) { if _, _, _, err := parseRun([]string{"--env-file=testdata/utf16.env", "img", "cmd"}); err == nil || !strings.Contains(err.Error(), e) {
t.Fatalf("Expected an error with message '%s', got %v", e, err) t.Fatalf("Expected an error with message '%s', got %v", e, err)
} }
@@ -916,13 +1078,13 @@ func TestParseEnvfileVariablesWithBOMUnicode(t *testing.T) {
} }
func TestParseLabelfileVariables(t *testing.T) { //nolint:dupl // verbatim copy from docker/cli tests func TestParseLabelfileVariables(t *testing.T) { //nolint:dupl // verbatim copy from docker/cli tests
e := "open nonexistent: no such file or directory" expErr := "--label-file: open nonexistent: no such file or directory"
if runtime.GOOS == "windows" { if runtime.GOOS == "windows" {
e = "open nonexistent: The system cannot find the file specified." expErr = "--label-file: open nonexistent: The system cannot find the file specified."
} }
// label ko // label ko
if _, _, _, err := parseRun([]string{"--label-file=nonexistent", "img", "cmd"}); err == nil || err.Error() != e { if _, _, _, err := parseRun([]string{"--label-file=nonexistent", "img", "cmd"}); err == nil || err.Error() != expErr {
t.Fatalf("Expected an error with message '%s', got %v", e, err) t.Fatalf("Expected an error with message '%s', got %v", expErr, err)
} }
// label ok // label ok
config, _, _, err := parseRun([]string{"--label-file=testdata/valid.label", "img", "cmd"}) config, _, _, err := parseRun([]string{"--label-file=testdata/valid.label", "img", "cmd"})
@@ -943,12 +1105,8 @@ func TestParseLabelfileVariables(t *testing.T) { //nolint:dupl // verbatim copy
func TestParseEntryPoint(t *testing.T) { func TestParseEntryPoint(t *testing.T) {
config, _, _, err := parseRun([]string{"--entrypoint=anything", "cmd", "img"}) config, _, _, err := parseRun([]string{"--entrypoint=anything", "cmd", "img"})
if err != nil { assert.NilError(t, err)
t.Fatal(err) assert.Check(t, is.DeepEqual(config.Entrypoint, []string{"anything"}))
}
if len(config.Entrypoint) != 1 && config.Entrypoint[0] != "anything" {
t.Fatalf("Expected entrypoint 'anything', got %v", config.Entrypoint)
}
} }
func TestValidateDevice(t *testing.T) { func TestValidateDevice(t *testing.T) {
@@ -995,13 +1153,11 @@ func TestValidateDevice(t *testing.T) {
for path, expectedError := range invalid { for path, expectedError := range invalid {
if _, err := validateDevice(path, runtime.GOOS); err == nil { if _, err := validateDevice(path, runtime.GOOS); err == nil {
t.Fatalf("ValidateDevice(`%q`) should have failed validation", path) t.Fatalf("ValidateDevice(`%q`) should have failed validation", path)
} else { } else if err.Error() != expectedError {
if err.Error() != expectedError {
t.Fatalf("ValidateDevice(`%q`) error should contain %q, got %q", path, expectedError, err.Error()) t.Fatalf("ValidateDevice(`%q`) error should contain %q, got %q", path, expectedError, err.Error())
} }
} }
} }
}
func TestValidateDeviceByServerOS(t *testing.T) { func TestValidateDeviceByServerOS(t *testing.T) {
tests := []struct { tests := []struct {
@@ -1073,10 +1229,12 @@ func TestDeviceCgroupRulesAndInvalidParameter(t *testing.T) {
if invalidParameter(nil) != nil { if invalidParameter(nil) != nil {
t.Fatal("invalidParameter(nil) should be nil") t.Fatal("invalidParameter(nil) should be nil")
} }
err = invalidParameter(errors.New("bad input")) cause := errors.New("bad input")
assert.Assert(t, err != nil) err = invalidParameter(cause)
var invalid interface{ InvalidParameter() } var invalid interface{ InvalidParameter() }
assert.Assert(t, errors.As(err, &invalid)) assert.Assert(t, errors.As(err, &invalid))
assert.Assert(t, errors.Is(err, cause))
assert.Equal(t, invalidParameter(err), err) // already invalid, so not wrapped twice
} }
func TestParseSystemPaths(t *testing.T) { func TestParseSystemPaths(t *testing.T) {

View File

@@ -57,7 +57,7 @@ func logDockerResponse(logger logrus.FieldLogger, dockerResponse io.ReadCloser,
if msg.ErrorDetail.Message != "" { if msg.ErrorDetail.Message != "" {
writeLog(logger, isError, "%s", msg.ErrorDetail.Message) writeLog(logger, isError, "%s", msg.ErrorDetail.Message)
return errors.New(msg.Error) return errors.New(msg.ErrorDetail.Message)
} }
if msg.Status != "" { if msg.Status != "" {

View File

@@ -8,12 +8,69 @@ package container
import ( import (
"context" "context"
"errors"
"fmt"
"strings"
"time"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
"github.com/moby/moby/client" "github.com/moby/moby/client"
) )
const (
networkCreateAttempts = 3
networkCreateRetryDelay = time.Second
// marks the networks a runner creates for its jobs, so it can tell its own leftovers from
// those of another runner sharing the daemon
runnerUUIDLabel = "com.gitea.runner.uuid"
)
// RemoveOrphanNetworks removes the networks this runner created for jobs whose teardown did
// not get to them: the runner died with the job, the teardown timed out, or the network still
// had an endpoint on it at the time. Each one holds a subnet of the daemon's address pool
// until it is removed. Networks created after createdBefore are left alone, so a job starting
// while this runs cannot lose the network it has created but not yet attached a container to.
func RemoveOrphanNetworks(ctx context.Context, runnerUUID string, createdBefore time.Time) error {
cli, err := GetDockerClient(ctx)
if err != nil {
return fmt.Errorf("failed to connect to the docker daemon: %w", err)
}
defer cli.Close()
return removeOrphanNetworks(ctx, cli, runnerUUID, createdBefore)
}
func removeOrphanNetworks(ctx context.Context, cli client.APIClient, runnerUUID string, createdBefore time.Time) error {
networks, err := cli.NetworkList(ctx, client.NetworkListOptions{
Filters: make(client.Filters).Add("label", runnerUUIDLabel+"="+runnerUUID),
})
if err != nil {
return err
}
var errs []error
for _, n := range networks.Items {
result, err := cli.NetworkInspect(ctx, n.ID, client.NetworkInspectOptions{})
if err != nil {
errs = append(errs, fmt.Errorf("failed to inspect network %s: %w", n.Name, err))
continue
}
// the emptiness check, not the label, is what keeps a live job of another process
// sharing this registration safe
if len(result.Network.Containers) != 0 || result.Network.Created.After(createdBefore) {
continue
}
if _, err := cli.NetworkRemove(ctx, n.ID, client.NetworkRemoveOptions{}); err != nil {
errs = append(errs, fmt.Errorf("failed to remove network %s: %w", n.Name, err))
continue
}
common.Logger(ctx).Infof("removed docker network %s left behind by an earlier job", n.Name)
}
return errors.Join(errs...)
}
func NewDockerNetworkCreateExecutor(name string, opts NewDockerNetworkCreateExecutorInput) common.Executor { func NewDockerNetworkCreateExecutor(name string, opts NewDockerNetworkCreateExecutorInput) common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
cli, err := GetDockerClient(ctx) cli, err := GetDockerClient(ctx)
@@ -36,18 +93,45 @@ func NewDockerNetworkCreateExecutor(name string, opts NewDockerNetworkCreateExec
} }
} }
_, err = cli.NetworkCreate(ctx, name, client.NetworkCreateOptions{ for i := range networkCreateAttempts {
if i > 0 {
common.Logger(ctx).Infof("Waiting for a free docker address pool to create network %s", name)
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(time.Duration(i) * networkCreateRetryDelay):
}
}
if _, err = cli.NetworkCreate(ctx, name, client.NetworkCreateOptions{
Driver: "bridge", Driver: "bridge",
Scope: "local", Scope: "local",
EnableIPv4: opts.EnableIPv4, EnableIPv4: opts.EnableIPv4,
EnableIPv6: opts.EnableIPv6, EnableIPv6: opts.EnableIPv6,
}) Labels: runnerLabels(opts.RunnerUUID),
if err != nil { }); err == nil {
return err
}
return nil return nil
} }
if !isAddressPoolExhausted(err) {
return err
}
}
return fmt.Errorf("docker has no address pool left for this job's network, lower runner.capacity or widen default-address-pools in the docker daemon config: %w", err)
}
}
func runnerLabels(runnerUUID string) map[string]string {
if runnerUUID == "" {
return nil
}
return map[string]string{runnerUUIDLabel: runnerUUID}
}
// The daemon reports this as a plain invalid-parameter error, the same kind it uses for every
// malformed request, so the message is the only discriminator.
func isAddressPoolExhausted(err error) bool {
msg := err.Error()
return strings.Contains(msg, "all predefined address pools have been fully subnetted") ||
strings.Contains(msg, "could not find an available, non-overlapping IPv4 address pool among the defaults") // docker 24 and older
} }
func NewDockerNetworkRemoveExecutor(name string) common.Executor { func NewDockerNetworkRemoveExecutor(name string) common.Executor {
@@ -66,6 +150,7 @@ func NewDockerNetworkRemoveExecutor(name string) common.Executor {
} }
// For Gitea, reduce log noise // For Gitea, reduce log noise
// common.Logger(ctx).Debugf("%v", networks) // common.Logger(ctx).Debugf("%v", networks)
var errs []error
for _, n := range networks.Items { for _, n := range networks.Items {
if n.Name == name { if n.Name == name {
result, err := cli.NetworkInspect(ctx, n.ID, client.NetworkInspectOptions{}) result, err := cli.NetworkInspect(ctx, n.ID, client.NetworkInspectOptions{})
@@ -73,16 +158,17 @@ func NewDockerNetworkRemoveExecutor(name string) common.Executor {
return err return err
} }
if len(result.Network.Containers) == 0 { // it holds a subnet out of the daemon's pool until something reclaims it
if _, err = cli.NetworkRemove(ctx, n.ID, client.NetworkRemoveOptions{}); err != nil { if len(result.Network.Containers) != 0 {
common.Logger(ctx).Debugf("%v", err) common.Logger(ctx).Warnf("Refusing to remove network %s because it still has active endpoints, the idle cleanup reclaims it once they are gone", name)
continue
} }
} else { if _, err = cli.NetworkRemove(ctx, n.ID, client.NetworkRemoveOptions{}); err != nil {
common.Logger(ctx).Debugf("Refusing to remove network %v because it still has active endpoints", name) errs = append(errs, fmt.Errorf("failed to remove network %s: %w", name, err))
} }
} }
} }
return err return errors.Join(errs...)
} }
} }

View File

@@ -0,0 +1,50 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package container
import (
"context"
"errors"
"testing"
"time"
cerrdefs "github.com/containerd/errdefs"
"github.com/moby/moby/api/types/network"
mobyclient "github.com/moby/moby/client"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestIsAddressPoolExhausted(t *testing.T) {
assert.True(t, isAddressPoolExhausted(cerrdefs.ErrInvalidArgument.WithMessage("Error response from daemon: all predefined address pools have been fully subnetted")))
assert.True(t, isAddressPoolExhausted(errors.New("could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network")))
assert.False(t, isAddressPoolExhausted(cerrdefs.ErrInvalidArgument.WithMessage("invalid subnet 10.0.0.0/8: it overlaps with an existing network")))
}
// Of this runner's networks, only the ones nothing is attached to and old enough to predate
// any job now starting are the runner's to reclaim. An unexpected NetworkRemove fails the
// test on its own, since testify has no expectation to match it against.
func TestRemoveOrphanNetworks(t *testing.T) {
ctx := context.Background()
cutoff := time.Date(2026, time.April, 29, 20, 0, 0, 0, time.UTC)
client := &mockDockerClient{}
client.On("NetworkList", ctx, mobyclient.NetworkListOptions{
Filters: make(mobyclient.Filters).Add("label", runnerUUIDLabel+"=runner-1"),
}).Return(mobyclient.NetworkListResult{Items: []network.Summary{
{Network: network.Network{ID: "orphan"}},
{Network: network.Network{ID: "busy"}},
{Network: network.Network{ID: "starting"}},
}}, nil)
client.On("NetworkInspect", ctx, "orphan", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{}, nil)
client.On("NetworkInspect", ctx, "busy", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{Network: network.Inspect{Containers: map[string]network.EndpointResource{"c": {}}}}, nil)
client.On("NetworkInspect", ctx, "starting", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{Network: network.Inspect{Network: network.Network{Created: cutoff.Add(time.Second)}}}, nil)
client.On("NetworkRemove", ctx, "orphan", mobyclient.NetworkRemoveOptions{}).
Return(mobyclient.NetworkRemoveResult{}, nil)
require.NoError(t, removeOrphanNetworks(ctx, client, "runner-1", cutoff))
client.AssertExpectations(t)
}

View File

@@ -30,22 +30,18 @@ func NewDockerPullExecutor(input NewDockerPullExecutorInput) common.Executor {
return nil return nil
} }
pull := input.ForcePull // skip the pull when the image is already here: either none was forced, or a digest
if !pull { // pins the content so a forced pull could only fetch the same bytes again
if !input.ForcePull || isPinnedImage(input.Image) {
imageExists, err := ImageExistsLocally(ctx, input.Image, input.Platform) imageExists, err := ImageExistsLocally(ctx, input.Image, input.Platform)
logger.Debugf("Image exists? %v", imageExists) logger.Debugf("Image exists? %v", imageExists)
if err != nil { if err != nil {
return fmt.Errorf("unable to determine if image already exists for image '%s' (%s): %w", input.Image, input.Platform, err) return fmt.Errorf("unable to determine if image already exists for image '%s' (%s): %w", input.Image, input.Platform, err)
} }
if imageExists {
if !imageExists {
pull = true
}
}
if !pull {
return nil return nil
} }
}
imageRef := cleanImage(ctx, input.Image) imageRef := cleanImage(ctx, input.Image)
logger.Debugf("pulling image '%v' (%s)", imageRef, input.Platform) logger.Debugf("pulling image '%v' (%s)", imageRef, input.Platform)
@@ -61,23 +57,33 @@ func NewDockerPullExecutor(input NewDockerPullExecutorInput) common.Executor {
return err return err
} }
reader, err := cli.ImagePull(ctx, imageRef, imagePullOptions) // the daemon reports a failure that happens after the first progress line in the
// stream rather than on the call itself, so both have to be checked
_ = logDockerResponse(logger, reader, err != nil) pullOnce := func(opts client.ImagePullOptions) error {
reader, err := cli.ImagePull(ctx, imageRef, opts)
streamErr := logDockerResponse(logger, reader, err != nil)
if err != nil { if err != nil {
if imagePullOptions.RegistryAuth != "" && strings.Contains(err.Error(), "unauthorized") { return err
}
return streamErr
}
err = pullOnce(imagePullOptions)
if err != nil && imagePullOptions.RegistryAuth != "" && strings.Contains(err.Error(), "unauthorized") {
logger.Errorf("pulling image '%v' (%s) failed with credentials %s retrying without them, please check for stale docker config files", imageRef, input.Platform, err.Error()) logger.Errorf("pulling image '%v' (%s) failed with credentials %s retrying without them, please check for stale docker config files", imageRef, input.Platform, err.Error())
imagePullOptions.RegistryAuth = "" imagePullOptions.RegistryAuth = ""
reader, err = cli.ImagePull(ctx, imageRef, imagePullOptions) err = pullOnce(imagePullOptions)
_ = logDockerResponse(logger, reader, err != nil)
}
if err != nil {
return fmt.Errorf("failed to pull image '%s' (%s): %w", imageRef, input.Platform, err)
}
} }
if err == nil {
return nil return nil
} }
// a registry that is down should not fail a job whose image is already here
if exists, existsErr := ImageExistsLocally(ctx, input.Image, input.Platform); existsErr == nil && exists {
logger.Warnf("could not update image '%s' (%s), continuing with the local copy: %v", imageRef, input.Platform, err)
return nil
}
return fmt.Errorf("failed to pull image '%s' (%s): %w", imageRef, input.Platform, err)
}
} }
func getImagePullOptions(ctx context.Context, input NewDockerPullExecutorInput) (client.ImagePullOptions, error) { func getImagePullOptions(ctx context.Context, input NewDockerPullExecutorInput) (client.ImagePullOptions, error) {
@@ -122,6 +128,15 @@ func getImagePullOptions(ctx context.Context, input NewDockerPullExecutorInput)
return imagePullOptions, nil return imagePullOptions, nil
} }
func isPinnedImage(image string) bool {
ref, err := reference.ParseAnyReference(image)
if err != nil {
return false
}
_, pinned := ref.(reference.Canonical)
return pinned
}
func cleanImage(ctx context.Context, imageName string) string { func cleanImage(ctx context.Context, imageName string) string {
ref, err := reference.ParseAnyReference(imageName) ref, err := reference.ParseAnyReference(imageName)
if err != nil { if err != nil {

View File

@@ -6,11 +6,15 @@ package container
import ( import (
"context" "context"
"io"
"strings"
"testing" "testing"
"github.com/docker/cli/cli/config" "github.com/docker/cli/cli/config"
log "github.com/sirupsen/logrus" log "github.com/sirupsen/logrus"
"github.com/sirupsen/logrus/hooks/test"
assert "github.com/stretchr/testify/assert" assert "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
) )
func init() { func init() {
@@ -65,3 +69,21 @@ func TestGetImagePullOptions(t *testing.T) {
assert.NoError(t, err, "Failed to create ImagePullOptions") //nolint:testifylint // pre-existing issue from nektos/act assert.NoError(t, err, "Failed to create ImagePullOptions") //nolint:testifylint // pre-existing issue from nektos/act
assert.Equal(t, "eyJ1c2VybmFtZSI6InVzZXJuYW1lIiwicGFzc3dvcmQiOiJwYXNzd29yZFxuIiwic2VydmVyYWRkcmVzcyI6Imh0dHBzOi8vaW5kZXguZG9ja2VyLmlvL3YxLyJ9", options.RegistryAuth, "RegistryAuth should be taken from local docker config") assert.Equal(t, "eyJ1c2VybmFtZSI6InVzZXJuYW1lIiwicGFzc3dvcmQiOiJwYXNzd29yZFxuIiwic2VydmVyYWRkcmVzcyI6Imh0dHBzOi8vaW5kZXguZG9ja2VyLmlvL3YxLyJ9", options.RegistryAuth, "RegistryAuth should be taken from local docker config")
} }
// A digest-pinned image is immutable, so its local copy is always current.
func TestIsPinnedImage(t *testing.T) {
assert.True(t, isPinnedImage("alpine@sha256:28bd5fe8b56d1bd048e5babf5b10710ebe0bae67db86916198a6eec434943f8b"))
assert.False(t, isPinnedImage("alpine:latest"))
}
// The pull path reports a failure the daemon sent mid-stream, so it must carry the reason
// whichever of the two shapes the daemon used.
func TestLogDockerResponseError(t *testing.T) {
logger, _ := test.NewNullLogger()
streamErr := func(line string) error {
return logDockerResponse(logger, io.NopCloser(strings.NewReader(line)), false)
}
require.EqualError(t, streamErr(`{"error":"toomanyrequests: rate limit exceeded"}`), "toomanyrequests: rate limit exceeded")
require.EqualError(t, streamErr(`{"errorDetail":{"message":"unexpected EOF"}}`), "unexpected EOF")
require.NoError(t, streamErr(`{"status":"Downloading"}`))
}

View File

@@ -14,6 +14,7 @@ import (
"fmt" "fmt"
"io" "io"
"os" "os"
"path"
"path/filepath" "path/filepath"
"regexp" "regexp"
"runtime" "runtime"
@@ -26,7 +27,6 @@ import (
"gitea.com/gitea/runner/act/filecollector" "gitea.com/gitea/runner/act/filecollector"
"dario.cat/mergo" "dario.cat/mergo"
"github.com/Masterminds/semver"
cerrdefs "github.com/containerd/errdefs" cerrdefs "github.com/containerd/errdefs"
"github.com/docker/cli/cli/compose/loader" "github.com/docker/cli/cli/compose/loader"
"github.com/docker/cli/cli/connhelper" "github.com/docker/cli/cli/connhelper"
@@ -41,7 +41,9 @@ import (
"github.com/moby/moby/api/types/network" "github.com/moby/moby/api/types/network"
"github.com/moby/moby/api/types/system" "github.com/moby/moby/api/types/system"
"github.com/moby/moby/client" "github.com/moby/moby/client"
"github.com/moby/moby/client/pkg/versions"
specs "github.com/opencontainers/image-spec/specs-go/v1" specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/sirupsen/logrus"
) )
// drainGracePeriod bounds how long we wait for an output-copy goroutine to // drainGracePeriod bounds how long we wait for an output-copy goroutine to
@@ -90,19 +92,11 @@ func (cr *containerReference) connectToNetwork(name string, aliases []string) co
// supportsContainerImagePlatform returns true if the underlying Docker server // supportsContainerImagePlatform returns true if the underlying Docker server
// API version is 1.41 and beyond // API version is 1.41 and beyond
func supportsContainerImagePlatform(ctx context.Context, cli client.APIClient) bool { func supportsContainerImagePlatform(ctx context.Context, cli client.APIClient) bool {
logger := common.Logger(ctx)
ver, err := cli.ServerVersion(ctx, client.ServerVersionOptions{}) ver, err := cli.ServerVersion(ctx, client.ServerVersionOptions{})
if err != nil { if err != nil {
logger.Panicf("Failed to get Docker API Version: %s", err) common.Logger(ctx).Panicf("Failed to get Docker API Version: %s", err)
return false
} }
sv, err := semver.NewVersion(ver.APIVersion) return versions.GreaterThanOrEqualTo(ver.APIVersion, "1.41")
if err != nil {
logger.Panicf("Failed to unmarshal Docker Version: %s", err)
return false
}
constraint, _ := semver.NewConstraint(">= 1.41")
return constraint.Check(sv)
} }
func (cr *containerReference) Create(capAdd, capDrop []string) common.Executor { func (cr *containerReference) Create(capAdd, capDrop []string) common.Executor {
@@ -204,6 +198,109 @@ func (cr *containerReference) GetContainerArchive(ctx context.Context, srcPath s
return result.Content, nil return result.Content, nil
} }
// Inspect resolves the container by name when its id is not known yet. One the daemon no
// longer knows is reported as ErrContainerNotFound.
func (cr *containerReference) Inspect(ctx context.Context) (*Info, error) {
if common.Dryrun(ctx) {
return &Info{Health: HealthNone, Ports: map[string]string{}}, nil
}
if err := cr.connect()(ctx); err != nil {
return nil, err
}
if cr.id == "" { // a known id is trusted, find() would spend a call validating it
if err := cr.find()(ctx); err != nil {
return nil, err
}
}
if cr.id == "" {
return nil, cr.missingContainerError("inspect it")
}
result, err := cr.cli.ContainerInspect(ctx, cr.id, client.ContainerInspectOptions{})
if cerrdefs.IsNotFound(err) {
return nil, cr.missingContainerError("inspect it")
} else if err != nil {
return nil, err
}
return containerInfoFromInspect(result.Container), nil
}
// DumpLogs copies the container's log so far to its output writers.
func (cr *containerReference) DumpLogs(ctx context.Context) error {
if common.Dryrun(ctx) {
return nil
}
if err := cr.connect()(ctx); err != nil {
return err
}
if cr.id == "" {
return cr.missingContainerError("read its logs")
}
logs, err := cr.cli.ContainerLogs(ctx, cr.id, client.ContainerLogsOptions{ShowStdout: true, ShowStderr: true})
if err != nil {
return err
}
defer logs.Close()
return cr.copyOutput(logs)
}
// copyOutput writes a container stream to the writers the container was created with,
// demultiplexing it unless the container has a TTY, which sends a single raw stream.
func (cr *containerReference) copyOutput(stream io.Reader) error {
outWriter := cr.input.Stdout
if outWriter == nil {
outWriter = os.Stdout
}
errWriter := cr.input.Stderr
if errWriter == nil {
errWriter = os.Stderr
}
var err error
if !cr.input.AllocatePTY || os.Getenv("NORAW") != "" {
_, err = stdcopy.StdCopy(outWriter, errWriter, stream)
} else {
_, err = io.Copy(outWriter, stream)
}
// Flush any buffered, not-yet-newline-terminated trailing line so the final line of
// the output is not lost when it is not newline-terminated.
common.FlushWriter(outWriter)
common.FlushWriter(errWriter)
return err
}
func containerInfoFromInspect(inspect container.InspectResponse) *Info {
info := &Info{
ID: inspect.ID,
Health: HealthNone,
Ports: map[string]string{}, // an empty map, never null, in the expression context
}
if state := inspect.State; state != nil {
info.State = string(state.Status)
info.ExitCode = state.ExitCode
if health := state.Health; health != nil {
info.Health = string(health.Status)
if len(health.Log) > 0 {
info.HealthOutput = strings.TrimSpace(health.Log[len(health.Log)-1].Output)
}
}
}
if settings := inspect.NetworkSettings; settings != nil {
for port, bindings := range settings.Ports {
for _, binding := range bindings { // the last binding wins, a port maps to one host port
if binding.HostPort != "" {
info.Ports[port.Port()] = binding.HostPort
}
}
}
}
return info
}
func (cr *containerReference) UpdateFromEnv(srcPath string, env *map[string]string) common.Executor { func (cr *containerReference) UpdateFromEnv(srcPath string, env *map[string]string) common.Executor {
return parseEnvFile(cr, srcPath, env).IfNot(common.Dryrun) return parseEnvFile(cr, srcPath, env).IfNot(common.Dryrun)
} }
@@ -349,10 +446,10 @@ func (cr *containerReference) Close() common.Executor {
} }
} }
// missingContainerError is the shared "container X does not exist" error // missingContainerError is the shared "container X does not exist" error used by ops that
// used by ops that need a live cr.id. // need a live cr.id, wrapping ErrContainerNotFound so a caller can tell it from a failing daemon.
func (cr *containerReference) missingContainerError(format string, args ...any) error { func (cr *containerReference) missingContainerError(format string, args ...any) error {
return fmt.Errorf("container %q does not exist; cannot "+format, append([]any{cr.input.Name}, args...)...) return fmt.Errorf("container %q %w; cannot "+format, append([]any{cr.input.Name, ErrContainerNotFound}, args...)...)
} }
func (cr *containerReference) find() common.Executor { func (cr *containerReference) find() common.Executor {
@@ -386,32 +483,71 @@ func (cr *containerReference) find() common.Executor {
} }
} }
// isContainerGone reports whether a failed remove still left the container gone (NotFound or Conflict).
func isContainerGone(err error) bool {
return cerrdefs.IsNotFound(err) || cerrdefs.IsConflict(err)
}
func (cr *containerReference) remove() common.Executor { func (cr *containerReference) remove() common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
if cr.id == "" { idOrName := cr.id
if idOrName == "" && cr.input != nil {
idOrName = cr.input.Name
}
if idOrName == "" {
return nil return nil
} }
logger := common.Logger(ctx) logger := common.Logger(ctx)
_, err := cr.cli.ContainerRemove(ctx, cr.id, client.ContainerRemoveOptions{ // Kill first so removal never waits out a daemon's stop timeout: Docker kills outright
// on a forced remove, Podman sends SIGTERM and waits. Only worth it for a container
// this started, and removal can still deal with one it could not kill.
if cr.id != "" {
_, err := cr.cli.ContainerKill(ctx, cr.id, client.ContainerKillOptions{Signal: "SIGKILL"})
if err != nil && !cerrdefs.IsConflict(err) && !cerrdefs.IsNotFound(err) {
logger.Debugf("Container %s could not be killed: %v", cr.id, err)
}
}
_, err := cr.cli.ContainerRemove(ctx, idOrName, client.ContainerRemoveOptions{
RemoveVolumes: true, RemoveVolumes: true,
Force: true, Force: true,
}) })
if err != nil && !isContainerGone(err) { switch {
logger.Error(fmt.Errorf("failed to remove container: %w", err)) case cerrdefs.IsConflict(err):
// the daemon's own AutoRemove teardown is running, and it releases the volume
// references and the network endpoint only once it finishes
cr.waitForRemoval(ctx, idOrName)
case err != nil && !cerrdefs.IsNotFound(err):
logger.Error(fmt.Errorf("failed to remove container %s: %w", idOrName, err))
return nil // keep the id, the container is still there for a later Remove()
} }
logger.Debugf("Removed container: %v", cr.id) logger.Debugf("Removed container: %v", idOrName)
cr.id = "" cr.id = ""
return nil return nil
} }
} }
func (cr *containerReference) waitForRemoval(ctx context.Context, idOrName string) {
// per container, against the one minute the post-job executor allows for the whole
// cleanup, so a job with several services can spend most of that budget here
ctx, cancel := context.WithTimeout(ctx, 15*time.Second)
defer cancel()
waitResult := cr.cli.ContainerWait(ctx, idOrName, client.ContainerWaitOptions{
Condition: container.WaitConditionRemoved,
})
select {
case <-waitResult.Result:
case <-waitResult.Error:
case <-ctx.Done():
// the client delivers the result over an unbuffered channel, so leave a receiver
// behind or its goroutine parks on the send for the lifetime of the process
go func() {
select {
case <-waitResult.Result:
case <-waitResult.Error:
}
}()
common.Logger(ctx).Warnf("Timed out waiting for the daemon to remove container %s, its volumes and network may be left behind", idOrName)
}
}
func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config *container.Config, hostConfig *container.HostConfig) (*container.Config, *container.HostConfig, error) { func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config *container.Config, hostConfig *container.HostConfig) (*container.Config, *container.HostConfig, error) {
logger := common.Logger(ctx) logger := common.Logger(ctx)
input := cr.input input := cr.input
@@ -461,6 +597,16 @@ func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config
return nil, nil, fmt.Errorf("Cannot process container options: '%s': '%w'", input.Options, err) return nil, nil, fmt.Errorf("Cannot process container options: '%s': '%w'", input.Options, err)
} }
// For Gitea
// When privileged mode is disabled, container.options is workflow-controlled
// untrusted input. Strip the HostConfig fields that would let a workflow break
// out of the container (host namespaces, capability expansion, security profile
// overrides, device and runtime access). Otherwise these survive into the final
// HostConfig even though --privileged is forced off.
if !hostConfig.Privileged {
sanitizeOptionsHostConfig(logger, containerConfig.HostConfig)
}
logger.Debugf("Custom container.Config from options ==> %+v", containerConfig.Config) logger.Debugf("Custom container.Config from options ==> %+v", containerConfig.Config)
err = mergo.Merge(config, containerConfig.Config, mergo.WithOverride, mergo.WithAppendSlice) err = mergo.Merge(config, containerConfig.Config, mergo.WithOverride, mergo.WithAppendSlice)
@@ -537,7 +683,7 @@ func (cr *containerReference) create(capAdd, capDrop []string) common.Executor {
} }
var platSpecs *specs.Platform var platSpecs *specs.Platform
if supportsContainerImagePlatform(ctx, cr.cli) && cr.input.Platform != "" { if cr.input.Platform != "" && supportsContainerImagePlatform(ctx, cr.cli) {
platSpecs, err = parsePlatform(cr.input.Platform) platSpecs, err = parsePlatform(cr.input.Platform)
if err != nil { if err != nil {
return err return err
@@ -694,7 +840,7 @@ func (cr *containerReference) exec(cmd []string, env map[string]string, user, wo
} }
defer resp.Close() defer resp.Close()
err = cr.waitForCommand(ctx, isTerminal, resp.HijackedResponse, idResp, user, workdir) err = cr.waitForCommand(ctx, resp.HijackedResponse, idResp, user, workdir)
if err != nil { if err != nil {
return err return err
} }
@@ -752,7 +898,7 @@ func (cr *containerReference) tryReadGID() common.Executor {
return cr.tryReadID("-g", func(id int) { cr.GID = id }) return cr.tryReadID("-g", func(id int) { cr.GID = id })
} }
func (cr *containerReference) waitForCommand(ctx context.Context, isTerminal bool, resp client.HijackedResponse, _ client.ExecCreateResult, _, _ string) error { func (cr *containerReference) waitForCommand(ctx context.Context, resp client.HijackedResponse, _ client.ExecCreateResult, _, _ string) error {
logger := common.Logger(ctx) logger := common.Logger(ctx)
// Buffered so the copy goroutine never blocks on send if the grace-period // Buffered so the copy goroutine never blocks on send if the grace-period
@@ -760,28 +906,7 @@ func (cr *containerReference) waitForCommand(ctx context.Context, isTerminal boo
cmdResponse := make(chan error, 1) cmdResponse := make(chan error, 1)
go func() { go func() {
var outWriter io.Writer cmdResponse <- cr.copyOutput(resp.Reader)
outWriter = cr.input.Stdout
if outWriter == nil {
outWriter = os.Stdout
}
errWriter := cr.input.Stderr
if errWriter == nil {
errWriter = os.Stderr
}
var err error
if !isTerminal || os.Getenv("NORAW") != "" {
_, err = stdcopy.StdCopy(outWriter, errWriter, resp.Reader)
} else {
_, err = io.Copy(outWriter, resp.Reader)
}
// Flush any buffered, not-yet-newline-terminated trailing line so the
// final line of a command's output is not lost (e.g. an error message
// printed without a trailing newline before the process exits).
common.FlushWriter(outWriter)
common.FlushWriter(errWriter)
cmdResponse <- err
}() }()
select { select {
@@ -814,23 +939,59 @@ func (cr *containerReference) waitForCommand(ctx context.Context, isTerminal boo
} }
} }
// mkdirInContainer creates containerPath and returns it with the symlinked components
// replaced by the targets the daemon reports for them. Docker 29.7 rejects tar entries
// traversing a symlink to an absolute target, like the "/var/run" of most images, with
// "path escapes from parent", and not every daemon creates the implied parents of a
// directory entry, so one entry per missing component is extracted at the deepest
// existing ancestor.
// WORKAROUND: https://github.com/moby/moby/issues/53258
func (cr *containerReference) mkdirInContainer(ctx context.Context, containerPath string) (string, error) {
parts := strings.Split(strings.Trim(path.Clean(containerPath), "/"), "/")
existing := "/"
for i, part := range parts {
if part == "" {
return existing, nil
}
stat, err := cr.cli.ContainerStatPath(ctx, cr.id, client.ContainerStatPathOptions{Path: path.Join(existing, part)})
if err != nil {
// nothing below exists either, so create the remaining components
return path.Join(existing, path.Join(parts[i:]...)), cr.mkdirEntries(ctx, existing, parts[i:])
}
existing = path.Join(existing, part)
if target := stat.Stat.LinkTarget; target != "" {
if !path.IsAbs(target) {
target = path.Join(path.Dir(existing), target)
}
existing = target
}
}
return existing, nil
}
func (cr *containerReference) mkdirEntries(ctx context.Context, destPath string, missing []string) error {
buf := &bytes.Buffer{}
tw := tar.NewWriter(buf)
for i := range missing {
_ = tw.WriteHeader(&tar.Header{
Name: path.Join(missing[:i+1]...),
Mode: 0o777,
Typeflag: tar.TypeDir,
})
}
tw.Close()
_, err := cr.cli.CopyToContainer(ctx, cr.id, client.CopyToContainerOptions{
DestinationPath: destPath,
Content: buf,
})
return err
}
func (cr *containerReference) CopyTarStream(ctx context.Context, destPath string, tarStream io.Reader) error { func (cr *containerReference) CopyTarStream(ctx context.Context, destPath string, tarStream io.Reader) error {
if cr.id == "" { if cr.id == "" {
return cr.missingContainerError("copy to %s", destPath) return cr.missingContainerError("copy to %s", destPath)
} }
// Mkdir destPath, err := cr.mkdirInContainer(ctx, destPath)
buf := &bytes.Buffer{}
tw := tar.NewWriter(buf)
_ = tw.WriteHeader(&tar.Header{
Name: destPath,
Mode: 0o777,
Typeflag: tar.TypeDir,
})
tw.Close()
_, err := cr.cli.CopyToContainer(ctx, cr.id, client.CopyToContainerOptions{
DestinationPath: "/",
Content: buf,
})
if err != nil { if err != nil {
return fmt.Errorf("failed to mkdir to copy content to container: %w", err) return fmt.Errorf("failed to mkdir to copy content to container: %w", err)
} }
@@ -855,6 +1016,10 @@ func (cr *containerReference) copyDir(dstPath, srcPath string, useGitIgnore bool
return cr.missingContainerError("copy directory to %s", dstPath) return cr.missingContainerError("copy directory to %s", dstPath)
} }
logger := common.Logger(ctx) logger := common.Logger(ctx)
dstPath, err := cr.mkdirInContainer(ctx, dstPath)
if err != nil {
return fmt.Errorf("failed to mkdir to copy directory to container: %w", err)
}
tarFile, err := os.CreateTemp("", "act") tarFile, err := os.CreateTemp("", "act")
if err != nil { if err != nil {
return err return err
@@ -976,33 +1141,11 @@ func (cr *containerReference) attach() common.Executor {
if err != nil { if err != nil {
return fmt.Errorf("failed to attach to container: %w", err) return fmt.Errorf("failed to attach to container: %w", err)
} }
isTerminal := cr.input.AllocatePTY
var outWriter io.Writer
outWriter = cr.input.Stdout
if outWriter == nil {
outWriter = os.Stdout
}
errWriter := cr.input.Stderr
if errWriter == nil {
errWriter = os.Stderr
}
done := make(chan struct{}) done := make(chan struct{})
cr.attachDone = done cr.attachDone = done
go func() { go func() {
defer close(done) defer close(done)
var copyErr error if copyErr := cr.copyOutput(out.Reader); copyErr != nil {
if !isTerminal || os.Getenv("NORAW") != "" {
_, copyErr = stdcopy.StdCopy(outWriter, errWriter, out.Reader)
} else {
_, copyErr = io.Copy(outWriter, out.Reader)
}
// Flush any buffered, not-yet-newline-terminated trailing line once
// the stream reaches EOF, so the final line of the container's
// output is not lost when it is not newline-terminated.
common.FlushWriter(outWriter)
common.FlushWriter(errWriter)
if copyErr != nil {
common.Logger(ctx).Error(copyErr) common.Logger(ctx).Error(copyErr)
} }
}() }()
@@ -1062,6 +1205,77 @@ func (cr *containerReference) wait() common.Executor {
} }
} }
// For Gitea
// sanitizeOptionsHostConfig clears the HostConfig fields parsed from a
// workflow-controlled container.options string that could be used to escape the
// container when privileged mode is disabled. It must only be called when the
// runner has privileged mode turned off; with privileged mode enabled the
// administrator has already opted into host access.
func sanitizeOptionsHostConfig(logger logrus.FieldLogger, hostConfig *container.HostConfig) {
warn := func(option string) {
logger.Warnf("container option %q is not allowed when privileged mode is disabled and will be ignored", option)
}
if hostConfig.PidMode != "" {
warn("--pid")
hostConfig.PidMode = ""
}
if hostConfig.IpcMode != "" {
warn("--ipc")
hostConfig.IpcMode = ""
}
if hostConfig.UTSMode != "" {
warn("--uts")
hostConfig.UTSMode = ""
}
if hostConfig.CgroupnsMode != "" {
warn("--cgroupns")
hostConfig.CgroupnsMode = ""
}
// UsernsMode is set from the runner-controlled input; never let options
// override it (e.g. --userns=host disables user namespace remapping).
if hostConfig.UsernsMode != "" {
warn("--userns")
hostConfig.UsernsMode = ""
}
if len(hostConfig.CapAdd) > 0 {
warn("--cap-add")
hostConfig.CapAdd = nil
}
if len(hostConfig.SecurityOpt) > 0 {
warn("--security-opt")
hostConfig.SecurityOpt = nil
}
if len(hostConfig.Devices) > 0 {
warn("--device")
hostConfig.Devices = nil
}
if len(hostConfig.DeviceCgroupRules) > 0 {
warn("--device-cgroup-rule")
hostConfig.DeviceCgroupRules = nil
}
if len(hostConfig.DeviceRequests) > 0 {
warn("--gpus")
hostConfig.DeviceRequests = nil
}
if len(hostConfig.VolumesFrom) > 0 {
warn("--volumes-from")
hostConfig.VolumesFrom = nil
}
if hostConfig.Runtime != "" {
warn("--runtime")
hostConfig.Runtime = ""
}
if hostConfig.CgroupParent != "" {
warn("--cgroup-parent")
hostConfig.CgroupParent = ""
}
if len(hostConfig.Sysctls) > 0 {
warn("--sysctl")
hostConfig.Sysctls = nil
}
}
// For Gitea // For Gitea
// sanitizeConfig remove the invalid configurations from `config` and `hostConfig` // sanitizeConfig remove the invalid configurations from `config` and `hostConfig`
func (cr *containerReference) sanitizeConfig(ctx context.Context, config *container.Config, hostConfig *container.HostConfig) (*container.Config, *container.HostConfig) { func (cr *containerReference) sanitizeConfig(ctx context.Context, config *container.Config, hostConfig *container.HostConfig) (*container.Config, *container.HostConfig) {

View File

@@ -5,6 +5,7 @@
package container package container
import ( import (
"archive/tar"
"bufio" "bufio"
"bytes" "bytes"
"context" "context"
@@ -24,6 +25,7 @@ import (
"github.com/moby/moby/api/pkg/stdcopy" "github.com/moby/moby/api/pkg/stdcopy"
"github.com/moby/moby/api/types/container" "github.com/moby/moby/api/types/container"
"github.com/moby/moby/api/types/mount" "github.com/moby/moby/api/types/mount"
"github.com/moby/moby/api/types/network"
mobyclient "github.com/moby/moby/client" mobyclient "github.com/moby/moby/client"
"github.com/sirupsen/logrus/hooks/test" "github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
@@ -92,6 +94,11 @@ func (m *mockDockerClient) ExecInspect(ctx context.Context, execID string, opts
return args.Get(0).(mobyclient.ExecInspectResult), args.Error(1) return args.Get(0).(mobyclient.ExecInspectResult), args.Error(1)
} }
func (m *mockDockerClient) ContainerStatPath(ctx context.Context, containerID string, opts mobyclient.ContainerStatPathOptions) (mobyclient.ContainerStatPathResult, error) {
args := m.Called(ctx, containerID, opts)
return args.Get(0).(mobyclient.ContainerStatPathResult), args.Error(1)
}
func (m *mockDockerClient) ContainerAttach(ctx context.Context, containerID string, opts mobyclient.ContainerAttachOptions) (mobyclient.ContainerAttachResult, error) { func (m *mockDockerClient) ContainerAttach(ctx context.Context, containerID string, opts mobyclient.ContainerAttachOptions) (mobyclient.ContainerAttachResult, error) {
args := m.Called(ctx, containerID, opts) args := m.Called(ctx, containerID, opts)
return args.Get(0).(mobyclient.ContainerAttachResult), args.Error(1) return args.Get(0).(mobyclient.ContainerAttachResult), args.Error(1)
@@ -122,6 +129,26 @@ func (m *mockDockerClient) ContainerRemove(ctx context.Context, id string, opts
return args.Get(0).(mobyclient.ContainerRemoveResult), args.Error(1) return args.Get(0).(mobyclient.ContainerRemoveResult), args.Error(1)
} }
func (m *mockDockerClient) ContainerKill(ctx context.Context, id string, opts mobyclient.ContainerKillOptions) (mobyclient.ContainerKillResult, error) {
args := m.Called(ctx, id, opts)
return args.Get(0).(mobyclient.ContainerKillResult), args.Error(1)
}
func (m *mockDockerClient) NetworkList(ctx context.Context, opts mobyclient.NetworkListOptions) (mobyclient.NetworkListResult, error) {
args := m.Called(ctx, opts)
return args.Get(0).(mobyclient.NetworkListResult), args.Error(1)
}
func (m *mockDockerClient) NetworkInspect(ctx context.Context, id string, opts mobyclient.NetworkInspectOptions) (mobyclient.NetworkInspectResult, error) {
args := m.Called(ctx, id, opts)
return args.Get(0).(mobyclient.NetworkInspectResult), args.Error(1)
}
func (m *mockDockerClient) NetworkRemove(ctx context.Context, id string, opts mobyclient.NetworkRemoveOptions) (mobyclient.NetworkRemoveResult, error) {
args := m.Called(ctx, id, opts)
return args.Get(0).(mobyclient.NetworkRemoveResult), args.Error(1)
}
type endlessReader struct { type endlessReader struct {
io.Reader io.Reader
} }
@@ -315,15 +342,37 @@ func TestDockerWaitFailure(t *testing.T) {
client.AssertExpectations(t) client.AssertExpectations(t)
} }
// stubStatPath answers path resolution: the given paths exist, mapped to their target
// when they are a symlink, everything else does not exist.
func stubStatPath(client *mockDockerClient, existing map[string]string) {
for containerPath, target := range existing {
client.On("ContainerStatPath", mock.Anything, "123", mobyclient.ContainerStatPathOptions{Path: containerPath}).
Return(mobyclient.ContainerStatPathResult{Stat: container.PathStat{LinkTarget: target}}, nil).Maybe()
}
client.On("ContainerStatPath", mock.Anything, "123", mock.Anything).
Return(mobyclient.ContainerStatPathResult{}, cerrdefs.ErrNotFound).Maybe()
}
// The mkdir tarball is extracted at the deepest existing ancestor, with entries relative
// to it that never traverse the "/var/run" symlink, see moby/moby#53258.
func TestDockerCopyTarStream(t *testing.T) { func TestDockerCopyTarStream(t *testing.T) {
ctx := context.Background() ctx := context.Background()
var mkdirNames []string
client := &mockDockerClient{} client := &mockDockerClient{}
stubStatPath(client, map[string]string{"/var": "", "/var/run": "/run", "/run": ""})
client.On("CopyToContainer", ctx, "123", mock.MatchedBy(func(opts mobyclient.CopyToContainerOptions) bool { client.On("CopyToContainer", ctx, "123", mock.MatchedBy(func(opts mobyclient.CopyToContainerOptions) bool {
return opts.DestinationPath == "/" && opts.Content != nil if opts.DestinationPath != "/run" || opts.Content == nil {
return false
}
tr := tar.NewReader(opts.Content)
for hdr, err := tr.Next(); err == nil; hdr, err = tr.Next() {
mkdirNames = append(mkdirNames, hdr.Name)
}
return true
})).Return(mobyclient.CopyToContainerResult{}, nil) })).Return(mobyclient.CopyToContainerResult{}, nil)
client.On("CopyToContainer", ctx, "123", mock.MatchedBy(func(opts mobyclient.CopyToContainerOptions) bool { client.On("CopyToContainer", ctx, "123", mock.MatchedBy(func(opts mobyclient.CopyToContainerOptions) bool {
return opts.DestinationPath == "/var/run/act" && opts.Content != nil return opts.DestinationPath == "/run/act" && opts.Content != nil
})).Return(mobyclient.CopyToContainerResult{}, nil) })).Return(mobyclient.CopyToContainerResult{}, nil)
cr := &containerReference{ cr := &containerReference{
id: "123", id: "123",
@@ -333,46 +382,33 @@ func TestDockerCopyTarStream(t *testing.T) {
}, },
} }
_ = cr.CopyTarStream(ctx, "/var/run/act", &bytes.Buffer{}) require.NoError(t, cr.CopyTarStream(ctx, "/var/run/act", &bytes.Buffer{}))
assert.Equal(t, []string{"act"}, mkdirNames)
client.AssertExpectations(t) client.AssertExpectations(t)
} }
func TestDockerCopyTarStreamErrorInCopyFiles(t *testing.T) { func TestDockerCopyTarStreamErrors(t *testing.T) {
merr := errors.New("Failure")
for _, testCase := range []struct {
name string
mkdirErr error
copyErr error
}{
{"mkdir", merr, nil},
{"copy content", nil, merr},
} {
t.Run(testCase.name, func(t *testing.T) {
ctx := context.Background() ctx := context.Background()
merr := errors.New("Failure")
client := &mockDockerClient{} client := &mockDockerClient{}
stubStatPath(client, map[string]string{"/var": "", "/var/run": ""})
client.On("CopyToContainer", ctx, "123", mock.MatchedBy(func(opts mobyclient.CopyToContainerOptions) bool { client.On("CopyToContainer", ctx, "123", mock.MatchedBy(func(opts mobyclient.CopyToContainerOptions) bool {
return opts.DestinationPath == "/" && opts.Content != nil return opts.DestinationPath == "/var/run" && opts.Content != nil
})).Return(mobyclient.CopyToContainerResult{}, merr) })).Return(mobyclient.CopyToContainerResult{}, testCase.mkdirErr)
cr := &containerReference{
id: "123",
cli: client,
input: &NewContainerInput{
Image: "image",
},
}
err := cr.CopyTarStream(ctx, "/var/run/act", &bytes.Buffer{})
assert.ErrorIs(t, err, merr) //nolint:testifylint // pre-existing issue from nektos/act
client.AssertExpectations(t)
}
func TestDockerCopyTarStreamErrorInMkdir(t *testing.T) {
ctx := context.Background()
merr := errors.New("Failure")
client := &mockDockerClient{}
client.On("CopyToContainer", ctx, "123", mock.MatchedBy(func(opts mobyclient.CopyToContainerOptions) bool {
return opts.DestinationPath == "/" && opts.Content != nil
})).Return(mobyclient.CopyToContainerResult{}, nil)
client.On("CopyToContainer", ctx, "123", mock.MatchedBy(func(opts mobyclient.CopyToContainerOptions) bool { client.On("CopyToContainer", ctx, "123", mock.MatchedBy(func(opts mobyclient.CopyToContainerOptions) bool {
return opts.DestinationPath == "/var/run/act" && opts.Content != nil return opts.DestinationPath == "/var/run/act" && opts.Content != nil
})).Return(mobyclient.CopyToContainerResult{}, merr) })).Return(mobyclient.CopyToContainerResult{}, testCase.copyErr).Maybe()
cr := &containerReference{ cr := &containerReference{
id: "123", id: "123",
cli: client, cli: client,
@@ -381,39 +417,50 @@ func TestDockerCopyTarStreamErrorInMkdir(t *testing.T) {
}, },
} }
err := cr.CopyTarStream(ctx, "/var/run/act", &bytes.Buffer{}) require.ErrorIs(t, cr.CopyTarStream(ctx, "/var/run/act", &bytes.Buffer{}), merr)
assert.ErrorIs(t, err, merr) //nolint:testifylint // pre-existing issue from nektos/act
client.AssertExpectations(t) client.AssertExpectations(t)
})
}
} }
// A remove that raced the daemon's AutoRemove teardown is not a failure and must not // A remove that raced the daemon's AutoRemove teardown is not a failure and must not
// be logged as one. // be logged as one.
func TestRemoveIgnoresAutoRemoveRace(t *testing.T) { func TestRemoveIgnoresAutoRemoveRace(t *testing.T) {
removeOpts := mobyclient.ContainerRemoveOptions{RemoveVolumes: true, Force: true} removeOpts := mobyclient.ContainerRemoveOptions{RemoveVolumes: true, Force: true}
killOpts := mobyclient.ContainerKillOptions{Signal: "SIGKILL"}
for _, tc := range []struct { for _, tc := range []struct {
name string name string
err error err error
wantLogs bool wantWait bool
wantFailure bool
}{ }{
{name: "removal in progress", err: cerrdefs.ErrConflict.WithMessage("removal of container abc is already in progress")}, {name: "removal in progress", err: cerrdefs.ErrConflict.WithMessage("removal of container abc is already in progress"), wantWait: true},
{name: "already removed", err: cerrdefs.ErrNotFound.WithMessage("No such container: abc")}, {name: "already removed", err: cerrdefs.ErrNotFound.WithMessage("No such container: abc")},
{name: "removed cleanly", err: nil}, {name: "removed cleanly", err: nil},
{name: "real failure", err: errors.New("driver failed to remove root filesystem"), wantLogs: true}, {name: "real failure", err: errors.New("driver failed to remove root filesystem"), wantFailure: true},
} { } {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
logger, hook := test.NewNullLogger() logger, hook := test.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger) ctx := common.WithLogger(context.Background(), logger)
client := &mockDockerClient{} client := &mockDockerClient{}
client.On("ContainerKill", ctx, "abc", killOpts).Return(mobyclient.ContainerKillResult{}, nil)
client.On("ContainerRemove", ctx, "abc", removeOpts).Return(mobyclient.ContainerRemoveResult{}, tc.err) client.On("ContainerRemove", ctx, "abc", removeOpts).Return(mobyclient.ContainerRemoveResult{}, tc.err)
if tc.wantWait {
removed := make(chan container.WaitResponse, 1)
removed <- container.WaitResponse{}
client.On("ContainerWait", mock.Anything, "abc", mobyclient.ContainerWaitOptions{Condition: container.WaitConditionRemoved}).
Return(mobyclient.ContainerWaitResult{Result: removed})
}
cr := &containerReference{id: "abc", cli: client} cr := &containerReference{id: "abc", cli: client}
require.NoError(t, cr.remove()(ctx)) require.NoError(t, cr.remove()(ctx))
assert.Empty(t, cr.id) // a failure keeps the id, so a later Remove() can retry it
if tc.wantFailure {
if tc.wantLogs { assert.Equal(t, "abc", cr.id)
assert.Len(t, hook.AllEntries(), 1) assert.Len(t, hook.AllEntries(), 1)
} else { } else {
assert.Empty(t, cr.id)
assert.Empty(t, hook.AllEntries()) assert.Empty(t, hook.AllEntries())
} }
client.AssertExpectations(t) client.AssertExpectations(t)
@@ -421,6 +468,20 @@ func TestRemoveIgnoresAutoRemoveRace(t *testing.T) {
} }
} }
// A container whose id was never learned, because find() could not reach the daemon or
// create() lost its reply, must still be removed rather than leaking with its network. It
// was never started here, so it is not worth a kill of its own.
func TestRemoveWithoutIDUsesName(t *testing.T) {
ctx := context.Background()
client := &mockDockerClient{}
client.On("ContainerRemove", ctx, "job-1", mobyclient.ContainerRemoveOptions{RemoveVolumes: true, Force: true}).
Return(mobyclient.ContainerRemoveResult{}, nil)
cr := &containerReference{cli: client, input: &NewContainerInput{Name: "job-1"}}
require.NoError(t, cr.remove()(ctx))
client.AssertExpectations(t)
}
// find() must drop a stale cached id so later Copy/Exec don't hit the // find() must drop a stale cached id so later Copy/Exec don't hit the
// daemon with a torn-down container. // daemon with a torn-down container.
func TestFindRevalidatesStaleID(t *testing.T) { func TestFindRevalidatesStaleID(t *testing.T) {
@@ -488,7 +549,7 @@ func TestRejectsMissingContainer(t *testing.T) {
cr := &containerReference{cli: client, input: &NewContainerInput{Name: "job-1"}} cr := &containerReference{cli: client, input: &NewContainerInput{Name: "job-1"}}
check := func(op string, err error) { check := func(op string, err error) {
t.Helper() t.Helper()
require.Error(t, err, op) require.ErrorIs(t, err, ErrContainerNotFound, op)
assert.Contains(t, err.Error(), `container "job-1" does not exist`, op) assert.Contains(t, err.Error(), `container "job-1" does not exist`, op)
} }
check("copyContent", cr.copyContent("/var/run/act", &FileEntry{Name: "x", Mode: 0o644})(ctx)) check("copyContent", cr.copyContent("/var/run/act", &FileEntry{Name: "x", Mode: 0o644})(ctx))
@@ -497,6 +558,15 @@ func TestRejectsMissingContainer(t *testing.T) {
check("exec", cr.exec([]string{"echo"}, nil, "", "")(ctx)) check("exec", cr.exec([]string{"echo"}, nil, "", "")(ctx))
_, err := cr.GetContainerArchive(ctx, "/var/run/act/x") _, err := cr.GetContainerArchive(ctx, "/var/run/act/x")
check("GetContainerArchive", err) check("GetContainerArchive", err)
_, err = cr.Inspect(ctx)
check("Inspect", err)
// a known id the daemon has since dropped
client.On("ContainerInspect", ctx, "gone", mobyclient.ContainerInspectOptions{}).
Return(mobyclient.ContainerInspectResult{}, cerrdefs.ErrNotFound)
removed := &containerReference{id: "gone", cli: client, input: &NewContainerInput{Name: "job-1"}}
_, err = removed.Inspect(ctx)
check("Inspect after removal", err)
} }
// End-to-end: a stale cr.id is cleared, repopulated from name lookup, // End-to-end: a stale cr.id is cleared, repopulated from name lookup,
@@ -542,9 +612,8 @@ func TestDockerCopyToSymlinkPath(t *testing.T) {
_ = rc.Close()(ctx) _ = rc.Close()(ctx)
}) })
// CopyTarStream first creates the destination directory by extracting a tar at "/", // CopyTarStream resolves the var/run symlink and creates act below its target, the
// which makes the daemon mkdir var, then var/run (the symlink), then act — the exact // exact step that fails on a broken daemon.
// step that fails on the broken daemon.
err := rc.CopyTarStream(ctx, "/var/run/act", &bytes.Buffer{}) err := rc.CopyTarStream(ctx, "/var/run/act", &bytes.Buffer{})
require.NoError(t, err) require.NoError(t, err)
} }
@@ -625,6 +694,110 @@ func TestCheckVolumes(t *testing.T) {
} }
} }
func TestSanitizeOptionsHostConfig(t *testing.T) {
logger, _ := test.NewNullLogger()
dangerous := func() *container.HostConfig {
return &container.HostConfig{
PidMode: "host",
IpcMode: "host",
UTSMode: "host",
CgroupnsMode: "host",
UsernsMode: "host",
CapAdd: []string{"ALL"},
SecurityOpt: []string{"seccomp=unconfined", "apparmor=unconfined"},
VolumesFrom: []string{"other"},
Runtime: "runc",
Resources: container.Resources{
CgroupParent: "/custom",
Devices: []container.DeviceMapping{{PathOnHost: "/dev/sda", PathInContainer: "/dev/sda", CgroupPermissions: "rwm"}},
DeviceCgroupRules: []string{"a *:* rwm"},
},
Sysctls: map[string]string{"net.ipv4.ip_forward": "1"},
}
}
hostConfig := dangerous()
sanitizeOptionsHostConfig(logger, hostConfig)
assert.Empty(t, string(hostConfig.PidMode))
assert.Empty(t, string(hostConfig.IpcMode))
assert.Empty(t, string(hostConfig.UTSMode))
assert.Empty(t, string(hostConfig.CgroupnsMode))
assert.Empty(t, string(hostConfig.UsernsMode))
assert.Empty(t, hostConfig.CapAdd)
assert.Empty(t, hostConfig.SecurityOpt)
assert.Empty(t, hostConfig.Devices)
assert.Empty(t, hostConfig.DeviceCgroupRules)
assert.Empty(t, hostConfig.VolumesFrom)
assert.Empty(t, hostConfig.Runtime)
assert.Empty(t, hostConfig.CgroupParent)
assert.Empty(t, hostConfig.Sysctls)
}
func TestMergeContainerConfigsStripsDangerousOptionsWhenUnprivileged(t *testing.T) {
// OS-independent options only: --device parsing requires a linux/windows
// server OS, which is not guaranteed for the test host.
const dangerousOptions = "--pid=host --ipc=host --uts=host --cgroupns=host " +
"--userns=host --cap-add=ALL --security-opt seccomp=unconfined " +
"--security-opt apparmor=unconfined --volumes-from other " +
"--runtime runc --cgroup-parent /custom --sysctl net.ipv4.ip_forward=1"
t.Run("unprivileged strips host-escape options", func(t *testing.T) {
logger, _ := test.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger)
cr := &containerReference{
input: &NewContainerInput{
Options: dangerousOptions,
NetworkMode: "bridge",
UsernsMode: "private",
},
}
_, hostConfig, err := cr.mergeContainerConfigs(ctx, &container.Config{}, &container.HostConfig{
Privileged: false,
UsernsMode: container.UsernsMode("private"),
NetworkMode: container.NetworkMode("bridge"),
})
require.NoError(t, err)
assert.False(t, hostConfig.Privileged)
assert.Empty(t, string(hostConfig.PidMode))
assert.Empty(t, string(hostConfig.IpcMode))
assert.Empty(t, string(hostConfig.UTSMode))
assert.Empty(t, string(hostConfig.CgroupnsMode))
// UsernsMode must keep the runner-controlled value, not the one from options.
assert.Equal(t, "private", string(hostConfig.UsernsMode))
assert.Empty(t, hostConfig.CapAdd)
assert.Empty(t, hostConfig.SecurityOpt)
assert.Empty(t, hostConfig.VolumesFrom)
assert.Empty(t, hostConfig.Runtime)
assert.Empty(t, hostConfig.CgroupParent)
assert.Empty(t, hostConfig.Sysctls)
})
t.Run("privileged preserves options", func(t *testing.T) {
logger, _ := test.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger)
cr := &containerReference{
input: &NewContainerInput{
Options: "--pid=host --cap-add=ALL --security-opt seccomp=unconfined",
NetworkMode: "bridge",
},
}
_, hostConfig, err := cr.mergeContainerConfigs(ctx, &container.Config{}, &container.HostConfig{
Privileged: true,
NetworkMode: container.NetworkMode("bridge"),
})
require.NoError(t, err)
assert.Equal(t, "host", string(hostConfig.PidMode))
assert.Equal(t, []string{"ALL"}, hostConfig.CapAdd)
assert.Equal(t, []string{"seccomp=unconfined"}, hostConfig.SecurityOpt)
})
}
func TestCheckVolumesRejectsEscapingHostPaths(t *testing.T) { func TestCheckVolumesRejectsEscapingHostPaths(t *testing.T) {
logger, _ := test.NewNullLogger() logger, _ := test.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger) ctx := common.WithLogger(context.Background(), logger)
@@ -662,6 +835,59 @@ func TestCheckVolumesRejectsEscapingHostPaths(t *testing.T) {
assert.Empty(t, hostConf.Binds) assert.Empty(t, hostConf.Binds)
} }
func TestContainerInfoFromInspect(t *testing.T) {
t.Run("reports no healthcheck when the image declares none", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{
ID: "abc123",
State: &container.State{Status: "running", Running: true},
})
assert.Equal(t, "abc123", info.ID)
assert.Equal(t, "running", info.State)
assert.Equal(t, HealthNone, info.Health)
assert.Empty(t, info.Ports)
})
t.Run("reports the health status and the last probe output", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{
State: &container.State{
Status: "running",
Health: &container.Health{
Status: container.Unhealthy,
Log: []*container.HealthcheckResult{
{Output: "first\n"},
{Output: "connection refused\n"},
},
},
},
})
assert.Equal(t, HealthUnhealthy, info.Health)
assert.Equal(t, "connection refused", info.HealthOutput)
})
t.Run("reports the published ports", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{
State: &container.State{Status: "running"},
NetworkSettings: &container.NetworkSettings{
Ports: network.PortMap{
network.MustParsePort("5432/tcp"): []network.PortBinding{{HostPort: "49153"}},
network.MustParsePort("6379/tcp"): nil,
},
},
})
assert.Equal(t, map[string]string{"5432": "49153"}, info.Ports)
})
t.Run("tolerates a container without state", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{ID: "abc123"})
assert.Equal(t, "abc123", info.ID)
assert.Equal(t, HealthNone, info.Health)
})
}
func TestMergeContainerConfigsVolumesReplaceRunnerMounts(t *testing.T) { func TestMergeContainerConfigsVolumesReplaceRunnerMounts(t *testing.T) {
logger, _ := test.NewNullLogger() logger, _ := test.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger) ctx := common.WithLogger(context.Background(), logger)

View File

@@ -8,12 +8,13 @@ package container
import ( import (
"context" "context"
"errors"
"runtime" "runtime"
"time"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
"github.com/moby/moby/api/types/system" "github.com/moby/moby/api/types/system"
"github.com/pkg/errors"
) )
// ImageExistsLocally returns a boolean indicating if an image with the // ImageExistsLocally returns a boolean indicating if an image with the
@@ -72,3 +73,7 @@ func NewDockerNetworkRemoveExecutor(name string) common.Executor {
return nil return nil
} }
} }
func RemoveOrphanNetworks(ctx context.Context, runnerUUID string, createdBefore time.Time) error {
return nil
}

View File

@@ -154,6 +154,14 @@ func (e *HostEnvironment) CopyDir(destPath, srcPath string, useGitIgnore bool) c
} }
} }
func (e *HostEnvironment) DumpLogs(_ context.Context) error {
return nil
}
func (e *HostEnvironment) Inspect(_ context.Context) (*Info, error) {
return &Info{Health: HealthNone, Ports: map[string]string{}}, nil
}
func (e *HostEnvironment) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) { func (e *HostEnvironment) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) {
buf := &bytes.Buffer{} buf := &bytes.Buffer{}
tw := tar.NewWriter(buf) tw := tar.NewWriter(buf)

View File

@@ -28,8 +28,8 @@ func (impl *interperterImpl) contains(search, item reflect.Value) (bool, error)
switch search.Kind() { switch search.Kind() {
case reflect.String, reflect.Int, reflect.Float64, reflect.Bool, reflect.Invalid: case reflect.String, reflect.Int, reflect.Float64, reflect.Bool, reflect.Invalid:
return strings.Contains( return strings.Contains(
strings.ToLower(impl.coerceToString(search).String()), strings.ToLower(CoerceToString(search)),
strings.ToLower(impl.coerceToString(item).String()), strings.ToLower(CoerceToString(item)),
), nil ), nil
case reflect.Slice: case reflect.Slice:
@@ -51,15 +51,15 @@ func (impl *interperterImpl) contains(search, item reflect.Value) (bool, error)
func (impl *interperterImpl) startsWith(searchString, searchValue reflect.Value) (bool, error) { //nolint:unparam // pre-existing issue from nektos/act func (impl *interperterImpl) startsWith(searchString, searchValue reflect.Value) (bool, error) { //nolint:unparam // pre-existing issue from nektos/act
return strings.HasPrefix( return strings.HasPrefix(
strings.ToLower(impl.coerceToString(searchString).String()), strings.ToLower(CoerceToString(searchString)),
strings.ToLower(impl.coerceToString(searchValue).String()), strings.ToLower(CoerceToString(searchValue)),
), nil ), nil
} }
func (impl *interperterImpl) endsWith(searchString, searchValue reflect.Value) (bool, error) { //nolint:unparam // pre-existing issue from nektos/act func (impl *interperterImpl) endsWith(searchString, searchValue reflect.Value) (bool, error) { //nolint:unparam // pre-existing issue from nektos/act
return strings.HasSuffix( return strings.HasSuffix(
strings.ToLower(impl.coerceToString(searchString).String()), strings.ToLower(CoerceToString(searchString)),
strings.ToLower(impl.coerceToString(searchValue).String()), strings.ToLower(CoerceToString(searchValue)),
), nil ), nil
} }
@@ -70,7 +70,7 @@ const (
) )
func (impl *interperterImpl) format(str reflect.Value, replaceValue ...reflect.Value) (string, error) { func (impl *interperterImpl) format(str reflect.Value, replaceValue ...reflect.Value) (string, error) {
input := impl.coerceToString(str).String() input := CoerceToString(str)
var output strings.Builder var output strings.Builder
replacementIndex := "" replacementIndex := ""
@@ -108,7 +108,7 @@ func (impl *interperterImpl) format(str reflect.Value, replaceValue ...reflect.V
return "", fmt.Errorf("The following format string references more arguments than were supplied: '%s'", input) return "", fmt.Errorf("The following format string references more arguments than were supplied: '%s'", input)
} }
output.WriteString(impl.coerceToString(replaceValue[index]).String()) output.WriteString(CoerceToString(replaceValue[index]))
state = passThrough state = passThrough
@@ -124,7 +124,7 @@ func (impl *interperterImpl) format(str reflect.Value, replaceValue ...reflect.V
state = passThrough state = passThrough
default: default:
panic("Invalid format parser state") return "", fmt.Errorf("Closing bracket without opening one. The following format string is invalid: '%s'", input)
} }
} }
} }
@@ -143,17 +143,17 @@ func (impl *interperterImpl) format(str reflect.Value, replaceValue ...reflect.V
} }
func (impl *interperterImpl) join(array, sep reflect.Value) (string, error) { //nolint:unparam // pre-existing issue from nektos/act func (impl *interperterImpl) join(array, sep reflect.Value) (string, error) { //nolint:unparam // pre-existing issue from nektos/act
separator := impl.coerceToString(sep).String() separator := CoerceToString(sep)
switch array.Kind() { switch array.Kind() {
case reflect.Slice: case reflect.Slice:
var items []string var items []string
for i := 0; i < array.Len(); i++ { for i := 0; i < array.Len(); i++ {
items = append(items, impl.coerceToString(array.Index(i).Elem()).String()) items = append(items, CoerceToString(array.Index(i)))
} }
return strings.Join(items, separator), nil return strings.Join(items, separator), nil
default: default:
return strings.Join([]string{impl.coerceToString(array).String()}, separator), nil return strings.Join([]string{CoerceToString(array)}, separator), nil
} }
} }

View File

@@ -121,6 +121,7 @@ func TestFunctionJoin(t *testing.T) {
{"join(fromJSON('[\"a\", \"b\", null]'), null)", "ab", "join-number"}, {"join(fromJSON('[\"a\", \"b\", null]'), null)", "ab", "join-number"},
{"join(fromJSON('[\"a\", \"b\"]'))", "a,b", "join-number"}, {"join(fromJSON('[\"a\", \"b\"]'))", "a,b", "join-number"},
{"join(fromJSON('[\"a\", \"b\", null]'), 1)", "a1b1", "join-number"}, {"join(fromJSON('[\"a\", \"b\", null]'), 1)", "a1b1", "join-number"},
{"join(fromJSON('[1, true, null]'), '-')", "1-true-", "join-mixed-types"},
} }
env := &EvaluationEnvironment{} env := &EvaluationEnvironment{}
@@ -230,8 +231,10 @@ func TestFunctionFormat(t *testing.T) {
{`format('Hello "{0}" {1} {2} {3} {4}', null, true, -3.14, NaN, Infinity)`, `Hello "" true -3.14 NaN Infinity`, nil, "format-with-primitives"}, {`format('Hello "{0}" {1} {2} {3} {4}', null, true, -3.14, NaN, Infinity)`, `Hello "" true -3.14 NaN Infinity`, nil, "format-with-primitives"},
{`format('Hello "{0}" {1} {2}', fromJSON('[0, true, "abc"]'), fromJSON('[{"a":1}]'), fromJSON('{"a":{"b":1}}'))`, `Hello "Array" Array Object`, nil, "format-with-complex-types"}, {`format('Hello "{0}" {1} {2}', fromJSON('[0, true, "abc"]'), fromJSON('[{"a":1}]'), fromJSON('{"a":{"b":1}}'))`, `Hello "Array" Array Object`, nil, "format-with-complex-types"},
{"format(true)", "true", nil, "format-with-primitive-args"}, {"format(true)", "true", nil, "format-with-primitive-args"},
{"format('{0}', github)", "Object", nil, "format-with-context"},
{"format('echo Hello {0} ${{Test}}', github.undefined_property)", "echo Hello ${Test}", nil, "format-with-undefined-value"}, {"format('echo Hello {0} ${{Test}}', github.undefined_property)", "echo Hello ${Test}", nil, "format-with-undefined-value"},
{"format('{0}}', '{1}', 'World')", nil, "Closing bracket without opening one. The following format string is invalid: '{0}}'", "format-invalid-format-string"}, {"format('{0}}', '{1}', 'World')", nil, "Closing bracket without opening one. The following format string is invalid: '{0}}'", "format-invalid-format-string"},
{"format('a}b')", nil, "Closing bracket without opening one. The following format string is invalid: 'a}b'", "format-unmatched-closing-brace"},
{"format('{0', '{1}', 'World')", nil, "Unclosed brackets. The following format string is invalid: '{0'", "format-invalid-format-string"}, {"format('{0', '{1}', 'World')", nil, "Unclosed brackets. The following format string is invalid: '{0'", "format-invalid-format-string"},
{"format('{2}', '{1}', 'World')", "", "The following format string references more arguments than were supplied: '{2}'", "format-invalid-replacement-reference"}, {"format('{2}', '{1}', 'World')", "", "The following format string references more arguments than were supplied: '{2}'", "format-invalid-replacement-reference"},
{"format('{2147483648}')", "", "The following format string is invalid: '{2147483648}'", "format-invalid-replacement-reference"}, {"format('{2147483648}')", "", "The following format string is invalid: '{2147483648}'", "format-invalid-replacement-reference"},

View File

@@ -10,6 +10,7 @@ import (
"fmt" "fmt"
"math" "math"
"reflect" "reflect"
"strconv"
"strings" "strings"
"gitea.com/gitea/runner/act/model" "gitea.com/gitea/runner/act/model"
@@ -429,41 +430,54 @@ func (impl *interperterImpl) coerceToNumber(value reflect.Value) reflect.Value {
return reflect.ValueOf(math.NaN()) return reflect.ValueOf(math.NaN())
} }
func (impl *interperterImpl) coerceToString(value reflect.Value) reflect.Value { // CoerceToString converts an evaluated expression value to a string the way GitHub does,
// see https://docs.github.com/en/actions/reference/workflows-and-actions/expressions#operators
// An already reflected value is accepted as-is, since Interface() would panic on an invalid one.
func CoerceToString(v any) string {
value, ok := v.(reflect.Value)
if !ok {
value = reflect.ValueOf(v)
}
switch value.Kind() { switch value.Kind() {
case reflect.Invalid: case reflect.Invalid:
return reflect.ValueOf("") return ""
case reflect.Bool: case reflect.Bool:
switch value.Bool() { return strconv.FormatBool(value.Bool())
case true:
return reflect.ValueOf("true")
case false:
return reflect.ValueOf("false")
}
case reflect.String: case reflect.String:
return value return value.String()
case reflect.Int: case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return reflect.ValueOf(fmt.Sprint(value)) return strconv.FormatInt(value.Int(), 10)
case reflect.Float64: case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
return strconv.FormatUint(value.Uint(), 10)
case reflect.Float32, reflect.Float64:
if math.IsInf(value.Float(), 1) { if math.IsInf(value.Float(), 1) {
return reflect.ValueOf("Infinity") return "Infinity"
} else if math.IsInf(value.Float(), -1) { } else if math.IsInf(value.Float(), -1) {
return reflect.ValueOf("-Infinity") return "-Infinity"
} }
return reflect.ValueOf(fmt.Sprintf("%.15G", value.Float())) return fmt.Sprintf("%.15G", value.Float())
case reflect.Slice: case reflect.Slice, reflect.Array:
return reflect.ValueOf("Array") return "Array"
case reflect.Map: // contexts such as `github` are pointers to structs, so they stringify as objects too
return reflect.ValueOf("Object") case reflect.Map, reflect.Struct:
return "Object"
case reflect.Interface, reflect.Pointer:
if value.IsNil() {
return ""
}
return CoerceToString(value.Elem())
} }
return value return fmt.Sprintf("%v", value)
} }
func (impl *interperterImpl) compareString(left, right string, kind actionlint.CompareOpNodeKind) (bool, error) { func (impl *interperterImpl) compareString(left, right string, kind actionlint.CompareOpNodeKind) (bool, error) {

View File

@@ -6,6 +6,7 @@ package exprparser
import ( import (
"math" "math"
"reflect"
"testing" "testing"
"gitea.com/gitea/runner/act/model" "gitea.com/gitea/runner/act/model"
@@ -633,3 +634,51 @@ func TestContexts(t *testing.T) {
}) })
} }
} }
func TestCoerceToString(t *testing.T) {
type object struct{ Name string }
obj := object{Name: "x"}
var nilPointer *object
var nilMap map[string]any
var nilSlice []any
table := []struct {
input any
expected string
name string
}{
{nil, "", "null"},
{true, "true", "true"},
{false, "false", "false"},
{"foo", "foo", "string"},
{"", "", "empty-string"},
{123, "123", "int"},
{int64(-9), "-9", "int64"},
{uint8(7), "7", "uint8"},
{1.0, "1", "float-integral"},
{-9.7, "-9.7", "float"},
{2.99e-2, "0.0299", "float-exponential"},
{1e21, "1E+21", "float-large"},
{float32(1.5), "1.5", "float32"},
{math.NaN(), "NaN", "nan"},
{math.Inf(1), "Infinity", "positive-infinity"},
{math.Inf(-1), "-Infinity", "negative-infinity"},
{[]any{1, 2}, "Array", "slice"},
{nilSlice, "Array", "nil-slice"},
{[2]int{1, 2}, "Array", "fixed-size-array"},
{map[string]any{"a": 1}, "Object", "map"},
{nilMap, "Object", "nil-map"},
{obj, "Object", "struct"},
{&obj, "Object", "pointer-to-struct"},
{nilPointer, "", "nil-pointer"},
{&model.GithubContext{Action: "push"}, "Object", "github-context"},
{reflect.ValueOf(42), "42", "reflected-value"},
{reflect.Value{}, "", "invalid-reflected-value"},
}
for _, tt := range table {
t.Run(tt.name, func(t *testing.T) {
assert.Equal(t, tt.expected, CoerceToString(tt.input))
})
}
}

View File

@@ -6,11 +6,17 @@ package model
type JobContext struct { type JobContext struct {
Status string `json:"status"` Status string `json:"status"`
Container struct { Container JobContainerContext `json:"container"`
Services map[string]JobService `json:"services"`
}
type JobContainerContext struct {
ID string `json:"id"` ID string `json:"id"`
Network string `json:"network"` Network string `json:"network"`
} `json:"container"` }
Services map[string]struct {
ID string `json:"id"` type JobService struct {
} `json:"services"` ID string `json:"id"`
Network string `json:"network"`
Ports map[string]string `json:"ports"` // container port to the published host port
} }

View File

@@ -595,8 +595,8 @@ func actionStagePaths(step actionStep) (actionDir, actionPath, actionName, conta
rc := step.getRunContext() rc := step.getRunContext()
stepModel := step.getStepModel() stepModel := step.getStepModel()
if _, ok := step.(*stepActionRemote); ok { if sar, ok := step.(*stepActionRemote); ok {
actionDir = fmt.Sprintf("%s/%s", rc.ActionCacheDir(), stepModel.UsesHash()) actionDir = sar.actionDir()
actionPath = newRemoteAction(stepModel.Uses).Path actionPath = newRemoteAction(stepModel.Uses).Path
} else { } else {
actionDir = filepath.Join(rc.Config.Workdir, stepModel.Uses) actionDir = filepath.Join(rc.Config.Workdir, stepModel.Uses)

View File

@@ -186,10 +186,10 @@ func (rc *RunContext) compositeExecutor(action *model.Action) *compositeSteps {
err := rc.newCompositeCommandExecutor(step.main())(ctx) err := rc.newCompositeCommandExecutor(step.main())(ctx)
if err != nil { if err != nil {
logger.Errorf("%v", err) logger.Errorf("##[error]%s", EscapeCommandData(err.Error()))
common.SetJobError(ctx, err) common.SetJobError(ctx, err)
} else if ctx.Err() != nil { } else if ctx.Err() != nil {
logger.Errorf("%v", ctx.Err()) logger.Errorf("##[error]%s", EscapeCommandData(ctx.Err().Error()))
common.SetJobError(ctx, ctx.Err()) common.SetJobError(ctx, ctx.Err())
} }
return nil return nil
@@ -248,10 +248,10 @@ func newCompositeStepLogExecutor(runStep common.Executor, stepID string) common.
logger := common.Logger(ctx) logger := common.Logger(ctx)
err := runStep(ctx) err := runStep(ctx)
if err != nil { if err != nil {
logger.Errorf("%v", err) logger.Errorf("##[error]%s", EscapeCommandData(err.Error()))
common.SetJobError(ctx, err) common.SetJobError(ctx, err)
} else if ctx.Err() != nil { } else if ctx.Err() != nil {
logger.Errorf("%v", ctx.Err()) logger.Errorf("##[error]%s", EscapeCommandData(ctx.Err().Error()))
common.SetJobError(ctx, ctx.Err()) common.SetJobError(ctx, ctx.Err())
} }
return nil return nil

View File

@@ -6,6 +6,7 @@ package runner
import ( import (
"context" "context"
"fmt"
"regexp" "regexp"
"strings" "strings"
@@ -45,17 +46,24 @@ func (rc *RunContext) commandHandler(ctx context.Context) common.LineHandler {
return true return true
} }
if resumeCommand != "" && command != resumeCommand { if resumeCommand != "" {
// There should not be any emojis in the log output for Gitea. // There should not be any emojis in the log output for Gitea.
// The code in the switch statement is the same.
// Return true (not false) so the line still reaches the raw_output // Return true (not false) so the line still reaches the raw_output
// log handler; otherwise everything between ::stop-commands:: and // log handler; otherwise everything between ::stop-commands:: and
// its end token is silently dropped from the step log. // its end token is silently dropped from the step log.
logger.Infof("%s", line) logger.Infof("%s", line)
// Resumed here rather than from the switch, because the end token is arbitrary
// and a token naming a real command would otherwise never resume.
if command == resumeCommand {
resumeCommand = ""
}
return true return true
} }
arg = UnescapeCommandData(arg) arg = UnescapeCommandData(arg)
kvPairs = unescapeKvPairs(kvPairs) kvPairs = unescapeKvPairs(kvPairs)
if (command == "set-env" || command == "add-path") && rc.refuseUnsecureCommand(ctx, command) {
return true
}
switch command { switch command {
case "set-env": case "set-env":
rc.setEnv(ctx, kvPairs, arg) rc.setEnv(ctx, kvPairs, arg)
@@ -63,27 +71,20 @@ func (rc *RunContext) commandHandler(ctx context.Context) common.LineHandler {
rc.setOutput(ctx, kvPairs, arg) rc.setOutput(ctx, kvPairs, arg)
case "add-path": case "add-path":
rc.addPath(ctx, arg) rc.addPath(ctx, arg)
case "debug":
logger.Infof("%s", line)
case "warning":
logger.Infof("%s", line)
case "error":
logger.Infof("%s", line)
case "add-mask": case "add-mask":
rc.AddMask(arg) rc.AddMask(arg)
logger.Infof("%s", "***") logger.Infof("%s", "***")
// The raw line is still forwarded, carrying the secret: that is how the reporter
// learns the mask, and it drops the row rather than writing it out.
case "stop-commands": case "stop-commands":
resumeCommand = arg resumeCommand = arg
logger.Infof("%s", line) logger.Infof("%s", line)
case resumeCommand:
resumeCommand = ""
logger.Infof("%s", line)
case "save-state": case "save-state":
logger.Infof("%s", line) logger.Infof("%s", line)
rc.saveState(ctx, kvPairs, arg) rc.saveState(ctx, kvPairs, arg)
case "add-matcher":
logger.Infof("%s", line)
default: default:
// ::debug::, ::error::, ::warning::, ::add-matcher:: and anything unrecognised are
// passed through for the reporter and Gitea's web UI to render.
logger.Infof("%s", line) logger.Infof("%s", line)
} }
@@ -92,6 +93,52 @@ func (rc *RunContext) commandHandler(ctx context.Context) common.LineHandler {
} }
} }
const allowUnsecureCommandsVar = "ACTIONS_ALLOW_UNSECURE_COMMANDS"
// refuseUnsecureCommand reports whether a deprecated ::set-env:: or ::add-path:: command must
// not run, recording the error that fails the step. GitHub disabled both because a step that
// echoes untrusted content can use them to set NODE_OPTIONS or PATH for every later step.
func (rc *RunContext) refuseUnsecureCommand(ctx context.Context, command string) bool {
if rc.allowUnsecureCommandsOptIn() {
return false
}
// The step executor logs the failure itself, so keep this line's wording distinct.
common.Logger(ctx).WithField(rawOutputField, true).Errorf("##[error]%s", EscapeCommandData(fmt.Sprintf(
"The `%s` command is disabled: it can set the environment of every later step from untrusted output. "+
"Write to $GITHUB_ENV or $GITHUB_PATH instead, or set ACTIONS_ALLOW_UNSECURE_COMMANDS to allow it",
command)))
rc.unsecureCommandMu.Lock()
defer rc.unsecureCommandMu.Unlock()
if rc.unsecureCommandErr == nil {
rc.unsecureCommandErr = fmt.Errorf("the `%s` workflow command is disabled", command)
}
return true
}
// allowUnsecureCommandsOptIn reports whether the workflow itself asked for the deprecated
// commands, from any env scope, as it can on GitHub.
func (rc *RunContext) allowUnsecureCommandsOptIn() bool {
return isTruthyEnv(rc.currentStepEnv()[allowUnsecureCommandsVar]) ||
isTruthyEnv(rc.Env[allowUnsecureCommandsVar]) ||
isTruthyEnv(rc.GlobalEnv[allowUnsecureCommandsVar])
}
// isTruthyEnv mirrors GitHub's bool.TryParse: only "true", in any casing.
func isTruthyEnv(v string) bool {
return strings.EqualFold(strings.TrimSpace(v), "true")
}
// takeUnsecureCommandError returns and clears the error left by a refused command.
func (rc *RunContext) takeUnsecureCommandError() error {
rc.unsecureCommandMu.Lock()
defer rc.unsecureCommandMu.Unlock()
err := rc.unsecureCommandErr
rc.unsecureCommandErr = nil
return err
}
func (rc *RunContext) setEnv(ctx context.Context, kvPairs map[string]string, arg string) { func (rc *RunContext) setEnv(ctx context.Context, kvPairs map[string]string, arg string) {
name := kvPairs["name"] name := kvPairs["name"]
common.Logger(ctx).Infof("::set-env:: %s=%s", name, arg) common.Logger(ctx).Infof("::set-env:: %s=%s", name, arg)
@@ -161,9 +208,9 @@ var (
commandPropertyUnescaper = strings.NewReplacer("%25", "%", "%0D", "\r", "%0A", "\n", "%3A", ":", "%2C", ",") commandPropertyUnescaper = strings.NewReplacer("%25", "%", "%0D", "\r", "%0A", "\n", "%3A", ":", "%2C", ",")
) )
// escapeCommandData encodes the data part of a "::cmd::" or "##[cmd]" line the runner writes itself, // EscapeCommandData encodes the data part of a "::cmd::" or "##[cmd]" line the runner writes itself,
// so the log renderer decodes it back. Lines forwarded from step output are already escaped. // so the log renderer decodes it back. Lines forwarded from step output are already escaped.
func escapeCommandData(arg string) string { func EscapeCommandData(arg string) string {
return commandDataEscaper.Replace(arg) return commandDataEscaper.Replace(arg)
} }

View File

@@ -16,12 +16,18 @@ import (
"github.com/sirupsen/logrus/hooks/test" "github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
) )
// unsecureRC opts into ::set-env:: and ::add-path::, which are refused without it.
func unsecureRC() *RunContext {
return &RunContext{Env: map[string]string{allowUnsecureCommandsVar: "true"}}
}
func TestSetEnv(t *testing.T) { func TestSetEnv(t *testing.T) {
a := assert.New(t) a := assert.New(t)
ctx := context.Background() ctx := context.Background()
rc := new(RunContext) rc := unsecureRC()
handler := rc.commandHandler(ctx) handler := rc.commandHandler(ctx)
handler("::set-env name=x::valz\n") handler("::set-env name=x::valz\n")
@@ -31,7 +37,7 @@ func TestSetEnv(t *testing.T) {
func TestStopCommandsKeepsSuppressedLinesInLog(t *testing.T) { func TestStopCommandsKeepsSuppressedLinesInLog(t *testing.T) {
a := assert.New(t) a := assert.New(t)
ctx := context.Background() ctx := context.Background()
rc := new(RunContext) rc := unsecureRC()
handler := rc.commandHandler(ctx) handler := rc.commandHandler(ctx)
// Stop command processing until the matching end token is seen. // Stop command processing until the matching end token is seen.
@@ -84,7 +90,7 @@ func TestSetOutput(t *testing.T) {
func TestAddpath(t *testing.T) { func TestAddpath(t *testing.T) {
a := assert.New(t) a := assert.New(t)
ctx := context.Background() ctx := context.Background()
rc := new(RunContext) rc := unsecureRC()
handler := rc.commandHandler(ctx) handler := rc.commandHandler(ctx)
handler("::add-path::/zoo\n") handler("::add-path::/zoo\n")
@@ -99,7 +105,7 @@ func TestStopCommands(t *testing.T) {
a := assert.New(t) a := assert.New(t)
ctx := common.WithLogger(context.Background(), logger) ctx := common.WithLogger(context.Background(), logger)
rc := new(RunContext) rc := unsecureRC()
handler := rc.commandHandler(ctx) handler := rc.commandHandler(ctx)
handler("::set-env name=x::valz\n") handler("::set-env name=x::valz\n")
@@ -119,10 +125,26 @@ func TestStopCommands(t *testing.T) {
a.Contains(messages, "::set-env name=x::abcd\n") a.Contains(messages, "::set-env name=x::abcd\n")
} }
// The end token is arbitrary, so one that happens to name a real command must still resume
// rather than being swallowed by that command's case.
func TestStopCommandsResumesOnCommandNamedToken(t *testing.T) {
a := assert.New(t)
rc := unsecureRC()
handler := rc.commandHandler(context.Background())
handler("::stop-commands::add-mask\n")
handler("::set-env name=x::suppressed\n")
a.NotContains(rc.Env, "x")
handler("::add-mask::\n")
handler("::set-env name=x::resumed\n")
a.Equal("resumed", rc.Env["x"])
}
func TestAddpathADO(t *testing.T) { func TestAddpathADO(t *testing.T) {
a := assert.New(t) a := assert.New(t)
ctx := context.Background() ctx := context.Background()
rc := new(RunContext) rc := unsecureRC()
handler := rc.commandHandler(ctx) handler := rc.commandHandler(ctx)
handler("##[add-path]/zoo\n") handler("##[add-path]/zoo\n")
@@ -218,6 +240,44 @@ func TestSaveState(t *testing.T) {
func TestEscapeCommandData(t *testing.T) { func TestEscapeCommandData(t *testing.T) {
a := assert.New(t) a := assert.New(t)
a.Equal("a%25b%0Dc%0Ad%250A", escapeCommandData("a%b\rc\nd%0A")) a.Equal("a%25b%0Dc%0Ad%250A", EscapeCommandData("a%b\rc\nd%0A"))
a.Equal("a%b\rc\nd%0A", UnescapeCommandData("a%25b%0Dc%0Ad%250A")) a.Equal("a%b\rc\nd%0A", UnescapeCommandData("a%25b%0Dc%0Ad%250A"))
} }
func TestUnsecureCommands(t *testing.T) {
tests := []struct {
name string
jobEnv map[string]string
stepEnv map[string]string
optedIn bool
}{
{name: "refused with no opt-in"},
// GitHub reads the opt-in with bool.TryParse, so "1" is not one.
{name: "refused for a value bool.TryParse rejects", jobEnv: map[string]string{allowUnsecureCommandsVar: "1"}},
{name: "opted in through the step environment", stepEnv: map[string]string{allowUnsecureCommandsVar: "true"}, optedIn: true},
{name: "opted in through the job environment", jobEnv: map[string]string{allowUnsecureCommandsVar: "TRUE"}, optedIn: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
a := assert.New(t)
rc := &RunContext{Env: tt.jobEnv}
rc.setCurrentStepEnv(tt.stepEnv)
handler := rc.commandHandler(context.Background())
handler("::set-env name=x::valz\n")
handler("::add-path::/opt/bin\n")
if !tt.optedIn {
a.Empty(rc.Env["x"])
a.Empty(rc.ExtraPath)
// The refusal fails the step that produced it, once.
require.ErrorContains(t, rc.takeUnsecureCommandError(), "set-env")
a.NoError(rc.takeUnsecureCommandError())
return
}
a.Equal("valz", rc.Env["x"])
a.Equal([]string{"/opt/bin"}, rc.ExtraPath)
a.NoError(rc.takeUnsecureCommandError())
})
}
}

View File

@@ -78,3 +78,14 @@ func (cm *containerMock) GetContainerArchive(ctx context.Context, srcPath string
} }
return args.Get(0).(io.ReadCloser), err return args.Get(0).(io.ReadCloser), err
} }
func (cm *containerMock) DumpLogs(ctx context.Context) error {
return cm.Called(ctx).Error(0)
}
func (cm *containerMock) Inspect(ctx context.Context) (*container.Info, error) {
args := cm.Called(ctx)
info, _ := args.Get(0).(*container.Info)
err, _ := args.Get(1).(error)
return info, err
}

View File

@@ -66,7 +66,7 @@ func reportStepError(ctx context.Context, rc *RunContext, err error) {
rc.markInterrupted(ctx.Err()) rc.markInterrupted(ctx.Err())
return return
} }
common.Logger(ctx).Errorf("##[error]%s", escapeCommandData(err.Error())) common.Logger(ctx).Errorf("##[error]%s", EscapeCommandData(err.Error()))
common.SetJobError(ctx, err) common.SetJobError(ctx, err)
rc.markFailed() rc.markFailed()
} }
@@ -260,7 +260,7 @@ func newJobExecutor(info jobInfo, sf stepFactory, rc *RunContext) common.Executo
logger.Infof("Cleaning up container for job %s", rc.JobName) logger.Infof("Cleaning up container for job %s", rc.JobName)
if err = info.stopContainer()(ctx); err != nil { if err = info.stopContainer()(ctx); err != nil {
logger.Errorf("Error while stop job container: %v", err) logger.Errorf("##[error]%s", EscapeCommandData("Error while stop job container: "+err.Error()))
} }
// For Gitea // For Gitea

View File

@@ -45,7 +45,7 @@ func (rc *RunContext) runJobHook(ctx context.Context, hookPath, name string) err
cmd, shell := hookCommand(hookPath) cmd, shell := hookCommand(hookPath)
rawLogger := common.Logger(ctx).WithField(rawOutputField, true) rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
defer rawLogger.Infof("::endgroup::") defer rawLogger.Infof("::endgroup::")
rawLogger.Infof("::group::Run '%s'", escapeCommandData(hookPath)) rawLogger.Infof("::group::Run '%s'", EscapeCommandData(hookPath))
rawLogger.Infof("A %s hook has been configured by the runner administrator", name) rawLogger.Infof("A %s hook has been configured by the runner administrator", name)
if shell != "" { if shell != "" {
rawLogger.Infof("shell: %s", shell) rawLogger.Infof("shell: %s", shell)

View File

@@ -7,8 +7,11 @@ package runner
import ( import (
"bytes" "bytes"
"context" "context"
"encoding/base64"
"encoding/json"
"fmt" "fmt"
"io" "io"
"net/url"
"os" "os"
"slices" "slices"
"strings" "strings"
@@ -167,6 +170,76 @@ func withStepLogger(ctx context.Context, stepNumber int, stepID, stepName, stage
type entryProcessor func(entry *logrus.Entry) *logrus.Entry type entryProcessor func(entry *logrus.Entry) *logrus.Entry
// secretValueEncoders are the shapes a secret takes on its way into a log: a base64
// payload, a JSON string, or a URL component. An action that serializes a secret leaks
// it in one of these forms, which a mask of the verbatim value alone does not catch, so
// every form is masked as well. This mirrors the value encoders of GitHub's runner.
var secretValueEncoders = []func(string) string{
func(v string) string { return base64.StdEncoding.EncodeToString([]byte(v)) },
base64ShiftEncoder(1),
base64ShiftEncoder(2),
jsonStringEscape,
jsonStringEscapeNoHTML,
url.QueryEscape,
url.PathEscape,
}
// minShiftedBase64Len is the shortest shifted base64 fragment worth masking. A shorter
// one carries too few bytes of the secret to identify it and would mask unrelated output.
const minShiftedBase64Len = 8
// base64ShiftEncoder returns the part of a secret's base64 form that survives when the
// secret does not start on a 3-byte boundary of the payload it is embedded in. base64
// encodes three bytes at a time, so `Authorization: Basic base64("user:token")` contains
// the base64 of the token alone only when the prefix length happens to be a multiple of
// three; at the other two alignments the encoding of the whole value differs. Encoding
// the secret behind shift filler bytes reproduces those alignments, which is what the
// Base64StringEscapeShift1/2 encoders of GitHub's runner do.
//
// The leading group (filler mixed with the secret's first bytes) and the trailing group
// (padded here, but continuing into whatever follows the secret) are dropped, leaving the
// group-aligned middle that does appear verbatim in the log.
func base64ShiftEncoder(shift int) func(string) string {
return func(v string) string {
buf := make([]byte, shift+len(v))
copy(buf[shift:], v)
encoded := base64.StdEncoding.EncodeToString(buf)
// Keep only the aligned middle, and only when enough of it is left to be a
// distinctive pattern rather than a fragment that matches unrelated output.
if len(encoded) < 8+minShiftedBase64Len {
return ""
}
return encoded[4 : len(encoded)-4]
}
}
// jsonStringEscape returns v as it appears inside a JSON string, without the quotes,
// which is what `toJSON(secrets)` or any action logging a JSON body produces. Go's encoder
// escapes <, >, & (as act's own toJSON does); the non-HTML variant below covers the runtimes
// that do not. When v has none of those characters both forms are equal and deduplicated.
func jsonStringEscape(v string) string {
encoded, err := json.Marshal(v)
if err != nil {
return v
}
return string(encoded[1 : len(encoded)-1])
}
// jsonStringEscapeNoHTML is jsonStringEscape without HTML escaping, matching the JSON a
// JavaScript (JSON.stringify) or .NET action emits, so a secret containing < > or & is
// masked in that form too.
func jsonStringEscapeNoHTML(v string) string {
var buf bytes.Buffer
enc := json.NewEncoder(&buf)
enc.SetEscapeHTML(false)
if err := enc.Encode(v); err != nil {
return v
}
// Encode appends a newline; drop it along with the surrounding quotes.
encoded := strings.TrimRight(buf.String(), "\n")
return encoded[1 : len(encoded)-1]
}
func AppendSecretMasker(oldnew []string, v string) []string { func AppendSecretMasker(oldnew []string, v string) []string {
ret := oldnew ret := oldnew
@@ -177,11 +250,26 @@ func AppendSecretMasker(oldnew []string, v string) []string {
ret = append(ret, tm, "***") ret = append(ret, tm, "***")
// command data reaches the log escaped, so "pass%word" also arrives as "pass%25word" // command data reaches the log escaped, so "pass%word" also arrives as "pass%25word"
if strings.ContainsAny(tm, "%\r\n") { if strings.ContainsAny(tm, "%\r\n") {
ret = append(ret, escapeCommandData(tm), "***") ret = append(ret, EscapeCommandData(tm), "***")
} }
} }
} }
// The encoded forms are derived from the whole value: a multi-line secret is
// encoded as one string, not line by line.
trimmed := strings.TrimSpace(v)
if len(trimmed) <= 1 {
return ret
}
for _, encode := range secretValueEncoders {
encoded := encode(trimmed)
// An encoding that leaves the value unchanged is already masked above.
if encoded == trimmed || len(encoded) <= 1 || slices.Contains(ret, encoded) {
continue
}
ret = append(ret, encoded, "***")
}
return ret return ret
} }
@@ -194,6 +282,18 @@ func valueMasker(insecureSecrets bool, secrets map[string]string) entryProcessor
} }
oldnew = slices.Clip(oldnew) oldnew = slices.Clip(oldnew)
defReplacer := strings.NewReplacer(oldnew...) defReplacer := strings.NewReplacer(oldnew...)
// A ::add-mask:: only ever appends to the job's mask slice, so the replacer built for
// it stays valid until the slice grows. Cache it, keyed by the slice itself and its
// length, instead of encoding every secret and mask again for each log line.
var (
mu sync.Mutex
masksRef *[]string
pairs []string
masked int
replacer *strings.Replacer
)
return func(entry *logrus.Entry) *logrus.Entry { return func(entry *logrus.Entry) *logrus.Entry {
if insecureSecrets { if insecureSecrets {
return entry return entry
@@ -203,15 +303,26 @@ func valueMasker(insecureSecrets bool, secrets map[string]string) entryProcessor
if len(*masks) == 0 { if len(*masks) == 0 {
entry.Message = defReplacer.Replace(entry.Message) entry.Message = defReplacer.Replace(entry.Message)
} else { return entry
cmasker := oldnew
for _, v := range *masks {
cmasker = AppendSecretMasker(cmasker, v)
} }
entry.Message = strings.NewReplacer(cmasker...).Replace(entry.Message) mu.Lock()
// A composite action logs through the same masker with its own mask slice, so a
// different slice starts the cache over.
if masksRef != masks {
masksRef, pairs, masked, replacer = masks, oldnew, 0, nil
} }
if replacer == nil || masked != len(*masks) {
for _, v := range (*masks)[masked:] {
pairs = AppendSecretMasker(pairs, v)
}
masked = len(*masks)
replacer = strings.NewReplacer(pairs...)
}
cmasker := replacer
mu.Unlock()
entry.Message = cmasker.Replace(entry.Message)
return entry return entry
} }

View File

@@ -4,7 +4,9 @@
package runner package runner
import ( import (
"encoding/base64"
"io" "io"
"net/url"
"strings" "strings"
"testing" "testing"
@@ -59,6 +61,136 @@ func TestValueMasker(t *testing.T) {
} }
} }
// A secret that reaches the log through an encoding — a base64 payload, a JSON body, a
// URL — must be masked as well: masking only the verbatim value leaks it.
func TestValueMaskerEncodedSecrets(t *testing.T) {
secret := `p@ss w"rd/1`
masker := valueMasker(false, map[string]string{"TOKEN": secret})
for _, tc := range []struct {
name string
line string
}{
{"verbatim", "the token is " + secret},
{"base64", "Authorization: Basic " + base64.StdEncoding.EncodeToString([]byte(secret))},
{"json", `{"token":"` + jsonStringEscape(secret) + `"}`},
{"query escaped", "https://example.com/?token=" + url.QueryEscape(secret)},
{"path escaped", "https://example.com/" + url.PathEscape(secret) + "/x"},
} {
t.Run(tc.name, func(t *testing.T) {
entry := masker(&logrus.Entry{Context: t.Context(), Message: tc.line})
assert.Contains(t, entry.Message, "***")
assert.NotContains(t, entry.Message, secret)
assert.NotContains(t, entry.Message, base64.StdEncoding.EncodeToString([]byte(secret)))
assert.NotContains(t, entry.Message, url.QueryEscape(secret))
})
}
}
// A secret containing " together with <, > or & serializes to JSON differently depending
// on the runtime: act's own toJSON (and Go) HTML-escape <>&, while a JavaScript
// (JSON.stringify) or .NET action leaves them literal. The secret must be masked in either
// form, so a JS-serialized JSON body does not leak it.
func TestValueMaskerJSONEscapesBothWays(t *testing.T) {
secret := `a"<b>&c`
masker := valueMasker(false, map[string]string{"TOKEN": secret})
for _, tc := range []struct {
name string
form string
}{
{"html escaped (act toJSON / Go)", jsonStringEscape(secret)},
{"literal (JS JSON.stringify / .NET)", jsonStringEscapeNoHTML(secret)},
} {
t.Run(tc.name, func(t *testing.T) {
entry := masker(&logrus.Entry{Context: t.Context(), Message: `{"t":"` + tc.form + `"}`})
assert.Contains(t, entry.Message, "***")
assert.NotContains(t, entry.Message, tc.form)
})
}
}
// ::add-mask:: values go through the same masker, so they get the same treatment.
func TestValueMaskerEncodedMasks(t *testing.T) {
masks := []string{"s3cr3t value"}
masker := valueMasker(false, nil)
entry := masker(&logrus.Entry{
Context: WithMasks(t.Context(), &masks),
Message: "encoded: " + base64.StdEncoding.EncodeToString([]byte("s3cr3t value")),
})
assert.Equal(t, "encoded: ***", entry.Message)
}
// A token in a Basic auth header is base64'd together with the user name, so the token's
// own base64 only appears when the prefix length is a multiple of three. The other two
// alignments must be masked as well, or `Authorization: Basic base64("user:token")` leaks
// the token to anyone who can decode the log.
func TestValueMaskerBase64Alignments(t *testing.T) {
secret := "s3cr3t-token-value"
masker := valueMasker(false, map[string]string{"TOKEN": secret})
// One prefix per alignment: len%3 of 0, 1 and 2.
for _, prefix := range []string{"x-access-token:", "user:", "ab:"} {
t.Run(prefix, func(t *testing.T) {
encoded := base64.StdEncoding.EncodeToString([]byte(prefix + secret))
entry := masker(&logrus.Entry{Context: t.Context(), Message: "Authorization: Basic " + encoded})
assert.Contains(t, entry.Message, "***")
// The aligned middle of the secret must be gone, so the payload can no longer be
// decoded back into the token.
assert.NotEqual(t, "Authorization: Basic "+encoded, entry.Message)
decodable := strings.TrimPrefix(entry.Message, "Authorization: Basic ")
decoded, err := base64.StdEncoding.DecodeString(decodable)
if err == nil {
assert.NotContains(t, string(decoded), secret)
}
})
}
}
// The masker caches its replacer, so it has to notice both a mask appended to the same
// slice and a composite action logging with a slice of its own.
func TestValueMaskerCachedReplacerSeesNewMasks(t *testing.T) {
masker := valueMasker(false, map[string]string{"TOKEN": "secret-token"})
mask := func(masks *[]string, message string) string {
return masker(&logrus.Entry{Context: WithMasks(t.Context(), masks), Message: message}).Message
}
job := []string{"first mask"}
assert.Equal(t, "a *** and ***", mask(&job, "a first mask and secret-token"))
// ::add-mask:: appends to the same slice
job = append(job, "second mask")
assert.Equal(t, "*** and ***", mask(&job, "first mask and second mask"))
// a composite action brings its own slice
composite := []string{"composite mask"}
assert.Equal(t, "*** but first mask", mask(&composite, "composite mask but first mask"))
// and the job's masks still apply once it is back
assert.Equal(t, "*** and *** but composite mask", mask(&job, "first mask and second mask but composite mask"))
}
func TestAppendSecretMaskerSkipsUselessEncodings(t *testing.T) {
// A token with no character an escape would touch only gains its base64 forms:
// JSON, query and path escaping all leave it unchanged.
pairs := AppendSecretMasker(nil, "plaintoken")
assert.Equal(t, []string{
"plaintoken", "***",
base64.StdEncoding.EncodeToString([]byte("plaintoken")), "***",
// The two shifted alignments, each without its leading and trailing group.
"YWludG9r", "***",
"bGFpbnRv", "***",
}, pairs)
// Too short to mask.
assert.Empty(t, AppendSecretMasker(nil, "x"))
}
func TestJobLogFormatterDecodesCommandData(t *testing.T) { func TestJobLogFormatterDecodesCommandData(t *testing.T) {
logger := logrus.New() logger := logrus.New()
logger.Out = io.Discard logger.Out = io.Discard

View File

@@ -22,6 +22,7 @@ import (
"runtime" "runtime"
"slices" "slices"
"strings" "strings"
"sync"
"time" "time"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
@@ -34,6 +35,7 @@ import (
"github.com/docker/go-connections/nat" "github.com/docker/go-connections/nat"
"github.com/moby/moby/api/types/mount" "github.com/moby/moby/api/types/mount"
"github.com/opencontainers/selinux/go-selinux" "github.com/opencontainers/selinux/go-selinux"
"golang.org/x/sync/errgroup"
) )
// RunContext contains info about current job // RunContext contains info about current job
@@ -55,7 +57,7 @@ type RunContext struct {
IntraActionState map[string]map[string]string IntraActionState map[string]map[string]string
ExprEval ExpressionEvaluator ExprEval ExpressionEvaluator
JobContainer container.ExecutionsEnvironment JobContainer container.ExecutionsEnvironment
ServiceContainers []container.ExecutionsEnvironment serviceContainers []*serviceContainer
OutputMappings map[MappableOutput]MappableOutput OutputMappings map[MappableOutput]MappableOutput
JobName string JobName string
ActionPath string ActionPath string
@@ -83,6 +85,37 @@ type RunContext struct {
// failures. Those failures must still make success() false and failure() true for later // failures. Those failures must still make success() false and failure() true for later
// main-step if evaluation. // main-step if evaluation.
jobFailed bool jobFailed bool
// empty for a host-mode job, which starts no container
jobContainerID string
jobNetworkName string
// stepEnv is a copy of the running step's environment, so that workflow commands parsed out
// of the container's output can be judged against it. Written by runStepExecutor and read on
// the log-writer goroutine, hence unsecureCommandMu, which also guards unsecureCommandErr.
stepEnv map[string]string
unsecureCommandErr error // refused ::set-env::/::add-path::, turned into a step failure
unsecureCommandMu sync.Mutex
}
// serviceContainer pairs a service container with the workflow id that keys job.services.
type serviceContainer struct {
name string
image string
container container.ExecutionsEnvironment
logsDumped bool
info *container.Info // last poll, the source of the `job.services` entry
}
// setCurrentStepEnv records the environment of the step about to run.
func (rc *RunContext) setCurrentStepEnv(env map[string]string) {
rc.unsecureCommandMu.Lock()
defer rc.unsecureCommandMu.Unlock()
rc.stepEnv = env
}
func (rc *RunContext) currentStepEnv() map[string]string {
rc.unsecureCommandMu.Lock()
defer rc.unsecureCommandMu.Unlock()
return rc.stepEnv
} }
// markCancelled flags the job as cancelled so subsequent step `if` evaluations and the // markCancelled flags the job as cancelled so subsequent step `if` evaluations and the
@@ -488,7 +521,7 @@ func (rc *RunContext) startJobContainer() common.Executor {
Privileged: rc.Config.Privileged, Privileged: rc.Config.Privileged,
UsernsMode: rc.Config.UsernsMode, UsernsMode: rc.Config.UsernsMode,
Platform: rc.Config.ContainerArchitecture, Platform: rc.Config.ContainerArchitecture,
AutoRemove: rc.Config.AutoRemove, AutoRemove: false, // so a dead service's log survives, cleanupJobResources removes it
Options: rc.ExprEval.Interpolate(ctx, spec.Options), Options: rc.ExprEval.Interpolate(ctx, spec.Options),
NetworkMode: networkName, NetworkMode: networkName,
NetworkAliases: []string{serviceID}, NetworkAliases: []string{serviceID},
@@ -496,7 +529,7 @@ func (rc *RunContext) startJobContainer() common.Executor {
PortBindings: portBindings, PortBindings: portBindings,
AllocatePTY: rc.Config.AllocatePTY, AllocatePTY: rc.Config.AllocatePTY,
}) })
rc.ServiceContainers = append(rc.ServiceContainers, c) rc.serviceContainers = append(rc.serviceContainers, &serviceContainer{name: serviceID, image: serviceImage, container: c})
} }
rc.cleanUpJobContainer = rc.cleanupJobResources(networkName, createAndDeleteNetwork) rc.cleanUpJobContainer = rc.cleanupJobResources(networkName, createAndDeleteNetwork)
@@ -531,6 +564,8 @@ func (rc *RunContext) startJobContainer() common.Executor {
return errors.New("Failed to create job container") return errors.New("Failed to create job container")
} }
rc.jobNetworkName = networkName
defer printStartJobContainerGroup(ctx, image, name, networkName)() defer printStartJobContainerGroup(ctx, image, name, networkName)()
return common.NewPipelineExecutor( return common.NewPipelineExecutor(
rc.pullServicesImages(rc.Config.ForcePull), rc.pullServicesImages(rc.Config.ForcePull),
@@ -538,9 +573,12 @@ func (rc *RunContext) startJobContainer() common.Executor {
rc.stopJobContainer(), rc.stopJobContainer(),
container.NewDockerNetworkCreateExecutor(networkName, rc.Config.ContainerNetworkCreateOptions). container.NewDockerNetworkCreateExecutor(networkName, rc.Config.ContainerNetworkCreateOptions).
IfBool(createAndDeleteNetwork), IfBool(createAndDeleteNetwork),
rc.startServiceContainers(networkName), rc.startServiceContainers(),
rc.reportUnstartedServices(),
rc.waitForServiceContainers(),
rc.JobContainer.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop), rc.JobContainer.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
rc.JobContainer.Start(false), rc.JobContainer.Start(false),
rc.captureJobContainerInfo(),
rc.JobContainer.Copy(rc.JobContainer.GetActPath()+"/", &container.FileEntry{ rc.JobContainer.Copy(rc.JobContainer.GetActPath()+"/", &container.FileEntry{
Name: "workflow/event.json", Name: "workflow/event.json",
Mode: 0o644, Mode: 0o644,
@@ -565,7 +603,7 @@ func (rc *RunContext) cleanupJobResources(networkName string, createAndDeleteNet
if removeJobContainer { if removeJobContainer {
errs = append(errs, rc.JobContainer.Remove()(ctx)) errs = append(errs, rc.JobContainer.Remove()(ctx))
} }
if len(rc.ServiceContainers) > 0 { if len(rc.serviceContainers) > 0 {
logger.Infof("Cleaning up services for job %s", rc.JobName) logger.Infof("Cleaning up services for job %s", rc.JobName)
if err := rc.stopServiceContainers()(ctx); err != nil { if err := rc.stopServiceContainers()(ctx); err != nil {
logger.Errorf("Error while cleaning services: %v", err) logger.Errorf("Error while cleaning services: %v", err)
@@ -662,21 +700,21 @@ func (rc *RunContext) stopJobContainer() common.Executor {
func (rc *RunContext) pullServicesImages(forcePull bool) common.Executor { func (rc *RunContext) pullServicesImages(forcePull bool) common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
execs := []common.Executor{} execs := []common.Executor{}
for _, c := range rc.ServiceContainers { for _, svc := range rc.serviceContainers {
execs = append(execs, c.Pull(forcePull)) execs = append(execs, svc.container.Pull(forcePull))
} }
return common.NewParallelExecutor(len(execs), execs...)(ctx) return common.NewParallelExecutor(len(execs), execs...)(ctx)
} }
} }
func (rc *RunContext) startServiceContainers(_ string) common.Executor { func (rc *RunContext) startServiceContainers() common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
execs := []common.Executor{} execs := []common.Executor{}
for _, c := range rc.ServiceContainers { for _, svc := range rc.serviceContainers {
execs = append(execs, common.NewPipelineExecutor( execs = append(execs, common.NewPipelineExecutor(
c.Pull(false), svc.container.Pull(false),
c.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop), svc.container.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
c.Start(false), svc.container.Start(false),
)) ))
} }
return common.NewParallelExecutor(len(execs), execs...)(ctx) return common.NewParallelExecutor(len(execs), execs...)(ctx)
@@ -686,13 +724,159 @@ func (rc *RunContext) startServiceContainers(_ string) common.Executor {
func (rc *RunContext) stopServiceContainers() common.Executor { func (rc *RunContext) stopServiceContainers() common.Executor {
return func(ctx context.Context) error { return func(ctx context.Context) error {
execs := []common.Executor{} execs := []common.Executor{}
for _, c := range rc.ServiceContainers { for _, svc := range rc.serviceContainers {
execs = append(execs, c.Remove().Finally(c.Close())) execs = append(execs, svc.container.Remove().Finally(svc.container.Close()))
} }
return common.NewParallelExecutor(len(execs), execs...)(ctx) return common.NewParallelExecutor(len(execs), execs...)(ctx)
} }
} }
const (
defaultServiceReadyTimeout = 5 * time.Minute
serviceReadyPollMax = 32 * time.Second
)
var serviceReadyPollInterval = 2 * time.Second // a variable so tests need not wait
// reportUnstartedServices logs a service that did not start. The steps that need it
// report it better than the runner can, so the job carries on.
func (rc *RunContext) reportUnstartedServices() common.Executor {
return func(ctx context.Context) error {
logger := common.Logger(ctx)
for _, svc := range rc.serviceContainers {
info, err := svc.inspect(ctx)
if err != nil {
logger.Debugf("unable to inspect service '%s': %v", svc.name, err)
continue
}
if info.State == container.StateRunning {
continue
}
svc.dumpLogs(ctx)
logger.Warnf("Docker container %s is not in running state: %s (%d)", info.ID, info.State, info.ExitCode)
}
return nil
}
}
// waitForServiceContainers blocks until every service that declares a healthcheck reports
// healthy, as GitHub does, so a first step cannot connect before the service listens.
func (rc *RunContext) waitForServiceContainers() common.Executor {
return func(ctx context.Context) error {
if len(rc.serviceContainers) == 0 {
return nil
}
timeout := rc.Config.ServiceReadyTimeout
switch {
case timeout < 0:
// disabled, but still describe the containers for `job.services`
for _, svc := range rc.serviceContainers {
if _, err := svc.inspect(ctx); err != nil && !errors.Is(err, container.ErrContainerNotFound) {
return err
}
}
return nil
case timeout == 0:
timeout = defaultServiceReadyTimeout
}
// the first error cancels the rest, so a failure does not wait out a sibling's timeout
group, groupCtx := errgroup.WithContext(ctx)
for _, svc := range rc.serviceContainers {
group.Go(func() error {
return svc.waitUntilHealthy(groupCtx, timeout)
})
}
return group.Wait()
}
}
// waitUntilHealthy waits on the healthcheck alone, so a container that declares none is
// ready at once and one that exited is left to the steps that need it.
func (svc *serviceContainer) waitUntilHealthy(ctx context.Context, timeout time.Duration) error {
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
interval := serviceReadyPollInterval
ctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
for {
info, err := svc.inspect(ctx)
if ctxErr := ctx.Err(); ctxErr != nil { // the wait ended, an inspect error only noticed it
if errors.Is(ctxErr, context.DeadlineExceeded) {
return fmt.Errorf("the service '%s' did not become healthy within %s%s", svc.name, timeout, svc.healthOutputSuffix())
}
return ctxErr
}
switch {
case errors.Is(err, container.ErrContainerNotFound):
return nil // gone, so there is no health left to wait on
case err != nil:
return err
}
switch {
case info.Health == container.HealthUnhealthy:
svc.dumpLogs(ctx)
common.Logger(ctx).Errorf("Failed to initialize container %s", svc.image)
return fmt.Errorf("the service '%s' is unhealthy%s", svc.name, svc.healthOutputSuffix())
case info.Health != container.HealthStarting:
rawLogger.Infof("%s service is healthy.", svc.name)
return nil
}
rawLogger.Infof("%s service is starting, waiting %d seconds before checking again.", svc.name, int(interval.Seconds()))
select {
case <-ctx.Done(): // reported at the top of the loop
case <-time.After(interval):
}
interval = min(interval*2, serviceReadyPollMax)
}
}
// dumpLogs writes the container's log to the job log once, however often it is reported.
func (svc *serviceContainer) dumpLogs(ctx context.Context) {
if svc.logsDumped {
return
}
svc.logsDumped = true
if err := svc.container.DumpLogs(ctx); err != nil {
common.Logger(ctx).Debugf("unable to read the log of service '%s': %v", svc.name, err)
}
}
// inspect also records the state for the `job.services` context.
func (svc *serviceContainer) inspect(ctx context.Context) (*container.Info, error) {
info, err := svc.container.Inspect(ctx)
if err != nil {
return nil, fmt.Errorf("failed to inspect service '%s': %w", svc.name, err)
}
svc.info = info
return info, nil
}
func (svc *serviceContainer) healthOutputSuffix() string {
if svc.info == nil || svc.info.HealthOutput == "" {
return ""
}
return ": " + svc.info.HealthOutput
}
// captureJobContainerInfo is a convenience: failing to describe the container must not
// fail the job.
func (rc *RunContext) captureJobContainerInfo() common.Executor {
return func(ctx context.Context) error {
info, err := rc.JobContainer.Inspect(ctx)
if err != nil {
common.Logger(ctx).Debugf("unable to inspect the job container: %v", err)
return nil
}
rc.jobContainerID = info.ID
return nil
}
}
// Prepare the mounts and binds for the worker // Prepare the mounts and binds for the worker
// ActionCacheDir is for rc // ActionCacheDir is for rc
@@ -1033,9 +1217,26 @@ func (rc *RunContext) getJobContext() *model.JobContext {
if rc.jobCancelled { if rc.jobCancelled {
jobStatus = "cancelled" jobStatus = "cancelled"
} }
return &model.JobContext{
jobContext := &model.JobContext{
Status: jobStatus, Status: jobStatus,
Services: map[string]model.JobService{}, // an empty map, never null
} }
if rc.jobContainerID != "" {
jobContext.Container.ID = rc.jobContainerID
jobContext.Container.Network = rc.jobNetworkName
}
for _, svc := range rc.serviceContainers {
if svc.info == nil {
continue
}
jobContext.Services[svc.name] = model.JobService{
ID: svc.info.ID,
Network: rc.jobNetworkName,
Ports: svc.info.Ports,
}
}
return jobContext
} }
func (rc *RunContext) getStepsContext() map[string]*model.StepResult { func (rc *RunContext) getStepsContext() map[string]*model.StepResult {

View File

@@ -13,6 +13,7 @@ import (
"runtime" "runtime"
"strings" "strings"
"testing" "testing"
"time"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/container" "gitea.com/gitea/runner/act/container"
@@ -22,6 +23,7 @@ import (
"github.com/docker/cli/cli/compose/loader" "github.com/docker/cli/cli/compose/loader"
log "github.com/sirupsen/logrus" log "github.com/sirupsen/logrus"
assert "github.com/stretchr/testify/assert" assert "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
require "github.com/stretchr/testify/require" require "github.com/stretchr/testify/require"
yaml "go.yaml.in/yaml/v4" yaml "go.yaml.in/yaml/v4"
) )
@@ -225,6 +227,12 @@ func (fakeContainer) Copy(string, ...*container.FileEntry) common.Executor {
return func(context.Context) error { return nil } return func(context.Context) error { return nil }
} }
func (fakeContainer) Inspect(context.Context) (*container.Info, error) {
return &container.Info{ID: "fake", State: "running", Health: container.HealthNone}, nil
}
func (fakeContainer) DumpLogs(context.Context) error { return nil }
// Regression test: a service without a `credentials:` block resolves to empty // Regression test: a service without a `credentials:` block resolves to empty
// credentials, which used to overwrite the job container's own credentials. // credentials, which used to overwrite the job container's own credentials.
func TestStartJobContainerKeepsJobCredentialsWithServices(t *testing.T) { func TestStartJobContainerKeepsJobCredentialsWithServices(t *testing.T) {
@@ -563,7 +571,7 @@ func TestCleanupJobResourcesCleansServicesWithoutJobContainer(t *testing.T) {
rc := &RunContext{ rc := &RunContext{
Config: &Config{}, Config: &Config{},
ServiceContainers: []container.ExecutionsEnvironment{service}, serviceContainers: []*serviceContainer{{name: "svc", container: service}},
} }
err := rc.cleanupJobResources("external-network", false)(context.Background()) err := rc.cleanupJobResources("external-network", false)(context.Background())
@@ -586,7 +594,7 @@ func TestCleanupJobResourcesContinuesAfterFailure(t *testing.T) {
Config: &Config{}, Config: &Config{},
Run: &model.Run{Workflow: &model.Workflow{Name: "wf"}, JobID: "job"}, Run: &model.Run{Workflow: &model.Workflow{Name: "wf"}, JobID: "job"},
JobContainer: jobContainer, JobContainer: jobContainer,
ServiceContainers: []container.ExecutionsEnvironment{service}, serviceContainers: []*serviceContainer{{name: "svc", container: service}},
} }
ctx, cancel := context.WithCancel(context.Background()) ctx, cancel := context.WithCancel(context.Background())
@@ -1045,6 +1053,199 @@ func TestRunContext_cleanupFailedStart(t *testing.T) {
}) })
} }
func TestWaitForServiceContainers(t *testing.T) {
origInterval := serviceReadyPollInterval
serviceReadyPollInterval = time.Millisecond
defer func() { serviceReadyPollInterval = origInterval }()
newRunContext := func(timeout time.Duration, services ...*serviceContainer) *RunContext {
return &RunContext{
Config: &Config{ServiceReadyTimeout: timeout},
serviceContainers: services,
}
}
t.Run("returns as soon as a service without a healthcheck runs", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return(&container.Info{ID: "id", State: "running", Health: container.HealthNone}, nil).Once()
rc := newRunContext(0, &serviceContainer{name: "redis", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
service.AssertExpectations(t)
})
t.Run("waits while a service is still starting", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return(&container.Info{ID: "id", State: "running", Health: container.HealthStarting}, nil).Twice()
service.On("Inspect", mock.Anything).
Return(&container.Info{ID: "id", State: "running", Health: container.HealthHealthy}, nil).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
service.AssertExpectations(t)
})
t.Run("fails with the probe output when a service is unhealthy", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).Return(&container.Info{
State: "running",
Health: container.HealthUnhealthy,
HealthOutput: "connection refused",
}, nil).Once()
service.On("DumpLogs", mock.Anything).Return(nil).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "the service 'postgres' is unhealthy: connection refused")
service.AssertExpectations(t)
})
t.Run("lets the steps run when a service exits without a healthcheck", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return(&container.Info{State: "exited", ExitCode: 2, Health: container.HealthNone}, nil).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
})
t.Run("proceeds when the container is gone", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return((*container.Info)(nil), container.ErrContainerNotFound).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
})
t.Run("fails right away when one service fails while another is still starting", func(t *testing.T) {
failing := &containerMock{}
failing.On("Inspect", mock.Anything).
Return(&container.Info{State: "running", Health: container.HealthUnhealthy}, nil)
failing.On("DumpLogs", mock.Anything).Return(nil).Once()
starting := &containerMock{}
starting.On("Inspect", mock.Anything).
Return(&container.Info{State: "running", Health: container.HealthStarting}, nil)
rc := newRunContext(10*time.Second,
&serviceContainer{name: "failing", container: failing},
&serviceContainer{name: "starting", container: starting})
done := make(chan error, 1)
go func() { done <- rc.waitForServiceContainers()(context.Background()) }()
select {
case err := <-done:
require.Error(t, err)
assert.Contains(t, err.Error(), "the service 'failing' is unhealthy")
case <-time.After(2 * time.Second):
t.Fatal("waitForServiceContainers did not fail fast; it waited for the starting service")
}
})
t.Run("gives up once the timeout expires", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return(&container.Info{State: "running", Health: container.HealthStarting}, nil)
rc := newRunContext(20*time.Millisecond, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "did not become healthy within")
})
t.Run("gives up with the same message when the deadline stops an inspect", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Run(func(args mock.Arguments) { <-args.Get(0).(context.Context).Done() }).
Return((*container.Info)(nil), errors.New("inspect aborted"))
rc := newRunContext(20*time.Millisecond, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "did not become healthy within")
})
t.Run("does not wait when the timeout is negative", func(t *testing.T) {
service := &containerMock{}
// Still described once, so the `job.services` context is filled either way.
service.On("Inspect", mock.Anything).
Return(&container.Info{ID: "id", State: "running", Health: container.HealthStarting}, nil).Once()
svc := &serviceContainer{name: "postgres", container: service}
rc := newRunContext(-1, svc)
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
service.AssertExpectations(t)
assert.Equal(t, "id", svc.info.ID)
})
t.Run("fails on an inspect error even when the timeout is negative", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).Return((*container.Info)(nil), errors.New("daemon is gone")).Once()
rc := newRunContext(-1, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "failed to inspect service 'postgres'")
})
t.Run("is a no-op without services", func(t *testing.T) {
require.NoError(t, newRunContext(0).waitForServiceContainers()(context.Background()))
})
}
func TestReportUnstartedServices(t *testing.T) {
dead := &containerMock{}
dead.On("Inspect", mock.Anything).Return(&container.Info{ID: "dead-id", State: "exited", ExitCode: 1}, nil).Once()
dead.On("DumpLogs", mock.Anything).Return(nil).Once()
running := &containerMock{}
running.On("Inspect", mock.Anything).Return(&container.Info{ID: "run-id", State: "running"}, nil).Once()
rc := &RunContext{serviceContainers: []*serviceContainer{
{name: "postgres", container: dead},
{name: "redis", container: running},
}}
require.NoError(t, rc.reportUnstartedServices()(context.Background()))
dead.AssertExpectations(t)
running.AssertExpectations(t)
}
func TestGetJobContextReportsContainers(t *testing.T) {
rc := &RunContext{
jobNetworkName: "job-network",
jobContainerID: "job-container-id",
serviceContainers: []*serviceContainer{
{name: "postgres", info: &container.Info{ID: "svc-id", Ports: map[string]string{"5432": "49153"}}},
// A service that publishes no port reports an empty map, as GitHub does.
{name: "redis", info: &container.Info{ID: "redis-id", Ports: map[string]string{}}},
// A service that never reported is left out rather than reported as empty.
{name: "mailhog"},
},
}
jobContext := rc.getJobContext()
assert.Equal(t, "job-container-id", jobContext.Container.ID)
assert.Equal(t, "job-network", jobContext.Container.Network)
assert.Equal(t, map[string]model.JobService{
"postgres": {ID: "svc-id", Network: "job-network", Ports: map[string]string{"5432": "49153"}},
"redis": {ID: "redis-id", Network: "job-network", Ports: map[string]string{}},
}, jobContext.Services)
}
// A job that never started a container reports an empty context, not a placeholder.
func TestGetJobContextWithoutContainer(t *testing.T) {
jobContext := (&RunContext{}).getJobContext()
assert.Empty(t, jobContext.Container.ID)
assert.Empty(t, jobContext.Container.Network)
assert.Empty(t, jobContext.Services)
}
func TestImageOSFromImage(t *testing.T) { func TestImageOSFromImage(t *testing.T) {
for _, tc := range []struct { for _, tc := range []struct {
image string image string

View File

@@ -74,6 +74,7 @@ type Config struct {
ContainerNetworkCreateOptions container.NewDockerNetworkCreateExecutorInput // the default network create options ContainerNetworkCreateOptions container.NewDockerNetworkCreateExecutorInput // the default network create options
ActionCache ActionCache // Use a custom ActionCache Implementation ActionCache ActionCache // Use a custom ActionCache Implementation
ProxyEnv map[string]string // the proxy variables the job runs with, also given to service containers and image builds ProxyEnv map[string]string // the proxy variables the job runs with, also given to service containers and image builds
PatchToolkit bool // edit the @actions toolkit bundled into an action so it works against Gitea, see toolkit_patch.go
PresetGitHubContext *model.GithubContext // the preset github context, overrides some fields like DefaultBranch, Env, Secrets etc. PresetGitHubContext *model.GithubContext // the preset github context, overrides some fields like DefaultBranch, Env, Secrets etc.
EventJSON string // the content of JSON file to use for event.json in containers, overrides EventPath EventJSON string // the content of JSON file to use for event.json in containers, overrides EventPath
@@ -93,6 +94,7 @@ type Config struct {
InsecureSkipTLS bool // whether to skip verifying TLS certificate of the Gitea instance InsecureSkipTLS bool // whether to skip verifying TLS certificate of the Gitea instance
MaxParallel int // max parallel jobs to run across all workflows (0 = no limit, uses CPU count) MaxParallel int // max parallel jobs to run across all workflows (0 = no limit, uses CPU count)
AllocatePTY bool // allocate a pseudo-TTY for each step's process AllocatePTY bool // allocate a pseudo-TTY for each step's process
ServiceReadyTimeout time.Duration // how long a job waits for its service containers to report healthy (0 uses the default)
RunnerName string // name this runner registered with, reported as `runner.name`, defaults to the hostname RunnerName string // name this runner registered with, reported as `runner.name`, defaults to the hostname
JobStartedHook string // script run inside the job environment before the job's first step; ACTIONS_RUNNER_HOOK_JOB_STARTED is read from Env when empty JobStartedHook string // script run inside the job environment before the job's first step; ACTIONS_RUNNER_HOOK_JOB_STARTED is read from Env when empty
JobCompletedHook string // script run inside the job environment after the job's last step; ACTIONS_RUNNER_HOOK_JOB_COMPLETED is read from Env when empty JobCompletedHook string // script run inside the job environment after the job's last step; ACTIONS_RUNNER_HOOK_JOB_COMPLETED is read from Env when empty

View File

@@ -165,9 +165,22 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
} }
_ = rc.JobContainer.Copy(actPath, files...)(ctx) _ = rc.JobContainer.Copy(actPath, files...)(ctx)
// The command handler needs the step's env to judge ACTIONS_ALLOW_UNSECURE_COMMANDS.
// Cloned: the step executor keeps writing to its own env map after this point, on a
// different goroutine from the command handler that reads it.
rc.setCurrentStepEnv(maps0.Clone(*step.getEnv()))
defer rc.setCurrentStepEnv(nil)
_ = rc.takeUnsecureCommandError() // a refusal from before any step belongs to no step
timeoutctx, cancelTimeOut := evaluateStepTimeout(ctx, rc.ExprEval, stepModel) timeoutctx, cancelTimeOut := evaluateStepTimeout(ctx, rc.ExprEval, stepModel)
defer cancelTimeOut() defer cancelTimeOut()
err = executor(timeoutctx) err = executor(timeoutctx)
// Always take it, so the job-scoped error cannot leak onto a later step. A refusal
// fails the step as it does on GitHub, but the executor's own error wins.
insecureErr := rc.takeUnsecureCommandError()
if err == nil {
err = insecureErr
}
if err == nil { if err == nil {
logger.WithField("stepResult", stepResult.Outcome).Infof("Success - %s %s", stage, stepString) logger.WithField("stepResult", stepResult.Outcome).Infof("Success - %s %s", stage, stepString)
@@ -181,7 +194,7 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
} }
if continueOnError { if continueOnError {
logger.Errorf("##[error]%s", escapeCommandData(err.Error())) logger.Errorf("##[error]%s", EscapeCommandData(err.Error()))
logger.Infof("Failed but continue next step") logger.Infof("Failed but continue next step")
err = nil err = nil
stepResult.Conclusion = model.StepStatusSuccess stepResult.Conclusion = model.StepStatusSuccess

View File

@@ -69,6 +69,8 @@ func (sar *stepActionRemote) prepareActionExecutor() common.Executor {
github.Token = sar.RunContext.Config.ReplaceGheActionTokenWithGithubCom github.Token = sar.RunContext.Config.ReplaceGheActionTokenWithGithubCom
} }
} }
// Actions served from the action cache are read out of a git object store rather than a
// directory, so they never reach the bundle patch below and keep to the v1 cache API.
if sar.RunContext.Config.ActionCache != nil { if sar.RunContext.Config.ActionCache != nil {
cache := sar.RunContext.Config.ActionCache cache := sar.RunContext.Config.ActionCache
@@ -112,7 +114,7 @@ func (sar *stepActionRemote) prepareActionExecutor() common.Executor {
return err return err
} }
actionDir := fmt.Sprintf("%s/%s", sar.RunContext.ActionCacheDir(), sar.Step.UsesHash()) actionDir := sar.actionDir()
defaultActionURL := sar.RunContext.Config.DefaultActionURL() defaultActionURL := sar.RunContext.Config.DefaultActionURL()
// For Gitea // For Gitea
// A composite RunContext nils Config.Secrets, so getGitCloneToken would yield an // A composite RunContext nils Config.Secrets, so getGitCloneToken would yield an
@@ -171,6 +173,9 @@ func (sar *stepActionRemote) prepareActionExecutor() common.Executor {
sar.action = actionModel sar.action = actionModel
return err return err
}, },
// A stage of its own: it takes the same clone lock, and it has to land before
// runAction copies the action into the job container.
sar.patchActionToolkit,
)(ctx) )(ctx)
} }
} }
@@ -189,7 +194,7 @@ func (sar *stepActionRemote) pre() common.Executor {
return common.NewPipelineExecutor( return common.NewPipelineExecutor(
sar.prepareActionExecutor(), sar.prepareActionExecutor(),
runStepExecutor(sar, stepStagePre, runPreStep(sar)).If(hasPreStep(sar)).If(shouldRunPreStep(sar))) runStepExecutor(sar, stepStagePre, sar.revertToolkitOnFailure(runPreStep(sar))).If(hasPreStep(sar)).If(shouldRunPreStep(sar)))
} }
func (sar *stepActionRemote) main() common.Executor { func (sar *stepActionRemote) main() common.Executor {
@@ -211,15 +216,51 @@ func (sar *stepActionRemote) main() common.Executor {
return sar.RunContext.JobContainer.CopyDir(copyToPath, sar.RunContext.Config.Workdir+string(filepath.Separator)+".", sar.RunContext.Config.UseGitIgnore)(ctx) return sar.RunContext.JobContainer.CopyDir(copyToPath, sar.RunContext.Config.Workdir+string(filepath.Separator)+".", sar.RunContext.Config.UseGitIgnore)(ctx)
} }
actionDir := fmt.Sprintf("%s/%s", sar.RunContext.ActionCacheDir(), sar.Step.UsesHash()) actionDir := sar.actionDir()
return sar.runAction(sar, actionDir, sar.remoteAction)(ctx) return sar.revertToolkitOnFailure(sar.runAction(sar, actionDir, sar.remoteAction))(ctx)
}), }),
) )
} }
func (sar *stepActionRemote) post() common.Executor { func (sar *stepActionRemote) post() common.Executor {
return runStepExecutor(sar, stepStagePost, runPostStep(sar)).If(hasPostStep(sar)).If(shouldRunPostStep(sar)) return runStepExecutor(sar, stepStagePost, sar.revertToolkitOnFailure(runPostStep(sar))).If(hasPostStep(sar)).If(shouldRunPostStep(sar))
}
// toolkitBundles is the action directory and the entrypoints the toolkit may live in.
func (sar *stepActionRemote) toolkitBundles() (string, []string) {
if sar.remoteAction == nil {
return "", nil
}
dir := sar.actionDir()
return dir, actionScriptPaths(filepath.Join(dir, sar.remoteAction.Path), sar.action)
}
// patchActionToolkit edits the bundled toolkit so it works against Gitea: the artifact actions
// stop refusing, and the cache client keeps to the cache server whichever API version it picks.
func (sar *stepActionRemote) patchActionToolkit(ctx context.Context) error {
if sar.RunContext.Config.PatchToolkit {
dir, scripts := sar.toolkitBundles()
patchToolkit(ctx, dir, scripts)
}
return nil
}
// revertToolkitOnFailure restores the untouched bundles when the action fails, so a later job
// runs it as shipped rather than repeating a failure the patch may have caused.
func (sar *stepActionRemote) revertToolkitOnFailure(exec common.Executor) common.Executor {
return func(ctx context.Context) error {
err := exec(ctx)
if err != nil {
dir, scripts := sar.toolkitBundles()
revertToolkit(ctx, dir, scripts)
}
return err
}
}
func (sar *stepActionRemote) actionDir() string {
return fmt.Sprintf("%s/%s", sar.RunContext.ActionCacheDir(), sar.Step.UsesHash())
} }
func (sar *stepActionRemote) getRunContext() *RunContext { func (sar *stepActionRemote) getRunContext() *RunContext {
@@ -270,7 +311,7 @@ func (sar *stepActionRemote) getActionModel() *model.Action {
func (sar *stepActionRemote) getCompositeRunContext(ctx context.Context) *RunContext { func (sar *stepActionRemote) getCompositeRunContext(ctx context.Context) *RunContext {
if sar.compositeRunContext == nil { if sar.compositeRunContext == nil {
actionDir := fmt.Sprintf("%s/%s", sar.RunContext.ActionCacheDir(), sar.Step.UsesHash()) actionDir := sar.actionDir()
actionLocation := path.Join(actionDir, sar.remoteAction.Path) actionLocation := path.Join(actionDir, sar.remoteAction.Path)
_, containerActionDir := getContainerActionPaths(sar.getStepModel(), actionLocation, sar.RunContext) _, containerActionDir := getContainerActionPaths(sar.getStepModel(), actionLocation, sar.RunContext)

View File

@@ -18,6 +18,7 @@ import (
"gitea.com/gitea/runner/act/model" "gitea.com/gitea/runner/act/model"
"github.com/kballard/go-shellquote" "github.com/kballard/go-shellquote"
"github.com/sirupsen/logrus"
yaml "go.yaml.in/yaml/v4" yaml "go.yaml.in/yaml/v4"
) )
@@ -63,7 +64,7 @@ func (sr *stepRun) printRunScriptActionDetails(ctx context.Context) {
normalized := strings.TrimRight(strings.ReplaceAll(sr.interpolatedScript, "\r\n", "\n"), "\n") normalized := strings.TrimRight(strings.ReplaceAll(sr.interpolatedScript, "\r\n", "\n"), "\n")
rawLogger.Infof("::group::Run %s", escapeCommandData(sr.runScriptGroupTitle(normalized))) rawLogger.Infof("::group::Run %s", EscapeCommandData(sr.runScriptGroupTitle(normalized)))
if normalized != "" { if normalized != "" {
for line := range strings.SplitSeq(normalized, "\n") { for line := range strings.SplitSeq(normalized, "\n") {
@@ -90,12 +91,12 @@ func printRunActionHeader(ctx context.Context, step *model.Step, env map[string]
if step.Name != "" { if step.Name != "" {
title = step.Name title = step.Name
} }
rawLogger.Infof("::group::Run %s", escapeCommandData(title)) rawLogger.Infof("::group::Run %s", EscapeCommandData(title))
if len(step.With) > 0 { if len(step.With) > 0 {
rawLogger.Infof("with:") rawLogger.Infof("with:")
for _, k := range slices.Sorted(maps.Keys(step.With)) { for _, k := range slices.Sorted(maps.Keys(step.With)) {
rawLogger.Infof(" %s: %s", k, step.With[k]) logKeyedValue(rawLogger, k, step.With[k])
} }
} }
@@ -129,7 +130,17 @@ func printStepEnvBlock(ctx context.Context, step *model.Step, env map[string]str
if caseInsensitive { if caseInsensitive {
lookupKey = strings.ToUpper(k) lookupKey = strings.ToUpper(k)
} }
rawLogger.Infof(" %s: %s", k, envLookup[lookupKey]) logKeyedValue(rawLogger, k, envLookup[lookupKey])
}
}
// logKeyedValue prints one row per line of value: Gitea stores one log row per line, so an
// embedded newline would reach the user as a literal "\n".
func logKeyedValue(rawLogger *logrus.Entry, key, value string) {
lines := strings.Split(strings.ReplaceAll(value, "\r\n", "\n"), "\n")
rawLogger.Infof(" %s: %s", key, lines[0])
for _, line := range lines[1:] {
rawLogger.Infof(" %s", line)
} }
} }

View File

@@ -6,6 +6,7 @@ package runner
import ( import (
"context" "context"
"errors"
"testing" "testing"
"gitea.com/gitea/runner/act/common" "gitea.com/gitea/runner/act/common"
@@ -14,6 +15,7 @@ import (
log "github.com/sirupsen/logrus" log "github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock" "github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
yaml "go.yaml.in/yaml/v4" yaml "go.yaml.in/yaml/v4"
) )
@@ -354,3 +356,48 @@ func TestIsContinueOnError(t *testing.T) {
assertObject.False(continueOnError) assertObject.False(continueOnError)
assertObject.Error(err) assertObject.Error(err)
} }
// A refused ::set-env::/::add-path:: records a job-scoped error. When the step that
// produced it also fails on its own, the refusal must be cleared at the step boundary, so
// it fails only that step and never leaks onto a later step that runs anyway (if: always()).
func TestRunStepExecutorDoesNotLeakRefusalToNextStep(t *testing.T) {
cm := &containerMock{}
noop := func(context.Context) error { return nil }
cm.On("Copy", mock.Anything, mock.Anything).Return(noop)
cm.On("UpdateFromEnv", mock.Anything, mock.Anything).Return(noop)
rc := &RunContext{
Config: &Config{Env: map[string]string{}},
Run: &model.Run{
JobID: "1",
Workflow: &model.Workflow{Jobs: map[string]*model.Job{"1": {}}},
},
Env: map[string]string{},
StepResults: map[string]*model.StepResult{},
JobContainer: cm,
}
rc.ExprEval = rc.NewExpressionEvaluator(context.Background())
// Dryrun skips reading the path file back from the (mocked) container.
ctx := common.WithDryrun(context.Background(), true)
// A refusal parsed out of the job container's own output belongs to no step, so the
// first step must not be failed by it.
rc.commandHandler(ctx)("::set-env name=setup::y\n")
stepSetup := &stepRun{RunContext: rc, Step: &model.Step{ID: "setup"}, env: map[string]string{}}
require.NoError(t, runStepExecutor(stepSetup, stepStageMain, func(context.Context) error { return nil })(ctx))
// Step A refuses a ::set-env:: and then fails on its own.
stepA := &stepRun{RunContext: rc, Step: &model.Step{ID: "a"}, env: map[string]string{}}
errA := runStepExecutor(stepA, stepStageMain, func(context.Context) error {
rc.commandHandler(ctx)("::set-env name=x::y\n")
return errors.New("boom")
})(ctx)
// The step fails with its own error, not the refusal.
require.ErrorContains(t, errA, "boom")
// Step B runs despite step A's failure (if: always()) and issues no unsecure command;
// it must not inherit step A's refusal.
stepB := &stepRun{RunContext: rc, Step: &model.Step{ID: "b", If: yaml.Node{Value: "always()"}}, env: map[string]string{}}
errB := runStepExecutor(stepB, stepStageMain, func(context.Context) error { return nil })(ctx)
require.NoError(t, errB)
}

View File

@@ -3,6 +3,7 @@ jobs:
_: _:
runs-on: ubuntu-latest runs-on: ubuntu-latest
env: env:
ACTIONS_ALLOW_UNSECURE_COMMANDS: 'true'
MYGLOBALENV3: myglobalval3 MYGLOBALENV3: myglobalval3
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4

View File

@@ -4,6 +4,8 @@ on: push
jobs: jobs:
build: build:
runs-on: ubuntu-latest runs-on: ubuntu-latest
env:
ACTIONS_ALLOW_UNSECURE_COMMANDS: 'true'
steps: steps:
- name: TEST set-env - name: TEST set-env
run: echo "::set-env name=foo::bar" run: echo "::set-env name=foo::bar"

View File

@@ -15,3 +15,9 @@ jobs:
echo "id: ${{ job.services.postgres.id }}" echo "id: ${{ job.services.postgres.id }}"
echo "network: ${{ job.services.postgres.network }}" echo "network: ${{ job.services.postgres.network }}"
echo "ports: ${{ job.services.postgres.ports }}" echo "ports: ${{ job.services.postgres.ports }}"
- name: The job context describes the started containers
run: |
test -n "${{ job.container.id }}"
test -n "${{ job.services.postgres.id }}"
test -n "${{ job.services.postgres.ports['80'] }}"
test "${{ job.services.postgres.network }}" = "${{ job.container.network }}"

262
act/runner/toolkit_patch.go Normal file
View File

@@ -0,0 +1,262 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package runner
import (
"bytes"
"context"
"os"
"path/filepath"
"regexp"
"strings"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/git"
"gitea.com/gitea/runner/act/model"
)
// Actions bundle the @actions toolkit into their own JavaScript, and two of its lines keep it
// from working against Gitea. Both are edited out of the bundle the runner downloaded.
//
// isGhes() takes any host that is not github.com, *.ghe.com or *.localhost for GitHub
// Enterprise. @actions/cache then forces the v1 API, and @actions/artifact refuses outright,
// which is why the stock upload-artifact aborts here. The edit empties the last of the three
// hostname tests, so `endsWith('.LOCALHOST')` becomes `endsWith(”)`, which every hostname
// satisfies: one string literal, no call sites to resolve, and the same answer the toolkit's own
// proposed ACTIONS_VENDOR switch would give. Gitea already makes this edit by hand in its fork
// of upload-artifact.
//
// getCacheServiceURL() then resolves the cache service from ACTIONS_RESULTS_URL alone, where v1
// reads ACTIONS_CACHE_URL first. Both reads there are given the same preference, which is what
// keeps the runner out of the artifact path: the results URL still points at Gitea.
//
// Either of these landing upstream makes this file deletable:
//
// https://github.com/actions/toolkit/pull/2123 — an ACTIONS_VENDOR switch, naming Gitea
// https://github.com/actions/toolkit/issues/2439 — treat ACTIONS_RESULTS_URL as the signal
const (
CacheServiceV2Env = "ACTIONS_CACHE_SERVICE_V2"
cacheURLEnv = "ACTIONS_CACHE_URL"
resultsURLEnv = "ACTIONS_RESULTS_URL"
// localhostHost is the suffix isGhes accepts; emptying the test is what opens the gate,
// because every hostname ends with the empty string.
localhostHost = ".LOCALHOST"
// artifactRefusal is the only thing the gate guards in @actions/artifact, which is what makes
// such a bundle safe to open. A bundle carrying neither toolkit uses isGhes for something this
// runner has not looked at, and is left alone.
artifactRefusal = "GHESNotSupportedError"
// sidecarSuffix names the directory of untouched copies, a sibling of the action directory
// because that directory is copied wholesale into job containers.
sidecarSuffix = ".toolkit-patch"
// skipMarker in the sidecar means a patched bundle already failed once here.
skipMarker = "skip"
maxBundleSize = 64 << 20
)
var (
// localhostTest matches the third hostname test of isGhes, in any quoting. The match is case
// sensitive on purpose, and that is load-bearing: isGhes uppercases the hostname before
// testing it, while undici, bundled into all of these actions, tests a lowercase ".localhost"
// in isURLPotentiallyTrustworthy. Opening that one would tell its HTTP client that every URL
// is trustworthy. Uppercase, the literal occurs nowhere but this test, across 118 bundles
// covering every major version of sixteen actions.
localhostTest = regexp.MustCompile(`endsWith\s*\(\s*` + quoted(regexp.QuoteMeta(localhostHost)) + `\s*\)`)
// serviceURLBranches matches both branches of getCacheServiceURL at once: the v1 branch reads
// the cache URL and falls back to the results URL, the v2 branch just below reads the results
// URL alone. That `||` pairing is the only place the two variables are read together, so
// matching them as one expression is what keeps the edit inside this function rather than
// anywhere they happen to sit near each other. The branches are 21 bytes apart minified and
// 63 not, across every bundle measured.
serviceURLBranches = regexp.MustCompile(`(` + envRead(cacheURLEnv) + `\s*\|\|\s*)(` +
envRead(resultsURLEnv) + `)((?s).{0,256}?)(` + envRead(resultsURLEnv) + `)`)
// cacheURLFirst gives both reads the preference the v1 branch already had.
cacheURLFirst = []byte(`${1}(process.env.` + cacheURLEnv + `||${2})${3}(process.env.` + cacheURLEnv + `||${4})`)
)
func envRead(name string) string {
return `process\s*\.\s*env\s*(?:\.\s*` + name + `\b|\[\s*` + quoted(name) + `\s*\])`
}
// quoted matches a string literal in any of the three quote characters. RE2 has no
// backreferences, so the pairs are spelled out.
func quoted(pattern string) string {
return "(?:'" + pattern + "'|\"" + pattern + "\"|`" + pattern + "`)"
}
// actionScriptPaths returns the entrypoints of a node action, the only kind with a bundle. Only
// remote actions get here: a local one lives in the user's checkout, which the runner does not
// rewrite.
func actionScriptPaths(dir string, action *model.Action) []string {
if action == nil || !action.Runs.Using.IsNode() {
return nil
}
var paths []string
for _, script := range []string{action.Runs.Pre, action.Runs.Main, action.Runs.Post} {
if script != "" {
paths = append(paths, filepath.Join(dir, script))
}
}
return paths
}
// patchToolkit edits the toolkit in an action's bundles, keeping each original beside them. Every
// failure is silent and leaves the bundle as it was, which costs the cache client the v2 API and
// an artifact action nothing at all.
func patchToolkit(ctx context.Context, actionDir string, scripts []string) {
if len(scripts) == 0 {
return
}
if _, err := os.Stat(filepath.Join(sidecarDir(actionDir), skipMarker)); err == nil {
return
}
defer git.AcquireCloneLock(actionDir)()
for _, script := range scripts {
if err := patchBundle(script, originalFor(actionDir, script)); err != nil {
common.Logger(ctx).Debugf("actions toolkit: %s left unpatched: %v", filepath.Base(script), err)
}
}
}
// revertToolkit puts the originals back and stops this action being patched again, so the next job
// runs it exactly as shipped. Called when a step failed with a patched bundle; it does not re-run
// the step, because a step's outputs and env-file writes are already recorded by then.
func revertToolkit(ctx context.Context, actionDir string, scripts []string) {
if len(scripts) == 0 {
return
}
if _, err := os.Stat(sidecarDir(actionDir)); err != nil {
return
}
defer git.AcquireCloneLock(actionDir)()
reverted := false
for _, script := range scripts {
original := originalFor(actionDir, script)
if !isPatchOf(original, script) {
continue
}
if err := os.Rename(original, script); err == nil {
reverted = true
}
}
if reverted {
_ = os.WriteFile(filepath.Join(sidecarDir(actionDir), skipMarker), nil, 0o600)
common.Logger(ctx).Warnf("actions toolkit: restored the original %s, it will not be patched again", filepath.Base(actionDir))
}
}
// sidecarDir holds an action's untouched bundles, and the marker that stops it being patched.
func sidecarDir(actionDir string) string {
return actionDir + sidecarSuffix
}
// originalFor is where a script's untouched copy lives, or "" for a script the action's own
// `runs` keys placed outside its directory, which is not this runner's to rewrite.
func originalFor(actionDir, script string) string {
rel, err := filepath.Rel(actionDir, script)
if err != nil || strings.HasPrefix(rel, "..") {
return ""
}
return filepath.Join(sidecarDir(actionDir), rel)
}
// patchBundle rewrites one entrypoint in place. The untouched copy kept beside it is what marks
// the bundle as already patched.
func patchBundle(script, original string) error {
if original == "" {
return nil
}
if _, err := os.Stat(original); err == nil {
if isPatchOf(original, script) {
return nil
}
// The action's ref moved and git checked the new bundle out over the patched one, so
// the pair no longer belongs together. Patch afresh rather than keep an original that
// would restore an older version of the action.
if err := os.Remove(original); err != nil {
return err
}
}
info, err := os.Stat(script)
if err != nil {
return err
}
if info.Size() > maxBundleSize {
return nil
}
data, err := os.ReadFile(script)
if err != nil {
return err
}
patched, ok := patchedBundle(data)
if !ok {
return nil
}
if err := os.MkdirAll(filepath.Dir(original), 0o755); err != nil {
return err
}
// The copy is taken before the bundle is replaced, so a write that fails part way can put the
// action back as it was. A crash needs no handling: the clone executor checks the action out
// and hard resets it on every prepare, so a half-written bundle never outlives the job.
if err := os.WriteFile(original, data, info.Mode().Perm()); err != nil {
return err
}
if err := os.WriteFile(script, patched, info.Mode().Perm()); err != nil {
_ = os.Rename(original, script)
return err
}
return nil
}
// isPatchOf reports whether script is exactly what patching original produced. It is what proves
// the two still belong together: an action whose ref moved is checked out over the patched bundle,
// leaving an original that would restore the version before the move.
func isPatchOf(original, script string) bool {
data, err := os.ReadFile(original)
if err != nil {
return false
}
current, err := os.ReadFile(script)
if err != nil {
return false
}
patched, ok := patchedBundle(data)
return ok && bytes.Equal(patched, current)
}
// patchedBundle opens the GHES gate, and where the cache toolkit is present, points the cache
// service at the cache server. A bundle this runner cannot account for comes back untouched.
func patchedBundle(data []byte) ([]byte, bool) {
if !localhostTest.Match(data) {
return data, false
}
switch {
case bytes.Contains(data, []byte(CacheServiceV2Env)):
// The cache toolkit: both edits or neither, because choosing v2 without redirecting the
// URL would send the client to a results URL that serves no cache service.
if !serviceURLBranches.Match(data) {
return data, false
}
case bytes.Contains(data, []byte(artifactRefusal)):
// The artifact toolkit, where the gate is a plain refusal and there is no URL to move:
// artifacts already go to Gitea, which implements that service.
default:
return data, false
}
opened := localhostTest.ReplaceAllFunc(data, func(test []byte) []byte {
// Drop the hostname from the test rather than rewriting the call, so the bundle's own
// quoting survives and the result stays valid even inside a string literal.
return bytes.Replace(test, []byte(localhostHost), nil, 1)
})
return serviceURLBranches.ReplaceAll(opened, cacheURLFirst), true
}

View File

@@ -0,0 +1,368 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package runner
import (
"context"
"encoding/base64"
"io"
"net/http"
"net/http/httptest"
"os"
"os/exec"
"path"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
"gitea.com/gitea/runner/act/artifactcache"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// actionsCacheRef pins the actions/cache release this is verified against. Bump it
// deliberately: a new release is exactly what can stop the patch matching.
const actionsCacheRef = "v6.1.0"
// bundleFromGitHub downloads one entrypoint, keeping it in the user cache dir so repeated runs
// cost nothing. The bundles are megabytes, too large to vendor.
func bundleFromGitHub(t *testing.T, repo, ref, path string) string {
t.Helper()
cacheDir, err := os.UserCacheDir()
require.NoError(t, err)
dir := filepath.Join(cacheDir, "gitea-runner-test", strings.ReplaceAll(repo, "/", "-")+"-"+ref)
bundle := filepath.Join(dir, strings.ReplaceAll(path, "/", "-"))
if _, err := os.Stat(bundle); err == nil {
return bundle
}
require.NoError(t, os.MkdirAll(dir, 0o755))
url := "https://raw.githubusercontent.com/" + repo + "/" + ref + "/" + path
ctx, cancel := context.WithTimeout(t.Context(), time.Minute)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
require.NoError(t, err)
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Skipf("cannot reach %s: %v", url, err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Skipf("GET %s: %s", url, resp.Status)
}
file, err := os.Create(bundle)
require.NoError(t, err)
_, err = io.Copy(file, resp.Body)
require.NoError(t, file.Close())
require.NoError(t, err)
return bundle
}
// jobEnv is the environment a job gets from this runner, which is what decides where an action's
// toolkit looks for the cache and artifact services.
type jobEnv struct {
workspace, runnerTemp string
cacheURL, resultsURL string
token string
}
// runActionEntrypoint runs one action entrypoint the way a job would. Adding another action to these tests
// means downloading its entrypoint with bundleFromGitHub and calling this with its inputs, whose
// names are the ones the action's own action.yml uses.
func runActionEntrypoint(t *testing.T, script string, env jobEnv, inputs map[string]string) string {
t.Helper()
state := filepath.Join(env.runnerTemp, "state")
output := filepath.Join(env.runnerTemp, "output")
for _, name := range []string{state, output} {
require.NoError(t, os.WriteFile(name, nil, 0o600))
}
cmd := exec.CommandContext(t.Context(), "node", script)
cmd.Dir = env.workspace
cmd.Env = append(os.Environ(),
"ACTIONS_RUNTIME_TOKEN="+env.token,
"ACTIONS_CACHE_URL="+env.cacheURL+"/",
"ACTIONS_RESULTS_URL="+env.resultsURL,
"ACTIONS_CACHE_SERVICE_V2=true",
"GITHUB_SERVER_URL=https://gitea.example.com",
"GITHUB_REPOSITORY=testuser/testrepo",
"GITHUB_RUN_ID=1",
"GITHUB_REF=refs/heads/main",
"GITHUB_EVENT_NAME=push",
"GITHUB_WORKSPACE="+env.workspace,
"RUNNER_TEMP="+env.runnerTemp,
"GITHUB_STATE="+state,
"GITHUB_OUTPUT="+output,
)
for name, value := range inputs {
cmd.Env = append(cmd.Env, "INPUT_"+strings.ToUpper(name)+"="+value)
}
out, err := cmd.CombinedOutput()
t.Logf("%s:\n%s", filepath.Base(script), out)
require.NoError(t, err, "%s failed", script)
return string(out)
}
// patchedAction downloads one entrypoint and patches it exactly as a downloaded action would be,
// keeping the untouched original in the sidecar beside it.
func patchedAction(t *testing.T, repo, ref, entrypoint string) string {
t.Helper()
body, err := os.ReadFile(bundleFromGitHub(t, repo, ref, entrypoint))
require.NoError(t, err)
dir := tempDirPath(t)
script := filepath.Join(dir, filepath.Base(entrypoint))
require.NoError(t, os.WriteFile(script, body, 0o600))
patchToolkit(t.Context(), dir, []string{script})
return script
}
// tempDirPath is TempDir with symlinks resolved, because macOS hands out /var paths that resolve
// to /private/var and the client derives archive paths relative to the workspace.
func tempDirPath(t *testing.T) string {
t.Helper()
dir, err := filepath.EvalSymlinks(t.TempDir())
require.NoError(t, err)
return dir
}
// The whole chain against the pinned release, whose bundles ship unminified: patch them, run the
// real client with an ordinary Gitea server URL and a results URL that goes nowhere, and have it
// save and restore through this runner's cache server. The unreachable results URL is the point,
// it is what proves the cache reaches the runner without the runner fronting Gitea. If a release
// stops matching the patch the client falls back to v1 and this fails on the version line, which
// is the signal to look at the new bundle.
func TestCacheServiceV2EndToEnd(t *testing.T) {
requireHostTools(t, "node")
restore := patchedAction(t, "actions/cache", actionsCacheRef, "dist/restore/index.js")
save := patchedAction(t, "actions/cache", actionsCacheRef, "dist/save/index.js")
handler, err := artifactcache.StartHandler(filepath.Join(t.TempDir(), "cache"), "127.0.0.1", 0, "", nil)
require.NoError(t, err)
t.Cleanup(func() { _ = handler.Close() })
const token, repo = "e2e-runtime-token", "testuser/testrepo"
handler.RegisterJob(token, artifactcache.JobCredential{Repo: repo})
env := jobEnv{
workspace: tempDirPath(t),
runnerTemp: tempDirPath(t),
cacheURL: handler.ExternalURL(),
// The results service is the cache server's too, which is what the runner advertises.
resultsURL: handler.ExternalURL(),
token: token,
}
require.NoError(t, os.MkdirAll(filepath.Join(env.workspace, "to-cache"), 0o755))
content := []byte("cached through the patched gate")
require.NoError(t, os.WriteFile(filepath.Join(env.workspace, "to-cache", "data.txt"), content, 0o600))
const key = "patched-gate-key"
inputs := map[string]string{"path": "to-cache", "key": key}
missed := runActionEntrypoint(t, restore, env, inputs)
require.Contains(t, missed, "Cache service version: v2", "the patch did not take, the client stayed on v1")
require.Contains(t, missed, "Cache not found for input keys: "+key)
saved := runActionEntrypoint(t, save, env, inputs)
require.Contains(t, saved, "Cache saved with key: "+key)
env.workspace = tempDirPath(t)
hit := runActionEntrypoint(t, restore, env, inputs)
require.Contains(t, hit, "Cache restored from key: "+key)
got, err := os.ReadFile(filepath.Join(env.workspace, "to-cache", "data.txt"))
require.NoError(t, err)
assert.Equal(t, content, got)
// Untouched, the same client takes a Gitea host for GHES and stays on v1, which reaches the
// cache server on its own address. That is what a runner without a results service of its own
// leaves its jobs with, so it has to round trip too.
env.workspace = tempDirPath(t)
v1 := runActionEntrypoint(t, filepath.Join(sidecarDir(filepath.Dir(restore)), "index.js"), env, inputs)
require.Contains(t, v1, "Cache service version: v1")
require.Contains(t, v1, "Cache restored from key: "+key)
}
// The gate and the URL getter are separate functions, and a bundler may put either first: the gap
// between them runs from 159 to 1179 bytes across these actions, which is why neither edit is
// anchored on that distance. One entrypoint from each of the families that bundle the cache
// toolkit, patched but not run, is what keeps a future release from quietly matching only one of
// the two shapes and leaving every cache on v1.
func TestToolkitPatchAcrossActions(t *testing.T) {
for _, tc := range []struct {
repo, ref, path string
wantPatched bool
}{
// The cache toolkit, in each bundler shape and from a spread of ecosystems, including the
// actions that drive a Go or a Rust cache client of their own.
{"actions/cache", actionsCacheRef, "dist/restore/index.js", true},
{"actions/setup-node", "v7.0.0", "dist/cache-save/index.js", true},
{"actions/setup-python", "v7.0.0", "dist/setup/index.js", true},
{"actions/setup-go", "v7.0.0", "dist/setup/index.js", true},
{"actions/setup-java", "v5.7.0", "dist/setup/index.js", true},
{"ruby/setup-ruby", "v1.321.0", "dist/index.js", true},
{"pnpm/action-setup", "v6.0.9", "dist/index.js", true},
{"oven-sh/setup-bun", "v2.2.0", "dist/setup/index.js", true},
{"Swatinem/rust-cache", "v2.9.1", "dist/restore/index.js", true},
{"docker/build-push-action", "v7.3.0", "dist/index.cjs", true},
// The artifact toolkit, where the gate is a refusal and there is nothing to redirect.
// v4.4.0 is the first release whose gate carries the localhost test this matches; the
// releases before it refuse in a shape the runner leaves alone.
{"actions/upload-artifact", "v4.4.0", "dist/upload/index.js", true},
{"actions/upload-artifact", "v7.0.1", "dist/upload/index.js", true},
{"actions/download-artifact", "v8.0.1", "dist/index.js", true},
// Neither toolkit's gate, so these have to come back byte for byte. sccache-action is the
// one that exports ACTIONS_CACHE_SERVICE_V2 itself, for the Rust client it installs.
{"actions/checkout", "v7.0.1", "dist/index.js", false},
{"mozilla-actions/sccache-action", "v0.0.11", "dist/setup/index.js", false},
} {
t.Run(tc.repo+"@"+tc.ref, func(t *testing.T) {
t.Parallel()
data, err := os.ReadFile(bundleFromGitHub(t, tc.repo, tc.ref, tc.path))
require.NoError(t, err)
out, patched := patchedBundle(data)
require.Equal(t, tc.wantPatched, patched)
if !tc.wantPatched {
assert.Equal(t, data, out, "an untouched bundle must come back byte for byte")
return
}
assert.NotContains(t, string(out), ".LOCALHOST", "a copy of the gate was missed")
if !strings.Contains(string(data), CacheServiceV2Env) {
return // the artifact toolkit: a refusal to open, and no URL to move
}
// Only the reads inside getCacheServiceURL are rewritten. The others, such as the
// feature-availability check, must be left as they are.
assert.NotZero(t, strings.Count(string(out), "(process.env."+cacheURLEnv+"||process.env"),
"the cache service URL was not redirected")
assert.Equal(t, strings.Count(string(data), resultsURLEnv), strings.Count(string(out), resultsURLEnv),
"a read of the results URL was lost, it must stay as the fallback")
})
}
}
// The stock artifact actions refuse on a Gitea host until the gate is opened, and then they talk
// to the results service, which is this runner's cache server forwarding the artifact half on to
// Gitea. Running the real upload-artifact against a stand-in Gitea covers both halves at once:
// the patch, and the forwarding the job's registration set up.
func TestUploadArtifactThroughTheResultsService(t *testing.T) {
requireHostTools(t, "node")
var called []string
var zipped []byte
gitea := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
method := path.Base(r.URL.Path)
called = append(called, method)
w.Header().Set("x-ms-request-id", "stub")
switch method {
case "CreateArtifact":
_, _ = io.WriteString(w, `{"ok":true,"signed_upload_url":"http://`+r.Host+
`/twirp/github.actions.results.api.v1.ArtifactService/UploadArtifact?sig=x"}`)
case "FinalizeArtifact":
_, _ = io.WriteString(w, `{"ok":true,"artifact_id":"1"}`)
case "ListArtifacts":
_, _ = io.WriteString(w, `{"artifacts":[{"workflow_run_backend_id":"11",`+
`"workflow_job_run_backend_id":"22","database_id":"1","name":"an-artifact","size":"`+
strconv.Itoa(len(zipped))+`"}]}`)
case "GetSignedArtifactURL":
_, _ = io.WriteString(w, `{"signed_url":"http://`+r.Host+`/download"}`)
case "download":
w.Header().Set("Content-Type", "application/zip")
_, _ = w.Write(zipped)
default: // the zip on its way up, in the blocks the Azure protocol puts it in
body, _ := io.ReadAll(r.Body)
switch r.URL.Query().Get("comp") {
case "block":
zipped = append(zipped, body...)
case "blocklist": // the ordering document, not content
default:
zipped = body
}
w.WriteHeader(http.StatusCreated)
}
}))
defer gitea.Close()
handler, err := artifactcache.StartHandler(filepath.Join(t.TempDir(), "cache"), "127.0.0.1", 0, "", nil)
require.NoError(t, err)
t.Cleanup(func() { _ = handler.Close() })
// The artifact client decodes the runtime token for the run ids it puts in its requests, where
// the cache client only presents it, so this one has to be shaped like Gitea's.
token := "e30." + base64.RawURLEncoding.EncodeToString([]byte(`{"scp":"Actions.Results:11:22"}`)) + ".sig"
defer handler.RegisterJob(token, artifactcache.JobCredential{Repo: "testuser/testrepo", Results: gitea.URL})()
upload := patchedAction(t, "actions/upload-artifact", "v7.0.1", "dist/upload/index.js")
env := jobEnv{
workspace: tempDirPath(t),
runnerTemp: tempDirPath(t),
cacheURL: handler.ExternalURL(),
resultsURL: handler.ExternalURL(),
token: token,
}
uploaded := []byte("through the results service")
require.NoError(t, os.WriteFile(filepath.Join(env.workspace, "artifact.txt"), uploaded, 0o600))
out := runActionEntrypoint(t, upload, env, map[string]string{
"name": "an-artifact", "path": "artifact.txt", "if-no-files-found": "error",
"retention-days": "0", "compression-level": "6", "overwrite": "false",
"include-hidden-files": "false", "archive": "true",
})
require.Contains(t, out, "has been successfully uploaded")
// And back down again: listing and downloading go the same way, and the signed URL the
// artifact service hands out is fetched straight from it.
download := patchedAction(t, "actions/download-artifact", "v8.0.1", "dist/index.js")
env.workspace = tempDirPath(t)
out = runActionEntrypoint(t, download, env, map[string]string{
"name": "an-artifact", "path": "downloaded", "merge-multiple": "false",
"skip-decompress": "false", "include-hidden-files": "false", "github-token": "",
})
require.Contains(t, out, "Artifact download completed")
assert.Subset(t, called,
[]string{"CreateArtifact", "UploadArtifact", "FinalizeArtifact", "ListArtifacts", "GetSignedArtifactURL"},
"the artifact service was not reached through the cache server")
got, err := os.ReadFile(filepath.Join(env.workspace, "downloaded", "artifact.txt"))
require.NoError(t, err)
assert.Equal(t, uploaded, got)
}
// The setup actions carry the same toolkit and reach the same service, from a key of their own
// making. setup-node is the cheapest of them to run: given a lockfile and no version to install,
// it does the cache lookup and nothing else.
func TestSetupActionFindsTheCacheService(t *testing.T) {
requireHostTools(t, "node", "npm")
setup := patchedAction(t, "actions/setup-node", "v7.0.0", "dist/setup/index.js")
handler, err := artifactcache.StartHandler(filepath.Join(t.TempDir(), "cache"), "127.0.0.1", 0, "", nil)
require.NoError(t, err)
t.Cleanup(func() { _ = handler.Close() })
const token = "setup-runtime-token"
defer handler.RegisterJob(token, artifactcache.JobCredential{Repo: "testuser/testrepo"})()
env := jobEnv{
workspace: tempDirPath(t),
runnerTemp: tempDirPath(t),
cacheURL: handler.ExternalURL(),
resultsURL: handler.ExternalURL(),
token: token,
}
require.NoError(t, os.WriteFile(filepath.Join(env.workspace, "package-lock.json"),
[]byte(`{"lockfileVersion":3}`), 0o600))
out := runActionEntrypoint(t, setup, env, map[string]string{"cache": "npm"})
require.Contains(t, out, "Cache service version: v2")
require.Contains(t, out, "npm cache is not found", "the lookup did not reach the cache server")
}

View File

@@ -0,0 +1,327 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package runner
import (
"context"
"errors"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"gitea.com/gitea/runner/act/model"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// The three shapes real bundlers emit, reduced to the bytes that matter: the version gate, and
// the URL getter that follows it. tsc keeps the names, webpack prefixes them, esbuild mangles
// them, writes ternaries in place of the switch, and records the real name in the export
// assignment. Each carries both reads of the results URL, as the real getter does.
const (
urlTSC = `function getCacheServiceURL() {` + "\n" + ` switch (getCacheServiceVersion()) {` + "\n" + ` case 'v1':` + "\n" + ` return (process.env['ACTIONS_CACHE_URL'] || process.env['ACTIONS_RESULTS_URL'] || '');` + "\n" + ` case 'v2':` + "\n" + ` return process.env['ACTIONS_RESULTS_URL'] || '';` + "\n" + ` }` + "\n" + `}`
urlEsbuild = `function YK(){let e=XK();return e==="v1"?process.env.ACTIONS_CACHE_URL||process.env.ACTIONS_RESULTS_URL||"":e==="v2"?process.env.ACTIONS_RESULTS_URL||"":""}`
isGhesTSC = `function isGhes(){const h=new URL(process.env['GITHUB_SERVER_URL']||'https://github.com').hostname.toUpperCase();return h!=='GITHUB.COM'&&!h.endsWith('.GHE.COM')&&!h.endsWith('.LOCALHOST')}`
gateTSC = isGhesTSC + "\n" + `function getCacheServiceVersion() {` + "\n" + ` if (isGhes())` + "\n" + ` return 'v1';` + "\n" + ` return process.env['ACTIONS_CACHE_SERVICE_V2'] ? 'v2' : 'v1';` + "\n" + `}` + "\n" + urlTSC
gateWebpack = `function config_isGhes(){const h=new URL(process.env['GITHUB_SERVER_URL']||'https://github.com').hostname.toUpperCase();return h!=='GITHUB.COM'&&!h.endsWith('.GHE.COM')&&!h.endsWith('.LOCALHOST')}` + "\n" + `function config_getCacheServiceVersion() {` + "\n" + ` if (config_isGhes())` + "\n" + ` return 'v1';` + "\n" + ` return process.env['ACTIONS_CACHE_SERVICE_V2'] ? 'v2' : 'v1';` + "\n" + `}` + "\n" + urlTSC
gateEsbuild = `vu.isGhes=$K;vu.getCacheServiceVersion=XK;function $K(){let e=new URL(process.env.GITHUB_SERVER_URL||"https://github.com").hostname.toUpperCase(),r=e==="GITHUB.COM",n=e.endsWith(".GHE.COM"),i=e.endsWith(".LOCALHOST");return!r&&!n&&!i}function XK(){return $K()?"v1":process.env.ACTIONS_CACHE_SERVICE_V2?"v2":"v1"}` + urlEsbuild
)
func TestPatchedBundle(t *testing.T) {
for _, tc := range []struct {
name, body string
wantPatched bool
}{
{"tsc keeps the names", gateTSC, true},
{"webpack prefixes them", gateWebpack, true},
{"esbuild mangles and minifies them", gateEsbuild, true},
// A bundler picks its own quoting; gateTSC is single-quoted already.
{"double-quoted", requoted(`"`), true},
{"backtick-quoted", requoted("`"), true},
// sccache-action sets the variable itself; there is no gate to open.
{"mentions the variable without the gate", `core.exportVariable("ACTIONS_CACHE_SERVICE_V2","on")`, false},
// Both edits or neither: a gate patched without the URL would send the client to a
// results URL that serves no cache service.
{"gate without a recognisable url getter", strings.TrimSuffix(gateTSC, "\n"+urlTSC), false},
// And the other way round: an action that reads both variables but has no gate to open.
{"url getter without a gate", urlTSC, false},
} {
t.Run(tc.name, func(t *testing.T) {
out, patched := patchedBundle([]byte(tc.body))
assert.Equal(t, tc.wantPatched, patched)
if !tc.wantPatched {
assert.Equal(t, tc.body, string(out), "an unpatched bundle must come back byte for byte")
return
}
assert.True(t, gateOpened(string(out)))
// The other two hostname tests are left alone, so a host that really is GitHub or
// GHES is still recognised as such.
assert.NotContains(t, string(out), ".LOCALHOST", "the localhost test is the one that opens")
assert.Contains(t, string(out), ".GHE.COM")
// Every read of the results URL now prefers the cache URL, and none was lost: the
// results URL stays the fallback, so a runner not serving the cache still works.
assert.Equal(t, strings.Count(tc.body, "ACTIONS_RESULTS_URL"), strings.Count(string(out), "ACTIONS_RESULTS_URL"))
assert.Equal(t, strings.Count(tc.body, "ACTIONS_RESULTS_URL"),
strings.Count(string(out), "(process.env.ACTIONS_CACHE_URL||process.env"))
})
}
}
// undici, bundled into every one of these actions, decides whether to trust a URL with a
// lowercase test that reads almost the same. Opening it would tell the HTTP client that every URL
// is trustworthy, so the uppercase the toolkit produces is what separates them.
func TestPatchedBundleLeavesTrustworthyURLCheckAlone(t *testing.T) {
const undici = `if(n.hostname==="localhost"||n.hostname.includes("localhost.")||n.hostname.endsWith(".localhost")){return true}`
out, patched := patchedBundle([]byte(undici + gateTSC))
require.True(t, patched)
assert.Contains(t, string(out), undici, "the trustworthy-URL check must survive byte for byte")
assert.True(t, gateOpened(string(out)))
}
// The artifact toolkit puts the same gate in front of a plain refusal, with no URL to move, so
// opening it is what lets the stock upload-artifact work against Gitea instead of aborting.
func TestPatchedBundleOpensTheArtifactRefusal(t *testing.T) {
const artifact = isGhesTSC + "\n" + `uploadArtifact(){if(isGhes()){throw new GHESNotSupportedError()}}`
out, patched := patchedBundle([]byte(artifact))
assert.True(t, patched)
assert.True(t, gateOpened(string(out)))
assert.Contains(t, string(out), "GHESNotSupportedError", "the refusal itself is left in place, it just stops firing")
// A bundle using the gate for something this runner has not accounted for is not touched.
unknown := strings.Replace(artifact, "GHESNotSupportedError", "SomeOtherError", 1)
out, patched = patchedBundle([]byte(unknown))
assert.False(t, patched)
assert.Equal(t, unknown, string(out))
}
// requoted respells gateTSC's string literals with another quote character.
func requoted(quote string) string {
gate := strings.ReplaceAll(gateTSC, `'.LOCALHOST'`, quote+".LOCALHOST"+quote)
gate = strings.ReplaceAll(gate, `['ACTIONS_RESULTS_URL']`, "["+quote+"ACTIONS_RESULTS_URL"+quote+"]")
return strings.ReplaceAll(gate, `['ACTIONS_CACHE_URL']`, "["+quote+"ACTIONS_CACHE_URL"+quote+"]")
}
// gateOpened reports whether the hostname test was emptied, in whatever quoting the bundle used.
func gateOpened(body string) bool {
return strings.Contains(body, "endsWith(") && !strings.Contains(body, ".LOCALHOST")
}
// The patched bundle must still be JavaScript, and must resolve the way the runner needs: v2 for
// an ordinary Gitea host, the cache server for the service URL, and the results URL when there is
// no cache server. Unpatched, the same bundle must still choose v1, or the patch proves nothing.
func TestPatchedBundleBehavesInNode(t *testing.T) {
requireHostTools(t, "node")
eval := func(t *testing.T, bundle, prelude, cacheURL string) string {
t.Helper()
script := prelude + bundle + "\nprocess.stdout.write(getCacheServiceVersion()+' '+getCacheServiceURL())"
cmd := exec.CommandContext(t.Context(), "node", "-e", script)
cmd.Env = append(os.Environ(),
"ACTIONS_CACHE_SERVICE_V2=true",
"ACTIONS_CACHE_URL="+cacheURL,
"ACTIONS_RESULTS_URL=https://gitea.example",
"GITHUB_SERVER_URL=https://gitea.example",
)
out, err := cmd.CombinedOutput()
require.NoError(t, err, "%s", out)
return string(out)
}
for _, tc := range []struct{ name, bundle, prelude string }{
{"tsc", gateTSC, ""},
{"webpack", gateWebpack, "const getCacheServiceVersion=()=>config_getCacheServiceVersion();"},
{"esbuild", gateEsbuild, "var vu={};const getCacheServiceVersion=()=>XK(),getCacheServiceURL=()=>YK();"},
} {
t.Run(tc.name, func(t *testing.T) {
// Unpatched, a Gitea host is taken for GHES: v1, whose branch already reads the
// cache URL. The patch has to move the version without moving that.
assert.Equal(t, "v1 http://cache:8088/", eval(t, tc.bundle, tc.prelude, "http://cache:8088/"))
patched, ok := patchedBundle([]byte(tc.bundle))
require.True(t, ok)
assert.Equal(t, "v2 http://cache:8088/", eval(t, string(patched), tc.prelude, "http://cache:8088/"))
assert.Equal(t, "v2 https://gitea.example", eval(t, string(patched), tc.prelude, ""),
"with no cache server the results URL is still the fallback")
})
}
}
// A bundler that embeds module sources as strings, such as webpack with devtool: eval, carries
// the gate inside a double-quoted literal. Rewriting the call rather than emptying its argument
// would end that string early and leave the bundle unparseable.
func TestPatchedBundleSurvivesInsideAStringLiteral(t *testing.T) {
requireHostTools(t, "node")
escaped := strings.ReplaceAll(gateTSC, `"`, `\"`)
embedded := `eval("` + strings.ReplaceAll(escaped, "\n", `\n`) + `");`
out, patched := patchedBundle([]byte(embedded))
require.True(t, patched)
file := filepath.Join(t.TempDir(), "bundle.js")
require.NoError(t, os.WriteFile(file, out, 0o600))
checked, err := exec.CommandContext(t.Context(), "node", "--check", file).CombinedOutput()
require.NoError(t, err, "%s", checked)
}
func TestPatchBundleKeepsTheOriginal(t *testing.T) {
dir, script := bundleFile(t, gateTSC)
original := originalFor(dir, script)
require.NoError(t, patchBundle(script, original))
patched, err := os.ReadFile(script)
require.NoError(t, err)
assert.True(t, gateOpened(string(patched)))
kept, err := os.ReadFile(original)
require.NoError(t, err)
assert.Equal(t, gateTSC, string(kept), "the untouched bundle is kept outside the action tree")
assert.NotContains(t, original, dir+string(filepath.Separator), "originals must not ship into job containers")
// Patching again must not stack, and must not overwrite the kept original.
require.NoError(t, patchBundle(script, original))
again, err := os.ReadFile(script)
require.NoError(t, err)
assert.Equal(t, string(patched), string(again))
kept, err = os.ReadFile(original)
require.NoError(t, err)
assert.Equal(t, gateTSC, string(kept))
}
// A bundle with nothing to patch is left exactly as it was, with no original kept beside it.
func TestPatchBundleLeavesOtherActionsAlone(t *testing.T) {
dir, script := bundleFile(t, `console.log("checkout")`)
original := originalFor(dir, script)
require.NoError(t, patchBundle(script, original))
body, err := os.ReadFile(script)
require.NoError(t, err)
assert.Equal(t, `console.log("checkout")`, string(body))
_, err = os.Stat(original)
assert.True(t, os.IsNotExist(err), "no original is kept for a bundle that was not patched")
}
// bundleFile writes one entrypoint into a fresh action directory.
func bundleFile(t *testing.T, body string) (dir, script string) {
t.Helper()
dir = t.TempDir()
script = filepath.Join(dir, "index.js")
require.NoError(t, os.WriteFile(script, []byte(body), 0o600))
return dir, script
}
func TestActionScriptPaths(t *testing.T) {
node := &model.Action{Runs: model.ActionRuns{Using: "node20", Main: "dist/restore/index.js", Post: "dist/save/index.js"}}
assert.Equal(t, []string{"/a/dist/restore/index.js", "/a/dist/save/index.js"}, actionScriptPaths("/a", node))
// Only a node action has a bundle to patch.
assert.Nil(t, actionScriptPaths("/a", &model.Action{Runs: model.ActionRuns{Using: "docker", Image: "alpine"}}))
assert.Nil(t, actionScriptPaths("/a", nil))
}
// A step that fails with a patched bundle gets the untouched bundle back, and the action is not
// patched again, so later jobs run it exactly as its author shipped it.
func TestRevertToolkit(t *testing.T) {
dir, script := bundleFile(t, gateTSC)
scripts := []string{script}
patchToolkit(t.Context(), dir, scripts)
body, err := os.ReadFile(script)
require.NoError(t, err)
require.True(t, gateOpened(string(body)), "precondition: the bundle is patched")
revertToolkit(t.Context(), dir, scripts)
body, err = os.ReadFile(script)
require.NoError(t, err)
assert.Equal(t, gateTSC, string(body), "the original bundle is back")
// The skip marker survives, so the action stays unpatched from now on.
patchToolkit(t.Context(), dir, scripts)
body, err = os.ReadFile(script)
require.NoError(t, err)
assert.Equal(t, gateTSC, string(body), "a reverted action stays unpatched")
}
// An action whose ref moves is checked out over the patched bundle. The kept original then
// belongs to the version before the move, and must not be restored over the new one.
func TestPatchBundleAfterTheActionMoved(t *testing.T) {
dir, script := bundleFile(t, gateTSC)
original := originalFor(dir, script)
scripts := []string{script}
patchToolkit(t.Context(), dir, scripts)
require.NoError(t, os.WriteFile(script, []byte(gateWebpack), 0o600)) // the new version lands
// Reverting must not roll the action back to the version the original came from.
revertToolkit(t.Context(), dir, scripts)
body, err := os.ReadFile(script)
require.NoError(t, err)
assert.Equal(t, gateWebpack, string(body))
// Nothing was reverted, so the action is not marked off either: the new version is patched
// in its own right, and keeps its own original.
require.NoFileExists(t, filepath.Join(sidecarDir(dir), skipMarker))
require.NoError(t, patchBundle(script, original))
body, err = os.ReadFile(script)
require.NoError(t, err)
assert.True(t, gateOpened(string(body)))
kept, err := os.ReadFile(original)
require.NoError(t, err)
assert.Equal(t, gateWebpack, string(kept))
}
// The wiring: a step patches its own bundles only when the runner serves the v2 API, and a step
// that fails gets them back. The action's path inside its repository is part of where they live.
func TestStepActionRemoteToolkitPatch(t *testing.T) {
newStep := func(t *testing.T, patch bool) (*stepActionRemote, string) {
t.Helper()
sar := &stepActionRemote{
Step: &model.Step{Uses: "owner/repo/sub@v1"},
remoteAction: &remoteAction{Org: "owner", Repo: "repo", Path: "sub", Ref: "v1"},
action: &model.Action{Runs: model.ActionRuns{Using: "node20", Main: "index.js"}},
RunContext: &RunContext{
Config: &Config{ActionCacheDir: t.TempDir(), PatchToolkit: patch},
},
}
script := filepath.Join(sar.actionDir(), "sub", "index.js")
require.NoError(t, os.MkdirAll(filepath.Dir(script), 0o755))
require.NoError(t, os.WriteFile(script, []byte(gateTSC), 0o600))
return sar, script
}
t.Run("left alone when the runner does not patch", func(t *testing.T) {
sar, script := newStep(t, false)
require.NoError(t, sar.patchActionToolkit(t.Context()))
body, err := os.ReadFile(script)
require.NoError(t, err)
assert.Equal(t, gateTSC, string(body))
})
t.Run("patched, and put back when the step fails", func(t *testing.T) {
sar, script := newStep(t, true)
require.NoError(t, sar.patchActionToolkit(t.Context()))
body, err := os.ReadFile(script)
require.NoError(t, err)
require.True(t, gateOpened(string(body)))
failed := errors.New("the step failed")
require.ErrorIs(t, sar.revertToolkitOnFailure(func(context.Context) error { return failed })(t.Context()), failed)
body, err = os.ReadFile(script)
require.NoError(t, err)
assert.Equal(t, gateTSC, string(body))
})
}

View File

@@ -16,7 +16,7 @@ the runner as a background service on a systemd host.
`.runner` file ends up in the working directory: `.runner` file ends up in the working directory:
```bash ```bash
sudo -u gitea-runner gitea-runner generate-config > /etc/gitea-runner/config.yaml sudo -u gitea-runner gitea-runner config generate > /etc/gitea-runner/config.yaml
cd /var/lib/gitea-runner cd /var/lib/gitea-runner
sudo -u gitea-runner gitea-runner register --config /etc/gitea-runner/config.yaml sudo -u gitea-runner gitea-runner register --config /etc/gitea-runner/config.yaml
``` ```

View File

@@ -52,7 +52,7 @@ export DOCKER_HOST=unix:///run/user/$(id -u)/docker.sock
- Generate a `gitea-runner` configuration file in the data directory. Edit the file to adjust for the system. - Generate a `gitea-runner` configuration file in the data directory. Edit the file to adjust for the system.
```bash ```bash
gitea-runner generate-config >/home/rootless/gitea-runner/config gitea-runner config generate >/home/rootless/gitea-runner/config
``` ```
- Create a new user-level`systemd` unit file as `/home/rootless/.config/systemd/user/gitea-runner.service` with the following contents: - Create a new user-level`systemd` unit file as `/home/rootless/.config/systemd/user/gitea-runner.service` with the following contents:

28
go.mod
View File

@@ -6,29 +6,27 @@ require (
connectrpc.com/connect v1.20.0 connectrpc.com/connect v1.20.0
dario.cat/mergo v1.0.2 dario.cat/mergo v1.0.2
gitea.dev/actions-proto-go v0.6.0 gitea.dev/actions-proto-go v0.6.0
github.com/Masterminds/semver v1.5.0
github.com/avast/retry-go/v5 v5.0.0 github.com/avast/retry-go/v5 v5.0.0
github.com/containerd/errdefs v1.0.0 github.com/containerd/errdefs v1.0.0
github.com/creack/pty v1.1.24 github.com/creack/pty v1.1.24
github.com/distribution/reference v0.6.0 github.com/distribution/reference v0.6.0
github.com/docker/cli v29.6.2+incompatible github.com/docker/cli v29.6.2+incompatible
github.com/docker/go-connections v0.7.0 github.com/docker/go-connections v0.8.1
github.com/go-git/go-billy/v5 v5.9.0 github.com/go-git/go-billy/v5 v5.9.1
github.com/go-git/go-git/v5 v5.19.1 github.com/go-git/go-git/v5 v5.19.1
github.com/gobwas/glob v0.2.3 github.com/gobwas/glob v0.2.3
github.com/google/go-cmp v0.7.0 github.com/google/go-cmp v0.7.0
github.com/joho/godotenv v1.5.1 github.com/joho/godotenv v1.5.1
github.com/julienschmidt/httprouter v1.3.0 github.com/julienschmidt/httprouter v1.3.0
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
github.com/mattn/go-isatty v0.0.23 github.com/mattn/go-isatty v0.0.24
github.com/moby/go-archive v0.2.0 github.com/moby/go-archive v0.2.1
github.com/moby/moby/api v1.55.0 github.com/moby/moby/api v1.55.0
github.com/moby/moby/client v0.5.0 github.com/moby/moby/client v0.5.1
github.com/moby/patternmatcher v0.6.1 github.com/moby/patternmatcher v0.6.1
github.com/opencontainers/image-spec v1.1.1 github.com/opencontainers/image-spec v1.1.1
github.com/opencontainers/selinux v1.15.1 github.com/opencontainers/selinux v1.15.1
github.com/pkg/errors v0.9.1 github.com/prometheus/client_golang v1.24.1
github.com/prometheus/client_golang v1.24.0
github.com/prometheus/client_model v0.6.2 github.com/prometheus/client_model v0.6.2
github.com/rhysd/actionlint v1.7.12 github.com/rhysd/actionlint v1.7.12
github.com/sirupsen/logrus v1.9.4 github.com/sirupsen/logrus v1.9.4
@@ -38,7 +36,8 @@ require (
github.com/timshannon/bolthold v0.0.0-20240314194003-30aac6950928 github.com/timshannon/bolthold v0.0.0-20240314194003-30aac6950928
go.etcd.io/bbolt v1.5.0 go.etcd.io/bbolt v1.5.0
go.yaml.in/yaml/v4 v4.0.0-rc.3 go.yaml.in/yaml/v4 v4.0.0-rc.3
golang.org/x/net v0.56.0 golang.org/x/net v0.57.0
golang.org/x/sync v0.22.0
golang.org/x/sys v0.47.0 golang.org/x/sys v0.47.0
golang.org/x/term v0.45.0 golang.org/x/term v0.45.0
golang.org/x/text v0.40.0 golang.org/x/text v0.40.0
@@ -74,20 +73,20 @@ require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/kevinburke/ssh_config v1.6.0 // indirect github.com/kevinburke/ssh_config v1.6.0 // indirect
github.com/klauspost/compress v1.19.0 // indirect github.com/klauspost/compress v1.19.1 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-runewidth v0.0.21 // indirect github.com/mattn/go-runewidth v0.0.21 // indirect
github.com/mattn/go-shellwords v1.0.12 // indirect github.com/mattn/go-shellwords v1.0.12 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect github.com/moby/sys/sequential v0.7.0 // indirect
github.com/moby/sys/user v0.4.0 // indirect github.com/moby/sys/user v0.4.1 // indirect
github.com/moby/sys/userns v0.1.0 // indirect github.com/moby/sys/userns v0.1.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/pjbgf/sha1cd v0.6.0 // indirect github.com/pjbgf/sha1cd v0.6.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/common v0.70.0 // indirect github.com/prometheus/common v0.70.1 // indirect
github.com/prometheus/procfs v0.21.1 // indirect github.com/prometheus/procfs v0.21.1 // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect github.com/robfig/cron/v3 v3.0.1 // indirect
github.com/sergi/go-diff v1.4.0 // indirect github.com/sergi/go-diff v1.4.0 // indirect
@@ -105,8 +104,7 @@ require (
go.opentelemetry.io/otel/sdk/metric v1.43.0 // indirect go.opentelemetry.io/otel/sdk/metric v1.43.0 // indirect
go.opentelemetry.io/otel/trace v1.44.0 // indirect go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.53.0 // indirect golang.org/x/crypto v0.54.0 // indirect
golang.org/x/sync v0.22.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect
) )

50
go.sum
View File

@@ -8,8 +8,6 @@ gitea.dev/actions-proto-go v0.6.0 h1:gjllYQ5vmwlkqOeofTQu5qKTZpmf7kWsafoHvoPCSzY
gitea.dev/actions-proto-go v0.6.0/go.mod h1:p4RX+D9oqiEEzzkPMXscw2CmaGuYFPWFc6xIOmDNDqs= gitea.dev/actions-proto-go v0.6.0/go.mod h1:p4RX+D9oqiEEzzkPMXscw2CmaGuYFPWFc6xIOmDNDqs=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 h1:He8afgbRMd7mFxO99hRNu+6tazq8nFF9lIwo9JFroBk= github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 h1:He8afgbRMd7mFxO99hRNu+6tazq8nFF9lIwo9JFroBk=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8= github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8=
github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww=
github.com/Masterminds/semver v1.5.0/go.mod h1:MB6lktGJrhw8PrUyiEoblNEGEQ+RzHPF078ddwwvV3Y=
github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY= github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
@@ -51,8 +49,8 @@ github.com/docker/cli v29.6.2+incompatible h1:/bjePvcbbFTnRrMfWJBY7AjfICdsiLVgHn
github.com/docker/cli v29.6.2+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= github.com/docker/cli v29.6.2+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/docker-credential-helpers v0.9.6 h1:cT2PbRPSlnMmNTfT2TDMXRyQ1KMWHG7xoTLBcn1ZNv0= github.com/docker/docker-credential-helpers v0.9.6 h1:cT2PbRPSlnMmNTfT2TDMXRyQ1KMWHG7xoTLBcn1ZNv0=
github.com/docker/docker-credential-helpers v0.9.6/go.mod h1:v1S+hepowrQXITkEfw6o4+BMbGot02wiKpzWhGUZK6c= github.com/docker/docker-credential-helpers v0.9.6/go.mod h1:v1S+hepowrQXITkEfw6o4+BMbGot02wiKpzWhGUZK6c=
github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf0mN39c= github.com/docker/go-connections v0.8.1 h1:JibmG5hULs5qXSr/cp/w3Pw5fZuStt4MOHMUExb29/M=
github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q= github.com/docker/go-connections v0.8.1/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/elazarl/goproxy v1.7.2 h1:Y2o6urb7Eule09PjlhQRGNsqRfPmYI3KKQLFpCAV3+o= github.com/elazarl/goproxy v1.7.2 h1:Y2o6urb7Eule09PjlhQRGNsqRfPmYI3KKQLFpCAV3+o=
@@ -67,8 +65,8 @@ github.com/gliderlabs/ssh v0.3.8 h1:a4YXD1V7xMF9g5nTkdfnja3Sxy1PVDCj1Zg4Wb8vY6c=
github.com/gliderlabs/ssh v0.3.8/go.mod h1:xYoytBv1sV0aL3CavoDuJIQNURXkkfPA/wxQ1pL1fAU= github.com/gliderlabs/ssh v0.3.8/go.mod h1:xYoytBv1sV0aL3CavoDuJIQNURXkkfPA/wxQ1pL1fAU=
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI= github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI=
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic= github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic=
github.com/go-git/go-billy/v5 v5.9.0 h1:jItGXszUDRtR/AlferWPTMN4j38BQ88XnXKbilmmBPA= github.com/go-git/go-billy/v5 v5.9.1 h1:8U73XiOTfINdItHVa6z4Gv7ToObcZ6grkqQbLryLCdA=
github.com/go-git/go-billy/v5 v5.9.0/go.mod h1:jCnQMLj9eUgGU7+ludSTYoZL/GGmii14RxKFj7ROgHw= github.com/go-git/go-billy/v5 v5.9.1/go.mod h1:ExsU+jcGwXTBOnyilvAnEM1wug1IxHr4yP2ZXsNRtV0=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399 h1:eMje31YglSBqCdIqdhKBW8lokaMrL3uTkpGYlE2OOT4= github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399 h1:eMje31YglSBqCdIqdhKBW8lokaMrL3uTkpGYlE2OOT4=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399/go.mod h1:1OCfN199q1Jm3HZlxleg+Dw/mwps2Wbk9frAWm+4FII= github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399/go.mod h1:1OCfN199q1Jm3HZlxleg+Dw/mwps2Wbk9frAWm+4FII=
github.com/go-git/go-git/v5 v5.19.1 h1:nX27AnaU43/K5bKktKwgBmR9lawoYVe1Ckg0rgzzN00= github.com/go-git/go-git/v5 v5.19.1 h1:nX27AnaU43/K5bKktKwgBmR9lawoYVe1Ckg0rgzzN00=
@@ -102,8 +100,8 @@ github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNU
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8= github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/kevinburke/ssh_config v1.6.0 h1:J1FBfmuVosPHf5GRdltRLhPJtJpTlMdKTBjRgTaQBFY= github.com/kevinburke/ssh_config v1.6.0 h1:J1FBfmuVosPHf5GRdltRLhPJtJpTlMdKTBjRgTaQBFY=
github.com/kevinburke/ssh_config v1.6.0/go.mod h1:q2RIzfka+BXARoNexmF9gkxEX7DmvbW9P4hIVx2Kg4M= github.com/kevinburke/ssh_config v1.6.0/go.mod h1:q2RIzfka+BXARoNexmF9gkxEX7DmvbW9P4hIVx2Kg4M=
github.com/klauspost/compress v1.19.0 h1:sXLILfc9jV2QYWkzFOPWStmcUVH2RHEB1JCdY2oVvCQ= github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
github.com/klauspost/compress v1.19.0/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ= github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y= github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
@@ -117,26 +115,26 @@ github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE= github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8= github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.23 h1:cYwCQTQf3HB6xUC+BtyCLZNr7IzbOmoZbmssVNzSyiQ= github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
github.com/mattn/go-isatty v0.0.23/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A= github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/mattn/go-runewidth v0.0.21 h1:jJKAZiQH+2mIinzCJIaIG9Be1+0NR+5sz/lYEEjdM8w= github.com/mattn/go-runewidth v0.0.21 h1:jJKAZiQH+2mIinzCJIaIG9Be1+0NR+5sz/lYEEjdM8w=
github.com/mattn/go-runewidth v0.0.21/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs= github.com/mattn/go-runewidth v0.0.21/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk= github.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk=
github.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y= github.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0= github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo= github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/go-archive v0.2.0 h1:zg5QDUM2mi0JIM9fdQZWC7U8+2ZfixfTYoHL7rWUcP8= github.com/moby/go-archive v0.2.1 h1:fAa0wUS/ikZKyx7o/1fhUYmhZ7RgpthdeoDhJvunTLc=
github.com/moby/go-archive v0.2.0/go.mod h1:mNeivT14o8xU+5q1YnNrkQVpK+dnNe/K6fHqnTg4qPU= github.com/moby/go-archive v0.2.1/go.mod h1:Npdv43fFqlhZW7Xo8fbm3ZMYFvAGNviUPqX21VERbcE=
github.com/moby/moby/api v1.55.0 h1:2/sexvQyqIWS8pRSCFddBfpW2qE7vR7FCL+vN8pxwMc= github.com/moby/moby/api v1.55.0 h1:2/sexvQyqIWS8pRSCFddBfpW2qE7vR7FCL+vN8pxwMc=
github.com/moby/moby/api v1.55.0/go.mod h1:+RQ6wluLwtYaTd1WnPLykIDPekkuyD/ROWQClE83pzs= github.com/moby/moby/api v1.55.0/go.mod h1:+RQ6wluLwtYaTd1WnPLykIDPekkuyD/ROWQClE83pzs=
github.com/moby/moby/client v0.5.0 h1:5XhyPk2fuOWf6RlSFa3MkIIgDZkF25xToXW8Q/BH7cc= github.com/moby/moby/client v0.5.1 h1:tYNaJno4c0HXz12y5BiqEDy0rVTYkWzI26lGvnTMiJw=
github.com/moby/moby/client v0.5.0/go.mod h1:rcVpF8ncl9vo5gaIBdol6CnbEtSj1uxMvEV/UrykF/s= github.com/moby/moby/client v0.5.1/go.mod h1:odLstlZ6uSnfvAgVxMpvgmb8SUdd+siH2T0GBuxVAlM=
github.com/moby/patternmatcher v0.6.1 h1:qlhtafmr6kgMIJjKJMDmMWq7WLkKIo23hsrpR3x084U= github.com/moby/patternmatcher v0.6.1 h1:qlhtafmr6kgMIJjKJMDmMWq7WLkKIo23hsrpR3x084U=
github.com/moby/patternmatcher v0.6.1/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc= github.com/moby/patternmatcher v0.6.1/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
github.com/moby/sys/sequential v0.6.0 h1:qrx7XFUd/5DxtqcoH1h438hF5TmOvzC/lspjy7zgvCU= github.com/moby/sys/sequential v0.7.0 h1:ASQNGNROJSuOO6LL6bPHbKvuZu6NU8P4ldPWk31zj/8=
github.com/moby/sys/sequential v0.6.0/go.mod h1:uyv8EUTrca5PnDsdMGXhZe6CCe8U/UiTWd+lL+7b/Ko= github.com/moby/sys/sequential v0.7.0/go.mod h1:NfSTAp6V3fw4tmkD62PEcOKeZKquXT8VKCkf7aVR79o=
github.com/moby/sys/user v0.4.0 h1:jhcMKit7SA80hivmFJcbB1vqmw//wU61Zdui2eQXuMs= github.com/moby/sys/user v0.4.1 h1:RgjRlaDKi/Xmyrz4t8lyzXT6v2ooFeO/7xtchmhVWE0=
github.com/moby/sys/user v0.4.0/go.mod h1:bG+tYYYJgaMtRKgEmuueC0hJEAZWwtIbZTB+85uoHjs= github.com/moby/sys/user v0.4.1/go.mod h1:E9QsW5WRe1kUAf7kW8hXKwu1uhsZEAdPLYHYSDudF4Y=
github.com/moby/sys/userns v0.1.0 h1:tVLXkFOxVu9A64/yh59slHVv9ahO9UIev4JZusOLG/g= github.com/moby/sys/userns v0.1.0 h1:tVLXkFOxVu9A64/yh59slHVv9ahO9UIev4JZusOLG/g=
github.com/moby/sys/userns v0.1.0/go.mod h1:IHUYgu/kao6N8YZlp9Cf444ySSvCmDlmzUcYfDHOl28= github.com/moby/sys/userns v0.1.0/go.mod h1:IHUYgu/kao6N8YZlp9Cf444ySSvCmDlmzUcYfDHOl28=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
@@ -155,12 +153,12 @@ github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.24.0 h1:5XStIklKuAtJSNpdD3s8XJj/Yv78IQmE1kbNk87JrAI= github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU=
github.com/prometheus/client_golang v1.24.0/go.mod h1:QcsNdotprC2nS4BTM2ucbcqxd2CeXTEa9jW7zHO9iDE= github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
github.com/prometheus/common v0.70.0 h1:bcpru3tWPVnxGnETLgOV5jbp/JRXgYEyv65CuBLAMMI= github.com/prometheus/common v0.70.1 h1:1HvjP4D5oL3t8RsPlwxA9onvvStjtIHYE5XuuwOi/PY=
github.com/prometheus/common v0.70.0/go.mod h1:S/SFasQmgGiYH6C81LKCtYa8QACgthGg5zxL2udV7SY= github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI= github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY= github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
github.com/rhysd/actionlint v1.7.12 h1:vQ4GeJN86C0QH+gTUQcs8McmK62OLT3kmakPMtEWYnY= github.com/rhysd/actionlint v1.7.12 h1:vQ4GeJN86C0QH+gTUQcs8McmK62OLT3kmakPMtEWYnY=
@@ -233,13 +231,13 @@ go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
go.yaml.in/yaml/v4 v4.0.0-rc.3 h1:3h1fjsh1CTAPjW7q/EMe+C8shx5d8ctzZTrLcs/j8Go= go.yaml.in/yaml/v4 v4.0.0-rc.3 h1:3h1fjsh1CTAPjW7q/EMe+C8shx5d8ctzZTrLcs/j8Go=
go.yaml.in/yaml/v4 v4.0.0-rc.3/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0= go.yaml.in/yaml/v4 v4.0.0-rc.3/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto= golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio= golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM= golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80= golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o= golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE=
golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec= golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek= golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=

View File

@@ -5,10 +5,8 @@ package cmd
import ( import (
"context" "context"
"fmt"
"os" "os"
"gitea.com/gitea/runner/internal/pkg/config"
"gitea.com/gitea/runner/internal/pkg/ver" "gitea.com/gitea/runner/internal/pkg/ver"
"github.com/spf13/cobra" "github.com/spf13/cobra"
@@ -23,7 +21,7 @@ func Execute(ctx context.Context) {
SilenceUsage: true, SilenceUsage: true,
} }
configFile := "" configFile := ""
rootCmd.PersistentFlags().StringVarP(&configFile, "config", "c", "", "Config file path") rootCmd.PersistentFlags().StringVarP(&configFile, "config", "c", "", "Config file path. `config` subcommands fall back to config.yaml in the working directory or next to the executable")
// ./gitea-runner register // ./gitea-runner register
var regArgs registerArgs var regArgs registerArgs
@@ -61,14 +59,12 @@ func Execute(ctx context.Context) {
rootCmd.AddCommand(loadBugReportCmd()) rootCmd.AddCommand(loadBugReportCmd())
// ./gitea-runner config // ./gitea-runner config
rootCmd.AddCommand(&cobra.Command{ rootCmd.AddCommand(loadConfigCmd(&configFile))
Use: "generate-config",
Short: "Generate an example config file", // ./gitea-runner generate-config
Args: cobra.MaximumNArgs(0), generateConfigCmd := loadGenerateConfigCmd("generate-config")
Run: func(_ *cobra.Command, _ []string) { generateConfigCmd.Deprecated = "use `config generate` instead."
fmt.Printf("%s", config.Example) rootCmd.AddCommand(generateConfigCmd)
},
})
// ./gitea-runner cache-server // ./gitea-runner cache-server
var cacheArgs cacheServerArgs var cacheArgs cacheServerArgs

116
internal/app/cmd/config.go Normal file
View File

@@ -0,0 +1,116 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package cmd
import (
"fmt"
"os"
"path/filepath"
"slices"
"strings"
"gitea.com/gitea/runner/internal/pkg/config"
"github.com/spf13/cobra"
)
func loadConfigCmd(configFile *string) *cobra.Command {
configCmd := &cobra.Command{
Use: "config",
Short: "Generate, read and edit config files",
Args: cobra.MaximumNArgs(0),
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
}
configCmd.AddCommand(loadGenerateConfigCmd("generate"))
configCmd.AddCommand(&cobra.Command{
Use: "get <key>",
Short: "Print the value of a config key",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
file, err := resolveConfigFile(cmd, configFile)
if err != nil {
return err
}
value, err := config.GetValue(file, args[0])
if err != nil {
return err
}
fmt.Fprintln(cmd.OutOrStdout(), value)
return nil
},
})
for _, sub := range []struct {
use string
short string
edit func(file, key string, values ...string) error
}{
{"set <key> <value>...", "Set the value of a config key", config.SetValue},
{"add <key> <value>...", "Append values to a list config key", config.AddValue},
{"remove <key> <value>...", "Remove values from a list config key", config.RemoveValue},
} {
valueCmd := &cobra.Command{
Use: sub.use,
Short: sub.short,
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
file, err := resolveConfigFile(cmd, configFile)
if err != nil {
return err
}
return sub.edit(file, args[0], args[1:]...)
},
}
valueCmd.Flags().SetInterspersed(false) // so a value such as `--cpus 2` is not parsed as a flag
configCmd.AddCommand(valueCmd)
}
return configCmd
}
func loadGenerateConfigCmd(use string) *cobra.Command {
return &cobra.Command{
Use: use,
Short: "Generate an example config file",
Args: cobra.MaximumNArgs(0),
Run: func(cmd *cobra.Command, _ []string) {
fmt.Fprintf(cmd.OutOrStdout(), "%s", config.Example)
},
}
}
var defaultConfigFileNames = []string{"config.yaml", "config.yml"}
func resolveConfigFile(cmd *cobra.Command, configFile *string) (string, error) {
if *configFile != "" {
return *configFile, nil
}
var dirs []string
if wd, err := os.Getwd(); err == nil {
dirs = append(dirs, wd)
}
if exe, err := os.Executable(); err == nil {
if dir := filepath.Dir(exe); !slices.Contains(dirs, dir) {
dirs = append(dirs, dir)
}
}
for _, dir := range dirs {
for _, name := range defaultConfigFileNames {
candidate := filepath.Join(dir, name)
if stat, err := os.Stat(candidate); err == nil && !stat.IsDir() {
fmt.Fprintf(cmd.ErrOrStderr(), "using config file %q\n", candidate)
return candidate, nil
}
}
}
return "", fmt.Errorf("no %s found in %s, pass one with --config",
strings.Join(defaultConfigFileNames, " or "), strings.Join(dirs, " or "))
}

View File

@@ -0,0 +1,75 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package cmd
import (
"bytes"
"os"
"path/filepath"
"testing"
"gitea.com/gitea/runner/internal/pkg/config"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func runConfigCmd(t *testing.T, configFile string, args ...string) (string, string, error) {
t.Helper()
out, errOut := &bytes.Buffer{}, &bytes.Buffer{}
cmd := loadConfigCmd(&configFile)
cmd.SetOut(out)
cmd.SetErr(errOut)
cmd.SetArgs(args)
err := cmd.Execute()
return out.String(), errOut.String(), err
}
func TestConfigCmdGeneratePrintsTheExample(t *testing.T) {
out, _, err := runConfigCmd(t, "", "generate")
require.NoError(t, err)
assert.Equal(t, string(config.Example), out)
}
// The subcommands only wire arguments through, so one pass over all of them is enough.
func TestConfigCmdEditsTheFile(t *testing.T) {
file := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(file, []byte("runner:\n labels:\n - self-hosted\n"), 0o600))
_, _, err := runConfigCmd(t, file, "set", "container.options", "--cpus 2")
require.NoError(t, err)
_, _, err = runConfigCmd(t, file, "add", "runner.labels", "ubuntu:docker://node:22")
require.NoError(t, err)
_, _, err = runConfigCmd(t, file, "remove", "runner.labels", "self-hosted")
require.NoError(t, err)
out, _, err := runConfigCmd(t, file, "get", "runner.labels")
require.NoError(t, err)
assert.Equal(t, "ubuntu:docker://node:22\n", out)
out, _, err = runConfigCmd(t, file, "get", "container.options")
require.NoError(t, err)
assert.Equal(t, "--cpus 2\n", out)
}
func TestConfigCmdResolvesTheConfigFile(t *testing.T) {
t.Run("falls back to the working directory", func(t *testing.T) {
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, "config.yaml"), []byte("runner:\n capacity: 2\n"), 0o600))
t.Chdir(dir)
out, errOut, err := runConfigCmd(t, "", "get", "runner.capacity")
require.NoError(t, err)
assert.Equal(t, "2\n", out)
assert.Contains(t, errOut, "using config file")
})
t.Run("reports that none was found", func(t *testing.T) {
t.Chdir(t.TempDir())
_, _, err := runConfigCmd(t, "", "set", "runner.capacity", "4")
require.Error(t, err)
assert.Contains(t, err.Error(), "--config")
})
}

View File

@@ -22,6 +22,7 @@ import (
"gitea.com/gitea/runner/internal/pkg/config" "gitea.com/gitea/runner/internal/pkg/config"
"gitea.com/gitea/runner/internal/pkg/envcheck" "gitea.com/gitea/runner/internal/pkg/envcheck"
"gitea.com/gitea/runner/internal/pkg/labels" "gitea.com/gitea/runner/internal/pkg/labels"
"gitea.com/gitea/runner/internal/pkg/lock"
"gitea.com/gitea/runner/internal/pkg/metrics" "gitea.com/gitea/runner/internal/pkg/metrics"
"gitea.com/gitea/runner/internal/pkg/ver" "gitea.com/gitea/runner/internal/pkg/ver"
@@ -49,6 +50,25 @@ func runDaemon(ctx context.Context, daemArgs *daemonArgs, configFile *string) fu
return fmt.Errorf("failed to load registration file: %w", err) return fmt.Errorf("failed to load registration file: %w", err)
} }
// Guard against a second runner process sharing this runner file: two
// processes with the same identity are indistinguishable to Gitea and
// end up cancelling each other's jobs.
releaseLock, err := lock.TryLock(cfg.Runner.File)
if errors.Is(err, lock.ErrLocked) {
log.Errorf("another gitea-runner process is already using %q; each runner process needs its own runner file (runner.file)", cfg.Runner.File)
return err
} else if err != nil {
// Best-effort guard: if the lock file can't be created (e.g. a
// read-only runner-file mount), warn and start anyway rather than
// refusing to run.
log.Warnf("could not lock runner file %q, continuing without the single-process guard: %v", cfg.Runner.File, err)
} else {
// Held until shutdown finishes: the draining runner still owns this
// identity on the server, so releasing early would let a restart
// reintroduce the duplicate-identity job cancellations.
defer func() { _ = releaseLock() }()
}
lbls := resolveLabels(daemArgs.Labels, cfg.Runner.Labels, reg.Labels) lbls := resolveLabels(daemArgs.Labels, cfg.Runner.Labels, reg.Labels)
ls := labels.Labels{} ls := labels.Labels{}
@@ -141,6 +161,11 @@ func runDaemon(ctx context.Context, daemArgs *daemonArgs, configFile *string) fu
) )
runner := run.NewRunner(cfg, reg, cli) runner := run.NewRunner(cfg, reg, cli)
defer func() {
if err := runner.Close(); err != nil {
log.Warnf("runner %s: cache server shutdown: %v", reg.Name, err)
}
}()
// declare the labels of the runner before fetching tasks // declare the labels of the runner before fetching tasks
resp, err := runner.Declare(ctx, ls.Names()) resp, err := runner.Declare(ctx, ls.Names())

View File

@@ -132,6 +132,8 @@ func (i *executeArgs) LoadEnvs() map[string]string {
_ = readEnvs(i.Envfile(), envs) _ = readEnvs(i.Envfile(), envs)
envs["ACTIONS_CACHE_URL"] = i.cacheHandler.ExternalURL() + "/" envs["ACTIONS_CACHE_URL"] = i.cacheHandler.ExternalURL() + "/"
// The same server answers the cache service v2 API, so let the actions reach it.
envs[runner.CacheServiceV2Env] = "true"
return envs return envs
} }
@@ -414,7 +416,7 @@ func runExec(ctx context.Context, execArgs *executeArgs) func(cmd *cobra.Command
env[actionsRuntimeTokenEnvName] = actionsRuntimeToken env[actionsRuntimeTokenEnvName] = actionsRuntimeToken
os.Setenv(actionsRuntimeTokenEnvName, actionsRuntimeToken) os.Setenv(actionsRuntimeTokenEnvName, actionsRuntimeToken)
} }
handler.RegisterJob(actionsRuntimeToken, "__local/__exec") handler.RegisterJob(actionsRuntimeToken, artifactcache.JobCredential{Repo: "__local/__exec"})
// no service aliases: exec builds one config for the whole plan // no service aliases: exec builds one config for the whole plan
run.BypassProxyForDockerHost(os.Getenv("DOCKER_HOST")) run.BypassProxyForDockerHost(os.Getenv("DOCKER_HOST"))
@@ -425,6 +427,7 @@ func runExec(ctx context.Context, execArgs *executeArgs) func(cmd *cobra.Command
config := &runner.Config{ config := &runner.Config{
Workdir: execArgs.Workdir(), Workdir: execArgs.Workdir(),
BindWorkdir: false, BindWorkdir: false,
PatchToolkit: true, // the cache server started above is what the patch points at
ReuseContainers: false, ReuseContainers: false,
ForcePull: execArgs.forcePull, ForcePull: execArgs.forcePull,
ForceRebuild: execArgs.forceRebuild, ForceRebuild: execArgs.forceRebuild,

View File

@@ -18,6 +18,7 @@ import (
"gitea.com/gitea/runner/internal/pkg/client" "gitea.com/gitea/runner/internal/pkg/client"
"gitea.com/gitea/runner/internal/pkg/config" "gitea.com/gitea/runner/internal/pkg/config"
"gitea.com/gitea/runner/internal/pkg/labels" "gitea.com/gitea/runner/internal/pkg/labels"
"gitea.com/gitea/runner/internal/pkg/lock"
"gitea.com/gitea/runner/internal/pkg/ver" "gitea.com/gitea/runner/internal/pkg/ver"
"connectrpc.com/connect" "connectrpc.com/connect"
@@ -346,6 +347,19 @@ func registerNoInteractive(ctx context.Context, configFile string, regArgs *regi
} }
func doRegister(ctx context.Context, cfg *config.Config, inputs *registerInputs) error { func doRegister(ctx context.Context, cfg *config.Config, inputs *registerInputs) error {
// Refuse to rewrite the runner file while another process is using it.
releaseLock, err := lock.TryLock(cfg.Runner.File)
if errors.Is(err, lock.ErrLocked) {
return fmt.Errorf("another process is already using %q; stop it before re-registering", cfg.Runner.File)
} else if err != nil {
// Best-effort guard: if the lock file can't be created, warn and
// register anyway; writing the runner file will surface any real
// permission problem with a clearer error.
log.Warnf("could not lock runner file %q, continuing without the single-process guard: %v", cfg.Runner.File, err)
} else {
defer func() { _ = releaseLock() }()
}
// initial http client // initial http client
cli := client.New( cli := client.New(
inputs.InstanceAddr, inputs.InstanceAddr,

View File

@@ -54,6 +54,7 @@ func RunnerCapabilities() []string {
// Runner runs the pipeline. // Runner runs the pipeline.
type Runner struct { type Runner struct {
name string name string
uuid string
cfg *config.Config cfg *config.Config
@@ -88,7 +89,8 @@ func NewRunner(cfg *config.Config, reg *config.Registration, cli client.Client)
var cacheHandler *artifactcache.Handler var cacheHandler *artifactcache.Handler
if cfg.Cache.Enabled == nil || *cfg.Cache.Enabled { if cfg.Cache.Enabled == nil || *cfg.Cache.Enabled {
if cfg.Cache.ExternalServer != "" { if cfg.Cache.ExternalServer != "" {
envs["ACTIONS_CACHE_URL"] = cfg.Cache.ExternalServer // The v1 client appends its path to this without a separator, so the slash is required.
envs["ACTIONS_CACHE_URL"] = strings.TrimRight(cfg.Cache.ExternalServer, "/") + "/"
} else { } else {
warnIgnoredCacheSecret(cfg) warnIgnoredCacheSecret(cfg)
handler, err := artifactcache.StartHandler( handler, err := artifactcache.StartHandler(
@@ -119,6 +121,7 @@ func NewRunner(cfg *config.Config, reg *config.Registration, cli client.Client)
runner := &Runner{ runner := &Runner{
name: reg.Name, name: reg.Name,
uuid: reg.UUID,
cfg: cfg, cfg: cfg,
client: cli, client: cli,
labels: ls, labels: ls,
@@ -130,6 +133,14 @@ func NewRunner(cfg *config.Config, reg *config.Registration, cli client.Client)
return runner return runner
} }
// Close shuts down the cache server this runner exposes to job containers.
func (r *Runner) Close() error {
return r.cacheHandler.Close()
}
// removeOrphanNetworks is a variable so tests can substitute one that needs no Docker daemon.
var removeOrphanNetworks = container.RemoveOrphanNetworks
// OnIdle performs lightweight maintenance during polling idle windows. // OnIdle performs lightweight maintenance during polling idle windows.
// It runs synchronously on the poller goroutine; shouldRunIdleCleanup // It runs synchronously on the poller goroutine; shouldRunIdleCleanup
// throttles invocations to runner.idle_cleanup_interval so the impact on // throttles invocations to runner.idle_cleanup_interval so the impact on
@@ -151,6 +162,21 @@ func (r *Runner) OnIdle(ctx context.Context) {
if hostRoot := filepath.FromSlash(r.cfg.Host.WorkdirParent); hostRoot != "" { if hostRoot := filepath.FromSlash(r.cfg.Host.WorkdirParent); hostRoot != "" {
r.cleanupStaleDirs(ctx, hostRoot, isHostScratchDir) r.cleanupStaleDirs(ctx, hostRoot, isHostScratchDir)
} }
r.cleanupOrphanNetworks(ctx)
}
// cleanupOrphanNetworks reclaims the per-job networks of jobs this runner did not live to
// tear down. A labelled network with no containers on it is finished with, and as for the
// directories above, a task beginning during the pass is safe because the cutoff keeps a
// network it has created but not yet attached a container to out of scope.
func (r *Runner) cleanupOrphanNetworks(ctx context.Context) {
if r.uuid == "" || !r.labels.RequireDocker() && !r.cfg.Container.RequireDocker {
return
}
cutoff := r.now().Add(-r.cfg.Runner.WorkdirCleanupAge)
if err := removeOrphanNetworks(ctx, r.uuid, cutoff); err != nil {
log.Warnf("failed to clean up networks left behind by earlier jobs: %v", err)
}
} }
func (r *Runner) shouldRunIdleCleanup() bool { func (r *Runner) shouldRunIdleCleanup() bool {
@@ -409,7 +435,13 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
// lifetime. Only applies to the embedded cache server; when the operator // lifetime. Only applies to the embedded cache server; when the operator
// points the runner at an external cache via cfg.Cache.ExternalServer, it // points the runner at an external cache via cfg.Cache.ExternalServer, it
// is that server's responsibility to authenticate requests. // is that server's responsibility to authenticate requests.
defer r.registerCacheForTask(giteaRuntimeToken, preset.Repository, reporter)() revokeCache, resultsURL := r.registerCacheForTask(giteaRuntimeToken, preset.Repository, reporter)
defer revokeCache()
// A cache server that agreed to forward the artifact half is the whole results service, so
// the job is pointed at it and the v2 variable is finally true.
if resultsURL != "" {
envs["ACTIONS_RESULTS_URL"], envs[runner.CacheServiceV2Env] = resultsURL, "true"
}
eventJSON, err := json.Marshal(preset.Event) eventJSON, err := json.Marshal(preset.Event)
if err != nil { if err != nil {
@@ -450,6 +482,7 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
AllocatePTY: r.cfg.Runner.AllocatePTY, AllocatePTY: r.cfg.Runner.AllocatePTY,
ActionOfflineMode: r.cfg.Cache.OfflineMode, ActionOfflineMode: r.cfg.Cache.OfflineMode,
ActionCloneDepth: actionCloneDepth, ActionCloneDepth: actionCloneDepth,
PatchToolkit: r.patchToolkit(),
ReuseContainers: false, ReuseContainers: false,
ForcePull: r.cfg.Container.ForcePull, ForcePull: r.cfg.Container.ForcePull,
@@ -472,8 +505,12 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
ContainerNetworkCreateOptions: container.NewDockerNetworkCreateExecutorInput{ ContainerNetworkCreateOptions: container.NewDockerNetworkCreateExecutorInput{
EnableIPv4: r.cfg.Container.NetworkCreateOptions.EnableIPv4, EnableIPv4: r.cfg.Container.NetworkCreateOptions.EnableIPv4,
EnableIPv6: r.cfg.Container.NetworkCreateOptions.EnableIPv6, EnableIPv6: r.cfg.Container.NetworkCreateOptions.EnableIPv6,
// so a network this job leaks, if the runner dies before its teardown, can be
// told apart from one belonging to another runner on the same daemon
RunnerUUID: r.uuid,
}, },
ContainerOptions: r.cfg.Container.Options, ContainerOptions: r.cfg.Container.Options,
ServiceReadyTimeout: r.cfg.Container.ServiceReadyTimeout,
ContainerDaemonSocket: r.cfg.Container.DockerHost, ContainerDaemonSocket: r.cfg.Container.DockerHost,
Privileged: r.cfg.Container.Privileged, Privileged: r.cfg.Container.Privileged,
DefaultActionInstance: r.getDefaultActionsURL(task), DefaultActionInstance: r.getDefaultActionsURL(task),
@@ -519,6 +556,12 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
return execErr return execErr
} }
// patchToolkit reports whether act should edit the toolkit bundled into an action. It follows the
// cache URL, because that is what the edits point the client at; see act/runner/toolkit_patch.go.
func (r *Runner) patchToolkit() bool {
return r.envs["ACTIONS_CACHE_URL"] != "" && (r.cfg.Cache.V2 == nil || *r.cfg.Cache.V2)
}
// registerCacheForTask tells the cache server to accept requests authenticated // registerCacheForTask tells the cache server to accept requests authenticated
// with the given runtime token for the duration of this task. Returns a // with the given runtime token for the duration of this task. Returns a
// function the caller must invoke (typically via defer) to revoke the // function the caller must invoke (typically via defer) to revoke the
@@ -531,18 +574,25 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
// repo scoping over the network. // repo scoping over the network.
// //
// Safe with an empty token (older Gitea did not issue one). // Safe with an empty token (older Gitea did not issue one).
func (r *Runner) registerCacheForTask(token, repo string, reporter *report.Reporter) func() { // It also returns the URL to advertise as ACTIONS_RESULTS_URL, which is the cache server itself
// when it agreed to forward this instance's artifact service, and "" when it did not.
func (r *Runner) registerCacheForTask(token, repo string, reporter *report.Reporter) (func(), string) {
if token == "" { if token == "" {
return func() {} return func() {}, ""
}
cred := artifactcache.JobCredential{
Repo: repo,
Results: r.envs["ACTIONS_RESULTS_URL"], // the instance as the job would reach it
InsecureTLS: r.cfg.Runner.Insecure,
} }
if r.cacheHandler != nil { if r.cacheHandler != nil {
return r.cacheHandler.RegisterJob(token, repo) return r.cacheHandler.RegisterJob(token, cred), r.cacheHandler.ResultsURL(cred)
} }
if r.cfg.Cache.ExternalServer != "" && r.cfg.Cache.ExternalSecret != "" { if r.cfg.Cache.ExternalServer != "" && r.cfg.Cache.ExternalSecret != "" {
return r.registerExternalCacheJob(token, repo, reporter) return r.registerExternalCacheJob(token, cred, reporter)
} }
// No cache server to register against: caching is disabled, or the built-in server failed to start. // No cache server to register against: caching is disabled, or the built-in server failed to start.
return func() {} return func() {}, ""
} }
// registerExternalCacheJob POSTs to the remote cache-server's control-plane. // registerExternalCacheJob POSTs to the remote cache-server's control-plane.
@@ -550,47 +600,56 @@ func (r *Runner) registerCacheForTask(token, repo string, reporter *report.Repor
// 401 the job's requests — better than failing the whole task for a cache // 401 the job's requests — better than failing the whole task for a cache
// outage. The warning is mirrored to the job log so users can see why their // outage. The warning is mirrored to the job log so users can see why their
// cache calls 401, instead of having to read the runner daemon's stderr. // cache calls 401, instead of having to read the runner daemon's stderr.
func (r *Runner) registerExternalCacheJob(token, repo string, reporter *report.Reporter) func() { func (r *Runner) registerExternalCacheJob(token string, cred artifactcache.JobCredential, reporter *report.Reporter) (func(), string) {
base := strings.TrimRight(r.cfg.Cache.ExternalServer, "/") base := strings.TrimRight(r.cfg.Cache.ExternalServer, "/")
if err := postInternalCache(base+"/_internal/register", r.cfg.Cache.ExternalSecret, resultsURL := ""
map[string]string{"token": token, "repo": repo}); err != nil { if body, err := postInternalCache(base+"/_internal/register", r.cfg.Cache.ExternalSecret, map[string]any{
"token": token, "repo": cred.Repo, "results": cred.Results, "insecure_tls": cred.InsecureTLS,
}); err != nil {
log.Warnf("cache external_server register failed (%s): %v", base, err) log.Warnf("cache external_server register failed (%s): %v", base, err)
if reporter != nil { if reporter != nil {
reporter.Logf("::warning::cache external_server register failed (%s): %v — cache requests from this job will be unauthenticated and likely return 401", base, err) reporter.Logf("::warning::%s", runner.EscapeCommandData(fmt.Sprintf(
"cache external_server register failed (%s): %v — cache requests from this job will be unauthenticated and likely return 401", base, err)))
} }
} else {
resultsURL, _ = body["results_url"].(string) // absent from a server too old to forward
} }
return func() { return func() {
if err := postInternalCache(base+"/_internal/revoke", r.cfg.Cache.ExternalSecret, if _, err := postInternalCache(base+"/_internal/revoke", r.cfg.Cache.ExternalSecret,
map[string]string{"token": token}); err != nil { map[string]any{"token": token}); err != nil {
log.Warnf("cache external_server revoke failed (%s): %v", base, err) log.Warnf("cache external_server revoke failed (%s): %v", base, err)
if reporter != nil { if reporter != nil {
reporter.Logf("::warning::cache external_server revoke failed (%s): %v", base, err) reporter.Logf("::warning::%s", runner.EscapeCommandData(fmt.Sprintf(
} "cache external_server revoke failed (%s): %v", base, err)))
} }
} }
}, resultsURL
} }
func postInternalCache(url, secret string, body map[string]string) error { func postInternalCache(url, secret string, body map[string]any) (map[string]any, error) {
buf, err := json.Marshal(body) buf, err := json.Marshal(body)
if err != nil { if err != nil {
return err return nil, err
} }
req, err := http.NewRequest(http.MethodPost, url, bytes.NewReader(buf)) req, err := http.NewRequest(http.MethodPost, url, bytes.NewReader(buf))
if err != nil { if err != nil {
return err return nil, err
} }
req.Header.Set("Authorization", "Bearer "+secret) req.Header.Set("Authorization", "Bearer "+secret)
req.Header.Set("Content-Type", "application/json") req.Header.Set("Content-Type", "application/json")
client := &http.Client{Timeout: 5 * time.Second} client := &http.Client{Timeout: 5 * time.Second}
resp, err := client.Do(req) resp, err := client.Do(req)
if err != nil { if err != nil {
return err return nil, err
} }
defer resp.Body.Close() defer resp.Body.Close()
if resp.StatusCode/100 != 2 { if resp.StatusCode/100 != 2 {
return fmt.Errorf("status %d", resp.StatusCode) return nil, fmt.Errorf("status %d", resp.StatusCode)
} }
return nil answer := map[string]any{}
// A server too old to answer with a body is not an error, it simply tells us nothing.
_ = json.NewDecoder(resp.Body).Decode(&answer)
return answer, nil
} }
func (r *Runner) RunningCount() int64 { func (r *Runner) RunningCount() int64 {

View File

@@ -7,7 +7,9 @@ import (
"bytes" "bytes"
"encoding/json" "encoding/json"
"fmt" "fmt"
"io"
"net/http" "net/http"
"net/http/httptest"
"path/filepath" "path/filepath"
"strings" "strings"
"testing" "testing"
@@ -29,7 +31,7 @@ func TestRunner_registerCacheForTask(t *testing.T) {
r := &Runner{cfg: emptyCfg(), cacheHandler: handler} r := &Runner{cfg: emptyCfg(), cacheHandler: handler}
token := "run-token-123" token := "run-token-123"
unregister := r.registerCacheForTask(token, "owner/repo", nil) unregister, _ := r.registerCacheForTask(token, "owner/repo", nil)
base := handler.ExternalURL() + "/_apis/artifactcache" base := handler.ExternalURL() + "/_apis/artifactcache"
probe := func() int { probe := func() int {
@@ -53,7 +55,7 @@ func TestRunner_registerCacheForTask(t *testing.T) {
func TestRunner_registerCacheForTask_NoOps(t *testing.T) { func TestRunner_registerCacheForTask_NoOps(t *testing.T) {
t.Run("nil cacheHandler", func(t *testing.T) { t.Run("nil cacheHandler", func(t *testing.T) {
r := &Runner{cfg: emptyCfg()} r := &Runner{cfg: emptyCfg()}
unregister := r.registerCacheForTask("tok", "owner/repo", nil) unregister, _ := r.registerCacheForTask("tok", "owner/repo", nil)
require.NotNil(t, unregister) require.NotNil(t, unregister)
unregister() unregister()
}) })
@@ -65,7 +67,7 @@ func TestRunner_registerCacheForTask_NoOps(t *testing.T) {
defer handler.Close() defer handler.Close()
r := &Runner{cfg: emptyCfg(), cacheHandler: handler} r := &Runner{cfg: emptyCfg(), cacheHandler: handler}
unregister := r.registerCacheForTask("", "owner/repo", nil) unregister, _ := r.registerCacheForTask("", "owner/repo", nil)
require.NotNil(t, unregister) require.NotNil(t, unregister)
unregister() unregister()
}) })
@@ -81,7 +83,7 @@ func TestRunner_CacheFullFlow_MatchesToolkit(t *testing.T) {
r := &Runner{cfg: emptyCfg(), cacheHandler: handler} r := &Runner{cfg: emptyCfg(), cacheHandler: handler}
token := "full-flow-token" token := "full-flow-token"
unregister := r.registerCacheForTask(token, "owner/repo", nil) unregister, _ := r.registerCacheForTask(token, "owner/repo", nil)
defer unregister() defer unregister()
base := handler.ExternalURL() + "/_apis/artifactcache" base := handler.ExternalURL() + "/_apis/artifactcache"
@@ -154,18 +156,27 @@ func decodeJSON(resp *http.Response, v any) error {
} }
// End-to-end against a remote cache-server: token unknown → 401, register → // End-to-end against a remote cache-server: token unknown → 401, register →
// reserve/upload/commit/find/download all OK, revoke → 401 again. // reserve/upload/commit/find/download all OK, revoke → 401 again. Registering also names the
// instance, and the server answering with its own address is what makes a shared cache server the
// whole results service, as the built-in one is.
func TestRunner_ExternalCacheServer_RegisterRevoke(t *testing.T) { func TestRunner_ExternalCacheServer_RegisterRevoke(t *testing.T) {
dir := filepath.Join(t.TempDir(), "remote-cache") dir := filepath.Join(t.TempDir(), "remote-cache")
const secret = "shared-secret-for-tests" const secret = "shared-secret-for-tests"
remote, err := artifactcache.StartHandler(dir, "127.0.0.1", 0, secret, nil) remote, err := artifactcache.StartHandler(dir, "127.0.0.1", 0, secret, nil)
require.NoError(t, err) require.NoError(t, err)
defer remote.Close() defer remote.Close()
gitea := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, `{"ok":true}`)
}))
defer gitea.Close()
r := &Runner{cfg: &config.Config{Cache: config.Cache{ r := &Runner{
cfg: &config.Config{Cache: config.Cache{
ExternalServer: remote.ExternalURL(), ExternalServer: remote.ExternalURL(),
ExternalSecret: secret, ExternalSecret: secret,
}}} }},
envs: map[string]string{"ACTIONS_RESULTS_URL": gitea.URL},
}
token := "external-task-token" token := "external-task-token"
repo := "owner/repoX" repo := "owner/repoX"
@@ -182,10 +193,21 @@ func TestRunner_ExternalCacheServer_RegisterRevoke(t *testing.T) {
require.Equal(t, http.StatusUnauthorized, probe(), require.Equal(t, http.StatusUnauthorized, probe(),
"token must be unknown to the remote server before registration") "token must be unknown to the remote server before registration")
unregister := r.registerCacheForTask(token, repo, nil) unregister, resultsURL := r.registerCacheForTask(token, repo, nil)
require.NotEqual(t, http.StatusUnauthorized, probe(), require.NotEqual(t, http.StatusUnauthorized, probe(),
"token must be accepted after registerCacheForTask") "token must be accepted after registerCacheForTask")
// The server took the results service over, so the artifact half reaches Gitea through it.
require.Equal(t, remote.ExternalURL(), resultsURL)
artifact, err := http.NewRequestWithContext(t.Context(), http.MethodPost,
resultsURL+"/twirp/github.actions.results.api.v1.ArtifactService/CreateArtifact", nil)
require.NoError(t, err)
artifact.Header.Set("Authorization", "Bearer "+token)
forwarded, err := http.DefaultClient.Do(artifact)
require.NoError(t, err)
forwarded.Body.Close()
require.Equal(t, http.StatusOK, forwarded.StatusCode)
// Full reserve→upload→commit→find→download cycle, identical to what // Full reserve→upload→commit→find→download cycle, identical to what
// @actions/cache does, against the remote (external) server. // @actions/cache does, against the remote (external) server.
body := []byte("payload-from-task") body := []byte("payload-from-task")

View File

@@ -297,3 +297,37 @@ func TestRunnerOnIdleSkipsWhenAlreadyCancelled(t *testing.T) {
assert.DirExists(t, stale) assert.DirExists(t, stale)
} }
// The idle sweep reclaims the docker networks of jobs this runner did not live to tear down,
// and stays out of the way of runners that share the daemon but not the registration.
func TestRunnerOnIdleRemovesOrphanNetworks(t *testing.T) {
now := time.Date(2026, time.April, 29, 20, 0, 0, 0, time.UTC)
cfg := &config.Config{
Container: config.Container{RequireDocker: true},
Runner: config.Runner{
WorkdirCleanupAge: 24 * time.Hour,
IdleCleanupInterval: time.Minute,
},
}
var swept []string
var sweptCutoff time.Time
origRemoveOrphanNetworks := removeOrphanNetworks
removeOrphanNetworks = func(_ context.Context, runnerUUID string, createdBefore time.Time) error {
swept = append(swept, runnerUUID)
sweptCutoff = createdBefore
return nil
}
t.Cleanup(func() { removeOrphanNetworks = origRemoveOrphanNetworks })
r := &Runner{uuid: "runner-1", cfg: cfg, now: func() time.Time { return now }}
r.OnIdle(context.Background())
assert.Equal(t, []string{"runner-1"}, swept)
// a network of a job starting during the pass is younger than this and so out of scope
assert.Equal(t, now.Add(-24*time.Hour), sweptCutoff)
// a host-only runner has no daemon to sweep
hostOnly := &Runner{uuid: "runner-2", cfg: &config.Config{Runner: cfg.Runner}, now: func() time.Time { return now }}
hostOnly.OnIdle(context.Background())
assert.Equal(t, []string{"runner-1"}, swept)
}

View File

@@ -5,14 +5,18 @@ package run
import ( import (
"context" "context"
"net/http"
"strings"
"testing" "testing"
"gitea.com/gitea/runner/act/runner"
clientmocks "gitea.com/gitea/runner/internal/pkg/client/mocks" clientmocks "gitea.com/gitea/runner/internal/pkg/client/mocks"
"gitea.com/gitea/runner/internal/pkg/config" "gitea.com/gitea/runner/internal/pkg/config"
"gitea.com/gitea/runner/internal/pkg/ver" "gitea.com/gitea/runner/internal/pkg/ver"
"connectrpc.com/connect" "connectrpc.com/connect"
runnerv1 "gitea.dev/actions-proto-go/runner/v1" runnerv1 "gitea.dev/actions-proto-go/runner/v1"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock" "github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"google.golang.org/protobuf/types/known/structpb" "google.golang.org/protobuf/types/known/structpb"
@@ -91,6 +95,7 @@ func TestNewRunnerInitializesLabelsAndEnvironment(t *testing.T) {
require.Equal(t, "true", r.envs["GITEA_ACTIONS"]) require.Equal(t, "true", r.envs["GITEA_ACTIONS"])
require.NotEmpty(t, r.envs["GITEA_ACTIONS_RUNNER_VERSION"]) require.NotEmpty(t, r.envs["GITEA_ACTIONS_RUNNER_VERSION"])
require.Nil(t, r.cacheHandler) require.Nil(t, r.cacheHandler)
require.Empty(t, r.envs[runner.CacheServiceV2Env], "no cache server, nothing to serve v2 from")
} }
// Proxy variables are assembled per task, because a job's service containers have to be // Proxy variables are assembled per task, because a job's service containers have to be
@@ -120,3 +125,58 @@ func taskWithDefaultActionsURL(url string) *runnerv1.Task {
}, },
} }
} }
// The results service is decided per task, because that is where the job's token and its
// instance are both known. NewRunner only leaves the address Gitea serves.
func TestNewRunnerCacheServiceV2(t *testing.T) {
cfg := &config.Config{}
cfg.Cache.Dir, cfg.Cache.Host = t.TempDir(), "127.0.0.1"
cli := clientmocks.NewClient(t)
cli.On("Address").Return("https://gitea.example/").Maybe()
r := NewRunner(cfg, &config.Registration{Name: "runner"}, cli)
t.Cleanup(func() { _ = r.Close() })
const token = "task-token"
assert.Equal(t, "https://gitea.example", r.envs["ACTIONS_RESULTS_URL"])
assert.Empty(t, r.envs[runner.CacheServiceV2Env], "a promise the runner has not made yet")
assert.True(t, r.patchToolkit())
// The registration is what makes it true: the cache server takes the results service over,
// having been told which instance to forward the artifact half to.
revoke, resultsURL := r.registerCacheForTask(token, "owner/repo", nil)
defer revoke()
require.Equal(t, r.cacheHandler.ExternalURL(), resultsURL)
// And what is advertised has to answer the cache service. A client without the GHES escape
// hatch, docker buildx among them, posts its cache calls at exactly this URL and nowhere else,
// so a 404 here is the failure this whole change is about.
req, err := http.NewRequestWithContext(t.Context(), http.MethodPost,
resultsURL+"/twirp/github.actions.results.api.v1.CacheService/GetCacheEntryDownloadURL",
strings.NewReader(`{"key":"k","version":"v"}`))
require.NoError(t, err)
req.Header.Set("Authorization", "Bearer "+token)
resp, err := http.DefaultClient.Do(req)
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode, "the advertised results service serves no cache service")
}
// The v1 cache client appends its path to ACTIONS_CACHE_URL without a separator, so a configured
// server that is missing the slash would send it to a URL whose port swallows the path.
func TestNewRunnerNormalizesTheExternalCacheServer(t *testing.T) {
cfg := &config.Config{}
cfg.Cache.ExternalServer = "http://cache.local:8088//"
cli := clientmocks.NewClient(t)
cli.On("Address").Return("https://gitea.example/").Maybe()
r := NewRunner(cfg, &config.Registration{Name: "runner"}, cli)
assert.Equal(t, "http://cache.local:8088/", r.envs["ACTIONS_CACHE_URL"])
// Nothing to front the results service with, so the variable stays unset, but the bundles are
// still patched: artifacts v4 need that, and the patch keeps the cache client on the cache URL.
assert.Equal(t, "https://gitea.example", r.envs["ACTIONS_RESULTS_URL"])
assert.Empty(t, r.envs[runner.CacheServiceV2Env])
assert.True(t, r.patchToolkit())
}

View File

@@ -1,7 +1,7 @@
# Example configuration file, it's safe to copy this as the default config file without any modification. # Example configuration file, it's safe to copy this as the default config file without any modification.
# You don't have to copy this file to your instance, # You don't have to copy this file to your instance,
# just run `./gitea-runner generate-config > config.yaml` to generate a config file. # just run `./gitea-runner config generate > config.yaml` to generate a config file.
# Logging for the runner process itself (messages printed to stderr). # Logging for the runner process itself (messages printed to stderr).
# This does not control how workflow step output is streamed to the Gitea UI; # This does not control how workflow step output is streamed to the Gitea UI;
@@ -15,6 +15,9 @@ runner:
# Where to store the registration result. # Where to store the registration result.
file: .runner file: .runner
# Execute how many tasks concurrently at the same time. # Execute how many tasks concurrently at the same time.
# With `container.network` empty, each concurrent docker job takes a subnet from the
# daemon's address pool, so a high capacity can exhaust it. See `default-address-pools`
# in the docker daemon config.
capacity: 1 capacity: 1
# Extra environment variables to run jobs. # Extra environment variables to run jobs.
envs: envs:
@@ -43,9 +46,12 @@ runner:
# While idle, remove stale bind-workdir task directories and orphaned host-mode # While idle, remove stale bind-workdir task directories and orphaned host-mode
# scratch directories (left behind when a host cleanup delete stalls) older than # scratch directories (left behind when a host cleanup delete stalls) older than
# this duration. Setting either workdir_cleanup_age or idle_cleanup_interval to 0 # this duration. Setting either workdir_cleanup_age or idle_cleanup_interval to 0
# (or any non-positive value) disables stale-directory cleanup entirely. # (or any non-positive value) disables stale-directory cleanup entirely, along with
# the docker network cleanup below.
workdir_cleanup_age: 24h workdir_cleanup_age: 24h
# Cadence for the idle stale-directory cleanup pass. # Cadence for the idle cleanup pass. Besides the directories above, on runners that use
# docker it removes the per-job networks of jobs this runner did not live to tear down,
# which would otherwise hold a subnet of the daemon address pool until the host is rebuilt.
idle_cleanup_interval: 10m idle_cleanup_interval: 10m
# The base interval for periodic log flush to the Gitea instance. # The base interval for periodic log flush to the Gitea instance.
# Logs may be sent earlier if the buffer reaches log_report_batch_size # Logs may be sent earlier if the buffer reaches log_report_batch_size
@@ -132,7 +138,7 @@ cache:
# Ignored when external_server is set. # Ignored when external_server is set.
port: 0 port: 0
# URL of a shared `gitea-runner cache-server` to use instead of starting a local one. # URL of a shared `gitea-runner cache-server` to use instead of starting a local one.
# Set on every runner that should share a cache pool. Must end with "/". # Set on every runner that should share a cache pool. A trailing slash is optional.
# Example: "http://cache-host:8088/" # Example: "http://cache-host:8088/"
# Requires external_secret (below) to match the value on the cache-server. # Requires external_secret (below) to match the value on the cache-server.
external_server: "" external_server: ""
@@ -149,6 +155,11 @@ cache:
# A moved tag (e.g. a re-tagged "v6") or an updated branch stays at the cached commit # A moved tag (e.g. a re-tagged "v6") or an updated branch stays at the cached commit
# until its cache entry expires or is manually removed. # until its cache entry expires or is manually removed.
offline_mode: false offline_mode: false
# Serve the actions cache service v2 API, used by actions/cache@v4.2 and later. Those actions
# refuse any host they do not take for GitHub, so reaching it means editing that check out of
# the action's own bundle, keeping the untouched copy beside it. The same edit lets the stock
# upload-artifact and download-artifact work here. A bundle that does not match is left alone.
v2: true
container: container:
# Specifies the network to which the container will connect. # Specifies the network to which the container will connect.
@@ -161,7 +172,9 @@ container:
# network_create_options only apply when `network` is left empty and the runner # network_create_options only apply when `network` is left empty and the runner
# auto-creates a per-job network that does not already exist. They have no effect # auto-creates a per-job network that does not already exist. They have no effect
# when a custom `network` name is set, because that network is used as-is and never # when a custom `network` name is set, because that network is used as-is and never
# created by the runner. Omit the entire block to use Docker's defaults. # created by the runner. Omit the entire block to use Docker's defaults. An auto-created
# network is labelled com.gitea.runner.uuid=<this runner's uuid>, which is how the idle
# cleanup tells its own leftovers apart from those of other runners on the same daemon.
network_create_options: network_create_options:
enable_ipv4: true # Omit to use Docker's default (IPv4 enabled). Set false to disable IPv4. enable_ipv4: true # Omit to use Docker's default (IPv4 enabled). Set false to disable IPv4.
enable_ipv6: false # Omit to use Docker's default (IPv6 disabled). Enabling it requires dockerd started with --ipv6. enable_ipv6: false # Omit to use Docker's default (IPv6 disabled). Enabling it requires dockerd started with --ipv6.
@@ -197,6 +210,9 @@ container:
docker_host: "" docker_host: ""
# Pull docker image(s) even if already present. # Pull docker image(s) even if already present.
# Defaults to false when the key is omitted. # Defaults to false when the key is omitted.
# Two exceptions: an image pinned by digest (image@sha256:...) cannot change, so it is never
# re-pulled, and a pull that fails while a copy is already on the host does not fail the job,
# which runs on that copy with a warning in its log.
force_pull: false force_pull: false
# Rebuild docker image(s) even if already present # Rebuild docker image(s) even if already present
force_rebuild: false force_rebuild: false
@@ -210,6 +226,9 @@ container:
# from the DinD daemon's filesystem. When enabled, ensure the workspace parent # from the DinD daemon's filesystem. When enabled, ensure the workspace parent
# directory is also mounted into the runner container and listed in valid_volumes. # directory is also mounted into the runner container and listed in valid_volumes.
bind_workdir: false bind_workdir: false
# How long a job waits for a service container that declares a healthcheck to become
# healthy. A negative value (e.g. -1s) starts the steps without waiting.
service_ready_timeout: 5m
host: host:
# The parent directory of a job's working directory. # The parent directory of a job's working directory.

View File

@@ -42,7 +42,7 @@ type Runner struct {
FetchInterval time.Duration `yaml:"fetch_interval"` // FetchInterval specifies the interval duration for fetching resources. FetchInterval time.Duration `yaml:"fetch_interval"` // FetchInterval specifies the interval duration for fetching resources.
FetchIntervalMax time.Duration `yaml:"fetch_interval_max"` // FetchIntervalMax specifies the maximum backoff interval when idle. FetchIntervalMax time.Duration `yaml:"fetch_interval_max"` // FetchIntervalMax specifies the maximum backoff interval when idle.
WorkdirCleanupAge time.Duration `yaml:"workdir_cleanup_age"` // WorkdirCleanupAge removes stale bind-workdir task directories and orphaned host-mode scratch dirs older than this duration during idle cleanup. WorkdirCleanupAge time.Duration `yaml:"workdir_cleanup_age"` // WorkdirCleanupAge removes stale bind-workdir task directories and orphaned host-mode scratch dirs older than this duration during idle cleanup.
IdleCleanupInterval time.Duration `yaml:"idle_cleanup_interval"` // IdleCleanupInterval runs stale-directory cleanup periodically while the runner is idle. Set to 0 to disable cleanup cadence. IdleCleanupInterval time.Duration `yaml:"idle_cleanup_interval"` // IdleCleanupInterval runs the idle cleanup (stale directories and orphaned docker networks) periodically while the runner is idle. Set to 0 to disable cleanup cadence.
LogReportInterval time.Duration `yaml:"log_report_interval"` // LogReportInterval specifies the base interval for periodic log flush. LogReportInterval time.Duration `yaml:"log_report_interval"` // LogReportInterval specifies the base interval for periodic log flush.
LogReportMaxLatency time.Duration `yaml:"log_report_max_latency"` // LogReportMaxLatency specifies the max time a log row can wait before being sent. LogReportMaxLatency time.Duration `yaml:"log_report_max_latency"` // LogReportMaxLatency specifies the max time a log row can wait before being sent.
LogReportBatchSize int `yaml:"log_report_batch_size"` // LogReportBatchSize triggers immediate log flush when buffer reaches this size. LogReportBatchSize int `yaml:"log_report_batch_size"` // LogReportBatchSize triggers immediate log flush when buffer reaches this size.
@@ -74,6 +74,7 @@ type Cache struct {
ExternalSecret string `yaml:"external_secret"` // ExternalSecret is a shared secret between this runner and an external gitea-runner cache-server, enabling per-job ACTIONS_RUNTIME_TOKEN authentication and repo scoping over the network. Required whenever ExternalServer is set; ExternalSecretFile is the alternative way to provide it. ExternalSecret string `yaml:"external_secret"` // ExternalSecret is a shared secret between this runner and an external gitea-runner cache-server, enabling per-job ACTIONS_RUNTIME_TOKEN authentication and repo scoping over the network. Required whenever ExternalServer is set; ExternalSecretFile is the alternative way to provide it.
ExternalSecretFile string `yaml:"external_secret_file"` // ExternalSecretFile is the path to a file holding the ExternalSecret value, so the secret can be mounted instead of stored in the config file. LoadDefault reads it into ExternalSecret; setting both is an error. ExternalSecretFile string `yaml:"external_secret_file"` // ExternalSecretFile is the path to a file holding the ExternalSecret value, so the secret can be mounted instead of stored in the config file. LoadDefault reads it into ExternalSecret; setting both is an error.
OfflineMode bool `yaml:"offline_mode"` // OfflineMode reuses a cached action without fetching from the remote; a moved tag or branch stays at the cached commit until the cache entry is removed. OfflineMode bool `yaml:"offline_mode"` // OfflineMode reuses a cached action without fetching from the remote; a moved tag or branch stays at the cached commit until the cache entry is removed.
V2 *bool `yaml:"v2"` // V2 serves the actions cache service v2 API to jobs, used by actions/cache@v4.2 and later, and edits the action bundles that would otherwise refuse it. Unset means enabled.
} }
// Container represents the configuration for the container. // Container represents the configuration for the container.
@@ -86,11 +87,12 @@ type Container struct {
WorkdirParent string `yaml:"workdir_parent"` // WorkdirParent specifies the parent directory for the container's working directory. WorkdirParent string `yaml:"workdir_parent"` // WorkdirParent specifies the parent directory for the container's working directory.
ValidVolumes []string `yaml:"valid_volumes"` // ValidVolumes specifies the volumes (including bind mounts) can be mounted to containers. ValidVolumes []string `yaml:"valid_volumes"` // ValidVolumes specifies the volumes (including bind mounts) can be mounted to containers.
DockerHost string `yaml:"docker_host"` // DockerHost specifies the Docker host. It overrides the value specified in environment variable DOCKER_HOST. DockerHost string `yaml:"docker_host"` // DockerHost specifies the Docker host. It overrides the value specified in environment variable DOCKER_HOST.
ForcePull bool `yaml:"force_pull"` // Pull docker image(s) even if already present ForcePull bool `yaml:"force_pull"` // Pull docker image(s) even if already present, except digest-pinned ones. A pull that fails while a local copy exists is a warning, not a job failure.
ForceRebuild bool `yaml:"force_rebuild"` // Rebuild docker image(s) even if already present ForceRebuild bool `yaml:"force_rebuild"` // Rebuild docker image(s) even if already present
RequireDocker bool `yaml:"require_docker"` // Always require a reachable docker daemon, even if not required by runner RequireDocker bool `yaml:"require_docker"` // Always require a reachable docker daemon, even if not required by runner
DockerTimeout time.Duration `yaml:"docker_timeout"` // Timeout to wait for the docker daemon to be reachable, if docker is required by require_docker or runner DockerTimeout time.Duration `yaml:"docker_timeout"` // Timeout to wait for the docker daemon to be reachable, if docker is required by require_docker or runner
BindWorkdir bool `yaml:"bind_workdir"` // BindWorkdir binds the workspace to the host filesystem instead of using Docker volumes. Required for DinD when jobs use docker compose with bind mounts. BindWorkdir bool `yaml:"bind_workdir"` // BindWorkdir binds the workspace to the host filesystem instead of using Docker volumes. Required for DinD when jobs use docker compose with bind mounts.
ServiceReadyTimeout time.Duration `yaml:"service_ready_timeout"` // ServiceReadyTimeout bounds how long a job waits for a service container that declares a healthcheck to report healthy. Negative disables waiting.
} }
type ContainerNetworkCreateOptions struct { type ContainerNetworkCreateOptions struct {

View File

@@ -347,3 +347,13 @@ cache:
require.Error(t, err) require.Error(t, err)
assert.Contains(t, err.Error(), "contains no secret") assert.Contains(t, err.Error(), "contains no secret")
} }
// The shipped example must parse, and every key in it must be one the config knows.
func TestLoadDefault_ExampleConfigParses(t *testing.T) {
hook := test.NewGlobal()
defer hook.Reset()
_, err := LoadDefault("config.example.yaml")
require.NoError(t, err)
assert.Empty(t, hook.AllEntries())
}

530
internal/pkg/config/edit.go Normal file
View File

@@ -0,0 +1,530 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package config
import (
"bytes"
"errors"
"fmt"
"os"
"path/filepath"
"reflect"
"slices"
"strconv"
"strings"
"time"
"go.yaml.in/yaml/v4"
)
type fieldKind int
const (
kindScalar fieldKind = iota
kindSequence
kindSection
)
var durationType = reflect.TypeFor[time.Duration]()
// GetValue renders a flat list or mapping one entry per line, and anything nested as YAML.
func GetValue(file, path string) (string, error) {
session, err := loadForEdit(file, path)
if err != nil {
return "", err
}
node, err := lookupNode(session.root, session.segments)
if err != nil {
return "", err
}
return renderNode(node)
}
func SetValue(file, path string, values ...string) error {
session, err := loadForEdit(file, path)
if err != nil {
return err
}
var replacement *yaml.Node
switch session.field.kind {
case kindSequence:
if len(values) == 0 {
return fmt.Errorf("%q needs at least one value", path)
}
items, err := session.scalars(values)
if err != nil {
return err
}
replacement = &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq", Content: items}
case kindScalar:
if len(values) != 1 {
return fmt.Errorf("%q takes exactly one value", path)
}
items, err := session.scalars(values)
if err != nil {
return err
}
replacement = items[0]
default:
return fmt.Errorf("%q is a section, set one of its keys instead", path)
}
node, err := ensureNode(session.root, session.segments)
if err != nil {
return err
}
replaceNode(node, replacement)
return session.write()
}
func AddValue(file, path string, values ...string) error {
session, err := loadSequenceEdit(file, path, values)
if err != nil {
return err
}
items, err := session.scalars(values)
if err != nil {
return err
}
node, err := ensureNode(session.root, session.segments)
if err != nil {
return err
}
if node.Kind != yaml.SequenceNode {
replaceNode(node, &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq"})
}
for _, item := range items {
if indexOfValue(node, item.Value) >= 0 {
return fmt.Errorf("%s already contains %q", path, item.Value)
}
node.Content = append(node.Content, item)
}
return session.write()
}
func RemoveValue(file, path string, values ...string) error {
session, err := loadSequenceEdit(file, path, values)
if err != nil {
return err
}
items, err := session.scalars(values)
if err != nil {
return err
}
node, err := lookupNode(session.root, session.segments)
if err != nil {
return err
}
if node.Kind != yaml.SequenceNode {
return fmt.Errorf("%s is not a list in %q", path, file)
}
for _, item := range items {
index := indexOfValue(node, item.Value)
if index < 0 {
return fmt.Errorf("%s does not contain %q", path, item.Value)
}
node.Content = slices.Delete(node.Content, index, index+1)
}
return session.write()
}
func indexOfValue(seq *yaml.Node, value string) int {
for i, item := range seq.Content {
if item.Kind == yaml.ScalarNode && item.Value == value {
return i
}
}
return -1
}
// replaceNode assigns field by field, as *node = *with would drop the comments attached to node.
func replaceNode(node, with *yaml.Node) {
node.Kind, node.Tag, node.Style, node.Value, node.Content = with.Kind, with.Tag, with.Style, with.Value, with.Content
}
type editSession struct {
file string
path string
original []byte
root *yaml.Node
field *fieldInfo
segments []string
}
// loadForEdit validates the path and parses the file, so every caller fails before anything is written.
func loadForEdit(file, path string) (*editSession, error) {
if path == "" {
return nil, errors.New("no config key given")
}
segments := strings.Split(path, ".")
field, err := resolvePath(segments)
if err != nil {
return nil, err
}
content, err := os.ReadFile(file)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return nil, fmt.Errorf("config file %q does not exist, create one with `config generate`", file)
}
return nil, err
}
var root yaml.Node
if err := yaml.Unmarshal(content, &root); err != nil {
return nil, fmt.Errorf("parse config file %q: %w", file, err)
}
if root.Kind == 0 || len(root.Content) == 0 {
root = yaml.Node{
Kind: yaml.DocumentNode,
Content: []*yaml.Node{{Kind: yaml.MappingNode, Tag: "!!map"}},
}
}
if root.Content[0].Kind != yaml.MappingNode {
return nil, fmt.Errorf("config file %q is not a YAML mapping", file)
}
return &editSession{file: file, path: path, original: content, root: &root, field: field, segments: segments}, nil
}
func loadSequenceEdit(file, path string, values []string) (*editSession, error) {
session, err := loadForEdit(file, path)
if err != nil {
return nil, err
}
if session.field.kind != kindSequence {
return nil, fmt.Errorf("%q is not a list, use `config set` instead", path)
}
if len(values) == 0 {
return nil, fmt.Errorf("%q needs at least one value", path)
}
return session, nil
}
func (s *editSession) scalars(values []string) ([]*yaml.Node, error) {
nodes := make([]*yaml.Node, 0, len(values))
for _, value := range values {
node, err := scalarNode(s.field.typ, value)
if err != nil {
return nil, fmt.Errorf("%s: %w", s.path, err)
}
nodes = append(nodes, node)
}
return nodes, nil
}
func lookupNode(root *yaml.Node, segments []string) (*yaml.Node, error) {
node := root.Content[0]
for i, segment := range segments {
if node.Kind != yaml.MappingNode {
return nil, fmt.Errorf("%q is not set", strings.Join(segments[:i], "."))
}
value := mappingValue(node, segment)
if value == nil {
return nil, fmt.Errorf("%q is not set", strings.Join(segments[:i+1], "."))
}
node = value
}
return node, nil
}
func ensureNode(root *yaml.Node, segments []string) (*yaml.Node, error) {
node := root.Content[0]
for i, segment := range segments {
if node.Kind != yaml.MappingNode {
if node.Kind == yaml.ScalarNode && node.Tag == "!!null" {
node.Kind, node.Tag, node.Style, node.Value = yaml.MappingNode, "!!map", 0, ""
} else {
return nil, fmt.Errorf("%q is not a section", strings.Join(segments[:i], "."))
}
}
value := mappingValue(node, segment)
if value == nil {
value = &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!null"}
node.Content = append(node.Content,
&yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: segment},
value)
}
node = value
}
return node, nil
}
func mappingValue(mapping *yaml.Node, key string) *yaml.Node {
for i := 0; i+1 < len(mapping.Content); i += 2 {
if mapping.Content[i].Value == key {
return mapping.Content[i+1]
}
}
return nil
}
func renderNode(node *yaml.Node) (string, error) {
if !allScalars(node.Content) {
encoded, err := encodeYAML(node)
if err != nil {
return "", err
}
return strings.TrimRight(string(encoded), "\n"), nil
}
switch node.Kind {
case yaml.SequenceNode:
lines := make([]string, 0, len(node.Content))
for _, item := range node.Content {
lines = append(lines, item.Value)
}
return strings.Join(lines, "\n"), nil
case yaml.MappingNode:
lines := make([]string, 0, len(node.Content)/2)
for i := 0; i+1 < len(node.Content); i += 2 {
lines = append(lines, node.Content[i].Value+"="+node.Content[i+1].Value)
}
return strings.Join(lines, "\n"), nil
default:
return node.Value, nil
}
}
func allScalars(nodes []*yaml.Node) bool {
for _, node := range nodes {
if node.Kind != yaml.ScalarNode {
return false
}
}
return true
}
func encodeYAML(node *yaml.Node) ([]byte, error) {
var buf bytes.Buffer
encoder := yaml.NewEncoder(&buf)
encoder.SetIndent(2)
if err := encoder.Encode(node); err != nil {
return nil, err
}
if err := encoder.Close(); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// restoreBlankLines re-inserts the blank lines between top-level sections that the encoder drops.
func restoreBlankLines(original, generated []byte) []byte {
spaced := map[string]bool{}
blank := false
for line := range strings.Lines(string(original)) {
line = strings.TrimRight(line, "\r\n")
switch {
case strings.TrimSpace(line) == "":
blank = true
case strings.HasPrefix(line, "#"): // the block belongs to the key below it
default:
if key, ok := topLevelKey(line); ok && blank {
spaced[key] = true
}
blank = false
}
}
var out []string
for line := range strings.Lines(string(generated)) {
line = strings.TrimRight(line, "\r\n")
if key, ok := topLevelKey(line); ok && spaced[key] {
start := len(out)
for start > 0 && strings.HasPrefix(out[start-1], "#") {
start--
}
if start > 0 && strings.TrimSpace(out[start-1]) != "" {
out = slices.Insert(out, start, "")
}
}
out = append(out, line)
}
if bytes.HasSuffix(generated, []byte("\n")) {
out = append(out, "")
}
return []byte(strings.Join(out, "\n"))
}
func topLevelKey(line string) (string, bool) {
if line == "" || line[0] == ' ' || line[0] == '\t' || line[0] == '#' || line[0] == '-' {
return "", false
}
key, _, ok := strings.Cut(line, ":")
return key, ok
}
func (s *editSession) write() error {
generated, err := encodeYAML(s.root)
if err != nil {
return err
}
// A file the runner already refused to load stays the user's to fix, only a regression is rejected.
if err := yaml.Unmarshal(generated, &Config{}); err != nil && yaml.Unmarshal(s.original, &Config{}) == nil {
return fmt.Errorf("the edit would produce a config the runner cannot load: %w", err)
}
content := restoreBlankLines(s.original, generated)
if bytes.Contains(s.original, []byte("\r\n")) { // the encoder only ever emits LF
content = bytes.ReplaceAll(content, []byte("\n"), []byte("\r\n"))
}
file := s.file
if resolved, err := filepath.EvalSymlinks(file); err == nil {
file = resolved // keeps a config linked in from elsewhere intact
}
var info os.FileInfo
mode := os.FileMode(0o600)
if stat, err := os.Stat(file); err == nil {
info, mode = stat, stat.Mode().Perm()
}
temp, err := os.CreateTemp(filepath.Dir(file), filepath.Base(file)+".*.tmp")
if err != nil {
return err
}
defer os.Remove(temp.Name())
if _, err := temp.Write(content); err != nil {
temp.Close()
return err
}
if err := temp.Sync(); err != nil {
temp.Close()
return err
}
if err := temp.Close(); err != nil {
return err
}
if info != nil { // before the chmod, as a chown can clear mode bits
if err := preserveOwner(temp.Name(), info); err != nil {
return err
}
}
if err := os.Chmod(temp.Name(), mode); err != nil {
return err
}
return os.Rename(temp.Name(), file)
}
type fieldInfo struct {
kind fieldKind
typ reflect.Type // the element type for a sequence
}
// resolvePath walks the Config struct through the yaml tags of a dotted path.
func resolvePath(segments []string) (*fieldInfo, error) {
typ := reflect.TypeFor[Config]()
for i, segment := range segments {
switch typ.Kind() {
case reflect.Struct:
field, ok := fieldByYAMLName(typ, segment)
if !ok {
return nil, fmt.Errorf("unknown config key %q, valid keys here: %s",
strings.Join(segments[:i+1], "."), strings.Join(yamlNames(typ), ", "))
}
typ = field.Type
case reflect.Map:
// The segment names a user-defined entry, so the walk ends here.
if i != len(segments)-1 {
return nil, fmt.Errorf("%q has no sub-keys", strings.Join(segments[:i+1], "."))
}
return &fieldInfo{kind: kindScalar, typ: typ.Elem()}, nil
default:
return nil, fmt.Errorf("%q is a value, not a section", strings.Join(segments[:i], "."))
}
}
switch typ.Kind() {
case reflect.Slice:
return &fieldInfo{kind: kindSequence, typ: typ.Elem()}, nil
case reflect.Map, reflect.Struct:
return &fieldInfo{kind: kindSection}, nil
default:
return &fieldInfo{kind: kindScalar, typ: typ}, nil
}
}
func fieldByYAMLName(typ reflect.Type, name string) (reflect.StructField, bool) {
for field := range typ.Fields() {
if yamlName(field) == name {
return field, true
}
}
return reflect.StructField{}, false
}
func yamlNames(typ reflect.Type) []string {
names := make([]string, 0, typ.NumField())
for field := range typ.Fields() {
if name := yamlName(field); name != "-" {
names = append(names, name)
}
}
slices.Sort(names)
return names
}
func yamlName(field reflect.StructField) string {
name, _, _ := strings.Cut(field.Tag.Get("yaml"), ",")
if name == "" {
return strings.ToLower(field.Name)
}
return name
}
// scalarNode types the value, so a bad one is reported instead of landing in the file as a string.
func scalarNode(typ reflect.Type, value string) (*yaml.Node, error) {
if typ.Kind() == reflect.Pointer {
typ = typ.Elem()
}
if typ == durationType {
duration, err := time.ParseDuration(value)
if err != nil {
return nil, fmt.Errorf("%q is not a duration such as 30s, 5m or 3h", value)
}
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: duration.String()}, nil
}
switch typ.Kind() {
case reflect.Bool:
parsed, err := strconv.ParseBool(value)
if err != nil {
return nil, fmt.Errorf("%q is not a boolean", value)
}
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!bool", Value: strconv.FormatBool(parsed)}, nil
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
parsed, err := strconv.ParseInt(value, 10, typ.Bits())
if err != nil {
return nil, fmt.Errorf("%q is not a valid %s", value, typ.Kind())
}
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!int", Value: strconv.FormatInt(parsed, 10)}, nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
parsed, err := strconv.ParseUint(value, 10, typ.Bits())
if err != nil {
return nil, fmt.Errorf("%q is not a valid %s", value, typ.Kind())
}
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!int", Value: strconv.FormatUint(parsed, 10)}, nil
case reflect.String:
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: value}, nil
default:
return nil, fmt.Errorf("unsupported config value type %s", typ)
}
}

View File

@@ -0,0 +1,13 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build windows || plan9
package config
import "os"
// preserveOwner is a no-op where a new file inherits its ownership from the directory.
func preserveOwner(_ string, _ os.FileInfo) error {
return nil
}

View File

@@ -0,0 +1,267 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package config
import (
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const editFixture = `# A leading comment.
log:
# The logging level.
level: info
runner:
capacity: 1
envs:
EXISTING: value
timeout: 3h
labels:
- ubuntu-latest:docker://node:20
- self-hosted
`
func writeEditFixture(t *testing.T) string {
t.Helper()
path := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(path, []byte(editFixture), 0o600))
return path
}
func TestEditValues(t *testing.T) {
tests := []struct {
name string
edit func(file string) error
assert func(t *testing.T, cfg *Config, content string)
}{
{
name: "set scalar",
edit: func(file string) error { return SetValue(file, "runner.capacity", "4") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, 4, cfg.Runner.Capacity)
},
},
{
name: "set duration",
edit: func(file string) error { return SetValue(file, "runner.timeout", "90m") },
assert: func(t *testing.T, cfg *Config, content string) {
assert.Equal(t, 90*time.Minute, cfg.Runner.Timeout)
assert.Contains(t, content, "timeout: 1h30m0s")
},
},
{
name: "set a pointer field in a missing section",
edit: func(file string) error {
return SetValue(file, "container.network_create_options.enable_ipv4", "false")
},
assert: func(t *testing.T, cfg *Config, _ string) {
require.NotNil(t, cfg.Container.NetworkCreateOptions.EnableIPv4)
assert.False(t, *cfg.Container.NetworkCreateOptions.EnableIPv4)
},
},
{
name: "set map entry",
edit: func(file string) error { return SetValue(file, "runner.envs.ADDED", "yes") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, map[string]string{"EXISTING": "value", "ADDED": "yes"}, cfg.Runner.Envs)
},
},
{
name: "set replaces a list",
edit: func(file string) error { return SetValue(file, "runner.labels", "one", "two") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, []string{"one", "two"}, cfg.Runner.Labels)
},
},
{
name: "add appends to a list",
edit: func(file string) error { return AddValue(file, "runner.labels", "ubuntu:docker://node:22") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, []string{"ubuntu-latest:docker://node:20", "self-hosted", "ubuntu:docker://node:22"}, cfg.Runner.Labels)
},
},
{
name: "remove drops a list entry",
edit: func(file string) error { return RemoveValue(file, "runner.labels", "self-hosted") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, []string{"ubuntu-latest:docker://node:20"}, cfg.Runner.Labels)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
file := writeEditFixture(t)
require.NoError(t, tt.edit(file))
raw, err := os.ReadFile(file)
require.NoError(t, err)
content := string(raw)
cfg, err := LoadDefault(file)
require.NoError(t, err)
tt.assert(t, cfg, content)
assert.Contains(t, content, "# A leading comment.")
assert.Contains(t, content, " # The logging level.")
assert.Contains(t, content, "\n\nrunner:")
})
}
}
func TestEditValuesRejectsBadInput(t *testing.T) {
tests := []struct {
name string
edit func(file string) error
wantErr string
}{
{
name: "unknown key",
edit: func(file string) error { return SetValue(file, "runner.labl", "x") },
wantErr: `unknown config key "runner.labl"`,
},
{
name: "value is not a number",
edit: func(file string) error { return SetValue(file, "runner.capacity", "many") },
wantErr: `"many" is not a valid int`,
},
{
name: "value is not a duration",
edit: func(file string) error { return SetValue(file, "runner.timeout", "soon") },
wantErr: `"soon" is not a duration`,
},
{
name: "value is not a boolean",
edit: func(file string) error { return SetValue(file, "runner.insecure", "maybe") },
wantErr: `"maybe" is not a boolean`,
},
{
name: "set needs a single value",
edit: func(file string) error { return SetValue(file, "runner.capacity", "1", "2") },
wantErr: "takes exactly one value",
},
{
name: "set on a section",
edit: func(file string) error { return SetValue(file, "runner", "x") },
wantErr: "is a section",
},
{
name: "add on a scalar",
edit: func(file string) error { return AddValue(file, "runner.capacity", "4") },
wantErr: "is not a list",
},
{
name: "add a duplicate",
edit: func(file string) error { return AddValue(file, "runner.labels", "self-hosted") },
wantErr: `already contains "self-hosted"`,
},
{
name: "remove a missing entry",
edit: func(file string) error { return RemoveValue(file, "runner.labels", "absent") },
wantErr: `does not contain "absent"`,
},
{
name: "sub-key of a free-form map entry",
edit: func(file string) error { return SetValue(file, "runner.envs.A.B", "x") },
wantErr: "has no sub-keys",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
file := writeEditFixture(t)
err := tt.edit(file)
require.Error(t, err)
assert.Contains(t, err.Error(), tt.wantErr)
content, err := os.ReadFile(file)
require.NoError(t, err)
assert.Equal(t, editFixture, string(content), "a rejected edit must leave the file untouched")
})
}
}
func TestGetValue(t *testing.T) {
file := writeEditFixture(t)
value, err := GetValue(file, "runner.capacity")
require.NoError(t, err)
assert.Equal(t, "1", value)
value, err = GetValue(file, "runner.labels")
require.NoError(t, err)
assert.Equal(t, "ubuntu-latest:docker://node:20\nself-hosted", value)
value, err = GetValue(file, "runner.envs")
require.NoError(t, err)
assert.Equal(t, "EXISTING=value", value)
// A section has no single-line rendering.
value, err = GetValue(file, "runner")
require.NoError(t, err)
assert.Contains(t, value, "labels:\n - ubuntu-latest:docker://node:20")
_, err = GetValue(file, "metrics.addr")
require.Error(t, err)
assert.Contains(t, err.Error(), "is not set")
}
func TestEditValuesFileHandling(t *testing.T) {
t.Run("reports a missing file", func(t *testing.T) {
err := SetValue(filepath.Join(t.TempDir(), "absent.yaml"), "runner.capacity", "4")
require.Error(t, err)
assert.Contains(t, err.Error(), "does not exist")
})
t.Run("writes through a symlink", func(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "real.yaml")
link := filepath.Join(dir, "config.yaml")
require.NoError(t, os.WriteFile(target, []byte(editFixture), 0o600))
require.NoError(t, os.Symlink(target, link))
require.NoError(t, SetValue(link, "runner.capacity", "4"))
info, err := os.Lstat(link)
require.NoError(t, err)
assert.NotZero(t, info.Mode()&os.ModeSymlink, "the symlink must not be replaced by a regular file")
content, err := os.ReadFile(target)
require.NoError(t, err)
assert.Contains(t, string(content), "capacity: 4")
})
t.Run("keeps CRLF line endings", func(t *testing.T) {
file := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(file, []byte(strings.ReplaceAll(editFixture, "\n", "\r\n")), 0o600))
require.NoError(t, SetValue(file, "runner.capacity", "4"))
content, err := os.ReadFile(file)
require.NoError(t, err)
assert.Contains(t, string(content), "capacity: 4\r\n")
assert.NotContains(t, strings.ReplaceAll(string(content), "\r\n", ""), "\n")
})
}
// The example config is the file users edit, so it has to stay written the way
// the encoder emits it, down to the single space before a trailing comment.
func TestEditValuesKeepsExampleConfigIntact(t *testing.T) {
file := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(file, Example, 0o600))
require.NoError(t, AddValue(file, "runner.labels", "ubuntu:docker://node:22"))
content, err := os.ReadFile(file)
require.NoError(t, err)
withoutAdded := strings.Replace(string(content), " - ubuntu:docker://node:22\n", "", 1)
assert.Equal(t, string(Example), withoutAdded, "only the appended label may differ")
}

View File

@@ -0,0 +1,25 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build !windows && !plan9
package config
import (
"errors"
"os"
"syscall"
)
// preserveOwner keeps a config that root edits owned by the service user it was created for.
func preserveOwner(file string, info os.FileInfo) error {
stat, ok := info.Sys().(*syscall.Stat_t)
if !ok {
return nil
}
// A caller that may replace the file but not chown it is no worse off than before.
if err := os.Chown(file, int(stat.Uid), int(stat.Gid)); err != nil && !errors.Is(err, os.ErrPermission) {
return err
}
return nil
}

35
internal/pkg/lock/lock.go Normal file
View File

@@ -0,0 +1,35 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
// Package lock provides a cross-platform, non-blocking advisory file lock used
// to ensure a single runner process owns a given runner file.
package lock
import (
"errors"
"fmt"
)
// ErrLocked is returned by TryLock when another process already holds the lock.
var ErrLocked = errors.New("runner file is already locked by another process")
// TryLock takes a non-blocking exclusive advisory lock tied to runnerFile. The
// lock is placed on a sibling "<runnerFile>.lock" so it never interferes with
// in-place rewrites of the runner file itself.
//
// It returns a release function that drops the lock. The operating system also
// releases the lock automatically when the process exits, including on a hard
// kill, so a crashed runner never leaves a stale lock behind.
//
// If another process already holds the lock, it returns ErrLocked.
func TryLock(runnerFile string) (func() error, error) {
path := runnerFile + ".lock"
release, err := tryLock(path)
if errors.Is(err, ErrLocked) {
return nil, ErrLocked
}
if err != nil {
return nil, fmt.Errorf("lock %q: %w", path, err)
}
return release, nil
}

View File

@@ -0,0 +1,11 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build plan9
package lock
// tryLock is a best-effort no-op on plan9, which lacks flock/LockFileEx.
func tryLock(_ string) (func() error, error) {
return func() error { return nil }, nil
}

View File

@@ -0,0 +1,54 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build !plan9
package lock
import (
"errors"
"path/filepath"
"testing"
)
func TestTryLock(t *testing.T) {
runnerFile := filepath.Join(t.TempDir(), ".runner")
release, err := TryLock(runnerFile)
if err != nil {
t.Fatalf("first TryLock failed: %v", err)
}
// A second lock on the same file must be refused while the first is held.
if _, err := TryLock(runnerFile); !errors.Is(err, ErrLocked) {
t.Fatalf("second TryLock: want ErrLocked, got %v", err)
}
if err := release(); err != nil {
t.Fatalf("release failed: %v", err)
}
// After release the lock is available again.
release2, err := TryLock(runnerFile)
if err != nil {
t.Fatalf("TryLock after release failed: %v", err)
}
if err := release2(); err != nil {
t.Fatalf("second release failed: %v", err)
}
}
// TestTryLockUncreatable ensures a lock file that cannot be created reports a
// non-ErrLocked error, so callers can tell "already locked by another process"
// apart from "couldn't lock" and degrade gracefully (e.g. a read-only mount).
func TestTryLockUncreatable(t *testing.T) {
runnerFile := filepath.Join(t.TempDir(), "missing-dir", ".runner")
_, err := TryLock(runnerFile)
if err == nil {
t.Fatal("TryLock on an uncreatable lock file: want error, got nil")
}
if errors.Is(err, ErrLocked) {
t.Fatal("TryLock on an uncreatable lock file: want non-ErrLocked error, got ErrLocked")
}
}

View File

@@ -0,0 +1,35 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build !windows && !plan9
package lock
import (
"errors"
"os"
"golang.org/x/sys/unix"
)
// tryLock opens (creating if needed) the lock file and takes a non-blocking
// exclusive flock on it. The returned release closes the file, which drops the
// lock; the kernel also drops it when the process exits.
func tryLock(path string) (func() error, error) {
f, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR, 0o600)
if err != nil {
return nil, err
}
if err := unix.Flock(int(f.Fd()), unix.LOCK_EX|unix.LOCK_NB); err != nil {
_ = f.Close()
if errors.Is(err, unix.EWOULDBLOCK) || errors.Is(err, unix.EAGAIN) {
return nil, ErrLocked
}
return nil, err
}
return func() error {
// Best-effort unlock; closing the fd releases the lock regardless.
_ = unix.Flock(int(f.Fd()), unix.LOCK_UN)
return f.Close()
}, nil
}

View File

@@ -0,0 +1,36 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build windows
package lock
import (
"errors"
"os"
"golang.org/x/sys/windows"
)
// tryLock opens (creating if needed) the lock file and takes a non-blocking
// exclusive lock on it via LockFileEx. The returned release closes the file,
// which drops the lock; Windows also drops it when the process exits.
func tryLock(path string) (func() error, error) {
f, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR, 0o600)
if err != nil {
return nil, err
}
handle := windows.Handle(f.Fd())
overlapped := new(windows.Overlapped)
if err := windows.LockFileEx(handle, windows.LOCKFILE_EXCLUSIVE_LOCK|windows.LOCKFILE_FAIL_IMMEDIATELY, 0, 1, 0, overlapped); err != nil {
_ = f.Close()
if errors.Is(err, windows.ERROR_LOCK_VIOLATION) {
return nil, ErrLocked
}
return nil, err
}
return func() error {
_ = windows.UnlockFileEx(handle, 0, 1, 0, overlapped)
return f.Close()
}, nil
}

View File

@@ -255,6 +255,9 @@ func (r *Reporter) Fire(entry *log.Entry) error {
if step.StartedAt == nil { if step.StartedAt == nil {
step.StartedAt = timestamppb.New(timestamp) step.StartedAt = timestamppb.New(timestamp)
urgentState = true urgentState = true
// The runner's own handler is per step, so an unresumed ::stop-commands:: must not
// leave the reporter suppressed, and no longer registering masks, for the whole job.
r.stopCommandEndToken = ""
} }
// Force reporting log errors as raw output to prevent silent failures // Force reporting log errors as raw output to prevent silent failures
@@ -396,10 +399,9 @@ func (r *Reporter) Logf(format string, a ...any) {
func (r *Reporter) logf(format string, a ...any) { func (r *Reporter) logf(format string, a ...any) {
if !r.duringSteps() { if !r.duringSteps() {
r.logRows = append(r.logRows, &runnerv1.LogRow{ // Masked like any other row: these bypass parseLogRow, but a caller can still
Time: timestamppb.Now(), // interpolate a secret, such as a configured URL carrying credentials.
Content: fmt.Sprintf(format, a...), r.logRows = append(r.logRows, r.newLogRow(timestamppb.Now(), fmt.Sprintf(format, a...)))
})
} }
} }
@@ -700,66 +702,128 @@ func (r *Reporter) parseResult(result any) (runnerv1.Result, bool) {
return ret, ok return ret, ok
} }
var cmdRegex = regexp.MustCompile(`^::([^ :]+)( .*)?::(.*)$`) // A property value never contains a raw ':' (GitHub escapes it as %3A), so excluding ':' ends
// the property list at the first '::' as GitHub does; greedily would swallow a '::' message.
var cmdRegex = regexp.MustCompile(`^::([^ :]+)( [^:]*)?::(.*)$`)
func (r *Reporter) handleCommand(originalContent, command, value string) *string { // handleCommand takes value still escaped, so that the web UI decodes it exactly once. Only
if r.stopCommandEndToken != "" && command != r.stopCommandEndToken { // the branches that consume the payload here decode it.
func (r *Reporter) handleCommand(originalContent, command, properties, value string) *string {
if r.stopCommandEndToken != "" {
if command != r.stopCommandEndToken {
return &originalContent return &originalContent
} }
// Resumed here rather than from the switch, because the end token is arbitrary and a
// token naming a real command would otherwise never resume.
r.stopCommandEndToken = ""
return nil
}
switch command { switch command {
case "add-mask": case "add-mask":
r.addMask(value) r.addMask(runner.UnescapeCommandData(value))
return nil return nil
case "debug": case "debug":
if r.debugOutputEnabled { if r.debugOutputEnabled {
return &value return &originalContent // kept as ::debug::, so the web UI labels and decodes it
} }
return nil return nil
case "notice": case "notice", "warning", "error":
// Not implemented yet, so just return the original content. // Gitea has no annotation store, so the annotation is rendered into the log with
return &originalContent // its source location instead of being dropped: that location is the whole point
case "warning": // of the command for compiler and linter output.
// Not implemented yet, so just return the original content. annotation := formatAnnotation(command, properties, value)
return &originalContent return &annotation
case "error": case "group", "endgroup":
// Not implemented yet, so just return the original content. // Passed through: the web UI folds the log on these and decodes the payload itself.
return &originalContent
case "group":
// Returning the original content, because I think the frontend
// will use it when rendering the output.
return &originalContent
case "endgroup":
// Ditto
return &originalContent return &originalContent
case "stop-commands": case "stop-commands":
r.stopCommandEndToken = value r.stopCommandEndToken = runner.UnescapeCommandData(value)
return nil
case r.stopCommandEndToken:
r.stopCommandEndToken = ""
return nil return nil
} }
return &originalContent return &originalContent
} }
// formatAnnotation folds the file, line, column and title the command carries into its message,
// which the web UI otherwise drops along with the rest of the properties:
//
// ::error file=main.go,line=12,col=5,title=vet::undefined: x
// ::error::main.go:12:5: vet: undefined: x
//
// The ::-form prefix is deliberate, and value is not escaped here because it arrived escaped
// and must stay that way.
func formatAnnotation(level, properties, value string) string {
props := parseCommandProperties(properties)
prefix := props["file"]
if prefix != "" {
if props["line"] != "" {
prefix += ":" + props["line"]
if props["col"] != "" {
prefix += ":" + props["col"]
}
}
prefix += ": "
}
if props["title"] != "" {
prefix += props["title"] + ": "
}
return "::" + level + "::" + prefix + value
}
// parseCommandProperties parses the `file=main.go,line=12` part of a workflow command.
func parseCommandProperties(properties string) map[string]string {
properties = strings.TrimSpace(properties)
if properties == "" {
return nil
}
props := map[string]string{}
for pair := range strings.SplitSeq(properties, ",") {
key, value, ok := strings.Cut(pair, "=")
if !ok {
continue
}
// Only the property-list separators are decoded, the web UI decodes the rest.
value = strings.ReplaceAll(strings.ReplaceAll(value, "%3A", ":"), "%2C", ",")
// GitHub keys its property dictionary case-insensitively, so `File=` works there too.
props[strings.ToLower(strings.TrimSpace(key))] = value
}
// GitHub's toolkit emits `col`; accept `column` as well, which some tools write instead.
if props["col"] == "" {
props["col"] = props["column"]
}
return props
}
func (r *Reporter) parseLogRow(entry *log.Entry) *runnerv1.LogRow { func (r *Reporter) parseLogRow(entry *log.Entry) *runnerv1.LogRow {
content := strings.TrimRight(entry.Message, "\r\n") content := strings.TrimRight(entry.Message, "\r\n")
// cmdRegex only covers the ::cmd:: form, so the ##[add-mask] one would otherwise reach
// the log carrying its own secret. Registered and dropped like its ::add-mask:: twin.
if arg, ok := strings.CutPrefix(content, "##[add-mask]"); ok {
r.addMask(runner.UnescapeCommandData(arg))
return nil
}
matches := cmdRegex.FindStringSubmatch(content) matches := cmdRegex.FindStringSubmatch(content)
if matches != nil { if matches != nil {
if output := r.handleCommand(content, matches[1], runner.UnescapeCommandData(matches[3])); output != nil { if output := r.handleCommand(content, matches[1], matches[2], matches[3]); output != nil {
content = *output content = *output
} else { } else {
return nil return nil
} }
} }
content = r.logReplacer.Replace(content) return r.newLogRow(timestamppb.New(entry.Time), content)
}
// newLogRow applies the masking and validation every row must carry, whatever built it.
func (r *Reporter) newLogRow(t *timestamppb.Timestamp, content string) *runnerv1.LogRow {
return &runnerv1.LogRow{ return &runnerv1.LogRow{
Time: timestamppb.New(entry.Time), Time: t,
Content: strings.ToValidUTF8(content, "?"), Content: strings.ToValidUTF8(r.logReplacer.Replace(content), "?"),
} }
} }

View File

@@ -5,15 +5,18 @@ package report
import ( import (
"context" "context"
"encoding/base64"
"errors" "errors"
"fmt" "fmt"
"maps" "maps"
"net/url"
"slices" "slices"
"strings" "strings"
"sync/atomic" "sync/atomic"
"testing" "testing"
"time" "time"
"gitea.com/gitea/runner/act/runner"
"gitea.com/gitea/runner/internal/pkg/client/mocks" "gitea.com/gitea/runner/internal/pkg/client/mocks"
"gitea.com/gitea/runner/internal/pkg/config" "gitea.com/gitea/runner/internal/pkg/config"
@@ -69,9 +72,12 @@ func TestReporter_parseLogRow(t *testing.T) {
"Debug enabled", true, "Debug enabled", true,
[]string{ []string{
"::debug::GitHub Actions runtime token access controls", "::debug::GitHub Actions runtime token access controls",
// Left escaped: the web UI decodes it, and a real newline would not survive storage.
"::debug::first%0Asecond",
}, },
[]string{ []string{
"GitHub Actions runtime token access controls", "::debug::GitHub Actions runtime token access controls",
"::debug::first%0Asecond",
}, },
}, },
{ {
@@ -83,31 +89,46 @@ func TestReporter_parseLogRow(t *testing.T) {
"<nil>", "<nil>",
}, },
}, },
// The three annotation levels share one code path, so the property shapes are only
// exercised under "error"; notice and warning just prove the level token round-trips.
{ {
"notice", false, "notice", false,
[]string{ []string{
"::notice file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work", "::notice::Gosh, that's not going to work",
}, },
[]string{ []string{
"::notice file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work", "::notice::Gosh, that's not going to work",
}, },
}, },
{ {
"warning", false, "warning", false,
[]string{ []string{
"::warning file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work", "::warning::Gosh, that's not going to work",
}, },
[]string{ []string{
"::warning file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work", "::warning::Gosh, that's not going to work",
}, },
}, },
{ {
"error", false, "error", false,
[]string{ []string{
"::error file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work", "::error file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::error::Gosh, that's not going to work",
"::error file=file.name,line=42,col=7::Gosh, that's not going to work",
// The message keeps its own '::', the property list ends at the first one.
"::error file=main.cpp,line=12::no member named 'foo' in 'std::vector<int>'",
// GitHub matches property names case-insensitively.
"::error File=file.name,Line=42,Col=7::Gosh, that's not going to work",
// Only the property separators are decoded here, %25/%0A are left for the web UI.
"::error file=a%3Ab.go,title=100%252C::still %25 escaped%0Aand multi-line",
}, },
[]string{ []string{
"::error file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work", "::error::file.name:42: Cool Title: Gosh, that's not going to work",
"::error::Gosh, that's not going to work",
"::error::file.name:42:7: Gosh, that's not going to work",
"::error::main.cpp:12: no member named 'foo' in 'std::vector<int>'",
"::error::file.name:42:7: Gosh, that's not going to work",
"::error::a:b.go: 100%252C: still %25 escaped%0Aand multi-line",
}, },
}, },
{ {
@@ -146,6 +167,24 @@ func TestReporter_parseLogRow(t *testing.T) {
"*** bar baz ***", "*** bar baz ***",
}, },
}, },
{
// a token naming a real command must still resume
"stop-commands with a command-named token", false,
[]string{
"::stop-commands::add-mask",
"::set-output name=x::suppressed",
"::add-mask::",
"::add-mask::masked",
"masked",
},
[]string{
"<nil>",
"::set-output name=x::suppressed",
"<nil>",
"<nil>",
"***",
},
},
{ {
"unknown command", false, "unknown command", false,
[]string{ []string{
@@ -176,6 +215,19 @@ func TestReporter_parseLogRow(t *testing.T) {
} }
} }
// Both add-mask forms must register the secret and drop their own row: the runner forwards
// the raw line, so failing to consume it writes the secret straight to the job log.
func TestReporter_parseLogRowAddMask(t *testing.T) {
for _, line := range []string{"::add-mask::supersecret", "##[add-mask]supersecret"} {
r := &Reporter{logReplacer: strings.NewReplacer()}
assert.Nil(t, r.parseLogRow(&log.Entry{Message: line}), line)
row := r.parseLogRow(&log.Entry{Message: "using supersecret now"})
assert.Equal(t, "using *** now", row.Content, line)
}
}
func TestReporter_Fire(t *testing.T) { func TestReporter_Fire(t *testing.T) {
t.Run("ignore command lines", func(t *testing.T) { t.Run("ignore command lines", func(t *testing.T) {
client := mocks.NewClient(t) client := mocks.NewClient(t)
@@ -1010,7 +1062,7 @@ func TestReporter_Result(t *testing.T) {
} }
func TestReporter_SetOutputs(t *testing.T) { func TestReporter_SetOutputs(t *testing.T) {
r := &Reporter{state: &runnerv1.TaskState{}} r := &Reporter{state: &runnerv1.TaskState{}, logReplacer: strings.NewReplacer()}
r.SetOutputs(map[string]string{"foo": "bar"}) r.SetOutputs(map[string]string{"foo": "bar"})
got, ok := r.outputs["foo"] got, ok := r.outputs["foo"]
@@ -1096,3 +1148,24 @@ func TestReporter_ParseResult(t *testing.T) {
}) })
} }
} }
// A secret leaked in an encoded form — the shape it takes once an action puts it in a
// JSON body, a URL or a base64 payload — must be masked in the reported log as well.
func TestReporter_masksEncodedSecrets(t *testing.T) {
secret := `p@ss w"rd/1`
r := &Reporter{logReplacer: strings.NewReplacer()}
r.oldnew = runner.AppendSecretMasker(r.oldnew, secret)
r.logReplacer = strings.NewReplacer(r.oldnew...)
for _, line := range []string{
"token: " + secret,
"basic " + base64.StdEncoding.EncodeToString([]byte(secret)),
"https://example.com/?token=" + url.QueryEscape(secret),
} {
row := r.parseLogRow(&log.Entry{Message: line})
require.NotNil(t, row)
assert.Contains(t, row.Content, "***")
assert.NotContains(t, row.Content, secret)
assert.NotContains(t, row.Content, base64.StdEncoding.EncodeToString([]byte(secret)))
}
}