Compare commits

7 Commits
v3.0.1 ... main

Author SHA1 Message Date
silverwind
1d6c6ffef9 enhance: improve config, comment out values in example file (#1145)
`config.example.yaml` now has every value commented out, as gitea's `app.example.ini` does, so it documents each option with its default instead of imposing it. Copying it no longer pins values the runner would otherwise pick, and a changed default reaches configs that never named the option.

`config init` writes the file to configure: a header comment and no option, so every option keeps its default. It refuses to overwrite an existing config without `--force`, and writes `config.yaml` in the working directory when `-c` is absent.

`config set`, `add` and `remove` keep such a file readable. Comments go back to the indentation they were written at, which the encoder drops for a comment block that has no key below it, and a file of only comments keeps its text instead of being emptied by the first edit.

Also rewrote the config docs sections for clarity.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1145
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-08-06 05:15:32 +00:00
silverwind
70387cca44 chore: align go version handling with gitea (#1144)
Aligns Go version handling with gitea, see https://github.com/go-gitea/gitea/pull/38559. `toolchain` names the build version, `go` stays the minimum.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1144
Reviewed-by: techknowlogick <9+techknowlogick@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-08-05 22:50:11 +00:00
bircni
68547886a5 feat: wait for healthy services and fill the job context (#1107)
Service containers were started and then left alone, so a job's first step could run while a database was still starting up. The runner now waits for every service whose image or `options` declare a healthcheck, as GitHub does. An unhealthy service fails the job with its container log, one that never becomes healthy fails it after `container.service_ready_timeout` (default `5m`, negative disables the wait), and one that exits without a healthcheck only gets its log and a warning.

The started containers also fill the `job` context, whose fields existed but were never populated: `job.container.{id,network}` and `job.services.<id>.{id,network,ports}`. `ports` is keyed by the plain container port, so `job.services.postgres.ports['5432']` resolves to the host port Docker picked.

---------

Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1107
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-08-05 19:46:43 +00:00
bircni
8700adc933 feat: config command to edit config files (#1140)
Changing a setting after `generate-config` meant hand-editing YAML, which is awkward in provisioning scripts. `config` now edits an existing file in place:

```bash
./gitea-runner -c config.yaml config set runner.capacity 4
./gitea-runner -c config.yaml config set runner.envs.MY_VAR value
./gitea-runner -c config.yaml config add runner.labels 'ubuntu:docker://node:22'
./gitea-runner -c config.yaml config remove runner.labels 'ubuntu:docker://node:22'
./gitea-runner -c config.yaml config get runner.labels
```

Edits go through the YAML node tree, so comments, key order and the blank lines between top-level sections survive — a test asserts that appending a label to `config.example.yaml` changes nothing but the added line. Keys are resolved by reflecting over the `Config` struct's yaml tags, so an unknown key or a value of the wrong type is rejected before the file is touched. The write is atomic and keeps the file's symlink, owner, mode and line endings.

---------

Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1140
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-08-05 16:48:25 +00:00
bircni
b70ff6893a feat: gate set-env/add-path and render annotation locations (#1109)
`::set-env::` and `::add-path::` let a step rewrite the environment of every later step from its own output, which the runner honoured silently. They are now refused, as GitHub has done since 2020, and `ACTIONS_ALLOW_UNSECURE_COMMANDS` opts back in per step or job. Support for that variable is new here too, and is the only opt-in, matching GitHub rather than adding a runner config key on top.

Annotations keep their source location: Gitea has no annotation store and its web UI strips command properties, so `::error file=main.go,line=12::msg` is rendered as `::error::main.go:12: msg`.

`DEVELOPMENT.md` writes down the log line encoding rules this relies on.

---------

Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1109
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-08-05 16:43:17 +00:00
silverwind
3618385b28 fix: serve the whole results service from the cache server (#1141)
`ACTIONS_RESULTS_URL` names one origin serving every `github.actions.results.api.v1` service. Gitea serves the artifact half and this runner the cache half, so announcing `ACTIONS_CACHE_SERVICE_V2` while that URL pointed at Gitea was a promise the environment could not keep, and `docker buildx` posted its cache calls at Gitea and got a 404.

The cache server now forwards the artifact half to the instance each job registers with, so it is the whole results service and jobs are pointed at it. The announcement follows, and the bundle patch follows the cache URL instead.

Also fixes three things no JavaScript client reached: camelCase in the v2 responses where the Go clients read proto names, the missing `x-ms-request-id` on blob uploads that panics buildkit, and `cache.external_server` passed through without the trailing slash the v1 client concatenates onto.

Tests run the real actions against the services they look for: `actions/cache` over both API versions, the artifact actions up and back down through the forwarding, and `setup-node`. The regression itself is covered by asserting that whatever a job is handed as `ACTIONS_RESULTS_URL` answers a cache service call.

Fixes https://gitea.com/gitea/runner/issues/1139

Reviewed-on: https://gitea.com/gitea/runner/pulls/1141
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
2026-08-03 16:41:53 +00:00
Renovate Bot
55a625f733 chore(deps): update dependencies (#1138)
Co-authored-by: Renovate Bot <renovate-bot@gitea.com>
2026-08-03 09:22:17 +00:00
57 changed files with 2942 additions and 433 deletions

View File

@@ -35,6 +35,7 @@ jobs:
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7
with:
go-version-file: "go.mod"
check-latest: true
- name: goreleaser
uses: goreleaser/goreleaser-action@f06c13b6b1a9625abc9e6e439d9c05a8f2190e94 # v7
with:
@@ -85,7 +86,7 @@ jobs:
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4
- name: Login to DockerHub
uses: docker/login-action@abd2ef45e78c5afb21d64d4ca52ee8550d9572c7 # v4
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

View File

@@ -27,6 +27,7 @@ jobs:
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7
with:
go-version-file: "go.mod"
check-latest: true
- name: Import GPG key
id: import_gpg
uses: crazy-max/ghaction-import-gpg@2dc316deee8e90f13e1a351ab510b4d5bc0c82cd # v7
@@ -88,7 +89,7 @@ jobs:
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4
- name: Login to DockerHub
uses: docker/login-action@abd2ef45e78c5afb21d64d4ca52ee8550d9572c7 # v4
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

View File

@@ -21,6 +21,7 @@ jobs:
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7
with:
go-version-file: 'go.mod'
check-latest: true
- name: prepare anonymous docker config
run: mkdir -p "$DOCKER_CONFIG" && echo '{}' > "$DOCKER_CONFIG/config.json"
# Pre-pull act/runner's two largest base images so a slow pull can't dominate `make test`;

View File

@@ -8,6 +8,7 @@
- Never force-push, amend, or squash unless asked. Use new commits and normal push for pull request updates
- Comments: write almost none, short and preferably same-line, explaining why for a future reader. Never narrate code, the change or the prompt. Preserve existing ones that still apply
- Add the current year into the copyright header of new `.go` files
- Read `DEVELOPMENT.md` for internals and conventions
- Ensure no trailing whitespace in edited files
- Run `make fmt` after `.go` edits, `make tidy` after `go.mod` edits, and `make checks` for the non-lint source checks
- Lint what changed with `make lint-go`, and `make lint-go-windows` for Windows and platform-split files

32
DEVELOPMENT.md Normal file
View File

@@ -0,0 +1,32 @@
# Development
## Job log line format
Gitea stores one log row per line and its web UI decodes the payload, so getting the encoding
wrong never fails a test here, it only shows up in the browser.
**A row cannot contain a real newline.** `FormatLog` rewrites `\n` to a literal backslash-n and
truncates at 64 KiB on a byte boundary.
**The payload of a line starting with a recognised prefix is decoded**, with the escape set
depending on the prefix:
| prefix | decodes |
| --- | --- |
| `##[error]` `##[warning]` `##[notice]` `##[debug]` `##[group]` `##[endgroup]` `##[add-matcher]` | `%25` `%0D` `%0A` `%3B` `%5D` |
| `::error::` `::warning::` `::notice::` `::debug::` (with or without ` key=value` properties), `::group::` `::endgroup::` `::add-matcher::` | `%25` `%0D` `%0A` |
| `##[command]` `[command]`, or no recognised prefix | nothing |
### Rules
- **Emitting a command line?** Escape the payload with `runner.EscapeCommandData`. One escaper
covers both forms: it escapes `%` first, so a literal `%3B` becomes `%253B` that the extra
`##[…]` rules cannot match, and a raw `;` or `]` is never decoded. It is also what makes
multi-line work, `\n` becomes `%0A` and the UI turns it back into a line break.
- **Forwarding a command from step output?** Leave the payload alone, it arrived escaped and is
decoded once. Decoding here double-decodes and destroys multi-line.
- **No prefix?** Do not escape, and split multi-line values into one row each.
- **Interpolating a secret?** Masking runs after escaping, so `AppendSecretMasker` registers the
encoded forms too.
- Command *properties* also escape `%3A` and `%2C`, which the UI never decodes, so the reporter
decodes exactly those two when folding a location into an annotation.

View File

@@ -6,7 +6,7 @@ SHASUM ?= shasum -a 256
HAS_GO = $(shell hash $(GO) > /dev/null 2>&1 && echo "GO" || echo "NOGO" )
XGO_PACKAGE ?= src.techknowlogick.com/xgo@v1.9.0 # renovate: datasource=go
XGO_VERSION := go-1.26.x
GXZ_PACKAGE ?= github.com/ulikunitz/xz/cmd/gxz@v0.5.15 # renovate: datasource=go
GXZ_PACKAGE ?= github.com/ulikunitz/xz/cmd/gxz@v0.5.16 # renovate: datasource=go
LINUX_ARCHS ?= linux/amd64,linux/arm64
DARWIN_ARCHS ?= darwin-12/amd64,darwin-12/arm64
@@ -19,7 +19,7 @@ DOCKER_REF := $(DOCKER_IMAGE):$(DOCKER_TAG)
DOCKER_ROOTLESS_REF := $(DOCKER_IMAGE):$(DOCKER_TAG)-dind-rootless
GOLANGCI_LINT_PACKAGE ?= github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.12.2 # renovate: datasource=go
GOVULNCHECK_PACKAGE ?= golang.org/x/vuln/cmd/govulncheck@v1.3.0 # renovate: datasource=go
GOVULNCHECK_PACKAGE ?= golang.org/x/vuln/cmd/govulncheck@v1.6.0 # renovate: datasource=go
GOTEST_FLAGS ?= -race -timeout 20m -parallel 8
@@ -141,7 +141,12 @@ security-check:
.PHONY: tidy
tidy: ## run go mod tidy
$(eval GO_TOOLCHAIN := $(shell grep -Eo '^toolchain\s+go[0-9.]+' go.mod | cut -d' ' -f2))
$(GO) mod tidy
@# workaround https://github.com/golang/go/issues/75331: restore toolchain if tidy dropped it
@if [ -n "$(GO_TOOLCHAIN)" ] && ! grep -qE '^toolchain\s' go.mod; then \
$(GO) mod edit -toolchain=$(GO_TOOLCHAIN); \
fi
.PHONY: tidy-check
tidy-check: tidy

View File

@@ -129,27 +129,38 @@ Same idea as `dind`, but built on `docker:dind-rootless` so the bundled daemon a
### Configuration
The runner is configured with a YAML file. Generate a starting point (this matches what ships in the tree):
The runner reads a YAML file. Without one, every option keeps its default.
```bash
./gitea-runner generate-config > config.yaml
./gitea-runner config init # write config.yaml, with no option set
./gitea-runner config generate | less # read what the options do
./gitea-runner -c config.yaml daemon # -c also works on register and cache-server
```
Pass it with `-c` / `--config` on any command that loads configuration (`register`, `daemon`, `cache-server`):
`config generate` prints [config.example.yaml](internal/pkg/config/config.example.yaml). Every value in it is commented out, so copy the lines you want to change into your own file and uncomment them.
#### Editing a config file
`config` edits a file in place, which is handy in provisioning scripts:
```bash
./gitea-runner -c config.yaml register
./gitea-runner -c config.yaml daemon
./gitea-runner -c config.yaml cache-server
./gitea-runner config set runner.capacity 4
./gitea-runner config set runner.timeout 90m # written as 1h30m0s
./gitea-runner config set runner.envs.MY_VAR value
./gitea-runner config add runner.labels 'ubuntu:docker://node:22'
./gitea-runner config remove runner.labels 'ubuntu:docker://node:22'
./gitea-runner config get runner.labels
```
Every option is described in [config.example.yaml](internal/pkg/config/config.example.yaml) (the same content `generate-config` prints).
A key is its dotted YAML path. An unknown key, a value of the wrong type, or `add`/`remove` on anything but a list is refused before the file is touched. `set` replaces a whole list when you give it several values.
#### Without a config file
An edit keeps the comments and the key order of the file. Indentation becomes two spaces, and a blank line between two values is dropped.
If you omit `-c`, built-in defaults apply (same as an empty YAML document).
`config get`, `set`, `add` and `remove` use `config.yaml` (or `config.yml`) from the working directory, then from the directory of the binary, and print their choice to stderr. `config init` writes `config.yaml` in the working directory, and refuses to overwrite an existing config without `--force`. Pass `-c` for another path.
Earlier releases let a small set of environment variables (`GITEA_DEBUG`, `GITEA_TRACE`, `GITEA_RUNNER_CAPACITY`, `GITEA_RUNNER_FILE`, `GITEA_RUNNER_ENVIRON`, `GITEA_RUNNER_ENV_FILE`) override parts of the default config. Those overrides have been removed — use a YAML config file for all settings instead. For the Docker images, the entrypoint still understands a separate set of variables (such as `RUNNER_STATE_FILE`); see [scripts/run.sh](scripts/run.sh) and the container documentation below.
#### Environment variables
Earlier releases let a few environment variables (`GITEA_DEBUG`, `GITEA_TRACE`, `GITEA_RUNNER_CAPACITY`, `GITEA_RUNNER_FILE`, `GITEA_RUNNER_ENVIRON`, `GITEA_RUNNER_ENV_FILE`) override parts of the config. They are gone, use the YAML file for all settings. The Docker images still read their own variables, such as `RUNNER_STATE_FILE`, see [scripts/run.sh](scripts/run.sh) and the container documentation below.
### Labels
@@ -209,6 +220,26 @@ Whenever the resulting labels differ from the ones in the registration file, the
> **Note:** A runner that only exposes `host` labels still needs access to a Docker daemon (e.g. a mounted `/var/run/docker.sock`) whenever a job uses a `docker://` action or a service container. `host` labels only change where the job's own steps run; container-based steps and actions are still executed with Docker.
#### Service containers
A job's `services` are started before its steps run. When a service's image or its `options` declare a healthcheck, the runner waits for it to report healthy, so a workflow does not have to poll for its own services:
```yaml
services:
postgres:
image: postgres:17
options: >-
--health-cmd pg_isready
--health-interval 5s
--health-retries 10
```
A service that reports unhealthy fails the job right away, with its container log. One that never becomes healthy fails it after `container.service_ready_timeout` (default `5m`, negative disables the wait). A service that exits without declaring a healthcheck only gets its log and a warning.
A job in a container reaches a service by its id on the job network, on the port the service listens on, for example `psql -h postgres -p 5432`. The started containers also fill the `job` context: `job.container.{id,network}` and `job.services.<id>.{id,network,ports}`, where `ports` maps a container port to the host port Docker published it on, for the services that publish one.
Unlike GitHub, a job whose steps run on the host (a `host` label without `container:`) starts no service containers, so `job.services` and `job.container` stay empty. Give such a job a `container:` when it needs services.
#### Proxy
Set these variables in the runner's environment, with systemd `Environment=`, `docker run -e`, or Kubernetes `env:`:

View File

@@ -52,7 +52,13 @@ type credKey struct{}
// poison another repo's cache, even from inside a container that reaches the
// cache server over the docker bridge network.
type JobCredential struct {
Repo string
Repo string `json:"repo"`
// Results is the instance whose artifact service this server forwards for the job, and
// InsecureTLS how the runner reaches it; see results.go. The tags are the wire format a
// remote runner registers with.
Results string `json:"results"`
InsecureTLS bool `json:"insecure_tls"`
}
// credEntry holds a registered job's credential along with an active
@@ -165,6 +171,7 @@ func StartHandler(dir, outboundIP string, port uint16, internalSecret string, lo
router.POST(internalPath+"/register", h.internalAuth(h.internalRegister))
router.POST(internalPath+"/revoke", h.internalAuth(h.internalRevoke))
h.registerV2Routes(router)
router.NotFound = http.HandlerFunc(h.forwardOrNotFound)
h.router = router
@@ -211,10 +218,11 @@ func (h *Handler) ExternalURL() string {
// is only accepted while the job is running.
//
// Registrations are reference-counted: if a token is already registered, the
// existing repo is kept and the refcount is incremented. The entry is
// removed only when every revoker returned by RegisterJob has been called.
// credential it was registered with is kept and the refcount is incremented.
// The entry is removed only when every revoker returned by RegisterJob has
// been called.
// This keeps a stray re-registration from silently revoking a live job.
func (h *Handler) RegisterJob(token, repo string) func() {
func (h *Handler) RegisterJob(token string, cred JobCredential) func() {
if h == nil || token == "" {
return func() {}
}
@@ -223,7 +231,7 @@ func (h *Handler) RegisterJob(token, repo string) func() {
existing.refs++
} else {
h.creds[token] = &credEntry{
cred: JobCredential{Repo: repo},
cred: cred,
refs: 1,
}
}
@@ -619,7 +627,7 @@ func (h *Handler) internalAuth(handler httprouter.Handle) httprouter.Handle {
type internalRegisterBody struct {
Token string `json:"token"`
Repo string `json:"repo"`
JobCredential
}
type internalRevokeBody struct {
@@ -627,6 +635,15 @@ type internalRevokeBody struct {
}
// POST /_internal/register
// ResultsURL is what a job registered with cred should be given as ACTIONS_RESULTS_URL, or "" when
// the credential names no instance to forward the artifact half to.
func (h *Handler) ResultsURL(cred JobCredential) string {
if h == nil || cred.Results == "" {
return ""
}
return h.ExternalURL()
}
func (h *Handler) internalRegister(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
var body internalRegisterBody
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
@@ -637,8 +654,9 @@ func (h *Handler) internalRegister(w http.ResponseWriter, r *http.Request, _ htt
h.responseJSON(w, r, http.StatusBadRequest, errors.New("token is required"))
return
}
h.RegisterJob(body.Token, body.Repo)
h.responseJSON(w, r, http.StatusOK)
h.RegisterJob(body.Token, body.JobCredential)
// A server too old to forward answers without this, which is how the caller knows.
h.responseJSON(w, r, http.StatusOK, map[string]any{"results_url": h.ResultsURL(body.JobCredential)})
}
// POST /_internal/revoke

View File

@@ -52,7 +52,7 @@ func TestHandler(t *testing.T) {
dir := filepath.Join(t.TempDir(), "artifactcache")
handler, err := StartHandler(dir, "", 0, "", nil)
require.NoError(t, err)
handler.RegisterJob(testToken, testRepo)
handler.RegisterJob(testToken, JobCredential{Repo: testRepo})
base := fmt.Sprintf("%s%s", handler.ExternalURL(), apiPath)
@@ -890,7 +890,7 @@ func TestHandler_UnregisterRevokes(t *testing.T) {
require.NoError(t, err)
defer handler.Close()
unregister := handler.RegisterJob("tmp-token", testRepo)
unregister := handler.RegisterJob("tmp-token", JobCredential{Repo: testRepo})
base := handler.ExternalURL() + apiPath
req, err := http.NewRequest(http.MethodGet, base+"/cache?keys=x&version=y", nil)
@@ -920,8 +920,8 @@ func TestHandler_CrossRepoIsolation(t *testing.T) {
handler, err := StartHandler(dir, "", 0, "", nil)
require.NoError(t, err)
defer handler.Close()
handler.RegisterJob("token-a", "owner/repoA")
handler.RegisterJob("token-b", "owner/repoB")
handler.RegisterJob("token-a", JobCredential{Repo: "owner/repoA"})
handler.RegisterJob("token-b", JobCredential{Repo: "owner/repoB"})
base := handler.ExternalURL() + apiPath
key := "shared-key"
@@ -986,7 +986,7 @@ func TestHandler_ArtifactSignature(t *testing.T) {
handler, err := StartHandler(dir, "", 0, "", nil)
require.NoError(t, err)
defer handler.Close()
handler.RegisterJob(testToken, testRepo)
handler.RegisterJob(testToken, JobCredential{Repo: testRepo})
base := handler.ExternalURL() + apiPath
@@ -1059,7 +1059,7 @@ func TestHandler_ArtifactSignatureDownload(t *testing.T) {
handler, err := StartHandler(dir, "", 0, "", nil)
require.NoError(t, err)
defer handler.Close()
handler.RegisterJob(testToken, testRepo)
handler.RegisterJob(testToken, JobCredential{Repo: testRepo})
base := handler.ExternalURL() + apiPath
key := "download-key"
@@ -1100,8 +1100,8 @@ func TestHandler_RegisterJob_RefCounted(t *testing.T) {
require.NoError(t, err)
defer handler.Close()
first := handler.RegisterJob("shared", testRepo)
second := handler.RegisterJob("shared", testRepo)
first := handler.RegisterJob("shared", JobCredential{Repo: testRepo})
second := handler.RegisterJob("shared", JobCredential{Repo: testRepo})
base := handler.ExternalURL() + apiPath
probe := func() int {
@@ -1131,8 +1131,8 @@ func TestHandler_GC_PerRepoDedup(t *testing.T) {
handler, err := StartHandler(dir, "", 0, "", nil)
require.NoError(t, err)
defer handler.Close()
handler.RegisterJob("tok-a", "owner/repoA")
handler.RegisterJob("tok-b", "owner/repoB")
handler.RegisterJob("tok-a", JobCredential{Repo: "owner/repoA"})
handler.RegisterJob("tok-b", JobCredential{Repo: "owner/repoB"})
key := "shared-dedup-key"
version := "c19da02a2bd7e77277f1ac29ab45c09b7d46a4ee758284e26bb3045ad11d9d20"

View File

@@ -23,6 +23,9 @@ import (
// endpoints, and uploads the archive to the returned URL with the Azure blob protocol.
// Both API versions are served from the same store, so a repository keeps its cache
// when a workflow moves between action versions.
//
// Responses carry the proto field names, which is what Gitea's own results API emits and the only
// spelling the Go clients parse. The JavaScript toolkit accepts either.
const (
cacheServiceV2Path = "/twirp/github.actions.results.api.v1.CacheService"
@@ -93,8 +96,8 @@ func (h *Handler) v2CreateCacheEntry(w http.ResponseWriter, r *http.Request, _ h
}
h.responseJSON(w, r, http.StatusOK, map[string]any{
"ok": true,
"signedUploadUrl": h.signedURL(blobPath, blobUploadPurpose, cache.ID, time.Now().Add(blobUploadURLTTL)),
"ok": true,
"signed_upload_url": h.signedURL(blobPath, blobUploadPurpose, cache.ID, time.Now().Add(blobUploadURLTTL)),
})
}
@@ -134,7 +137,7 @@ func (h *Handler) v2FinalizeCacheEntryUpload(w http.ResponseWriter, r *http.Requ
h.responseJSON(w, r, http.StatusOK, map[string]any{
"ok": true,
// int64 fields travel as strings in the proto JSON mapping.
"entryId": strconv.FormatUint(cache.ID, 10),
"entry_id": strconv.FormatUint(cache.ID, 10),
})
}
@@ -164,9 +167,9 @@ func (h *Handler) v2GetCacheEntryDownloadURL(w http.ResponseWriter, r *http.Requ
}
h.responseJSON(w, r, http.StatusOK, map[string]any{
"ok": true,
"signedDownloadUrl": h.signedArtifactURL(cache.ID, time.Now().Add(artifactURLTTL)),
"matchedKey": cache.Key,
"ok": true,
"signed_download_url": h.signedArtifactURL(cache.ID, time.Now().Add(artifactURLTTL)),
"matched_key": cache.Key,
})
}
@@ -209,6 +212,8 @@ func (h *Handler) v2UploadBlob(w http.ResponseWriter, r *http.Request, params ht
return
}
// The Azure SDK client dereferences this without checking, so its absence panics the caller.
w.Header().Set("x-ms-request-id", strconv.FormatInt(time.Now().UnixNano(), 10))
w.WriteHeader(http.StatusCreated)
}

View File

@@ -45,6 +45,11 @@ func putBlob(t *testing.T, url string, content []byte) int {
resp, err := http.DefaultClient.Do(req)
require.NoError(t, err)
defer resp.Body.Close()
if resp.StatusCode == http.StatusCreated {
// The Azure SDK client dereferences this header without checking, so a blob upload that
// omits it panics the caller rather than failing it.
require.NotEmpty(t, resp.Header.Get("x-ms-request-id"))
}
return resp.StatusCode
}
@@ -67,7 +72,7 @@ func startTestHandler(t *testing.T) *Handler {
handler, err := StartHandler(filepath.Join(t.TempDir(), "artifactcache"), "127.0.0.1", 0, "", nil)
require.NoError(t, err)
t.Cleanup(func() { _ = handler.Close() })
handler.RegisterJob(testToken, testRepo)
handler.RegisterJob(testToken, JobCredential{Repo: testRepo})
return handler
}
@@ -78,7 +83,7 @@ func saveV2(t *testing.T, handler *Handler, key, version string, content []byte)
created := v2Call(t, handler, testClient, "CreateCacheEntry", map[string]any{"key": key, "version": version})
require.Equal(t, true, created["ok"])
uploadURL, _ = created["signedUploadUrl"].(string)
uploadURL, _ = created["signed_upload_url"].(string)
require.NotEmpty(t, uploadURL)
require.Equal(t, http.StatusCreated, putBlob(t, uploadURL, content))
@@ -100,7 +105,7 @@ func TestCacheServiceV2RoundTrip(t *testing.T) {
finalized, uploadURL := saveV2(t, handler, "deps-v1", "abc123", content)
require.Equal(t, true, finalized["ok"])
assert.NotEmpty(t, finalized["entryId"])
assert.NotEmpty(t, finalized["entry_id"])
// The upload URL outlives the finalize call, so replaying it must not poison the entry,
// and it is an upload URL only: nothing reads a blob back through it.
@@ -112,8 +117,8 @@ func TestCacheServiceV2RoundTrip(t *testing.T) {
got := v2Call(t, handler, testClient, "GetCacheEntryDownloadURL", map[string]any{"key": "deps-v1", "version": "abc123"})
require.Equal(t, true, got["ok"])
assert.Equal(t, "deps-v1", got["matchedKey"])
downloadURL, _ := got["signedDownloadUrl"].(string)
assert.Equal(t, "deps-v1", got["matched_key"])
downloadURL, _ := got["signed_download_url"].(string)
require.NotEmpty(t, downloadURL)
assert.Equal(t, content, getURL(t, downloadURL))
}
@@ -124,7 +129,7 @@ func TestCacheServiceV2BlockUpload(t *testing.T) {
handler := startTestHandler(t)
created := v2Call(t, handler, testClient, "CreateCacheEntry", map[string]any{"key": "blocks", "version": "v1"})
uploadURL, _ := created["signedUploadUrl"].(string)
uploadURL, _ := created["signed_upload_url"].(string)
require.NotEmpty(t, uploadURL)
blocks := map[string][]byte{}
@@ -154,7 +159,7 @@ func TestCacheServiceV2BlockUpload(t *testing.T) {
got := v2Call(t, handler, testClient, "GetCacheEntryDownloadURL", map[string]any{"key": "blocks", "version": "v1"})
require.Equal(t, true, got["ok"])
assert.Equal(t, "hello world!", string(getURL(t, got["signedDownloadUrl"].(string))))
assert.Equal(t, "hello world!", string(getURL(t, got["signed_download_url"].(string))))
}
func TestCacheServiceV2Lookups(t *testing.T) {
@@ -175,7 +180,7 @@ func TestCacheServiceV2Lookups(t *testing.T) {
"key": "deps-zzz", field: []string{"deps-"}, "version": "v1",
})
require.Equal(t, true, got["ok"])
assert.Equal(t, "deps-abc", got["matchedKey"])
assert.Equal(t, "deps-abc", got["matched_key"])
})
}
@@ -190,7 +195,7 @@ func TestCacheServiceV2Lookups(t *testing.T) {
t.Run("a prefix of an existing key is still reserved", func(t *testing.T) {
reserved := v2Call(t, handler, testClient, "CreateCacheEntry", map[string]any{"key": "deps", "version": "v1"})
require.Equal(t, true, reserved["ok"])
assert.NotEmpty(t, reserved["signedUploadUrl"])
assert.NotEmpty(t, reserved["signed_upload_url"])
})
t.Run("finalizing without a reservation is not ok", func(t *testing.T) {
@@ -203,7 +208,7 @@ func TestCacheServiceV2Lookups(t *testing.T) {
// The size the client declares is what Commit validates the assembled archive against.
t.Run("finalizing with the wrong size is not ok", func(t *testing.T) {
created := v2Call(t, handler, testClient, "CreateCacheEntry", map[string]any{"key": "wrong-size", "version": "v1"})
require.Equal(t, http.StatusCreated, putBlob(t, created["signedUploadUrl"].(string), []byte("four")))
require.Equal(t, http.StatusCreated, putBlob(t, created["signed_upload_url"].(string), []byte("four")))
got := v2Call(t, handler, testClient, "FinalizeCacheEntryUpload", map[string]any{
"key": "wrong-size", "version": "v1", "size_bytes": 99,
@@ -227,7 +232,7 @@ func TestCacheServiceV2Lookups(t *testing.T) {
// The cache of one repository must stay invisible to another, as it does for the v1 API.
t.Run("another repository sees nothing", func(t *testing.T) {
handler.RegisterJob("other-runtime-token", "other/repo")
handler.RegisterJob("other-runtime-token", JobCredential{Repo: "other/repo"})
otherClient := &http.Client{Transport: &bearerTransport{token: "other-runtime-token"}}
got := v2Call(t, handler, otherClient, "GetCacheEntryDownloadURL", map[string]any{"key": "deps-abc", "version": "v1"})

View File

@@ -0,0 +1,59 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package artifactcache
import (
"crypto/tls"
"net/http"
"net/http/httputil"
"net/url"
"strings"
)
// The results service is one origin serving every github.actions.results.api.v1 service, and
// Gitea implements only the artifact half of it. Forwarding that half from here makes this origin
// the whole service, so ACTIONS_RESULTS_URL can point at it truthfully, which is what the clients
// this runner cannot patch need, docker buildx among them.
//
// The instance to forward to travels with the job registration rather than with configuration, so
// a cache server shared between runners serves each of their instances.
const artifactServicePath = "/twirp/github.actions.results.api.v1.ArtifactService/"
// forwardOrNotFound is the router's fallback: the artifact service of the instance the job
// registered with, and the 404 the router would have written otherwise.
func (h *Handler) forwardOrNotFound(w http.ResponseWriter, r *http.Request) {
cred, ok := h.lookupCredential(bearerToken(r))
if !ok || cred.Results == "" || !strings.HasPrefix(r.URL.Path, artifactServicePath) {
http.NotFound(w, r)
return
}
target, err := url.Parse(strings.TrimSuffix(cred.Results, "/"))
if err != nil {
h.logger.Errorf("artifact service forward to %q: %v", cred.Results, err)
w.WriteHeader(http.StatusBadGateway)
return
}
h.logger.Debugf("%s %s: forwarding to %s", r.Method, r.URL.Path, target)
proxy := &httputil.ReverseProxy{
Rewrite: func(r *httputil.ProxyRequest) {
r.SetURL(target)
// Gitea builds the URLs it hands back from this Host, and their scheme from the
// connection unless a forwarded header overrides it, so artifact bodies go to Gitea
// directly and never through here.
r.Out.Host = target.Host
},
ErrorHandler: func(w http.ResponseWriter, _ *http.Request, err error) {
h.logger.Warnf("artifact service forward to %s: %v", target, err)
w.WriteHeader(http.StatusBadGateway)
},
}
if cred.InsecureTLS {
proxy.Transport = insecureTransport
}
proxy.ServeHTTP(w, r)
}
// insecureTransport is shared, because a transport per request would pool no connections.
var insecureTransport = &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}} //nolint:gosec // the runner reaches its instance on the operator's say-so

View File

@@ -0,0 +1,56 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package artifactcache
import (
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// The artifact half is forwarded under the Host Gitea knows itself by, so the URLs it hands back
// still point at Gitea, and nothing else is proxied.
func TestFrontResultsService(t *testing.T) {
var gotHost, gotPath, gotProto string
gitea := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotHost, gotPath, gotProto = r.Host, r.URL.Path, r.Header.Get("X-Forwarded-Proto")
_, _ = io.WriteString(w, `{"ok":true}`)
}))
defer gitea.Close()
handler, err := StartHandler(t.TempDir(), "127.0.0.1", 0, "", nil)
require.NoError(t, err)
defer handler.Close()
const token = "forward-token"
client := &http.Client{Transport: &bearerTransport{token: token}}
post := func(path string) int {
req, err := http.NewRequestWithContext(t.Context(), http.MethodPost, handler.ExternalURL()+path, nil)
require.NoError(t, err)
resp, err := client.Do(req)
require.NoError(t, err)
resp.Body.Close()
return resp.StatusCode
}
assert.Equal(t, http.StatusNotFound, post(artifactServicePath+"CreateArtifact"),
"an unregistered token is forwarded nowhere")
defer handler.RegisterJob(token, JobCredential{Repo: "owner/repo", Results: gitea.URL})()
assert.Equal(t, http.StatusOK, post(artifactServicePath+"CreateArtifact"))
assert.Equal(t, strings.TrimPrefix(gitea.URL, "http://"), gotHost, "Gitea must see the host it mints its URLs from")
assert.Empty(t, gotProto, "a forwarded scheme would make an https Gitea mint http URLs")
assert.Equal(t, artifactServicePath+"CreateArtifact", gotPath)
gotPath = ""
assert.Equal(t, http.StatusNotFound, post("/twirp/github.actions.results.api.v1.OtherService/Do"))
assert.Equal(t, http.StatusNotFound, post("/api/v1/repos/owner/repo"))
assert.Empty(t, gotPath, "only the artifact service is forwarded")
}

View File

@@ -6,12 +6,14 @@ package container
import (
"context"
"errors"
"fmt"
"io"
"gitea.com/gitea/runner/act/common"
"github.com/docker/go-connections/nat"
"github.com/moby/moby/api/types/container"
)
// ExitCodeError reports a non-zero process exit code from a container command.
@@ -57,6 +59,32 @@ type FileEntry struct {
Body string
}
// Container and healthcheck states, as plain strings so a caller of Info needs no docker
// SDK of its own.
const (
StateRunning = string(container.StateRunning)
HealthNone = string(container.NoHealthcheck)
HealthStarting = string(container.Starting)
HealthHealthy = string(container.Healthy)
HealthUnhealthy = string(container.Unhealthy)
)
// ErrContainerNotFound reports a container the daemon no longer knows. Its text is a
// fragment, missingContainerError composes it into the message every operation shares.
var ErrContainerNotFound = errors.New("does not exist")
// Info is a snapshot of a container, as of one inspect.
type Info struct {
ID string
State string // the docker container state: "created", "running", "exited", ...
ExitCode int
Health string // one of the Health* constants
// HealthOutput is the last healthcheck probe's output.
HealthOutput string
Ports map[string]string // container port ("5432") to the host port it is published on
}
// Container for managing docker run containers
type Container interface {
Create(capAdd, capDrop []string) common.Executor
@@ -65,6 +93,8 @@ type Container interface {
CopyTarStream(ctx context.Context, destPath string, tarStream io.Reader) error
CopyDir(destPath, srcPath string, useGitIgnore bool) common.Executor
GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error)
Inspect(ctx context.Context) (*Info, error)
DumpLogs(ctx context.Context) error
Pull(forcePull bool) common.Executor
Start(attach bool) common.Executor
Exec(command []string, env map[string]string, user, workdir string) common.Executor

View File

@@ -198,6 +198,109 @@ func (cr *containerReference) GetContainerArchive(ctx context.Context, srcPath s
return result.Content, nil
}
// Inspect resolves the container by name when its id is not known yet. One the daemon no
// longer knows is reported as ErrContainerNotFound.
func (cr *containerReference) Inspect(ctx context.Context) (*Info, error) {
if common.Dryrun(ctx) {
return &Info{Health: HealthNone, Ports: map[string]string{}}, nil
}
if err := cr.connect()(ctx); err != nil {
return nil, err
}
if cr.id == "" { // a known id is trusted, find() would spend a call validating it
if err := cr.find()(ctx); err != nil {
return nil, err
}
}
if cr.id == "" {
return nil, cr.missingContainerError("inspect it")
}
result, err := cr.cli.ContainerInspect(ctx, cr.id, client.ContainerInspectOptions{})
if cerrdefs.IsNotFound(err) {
return nil, cr.missingContainerError("inspect it")
} else if err != nil {
return nil, err
}
return containerInfoFromInspect(result.Container), nil
}
// DumpLogs copies the container's log so far to its output writers.
func (cr *containerReference) DumpLogs(ctx context.Context) error {
if common.Dryrun(ctx) {
return nil
}
if err := cr.connect()(ctx); err != nil {
return err
}
if cr.id == "" {
return cr.missingContainerError("read its logs")
}
logs, err := cr.cli.ContainerLogs(ctx, cr.id, client.ContainerLogsOptions{ShowStdout: true, ShowStderr: true})
if err != nil {
return err
}
defer logs.Close()
return cr.copyOutput(logs)
}
// copyOutput writes a container stream to the writers the container was created with,
// demultiplexing it unless the container has a TTY, which sends a single raw stream.
func (cr *containerReference) copyOutput(stream io.Reader) error {
outWriter := cr.input.Stdout
if outWriter == nil {
outWriter = os.Stdout
}
errWriter := cr.input.Stderr
if errWriter == nil {
errWriter = os.Stderr
}
var err error
if !cr.input.AllocatePTY || os.Getenv("NORAW") != "" {
_, err = stdcopy.StdCopy(outWriter, errWriter, stream)
} else {
_, err = io.Copy(outWriter, stream)
}
// Flush any buffered, not-yet-newline-terminated trailing line so the final line of
// the output is not lost when it is not newline-terminated.
common.FlushWriter(outWriter)
common.FlushWriter(errWriter)
return err
}
func containerInfoFromInspect(inspect container.InspectResponse) *Info {
info := &Info{
ID: inspect.ID,
Health: HealthNone,
Ports: map[string]string{}, // an empty map, never null, in the expression context
}
if state := inspect.State; state != nil {
info.State = string(state.Status)
info.ExitCode = state.ExitCode
if health := state.Health; health != nil {
info.Health = string(health.Status)
if len(health.Log) > 0 {
info.HealthOutput = strings.TrimSpace(health.Log[len(health.Log)-1].Output)
}
}
}
if settings := inspect.NetworkSettings; settings != nil {
for port, bindings := range settings.Ports {
for _, binding := range bindings { // the last binding wins, a port maps to one host port
if binding.HostPort != "" {
info.Ports[port.Port()] = binding.HostPort
}
}
}
}
return info
}
func (cr *containerReference) UpdateFromEnv(srcPath string, env *map[string]string) common.Executor {
return parseEnvFile(cr, srcPath, env).IfNot(common.Dryrun)
}
@@ -343,10 +446,10 @@ func (cr *containerReference) Close() common.Executor {
}
}
// missingContainerError is the shared "container X does not exist" error
// used by ops that need a live cr.id.
// missingContainerError is the shared "container X does not exist" error used by ops that
// need a live cr.id, wrapping ErrContainerNotFound so a caller can tell it from a failing daemon.
func (cr *containerReference) missingContainerError(format string, args ...any) error {
return fmt.Errorf("container %q does not exist; cannot "+format, append([]any{cr.input.Name}, args...)...)
return fmt.Errorf("container %q %w; cannot "+format, append([]any{cr.input.Name, ErrContainerNotFound}, args...)...)
}
func (cr *containerReference) find() common.Executor {
@@ -737,7 +840,7 @@ func (cr *containerReference) exec(cmd []string, env map[string]string, user, wo
}
defer resp.Close()
err = cr.waitForCommand(ctx, isTerminal, resp.HijackedResponse, idResp, user, workdir)
err = cr.waitForCommand(ctx, resp.HijackedResponse, idResp, user, workdir)
if err != nil {
return err
}
@@ -795,7 +898,7 @@ func (cr *containerReference) tryReadGID() common.Executor {
return cr.tryReadID("-g", func(id int) { cr.GID = id })
}
func (cr *containerReference) waitForCommand(ctx context.Context, isTerminal bool, resp client.HijackedResponse, _ client.ExecCreateResult, _, _ string) error {
func (cr *containerReference) waitForCommand(ctx context.Context, resp client.HijackedResponse, _ client.ExecCreateResult, _, _ string) error {
logger := common.Logger(ctx)
// Buffered so the copy goroutine never blocks on send if the grace-period
@@ -803,28 +906,7 @@ func (cr *containerReference) waitForCommand(ctx context.Context, isTerminal boo
cmdResponse := make(chan error, 1)
go func() {
var outWriter io.Writer
outWriter = cr.input.Stdout
if outWriter == nil {
outWriter = os.Stdout
}
errWriter := cr.input.Stderr
if errWriter == nil {
errWriter = os.Stderr
}
var err error
if !isTerminal || os.Getenv("NORAW") != "" {
_, err = stdcopy.StdCopy(outWriter, errWriter, resp.Reader)
} else {
_, err = io.Copy(outWriter, resp.Reader)
}
// Flush any buffered, not-yet-newline-terminated trailing line so the
// final line of a command's output is not lost (e.g. an error message
// printed without a trailing newline before the process exits).
common.FlushWriter(outWriter)
common.FlushWriter(errWriter)
cmdResponse <- err
cmdResponse <- cr.copyOutput(resp.Reader)
}()
select {
@@ -1059,33 +1141,11 @@ func (cr *containerReference) attach() common.Executor {
if err != nil {
return fmt.Errorf("failed to attach to container: %w", err)
}
isTerminal := cr.input.AllocatePTY
var outWriter io.Writer
outWriter = cr.input.Stdout
if outWriter == nil {
outWriter = os.Stdout
}
errWriter := cr.input.Stderr
if errWriter == nil {
errWriter = os.Stderr
}
done := make(chan struct{})
cr.attachDone = done
go func() {
defer close(done)
var copyErr error
if !isTerminal || os.Getenv("NORAW") != "" {
_, copyErr = stdcopy.StdCopy(outWriter, errWriter, out.Reader)
} else {
_, copyErr = io.Copy(outWriter, out.Reader)
}
// Flush any buffered, not-yet-newline-terminated trailing line once
// the stream reaches EOF, so the final line of the container's
// output is not lost when it is not newline-terminated.
common.FlushWriter(outWriter)
common.FlushWriter(errWriter)
if copyErr != nil {
if copyErr := cr.copyOutput(out.Reader); copyErr != nil {
common.Logger(ctx).Error(copyErr)
}
}()

View File

@@ -25,6 +25,7 @@ import (
"github.com/moby/moby/api/pkg/stdcopy"
"github.com/moby/moby/api/types/container"
"github.com/moby/moby/api/types/mount"
"github.com/moby/moby/api/types/network"
mobyclient "github.com/moby/moby/client"
"github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
@@ -548,7 +549,7 @@ func TestRejectsMissingContainer(t *testing.T) {
cr := &containerReference{cli: client, input: &NewContainerInput{Name: "job-1"}}
check := func(op string, err error) {
t.Helper()
require.Error(t, err, op)
require.ErrorIs(t, err, ErrContainerNotFound, op)
assert.Contains(t, err.Error(), `container "job-1" does not exist`, op)
}
check("copyContent", cr.copyContent("/var/run/act", &FileEntry{Name: "x", Mode: 0o644})(ctx))
@@ -557,6 +558,15 @@ func TestRejectsMissingContainer(t *testing.T) {
check("exec", cr.exec([]string{"echo"}, nil, "", "")(ctx))
_, err := cr.GetContainerArchive(ctx, "/var/run/act/x")
check("GetContainerArchive", err)
_, err = cr.Inspect(ctx)
check("Inspect", err)
// a known id the daemon has since dropped
client.On("ContainerInspect", ctx, "gone", mobyclient.ContainerInspectOptions{}).
Return(mobyclient.ContainerInspectResult{}, cerrdefs.ErrNotFound)
removed := &containerReference{id: "gone", cli: client, input: &NewContainerInput{Name: "job-1"}}
_, err = removed.Inspect(ctx)
check("Inspect after removal", err)
}
// End-to-end: a stale cr.id is cleared, repopulated from name lookup,
@@ -825,6 +835,59 @@ func TestCheckVolumesRejectsEscapingHostPaths(t *testing.T) {
assert.Empty(t, hostConf.Binds)
}
func TestContainerInfoFromInspect(t *testing.T) {
t.Run("reports no healthcheck when the image declares none", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{
ID: "abc123",
State: &container.State{Status: "running", Running: true},
})
assert.Equal(t, "abc123", info.ID)
assert.Equal(t, "running", info.State)
assert.Equal(t, HealthNone, info.Health)
assert.Empty(t, info.Ports)
})
t.Run("reports the health status and the last probe output", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{
State: &container.State{
Status: "running",
Health: &container.Health{
Status: container.Unhealthy,
Log: []*container.HealthcheckResult{
{Output: "first\n"},
{Output: "connection refused\n"},
},
},
},
})
assert.Equal(t, HealthUnhealthy, info.Health)
assert.Equal(t, "connection refused", info.HealthOutput)
})
t.Run("reports the published ports", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{
State: &container.State{Status: "running"},
NetworkSettings: &container.NetworkSettings{
Ports: network.PortMap{
network.MustParsePort("5432/tcp"): []network.PortBinding{{HostPort: "49153"}},
network.MustParsePort("6379/tcp"): nil,
},
},
})
assert.Equal(t, map[string]string{"5432": "49153"}, info.Ports)
})
t.Run("tolerates a container without state", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{ID: "abc123"})
assert.Equal(t, "abc123", info.ID)
assert.Equal(t, HealthNone, info.Health)
})
}
func TestMergeContainerConfigsVolumesReplaceRunnerMounts(t *testing.T) {
logger, _ := test.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger)

View File

@@ -154,6 +154,14 @@ func (e *HostEnvironment) CopyDir(destPath, srcPath string, useGitIgnore bool) c
}
}
func (e *HostEnvironment) DumpLogs(_ context.Context) error {
return nil
}
func (e *HostEnvironment) Inspect(_ context.Context) (*Info, error) {
return &Info{Health: HealthNone, Ports: map[string]string{}}, nil
}
func (e *HostEnvironment) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) {
buf := &bytes.Buffer{}
tw := tar.NewWriter(buf)

View File

@@ -5,12 +5,18 @@
package model
type JobContext struct {
Status string `json:"status"`
Container struct {
ID string `json:"id"`
Network string `json:"network"`
} `json:"container"`
Services map[string]struct {
ID string `json:"id"`
} `json:"services"`
Status string `json:"status"`
Container JobContainerContext `json:"container"`
Services map[string]JobService `json:"services"`
}
type JobContainerContext struct {
ID string `json:"id"`
Network string `json:"network"`
}
type JobService struct {
ID string `json:"id"`
Network string `json:"network"`
Ports map[string]string `json:"ports"` // container port to the published host port
}

View File

@@ -186,10 +186,10 @@ func (rc *RunContext) compositeExecutor(action *model.Action) *compositeSteps {
err := rc.newCompositeCommandExecutor(step.main())(ctx)
if err != nil {
logger.Errorf("%v", err)
logger.Errorf("##[error]%s", EscapeCommandData(err.Error()))
common.SetJobError(ctx, err)
} else if ctx.Err() != nil {
logger.Errorf("%v", ctx.Err())
logger.Errorf("##[error]%s", EscapeCommandData(ctx.Err().Error()))
common.SetJobError(ctx, ctx.Err())
}
return nil
@@ -248,10 +248,10 @@ func newCompositeStepLogExecutor(runStep common.Executor, stepID string) common.
logger := common.Logger(ctx)
err := runStep(ctx)
if err != nil {
logger.Errorf("%v", err)
logger.Errorf("##[error]%s", EscapeCommandData(err.Error()))
common.SetJobError(ctx, err)
} else if ctx.Err() != nil {
logger.Errorf("%v", ctx.Err())
logger.Errorf("##[error]%s", EscapeCommandData(ctx.Err().Error()))
common.SetJobError(ctx, ctx.Err())
}
return nil

View File

@@ -6,6 +6,7 @@ package runner
import (
"context"
"fmt"
"regexp"
"strings"
@@ -45,17 +46,24 @@ func (rc *RunContext) commandHandler(ctx context.Context) common.LineHandler {
return true
}
if resumeCommand != "" && command != resumeCommand {
if resumeCommand != "" {
// There should not be any emojis in the log output for Gitea.
// The code in the switch statement is the same.
// Return true (not false) so the line still reaches the raw_output
// log handler; otherwise everything between ::stop-commands:: and
// its end token is silently dropped from the step log.
logger.Infof("%s", line)
// Resumed here rather than from the switch, because the end token is arbitrary
// and a token naming a real command would otherwise never resume.
if command == resumeCommand {
resumeCommand = ""
}
return true
}
arg = UnescapeCommandData(arg)
kvPairs = unescapeKvPairs(kvPairs)
if (command == "set-env" || command == "add-path") && rc.refuseUnsecureCommand(ctx, command) {
return true
}
switch command {
case "set-env":
rc.setEnv(ctx, kvPairs, arg)
@@ -63,27 +71,20 @@ func (rc *RunContext) commandHandler(ctx context.Context) common.LineHandler {
rc.setOutput(ctx, kvPairs, arg)
case "add-path":
rc.addPath(ctx, arg)
case "debug":
logger.Infof("%s", line)
case "warning":
logger.Infof("%s", line)
case "error":
logger.Infof("%s", line)
case "add-mask":
rc.AddMask(arg)
logger.Infof("%s", "***")
// The raw line is still forwarded, carrying the secret: that is how the reporter
// learns the mask, and it drops the row rather than writing it out.
case "stop-commands":
resumeCommand = arg
logger.Infof("%s", line)
case resumeCommand:
resumeCommand = ""
logger.Infof("%s", line)
case "save-state":
logger.Infof("%s", line)
rc.saveState(ctx, kvPairs, arg)
case "add-matcher":
logger.Infof("%s", line)
default:
// ::debug::, ::error::, ::warning::, ::add-matcher:: and anything unrecognised are
// passed through for the reporter and Gitea's web UI to render.
logger.Infof("%s", line)
}
@@ -92,6 +93,52 @@ func (rc *RunContext) commandHandler(ctx context.Context) common.LineHandler {
}
}
const allowUnsecureCommandsVar = "ACTIONS_ALLOW_UNSECURE_COMMANDS"
// refuseUnsecureCommand reports whether a deprecated ::set-env:: or ::add-path:: command must
// not run, recording the error that fails the step. GitHub disabled both because a step that
// echoes untrusted content can use them to set NODE_OPTIONS or PATH for every later step.
func (rc *RunContext) refuseUnsecureCommand(ctx context.Context, command string) bool {
if rc.allowUnsecureCommandsOptIn() {
return false
}
// The step executor logs the failure itself, so keep this line's wording distinct.
common.Logger(ctx).WithField(rawOutputField, true).Errorf("##[error]%s", EscapeCommandData(fmt.Sprintf(
"The `%s` command is disabled: it can set the environment of every later step from untrusted output. "+
"Write to $GITHUB_ENV or $GITHUB_PATH instead, or set ACTIONS_ALLOW_UNSECURE_COMMANDS to allow it",
command)))
rc.unsecureCommandMu.Lock()
defer rc.unsecureCommandMu.Unlock()
if rc.unsecureCommandErr == nil {
rc.unsecureCommandErr = fmt.Errorf("the `%s` workflow command is disabled", command)
}
return true
}
// allowUnsecureCommandsOptIn reports whether the workflow itself asked for the deprecated
// commands, from any env scope, as it can on GitHub.
func (rc *RunContext) allowUnsecureCommandsOptIn() bool {
return isTruthyEnv(rc.currentStepEnv()[allowUnsecureCommandsVar]) ||
isTruthyEnv(rc.Env[allowUnsecureCommandsVar]) ||
isTruthyEnv(rc.GlobalEnv[allowUnsecureCommandsVar])
}
// isTruthyEnv mirrors GitHub's bool.TryParse: only "true", in any casing.
func isTruthyEnv(v string) bool {
return strings.EqualFold(strings.TrimSpace(v), "true")
}
// takeUnsecureCommandError returns and clears the error left by a refused command.
func (rc *RunContext) takeUnsecureCommandError() error {
rc.unsecureCommandMu.Lock()
defer rc.unsecureCommandMu.Unlock()
err := rc.unsecureCommandErr
rc.unsecureCommandErr = nil
return err
}
func (rc *RunContext) setEnv(ctx context.Context, kvPairs map[string]string, arg string) {
name := kvPairs["name"]
common.Logger(ctx).Infof("::set-env:: %s=%s", name, arg)
@@ -161,9 +208,9 @@ var (
commandPropertyUnescaper = strings.NewReplacer("%25", "%", "%0D", "\r", "%0A", "\n", "%3A", ":", "%2C", ",")
)
// escapeCommandData encodes the data part of a "::cmd::" or "##[cmd]" line the runner writes itself,
// EscapeCommandData encodes the data part of a "::cmd::" or "##[cmd]" line the runner writes itself,
// so the log renderer decodes it back. Lines forwarded from step output are already escaped.
func escapeCommandData(arg string) string {
func EscapeCommandData(arg string) string {
return commandDataEscaper.Replace(arg)
}

View File

@@ -16,12 +16,18 @@ import (
"github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// unsecureRC opts into ::set-env:: and ::add-path::, which are refused without it.
func unsecureRC() *RunContext {
return &RunContext{Env: map[string]string{allowUnsecureCommandsVar: "true"}}
}
func TestSetEnv(t *testing.T) {
a := assert.New(t)
ctx := context.Background()
rc := new(RunContext)
rc := unsecureRC()
handler := rc.commandHandler(ctx)
handler("::set-env name=x::valz\n")
@@ -31,7 +37,7 @@ func TestSetEnv(t *testing.T) {
func TestStopCommandsKeepsSuppressedLinesInLog(t *testing.T) {
a := assert.New(t)
ctx := context.Background()
rc := new(RunContext)
rc := unsecureRC()
handler := rc.commandHandler(ctx)
// Stop command processing until the matching end token is seen.
@@ -84,7 +90,7 @@ func TestSetOutput(t *testing.T) {
func TestAddpath(t *testing.T) {
a := assert.New(t)
ctx := context.Background()
rc := new(RunContext)
rc := unsecureRC()
handler := rc.commandHandler(ctx)
handler("::add-path::/zoo\n")
@@ -99,7 +105,7 @@ func TestStopCommands(t *testing.T) {
a := assert.New(t)
ctx := common.WithLogger(context.Background(), logger)
rc := new(RunContext)
rc := unsecureRC()
handler := rc.commandHandler(ctx)
handler("::set-env name=x::valz\n")
@@ -119,10 +125,26 @@ func TestStopCommands(t *testing.T) {
a.Contains(messages, "::set-env name=x::abcd\n")
}
// The end token is arbitrary, so one that happens to name a real command must still resume
// rather than being swallowed by that command's case.
func TestStopCommandsResumesOnCommandNamedToken(t *testing.T) {
a := assert.New(t)
rc := unsecureRC()
handler := rc.commandHandler(context.Background())
handler("::stop-commands::add-mask\n")
handler("::set-env name=x::suppressed\n")
a.NotContains(rc.Env, "x")
handler("::add-mask::\n")
handler("::set-env name=x::resumed\n")
a.Equal("resumed", rc.Env["x"])
}
func TestAddpathADO(t *testing.T) {
a := assert.New(t)
ctx := context.Background()
rc := new(RunContext)
rc := unsecureRC()
handler := rc.commandHandler(ctx)
handler("##[add-path]/zoo\n")
@@ -218,6 +240,44 @@ func TestSaveState(t *testing.T) {
func TestEscapeCommandData(t *testing.T) {
a := assert.New(t)
a.Equal("a%25b%0Dc%0Ad%250A", escapeCommandData("a%b\rc\nd%0A"))
a.Equal("a%25b%0Dc%0Ad%250A", EscapeCommandData("a%b\rc\nd%0A"))
a.Equal("a%b\rc\nd%0A", UnescapeCommandData("a%25b%0Dc%0Ad%250A"))
}
func TestUnsecureCommands(t *testing.T) {
tests := []struct {
name string
jobEnv map[string]string
stepEnv map[string]string
optedIn bool
}{
{name: "refused with no opt-in"},
// GitHub reads the opt-in with bool.TryParse, so "1" is not one.
{name: "refused for a value bool.TryParse rejects", jobEnv: map[string]string{allowUnsecureCommandsVar: "1"}},
{name: "opted in through the step environment", stepEnv: map[string]string{allowUnsecureCommandsVar: "true"}, optedIn: true},
{name: "opted in through the job environment", jobEnv: map[string]string{allowUnsecureCommandsVar: "TRUE"}, optedIn: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
a := assert.New(t)
rc := &RunContext{Env: tt.jobEnv}
rc.setCurrentStepEnv(tt.stepEnv)
handler := rc.commandHandler(context.Background())
handler("::set-env name=x::valz\n")
handler("::add-path::/opt/bin\n")
if !tt.optedIn {
a.Empty(rc.Env["x"])
a.Empty(rc.ExtraPath)
// The refusal fails the step that produced it, once.
require.ErrorContains(t, rc.takeUnsecureCommandError(), "set-env")
a.NoError(rc.takeUnsecureCommandError())
return
}
a.Equal("valz", rc.Env["x"])
a.Equal([]string{"/opt/bin"}, rc.ExtraPath)
a.NoError(rc.takeUnsecureCommandError())
})
}
}

View File

@@ -78,3 +78,14 @@ func (cm *containerMock) GetContainerArchive(ctx context.Context, srcPath string
}
return args.Get(0).(io.ReadCloser), err
}
func (cm *containerMock) DumpLogs(ctx context.Context) error {
return cm.Called(ctx).Error(0)
}
func (cm *containerMock) Inspect(ctx context.Context) (*container.Info, error) {
args := cm.Called(ctx)
info, _ := args.Get(0).(*container.Info)
err, _ := args.Get(1).(error)
return info, err
}

View File

@@ -66,7 +66,7 @@ func reportStepError(ctx context.Context, rc *RunContext, err error) {
rc.markInterrupted(ctx.Err())
return
}
common.Logger(ctx).Errorf("##[error]%s", escapeCommandData(err.Error()))
common.Logger(ctx).Errorf("##[error]%s", EscapeCommandData(err.Error()))
common.SetJobError(ctx, err)
rc.markFailed()
}
@@ -260,7 +260,7 @@ func newJobExecutor(info jobInfo, sf stepFactory, rc *RunContext) common.Executo
logger.Infof("Cleaning up container for job %s", rc.JobName)
if err = info.stopContainer()(ctx); err != nil {
logger.Errorf("Error while stop job container: %v", err)
logger.Errorf("##[error]%s", EscapeCommandData("Error while stop job container: "+err.Error()))
}
// For Gitea

View File

@@ -45,7 +45,7 @@ func (rc *RunContext) runJobHook(ctx context.Context, hookPath, name string) err
cmd, shell := hookCommand(hookPath)
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
defer rawLogger.Infof("::endgroup::")
rawLogger.Infof("::group::Run '%s'", escapeCommandData(hookPath))
rawLogger.Infof("::group::Run '%s'", EscapeCommandData(hookPath))
rawLogger.Infof("A %s hook has been configured by the runner administrator", name)
if shell != "" {
rawLogger.Infof("shell: %s", shell)

View File

@@ -250,7 +250,7 @@ func AppendSecretMasker(oldnew []string, v string) []string {
ret = append(ret, tm, "***")
// command data reaches the log escaped, so "pass%word" also arrives as "pass%25word"
if strings.ContainsAny(tm, "%\r\n") {
ret = append(ret, escapeCommandData(tm), "***")
ret = append(ret, EscapeCommandData(tm), "***")
}
}
}

View File

@@ -22,6 +22,7 @@ import (
"runtime"
"slices"
"strings"
"sync"
"time"
"gitea.com/gitea/runner/act/common"
@@ -34,6 +35,7 @@ import (
"github.com/docker/go-connections/nat"
"github.com/moby/moby/api/types/mount"
"github.com/opencontainers/selinux/go-selinux"
"golang.org/x/sync/errgroup"
)
// RunContext contains info about current job
@@ -55,7 +57,7 @@ type RunContext struct {
IntraActionState map[string]map[string]string
ExprEval ExpressionEvaluator
JobContainer container.ExecutionsEnvironment
ServiceContainers []container.ExecutionsEnvironment
serviceContainers []*serviceContainer
OutputMappings map[MappableOutput]MappableOutput
JobName string
ActionPath string
@@ -83,6 +85,37 @@ type RunContext struct {
// failures. Those failures must still make success() false and failure() true for later
// main-step if evaluation.
jobFailed bool
// empty for a host-mode job, which starts no container
jobContainerID string
jobNetworkName string
// stepEnv is a copy of the running step's environment, so that workflow commands parsed out
// of the container's output can be judged against it. Written by runStepExecutor and read on
// the log-writer goroutine, hence unsecureCommandMu, which also guards unsecureCommandErr.
stepEnv map[string]string
unsecureCommandErr error // refused ::set-env::/::add-path::, turned into a step failure
unsecureCommandMu sync.Mutex
}
// serviceContainer pairs a service container with the workflow id that keys job.services.
type serviceContainer struct {
name string
image string
container container.ExecutionsEnvironment
logsDumped bool
info *container.Info // last poll, the source of the `job.services` entry
}
// setCurrentStepEnv records the environment of the step about to run.
func (rc *RunContext) setCurrentStepEnv(env map[string]string) {
rc.unsecureCommandMu.Lock()
defer rc.unsecureCommandMu.Unlock()
rc.stepEnv = env
}
func (rc *RunContext) currentStepEnv() map[string]string {
rc.unsecureCommandMu.Lock()
defer rc.unsecureCommandMu.Unlock()
return rc.stepEnv
}
// markCancelled flags the job as cancelled so subsequent step `if` evaluations and the
@@ -488,7 +521,7 @@ func (rc *RunContext) startJobContainer() common.Executor {
Privileged: rc.Config.Privileged,
UsernsMode: rc.Config.UsernsMode,
Platform: rc.Config.ContainerArchitecture,
AutoRemove: rc.Config.AutoRemove,
AutoRemove: false, // so a dead service's log survives, cleanupJobResources removes it
Options: rc.ExprEval.Interpolate(ctx, spec.Options),
NetworkMode: networkName,
NetworkAliases: []string{serviceID},
@@ -496,7 +529,7 @@ func (rc *RunContext) startJobContainer() common.Executor {
PortBindings: portBindings,
AllocatePTY: rc.Config.AllocatePTY,
})
rc.ServiceContainers = append(rc.ServiceContainers, c)
rc.serviceContainers = append(rc.serviceContainers, &serviceContainer{name: serviceID, image: serviceImage, container: c})
}
rc.cleanUpJobContainer = rc.cleanupJobResources(networkName, createAndDeleteNetwork)
@@ -531,6 +564,8 @@ func (rc *RunContext) startJobContainer() common.Executor {
return errors.New("Failed to create job container")
}
rc.jobNetworkName = networkName
defer printStartJobContainerGroup(ctx, image, name, networkName)()
return common.NewPipelineExecutor(
rc.pullServicesImages(rc.Config.ForcePull),
@@ -538,9 +573,12 @@ func (rc *RunContext) startJobContainer() common.Executor {
rc.stopJobContainer(),
container.NewDockerNetworkCreateExecutor(networkName, rc.Config.ContainerNetworkCreateOptions).
IfBool(createAndDeleteNetwork),
rc.startServiceContainers(networkName),
rc.startServiceContainers(),
rc.reportUnstartedServices(),
rc.waitForServiceContainers(),
rc.JobContainer.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
rc.JobContainer.Start(false),
rc.captureJobContainerInfo(),
rc.JobContainer.Copy(rc.JobContainer.GetActPath()+"/", &container.FileEntry{
Name: "workflow/event.json",
Mode: 0o644,
@@ -565,7 +603,7 @@ func (rc *RunContext) cleanupJobResources(networkName string, createAndDeleteNet
if removeJobContainer {
errs = append(errs, rc.JobContainer.Remove()(ctx))
}
if len(rc.ServiceContainers) > 0 {
if len(rc.serviceContainers) > 0 {
logger.Infof("Cleaning up services for job %s", rc.JobName)
if err := rc.stopServiceContainers()(ctx); err != nil {
logger.Errorf("Error while cleaning services: %v", err)
@@ -662,21 +700,21 @@ func (rc *RunContext) stopJobContainer() common.Executor {
func (rc *RunContext) pullServicesImages(forcePull bool) common.Executor {
return func(ctx context.Context) error {
execs := []common.Executor{}
for _, c := range rc.ServiceContainers {
execs = append(execs, c.Pull(forcePull))
for _, svc := range rc.serviceContainers {
execs = append(execs, svc.container.Pull(forcePull))
}
return common.NewParallelExecutor(len(execs), execs...)(ctx)
}
}
func (rc *RunContext) startServiceContainers(_ string) common.Executor {
func (rc *RunContext) startServiceContainers() common.Executor {
return func(ctx context.Context) error {
execs := []common.Executor{}
for _, c := range rc.ServiceContainers {
for _, svc := range rc.serviceContainers {
execs = append(execs, common.NewPipelineExecutor(
c.Pull(false),
c.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
c.Start(false),
svc.container.Pull(false),
svc.container.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
svc.container.Start(false),
))
}
return common.NewParallelExecutor(len(execs), execs...)(ctx)
@@ -686,13 +724,159 @@ func (rc *RunContext) startServiceContainers(_ string) common.Executor {
func (rc *RunContext) stopServiceContainers() common.Executor {
return func(ctx context.Context) error {
execs := []common.Executor{}
for _, c := range rc.ServiceContainers {
execs = append(execs, c.Remove().Finally(c.Close()))
for _, svc := range rc.serviceContainers {
execs = append(execs, svc.container.Remove().Finally(svc.container.Close()))
}
return common.NewParallelExecutor(len(execs), execs...)(ctx)
}
}
const (
defaultServiceReadyTimeout = 5 * time.Minute
serviceReadyPollMax = 32 * time.Second
)
var serviceReadyPollInterval = 2 * time.Second // a variable so tests need not wait
// reportUnstartedServices logs a service that did not start. The steps that need it
// report it better than the runner can, so the job carries on.
func (rc *RunContext) reportUnstartedServices() common.Executor {
return func(ctx context.Context) error {
logger := common.Logger(ctx)
for _, svc := range rc.serviceContainers {
info, err := svc.inspect(ctx)
if err != nil {
logger.Debugf("unable to inspect service '%s': %v", svc.name, err)
continue
}
if info.State == container.StateRunning {
continue
}
svc.dumpLogs(ctx)
logger.Warnf("Docker container %s is not in running state: %s (%d)", info.ID, info.State, info.ExitCode)
}
return nil
}
}
// waitForServiceContainers blocks until every service that declares a healthcheck reports
// healthy, as GitHub does, so a first step cannot connect before the service listens.
func (rc *RunContext) waitForServiceContainers() common.Executor {
return func(ctx context.Context) error {
if len(rc.serviceContainers) == 0 {
return nil
}
timeout := rc.Config.ServiceReadyTimeout
switch {
case timeout < 0:
// disabled, but still describe the containers for `job.services`
for _, svc := range rc.serviceContainers {
if _, err := svc.inspect(ctx); err != nil && !errors.Is(err, container.ErrContainerNotFound) {
return err
}
}
return nil
case timeout == 0:
timeout = defaultServiceReadyTimeout
}
// the first error cancels the rest, so a failure does not wait out a sibling's timeout
group, groupCtx := errgroup.WithContext(ctx)
for _, svc := range rc.serviceContainers {
group.Go(func() error {
return svc.waitUntilHealthy(groupCtx, timeout)
})
}
return group.Wait()
}
}
// waitUntilHealthy waits on the healthcheck alone, so a container that declares none is
// ready at once and one that exited is left to the steps that need it.
func (svc *serviceContainer) waitUntilHealthy(ctx context.Context, timeout time.Duration) error {
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
interval := serviceReadyPollInterval
ctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
for {
info, err := svc.inspect(ctx)
if ctxErr := ctx.Err(); ctxErr != nil { // the wait ended, an inspect error only noticed it
if errors.Is(ctxErr, context.DeadlineExceeded) {
return fmt.Errorf("the service '%s' did not become healthy within %s%s", svc.name, timeout, svc.healthOutputSuffix())
}
return ctxErr
}
switch {
case errors.Is(err, container.ErrContainerNotFound):
return nil // gone, so there is no health left to wait on
case err != nil:
return err
}
switch {
case info.Health == container.HealthUnhealthy:
svc.dumpLogs(ctx)
common.Logger(ctx).Errorf("Failed to initialize container %s", svc.image)
return fmt.Errorf("the service '%s' is unhealthy%s", svc.name, svc.healthOutputSuffix())
case info.Health != container.HealthStarting:
rawLogger.Infof("%s service is healthy.", svc.name)
return nil
}
rawLogger.Infof("%s service is starting, waiting %d seconds before checking again.", svc.name, int(interval.Seconds()))
select {
case <-ctx.Done(): // reported at the top of the loop
case <-time.After(interval):
}
interval = min(interval*2, serviceReadyPollMax)
}
}
// dumpLogs writes the container's log to the job log once, however often it is reported.
func (svc *serviceContainer) dumpLogs(ctx context.Context) {
if svc.logsDumped {
return
}
svc.logsDumped = true
if err := svc.container.DumpLogs(ctx); err != nil {
common.Logger(ctx).Debugf("unable to read the log of service '%s': %v", svc.name, err)
}
}
// inspect also records the state for the `job.services` context.
func (svc *serviceContainer) inspect(ctx context.Context) (*container.Info, error) {
info, err := svc.container.Inspect(ctx)
if err != nil {
return nil, fmt.Errorf("failed to inspect service '%s': %w", svc.name, err)
}
svc.info = info
return info, nil
}
func (svc *serviceContainer) healthOutputSuffix() string {
if svc.info == nil || svc.info.HealthOutput == "" {
return ""
}
return ": " + svc.info.HealthOutput
}
// captureJobContainerInfo is a convenience: failing to describe the container must not
// fail the job.
func (rc *RunContext) captureJobContainerInfo() common.Executor {
return func(ctx context.Context) error {
info, err := rc.JobContainer.Inspect(ctx)
if err != nil {
common.Logger(ctx).Debugf("unable to inspect the job container: %v", err)
return nil
}
rc.jobContainerID = info.ID
return nil
}
}
// Prepare the mounts and binds for the worker
// ActionCacheDir is for rc
@@ -1033,9 +1217,26 @@ func (rc *RunContext) getJobContext() *model.JobContext {
if rc.jobCancelled {
jobStatus = "cancelled"
}
return &model.JobContext{
Status: jobStatus,
jobContext := &model.JobContext{
Status: jobStatus,
Services: map[string]model.JobService{}, // an empty map, never null
}
if rc.jobContainerID != "" {
jobContext.Container.ID = rc.jobContainerID
jobContext.Container.Network = rc.jobNetworkName
}
for _, svc := range rc.serviceContainers {
if svc.info == nil {
continue
}
jobContext.Services[svc.name] = model.JobService{
ID: svc.info.ID,
Network: rc.jobNetworkName,
Ports: svc.info.Ports,
}
}
return jobContext
}
func (rc *RunContext) getStepsContext() map[string]*model.StepResult {

View File

@@ -13,6 +13,7 @@ import (
"runtime"
"strings"
"testing"
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/container"
@@ -22,6 +23,7 @@ import (
"github.com/docker/cli/cli/compose/loader"
log "github.com/sirupsen/logrus"
assert "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
require "github.com/stretchr/testify/require"
yaml "go.yaml.in/yaml/v4"
)
@@ -225,6 +227,12 @@ func (fakeContainer) Copy(string, ...*container.FileEntry) common.Executor {
return func(context.Context) error { return nil }
}
func (fakeContainer) Inspect(context.Context) (*container.Info, error) {
return &container.Info{ID: "fake", State: "running", Health: container.HealthNone}, nil
}
func (fakeContainer) DumpLogs(context.Context) error { return nil }
// Regression test: a service without a `credentials:` block resolves to empty
// credentials, which used to overwrite the job container's own credentials.
func TestStartJobContainerKeepsJobCredentialsWithServices(t *testing.T) {
@@ -563,7 +571,7 @@ func TestCleanupJobResourcesCleansServicesWithoutJobContainer(t *testing.T) {
rc := &RunContext{
Config: &Config{},
ServiceContainers: []container.ExecutionsEnvironment{service},
serviceContainers: []*serviceContainer{{name: "svc", container: service}},
}
err := rc.cleanupJobResources("external-network", false)(context.Background())
@@ -586,7 +594,7 @@ func TestCleanupJobResourcesContinuesAfterFailure(t *testing.T) {
Config: &Config{},
Run: &model.Run{Workflow: &model.Workflow{Name: "wf"}, JobID: "job"},
JobContainer: jobContainer,
ServiceContainers: []container.ExecutionsEnvironment{service},
serviceContainers: []*serviceContainer{{name: "svc", container: service}},
}
ctx, cancel := context.WithCancel(context.Background())
@@ -1045,6 +1053,199 @@ func TestRunContext_cleanupFailedStart(t *testing.T) {
})
}
func TestWaitForServiceContainers(t *testing.T) {
origInterval := serviceReadyPollInterval
serviceReadyPollInterval = time.Millisecond
defer func() { serviceReadyPollInterval = origInterval }()
newRunContext := func(timeout time.Duration, services ...*serviceContainer) *RunContext {
return &RunContext{
Config: &Config{ServiceReadyTimeout: timeout},
serviceContainers: services,
}
}
t.Run("returns as soon as a service without a healthcheck runs", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return(&container.Info{ID: "id", State: "running", Health: container.HealthNone}, nil).Once()
rc := newRunContext(0, &serviceContainer{name: "redis", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
service.AssertExpectations(t)
})
t.Run("waits while a service is still starting", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return(&container.Info{ID: "id", State: "running", Health: container.HealthStarting}, nil).Twice()
service.On("Inspect", mock.Anything).
Return(&container.Info{ID: "id", State: "running", Health: container.HealthHealthy}, nil).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
service.AssertExpectations(t)
})
t.Run("fails with the probe output when a service is unhealthy", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).Return(&container.Info{
State: "running",
Health: container.HealthUnhealthy,
HealthOutput: "connection refused",
}, nil).Once()
service.On("DumpLogs", mock.Anything).Return(nil).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "the service 'postgres' is unhealthy: connection refused")
service.AssertExpectations(t)
})
t.Run("lets the steps run when a service exits without a healthcheck", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return(&container.Info{State: "exited", ExitCode: 2, Health: container.HealthNone}, nil).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
})
t.Run("proceeds when the container is gone", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return((*container.Info)(nil), container.ErrContainerNotFound).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
})
t.Run("fails right away when one service fails while another is still starting", func(t *testing.T) {
failing := &containerMock{}
failing.On("Inspect", mock.Anything).
Return(&container.Info{State: "running", Health: container.HealthUnhealthy}, nil)
failing.On("DumpLogs", mock.Anything).Return(nil).Once()
starting := &containerMock{}
starting.On("Inspect", mock.Anything).
Return(&container.Info{State: "running", Health: container.HealthStarting}, nil)
rc := newRunContext(10*time.Second,
&serviceContainer{name: "failing", container: failing},
&serviceContainer{name: "starting", container: starting})
done := make(chan error, 1)
go func() { done <- rc.waitForServiceContainers()(context.Background()) }()
select {
case err := <-done:
require.Error(t, err)
assert.Contains(t, err.Error(), "the service 'failing' is unhealthy")
case <-time.After(2 * time.Second):
t.Fatal("waitForServiceContainers did not fail fast; it waited for the starting service")
}
})
t.Run("gives up once the timeout expires", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return(&container.Info{State: "running", Health: container.HealthStarting}, nil)
rc := newRunContext(20*time.Millisecond, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "did not become healthy within")
})
t.Run("gives up with the same message when the deadline stops an inspect", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Run(func(args mock.Arguments) { <-args.Get(0).(context.Context).Done() }).
Return((*container.Info)(nil), errors.New("inspect aborted"))
rc := newRunContext(20*time.Millisecond, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "did not become healthy within")
})
t.Run("does not wait when the timeout is negative", func(t *testing.T) {
service := &containerMock{}
// Still described once, so the `job.services` context is filled either way.
service.On("Inspect", mock.Anything).
Return(&container.Info{ID: "id", State: "running", Health: container.HealthStarting}, nil).Once()
svc := &serviceContainer{name: "postgres", container: service}
rc := newRunContext(-1, svc)
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
service.AssertExpectations(t)
assert.Equal(t, "id", svc.info.ID)
})
t.Run("fails on an inspect error even when the timeout is negative", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).Return((*container.Info)(nil), errors.New("daemon is gone")).Once()
rc := newRunContext(-1, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "failed to inspect service 'postgres'")
})
t.Run("is a no-op without services", func(t *testing.T) {
require.NoError(t, newRunContext(0).waitForServiceContainers()(context.Background()))
})
}
func TestReportUnstartedServices(t *testing.T) {
dead := &containerMock{}
dead.On("Inspect", mock.Anything).Return(&container.Info{ID: "dead-id", State: "exited", ExitCode: 1}, nil).Once()
dead.On("DumpLogs", mock.Anything).Return(nil).Once()
running := &containerMock{}
running.On("Inspect", mock.Anything).Return(&container.Info{ID: "run-id", State: "running"}, nil).Once()
rc := &RunContext{serviceContainers: []*serviceContainer{
{name: "postgres", container: dead},
{name: "redis", container: running},
}}
require.NoError(t, rc.reportUnstartedServices()(context.Background()))
dead.AssertExpectations(t)
running.AssertExpectations(t)
}
func TestGetJobContextReportsContainers(t *testing.T) {
rc := &RunContext{
jobNetworkName: "job-network",
jobContainerID: "job-container-id",
serviceContainers: []*serviceContainer{
{name: "postgres", info: &container.Info{ID: "svc-id", Ports: map[string]string{"5432": "49153"}}},
// A service that publishes no port reports an empty map, as GitHub does.
{name: "redis", info: &container.Info{ID: "redis-id", Ports: map[string]string{}}},
// A service that never reported is left out rather than reported as empty.
{name: "mailhog"},
},
}
jobContext := rc.getJobContext()
assert.Equal(t, "job-container-id", jobContext.Container.ID)
assert.Equal(t, "job-network", jobContext.Container.Network)
assert.Equal(t, map[string]model.JobService{
"postgres": {ID: "svc-id", Network: "job-network", Ports: map[string]string{"5432": "49153"}},
"redis": {ID: "redis-id", Network: "job-network", Ports: map[string]string{}},
}, jobContext.Services)
}
// A job that never started a container reports an empty context, not a placeholder.
func TestGetJobContextWithoutContainer(t *testing.T) {
jobContext := (&RunContext{}).getJobContext()
assert.Empty(t, jobContext.Container.ID)
assert.Empty(t, jobContext.Container.Network)
assert.Empty(t, jobContext.Services)
}
func TestImageOSFromImage(t *testing.T) {
for _, tc := range []struct {
image string

View File

@@ -74,6 +74,7 @@ type Config struct {
ContainerNetworkCreateOptions container.NewDockerNetworkCreateExecutorInput // the default network create options
ActionCache ActionCache // Use a custom ActionCache Implementation
ProxyEnv map[string]string // the proxy variables the job runs with, also given to service containers and image builds
PatchToolkit bool // edit the @actions toolkit bundled into an action so it works against Gitea, see toolkit_patch.go
PresetGitHubContext *model.GithubContext // the preset github context, overrides some fields like DefaultBranch, Env, Secrets etc.
EventJSON string // the content of JSON file to use for event.json in containers, overrides EventPath
@@ -93,6 +94,7 @@ type Config struct {
InsecureSkipTLS bool // whether to skip verifying TLS certificate of the Gitea instance
MaxParallel int // max parallel jobs to run across all workflows (0 = no limit, uses CPU count)
AllocatePTY bool // allocate a pseudo-TTY for each step's process
ServiceReadyTimeout time.Duration // how long a job waits for its service containers to report healthy (0 uses the default)
RunnerName string // name this runner registered with, reported as `runner.name`, defaults to the hostname
JobStartedHook string // script run inside the job environment before the job's first step; ACTIONS_RUNNER_HOOK_JOB_STARTED is read from Env when empty
JobCompletedHook string // script run inside the job environment after the job's last step; ACTIONS_RUNNER_HOOK_JOB_COMPLETED is read from Env when empty

View File

@@ -165,9 +165,22 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
}
_ = rc.JobContainer.Copy(actPath, files...)(ctx)
// The command handler needs the step's env to judge ACTIONS_ALLOW_UNSECURE_COMMANDS.
// Cloned: the step executor keeps writing to its own env map after this point, on a
// different goroutine from the command handler that reads it.
rc.setCurrentStepEnv(maps0.Clone(*step.getEnv()))
defer rc.setCurrentStepEnv(nil)
_ = rc.takeUnsecureCommandError() // a refusal from before any step belongs to no step
timeoutctx, cancelTimeOut := evaluateStepTimeout(ctx, rc.ExprEval, stepModel)
defer cancelTimeOut()
err = executor(timeoutctx)
// Always take it, so the job-scoped error cannot leak onto a later step. A refusal
// fails the step as it does on GitHub, but the executor's own error wins.
insecureErr := rc.takeUnsecureCommandError()
if err == nil {
err = insecureErr
}
if err == nil {
logger.WithField("stepResult", stepResult.Outcome).Infof("Success - %s %s", stage, stepString)
@@ -181,7 +194,7 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
}
if continueOnError {
logger.Errorf("##[error]%s", escapeCommandData(err.Error()))
logger.Errorf("##[error]%s", EscapeCommandData(err.Error()))
logger.Infof("Failed but continue next step")
err = nil
stepResult.Conclusion = model.StepStatusSuccess

View File

@@ -236,10 +236,10 @@ func (sar *stepActionRemote) toolkitBundles() (string, []string) {
return dir, actionScriptPaths(filepath.Join(dir, sar.remoteAction.Path), sar.action)
}
// patchActionToolkit edits the bundled toolkit so it works against Gitea, which lets the cache
// client use the v2 API this runner serves. A no-op unless the runner serves it.
// patchActionToolkit edits the bundled toolkit so it works against Gitea: the artifact actions
// stop refusing, and the cache client keeps to the cache server whichever API version it picks.
func (sar *stepActionRemote) patchActionToolkit(ctx context.Context) error {
if sar.RunContext.GetEnv()[CacheServiceV2Env] != "" {
if sar.RunContext.Config.PatchToolkit {
dir, scripts := sar.toolkitBundles()
patchToolkit(ctx, dir, scripts)
}

View File

@@ -18,6 +18,7 @@ import (
"gitea.com/gitea/runner/act/model"
"github.com/kballard/go-shellquote"
"github.com/sirupsen/logrus"
yaml "go.yaml.in/yaml/v4"
)
@@ -63,7 +64,7 @@ func (sr *stepRun) printRunScriptActionDetails(ctx context.Context) {
normalized := strings.TrimRight(strings.ReplaceAll(sr.interpolatedScript, "\r\n", "\n"), "\n")
rawLogger.Infof("::group::Run %s", escapeCommandData(sr.runScriptGroupTitle(normalized)))
rawLogger.Infof("::group::Run %s", EscapeCommandData(sr.runScriptGroupTitle(normalized)))
if normalized != "" {
for line := range strings.SplitSeq(normalized, "\n") {
@@ -90,12 +91,12 @@ func printRunActionHeader(ctx context.Context, step *model.Step, env map[string]
if step.Name != "" {
title = step.Name
}
rawLogger.Infof("::group::Run %s", escapeCommandData(title))
rawLogger.Infof("::group::Run %s", EscapeCommandData(title))
if len(step.With) > 0 {
rawLogger.Infof("with:")
for _, k := range slices.Sorted(maps.Keys(step.With)) {
rawLogger.Infof(" %s: %s", k, step.With[k])
logKeyedValue(rawLogger, k, step.With[k])
}
}
@@ -129,7 +130,17 @@ func printStepEnvBlock(ctx context.Context, step *model.Step, env map[string]str
if caseInsensitive {
lookupKey = strings.ToUpper(k)
}
rawLogger.Infof(" %s: %s", k, envLookup[lookupKey])
logKeyedValue(rawLogger, k, envLookup[lookupKey])
}
}
// logKeyedValue prints one row per line of value: Gitea stores one log row per line, so an
// embedded newline would reach the user as a literal "\n".
func logKeyedValue(rawLogger *logrus.Entry, key, value string) {
lines := strings.Split(strings.ReplaceAll(value, "\r\n", "\n"), "\n")
rawLogger.Infof(" %s: %s", key, lines[0])
for _, line := range lines[1:] {
rawLogger.Infof(" %s", line)
}
}

View File

@@ -6,6 +6,7 @@ package runner
import (
"context"
"errors"
"testing"
"gitea.com/gitea/runner/act/common"
@@ -14,6 +15,7 @@ import (
log "github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
yaml "go.yaml.in/yaml/v4"
)
@@ -354,3 +356,48 @@ func TestIsContinueOnError(t *testing.T) {
assertObject.False(continueOnError)
assertObject.Error(err)
}
// A refused ::set-env::/::add-path:: records a job-scoped error. When the step that
// produced it also fails on its own, the refusal must be cleared at the step boundary, so
// it fails only that step and never leaks onto a later step that runs anyway (if: always()).
func TestRunStepExecutorDoesNotLeakRefusalToNextStep(t *testing.T) {
cm := &containerMock{}
noop := func(context.Context) error { return nil }
cm.On("Copy", mock.Anything, mock.Anything).Return(noop)
cm.On("UpdateFromEnv", mock.Anything, mock.Anything).Return(noop)
rc := &RunContext{
Config: &Config{Env: map[string]string{}},
Run: &model.Run{
JobID: "1",
Workflow: &model.Workflow{Jobs: map[string]*model.Job{"1": {}}},
},
Env: map[string]string{},
StepResults: map[string]*model.StepResult{},
JobContainer: cm,
}
rc.ExprEval = rc.NewExpressionEvaluator(context.Background())
// Dryrun skips reading the path file back from the (mocked) container.
ctx := common.WithDryrun(context.Background(), true)
// A refusal parsed out of the job container's own output belongs to no step, so the
// first step must not be failed by it.
rc.commandHandler(ctx)("::set-env name=setup::y\n")
stepSetup := &stepRun{RunContext: rc, Step: &model.Step{ID: "setup"}, env: map[string]string{}}
require.NoError(t, runStepExecutor(stepSetup, stepStageMain, func(context.Context) error { return nil })(ctx))
// Step A refuses a ::set-env:: and then fails on its own.
stepA := &stepRun{RunContext: rc, Step: &model.Step{ID: "a"}, env: map[string]string{}}
errA := runStepExecutor(stepA, stepStageMain, func(context.Context) error {
rc.commandHandler(ctx)("::set-env name=x::y\n")
return errors.New("boom")
})(ctx)
// The step fails with its own error, not the refusal.
require.ErrorContains(t, errA, "boom")
// Step B runs despite step A's failure (if: always()) and issues no unsecure command;
// it must not inherit step A's refusal.
stepB := &stepRun{RunContext: rc, Step: &model.Step{ID: "b", If: yaml.Node{Value: "always()"}}, env: map[string]string{}}
errB := runStepExecutor(stepB, stepStageMain, func(context.Context) error { return nil })(ctx)
require.NoError(t, errB)
}

View File

@@ -3,6 +3,7 @@ jobs:
_:
runs-on: ubuntu-latest
env:
ACTIONS_ALLOW_UNSECURE_COMMANDS: 'true'
MYGLOBALENV3: myglobalval3
steps:
- uses: actions/checkout@v4

View File

@@ -4,6 +4,8 @@ on: push
jobs:
build:
runs-on: ubuntu-latest
env:
ACTIONS_ALLOW_UNSECURE_COMMANDS: 'true'
steps:
- name: TEST set-env
run: echo "::set-env name=foo::bar"

View File

@@ -15,3 +15,9 @@ jobs:
echo "id: ${{ job.services.postgres.id }}"
echo "network: ${{ job.services.postgres.network }}"
echo "ports: ${{ job.services.postgres.ports }}"
- name: The job context describes the started containers
run: |
test -n "${{ job.container.id }}"
test -n "${{ job.services.postgres.id }}"
test -n "${{ job.services.postgres.ports['80'] }}"
test "${{ job.services.postgres.network }}" = "${{ job.container.network }}"

View File

@@ -5,11 +5,15 @@ package runner
import (
"context"
"encoding/base64"
"io"
"net/http"
"net/http/httptest"
"os"
"os/exec"
"path"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
@@ -59,41 +63,65 @@ func bundleFromGitHub(t *testing.T, repo, ref, path string) string {
return bundle
}
// runCacheAction runs one entrypoint the way a job would: a real Gitea server URL, and a results
// URL that points at Gitea rather than at the runner. Nothing about the environment is rewritten,
// so only the patch can make the client choose v2 and find the cache server.
func runCacheAction(t *testing.T, script, workspace, runnerTemp, cacheURL, token, key string) string {
// jobEnv is the environment a job gets from this runner, which is what decides where an action's
// toolkit looks for the cache and artifact services.
type jobEnv struct {
workspace, runnerTemp string
cacheURL, resultsURL string
token string
}
// runActionEntrypoint runs one action entrypoint the way a job would. Adding another action to these tests
// means downloading its entrypoint with bundleFromGitHub and calling this with its inputs, whose
// names are the ones the action's own action.yml uses.
func runActionEntrypoint(t *testing.T, script string, env jobEnv, inputs map[string]string) string {
t.Helper()
state := filepath.Join(runnerTemp, "state")
output := filepath.Join(runnerTemp, "output")
state := filepath.Join(env.runnerTemp, "state")
output := filepath.Join(env.runnerTemp, "output")
for _, name := range []string{state, output} {
require.NoError(t, os.WriteFile(name, nil, 0o600))
}
cmd := exec.CommandContext(t.Context(), "node", script)
cmd.Dir = workspace
cmd.Dir = env.workspace
cmd.Env = append(os.Environ(),
"INPUT_PATH=to-cache",
"INPUT_KEY="+key,
"ACTIONS_RUNTIME_TOKEN="+token,
"ACTIONS_CACHE_URL="+cacheURL+"/",
// Unreachable on purpose: the artifact service lives here, the cache service must not.
"ACTIONS_RESULTS_URL=https://gitea.example",
"ACTIONS_RUNTIME_TOKEN="+env.token,
"ACTIONS_CACHE_URL="+env.cacheURL+"/",
"ACTIONS_RESULTS_URL="+env.resultsURL,
"ACTIONS_CACHE_SERVICE_V2=true",
"GITHUB_SERVER_URL=https://gitea.example.com",
"GITHUB_REPOSITORY=testuser/testrepo",
"GITHUB_RUN_ID=1",
"GITHUB_REF=refs/heads/main",
"GITHUB_EVENT_NAME=push",
"GITHUB_WORKSPACE="+workspace,
"RUNNER_TEMP="+runnerTemp,
"GITHUB_WORKSPACE="+env.workspace,
"RUNNER_TEMP="+env.runnerTemp,
"GITHUB_STATE="+state,
"GITHUB_OUTPUT="+output,
)
for name, value := range inputs {
cmd.Env = append(cmd.Env, "INPUT_"+strings.ToUpper(name)+"="+value)
}
out, err := cmd.CombinedOutput()
t.Logf("%s:\n%s", filepath.Base(filepath.Dir(script)), out)
t.Logf("%s:\n%s", filepath.Base(script), out)
require.NoError(t, err, "%s failed", script)
return string(out)
}
// patchedAction downloads one entrypoint and patches it exactly as a downloaded action would be,
// keeping the untouched original in the sidecar beside it.
func patchedAction(t *testing.T, repo, ref, entrypoint string) string {
t.Helper()
body, err := os.ReadFile(bundleFromGitHub(t, repo, ref, entrypoint))
require.NoError(t, err)
dir := tempDirPath(t)
script := filepath.Join(dir, filepath.Base(entrypoint))
require.NoError(t, os.WriteFile(script, body, 0o600))
patchToolkit(t.Context(), dir, []string{script})
return script
}
// tempDirPath is TempDir with symlinks resolved, because macOS hands out /var paths that resolve
// to /private/var and the client derives archive paths relative to the workspace.
func tempDirPath(t *testing.T) string {
@@ -113,43 +141,52 @@ func tempDirPath(t *testing.T) string {
func TestCacheServiceV2EndToEnd(t *testing.T) {
requireHostTools(t, "node")
// A stand-in action directory, patched exactly as a downloaded one would be.
actionDir := tempDirPath(t)
scripts := map[string]string{}
for _, stage := range []string{"restore", "save"} {
body, err := os.ReadFile(bundleFromGitHub(t, "actions/cache", actionsCacheRef, "dist/"+stage+"/index.js"))
require.NoError(t, err)
scripts[stage] = filepath.Join(actionDir, stage+".js")
require.NoError(t, os.WriteFile(scripts[stage], body, 0o600))
}
patchToolkit(t.Context(), actionDir, []string{scripts["restore"], scripts["save"]})
restore := patchedAction(t, "actions/cache", actionsCacheRef, "dist/restore/index.js")
save := patchedAction(t, "actions/cache", actionsCacheRef, "dist/save/index.js")
handler, err := artifactcache.StartHandler(filepath.Join(t.TempDir(), "cache"), "127.0.0.1", 0, "", nil)
require.NoError(t, err)
t.Cleanup(func() { _ = handler.Close() })
const token, repo = "e2e-runtime-token", "testuser/testrepo"
handler.RegisterJob(token, repo)
handler.RegisterJob(token, artifactcache.JobCredential{Repo: repo})
workspace, runnerTemp := tempDirPath(t), tempDirPath(t)
require.NoError(t, os.MkdirAll(filepath.Join(workspace, "to-cache"), 0o755))
env := jobEnv{
workspace: tempDirPath(t),
runnerTemp: tempDirPath(t),
cacheURL: handler.ExternalURL(),
// The results service is the cache server's too, which is what the runner advertises.
resultsURL: handler.ExternalURL(),
token: token,
}
require.NoError(t, os.MkdirAll(filepath.Join(env.workspace, "to-cache"), 0o755))
content := []byte("cached through the patched gate")
require.NoError(t, os.WriteFile(filepath.Join(workspace, "to-cache", "data.txt"), content, 0o600))
require.NoError(t, os.WriteFile(filepath.Join(env.workspace, "to-cache", "data.txt"), content, 0o600))
const key = "patched-gate-key"
missed := runCacheAction(t, scripts["restore"], workspace, runnerTemp, handler.ExternalURL(), token, key)
inputs := map[string]string{"path": "to-cache", "key": key}
missed := runActionEntrypoint(t, restore, env, inputs)
require.Contains(t, missed, "Cache service version: v2", "the patch did not take, the client stayed on v1")
require.Contains(t, missed, "Cache not found for input keys: "+key)
saved := runCacheAction(t, scripts["save"], workspace, runnerTemp, handler.ExternalURL(), token, key)
saved := runActionEntrypoint(t, save, env, inputs)
require.Contains(t, saved, "Cache saved with key: "+key)
restored := tempDirPath(t)
hit := runCacheAction(t, scripts["restore"], restored, runnerTemp, handler.ExternalURL(), token, key)
env.workspace = tempDirPath(t)
hit := runActionEntrypoint(t, restore, env, inputs)
require.Contains(t, hit, "Cache restored from key: "+key)
got, err := os.ReadFile(filepath.Join(restored, "to-cache", "data.txt"))
got, err := os.ReadFile(filepath.Join(env.workspace, "to-cache", "data.txt"))
require.NoError(t, err)
assert.Equal(t, content, got)
// Untouched, the same client takes a Gitea host for GHES and stays on v1, which reaches the
// cache server on its own address. That is what a runner without a results service of its own
// leaves its jobs with, so it has to round trip too.
env.workspace = tempDirPath(t)
v1 := runActionEntrypoint(t, filepath.Join(sidecarDir(filepath.Dir(restore)), "index.js"), env, inputs)
require.Contains(t, v1, "Cache service version: v1")
require.Contains(t, v1, "Cache restored from key: "+key)
}
// The gate and the URL getter are separate functions, and a bundler may put either first: the gap
@@ -158,19 +195,32 @@ func TestCacheServiceV2EndToEnd(t *testing.T) {
// toolkit, patched but not run, is what keeps a future release from quietly matching only one of
// the two shapes and leaving every cache on v1.
func TestToolkitPatchAcrossActions(t *testing.T) {
for _, tc := range []struct{ repo, ref, path string }{
{"actions/setup-go", "v7.0.0", "dist/setup/index.js"},
{"actions/setup-node", "v6.0.0", "dist/cache-save/index.js"},
{"actions/setup-python", "v6.0.0", "dist/setup/index.js"},
{"ruby/setup-ruby", "v1.271.0", "dist/index.js"},
{"pnpm/action-setup", "v6.0.9", "dist/index.js"},
for _, tc := range []struct {
repo, ref, path string
wantPatched bool
}{
// The cache toolkit, in each bundler shape and from a spread of ecosystems, including the
// actions that drive a Go or a Rust cache client of their own.
{"actions/cache", actionsCacheRef, "dist/restore/index.js", true},
{"actions/setup-node", "v7.0.0", "dist/cache-save/index.js", true},
{"actions/setup-python", "v7.0.0", "dist/setup/index.js", true},
{"actions/setup-go", "v7.0.0", "dist/setup/index.js", true},
{"actions/setup-java", "v5.7.0", "dist/setup/index.js", true},
{"ruby/setup-ruby", "v1.321.0", "dist/index.js", true},
{"pnpm/action-setup", "v6.0.9", "dist/index.js", true},
{"oven-sh/setup-bun", "v2.2.0", "dist/setup/index.js", true},
{"Swatinem/rust-cache", "v2.9.1", "dist/restore/index.js", true},
{"docker/build-push-action", "v7.3.0", "dist/index.cjs", true},
// The artifact toolkit, where the gate is a refusal and there is nothing to redirect.
// v4.4.0 is the first release whose gate carries the localhost test this matches; the
// releases before it refuse in a shape the runner leaves alone.
{"actions/upload-artifact", "v4.4.0", "dist/upload/index.js"},
{"actions/upload-artifact", "v7.0.1", "dist/upload/index.js"},
{"actions/download-artifact", "v6.0.0", "dist/index.js"},
{"oven-sh/setup-bun", "v2.2.0", "dist/setup/index.js"},
{"actions/upload-artifact", "v4.4.0", "dist/upload/index.js", true},
{"actions/upload-artifact", "v7.0.1", "dist/upload/index.js", true},
{"actions/download-artifact", "v8.0.1", "dist/index.js", true},
// Neither toolkit's gate, so these have to come back byte for byte. sccache-action is the
// one that exports ACTIONS_CACHE_SERVICE_V2 itself, for the Rust client it installs.
{"actions/checkout", "v7.0.1", "dist/index.js", false},
{"mozilla-actions/sccache-action", "v0.0.11", "dist/setup/index.js", false},
} {
t.Run(tc.repo+"@"+tc.ref, func(t *testing.T) {
t.Parallel()
@@ -179,7 +229,11 @@ func TestToolkitPatchAcrossActions(t *testing.T) {
require.NoError(t, err)
out, patched := patchedBundle(data)
assert.True(t, patched, "the version gate was not patched")
require.Equal(t, tc.wantPatched, patched)
if !tc.wantPatched {
assert.Equal(t, data, out, "an untouched bundle must come back byte for byte")
return
}
assert.NotContains(t, string(out), ".LOCALHOST", "a copy of the gate was missed")
if !strings.Contains(string(data), CacheServiceV2Env) {
@@ -194,3 +248,121 @@ func TestToolkitPatchAcrossActions(t *testing.T) {
})
}
}
// The stock artifact actions refuse on a Gitea host until the gate is opened, and then they talk
// to the results service, which is this runner's cache server forwarding the artifact half on to
// Gitea. Running the real upload-artifact against a stand-in Gitea covers both halves at once:
// the patch, and the forwarding the job's registration set up.
func TestUploadArtifactThroughTheResultsService(t *testing.T) {
requireHostTools(t, "node")
var called []string
var zipped []byte
gitea := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
method := path.Base(r.URL.Path)
called = append(called, method)
w.Header().Set("x-ms-request-id", "stub")
switch method {
case "CreateArtifact":
_, _ = io.WriteString(w, `{"ok":true,"signed_upload_url":"http://`+r.Host+
`/twirp/github.actions.results.api.v1.ArtifactService/UploadArtifact?sig=x"}`)
case "FinalizeArtifact":
_, _ = io.WriteString(w, `{"ok":true,"artifact_id":"1"}`)
case "ListArtifacts":
_, _ = io.WriteString(w, `{"artifacts":[{"workflow_run_backend_id":"11",`+
`"workflow_job_run_backend_id":"22","database_id":"1","name":"an-artifact","size":"`+
strconv.Itoa(len(zipped))+`"}]}`)
case "GetSignedArtifactURL":
_, _ = io.WriteString(w, `{"signed_url":"http://`+r.Host+`/download"}`)
case "download":
w.Header().Set("Content-Type", "application/zip")
_, _ = w.Write(zipped)
default: // the zip on its way up, in the blocks the Azure protocol puts it in
body, _ := io.ReadAll(r.Body)
switch r.URL.Query().Get("comp") {
case "block":
zipped = append(zipped, body...)
case "blocklist": // the ordering document, not content
default:
zipped = body
}
w.WriteHeader(http.StatusCreated)
}
}))
defer gitea.Close()
handler, err := artifactcache.StartHandler(filepath.Join(t.TempDir(), "cache"), "127.0.0.1", 0, "", nil)
require.NoError(t, err)
t.Cleanup(func() { _ = handler.Close() })
// The artifact client decodes the runtime token for the run ids it puts in its requests, where
// the cache client only presents it, so this one has to be shaped like Gitea's.
token := "e30." + base64.RawURLEncoding.EncodeToString([]byte(`{"scp":"Actions.Results:11:22"}`)) + ".sig"
defer handler.RegisterJob(token, artifactcache.JobCredential{Repo: "testuser/testrepo", Results: gitea.URL})()
upload := patchedAction(t, "actions/upload-artifact", "v7.0.1", "dist/upload/index.js")
env := jobEnv{
workspace: tempDirPath(t),
runnerTemp: tempDirPath(t),
cacheURL: handler.ExternalURL(),
resultsURL: handler.ExternalURL(),
token: token,
}
uploaded := []byte("through the results service")
require.NoError(t, os.WriteFile(filepath.Join(env.workspace, "artifact.txt"), uploaded, 0o600))
out := runActionEntrypoint(t, upload, env, map[string]string{
"name": "an-artifact", "path": "artifact.txt", "if-no-files-found": "error",
"retention-days": "0", "compression-level": "6", "overwrite": "false",
"include-hidden-files": "false", "archive": "true",
})
require.Contains(t, out, "has been successfully uploaded")
// And back down again: listing and downloading go the same way, and the signed URL the
// artifact service hands out is fetched straight from it.
download := patchedAction(t, "actions/download-artifact", "v8.0.1", "dist/index.js")
env.workspace = tempDirPath(t)
out = runActionEntrypoint(t, download, env, map[string]string{
"name": "an-artifact", "path": "downloaded", "merge-multiple": "false",
"skip-decompress": "false", "include-hidden-files": "false", "github-token": "",
})
require.Contains(t, out, "Artifact download completed")
assert.Subset(t, called,
[]string{"CreateArtifact", "UploadArtifact", "FinalizeArtifact", "ListArtifacts", "GetSignedArtifactURL"},
"the artifact service was not reached through the cache server")
got, err := os.ReadFile(filepath.Join(env.workspace, "downloaded", "artifact.txt"))
require.NoError(t, err)
assert.Equal(t, uploaded, got)
}
// The setup actions carry the same toolkit and reach the same service, from a key of their own
// making. setup-node is the cheapest of them to run: given a lockfile and no version to install,
// it does the cache lookup and nothing else.
func TestSetupActionFindsTheCacheService(t *testing.T) {
requireHostTools(t, "node", "npm")
setup := patchedAction(t, "actions/setup-node", "v7.0.0", "dist/setup/index.js")
handler, err := artifactcache.StartHandler(filepath.Join(t.TempDir(), "cache"), "127.0.0.1", 0, "", nil)
require.NoError(t, err)
t.Cleanup(func() { _ = handler.Close() })
const token = "setup-runtime-token"
defer handler.RegisterJob(token, artifactcache.JobCredential{Repo: "testuser/testrepo"})()
env := jobEnv{
workspace: tempDirPath(t),
runnerTemp: tempDirPath(t),
cacheURL: handler.ExternalURL(),
resultsURL: handler.ExternalURL(),
token: token,
}
require.NoError(t, os.WriteFile(filepath.Join(env.workspace, "package-lock.json"),
[]byte(`{"lockfileVersion":3}`), 0o600))
out := runActionEntrypoint(t, setup, env, map[string]string{"cache": "npm"})
require.Contains(t, out, "Cache service version: v2")
require.Contains(t, out, "npm cache is not found", "the lookup did not reach the cache server")
}

View File

@@ -283,7 +283,7 @@ func TestPatchBundleAfterTheActionMoved(t *testing.T) {
// The wiring: a step patches its own bundles only when the runner serves the v2 API, and a step
// that fails gets them back. The action's path inside its repository is part of where they live.
func TestStepActionRemoteToolkitPatch(t *testing.T) {
newStep := func(t *testing.T, env map[string]string) (*stepActionRemote, string) {
newStep := func(t *testing.T, patch bool) (*stepActionRemote, string) {
t.Helper()
sar := &stepActionRemote{
@@ -291,8 +291,7 @@ func TestStepActionRemoteToolkitPatch(t *testing.T) {
remoteAction: &remoteAction{Org: "owner", Repo: "repo", Path: "sub", Ref: "v1"},
action: &model.Action{Runs: model.ActionRuns{Using: "node20", Main: "index.js"}},
RunContext: &RunContext{
Env: env,
Config: &Config{ActionCacheDir: t.TempDir()},
Config: &Config{ActionCacheDir: t.TempDir(), PatchToolkit: patch},
},
}
script := filepath.Join(sar.actionDir(), "sub", "index.js")
@@ -301,8 +300,8 @@ func TestStepActionRemoteToolkitPatch(t *testing.T) {
return sar, script
}
t.Run("left alone when the runner does not serve the v2 API", func(t *testing.T) {
sar, script := newStep(t, map[string]string{})
t.Run("left alone when the runner does not patch", func(t *testing.T) {
sar, script := newStep(t, false)
require.NoError(t, sar.patchActionToolkit(t.Context()))
body, err := os.ReadFile(script)
@@ -311,7 +310,7 @@ func TestStepActionRemoteToolkitPatch(t *testing.T) {
})
t.Run("patched, and put back when the step fails", func(t *testing.T) {
sar, script := newStep(t, map[string]string{CacheServiceV2Env: "true"})
sar, script := newStep(t, true)
require.NoError(t, sar.patchActionToolkit(t.Context()))
body, err := os.ReadFile(script)

View File

@@ -12,11 +12,13 @@ the runner as a background service on a systemd host.
sudo useradd --system --home-dir /var/lib/gitea-runner --create-home gitea-runner
```
3. Generate a config and register the runner (as the service user), so the
3. Write a config, hand it to the service user, and register as that user so the
`.runner` file ends up in the working directory:
```bash
sudo -u gitea-runner gitea-runner generate-config > /etc/gitea-runner/config.yaml
sudo mkdir -p /etc/gitea-runner
sudo gitea-runner config init --config /etc/gitea-runner/config.yaml
sudo chown gitea-runner /etc/gitea-runner/config.yaml
cd /var/lib/gitea-runner
sudo -u gitea-runner gitea-runner register --config /etc/gitea-runner/config.yaml
```

View File

@@ -49,10 +49,10 @@ export DOCKER_HOST=unix:///run/user/$(id -u)/docker.sock
gitea-runner register
```
- Generate a `gitea-runner` configuration file in the data directory. Edit the file to adjust for the system.
- Write a `gitea-runner` configuration file in the data directory. Edit the file to adjust for the system, `gitea-runner config generate` documents every option.
```bash
gitea-runner generate-config >/home/rootless/gitea-runner/config
gitea-runner config init --config /home/rootless/gitea-runner/config
```
- Create a new user-level`systemd` unit file as `/home/rootless/.config/systemd/user/gitea-runner.service` with the following contents:

28
go.mod
View File

@@ -2,6 +2,8 @@ module gitea.com/gitea/runner
go 1.26.0
toolchain go1.26.5
require (
connectrpc.com/connect v1.20.0
dario.cat/mergo v1.0.2
@@ -11,22 +13,22 @@ require (
github.com/creack/pty v1.1.24
github.com/distribution/reference v0.6.0
github.com/docker/cli v29.6.2+incompatible
github.com/docker/go-connections v0.7.0
github.com/go-git/go-billy/v5 v5.9.0
github.com/docker/go-connections v0.8.1
github.com/go-git/go-billy/v5 v5.9.1
github.com/go-git/go-git/v5 v5.19.1
github.com/gobwas/glob v0.2.3
github.com/google/go-cmp v0.7.0
github.com/joho/godotenv v1.5.1
github.com/julienschmidt/httprouter v1.3.0
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
github.com/mattn/go-isatty v0.0.23
github.com/moby/go-archive v0.2.0
github.com/mattn/go-isatty v0.0.24
github.com/moby/go-archive v0.2.1
github.com/moby/moby/api v1.55.0
github.com/moby/moby/client v0.5.0
github.com/moby/moby/client v0.5.1
github.com/moby/patternmatcher v0.6.1
github.com/opencontainers/image-spec v1.1.1
github.com/opencontainers/selinux v1.15.1
github.com/prometheus/client_golang v1.24.0
github.com/prometheus/client_golang v1.24.1
github.com/prometheus/client_model v0.6.2
github.com/rhysd/actionlint v1.7.12
github.com/sirupsen/logrus v1.9.4
@@ -36,7 +38,8 @@ require (
github.com/timshannon/bolthold v0.0.0-20240314194003-30aac6950928
go.etcd.io/bbolt v1.5.0
go.yaml.in/yaml/v4 v4.0.0-rc.3
golang.org/x/net v0.56.0
golang.org/x/net v0.57.0
golang.org/x/sync v0.22.0
golang.org/x/sys v0.47.0
golang.org/x/term v0.45.0
golang.org/x/text v0.40.0
@@ -72,20 +75,20 @@ require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/kevinburke/ssh_config v1.6.0 // indirect
github.com/klauspost/compress v1.19.0 // indirect
github.com/klauspost/compress v1.19.1 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-runewidth v0.0.21 // indirect
github.com/mattn/go-shellwords v1.0.12 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect
github.com/moby/sys/user v0.4.0 // indirect
github.com/moby/sys/sequential v0.7.0 // indirect
github.com/moby/sys/user v0.4.1 // indirect
github.com/moby/sys/userns v0.1.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/pjbgf/sha1cd v0.6.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/common v0.70.0 // indirect
github.com/prometheus/common v0.70.1 // indirect
github.com/prometheus/procfs v0.21.1 // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
github.com/sergi/go-diff v1.4.0 // indirect
@@ -103,8 +106,7 @@ require (
go.opentelemetry.io/otel/sdk/metric v1.43.0 // indirect
go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.53.0 // indirect
golang.org/x/sync v0.22.0 // indirect
golang.org/x/crypto v0.54.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

48
go.sum
View File

@@ -49,8 +49,8 @@ github.com/docker/cli v29.6.2+incompatible h1:/bjePvcbbFTnRrMfWJBY7AjfICdsiLVgHn
github.com/docker/cli v29.6.2+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/docker-credential-helpers v0.9.6 h1:cT2PbRPSlnMmNTfT2TDMXRyQ1KMWHG7xoTLBcn1ZNv0=
github.com/docker/docker-credential-helpers v0.9.6/go.mod h1:v1S+hepowrQXITkEfw6o4+BMbGot02wiKpzWhGUZK6c=
github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf0mN39c=
github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-connections v0.8.1 h1:JibmG5hULs5qXSr/cp/w3Pw5fZuStt4MOHMUExb29/M=
github.com/docker/go-connections v0.8.1/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/elazarl/goproxy v1.7.2 h1:Y2o6urb7Eule09PjlhQRGNsqRfPmYI3KKQLFpCAV3+o=
@@ -65,8 +65,8 @@ github.com/gliderlabs/ssh v0.3.8 h1:a4YXD1V7xMF9g5nTkdfnja3Sxy1PVDCj1Zg4Wb8vY6c=
github.com/gliderlabs/ssh v0.3.8/go.mod h1:xYoytBv1sV0aL3CavoDuJIQNURXkkfPA/wxQ1pL1fAU=
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI=
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic=
github.com/go-git/go-billy/v5 v5.9.0 h1:jItGXszUDRtR/AlferWPTMN4j38BQ88XnXKbilmmBPA=
github.com/go-git/go-billy/v5 v5.9.0/go.mod h1:jCnQMLj9eUgGU7+ludSTYoZL/GGmii14RxKFj7ROgHw=
github.com/go-git/go-billy/v5 v5.9.1 h1:8U73XiOTfINdItHVa6z4Gv7ToObcZ6grkqQbLryLCdA=
github.com/go-git/go-billy/v5 v5.9.1/go.mod h1:ExsU+jcGwXTBOnyilvAnEM1wug1IxHr4yP2ZXsNRtV0=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399 h1:eMje31YglSBqCdIqdhKBW8lokaMrL3uTkpGYlE2OOT4=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399/go.mod h1:1OCfN199q1Jm3HZlxleg+Dw/mwps2Wbk9frAWm+4FII=
github.com/go-git/go-git/v5 v5.19.1 h1:nX27AnaU43/K5bKktKwgBmR9lawoYVe1Ckg0rgzzN00=
@@ -100,8 +100,8 @@ github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNU
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/kevinburke/ssh_config v1.6.0 h1:J1FBfmuVosPHf5GRdltRLhPJtJpTlMdKTBjRgTaQBFY=
github.com/kevinburke/ssh_config v1.6.0/go.mod h1:q2RIzfka+BXARoNexmF9gkxEX7DmvbW9P4hIVx2Kg4M=
github.com/klauspost/compress v1.19.0 h1:sXLILfc9jV2QYWkzFOPWStmcUVH2RHEB1JCdY2oVvCQ=
github.com/klauspost/compress v1.19.0/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
@@ -115,26 +115,26 @@ github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.23 h1:cYwCQTQf3HB6xUC+BtyCLZNr7IzbOmoZbmssVNzSyiQ=
github.com/mattn/go-isatty v0.0.23/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/mattn/go-runewidth v0.0.21 h1:jJKAZiQH+2mIinzCJIaIG9Be1+0NR+5sz/lYEEjdM8w=
github.com/mattn/go-runewidth v0.0.21/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk=
github.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/go-archive v0.2.0 h1:zg5QDUM2mi0JIM9fdQZWC7U8+2ZfixfTYoHL7rWUcP8=
github.com/moby/go-archive v0.2.0/go.mod h1:mNeivT14o8xU+5q1YnNrkQVpK+dnNe/K6fHqnTg4qPU=
github.com/moby/go-archive v0.2.1 h1:fAa0wUS/ikZKyx7o/1fhUYmhZ7RgpthdeoDhJvunTLc=
github.com/moby/go-archive v0.2.1/go.mod h1:Npdv43fFqlhZW7Xo8fbm3ZMYFvAGNviUPqX21VERbcE=
github.com/moby/moby/api v1.55.0 h1:2/sexvQyqIWS8pRSCFddBfpW2qE7vR7FCL+vN8pxwMc=
github.com/moby/moby/api v1.55.0/go.mod h1:+RQ6wluLwtYaTd1WnPLykIDPekkuyD/ROWQClE83pzs=
github.com/moby/moby/client v0.5.0 h1:5XhyPk2fuOWf6RlSFa3MkIIgDZkF25xToXW8Q/BH7cc=
github.com/moby/moby/client v0.5.0/go.mod h1:rcVpF8ncl9vo5gaIBdol6CnbEtSj1uxMvEV/UrykF/s=
github.com/moby/moby/client v0.5.1 h1:tYNaJno4c0HXz12y5BiqEDy0rVTYkWzI26lGvnTMiJw=
github.com/moby/moby/client v0.5.1/go.mod h1:odLstlZ6uSnfvAgVxMpvgmb8SUdd+siH2T0GBuxVAlM=
github.com/moby/patternmatcher v0.6.1 h1:qlhtafmr6kgMIJjKJMDmMWq7WLkKIo23hsrpR3x084U=
github.com/moby/patternmatcher v0.6.1/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
github.com/moby/sys/sequential v0.6.0 h1:qrx7XFUd/5DxtqcoH1h438hF5TmOvzC/lspjy7zgvCU=
github.com/moby/sys/sequential v0.6.0/go.mod h1:uyv8EUTrca5PnDsdMGXhZe6CCe8U/UiTWd+lL+7b/Ko=
github.com/moby/sys/user v0.4.0 h1:jhcMKit7SA80hivmFJcbB1vqmw//wU61Zdui2eQXuMs=
github.com/moby/sys/user v0.4.0/go.mod h1:bG+tYYYJgaMtRKgEmuueC0hJEAZWwtIbZTB+85uoHjs=
github.com/moby/sys/sequential v0.7.0 h1:ASQNGNROJSuOO6LL6bPHbKvuZu6NU8P4ldPWk31zj/8=
github.com/moby/sys/sequential v0.7.0/go.mod h1:NfSTAp6V3fw4tmkD62PEcOKeZKquXT8VKCkf7aVR79o=
github.com/moby/sys/user v0.4.1 h1:RgjRlaDKi/Xmyrz4t8lyzXT6v2ooFeO/7xtchmhVWE0=
github.com/moby/sys/user v0.4.1/go.mod h1:E9QsW5WRe1kUAf7kW8hXKwu1uhsZEAdPLYHYSDudF4Y=
github.com/moby/sys/userns v0.1.0 h1:tVLXkFOxVu9A64/yh59slHVv9ahO9UIev4JZusOLG/g=
github.com/moby/sys/userns v0.1.0/go.mod h1:IHUYgu/kao6N8YZlp9Cf444ySSvCmDlmzUcYfDHOl28=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
@@ -153,12 +153,12 @@ github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.24.0 h1:5XStIklKuAtJSNpdD3s8XJj/Yv78IQmE1kbNk87JrAI=
github.com/prometheus/client_golang v1.24.0/go.mod h1:QcsNdotprC2nS4BTM2ucbcqxd2CeXTEa9jW7zHO9iDE=
github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU=
github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
github.com/prometheus/common v0.70.0 h1:bcpru3tWPVnxGnETLgOV5jbp/JRXgYEyv65CuBLAMMI=
github.com/prometheus/common v0.70.0/go.mod h1:S/SFasQmgGiYH6C81LKCtYa8QACgthGg5zxL2udV7SY=
github.com/prometheus/common v0.70.1 h1:1HvjP4D5oL3t8RsPlwxA9onvvStjtIHYE5XuuwOi/PY=
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
github.com/rhysd/actionlint v1.7.12 h1:vQ4GeJN86C0QH+gTUQcs8McmK62OLT3kmakPMtEWYnY=
@@ -231,13 +231,13 @@ go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
go.yaml.in/yaml/v4 v4.0.0-rc.3 h1:3h1fjsh1CTAPjW7q/EMe+C8shx5d8ctzZTrLcs/j8Go=
go.yaml.in/yaml/v4 v4.0.0-rc.3/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE=
golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=

View File

@@ -5,10 +5,8 @@ package cmd
import (
"context"
"fmt"
"os"
"gitea.com/gitea/runner/internal/pkg/config"
"gitea.com/gitea/runner/internal/pkg/ver"
"github.com/spf13/cobra"
@@ -23,7 +21,7 @@ func Execute(ctx context.Context) {
SilenceUsage: true,
}
configFile := ""
rootCmd.PersistentFlags().StringVarP(&configFile, "config", "c", "", "Config file path")
rootCmd.PersistentFlags().StringVarP(&configFile, "config", "c", "", "Config file path. `config` subcommands fall back to config.yaml in the working directory or next to the executable")
// ./gitea-runner register
var regArgs registerArgs
@@ -61,14 +59,12 @@ func Execute(ctx context.Context) {
rootCmd.AddCommand(loadBugReportCmd())
// ./gitea-runner config
rootCmd.AddCommand(&cobra.Command{
Use: "generate-config",
Short: "Generate an example config file",
Args: cobra.MaximumNArgs(0),
Run: func(_ *cobra.Command, _ []string) {
fmt.Printf("%s", config.Example)
},
})
rootCmd.AddCommand(loadConfigCmd(&configFile))
// ./gitea-runner generate-config
generateConfigCmd := loadGenerateConfigCmd("generate-config")
generateConfigCmd.Deprecated = "use `config generate` instead."
rootCmd.AddCommand(generateConfigCmd)
// ./gitea-runner cache-server
var cacheArgs cacheServerArgs

145
internal/app/cmd/config.go Normal file
View File

@@ -0,0 +1,145 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package cmd
import (
"fmt"
"os"
"path/filepath"
"slices"
"strings"
"gitea.com/gitea/runner/internal/pkg/config"
"github.com/spf13/cobra"
)
func loadConfigCmd(configFile *string) *cobra.Command {
configCmd := &cobra.Command{
Use: "config",
Short: "Generate, read and edit config files",
Args: cobra.MaximumNArgs(0),
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
}
configCmd.AddCommand(loadGenerateConfigCmd("generate"))
configCmd.AddCommand(loadInitConfigCmd(configFile))
configCmd.AddCommand(&cobra.Command{
Use: "get <key>",
Short: "Print the value of a config key",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
file, err := resolveConfigFile(cmd, configFile)
if err != nil {
return err
}
value, err := config.GetValue(file, args[0])
if err != nil {
return err
}
fmt.Fprintln(cmd.OutOrStdout(), value)
return nil
},
})
for _, sub := range []struct {
use string
short string
edit func(file, key string, values ...string) error
}{
{"set <key> <value>...", "Set the value of a config key", config.SetValue},
{"add <key> <value>...", "Append values to a list config key", config.AddValue},
{"remove <key> <value>...", "Remove values from a list config key", config.RemoveValue},
} {
valueCmd := &cobra.Command{
Use: sub.use,
Short: sub.short,
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
file, err := resolveConfigFile(cmd, configFile)
if err != nil {
return err
}
return sub.edit(file, args[0], args[1:]...)
},
}
valueCmd.Flags().SetInterspersed(false) // so a value such as `--cpus 2` is not parsed as a flag
configCmd.AddCommand(valueCmd)
}
return configCmd
}
func loadInitConfigCmd(configFile *string) *cobra.Command {
var force bool
initCmd := &cobra.Command{
Use: "init",
Short: "Write a minimal config file",
Long: "Write a minimal config file, leaving every option at its default.\nWithout --config it writes config.yaml in the working directory.",
Args: cobra.MaximumNArgs(0),
RunE: func(cmd *cobra.Command, _ []string) error {
file, taken := *configFile, []string{*configFile}
if file == "" {
file, taken = defaultConfigFileNames[0], defaultConfigFileNames // any of them would shadow the new file
}
for _, name := range taken {
if _, err := os.Stat(name); err == nil && !force {
return fmt.Errorf("config file %q already exists, pass --force to overwrite it", name)
}
}
if err := config.WriteFile(file, []byte(config.Minimal)); err != nil {
return err
}
fmt.Fprintf(cmd.OutOrStdout(), "wrote config file %q\n", file)
return nil
},
}
initCmd.Flags().BoolVarP(&force, "force", "f", false, "overwrite an existing config file")
return initCmd
}
func loadGenerateConfigCmd(use string) *cobra.Command {
return &cobra.Command{
Use: use,
Short: "Print the example config, which documents every option",
Args: cobra.MaximumNArgs(0),
Run: func(cmd *cobra.Command, _ []string) {
fmt.Fprintf(cmd.OutOrStdout(), "%s", config.Example)
},
}
}
var defaultConfigFileNames = []string{"config.yaml", "config.yml"}
func resolveConfigFile(cmd *cobra.Command, configFile *string) (string, error) {
if *configFile != "" {
return *configFile, nil
}
var dirs []string
if wd, err := os.Getwd(); err == nil {
dirs = append(dirs, wd)
}
if exe, err := os.Executable(); err == nil {
if dir := filepath.Dir(exe); !slices.Contains(dirs, dir) {
dirs = append(dirs, dir)
}
}
for _, dir := range dirs {
for _, name := range defaultConfigFileNames {
candidate := filepath.Join(dir, name)
if stat, err := os.Stat(candidate); err == nil && !stat.IsDir() {
fmt.Fprintf(cmd.ErrOrStderr(), "using config file %q\n", candidate)
return candidate, nil
}
}
}
return "", fmt.Errorf("no %s found in %s, pass one with --config",
strings.Join(defaultConfigFileNames, " or "), strings.Join(dirs, " or "))
}

View File

@@ -0,0 +1,99 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package cmd
import (
"bytes"
"os"
"path/filepath"
"testing"
"gitea.com/gitea/runner/internal/pkg/config"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func runConfigCmd(t *testing.T, configFile string, args ...string) (string, string, error) {
t.Helper()
out, errOut := &bytes.Buffer{}, &bytes.Buffer{}
cmd := loadConfigCmd(&configFile)
cmd.SetOut(out)
cmd.SetErr(errOut)
cmd.SetArgs(args)
err := cmd.Execute()
return out.String(), errOut.String(), err
}
func TestConfigCmdGeneratePrintsTheExample(t *testing.T) {
out, _, err := runConfigCmd(t, "", "generate")
require.NoError(t, err)
assert.Equal(t, string(config.Example), out)
}
func TestConfigCmdInitWritesTheMinimalConfig(t *testing.T) {
dir := t.TempDir()
file := filepath.Join(dir, "config.yaml")
out, _, err := runConfigCmd(t, file, "init")
require.NoError(t, err)
assert.Contains(t, out, file)
content, err := os.ReadFile(file)
require.NoError(t, err)
assert.Equal(t, config.Minimal, string(content))
_, _, err = runConfigCmd(t, file, "init")
require.Error(t, err)
assert.Contains(t, err.Error(), "--force")
_, _, err = runConfigCmd(t, file, "init", "--force")
require.NoError(t, err)
t.Chdir(t.TempDir())
_, _, err = runConfigCmd(t, "", "init")
require.NoError(t, err)
assert.FileExists(t, defaultConfigFileNames[0])
}
// The subcommands only wire arguments through, so one pass over all of them is enough.
func TestConfigCmdEditsTheFile(t *testing.T) {
file := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(file, []byte("runner:\n labels:\n - self-hosted\n"), 0o600))
_, _, err := runConfigCmd(t, file, "set", "container.options", "--cpus 2")
require.NoError(t, err)
_, _, err = runConfigCmd(t, file, "add", "runner.labels", "ubuntu:docker://node:22")
require.NoError(t, err)
_, _, err = runConfigCmd(t, file, "remove", "runner.labels", "self-hosted")
require.NoError(t, err)
out, _, err := runConfigCmd(t, file, "get", "runner.labels")
require.NoError(t, err)
assert.Equal(t, "ubuntu:docker://node:22\n", out)
out, _, err = runConfigCmd(t, file, "get", "container.options")
require.NoError(t, err)
assert.Equal(t, "--cpus 2\n", out)
}
func TestConfigCmdResolvesTheConfigFile(t *testing.T) {
t.Run("falls back to the working directory", func(t *testing.T) {
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, "config.yaml"), []byte("runner:\n capacity: 2\n"), 0o600))
t.Chdir(dir)
out, errOut, err := runConfigCmd(t, "", "get", "runner.capacity")
require.NoError(t, err)
assert.Equal(t, "2\n", out)
assert.Contains(t, errOut, "using config file")
})
t.Run("reports that none was found", func(t *testing.T) {
t.Chdir(t.TempDir())
_, _, err := runConfigCmd(t, "", "set", "runner.capacity", "4")
require.Error(t, err)
assert.Contains(t, err.Error(), "--config")
})
}

View File

@@ -416,7 +416,7 @@ func runExec(ctx context.Context, execArgs *executeArgs) func(cmd *cobra.Command
env[actionsRuntimeTokenEnvName] = actionsRuntimeToken
os.Setenv(actionsRuntimeTokenEnvName, actionsRuntimeToken)
}
handler.RegisterJob(actionsRuntimeToken, "__local/__exec")
handler.RegisterJob(actionsRuntimeToken, artifactcache.JobCredential{Repo: "__local/__exec"})
// no service aliases: exec builds one config for the whole plan
run.BypassProxyForDockerHost(os.Getenv("DOCKER_HOST"))
@@ -427,6 +427,7 @@ func runExec(ctx context.Context, execArgs *executeArgs) func(cmd *cobra.Command
config := &runner.Config{
Workdir: execArgs.Workdir(),
BindWorkdir: false,
PatchToolkit: true, // the cache server started above is what the patch points at
ReuseContainers: false,
ForcePull: execArgs.forcePull,
ForceRebuild: execArgs.forceRebuild,

View File

@@ -89,7 +89,8 @@ func NewRunner(cfg *config.Config, reg *config.Registration, cli client.Client)
var cacheHandler *artifactcache.Handler
if cfg.Cache.Enabled == nil || *cfg.Cache.Enabled {
if cfg.Cache.ExternalServer != "" {
envs["ACTIONS_CACHE_URL"] = cfg.Cache.ExternalServer
// The v1 client appends its path to this without a separator, so the slash is required.
envs["ACTIONS_CACHE_URL"] = strings.TrimRight(cfg.Cache.ExternalServer, "/") + "/"
} else {
warnIgnoredCacheSecret(cfg)
handler, err := artifactcache.StartHandler(
@@ -109,12 +110,6 @@ func NewRunner(cfg *config.Config, reg *config.Registration, cli client.Client)
}
}
if envs["ACTIONS_CACHE_URL"] != "" && (cfg.Cache.V2 == nil || *cfg.Cache.V2) {
// act patches the GHES check out of an action's bundle when it sees this, so the client
// uses the cache service v2 API this server also answers; see act/runner/toolkit_patch.go.
envs[runner.CacheServiceV2Env] = "true"
}
// set artifact gitea api
artifactGiteaAPI := strings.TrimSuffix(cli.Address(), "/") + "/api/actions_pipeline/"
envs["ACTIONS_RUNTIME_URL"] = artifactGiteaAPI
@@ -440,7 +435,13 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
// lifetime. Only applies to the embedded cache server; when the operator
// points the runner at an external cache via cfg.Cache.ExternalServer, it
// is that server's responsibility to authenticate requests.
defer r.registerCacheForTask(giteaRuntimeToken, preset.Repository, reporter)()
revokeCache, resultsURL := r.registerCacheForTask(giteaRuntimeToken, preset.Repository, reporter)
defer revokeCache()
// A cache server that agreed to forward the artifact half is the whole results service, so
// the job is pointed at it and the v2 variable is finally true.
if resultsURL != "" {
envs["ACTIONS_RESULTS_URL"], envs[runner.CacheServiceV2Env] = resultsURL, "true"
}
eventJSON, err := json.Marshal(preset.Event)
if err != nil {
@@ -481,6 +482,7 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
AllocatePTY: r.cfg.Runner.AllocatePTY,
ActionOfflineMode: r.cfg.Cache.OfflineMode,
ActionCloneDepth: actionCloneDepth,
PatchToolkit: r.patchToolkit(),
ReuseContainers: false,
ForcePull: r.cfg.Container.ForcePull,
@@ -508,6 +510,7 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
RunnerUUID: r.uuid,
},
ContainerOptions: r.cfg.Container.Options,
ServiceReadyTimeout: r.cfg.Container.ServiceReadyTimeout,
ContainerDaemonSocket: r.cfg.Container.DockerHost,
Privileged: r.cfg.Container.Privileged,
DefaultActionInstance: r.getDefaultActionsURL(task),
@@ -553,6 +556,12 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
return execErr
}
// patchToolkit reports whether act should edit the toolkit bundled into an action. It follows the
// cache URL, because that is what the edits point the client at; see act/runner/toolkit_patch.go.
func (r *Runner) patchToolkit() bool {
return r.envs["ACTIONS_CACHE_URL"] != "" && (r.cfg.Cache.V2 == nil || *r.cfg.Cache.V2)
}
// registerCacheForTask tells the cache server to accept requests authenticated
// with the given runtime token for the duration of this task. Returns a
// function the caller must invoke (typically via defer) to revoke the
@@ -565,18 +574,25 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
// repo scoping over the network.
//
// Safe with an empty token (older Gitea did not issue one).
func (r *Runner) registerCacheForTask(token, repo string, reporter *report.Reporter) func() {
// It also returns the URL to advertise as ACTIONS_RESULTS_URL, which is the cache server itself
// when it agreed to forward this instance's artifact service, and "" when it did not.
func (r *Runner) registerCacheForTask(token, repo string, reporter *report.Reporter) (func(), string) {
if token == "" {
return func() {}
return func() {}, ""
}
cred := artifactcache.JobCredential{
Repo: repo,
Results: r.envs["ACTIONS_RESULTS_URL"], // the instance as the job would reach it
InsecureTLS: r.cfg.Runner.Insecure,
}
if r.cacheHandler != nil {
return r.cacheHandler.RegisterJob(token, repo)
return r.cacheHandler.RegisterJob(token, cred), r.cacheHandler.ResultsURL(cred)
}
if r.cfg.Cache.ExternalServer != "" && r.cfg.Cache.ExternalSecret != "" {
return r.registerExternalCacheJob(token, repo, reporter)
return r.registerExternalCacheJob(token, cred, reporter)
}
// No cache server to register against: caching is disabled, or the built-in server failed to start.
return func() {}
return func() {}, ""
}
// registerExternalCacheJob POSTs to the remote cache-server's control-plane.
@@ -584,47 +600,56 @@ func (r *Runner) registerCacheForTask(token, repo string, reporter *report.Repor
// 401 the job's requests — better than failing the whole task for a cache
// outage. The warning is mirrored to the job log so users can see why their
// cache calls 401, instead of having to read the runner daemon's stderr.
func (r *Runner) registerExternalCacheJob(token, repo string, reporter *report.Reporter) func() {
func (r *Runner) registerExternalCacheJob(token string, cred artifactcache.JobCredential, reporter *report.Reporter) (func(), string) {
base := strings.TrimRight(r.cfg.Cache.ExternalServer, "/")
if err := postInternalCache(base+"/_internal/register", r.cfg.Cache.ExternalSecret,
map[string]string{"token": token, "repo": repo}); err != nil {
resultsURL := ""
if body, err := postInternalCache(base+"/_internal/register", r.cfg.Cache.ExternalSecret, map[string]any{
"token": token, "repo": cred.Repo, "results": cred.Results, "insecure_tls": cred.InsecureTLS,
}); err != nil {
log.Warnf("cache external_server register failed (%s): %v", base, err)
if reporter != nil {
reporter.Logf("::warning::cache external_server register failed (%s): %v — cache requests from this job will be unauthenticated and likely return 401", base, err)
reporter.Logf("::warning::%s", runner.EscapeCommandData(fmt.Sprintf(
"cache external_server register failed (%s): %v — cache requests from this job will be unauthenticated and likely return 401", base, err)))
}
} else {
resultsURL, _ = body["results_url"].(string) // absent from a server too old to forward
}
return func() {
if err := postInternalCache(base+"/_internal/revoke", r.cfg.Cache.ExternalSecret,
map[string]string{"token": token}); err != nil {
if _, err := postInternalCache(base+"/_internal/revoke", r.cfg.Cache.ExternalSecret,
map[string]any{"token": token}); err != nil {
log.Warnf("cache external_server revoke failed (%s): %v", base, err)
if reporter != nil {
reporter.Logf("::warning::cache external_server revoke failed (%s): %v", base, err)
reporter.Logf("::warning::%s", runner.EscapeCommandData(fmt.Sprintf(
"cache external_server revoke failed (%s): %v", base, err)))
}
}
}
}, resultsURL
}
func postInternalCache(url, secret string, body map[string]string) error {
func postInternalCache(url, secret string, body map[string]any) (map[string]any, error) {
buf, err := json.Marshal(body)
if err != nil {
return err
return nil, err
}
req, err := http.NewRequest(http.MethodPost, url, bytes.NewReader(buf))
if err != nil {
return err
return nil, err
}
req.Header.Set("Authorization", "Bearer "+secret)
req.Header.Set("Content-Type", "application/json")
client := &http.Client{Timeout: 5 * time.Second}
resp, err := client.Do(req)
if err != nil {
return err
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode/100 != 2 {
return fmt.Errorf("status %d", resp.StatusCode)
return nil, fmt.Errorf("status %d", resp.StatusCode)
}
return nil
answer := map[string]any{}
// A server too old to answer with a body is not an error, it simply tells us nothing.
_ = json.NewDecoder(resp.Body).Decode(&answer)
return answer, nil
}
func (r *Runner) RunningCount() int64 {

View File

@@ -7,7 +7,9 @@ import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"path/filepath"
"strings"
"testing"
@@ -29,7 +31,7 @@ func TestRunner_registerCacheForTask(t *testing.T) {
r := &Runner{cfg: emptyCfg(), cacheHandler: handler}
token := "run-token-123"
unregister := r.registerCacheForTask(token, "owner/repo", nil)
unregister, _ := r.registerCacheForTask(token, "owner/repo", nil)
base := handler.ExternalURL() + "/_apis/artifactcache"
probe := func() int {
@@ -53,7 +55,7 @@ func TestRunner_registerCacheForTask(t *testing.T) {
func TestRunner_registerCacheForTask_NoOps(t *testing.T) {
t.Run("nil cacheHandler", func(t *testing.T) {
r := &Runner{cfg: emptyCfg()}
unregister := r.registerCacheForTask("tok", "owner/repo", nil)
unregister, _ := r.registerCacheForTask("tok", "owner/repo", nil)
require.NotNil(t, unregister)
unregister()
})
@@ -65,7 +67,7 @@ func TestRunner_registerCacheForTask_NoOps(t *testing.T) {
defer handler.Close()
r := &Runner{cfg: emptyCfg(), cacheHandler: handler}
unregister := r.registerCacheForTask("", "owner/repo", nil)
unregister, _ := r.registerCacheForTask("", "owner/repo", nil)
require.NotNil(t, unregister)
unregister()
})
@@ -81,7 +83,7 @@ func TestRunner_CacheFullFlow_MatchesToolkit(t *testing.T) {
r := &Runner{cfg: emptyCfg(), cacheHandler: handler}
token := "full-flow-token"
unregister := r.registerCacheForTask(token, "owner/repo", nil)
unregister, _ := r.registerCacheForTask(token, "owner/repo", nil)
defer unregister()
base := handler.ExternalURL() + "/_apis/artifactcache"
@@ -154,18 +156,27 @@ func decodeJSON(resp *http.Response, v any) error {
}
// End-to-end against a remote cache-server: token unknown → 401, register →
// reserve/upload/commit/find/download all OK, revoke → 401 again.
// reserve/upload/commit/find/download all OK, revoke → 401 again. Registering also names the
// instance, and the server answering with its own address is what makes a shared cache server the
// whole results service, as the built-in one is.
func TestRunner_ExternalCacheServer_RegisterRevoke(t *testing.T) {
dir := filepath.Join(t.TempDir(), "remote-cache")
const secret = "shared-secret-for-tests"
remote, err := artifactcache.StartHandler(dir, "127.0.0.1", 0, secret, nil)
require.NoError(t, err)
defer remote.Close()
gitea := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, `{"ok":true}`)
}))
defer gitea.Close()
r := &Runner{cfg: &config.Config{Cache: config.Cache{
ExternalServer: remote.ExternalURL(),
ExternalSecret: secret,
}}}
r := &Runner{
cfg: &config.Config{Cache: config.Cache{
ExternalServer: remote.ExternalURL(),
ExternalSecret: secret,
}},
envs: map[string]string{"ACTIONS_RESULTS_URL": gitea.URL},
}
token := "external-task-token"
repo := "owner/repoX"
@@ -182,10 +193,21 @@ func TestRunner_ExternalCacheServer_RegisterRevoke(t *testing.T) {
require.Equal(t, http.StatusUnauthorized, probe(),
"token must be unknown to the remote server before registration")
unregister := r.registerCacheForTask(token, repo, nil)
unregister, resultsURL := r.registerCacheForTask(token, repo, nil)
require.NotEqual(t, http.StatusUnauthorized, probe(),
"token must be accepted after registerCacheForTask")
// The server took the results service over, so the artifact half reaches Gitea through it.
require.Equal(t, remote.ExternalURL(), resultsURL)
artifact, err := http.NewRequestWithContext(t.Context(), http.MethodPost,
resultsURL+"/twirp/github.actions.results.api.v1.ArtifactService/CreateArtifact", nil)
require.NoError(t, err)
artifact.Header.Set("Authorization", "Bearer "+token)
forwarded, err := http.DefaultClient.Do(artifact)
require.NoError(t, err)
forwarded.Body.Close()
require.Equal(t, http.StatusOK, forwarded.StatusCode)
// Full reserve→upload→commit→find→download cycle, identical to what
// @actions/cache does, against the remote (external) server.
body := []byte("payload-from-task")

View File

@@ -5,6 +5,8 @@ package run
import (
"context"
"net/http"
"strings"
"testing"
"gitea.com/gitea/runner/act/runner"
@@ -124,21 +126,57 @@ func taskWithDefaultActionsURL(url string) *runnerv1.Task {
}
}
// The cache service v2 API is announced to jobs unless it is turned off. Announcing it is what
// makes act patch the GHES check out of an action's bundle, so the client can reach it.
// The results service is decided per task, because that is where the job's token and its
// instance are both known. NewRunner only leaves the address Gitea serves.
func TestNewRunnerCacheServiceV2(t *testing.T) {
announced := func(v2 *bool) string {
cfg := &config.Config{}
cfg.Cache.V2, cfg.Cache.Dir, cfg.Cache.Host = v2, t.TempDir(), "127.0.0.1"
cli := clientmocks.NewClient(t)
cli.On("Address").Return("https://gitea.example/").Maybe()
cfg := &config.Config{}
cfg.Cache.Dir, cfg.Cache.Host = t.TempDir(), "127.0.0.1"
cli := clientmocks.NewClient(t)
cli.On("Address").Return("https://gitea.example/").Maybe()
r := NewRunner(cfg, &config.Registration{Name: "runner"}, cli)
t.Cleanup(func() { _ = r.Close() })
return r.envs[runner.CacheServiceV2Env]
}
r := NewRunner(cfg, &config.Registration{Name: "runner"}, cli)
t.Cleanup(func() { _ = r.Close() })
const token = "task-token"
off := false
assert.Equal(t, "true", announced(nil))
assert.Empty(t, announced(&off))
assert.Equal(t, "https://gitea.example", r.envs["ACTIONS_RESULTS_URL"])
assert.Empty(t, r.envs[runner.CacheServiceV2Env], "a promise the runner has not made yet")
assert.True(t, r.patchToolkit())
// The registration is what makes it true: the cache server takes the results service over,
// having been told which instance to forward the artifact half to.
revoke, resultsURL := r.registerCacheForTask(token, "owner/repo", nil)
defer revoke()
require.Equal(t, r.cacheHandler.ExternalURL(), resultsURL)
// And what is advertised has to answer the cache service. A client without the GHES escape
// hatch, docker buildx among them, posts its cache calls at exactly this URL and nowhere else,
// so a 404 here is the failure this whole change is about.
req, err := http.NewRequestWithContext(t.Context(), http.MethodPost,
resultsURL+"/twirp/github.actions.results.api.v1.CacheService/GetCacheEntryDownloadURL",
strings.NewReader(`{"key":"k","version":"v"}`))
require.NoError(t, err)
req.Header.Set("Authorization", "Bearer "+token)
resp, err := http.DefaultClient.Do(req)
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode, "the advertised results service serves no cache service")
}
// The v1 cache client appends its path to ACTIONS_CACHE_URL without a separator, so a configured
// server that is missing the slash would send it to a URL whose port swallows the path.
func TestNewRunnerNormalizesTheExternalCacheServer(t *testing.T) {
cfg := &config.Config{}
cfg.Cache.ExternalServer = "http://cache.local:8088//"
cli := clientmocks.NewClient(t)
cli.On("Address").Return("https://gitea.example/").Maybe()
r := NewRunner(cfg, &config.Registration{Name: "runner"}, cli)
assert.Equal(t, "http://cache.local:8088/", r.envs["ACTIONS_CACHE_URL"])
// Nothing to front the results service with, so the variable stays unset, but the bundles are
// still patched: artifacts v4 need that, and the patch keeps the cache client on the cache URL.
assert.Equal(t, "https://gitea.example", r.envs["ACTIONS_RESULTS_URL"])
assert.Empty(t, r.envs[runner.CacheServiceV2Env])
assert.True(t, r.patchToolkit())
}

View File

@@ -1,7 +1,5 @@
# Example configuration file, it's safe to copy this as the default config file without any modification.
# You don't have to copy this file to your instance,
# just run `./gitea-runner generate-config > config.yaml` to generate a config file.
# Every option with its default value, all commented out. Read this file, do not copy it.
# `./gitea-runner config init` writes a config file to copy the lines you change into.
# Logging for the runner process itself (messages printed to stderr).
# This does not control how workflow step output is streamed to the Gitea UI;
@@ -9,92 +7,92 @@
log:
# logrus severity: trace, debug, info, warn, error, fatal, panic.
# trace and debug turn on caller/file:line in log lines. Default if omitted: info.
level: info
#level: info
runner:
# Where to store the registration result.
file: .runner
#file: .runner
# Execute how many tasks concurrently at the same time.
# With `container.network` empty, each concurrent docker job takes a subnet from the
# daemon's address pool, so a high capacity can exhaust it. See `default-address-pools`
# in the docker daemon config.
capacity: 1
#capacity: 1
# Extra environment variables to run jobs.
envs:
A_TEST_ENV_NAME_1: a_test_env_value_1
A_TEST_ENV_NAME_2: a_test_env_value_2
#envs:
# A_TEST_ENV_NAME_1: a_test_env_value_1
# A_TEST_ENV_NAME_2: a_test_env_value_2
# Extra environment variables to run jobs from a file.
# It will be ignored if it's empty or the file doesn't exist.
env_file: .env
#env_file: .env
# The timeout for a job to be finished.
# Please note that the Gitea instance also has a timeout (3h by default) for the job.
# So the job could be stopped by the Gitea instance if its timeout is shorter than this.
timeout: 3h
#timeout: 3h
# The timeout for the runner to wait for running jobs to finish when shutting down.
# Any running jobs that haven't finished after this timeout will be cancelled.
shutdown_timeout: 0s
#shutdown_timeout: 0s
# Whether skip verifying the TLS certificate of the Gitea instance.
insecure: false
#insecure: false
# The timeout for fetching the job from the Gitea instance.
fetch_timeout: 5s
#fetch_timeout: 5s
# The interval for fetching the job from the Gitea instance.
fetch_interval: 2s
#fetch_interval: 2s
# The maximum interval for fetching the job from the Gitea instance.
# The runner uses exponential backoff when idle, increasing the interval up to this maximum.
# Set to 0 or same as fetch_interval to disable backoff.
fetch_interval_max: 5s
#fetch_interval_max: 5s
# While idle, remove stale bind-workdir task directories and orphaned host-mode
# scratch directories (left behind when a host cleanup delete stalls) older than
# this duration. Setting either workdir_cleanup_age or idle_cleanup_interval to 0
# (or any non-positive value) disables stale-directory cleanup entirely, along with
# the docker network cleanup below.
workdir_cleanup_age: 24h
#workdir_cleanup_age: 24h
# Cadence for the idle cleanup pass. Besides the directories above, on runners that use
# docker it removes the per-job networks of jobs this runner did not live to tear down,
# which would otherwise hold a subnet of the daemon address pool until the host is rebuilt.
idle_cleanup_interval: 10m
#idle_cleanup_interval: 10m
# The base interval for periodic log flush to the Gitea instance.
# Logs may be sent earlier if the buffer reaches log_report_batch_size
# or if log_report_max_latency expires after the first buffered row.
log_report_interval: 5s
#log_report_interval: 5s
# The maximum time a log row can wait before being sent.
# This ensures even a single log line appears on the frontend within this duration.
# Must be less than log_report_interval to have any effect.
log_report_max_latency: 3s
#log_report_max_latency: 3s
# Flush logs immediately when the buffer reaches this many rows.
# This ensures bursty output (e.g., npm install) is delivered promptly.
log_report_batch_size: 100
#log_report_batch_size: 100
# The interval for reporting task state (step status, timing) to the Gitea instance.
# State is also reported immediately on step transitions (start/stop).
state_report_interval: 5s
#state_report_interval: 5s
# Per-attempt deadline for flushing the final logs and task state when a job
# finishes, on a detached context so a server cancel can't block the acknowledgement.
report_close_timeout: 10s
#report_close_timeout: 10s
# The github_mirror of a runner is used to specify the mirror address of the github that pulls the action repository.
# It works when something like `uses: actions/checkout@v4` is used and DEFAULT_ACTIONS_URL is set to github,
# and github_mirror is not empty. In this case,
# it replaces https://github.com with the value here, which is useful for some special network environments.
github_mirror: ''
#github_mirror: ''
# When true (the default), fetch only the requested ref of an action repository (e.g. actions/checkout@v4) at depth 1 instead of cloning every branch's full history.
# Set to false to clone the full history.
action_shallow_clone: true
#action_shallow_clone: true
# When true (the default), inject the ACT=true environment variable into jobs.
# Set to false so workflows gated on `if: ${{ !env.ACT }}` behave like they do on GitHub.
set_act_env: true
#set_act_env: true
# The labels of a runner are used to determine which jobs the runner can run, and how to run them.
# Like: "macos-arm64:host" or "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"
# Find more images provided by Gitea at https://gitea.com/gitea/runner-images .
# If it's empty when registering, it will ask for inputting labels.
# If it's empty when execute `daemon`, will use labels in `.runner` file.
labels:
- "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"
- "ubuntu-24.04:docker://docker.gitea.com/runner-images:ubuntu-24.04"
- "ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"
#labels:
# - "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"
# - "ubuntu-24.04:docker://docker.gitea.com/runner-images:ubuntu-24.04"
# - "ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"
# Allocate a pseudo-TTY for each step's process. Applies to both host and docker backends.
# Default false matches GitHub actions/runner. Enable only for jobs that need an interactive
# terminal; tools like `docker build` emit redrawing progress frames into the captured log
# when a TTY is present.
allocate_pty: false
#allocate_pty: false
# Optional executable on the host, run once after each task's built-in cleanup
# (post-steps, container teardown, bind-workdir removal). Additive only.
#
@@ -107,24 +105,24 @@ runner:
# Windows: use .exe, .bat, or .cmd. PowerShell (.ps1) is not supported yet as
# the configured path; wrap PowerShell commands in a .cmd file instead.
# Full guide: docs/post-task-script.md
post_task_script: ''
#post_task_script: ''
# Hard limit on post_task_script runtime. Default if omitted: 5m.
post_task_script_timeout: 5m
#post_task_script_timeout: 5m
# Scripts run inside the job environment before the job's first step and after its last
# one, the equivalent of GitHub's ACTIONS_RUNNER_HOOK_JOB_STARTED and
# ACTIONS_RUNNER_HOOK_JOB_COMPLETED, which are read when these are unset. The paths are
# resolved inside the job environment. Either one failing fails the job.
# Full guide: docs/job-hooks.md
hooks:
job_started: ''
job_completed: ''
#hooks:
# job_started: ''
# job_completed: ''
cache:
# Enable the built-in cache server (used by actions/cache and similar actions).
enabled: true
#enabled: true
# Directory where cache blobs are stored on disk. Default: $HOME/.cache/actcache
# Ignored when external_server is set.
dir: ""
#dir: ""
# Outbound IP or hostname that job containers use to reach this runner's cache server.
# Leave empty to detect automatically. 0.0.0.0 is not valid here.
# If the runner itself runs in Docker, automatic detection can choose an
@@ -133,33 +131,33 @@ cache:
# to a hostname/IP reachable from job containers, and set port to a fixed
# published port or put the job containers on a shared Docker network.
# Ignored when external_server is set.
host: ""
#host: ""
# Port for the built-in cache server. 0 picks a random free port.
# Ignored when external_server is set.
port: 0
#port: 0
# URL of a shared `gitea-runner cache-server` to use instead of starting a local one.
# Set on every runner that should share a cache pool. Must end with "/".
# Set on every runner that should share a cache pool. A trailing slash is optional.
# Example: "http://cache-host:8088/"
# Requires external_secret (below) to match the value on the cache-server.
external_server: ""
#external_server: ""
# Shared secret between this runner and the external cache-server.
# Required when external_server is set. Must be identical on every runner and the cache-server.
# Generate with: openssl rand -hex 32
external_secret: ""
#external_secret: ""
# Path to a file containing the shared secret, as an alternative to external_secret.
# Use this to keep the secret out of this file.
# Surrounding whitespace is trimmed, so a trailing newline in the file is fine.
# Setting both external_secret and external_secret_file is an error.
external_secret_file: ""
#external_secret_file: ""
# When true, reuse a cached action instead of fetching from the remote on every job.
# A moved tag (e.g. a re-tagged "v6") or an updated branch stays at the cached commit
# until its cache entry expires or is manually removed.
offline_mode: false
#offline_mode: false
# Serve the actions cache service v2 API, used by actions/cache@v4.2 and later. Those actions
# refuse any host they do not take for GitHub, so reaching it means editing that check out of
# the action's own bundle, keeping the untouched copy beside it. The same edit lets the stock
# upload-artifact and download-artifact work here. A bundle that does not match is left alone.
v2: true
#v2: true
container:
# Specifies the network to which the container will connect.
@@ -168,31 +166,31 @@ container:
# For dockerized runners using the built-in cache server, a custom shared
# network can be required so job containers can reach cache.host/cache.port.
# Deprecated: `network_mode` is still accepted for old configs; use `network` instead.
network: ""
#network: ""
# network_create_options only apply when `network` is left empty and the runner
# auto-creates a per-job network that does not already exist. They have no effect
# when a custom `network` name is set, because that network is used as-is and never
# created by the runner. Omit the entire block to use Docker's defaults. An auto-created
# network is labelled com.gitea.runner.uuid=<this runner's uuid>, which is how the idle
# cleanup tells its own leftovers apart from those of other runners on the same daemon.
network_create_options:
enable_ipv4: true # Omit to use Docker's default (IPv4 enabled). Set false to disable IPv4.
enable_ipv6: false # Omit to use Docker's default (IPv6 disabled). Enabling it requires dockerd started with --ipv6.
#network_create_options:
# enable_ipv4: true # Omit to use Docker's default (IPv4 enabled). Set false to disable IPv4.
# enable_ipv6: false # Omit to use Docker's default (IPv6 disabled). Enabling it requires dockerd started with --ipv6.
# Whether to use privileged mode or not when launching task containers (privileged mode is required for Docker-in-Docker).
privileged: false
#privileged: false
# Any other options to be used when the container is started, for example:
# options: --add-host=my.gitea.url:host-gateway
# A volume declared here replaces the one the runner mounts on the same container path, so the
# tool cache can be kept on the host. Its source must also be allowed by valid_volumes below:
# options: --volume /host/toolcache:/opt/hostedtoolcache
options:
#options:
# The parent directory of a job's working directory.
# NOTE: There is no need to add the first '/' of the path as runner will add it automatically.
# If the path starts with '/', the '/' will be trimmed.
# For example, if the parent directory is /path/to/my/dir, workdir_parent should be path/to/my/dir
# If it's empty, /workspace will be used.
# Purely numeric subdirectories under this path are reserved for task workspaces and may be removed by idle cleanup.
workdir_parent:
#workdir_parent:
# Volumes (including bind mounts) can be mounted to containers. Glob syntax is supported, see https://github.com/gobwas/glob
# You can specify multiple volumes. If the sequence is empty, no volumes can be mounted.
# For example, if you only allow containers to mount the `data` volume and all the json files in `/src`, you should change the config to:
@@ -202,59 +200,62 @@ container:
# If you want to allow any volume, please use the following configuration:
# valid_volumes:
# - '**'
valid_volumes: []
#valid_volumes: []
# Overrides the docker client host with the specified one.
# If it's empty, runner will find an available docker host automatically.
# If it's "-", runner will find an available docker host automatically, but the docker host won't be mounted to the job containers and service containers.
# If it's not empty or "-", the specified docker host will be used. An error will be returned if it doesn't work.
docker_host: ""
#docker_host: ""
# Pull docker image(s) even if already present.
# Defaults to false when the key is omitted.
# Two exceptions: an image pinned by digest (image@sha256:...) cannot change, so it is never
# re-pulled, and a pull that fails while a copy is already on the host does not fail the job,
# which runs on that copy with a warning in its log.
force_pull: false
#force_pull: false
# Rebuild docker image(s) even if already present
force_rebuild: false
#force_rebuild: false
# Always require a reachable docker daemon, even if not required by runner
require_docker: false
#require_docker: false
# Timeout to wait for the docker daemon to be reachable, if docker is required by require_docker or runner
docker_timeout: 0s
#docker_timeout: 0s
# Bind the workspace to the host filesystem instead of using Docker volumes.
# This is required for Docker-in-Docker (DinD) setups when jobs use docker compose
# with bind mounts (e.g., ".:/app"), as volume-based workspaces are not accessible
# from the DinD daemon's filesystem. When enabled, ensure the workspace parent
# directory is also mounted into the runner container and listed in valid_volumes.
bind_workdir: false
#bind_workdir: false
# How long a job waits for a service container that declares a healthcheck to become
# healthy. A negative value (e.g. -1s) starts the steps without waiting.
#service_ready_timeout: 5m
host:
# The parent directory of a job's working directory.
# If it's empty, $HOME/.cache/act/ will be used.
workdir_parent:
#workdir_parent:
# Optional local task-admission checks. Disabled by default. When enabled, low
# disk space or a failing script pauses new task fetching; existing jobs continue.
# No health checks run while any job is active; the last result is reused until idle.
health_check:
enabled: false
#enabled: false
# Minimum free space required on the filesystem holding runner workspaces.
# Defaults to 1024 MiB when omitted or set to zero.
min_free_disk_space_mb: 1024
#min_free_disk_space_mb: 1024
# Optional additional executable. A non-zero exit, timeout, or startup failure
# marks the runner unavailable.
script: ''
#script: ''
# How long a script result is cached and its maximum execution time.
interval: 30s
timeout: 10s
#interval: 30s
#timeout: 10s
metrics:
# Enable the Prometheus metrics endpoint.
# When enabled, metrics are served at /metrics, liveness at /healthz, and
# task-admission readiness at /readyz.
enabled: false
#enabled: false
# The address for the metrics HTTP server to listen on.
# Defaults to localhost only. Set to ":9101" to allow external access,
# but ensure the port is firewall-protected as there is no authentication.
addr: "127.0.0.1:9101"
#addr: "127.0.0.1:9101"
# Consecutive polling failures may last this long before /readyz returns 503.
readiness_grace: 30s
#readiness_grace: 30s

View File

@@ -24,6 +24,12 @@ import (
// (so a programmatically built config still gets a sane bound).
const DefaultPostTaskScriptTimeout = 5 * time.Minute
// Minimal is the smallest config file that runs the runner: options it does not
// name keep their default, and it names none.
const Minimal = `# Minimal config file. Every option it does not set keeps its default.
# "gitea-runner config generate" prints all options, "config set <key> <value>" sets one here.
`
// Log represents the configuration for logging.
type Log struct {
Level string `yaml:"level"` // Level indicates the logging level.
@@ -92,6 +98,7 @@ type Container struct {
RequireDocker bool `yaml:"require_docker"` // Always require a reachable docker daemon, even if not required by runner
DockerTimeout time.Duration `yaml:"docker_timeout"` // Timeout to wait for the docker daemon to be reachable, if docker is required by require_docker or runner
BindWorkdir bool `yaml:"bind_workdir"` // BindWorkdir binds the workspace to the host filesystem instead of using Docker volumes. Required for DinD when jobs use docker compose with bind mounts.
ServiceReadyTimeout time.Duration `yaml:"service_ready_timeout"` // ServiceReadyTimeout bounds how long a job waits for a service container that declares a healthcheck to report healthy. Negative disables waiting.
}
type ContainerNetworkCreateOptions struct {

View File

@@ -348,12 +348,23 @@ cache:
assert.Contains(t, err.Error(), "contains no secret")
}
// The shipped example must parse, and every key in it must be one the config knows.
func TestLoadDefault_ExampleConfigParses(t *testing.T) {
// The shipped configs must parse, hold no key the config does not know, and leave
// every option at its default, as all of their values are commented out.
func TestLoadDefault_ShippedConfigsChangeNothing(t *testing.T) {
hook := test.NewGlobal()
defer hook.Reset()
_, err := LoadDefault("config.example.yaml")
defaults, err := LoadDefault("")
require.NoError(t, err)
dir := t.TempDir()
for name, content := range map[string][]byte{"config.example.yaml": Example, "minimal.yaml": []byte(Minimal)} {
file := filepath.Join(dir, name)
require.NoError(t, os.WriteFile(file, content, 0o600))
cfg, err := LoadDefault(file)
require.NoError(t, err, name)
assert.Equal(t, defaults, cfg, name)
}
assert.Empty(t, hook.AllEntries())
}

564
internal/pkg/config/edit.go Normal file
View File

@@ -0,0 +1,564 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package config
import (
"bytes"
"errors"
"fmt"
"os"
"path/filepath"
"reflect"
"slices"
"strconv"
"strings"
"time"
"go.yaml.in/yaml/v4"
)
type fieldKind int
const (
kindScalar fieldKind = iota
kindSequence
kindSection
)
var durationType = reflect.TypeFor[time.Duration]()
// GetValue renders a flat list or mapping one entry per line, and anything nested as YAML.
func GetValue(file, path string) (string, error) {
session, err := loadForEdit(file, path)
if err != nil {
return "", err
}
node, err := lookupNode(session.root, session.segments)
if err != nil {
return "", err
}
return renderNode(node)
}
func SetValue(file, path string, values ...string) error {
session, err := loadForEdit(file, path)
if err != nil {
return err
}
var replacement *yaml.Node
switch session.field.kind {
case kindSequence:
if len(values) == 0 {
return fmt.Errorf("%q needs at least one value", path)
}
items, err := session.scalars(values)
if err != nil {
return err
}
replacement = &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq", Content: items}
case kindScalar:
if len(values) != 1 {
return fmt.Errorf("%q takes exactly one value", path)
}
items, err := session.scalars(values)
if err != nil {
return err
}
replacement = items[0]
default:
return fmt.Errorf("%q is a section, set one of its keys instead", path)
}
node, err := ensureNode(session.root, session.segments)
if err != nil {
return err
}
replaceNode(node, replacement)
return session.write()
}
func AddValue(file, path string, values ...string) error {
session, err := loadSequenceEdit(file, path, values)
if err != nil {
return err
}
items, err := session.scalars(values)
if err != nil {
return err
}
node, err := ensureNode(session.root, session.segments)
if err != nil {
return err
}
if node.Kind != yaml.SequenceNode {
replaceNode(node, &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq"})
}
for _, item := range items {
if indexOfValue(node, item.Value) >= 0 {
return fmt.Errorf("%s already contains %q", path, item.Value)
}
node.Content = append(node.Content, item)
}
return session.write()
}
func RemoveValue(file, path string, values ...string) error {
session, err := loadSequenceEdit(file, path, values)
if err != nil {
return err
}
items, err := session.scalars(values)
if err != nil {
return err
}
node, err := lookupNode(session.root, session.segments)
if err != nil {
return err
}
if node.Kind != yaml.SequenceNode {
return fmt.Errorf("%s is not a list in %q", path, file)
}
for _, item := range items {
index := indexOfValue(node, item.Value)
if index < 0 {
return fmt.Errorf("%s does not contain %q", path, item.Value)
}
node.Content = slices.Delete(node.Content, index, index+1)
}
return session.write()
}
func indexOfValue(seq *yaml.Node, value string) int {
for i, item := range seq.Content {
if item.Kind == yaml.ScalarNode && item.Value == value {
return i
}
}
return -1
}
// replaceNode assigns field by field, as *node = *with would drop the comments attached to node.
func replaceNode(node, with *yaml.Node) {
node.Kind, node.Tag, node.Style, node.Value, node.Content = with.Kind, with.Tag, with.Style, with.Value, with.Content
}
type editSession struct {
file string
path string
original []byte
root *yaml.Node
field *fieldInfo
segments []string
preamble []byte // text of a file that holds no YAML node, which the encoder cannot give back
}
// loadForEdit validates the path and parses the file, so every caller fails before anything is written.
func loadForEdit(file, path string) (*editSession, error) {
if path == "" {
return nil, errors.New("no config key given")
}
segments := strings.Split(path, ".")
field, err := resolvePath(segments)
if err != nil {
return nil, err
}
content, err := os.ReadFile(file)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return nil, fmt.Errorf("config file %q does not exist, create one with `config init`", file)
}
return nil, err
}
var root yaml.Node
if err := yaml.Unmarshal(content, &root); err != nil {
return nil, fmt.Errorf("parse config file %q: %w", file, err)
}
var preamble []byte
if root.Kind == 0 || len(root.Content) == 0 {
preamble = bytes.TrimSpace(content) // all the file has is comments
root = yaml.Node{
Kind: yaml.DocumentNode,
Content: []*yaml.Node{{Kind: yaml.MappingNode, Tag: "!!map"}},
}
}
if root.Content[0].Kind != yaml.MappingNode {
return nil, fmt.Errorf("config file %q is not a YAML mapping", file)
}
return &editSession{file: file, path: path, original: content, root: &root, field: field, segments: segments, preamble: preamble}, nil
}
func loadSequenceEdit(file, path string, values []string) (*editSession, error) {
session, err := loadForEdit(file, path)
if err != nil {
return nil, err
}
if session.field.kind != kindSequence {
return nil, fmt.Errorf("%q is not a list, use `config set` instead", path)
}
if len(values) == 0 {
return nil, fmt.Errorf("%q needs at least one value", path)
}
return session, nil
}
func (s *editSession) scalars(values []string) ([]*yaml.Node, error) {
nodes := make([]*yaml.Node, 0, len(values))
for _, value := range values {
node, err := scalarNode(s.field.typ, value)
if err != nil {
return nil, fmt.Errorf("%s: %w", s.path, err)
}
nodes = append(nodes, node)
}
return nodes, nil
}
func lookupNode(root *yaml.Node, segments []string) (*yaml.Node, error) {
node := root.Content[0]
for i, segment := range segments {
if node.Kind != yaml.MappingNode {
return nil, fmt.Errorf("%q is not set", strings.Join(segments[:i], "."))
}
value := mappingValue(node, segment)
if value == nil {
return nil, fmt.Errorf("%q is not set", strings.Join(segments[:i+1], "."))
}
node = value
}
return node, nil
}
func ensureNode(root *yaml.Node, segments []string) (*yaml.Node, error) {
node := root.Content[0]
for i, segment := range segments {
if node.Kind != yaml.MappingNode {
if node.Kind == yaml.ScalarNode && node.Tag == "!!null" {
node.Kind, node.Tag, node.Style, node.Value = yaml.MappingNode, "!!map", 0, ""
} else {
return nil, fmt.Errorf("%q is not a section", strings.Join(segments[:i], "."))
}
}
value := mappingValue(node, segment)
if value == nil {
value = &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!null"}
node.Content = append(node.Content,
&yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: segment},
value)
}
node = value
}
return node, nil
}
func mappingValue(mapping *yaml.Node, key string) *yaml.Node {
for i := 0; i+1 < len(mapping.Content); i += 2 {
if mapping.Content[i].Value == key {
return mapping.Content[i+1]
}
}
return nil
}
func renderNode(node *yaml.Node) (string, error) {
if !allScalars(node.Content) {
encoded, err := encodeYAML(node)
if err != nil {
return "", err
}
return strings.TrimRight(string(encoded), "\n"), nil
}
switch node.Kind {
case yaml.SequenceNode:
lines := make([]string, 0, len(node.Content))
for _, item := range node.Content {
lines = append(lines, item.Value)
}
return strings.Join(lines, "\n"), nil
case yaml.MappingNode:
lines := make([]string, 0, len(node.Content)/2)
for i := 0; i+1 < len(node.Content); i += 2 {
lines = append(lines, node.Content[i].Value+"="+node.Content[i+1].Value)
}
return strings.Join(lines, "\n"), nil
default:
return node.Value, nil
}
}
func allScalars(nodes []*yaml.Node) bool {
for _, node := range nodes {
if node.Kind != yaml.ScalarNode {
return false
}
}
return true
}
func encodeYAML(node *yaml.Node) ([]byte, error) {
var buf bytes.Buffer
encoder := yaml.NewEncoder(&buf)
encoder.SetIndent(2)
if err := encoder.Encode(node); err != nil {
return nil, err
}
if err := encoder.Close(); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// restoreLayout re-applies the spacing the encoder drops: the blank lines between
// top-level sections, and the indentation of comments, which the encoder emits at the
// indentation of the node it attached them to rather than the one they were written at.
func restoreLayout(original, generated []byte) []byte {
type comment struct {
line string // as written, indentation included
trimmed string
blankBefore bool
}
var comments []comment
spaced := map[string]bool{}
blank := false
for line := range strings.Lines(string(original)) {
line = strings.TrimRight(line, "\r\n")
trimmed := strings.TrimSpace(line)
switch {
case trimmed == "":
blank = true
case strings.HasPrefix(trimmed, "#"):
comments = append(comments, comment{line: line, trimmed: trimmed, blankBefore: blank})
blank = false
default:
if key, ok := topLevelKey(line); ok && blank {
spaced[key] = true
}
blank = false
}
}
var out []string
appendBlank := func() {
if len(out) > 0 && strings.TrimSpace(out[len(out)-1]) != "" {
out = append(out, "")
}
}
for line := range strings.Lines(string(generated)) {
line = strings.TrimRight(line, "\r\n")
trimmed := strings.TrimSpace(line)
if strings.HasPrefix(trimmed, "#") {
if i := slices.IndexFunc(comments, func(c comment) bool { return c.trimmed == trimmed }); i >= 0 {
if comments[i].blankBefore {
appendBlank()
} else if len(out) > 0 && strings.TrimSpace(out[len(out)-1]) == "" {
out = out[:len(out)-1] // the encoder separates a comment block it moved
}
line = comments[i].line
comments = comments[i+1:] // the encoder keeps their order, so earlier ones cannot match again
}
} else if key, ok := topLevelKey(line); ok && spaced[key] {
appendBlank()
}
out = append(out, line)
}
if bytes.HasSuffix(generated, []byte("\n")) {
out = append(out, "")
}
return []byte(strings.Join(out, "\n"))
}
func topLevelKey(line string) (string, bool) {
if line == "" || line[0] == ' ' || line[0] == '\t' || line[0] == '#' || line[0] == '-' {
return "", false
}
key, _, ok := strings.Cut(line, ":")
return key, ok
}
func (s *editSession) write() error {
generated, err := encodeYAML(s.root)
if err != nil {
return err
}
// A file the runner already refused to load stays the user's to fix, only a regression is rejected.
if err := yaml.Unmarshal(generated, &Config{}); err != nil && yaml.Unmarshal(s.original, &Config{}) == nil {
return fmt.Errorf("the edit would produce a config the runner cannot load: %w", err)
}
if len(s.preamble) > 0 { // before restoreLayout, so that it spaces the preamble too
generated = slices.Concat(s.preamble, []byte("\n"), generated)
}
content := restoreLayout(s.original, generated)
if bytes.Contains(s.original, []byte("\r\n")) { // the encoder only ever emits LF
content = bytes.ReplaceAll(content, []byte("\n"), []byte("\r\n"))
}
return WriteFile(s.file, content)
}
// WriteFile replaces the config file in one step, keeping the mode and owner of the
// file it replaces, so a half-written config never reaches a runner reading it.
func WriteFile(file string, content []byte) error {
if resolved, err := filepath.EvalSymlinks(file); err == nil {
file = resolved // keeps a config linked in from elsewhere intact
}
var info os.FileInfo
mode := os.FileMode(0o600)
if stat, err := os.Stat(file); err == nil {
info, mode = stat, stat.Mode().Perm()
}
temp, err := os.CreateTemp(filepath.Dir(file), filepath.Base(file)+".*.tmp")
if err != nil {
return err
}
defer os.Remove(temp.Name())
if _, err := temp.Write(content); err != nil {
temp.Close()
return err
}
if err := temp.Sync(); err != nil {
temp.Close()
return err
}
if err := temp.Close(); err != nil {
return err
}
if info != nil { // before the chmod, as a chown can clear mode bits
if err := preserveOwner(temp.Name(), info); err != nil {
return err
}
}
if err := os.Chmod(temp.Name(), mode); err != nil {
return err
}
return os.Rename(temp.Name(), file)
}
type fieldInfo struct {
kind fieldKind
typ reflect.Type // the element type for a sequence
}
// resolvePath walks the Config struct through the yaml tags of a dotted path.
func resolvePath(segments []string) (*fieldInfo, error) {
typ := reflect.TypeFor[Config]()
for i, segment := range segments {
switch typ.Kind() {
case reflect.Struct:
field, ok := fieldByYAMLName(typ, segment)
if !ok {
return nil, fmt.Errorf("unknown config key %q, valid keys here: %s",
strings.Join(segments[:i+1], "."), strings.Join(yamlNames(typ), ", "))
}
typ = field.Type
case reflect.Map:
// The segment names a user-defined entry, so the walk ends here.
if i != len(segments)-1 {
return nil, fmt.Errorf("%q has no sub-keys", strings.Join(segments[:i+1], "."))
}
return &fieldInfo{kind: kindScalar, typ: typ.Elem()}, nil
default:
return nil, fmt.Errorf("%q is a value, not a section", strings.Join(segments[:i], "."))
}
}
switch typ.Kind() {
case reflect.Slice:
return &fieldInfo{kind: kindSequence, typ: typ.Elem()}, nil
case reflect.Map, reflect.Struct:
return &fieldInfo{kind: kindSection}, nil
default:
return &fieldInfo{kind: kindScalar, typ: typ}, nil
}
}
func fieldByYAMLName(typ reflect.Type, name string) (reflect.StructField, bool) {
for field := range typ.Fields() {
if yamlName(field) == name {
return field, true
}
}
return reflect.StructField{}, false
}
func yamlNames(typ reflect.Type) []string {
names := make([]string, 0, typ.NumField())
for field := range typ.Fields() {
if name := yamlName(field); name != "-" {
names = append(names, name)
}
}
slices.Sort(names)
return names
}
func yamlName(field reflect.StructField) string {
name, _, _ := strings.Cut(field.Tag.Get("yaml"), ",")
if name == "" {
return strings.ToLower(field.Name)
}
return name
}
// scalarNode types the value, so a bad one is reported instead of landing in the file as a string.
func scalarNode(typ reflect.Type, value string) (*yaml.Node, error) {
if typ.Kind() == reflect.Pointer {
typ = typ.Elem()
}
if typ == durationType {
duration, err := time.ParseDuration(value)
if err != nil {
return nil, fmt.Errorf("%q is not a duration such as 30s, 5m or 3h", value)
}
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: duration.String()}, nil
}
switch typ.Kind() {
case reflect.Bool:
parsed, err := strconv.ParseBool(value)
if err != nil {
return nil, fmt.Errorf("%q is not a boolean", value)
}
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!bool", Value: strconv.FormatBool(parsed)}, nil
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
parsed, err := strconv.ParseInt(value, 10, typ.Bits())
if err != nil {
return nil, fmt.Errorf("%q is not a valid %s", value, typ.Kind())
}
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!int", Value: strconv.FormatInt(parsed, 10)}, nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
parsed, err := strconv.ParseUint(value, 10, typ.Bits())
if err != nil {
return nil, fmt.Errorf("%q is not a valid %s", value, typ.Kind())
}
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!int", Value: strconv.FormatUint(parsed, 10)}, nil
case reflect.String:
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: value}, nil
default:
return nil, fmt.Errorf("unsupported config value type %s", typ)
}
}

View File

@@ -0,0 +1,13 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build windows || plan9
package config
import "os"
// preserveOwner is a no-op where a new file inherits its ownership from the directory.
func preserveOwner(_ string, _ os.FileInfo) error {
return nil
}

View File

@@ -0,0 +1,290 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package config
import (
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const editFixture = `# A leading comment.
log:
# The logging level.
level: info
runner:
capacity: 1
envs:
EXISTING: value
timeout: 3h
labels:
- ubuntu-latest:docker://node:20
- self-hosted
`
func writeEditFixture(t *testing.T) string {
t.Helper()
path := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(path, []byte(editFixture), 0o600))
return path
}
func TestEditValues(t *testing.T) {
tests := []struct {
name string
edit func(file string) error
assert func(t *testing.T, cfg *Config, content string)
}{
{
name: "set scalar",
edit: func(file string) error { return SetValue(file, "runner.capacity", "4") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, 4, cfg.Runner.Capacity)
},
},
{
name: "set duration",
edit: func(file string) error { return SetValue(file, "runner.timeout", "90m") },
assert: func(t *testing.T, cfg *Config, content string) {
assert.Equal(t, 90*time.Minute, cfg.Runner.Timeout)
assert.Contains(t, content, "timeout: 1h30m0s")
},
},
{
name: "set a pointer field in a missing section",
edit: func(file string) error {
return SetValue(file, "container.network_create_options.enable_ipv4", "false")
},
assert: func(t *testing.T, cfg *Config, _ string) {
require.NotNil(t, cfg.Container.NetworkCreateOptions.EnableIPv4)
assert.False(t, *cfg.Container.NetworkCreateOptions.EnableIPv4)
},
},
{
name: "set map entry",
edit: func(file string) error { return SetValue(file, "runner.envs.ADDED", "yes") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, map[string]string{"EXISTING": "value", "ADDED": "yes"}, cfg.Runner.Envs)
},
},
{
name: "set replaces a list",
edit: func(file string) error { return SetValue(file, "runner.labels", "one", "two") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, []string{"one", "two"}, cfg.Runner.Labels)
},
},
{
name: "add appends to a list",
edit: func(file string) error { return AddValue(file, "runner.labels", "ubuntu:docker://node:22") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, []string{"ubuntu-latest:docker://node:20", "self-hosted", "ubuntu:docker://node:22"}, cfg.Runner.Labels)
},
},
{
name: "remove drops a list entry",
edit: func(file string) error { return RemoveValue(file, "runner.labels", "self-hosted") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, []string{"ubuntu-latest:docker://node:20"}, cfg.Runner.Labels)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
file := writeEditFixture(t)
require.NoError(t, tt.edit(file))
raw, err := os.ReadFile(file)
require.NoError(t, err)
content := string(raw)
cfg, err := LoadDefault(file)
require.NoError(t, err)
tt.assert(t, cfg, content)
assert.Contains(t, content, "# A leading comment.")
assert.Contains(t, content, " # The logging level.")
assert.Contains(t, content, "\n\nrunner:")
})
}
}
func TestEditValuesRejectsBadInput(t *testing.T) {
tests := []struct {
name string
edit func(file string) error
wantErr string
}{
{
name: "unknown key",
edit: func(file string) error { return SetValue(file, "runner.labl", "x") },
wantErr: `unknown config key "runner.labl"`,
},
{
name: "value is not a number",
edit: func(file string) error { return SetValue(file, "runner.capacity", "many") },
wantErr: `"many" is not a valid int`,
},
{
name: "value is not a duration",
edit: func(file string) error { return SetValue(file, "runner.timeout", "soon") },
wantErr: `"soon" is not a duration`,
},
{
name: "value is not a boolean",
edit: func(file string) error { return SetValue(file, "runner.insecure", "maybe") },
wantErr: `"maybe" is not a boolean`,
},
{
name: "set needs a single value",
edit: func(file string) error { return SetValue(file, "runner.capacity", "1", "2") },
wantErr: "takes exactly one value",
},
{
name: "set on a section",
edit: func(file string) error { return SetValue(file, "runner", "x") },
wantErr: "is a section",
},
{
name: "add on a scalar",
edit: func(file string) error { return AddValue(file, "runner.capacity", "4") },
wantErr: "is not a list",
},
{
name: "add a duplicate",
edit: func(file string) error { return AddValue(file, "runner.labels", "self-hosted") },
wantErr: `already contains "self-hosted"`,
},
{
name: "remove a missing entry",
edit: func(file string) error { return RemoveValue(file, "runner.labels", "absent") },
wantErr: `does not contain "absent"`,
},
{
name: "sub-key of a free-form map entry",
edit: func(file string) error { return SetValue(file, "runner.envs.A.B", "x") },
wantErr: "has no sub-keys",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
file := writeEditFixture(t)
err := tt.edit(file)
require.Error(t, err)
assert.Contains(t, err.Error(), tt.wantErr)
content, err := os.ReadFile(file)
require.NoError(t, err)
assert.Equal(t, editFixture, string(content), "a rejected edit must leave the file untouched")
})
}
}
func TestGetValue(t *testing.T) {
file := writeEditFixture(t)
value, err := GetValue(file, "runner.capacity")
require.NoError(t, err)
assert.Equal(t, "1", value)
value, err = GetValue(file, "runner.labels")
require.NoError(t, err)
assert.Equal(t, "ubuntu-latest:docker://node:20\nself-hosted", value)
value, err = GetValue(file, "runner.envs")
require.NoError(t, err)
assert.Equal(t, "EXISTING=value", value)
// A section has no single-line rendering.
value, err = GetValue(file, "runner")
require.NoError(t, err)
assert.Contains(t, value, "labels:\n - ubuntu-latest:docker://node:20")
_, err = GetValue(file, "metrics.addr")
require.Error(t, err)
assert.Contains(t, err.Error(), "is not set")
}
func TestEditValuesFileHandling(t *testing.T) {
t.Run("reports a missing file", func(t *testing.T) {
err := SetValue(filepath.Join(t.TempDir(), "absent.yaml"), "runner.capacity", "4")
require.Error(t, err)
assert.Contains(t, err.Error(), "does not exist")
})
t.Run("writes through a symlink", func(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "real.yaml")
link := filepath.Join(dir, "config.yaml")
require.NoError(t, os.WriteFile(target, []byte(editFixture), 0o600))
require.NoError(t, os.Symlink(target, link))
require.NoError(t, SetValue(link, "runner.capacity", "4"))
info, err := os.Lstat(link)
require.NoError(t, err)
assert.NotZero(t, info.Mode()&os.ModeSymlink, "the symlink must not be replaced by a regular file")
content, err := os.ReadFile(target)
require.NoError(t, err)
assert.Contains(t, string(content), "capacity: 4")
})
t.Run("keeps CRLF line endings", func(t *testing.T) {
file := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(file, []byte(strings.ReplaceAll(editFixture, "\n", "\r\n")), 0o600))
require.NoError(t, SetValue(file, "runner.capacity", "4"))
content, err := os.ReadFile(file)
require.NoError(t, err)
assert.Contains(t, string(content), "capacity: 4\r\n")
assert.NotContains(t, strings.ReplaceAll(string(content), "\r\n", ""), "\n")
})
}
// An edit has to give the file back unchanged around it, down to the indentation of
// a commented-out option, as that documentation is what the user reads and edits.
func TestEditValuesPreservesFileText(t *testing.T) {
tests := []struct {
name string
content []byte
edit func(file string) error
added string // the only text the edit may add
}{
{
name: "example config",
content: Example,
edit: func(file string) error { return AddValue(file, "runner.labels", "ubuntu:docker://node:22") },
added: " labels:\n - ubuntu:docker://node:22\n",
},
{
name: "minimal config",
content: []byte(Minimal),
edit: func(file string) error { return SetValue(file, "runner.capacity", "4") },
added: "runner:\n capacity: 4\n",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
file := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(file, tc.content, 0o600))
require.NoError(t, tc.edit(file))
content, err := os.ReadFile(file)
require.NoError(t, err)
assert.Equal(t, string(tc.content), strings.Replace(string(content), tc.added, "", 1))
})
}
}

View File

@@ -0,0 +1,25 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build !windows && !plan9
package config
import (
"errors"
"os"
"syscall"
)
// preserveOwner keeps a config that root edits owned by the service user it was created for.
func preserveOwner(file string, info os.FileInfo) error {
stat, ok := info.Sys().(*syscall.Stat_t)
if !ok {
return nil
}
// A caller that may replace the file but not chown it is no worse off than before.
if err := os.Chown(file, int(stat.Uid), int(stat.Gid)); err != nil && !errors.Is(err, os.ErrPermission) {
return err
}
return nil
}

View File

@@ -255,6 +255,9 @@ func (r *Reporter) Fire(entry *log.Entry) error {
if step.StartedAt == nil {
step.StartedAt = timestamppb.New(timestamp)
urgentState = true
// The runner's own handler is per step, so an unresumed ::stop-commands:: must not
// leave the reporter suppressed, and no longer registering masks, for the whole job.
r.stopCommandEndToken = ""
}
// Force reporting log errors as raw output to prevent silent failures
@@ -396,10 +399,9 @@ func (r *Reporter) Logf(format string, a ...any) {
func (r *Reporter) logf(format string, a ...any) {
if !r.duringSteps() {
r.logRows = append(r.logRows, &runnerv1.LogRow{
Time: timestamppb.Now(),
Content: fmt.Sprintf(format, a...),
})
// Masked like any other row: these bypass parseLogRow, but a caller can still
// interpolate a secret, such as a configured URL carrying credentials.
r.logRows = append(r.logRows, r.newLogRow(timestamppb.Now(), fmt.Sprintf(format, a...)))
}
}
@@ -700,66 +702,128 @@ func (r *Reporter) parseResult(result any) (runnerv1.Result, bool) {
return ret, ok
}
var cmdRegex = regexp.MustCompile(`^::([^ :]+)( .*)?::(.*)$`)
// A property value never contains a raw ':' (GitHub escapes it as %3A), so excluding ':' ends
// the property list at the first '::' as GitHub does; greedily would swallow a '::' message.
var cmdRegex = regexp.MustCompile(`^::([^ :]+)( [^:]*)?::(.*)$`)
func (r *Reporter) handleCommand(originalContent, command, value string) *string {
if r.stopCommandEndToken != "" && command != r.stopCommandEndToken {
return &originalContent
// handleCommand takes value still escaped, so that the web UI decodes it exactly once. Only
// the branches that consume the payload here decode it.
func (r *Reporter) handleCommand(originalContent, command, properties, value string) *string {
if r.stopCommandEndToken != "" {
if command != r.stopCommandEndToken {
return &originalContent
}
// Resumed here rather than from the switch, because the end token is arbitrary and a
// token naming a real command would otherwise never resume.
r.stopCommandEndToken = ""
return nil
}
switch command {
case "add-mask":
r.addMask(value)
r.addMask(runner.UnescapeCommandData(value))
return nil
case "debug":
if r.debugOutputEnabled {
return &value
return &originalContent // kept as ::debug::, so the web UI labels and decodes it
}
return nil
case "notice":
// Not implemented yet, so just return the original content.
return &originalContent
case "warning":
// Not implemented yet, so just return the original content.
return &originalContent
case "error":
// Not implemented yet, so just return the original content.
return &originalContent
case "group":
// Returning the original content, because I think the frontend
// will use it when rendering the output.
return &originalContent
case "endgroup":
// Ditto
case "notice", "warning", "error":
// Gitea has no annotation store, so the annotation is rendered into the log with
// its source location instead of being dropped: that location is the whole point
// of the command for compiler and linter output.
annotation := formatAnnotation(command, properties, value)
return &annotation
case "group", "endgroup":
// Passed through: the web UI folds the log on these and decodes the payload itself.
return &originalContent
case "stop-commands":
r.stopCommandEndToken = value
return nil
case r.stopCommandEndToken:
r.stopCommandEndToken = ""
r.stopCommandEndToken = runner.UnescapeCommandData(value)
return nil
}
return &originalContent
}
// formatAnnotation folds the file, line, column and title the command carries into its message,
// which the web UI otherwise drops along with the rest of the properties:
//
// ::error file=main.go,line=12,col=5,title=vet::undefined: x
// ::error::main.go:12:5: vet: undefined: x
//
// The ::-form prefix is deliberate, and value is not escaped here because it arrived escaped
// and must stay that way.
func formatAnnotation(level, properties, value string) string {
props := parseCommandProperties(properties)
prefix := props["file"]
if prefix != "" {
if props["line"] != "" {
prefix += ":" + props["line"]
if props["col"] != "" {
prefix += ":" + props["col"]
}
}
prefix += ": "
}
if props["title"] != "" {
prefix += props["title"] + ": "
}
return "::" + level + "::" + prefix + value
}
// parseCommandProperties parses the `file=main.go,line=12` part of a workflow command.
func parseCommandProperties(properties string) map[string]string {
properties = strings.TrimSpace(properties)
if properties == "" {
return nil
}
props := map[string]string{}
for pair := range strings.SplitSeq(properties, ",") {
key, value, ok := strings.Cut(pair, "=")
if !ok {
continue
}
// Only the property-list separators are decoded, the web UI decodes the rest.
value = strings.ReplaceAll(strings.ReplaceAll(value, "%3A", ":"), "%2C", ",")
// GitHub keys its property dictionary case-insensitively, so `File=` works there too.
props[strings.ToLower(strings.TrimSpace(key))] = value
}
// GitHub's toolkit emits `col`; accept `column` as well, which some tools write instead.
if props["col"] == "" {
props["col"] = props["column"]
}
return props
}
func (r *Reporter) parseLogRow(entry *log.Entry) *runnerv1.LogRow {
content := strings.TrimRight(entry.Message, "\r\n")
// cmdRegex only covers the ::cmd:: form, so the ##[add-mask] one would otherwise reach
// the log carrying its own secret. Registered and dropped like its ::add-mask:: twin.
if arg, ok := strings.CutPrefix(content, "##[add-mask]"); ok {
r.addMask(runner.UnescapeCommandData(arg))
return nil
}
matches := cmdRegex.FindStringSubmatch(content)
if matches != nil {
if output := r.handleCommand(content, matches[1], runner.UnescapeCommandData(matches[3])); output != nil {
if output := r.handleCommand(content, matches[1], matches[2], matches[3]); output != nil {
content = *output
} else {
return nil
}
}
content = r.logReplacer.Replace(content)
return r.newLogRow(timestamppb.New(entry.Time), content)
}
// newLogRow applies the masking and validation every row must carry, whatever built it.
func (r *Reporter) newLogRow(t *timestamppb.Timestamp, content string) *runnerv1.LogRow {
return &runnerv1.LogRow{
Time: timestamppb.New(entry.Time),
Content: strings.ToValidUTF8(content, "?"),
Time: t,
Content: strings.ToValidUTF8(r.logReplacer.Replace(content), "?"),
}
}

View File

@@ -72,9 +72,12 @@ func TestReporter_parseLogRow(t *testing.T) {
"Debug enabled", true,
[]string{
"::debug::GitHub Actions runtime token access controls",
// Left escaped: the web UI decodes it, and a real newline would not survive storage.
"::debug::first%0Asecond",
},
[]string{
"GitHub Actions runtime token access controls",
"::debug::GitHub Actions runtime token access controls",
"::debug::first%0Asecond",
},
},
{
@@ -86,31 +89,46 @@ func TestReporter_parseLogRow(t *testing.T) {
"<nil>",
},
},
// The three annotation levels share one code path, so the property shapes are only
// exercised under "error"; notice and warning just prove the level token round-trips.
{
"notice", false,
[]string{
"::notice file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::notice::Gosh, that's not going to work",
},
[]string{
"::notice file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::notice::Gosh, that's not going to work",
},
},
{
"warning", false,
[]string{
"::warning file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::warning::Gosh, that's not going to work",
},
[]string{
"::warning file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::warning::Gosh, that's not going to work",
},
},
{
"error", false,
[]string{
"::error file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::error::Gosh, that's not going to work",
"::error file=file.name,line=42,col=7::Gosh, that's not going to work",
// The message keeps its own '::', the property list ends at the first one.
"::error file=main.cpp,line=12::no member named 'foo' in 'std::vector<int>'",
// GitHub matches property names case-insensitively.
"::error File=file.name,Line=42,Col=7::Gosh, that's not going to work",
// Only the property separators are decoded here, %25/%0A are left for the web UI.
"::error file=a%3Ab.go,title=100%252C::still %25 escaped%0Aand multi-line",
},
[]string{
"::error file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::error::file.name:42: Cool Title: Gosh, that's not going to work",
"::error::Gosh, that's not going to work",
"::error::file.name:42:7: Gosh, that's not going to work",
"::error::main.cpp:12: no member named 'foo' in 'std::vector<int>'",
"::error::file.name:42:7: Gosh, that's not going to work",
"::error::a:b.go: 100%252C: still %25 escaped%0Aand multi-line",
},
},
{
@@ -149,6 +167,24 @@ func TestReporter_parseLogRow(t *testing.T) {
"*** bar baz ***",
},
},
{
// a token naming a real command must still resume
"stop-commands with a command-named token", false,
[]string{
"::stop-commands::add-mask",
"::set-output name=x::suppressed",
"::add-mask::",
"::add-mask::masked",
"masked",
},
[]string{
"<nil>",
"::set-output name=x::suppressed",
"<nil>",
"<nil>",
"***",
},
},
{
"unknown command", false,
[]string{
@@ -179,6 +215,19 @@ func TestReporter_parseLogRow(t *testing.T) {
}
}
// Both add-mask forms must register the secret and drop their own row: the runner forwards
// the raw line, so failing to consume it writes the secret straight to the job log.
func TestReporter_parseLogRowAddMask(t *testing.T) {
for _, line := range []string{"::add-mask::supersecret", "##[add-mask]supersecret"} {
r := &Reporter{logReplacer: strings.NewReplacer()}
assert.Nil(t, r.parseLogRow(&log.Entry{Message: line}), line)
row := r.parseLogRow(&log.Entry{Message: "using supersecret now"})
assert.Equal(t, "using *** now", row.Content, line)
}
}
func TestReporter_Fire(t *testing.T) {
t.Run("ignore command lines", func(t *testing.T) {
client := mocks.NewClient(t)
@@ -1013,7 +1062,7 @@ func TestReporter_Result(t *testing.T) {
}
func TestReporter_SetOutputs(t *testing.T) {
r := &Reporter{state: &runnerv1.TaskState{}}
r := &Reporter{state: &runnerv1.TaskState{}, logReplacer: strings.NewReplacer()}
r.SetOutputs(map[string]string{"foo": "bar"})
got, ok := r.outputs["foo"]