Compare commits

3 Commits
v3.0.2 ... main

Author SHA1 Message Date
bircni
68547886a5 feat: wait for healthy services and fill the job context (#1107)
Service containers were started and then left alone, so a job's first step could run while a database was still starting up. The runner now waits for every service whose image or `options` declare a healthcheck, as GitHub does. An unhealthy service fails the job with its container log, one that never becomes healthy fails it after `container.service_ready_timeout` (default `5m`, negative disables the wait), and one that exits without a healthcheck only gets its log and a warning.

The started containers also fill the `job` context, whose fields existed but were never populated: `job.container.{id,network}` and `job.services.<id>.{id,network,ports}`. `ports` is keyed by the plain container port, so `job.services.postgres.ports['5432']` resolves to the host port Docker picked.

---------

Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1107
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-08-05 19:46:43 +00:00
bircni
8700adc933 feat: config command to edit config files (#1140)
Changing a setting after `generate-config` meant hand-editing YAML, which is awkward in provisioning scripts. `config` now edits an existing file in place:

```bash
./gitea-runner -c config.yaml config set runner.capacity 4
./gitea-runner -c config.yaml config set runner.envs.MY_VAR value
./gitea-runner -c config.yaml config add runner.labels 'ubuntu:docker://node:22'
./gitea-runner -c config.yaml config remove runner.labels 'ubuntu:docker://node:22'
./gitea-runner -c config.yaml config get runner.labels
```

Edits go through the YAML node tree, so comments, key order and the blank lines between top-level sections survive — a test asserts that appending a label to `config.example.yaml` changes nothing but the added line. Keys are resolved by reflecting over the `Config` struct's yaml tags, so an unknown key or a value of the wrong type is rejected before the file is touched. The write is atomic and keeps the file's symlink, owner, mode and line endings.

---------

Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1140
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-08-05 16:48:25 +00:00
bircni
b70ff6893a feat: gate set-env/add-path and render annotation locations (#1109)
`::set-env::` and `::add-path::` let a step rewrite the environment of every later step from its own output, which the runner honoured silently. They are now refused, as GitHub has done since 2020, and `ACTIONS_ALLOW_UNSECURE_COMMANDS` opts back in per step or job. Support for that variable is new here too, and is the only opt-in, matching GitHub rather than adding a runner config key on top.

Annotations keep their source location: Gitea has no annotation store and its web UI strips command properties, so `::error file=main.go,line=12::msg` is rendered as `::error::main.go:12: msg`.

`DEVELOPMENT.md` writes down the log line encoding rules this relies on.

---------

Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1109
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-08-05 16:43:17 +00:00
39 changed files with 2155 additions and 171 deletions

View File

@@ -8,6 +8,7 @@
- Never force-push, amend, or squash unless asked. Use new commits and normal push for pull request updates
- Comments: write almost none, short and preferably same-line, explaining why for a future reader. Never narrate code, the change or the prompt. Preserve existing ones that still apply
- Add the current year into the copyright header of new `.go` files
- Read `DEVELOPMENT.md` for internals and conventions
- Ensure no trailing whitespace in edited files
- Run `make fmt` after `.go` edits, `make tidy` after `go.mod` edits, and `make checks` for the non-lint source checks
- Lint what changed with `make lint-go`, and `make lint-go-windows` for Windows and platform-split files

32
DEVELOPMENT.md Normal file
View File

@@ -0,0 +1,32 @@
# Development
## Job log line format
Gitea stores one log row per line and its web UI decodes the payload, so getting the encoding
wrong never fails a test here, it only shows up in the browser.
**A row cannot contain a real newline.** `FormatLog` rewrites `\n` to a literal backslash-n and
truncates at 64 KiB on a byte boundary.
**The payload of a line starting with a recognised prefix is decoded**, with the escape set
depending on the prefix:
| prefix | decodes |
| --- | --- |
| `##[error]` `##[warning]` `##[notice]` `##[debug]` `##[group]` `##[endgroup]` `##[add-matcher]` | `%25` `%0D` `%0A` `%3B` `%5D` |
| `::error::` `::warning::` `::notice::` `::debug::` (with or without ` key=value` properties), `::group::` `::endgroup::` `::add-matcher::` | `%25` `%0D` `%0A` |
| `##[command]` `[command]`, or no recognised prefix | nothing |
### Rules
- **Emitting a command line?** Escape the payload with `runner.EscapeCommandData`. One escaper
covers both forms: it escapes `%` first, so a literal `%3B` becomes `%253B` that the extra
`##[…]` rules cannot match, and a raw `;` or `]` is never decoded. It is also what makes
multi-line work, `\n` becomes `%0A` and the UI turns it back into a line break.
- **Forwarding a command from step output?** Leave the payload alone, it arrived escaped and is
decoded once. Decoding here double-decodes and destroys multi-line.
- **No prefix?** Do not escape, and split multi-line values into one row each.
- **Interpolating a secret?** Masking runs after escaping, so `AppendSecretMasker` registers the
encoded forms too.
- Command *properties* also escape `%3A` and `%2C`, which the UI never decodes, so the reporter
decodes exactly those two when folding a location into an annotation.

View File

@@ -132,9 +132,11 @@ Same idea as `dind`, but built on `docker:dind-rootless` so the bundled daemon a
The runner is configured with a YAML file. Generate a starting point (this matches what ships in the tree):
```bash
./gitea-runner generate-config > config.yaml
./gitea-runner config generate > config.yaml
```
> The top-level `generate-config` command still does the same thing, but is deprecated in favour of `config generate`.
Pass it with `-c` / `--config` on any command that loads configuration (`register`, `daemon`, `cache-server`):
```bash
@@ -143,7 +145,26 @@ Pass it with `-c` / `--config` on any command that loads configuration (`registe
./gitea-runner -c config.yaml cache-server
```
Every option is described in [config.example.yaml](internal/pkg/config/config.example.yaml) (the same content `generate-config` prints).
Every option is described in [config.example.yaml](internal/pkg/config/config.example.yaml) (the same content `config generate` prints).
#### Editing a config file
`config` changes an existing file in place, keeping its comments and key order, which is handy in provisioning scripts:
```bash
./gitea-runner -c config.yaml config set runner.capacity 4
./gitea-runner -c config.yaml config set runner.timeout 90m # written as 1h30m0s
./gitea-runner -c config.yaml config set runner.envs.MY_VAR value
./gitea-runner -c config.yaml config add runner.labels 'ubuntu:docker://node:22'
./gitea-runner -c config.yaml config remove runner.labels 'ubuntu:docker://node:22'
./gitea-runner -c config.yaml config get runner.labels
```
`-c` is optional for these subcommands: without it they use `config.yaml` (or `config.yml`) from the working directory, falling back to the directory of the `gitea-runner` binary, and print which file they picked to stderr.
Keys are the dotted YAML path and are validated against the known options, so a typo is rejected instead of being written. `add` and `remove` only work on list options such as `runner.labels` and `container.valid_volumes`, and fail if the value is already present or missing. `set` replaces the whole list when given several values.
The file is re-encoded on every edit, so indentation is normalised to two spaces and blank lines inside a section are dropped.
#### Without a config file
@@ -209,6 +230,26 @@ Whenever the resulting labels differ from the ones in the registration file, the
> **Note:** A runner that only exposes `host` labels still needs access to a Docker daemon (e.g. a mounted `/var/run/docker.sock`) whenever a job uses a `docker://` action or a service container. `host` labels only change where the job's own steps run; container-based steps and actions are still executed with Docker.
#### Service containers
A job's `services` are started before its steps run. When a service's image or its `options` declare a healthcheck, the runner waits for it to report healthy, so a workflow does not have to poll for its own services:
```yaml
services:
postgres:
image: postgres:17
options: >-
--health-cmd pg_isready
--health-interval 5s
--health-retries 10
```
A service that reports unhealthy fails the job right away, with its container log. One that never becomes healthy fails it after `container.service_ready_timeout` (default `5m`, negative disables the wait). A service that exits without declaring a healthcheck only gets its log and a warning.
A job in a container reaches a service by its id on the job network, on the port the service listens on, for example `psql -h postgres -p 5432`. The started containers also fill the `job` context: `job.container.{id,network}` and `job.services.<id>.{id,network,ports}`, where `ports` maps a container port to the host port Docker published it on, for the services that publish one.
Unlike GitHub, a job whose steps run on the host (a `host` label without `container:`) starts no service containers, so `job.services` and `job.container` stay empty. Give such a job a `container:` when it needs services.
#### Proxy
Set these variables in the runner's environment, with systemd `Environment=`, `docker run -e`, or Kubernetes `env:`:

View File

@@ -6,12 +6,14 @@ package container
import (
"context"
"errors"
"fmt"
"io"
"gitea.com/gitea/runner/act/common"
"github.com/docker/go-connections/nat"
"github.com/moby/moby/api/types/container"
)
// ExitCodeError reports a non-zero process exit code from a container command.
@@ -57,6 +59,32 @@ type FileEntry struct {
Body string
}
// Container and healthcheck states, as plain strings so a caller of Info needs no docker
// SDK of its own.
const (
StateRunning = string(container.StateRunning)
HealthNone = string(container.NoHealthcheck)
HealthStarting = string(container.Starting)
HealthHealthy = string(container.Healthy)
HealthUnhealthy = string(container.Unhealthy)
)
// ErrContainerNotFound reports a container the daemon no longer knows. Its text is a
// fragment, missingContainerError composes it into the message every operation shares.
var ErrContainerNotFound = errors.New("does not exist")
// Info is a snapshot of a container, as of one inspect.
type Info struct {
ID string
State string // the docker container state: "created", "running", "exited", ...
ExitCode int
Health string // one of the Health* constants
// HealthOutput is the last healthcheck probe's output.
HealthOutput string
Ports map[string]string // container port ("5432") to the host port it is published on
}
// Container for managing docker run containers
type Container interface {
Create(capAdd, capDrop []string) common.Executor
@@ -65,6 +93,8 @@ type Container interface {
CopyTarStream(ctx context.Context, destPath string, tarStream io.Reader) error
CopyDir(destPath, srcPath string, useGitIgnore bool) common.Executor
GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error)
Inspect(ctx context.Context) (*Info, error)
DumpLogs(ctx context.Context) error
Pull(forcePull bool) common.Executor
Start(attach bool) common.Executor
Exec(command []string, env map[string]string, user, workdir string) common.Executor

View File

@@ -198,6 +198,109 @@ func (cr *containerReference) GetContainerArchive(ctx context.Context, srcPath s
return result.Content, nil
}
// Inspect resolves the container by name when its id is not known yet. One the daemon no
// longer knows is reported as ErrContainerNotFound.
func (cr *containerReference) Inspect(ctx context.Context) (*Info, error) {
if common.Dryrun(ctx) {
return &Info{Health: HealthNone, Ports: map[string]string{}}, nil
}
if err := cr.connect()(ctx); err != nil {
return nil, err
}
if cr.id == "" { // a known id is trusted, find() would spend a call validating it
if err := cr.find()(ctx); err != nil {
return nil, err
}
}
if cr.id == "" {
return nil, cr.missingContainerError("inspect it")
}
result, err := cr.cli.ContainerInspect(ctx, cr.id, client.ContainerInspectOptions{})
if cerrdefs.IsNotFound(err) {
return nil, cr.missingContainerError("inspect it")
} else if err != nil {
return nil, err
}
return containerInfoFromInspect(result.Container), nil
}
// DumpLogs copies the container's log so far to its output writers.
func (cr *containerReference) DumpLogs(ctx context.Context) error {
if common.Dryrun(ctx) {
return nil
}
if err := cr.connect()(ctx); err != nil {
return err
}
if cr.id == "" {
return cr.missingContainerError("read its logs")
}
logs, err := cr.cli.ContainerLogs(ctx, cr.id, client.ContainerLogsOptions{ShowStdout: true, ShowStderr: true})
if err != nil {
return err
}
defer logs.Close()
return cr.copyOutput(logs)
}
// copyOutput writes a container stream to the writers the container was created with,
// demultiplexing it unless the container has a TTY, which sends a single raw stream.
func (cr *containerReference) copyOutput(stream io.Reader) error {
outWriter := cr.input.Stdout
if outWriter == nil {
outWriter = os.Stdout
}
errWriter := cr.input.Stderr
if errWriter == nil {
errWriter = os.Stderr
}
var err error
if !cr.input.AllocatePTY || os.Getenv("NORAW") != "" {
_, err = stdcopy.StdCopy(outWriter, errWriter, stream)
} else {
_, err = io.Copy(outWriter, stream)
}
// Flush any buffered, not-yet-newline-terminated trailing line so the final line of
// the output is not lost when it is not newline-terminated.
common.FlushWriter(outWriter)
common.FlushWriter(errWriter)
return err
}
func containerInfoFromInspect(inspect container.InspectResponse) *Info {
info := &Info{
ID: inspect.ID,
Health: HealthNone,
Ports: map[string]string{}, // an empty map, never null, in the expression context
}
if state := inspect.State; state != nil {
info.State = string(state.Status)
info.ExitCode = state.ExitCode
if health := state.Health; health != nil {
info.Health = string(health.Status)
if len(health.Log) > 0 {
info.HealthOutput = strings.TrimSpace(health.Log[len(health.Log)-1].Output)
}
}
}
if settings := inspect.NetworkSettings; settings != nil {
for port, bindings := range settings.Ports {
for _, binding := range bindings { // the last binding wins, a port maps to one host port
if binding.HostPort != "" {
info.Ports[port.Port()] = binding.HostPort
}
}
}
}
return info
}
func (cr *containerReference) UpdateFromEnv(srcPath string, env *map[string]string) common.Executor {
return parseEnvFile(cr, srcPath, env).IfNot(common.Dryrun)
}
@@ -343,10 +446,10 @@ func (cr *containerReference) Close() common.Executor {
}
}
// missingContainerError is the shared "container X does not exist" error
// used by ops that need a live cr.id.
// missingContainerError is the shared "container X does not exist" error used by ops that
// need a live cr.id, wrapping ErrContainerNotFound so a caller can tell it from a failing daemon.
func (cr *containerReference) missingContainerError(format string, args ...any) error {
return fmt.Errorf("container %q does not exist; cannot "+format, append([]any{cr.input.Name}, args...)...)
return fmt.Errorf("container %q %w; cannot "+format, append([]any{cr.input.Name, ErrContainerNotFound}, args...)...)
}
func (cr *containerReference) find() common.Executor {
@@ -737,7 +840,7 @@ func (cr *containerReference) exec(cmd []string, env map[string]string, user, wo
}
defer resp.Close()
err = cr.waitForCommand(ctx, isTerminal, resp.HijackedResponse, idResp, user, workdir)
err = cr.waitForCommand(ctx, resp.HijackedResponse, idResp, user, workdir)
if err != nil {
return err
}
@@ -795,7 +898,7 @@ func (cr *containerReference) tryReadGID() common.Executor {
return cr.tryReadID("-g", func(id int) { cr.GID = id })
}
func (cr *containerReference) waitForCommand(ctx context.Context, isTerminal bool, resp client.HijackedResponse, _ client.ExecCreateResult, _, _ string) error {
func (cr *containerReference) waitForCommand(ctx context.Context, resp client.HijackedResponse, _ client.ExecCreateResult, _, _ string) error {
logger := common.Logger(ctx)
// Buffered so the copy goroutine never blocks on send if the grace-period
@@ -803,28 +906,7 @@ func (cr *containerReference) waitForCommand(ctx context.Context, isTerminal boo
cmdResponse := make(chan error, 1)
go func() {
var outWriter io.Writer
outWriter = cr.input.Stdout
if outWriter == nil {
outWriter = os.Stdout
}
errWriter := cr.input.Stderr
if errWriter == nil {
errWriter = os.Stderr
}
var err error
if !isTerminal || os.Getenv("NORAW") != "" {
_, err = stdcopy.StdCopy(outWriter, errWriter, resp.Reader)
} else {
_, err = io.Copy(outWriter, resp.Reader)
}
// Flush any buffered, not-yet-newline-terminated trailing line so the
// final line of a command's output is not lost (e.g. an error message
// printed without a trailing newline before the process exits).
common.FlushWriter(outWriter)
common.FlushWriter(errWriter)
cmdResponse <- err
cmdResponse <- cr.copyOutput(resp.Reader)
}()
select {
@@ -1059,33 +1141,11 @@ func (cr *containerReference) attach() common.Executor {
if err != nil {
return fmt.Errorf("failed to attach to container: %w", err)
}
isTerminal := cr.input.AllocatePTY
var outWriter io.Writer
outWriter = cr.input.Stdout
if outWriter == nil {
outWriter = os.Stdout
}
errWriter := cr.input.Stderr
if errWriter == nil {
errWriter = os.Stderr
}
done := make(chan struct{})
cr.attachDone = done
go func() {
defer close(done)
var copyErr error
if !isTerminal || os.Getenv("NORAW") != "" {
_, copyErr = stdcopy.StdCopy(outWriter, errWriter, out.Reader)
} else {
_, copyErr = io.Copy(outWriter, out.Reader)
}
// Flush any buffered, not-yet-newline-terminated trailing line once
// the stream reaches EOF, so the final line of the container's
// output is not lost when it is not newline-terminated.
common.FlushWriter(outWriter)
common.FlushWriter(errWriter)
if copyErr != nil {
if copyErr := cr.copyOutput(out.Reader); copyErr != nil {
common.Logger(ctx).Error(copyErr)
}
}()

View File

@@ -25,6 +25,7 @@ import (
"github.com/moby/moby/api/pkg/stdcopy"
"github.com/moby/moby/api/types/container"
"github.com/moby/moby/api/types/mount"
"github.com/moby/moby/api/types/network"
mobyclient "github.com/moby/moby/client"
"github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
@@ -548,7 +549,7 @@ func TestRejectsMissingContainer(t *testing.T) {
cr := &containerReference{cli: client, input: &NewContainerInput{Name: "job-1"}}
check := func(op string, err error) {
t.Helper()
require.Error(t, err, op)
require.ErrorIs(t, err, ErrContainerNotFound, op)
assert.Contains(t, err.Error(), `container "job-1" does not exist`, op)
}
check("copyContent", cr.copyContent("/var/run/act", &FileEntry{Name: "x", Mode: 0o644})(ctx))
@@ -557,6 +558,15 @@ func TestRejectsMissingContainer(t *testing.T) {
check("exec", cr.exec([]string{"echo"}, nil, "", "")(ctx))
_, err := cr.GetContainerArchive(ctx, "/var/run/act/x")
check("GetContainerArchive", err)
_, err = cr.Inspect(ctx)
check("Inspect", err)
// a known id the daemon has since dropped
client.On("ContainerInspect", ctx, "gone", mobyclient.ContainerInspectOptions{}).
Return(mobyclient.ContainerInspectResult{}, cerrdefs.ErrNotFound)
removed := &containerReference{id: "gone", cli: client, input: &NewContainerInput{Name: "job-1"}}
_, err = removed.Inspect(ctx)
check("Inspect after removal", err)
}
// End-to-end: a stale cr.id is cleared, repopulated from name lookup,
@@ -825,6 +835,59 @@ func TestCheckVolumesRejectsEscapingHostPaths(t *testing.T) {
assert.Empty(t, hostConf.Binds)
}
func TestContainerInfoFromInspect(t *testing.T) {
t.Run("reports no healthcheck when the image declares none", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{
ID: "abc123",
State: &container.State{Status: "running", Running: true},
})
assert.Equal(t, "abc123", info.ID)
assert.Equal(t, "running", info.State)
assert.Equal(t, HealthNone, info.Health)
assert.Empty(t, info.Ports)
})
t.Run("reports the health status and the last probe output", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{
State: &container.State{
Status: "running",
Health: &container.Health{
Status: container.Unhealthy,
Log: []*container.HealthcheckResult{
{Output: "first\n"},
{Output: "connection refused\n"},
},
},
},
})
assert.Equal(t, HealthUnhealthy, info.Health)
assert.Equal(t, "connection refused", info.HealthOutput)
})
t.Run("reports the published ports", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{
State: &container.State{Status: "running"},
NetworkSettings: &container.NetworkSettings{
Ports: network.PortMap{
network.MustParsePort("5432/tcp"): []network.PortBinding{{HostPort: "49153"}},
network.MustParsePort("6379/tcp"): nil,
},
},
})
assert.Equal(t, map[string]string{"5432": "49153"}, info.Ports)
})
t.Run("tolerates a container without state", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{ID: "abc123"})
assert.Equal(t, "abc123", info.ID)
assert.Equal(t, HealthNone, info.Health)
})
}
func TestMergeContainerConfigsVolumesReplaceRunnerMounts(t *testing.T) {
logger, _ := test.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger)

View File

@@ -154,6 +154,14 @@ func (e *HostEnvironment) CopyDir(destPath, srcPath string, useGitIgnore bool) c
}
}
func (e *HostEnvironment) DumpLogs(_ context.Context) error {
return nil
}
func (e *HostEnvironment) Inspect(_ context.Context) (*Info, error) {
return &Info{Health: HealthNone, Ports: map[string]string{}}, nil
}
func (e *HostEnvironment) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) {
buf := &bytes.Buffer{}
tw := tar.NewWriter(buf)

View File

@@ -5,12 +5,18 @@
package model
type JobContext struct {
Status string `json:"status"`
Container struct {
ID string `json:"id"`
Network string `json:"network"`
} `json:"container"`
Services map[string]struct {
ID string `json:"id"`
} `json:"services"`
Status string `json:"status"`
Container JobContainerContext `json:"container"`
Services map[string]JobService `json:"services"`
}
type JobContainerContext struct {
ID string `json:"id"`
Network string `json:"network"`
}
type JobService struct {
ID string `json:"id"`
Network string `json:"network"`
Ports map[string]string `json:"ports"` // container port to the published host port
}

View File

@@ -186,10 +186,10 @@ func (rc *RunContext) compositeExecutor(action *model.Action) *compositeSteps {
err := rc.newCompositeCommandExecutor(step.main())(ctx)
if err != nil {
logger.Errorf("%v", err)
logger.Errorf("##[error]%s", EscapeCommandData(err.Error()))
common.SetJobError(ctx, err)
} else if ctx.Err() != nil {
logger.Errorf("%v", ctx.Err())
logger.Errorf("##[error]%s", EscapeCommandData(ctx.Err().Error()))
common.SetJobError(ctx, ctx.Err())
}
return nil
@@ -248,10 +248,10 @@ func newCompositeStepLogExecutor(runStep common.Executor, stepID string) common.
logger := common.Logger(ctx)
err := runStep(ctx)
if err != nil {
logger.Errorf("%v", err)
logger.Errorf("##[error]%s", EscapeCommandData(err.Error()))
common.SetJobError(ctx, err)
} else if ctx.Err() != nil {
logger.Errorf("%v", ctx.Err())
logger.Errorf("##[error]%s", EscapeCommandData(ctx.Err().Error()))
common.SetJobError(ctx, ctx.Err())
}
return nil

View File

@@ -6,6 +6,7 @@ package runner
import (
"context"
"fmt"
"regexp"
"strings"
@@ -45,17 +46,24 @@ func (rc *RunContext) commandHandler(ctx context.Context) common.LineHandler {
return true
}
if resumeCommand != "" && command != resumeCommand {
if resumeCommand != "" {
// There should not be any emojis in the log output for Gitea.
// The code in the switch statement is the same.
// Return true (not false) so the line still reaches the raw_output
// log handler; otherwise everything between ::stop-commands:: and
// its end token is silently dropped from the step log.
logger.Infof("%s", line)
// Resumed here rather than from the switch, because the end token is arbitrary
// and a token naming a real command would otherwise never resume.
if command == resumeCommand {
resumeCommand = ""
}
return true
}
arg = UnescapeCommandData(arg)
kvPairs = unescapeKvPairs(kvPairs)
if (command == "set-env" || command == "add-path") && rc.refuseUnsecureCommand(ctx, command) {
return true
}
switch command {
case "set-env":
rc.setEnv(ctx, kvPairs, arg)
@@ -63,27 +71,20 @@ func (rc *RunContext) commandHandler(ctx context.Context) common.LineHandler {
rc.setOutput(ctx, kvPairs, arg)
case "add-path":
rc.addPath(ctx, arg)
case "debug":
logger.Infof("%s", line)
case "warning":
logger.Infof("%s", line)
case "error":
logger.Infof("%s", line)
case "add-mask":
rc.AddMask(arg)
logger.Infof("%s", "***")
// The raw line is still forwarded, carrying the secret: that is how the reporter
// learns the mask, and it drops the row rather than writing it out.
case "stop-commands":
resumeCommand = arg
logger.Infof("%s", line)
case resumeCommand:
resumeCommand = ""
logger.Infof("%s", line)
case "save-state":
logger.Infof("%s", line)
rc.saveState(ctx, kvPairs, arg)
case "add-matcher":
logger.Infof("%s", line)
default:
// ::debug::, ::error::, ::warning::, ::add-matcher:: and anything unrecognised are
// passed through for the reporter and Gitea's web UI to render.
logger.Infof("%s", line)
}
@@ -92,6 +93,52 @@ func (rc *RunContext) commandHandler(ctx context.Context) common.LineHandler {
}
}
const allowUnsecureCommandsVar = "ACTIONS_ALLOW_UNSECURE_COMMANDS"
// refuseUnsecureCommand reports whether a deprecated ::set-env:: or ::add-path:: command must
// not run, recording the error that fails the step. GitHub disabled both because a step that
// echoes untrusted content can use them to set NODE_OPTIONS or PATH for every later step.
func (rc *RunContext) refuseUnsecureCommand(ctx context.Context, command string) bool {
if rc.allowUnsecureCommandsOptIn() {
return false
}
// The step executor logs the failure itself, so keep this line's wording distinct.
common.Logger(ctx).WithField(rawOutputField, true).Errorf("##[error]%s", EscapeCommandData(fmt.Sprintf(
"The `%s` command is disabled: it can set the environment of every later step from untrusted output. "+
"Write to $GITHUB_ENV or $GITHUB_PATH instead, or set ACTIONS_ALLOW_UNSECURE_COMMANDS to allow it",
command)))
rc.unsecureCommandMu.Lock()
defer rc.unsecureCommandMu.Unlock()
if rc.unsecureCommandErr == nil {
rc.unsecureCommandErr = fmt.Errorf("the `%s` workflow command is disabled", command)
}
return true
}
// allowUnsecureCommandsOptIn reports whether the workflow itself asked for the deprecated
// commands, from any env scope, as it can on GitHub.
func (rc *RunContext) allowUnsecureCommandsOptIn() bool {
return isTruthyEnv(rc.currentStepEnv()[allowUnsecureCommandsVar]) ||
isTruthyEnv(rc.Env[allowUnsecureCommandsVar]) ||
isTruthyEnv(rc.GlobalEnv[allowUnsecureCommandsVar])
}
// isTruthyEnv mirrors GitHub's bool.TryParse: only "true", in any casing.
func isTruthyEnv(v string) bool {
return strings.EqualFold(strings.TrimSpace(v), "true")
}
// takeUnsecureCommandError returns and clears the error left by a refused command.
func (rc *RunContext) takeUnsecureCommandError() error {
rc.unsecureCommandMu.Lock()
defer rc.unsecureCommandMu.Unlock()
err := rc.unsecureCommandErr
rc.unsecureCommandErr = nil
return err
}
func (rc *RunContext) setEnv(ctx context.Context, kvPairs map[string]string, arg string) {
name := kvPairs["name"]
common.Logger(ctx).Infof("::set-env:: %s=%s", name, arg)
@@ -161,9 +208,9 @@ var (
commandPropertyUnescaper = strings.NewReplacer("%25", "%", "%0D", "\r", "%0A", "\n", "%3A", ":", "%2C", ",")
)
// escapeCommandData encodes the data part of a "::cmd::" or "##[cmd]" line the runner writes itself,
// EscapeCommandData encodes the data part of a "::cmd::" or "##[cmd]" line the runner writes itself,
// so the log renderer decodes it back. Lines forwarded from step output are already escaped.
func escapeCommandData(arg string) string {
func EscapeCommandData(arg string) string {
return commandDataEscaper.Replace(arg)
}

View File

@@ -16,12 +16,18 @@ import (
"github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// unsecureRC opts into ::set-env:: and ::add-path::, which are refused without it.
func unsecureRC() *RunContext {
return &RunContext{Env: map[string]string{allowUnsecureCommandsVar: "true"}}
}
func TestSetEnv(t *testing.T) {
a := assert.New(t)
ctx := context.Background()
rc := new(RunContext)
rc := unsecureRC()
handler := rc.commandHandler(ctx)
handler("::set-env name=x::valz\n")
@@ -31,7 +37,7 @@ func TestSetEnv(t *testing.T) {
func TestStopCommandsKeepsSuppressedLinesInLog(t *testing.T) {
a := assert.New(t)
ctx := context.Background()
rc := new(RunContext)
rc := unsecureRC()
handler := rc.commandHandler(ctx)
// Stop command processing until the matching end token is seen.
@@ -84,7 +90,7 @@ func TestSetOutput(t *testing.T) {
func TestAddpath(t *testing.T) {
a := assert.New(t)
ctx := context.Background()
rc := new(RunContext)
rc := unsecureRC()
handler := rc.commandHandler(ctx)
handler("::add-path::/zoo\n")
@@ -99,7 +105,7 @@ func TestStopCommands(t *testing.T) {
a := assert.New(t)
ctx := common.WithLogger(context.Background(), logger)
rc := new(RunContext)
rc := unsecureRC()
handler := rc.commandHandler(ctx)
handler("::set-env name=x::valz\n")
@@ -119,10 +125,26 @@ func TestStopCommands(t *testing.T) {
a.Contains(messages, "::set-env name=x::abcd\n")
}
// The end token is arbitrary, so one that happens to name a real command must still resume
// rather than being swallowed by that command's case.
func TestStopCommandsResumesOnCommandNamedToken(t *testing.T) {
a := assert.New(t)
rc := unsecureRC()
handler := rc.commandHandler(context.Background())
handler("::stop-commands::add-mask\n")
handler("::set-env name=x::suppressed\n")
a.NotContains(rc.Env, "x")
handler("::add-mask::\n")
handler("::set-env name=x::resumed\n")
a.Equal("resumed", rc.Env["x"])
}
func TestAddpathADO(t *testing.T) {
a := assert.New(t)
ctx := context.Background()
rc := new(RunContext)
rc := unsecureRC()
handler := rc.commandHandler(ctx)
handler("##[add-path]/zoo\n")
@@ -218,6 +240,44 @@ func TestSaveState(t *testing.T) {
func TestEscapeCommandData(t *testing.T) {
a := assert.New(t)
a.Equal("a%25b%0Dc%0Ad%250A", escapeCommandData("a%b\rc\nd%0A"))
a.Equal("a%25b%0Dc%0Ad%250A", EscapeCommandData("a%b\rc\nd%0A"))
a.Equal("a%b\rc\nd%0A", UnescapeCommandData("a%25b%0Dc%0Ad%250A"))
}
func TestUnsecureCommands(t *testing.T) {
tests := []struct {
name string
jobEnv map[string]string
stepEnv map[string]string
optedIn bool
}{
{name: "refused with no opt-in"},
// GitHub reads the opt-in with bool.TryParse, so "1" is not one.
{name: "refused for a value bool.TryParse rejects", jobEnv: map[string]string{allowUnsecureCommandsVar: "1"}},
{name: "opted in through the step environment", stepEnv: map[string]string{allowUnsecureCommandsVar: "true"}, optedIn: true},
{name: "opted in through the job environment", jobEnv: map[string]string{allowUnsecureCommandsVar: "TRUE"}, optedIn: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
a := assert.New(t)
rc := &RunContext{Env: tt.jobEnv}
rc.setCurrentStepEnv(tt.stepEnv)
handler := rc.commandHandler(context.Background())
handler("::set-env name=x::valz\n")
handler("::add-path::/opt/bin\n")
if !tt.optedIn {
a.Empty(rc.Env["x"])
a.Empty(rc.ExtraPath)
// The refusal fails the step that produced it, once.
require.ErrorContains(t, rc.takeUnsecureCommandError(), "set-env")
a.NoError(rc.takeUnsecureCommandError())
return
}
a.Equal("valz", rc.Env["x"])
a.Equal([]string{"/opt/bin"}, rc.ExtraPath)
a.NoError(rc.takeUnsecureCommandError())
})
}
}

View File

@@ -78,3 +78,14 @@ func (cm *containerMock) GetContainerArchive(ctx context.Context, srcPath string
}
return args.Get(0).(io.ReadCloser), err
}
func (cm *containerMock) DumpLogs(ctx context.Context) error {
return cm.Called(ctx).Error(0)
}
func (cm *containerMock) Inspect(ctx context.Context) (*container.Info, error) {
args := cm.Called(ctx)
info, _ := args.Get(0).(*container.Info)
err, _ := args.Get(1).(error)
return info, err
}

View File

@@ -66,7 +66,7 @@ func reportStepError(ctx context.Context, rc *RunContext, err error) {
rc.markInterrupted(ctx.Err())
return
}
common.Logger(ctx).Errorf("##[error]%s", escapeCommandData(err.Error()))
common.Logger(ctx).Errorf("##[error]%s", EscapeCommandData(err.Error()))
common.SetJobError(ctx, err)
rc.markFailed()
}
@@ -260,7 +260,7 @@ func newJobExecutor(info jobInfo, sf stepFactory, rc *RunContext) common.Executo
logger.Infof("Cleaning up container for job %s", rc.JobName)
if err = info.stopContainer()(ctx); err != nil {
logger.Errorf("Error while stop job container: %v", err)
logger.Errorf("##[error]%s", EscapeCommandData("Error while stop job container: "+err.Error()))
}
// For Gitea

View File

@@ -45,7 +45,7 @@ func (rc *RunContext) runJobHook(ctx context.Context, hookPath, name string) err
cmd, shell := hookCommand(hookPath)
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
defer rawLogger.Infof("::endgroup::")
rawLogger.Infof("::group::Run '%s'", escapeCommandData(hookPath))
rawLogger.Infof("::group::Run '%s'", EscapeCommandData(hookPath))
rawLogger.Infof("A %s hook has been configured by the runner administrator", name)
if shell != "" {
rawLogger.Infof("shell: %s", shell)

View File

@@ -250,7 +250,7 @@ func AppendSecretMasker(oldnew []string, v string) []string {
ret = append(ret, tm, "***")
// command data reaches the log escaped, so "pass%word" also arrives as "pass%25word"
if strings.ContainsAny(tm, "%\r\n") {
ret = append(ret, escapeCommandData(tm), "***")
ret = append(ret, EscapeCommandData(tm), "***")
}
}
}

View File

@@ -22,6 +22,7 @@ import (
"runtime"
"slices"
"strings"
"sync"
"time"
"gitea.com/gitea/runner/act/common"
@@ -34,6 +35,7 @@ import (
"github.com/docker/go-connections/nat"
"github.com/moby/moby/api/types/mount"
"github.com/opencontainers/selinux/go-selinux"
"golang.org/x/sync/errgroup"
)
// RunContext contains info about current job
@@ -55,7 +57,7 @@ type RunContext struct {
IntraActionState map[string]map[string]string
ExprEval ExpressionEvaluator
JobContainer container.ExecutionsEnvironment
ServiceContainers []container.ExecutionsEnvironment
serviceContainers []*serviceContainer
OutputMappings map[MappableOutput]MappableOutput
JobName string
ActionPath string
@@ -83,6 +85,37 @@ type RunContext struct {
// failures. Those failures must still make success() false and failure() true for later
// main-step if evaluation.
jobFailed bool
// empty for a host-mode job, which starts no container
jobContainerID string
jobNetworkName string
// stepEnv is a copy of the running step's environment, so that workflow commands parsed out
// of the container's output can be judged against it. Written by runStepExecutor and read on
// the log-writer goroutine, hence unsecureCommandMu, which also guards unsecureCommandErr.
stepEnv map[string]string
unsecureCommandErr error // refused ::set-env::/::add-path::, turned into a step failure
unsecureCommandMu sync.Mutex
}
// serviceContainer pairs a service container with the workflow id that keys job.services.
type serviceContainer struct {
name string
image string
container container.ExecutionsEnvironment
logsDumped bool
info *container.Info // last poll, the source of the `job.services` entry
}
// setCurrentStepEnv records the environment of the step about to run.
func (rc *RunContext) setCurrentStepEnv(env map[string]string) {
rc.unsecureCommandMu.Lock()
defer rc.unsecureCommandMu.Unlock()
rc.stepEnv = env
}
func (rc *RunContext) currentStepEnv() map[string]string {
rc.unsecureCommandMu.Lock()
defer rc.unsecureCommandMu.Unlock()
return rc.stepEnv
}
// markCancelled flags the job as cancelled so subsequent step `if` evaluations and the
@@ -488,7 +521,7 @@ func (rc *RunContext) startJobContainer() common.Executor {
Privileged: rc.Config.Privileged,
UsernsMode: rc.Config.UsernsMode,
Platform: rc.Config.ContainerArchitecture,
AutoRemove: rc.Config.AutoRemove,
AutoRemove: false, // so a dead service's log survives, cleanupJobResources removes it
Options: rc.ExprEval.Interpolate(ctx, spec.Options),
NetworkMode: networkName,
NetworkAliases: []string{serviceID},
@@ -496,7 +529,7 @@ func (rc *RunContext) startJobContainer() common.Executor {
PortBindings: portBindings,
AllocatePTY: rc.Config.AllocatePTY,
})
rc.ServiceContainers = append(rc.ServiceContainers, c)
rc.serviceContainers = append(rc.serviceContainers, &serviceContainer{name: serviceID, image: serviceImage, container: c})
}
rc.cleanUpJobContainer = rc.cleanupJobResources(networkName, createAndDeleteNetwork)
@@ -531,6 +564,8 @@ func (rc *RunContext) startJobContainer() common.Executor {
return errors.New("Failed to create job container")
}
rc.jobNetworkName = networkName
defer printStartJobContainerGroup(ctx, image, name, networkName)()
return common.NewPipelineExecutor(
rc.pullServicesImages(rc.Config.ForcePull),
@@ -538,9 +573,12 @@ func (rc *RunContext) startJobContainer() common.Executor {
rc.stopJobContainer(),
container.NewDockerNetworkCreateExecutor(networkName, rc.Config.ContainerNetworkCreateOptions).
IfBool(createAndDeleteNetwork),
rc.startServiceContainers(networkName),
rc.startServiceContainers(),
rc.reportUnstartedServices(),
rc.waitForServiceContainers(),
rc.JobContainer.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
rc.JobContainer.Start(false),
rc.captureJobContainerInfo(),
rc.JobContainer.Copy(rc.JobContainer.GetActPath()+"/", &container.FileEntry{
Name: "workflow/event.json",
Mode: 0o644,
@@ -565,7 +603,7 @@ func (rc *RunContext) cleanupJobResources(networkName string, createAndDeleteNet
if removeJobContainer {
errs = append(errs, rc.JobContainer.Remove()(ctx))
}
if len(rc.ServiceContainers) > 0 {
if len(rc.serviceContainers) > 0 {
logger.Infof("Cleaning up services for job %s", rc.JobName)
if err := rc.stopServiceContainers()(ctx); err != nil {
logger.Errorf("Error while cleaning services: %v", err)
@@ -662,21 +700,21 @@ func (rc *RunContext) stopJobContainer() common.Executor {
func (rc *RunContext) pullServicesImages(forcePull bool) common.Executor {
return func(ctx context.Context) error {
execs := []common.Executor{}
for _, c := range rc.ServiceContainers {
execs = append(execs, c.Pull(forcePull))
for _, svc := range rc.serviceContainers {
execs = append(execs, svc.container.Pull(forcePull))
}
return common.NewParallelExecutor(len(execs), execs...)(ctx)
}
}
func (rc *RunContext) startServiceContainers(_ string) common.Executor {
func (rc *RunContext) startServiceContainers() common.Executor {
return func(ctx context.Context) error {
execs := []common.Executor{}
for _, c := range rc.ServiceContainers {
for _, svc := range rc.serviceContainers {
execs = append(execs, common.NewPipelineExecutor(
c.Pull(false),
c.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
c.Start(false),
svc.container.Pull(false),
svc.container.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
svc.container.Start(false),
))
}
return common.NewParallelExecutor(len(execs), execs...)(ctx)
@@ -686,13 +724,159 @@ func (rc *RunContext) startServiceContainers(_ string) common.Executor {
func (rc *RunContext) stopServiceContainers() common.Executor {
return func(ctx context.Context) error {
execs := []common.Executor{}
for _, c := range rc.ServiceContainers {
execs = append(execs, c.Remove().Finally(c.Close()))
for _, svc := range rc.serviceContainers {
execs = append(execs, svc.container.Remove().Finally(svc.container.Close()))
}
return common.NewParallelExecutor(len(execs), execs...)(ctx)
}
}
const (
defaultServiceReadyTimeout = 5 * time.Minute
serviceReadyPollMax = 32 * time.Second
)
var serviceReadyPollInterval = 2 * time.Second // a variable so tests need not wait
// reportUnstartedServices logs a service that did not start. The steps that need it
// report it better than the runner can, so the job carries on.
func (rc *RunContext) reportUnstartedServices() common.Executor {
return func(ctx context.Context) error {
logger := common.Logger(ctx)
for _, svc := range rc.serviceContainers {
info, err := svc.inspect(ctx)
if err != nil {
logger.Debugf("unable to inspect service '%s': %v", svc.name, err)
continue
}
if info.State == container.StateRunning {
continue
}
svc.dumpLogs(ctx)
logger.Warnf("Docker container %s is not in running state: %s (%d)", info.ID, info.State, info.ExitCode)
}
return nil
}
}
// waitForServiceContainers blocks until every service that declares a healthcheck reports
// healthy, as GitHub does, so a first step cannot connect before the service listens.
func (rc *RunContext) waitForServiceContainers() common.Executor {
return func(ctx context.Context) error {
if len(rc.serviceContainers) == 0 {
return nil
}
timeout := rc.Config.ServiceReadyTimeout
switch {
case timeout < 0:
// disabled, but still describe the containers for `job.services`
for _, svc := range rc.serviceContainers {
if _, err := svc.inspect(ctx); err != nil && !errors.Is(err, container.ErrContainerNotFound) {
return err
}
}
return nil
case timeout == 0:
timeout = defaultServiceReadyTimeout
}
// the first error cancels the rest, so a failure does not wait out a sibling's timeout
group, groupCtx := errgroup.WithContext(ctx)
for _, svc := range rc.serviceContainers {
group.Go(func() error {
return svc.waitUntilHealthy(groupCtx, timeout)
})
}
return group.Wait()
}
}
// waitUntilHealthy waits on the healthcheck alone, so a container that declares none is
// ready at once and one that exited is left to the steps that need it.
func (svc *serviceContainer) waitUntilHealthy(ctx context.Context, timeout time.Duration) error {
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
interval := serviceReadyPollInterval
ctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
for {
info, err := svc.inspect(ctx)
if ctxErr := ctx.Err(); ctxErr != nil { // the wait ended, an inspect error only noticed it
if errors.Is(ctxErr, context.DeadlineExceeded) {
return fmt.Errorf("the service '%s' did not become healthy within %s%s", svc.name, timeout, svc.healthOutputSuffix())
}
return ctxErr
}
switch {
case errors.Is(err, container.ErrContainerNotFound):
return nil // gone, so there is no health left to wait on
case err != nil:
return err
}
switch {
case info.Health == container.HealthUnhealthy:
svc.dumpLogs(ctx)
common.Logger(ctx).Errorf("Failed to initialize container %s", svc.image)
return fmt.Errorf("the service '%s' is unhealthy%s", svc.name, svc.healthOutputSuffix())
case info.Health != container.HealthStarting:
rawLogger.Infof("%s service is healthy.", svc.name)
return nil
}
rawLogger.Infof("%s service is starting, waiting %d seconds before checking again.", svc.name, int(interval.Seconds()))
select {
case <-ctx.Done(): // reported at the top of the loop
case <-time.After(interval):
}
interval = min(interval*2, serviceReadyPollMax)
}
}
// dumpLogs writes the container's log to the job log once, however often it is reported.
func (svc *serviceContainer) dumpLogs(ctx context.Context) {
if svc.logsDumped {
return
}
svc.logsDumped = true
if err := svc.container.DumpLogs(ctx); err != nil {
common.Logger(ctx).Debugf("unable to read the log of service '%s': %v", svc.name, err)
}
}
// inspect also records the state for the `job.services` context.
func (svc *serviceContainer) inspect(ctx context.Context) (*container.Info, error) {
info, err := svc.container.Inspect(ctx)
if err != nil {
return nil, fmt.Errorf("failed to inspect service '%s': %w", svc.name, err)
}
svc.info = info
return info, nil
}
func (svc *serviceContainer) healthOutputSuffix() string {
if svc.info == nil || svc.info.HealthOutput == "" {
return ""
}
return ": " + svc.info.HealthOutput
}
// captureJobContainerInfo is a convenience: failing to describe the container must not
// fail the job.
func (rc *RunContext) captureJobContainerInfo() common.Executor {
return func(ctx context.Context) error {
info, err := rc.JobContainer.Inspect(ctx)
if err != nil {
common.Logger(ctx).Debugf("unable to inspect the job container: %v", err)
return nil
}
rc.jobContainerID = info.ID
return nil
}
}
// Prepare the mounts and binds for the worker
// ActionCacheDir is for rc
@@ -1033,9 +1217,26 @@ func (rc *RunContext) getJobContext() *model.JobContext {
if rc.jobCancelled {
jobStatus = "cancelled"
}
return &model.JobContext{
Status: jobStatus,
jobContext := &model.JobContext{
Status: jobStatus,
Services: map[string]model.JobService{}, // an empty map, never null
}
if rc.jobContainerID != "" {
jobContext.Container.ID = rc.jobContainerID
jobContext.Container.Network = rc.jobNetworkName
}
for _, svc := range rc.serviceContainers {
if svc.info == nil {
continue
}
jobContext.Services[svc.name] = model.JobService{
ID: svc.info.ID,
Network: rc.jobNetworkName,
Ports: svc.info.Ports,
}
}
return jobContext
}
func (rc *RunContext) getStepsContext() map[string]*model.StepResult {

View File

@@ -13,6 +13,7 @@ import (
"runtime"
"strings"
"testing"
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/container"
@@ -22,6 +23,7 @@ import (
"github.com/docker/cli/cli/compose/loader"
log "github.com/sirupsen/logrus"
assert "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
require "github.com/stretchr/testify/require"
yaml "go.yaml.in/yaml/v4"
)
@@ -225,6 +227,12 @@ func (fakeContainer) Copy(string, ...*container.FileEntry) common.Executor {
return func(context.Context) error { return nil }
}
func (fakeContainer) Inspect(context.Context) (*container.Info, error) {
return &container.Info{ID: "fake", State: "running", Health: container.HealthNone}, nil
}
func (fakeContainer) DumpLogs(context.Context) error { return nil }
// Regression test: a service without a `credentials:` block resolves to empty
// credentials, which used to overwrite the job container's own credentials.
func TestStartJobContainerKeepsJobCredentialsWithServices(t *testing.T) {
@@ -563,7 +571,7 @@ func TestCleanupJobResourcesCleansServicesWithoutJobContainer(t *testing.T) {
rc := &RunContext{
Config: &Config{},
ServiceContainers: []container.ExecutionsEnvironment{service},
serviceContainers: []*serviceContainer{{name: "svc", container: service}},
}
err := rc.cleanupJobResources("external-network", false)(context.Background())
@@ -586,7 +594,7 @@ func TestCleanupJobResourcesContinuesAfterFailure(t *testing.T) {
Config: &Config{},
Run: &model.Run{Workflow: &model.Workflow{Name: "wf"}, JobID: "job"},
JobContainer: jobContainer,
ServiceContainers: []container.ExecutionsEnvironment{service},
serviceContainers: []*serviceContainer{{name: "svc", container: service}},
}
ctx, cancel := context.WithCancel(context.Background())
@@ -1045,6 +1053,199 @@ func TestRunContext_cleanupFailedStart(t *testing.T) {
})
}
func TestWaitForServiceContainers(t *testing.T) {
origInterval := serviceReadyPollInterval
serviceReadyPollInterval = time.Millisecond
defer func() { serviceReadyPollInterval = origInterval }()
newRunContext := func(timeout time.Duration, services ...*serviceContainer) *RunContext {
return &RunContext{
Config: &Config{ServiceReadyTimeout: timeout},
serviceContainers: services,
}
}
t.Run("returns as soon as a service without a healthcheck runs", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return(&container.Info{ID: "id", State: "running", Health: container.HealthNone}, nil).Once()
rc := newRunContext(0, &serviceContainer{name: "redis", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
service.AssertExpectations(t)
})
t.Run("waits while a service is still starting", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return(&container.Info{ID: "id", State: "running", Health: container.HealthStarting}, nil).Twice()
service.On("Inspect", mock.Anything).
Return(&container.Info{ID: "id", State: "running", Health: container.HealthHealthy}, nil).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
service.AssertExpectations(t)
})
t.Run("fails with the probe output when a service is unhealthy", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).Return(&container.Info{
State: "running",
Health: container.HealthUnhealthy,
HealthOutput: "connection refused",
}, nil).Once()
service.On("DumpLogs", mock.Anything).Return(nil).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "the service 'postgres' is unhealthy: connection refused")
service.AssertExpectations(t)
})
t.Run("lets the steps run when a service exits without a healthcheck", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return(&container.Info{State: "exited", ExitCode: 2, Health: container.HealthNone}, nil).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
})
t.Run("proceeds when the container is gone", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return((*container.Info)(nil), container.ErrContainerNotFound).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
})
t.Run("fails right away when one service fails while another is still starting", func(t *testing.T) {
failing := &containerMock{}
failing.On("Inspect", mock.Anything).
Return(&container.Info{State: "running", Health: container.HealthUnhealthy}, nil)
failing.On("DumpLogs", mock.Anything).Return(nil).Once()
starting := &containerMock{}
starting.On("Inspect", mock.Anything).
Return(&container.Info{State: "running", Health: container.HealthStarting}, nil)
rc := newRunContext(10*time.Second,
&serviceContainer{name: "failing", container: failing},
&serviceContainer{name: "starting", container: starting})
done := make(chan error, 1)
go func() { done <- rc.waitForServiceContainers()(context.Background()) }()
select {
case err := <-done:
require.Error(t, err)
assert.Contains(t, err.Error(), "the service 'failing' is unhealthy")
case <-time.After(2 * time.Second):
t.Fatal("waitForServiceContainers did not fail fast; it waited for the starting service")
}
})
t.Run("gives up once the timeout expires", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Return(&container.Info{State: "running", Health: container.HealthStarting}, nil)
rc := newRunContext(20*time.Millisecond, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "did not become healthy within")
})
t.Run("gives up with the same message when the deadline stops an inspect", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).
Run(func(args mock.Arguments) { <-args.Get(0).(context.Context).Done() }).
Return((*container.Info)(nil), errors.New("inspect aborted"))
rc := newRunContext(20*time.Millisecond, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "did not become healthy within")
})
t.Run("does not wait when the timeout is negative", func(t *testing.T) {
service := &containerMock{}
// Still described once, so the `job.services` context is filled either way.
service.On("Inspect", mock.Anything).
Return(&container.Info{ID: "id", State: "running", Health: container.HealthStarting}, nil).Once()
svc := &serviceContainer{name: "postgres", container: service}
rc := newRunContext(-1, svc)
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
service.AssertExpectations(t)
assert.Equal(t, "id", svc.info.ID)
})
t.Run("fails on an inspect error even when the timeout is negative", func(t *testing.T) {
service := &containerMock{}
service.On("Inspect", mock.Anything).Return((*container.Info)(nil), errors.New("daemon is gone")).Once()
rc := newRunContext(-1, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "failed to inspect service 'postgres'")
})
t.Run("is a no-op without services", func(t *testing.T) {
require.NoError(t, newRunContext(0).waitForServiceContainers()(context.Background()))
})
}
func TestReportUnstartedServices(t *testing.T) {
dead := &containerMock{}
dead.On("Inspect", mock.Anything).Return(&container.Info{ID: "dead-id", State: "exited", ExitCode: 1}, nil).Once()
dead.On("DumpLogs", mock.Anything).Return(nil).Once()
running := &containerMock{}
running.On("Inspect", mock.Anything).Return(&container.Info{ID: "run-id", State: "running"}, nil).Once()
rc := &RunContext{serviceContainers: []*serviceContainer{
{name: "postgres", container: dead},
{name: "redis", container: running},
}}
require.NoError(t, rc.reportUnstartedServices()(context.Background()))
dead.AssertExpectations(t)
running.AssertExpectations(t)
}
func TestGetJobContextReportsContainers(t *testing.T) {
rc := &RunContext{
jobNetworkName: "job-network",
jobContainerID: "job-container-id",
serviceContainers: []*serviceContainer{
{name: "postgres", info: &container.Info{ID: "svc-id", Ports: map[string]string{"5432": "49153"}}},
// A service that publishes no port reports an empty map, as GitHub does.
{name: "redis", info: &container.Info{ID: "redis-id", Ports: map[string]string{}}},
// A service that never reported is left out rather than reported as empty.
{name: "mailhog"},
},
}
jobContext := rc.getJobContext()
assert.Equal(t, "job-container-id", jobContext.Container.ID)
assert.Equal(t, "job-network", jobContext.Container.Network)
assert.Equal(t, map[string]model.JobService{
"postgres": {ID: "svc-id", Network: "job-network", Ports: map[string]string{"5432": "49153"}},
"redis": {ID: "redis-id", Network: "job-network", Ports: map[string]string{}},
}, jobContext.Services)
}
// A job that never started a container reports an empty context, not a placeholder.
func TestGetJobContextWithoutContainer(t *testing.T) {
jobContext := (&RunContext{}).getJobContext()
assert.Empty(t, jobContext.Container.ID)
assert.Empty(t, jobContext.Container.Network)
assert.Empty(t, jobContext.Services)
}
func TestImageOSFromImage(t *testing.T) {
for _, tc := range []struct {
image string

View File

@@ -94,6 +94,7 @@ type Config struct {
InsecureSkipTLS bool // whether to skip verifying TLS certificate of the Gitea instance
MaxParallel int // max parallel jobs to run across all workflows (0 = no limit, uses CPU count)
AllocatePTY bool // allocate a pseudo-TTY for each step's process
ServiceReadyTimeout time.Duration // how long a job waits for its service containers to report healthy (0 uses the default)
RunnerName string // name this runner registered with, reported as `runner.name`, defaults to the hostname
JobStartedHook string // script run inside the job environment before the job's first step; ACTIONS_RUNNER_HOOK_JOB_STARTED is read from Env when empty
JobCompletedHook string // script run inside the job environment after the job's last step; ACTIONS_RUNNER_HOOK_JOB_COMPLETED is read from Env when empty

View File

@@ -165,9 +165,22 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
}
_ = rc.JobContainer.Copy(actPath, files...)(ctx)
// The command handler needs the step's env to judge ACTIONS_ALLOW_UNSECURE_COMMANDS.
// Cloned: the step executor keeps writing to its own env map after this point, on a
// different goroutine from the command handler that reads it.
rc.setCurrentStepEnv(maps0.Clone(*step.getEnv()))
defer rc.setCurrentStepEnv(nil)
_ = rc.takeUnsecureCommandError() // a refusal from before any step belongs to no step
timeoutctx, cancelTimeOut := evaluateStepTimeout(ctx, rc.ExprEval, stepModel)
defer cancelTimeOut()
err = executor(timeoutctx)
// Always take it, so the job-scoped error cannot leak onto a later step. A refusal
// fails the step as it does on GitHub, but the executor's own error wins.
insecureErr := rc.takeUnsecureCommandError()
if err == nil {
err = insecureErr
}
if err == nil {
logger.WithField("stepResult", stepResult.Outcome).Infof("Success - %s %s", stage, stepString)
@@ -181,7 +194,7 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
}
if continueOnError {
logger.Errorf("##[error]%s", escapeCommandData(err.Error()))
logger.Errorf("##[error]%s", EscapeCommandData(err.Error()))
logger.Infof("Failed but continue next step")
err = nil
stepResult.Conclusion = model.StepStatusSuccess

View File

@@ -18,6 +18,7 @@ import (
"gitea.com/gitea/runner/act/model"
"github.com/kballard/go-shellquote"
"github.com/sirupsen/logrus"
yaml "go.yaml.in/yaml/v4"
)
@@ -63,7 +64,7 @@ func (sr *stepRun) printRunScriptActionDetails(ctx context.Context) {
normalized := strings.TrimRight(strings.ReplaceAll(sr.interpolatedScript, "\r\n", "\n"), "\n")
rawLogger.Infof("::group::Run %s", escapeCommandData(sr.runScriptGroupTitle(normalized)))
rawLogger.Infof("::group::Run %s", EscapeCommandData(sr.runScriptGroupTitle(normalized)))
if normalized != "" {
for line := range strings.SplitSeq(normalized, "\n") {
@@ -90,12 +91,12 @@ func printRunActionHeader(ctx context.Context, step *model.Step, env map[string]
if step.Name != "" {
title = step.Name
}
rawLogger.Infof("::group::Run %s", escapeCommandData(title))
rawLogger.Infof("::group::Run %s", EscapeCommandData(title))
if len(step.With) > 0 {
rawLogger.Infof("with:")
for _, k := range slices.Sorted(maps.Keys(step.With)) {
rawLogger.Infof(" %s: %s", k, step.With[k])
logKeyedValue(rawLogger, k, step.With[k])
}
}
@@ -129,7 +130,17 @@ func printStepEnvBlock(ctx context.Context, step *model.Step, env map[string]str
if caseInsensitive {
lookupKey = strings.ToUpper(k)
}
rawLogger.Infof(" %s: %s", k, envLookup[lookupKey])
logKeyedValue(rawLogger, k, envLookup[lookupKey])
}
}
// logKeyedValue prints one row per line of value: Gitea stores one log row per line, so an
// embedded newline would reach the user as a literal "\n".
func logKeyedValue(rawLogger *logrus.Entry, key, value string) {
lines := strings.Split(strings.ReplaceAll(value, "\r\n", "\n"), "\n")
rawLogger.Infof(" %s: %s", key, lines[0])
for _, line := range lines[1:] {
rawLogger.Infof(" %s", line)
}
}

View File

@@ -6,6 +6,7 @@ package runner
import (
"context"
"errors"
"testing"
"gitea.com/gitea/runner/act/common"
@@ -14,6 +15,7 @@ import (
log "github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
yaml "go.yaml.in/yaml/v4"
)
@@ -354,3 +356,48 @@ func TestIsContinueOnError(t *testing.T) {
assertObject.False(continueOnError)
assertObject.Error(err)
}
// A refused ::set-env::/::add-path:: records a job-scoped error. When the step that
// produced it also fails on its own, the refusal must be cleared at the step boundary, so
// it fails only that step and never leaks onto a later step that runs anyway (if: always()).
func TestRunStepExecutorDoesNotLeakRefusalToNextStep(t *testing.T) {
cm := &containerMock{}
noop := func(context.Context) error { return nil }
cm.On("Copy", mock.Anything, mock.Anything).Return(noop)
cm.On("UpdateFromEnv", mock.Anything, mock.Anything).Return(noop)
rc := &RunContext{
Config: &Config{Env: map[string]string{}},
Run: &model.Run{
JobID: "1",
Workflow: &model.Workflow{Jobs: map[string]*model.Job{"1": {}}},
},
Env: map[string]string{},
StepResults: map[string]*model.StepResult{},
JobContainer: cm,
}
rc.ExprEval = rc.NewExpressionEvaluator(context.Background())
// Dryrun skips reading the path file back from the (mocked) container.
ctx := common.WithDryrun(context.Background(), true)
// A refusal parsed out of the job container's own output belongs to no step, so the
// first step must not be failed by it.
rc.commandHandler(ctx)("::set-env name=setup::y\n")
stepSetup := &stepRun{RunContext: rc, Step: &model.Step{ID: "setup"}, env: map[string]string{}}
require.NoError(t, runStepExecutor(stepSetup, stepStageMain, func(context.Context) error { return nil })(ctx))
// Step A refuses a ::set-env:: and then fails on its own.
stepA := &stepRun{RunContext: rc, Step: &model.Step{ID: "a"}, env: map[string]string{}}
errA := runStepExecutor(stepA, stepStageMain, func(context.Context) error {
rc.commandHandler(ctx)("::set-env name=x::y\n")
return errors.New("boom")
})(ctx)
// The step fails with its own error, not the refusal.
require.ErrorContains(t, errA, "boom")
// Step B runs despite step A's failure (if: always()) and issues no unsecure command;
// it must not inherit step A's refusal.
stepB := &stepRun{RunContext: rc, Step: &model.Step{ID: "b", If: yaml.Node{Value: "always()"}}, env: map[string]string{}}
errB := runStepExecutor(stepB, stepStageMain, func(context.Context) error { return nil })(ctx)
require.NoError(t, errB)
}

View File

@@ -3,6 +3,7 @@ jobs:
_:
runs-on: ubuntu-latest
env:
ACTIONS_ALLOW_UNSECURE_COMMANDS: 'true'
MYGLOBALENV3: myglobalval3
steps:
- uses: actions/checkout@v4

View File

@@ -4,6 +4,8 @@ on: push
jobs:
build:
runs-on: ubuntu-latest
env:
ACTIONS_ALLOW_UNSECURE_COMMANDS: 'true'
steps:
- name: TEST set-env
run: echo "::set-env name=foo::bar"

View File

@@ -15,3 +15,9 @@ jobs:
echo "id: ${{ job.services.postgres.id }}"
echo "network: ${{ job.services.postgres.network }}"
echo "ports: ${{ job.services.postgres.ports }}"
- name: The job context describes the started containers
run: |
test -n "${{ job.container.id }}"
test -n "${{ job.services.postgres.id }}"
test -n "${{ job.services.postgres.ports['80'] }}"
test "${{ job.services.postgres.network }}" = "${{ job.container.network }}"

View File

@@ -16,7 +16,7 @@ the runner as a background service on a systemd host.
`.runner` file ends up in the working directory:
```bash
sudo -u gitea-runner gitea-runner generate-config > /etc/gitea-runner/config.yaml
sudo -u gitea-runner gitea-runner config generate > /etc/gitea-runner/config.yaml
cd /var/lib/gitea-runner
sudo -u gitea-runner gitea-runner register --config /etc/gitea-runner/config.yaml
```

View File

@@ -52,7 +52,7 @@ export DOCKER_HOST=unix:///run/user/$(id -u)/docker.sock
- Generate a `gitea-runner` configuration file in the data directory. Edit the file to adjust for the system.
```bash
gitea-runner generate-config >/home/rootless/gitea-runner/config
gitea-runner config generate >/home/rootless/gitea-runner/config
```
- Create a new user-level`systemd` unit file as `/home/rootless/.config/systemd/user/gitea-runner.service` with the following contents:

2
go.mod
View File

@@ -37,6 +37,7 @@ require (
go.etcd.io/bbolt v1.5.0
go.yaml.in/yaml/v4 v4.0.0-rc.3
golang.org/x/net v0.57.0
golang.org/x/sync v0.22.0
golang.org/x/sys v0.47.0
golang.org/x/term v0.45.0
golang.org/x/text v0.40.0
@@ -104,7 +105,6 @@ require (
go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.54.0 // indirect
golang.org/x/sync v0.22.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

View File

@@ -5,10 +5,8 @@ package cmd
import (
"context"
"fmt"
"os"
"gitea.com/gitea/runner/internal/pkg/config"
"gitea.com/gitea/runner/internal/pkg/ver"
"github.com/spf13/cobra"
@@ -23,7 +21,7 @@ func Execute(ctx context.Context) {
SilenceUsage: true,
}
configFile := ""
rootCmd.PersistentFlags().StringVarP(&configFile, "config", "c", "", "Config file path")
rootCmd.PersistentFlags().StringVarP(&configFile, "config", "c", "", "Config file path. `config` subcommands fall back to config.yaml in the working directory or next to the executable")
// ./gitea-runner register
var regArgs registerArgs
@@ -61,14 +59,12 @@ func Execute(ctx context.Context) {
rootCmd.AddCommand(loadBugReportCmd())
// ./gitea-runner config
rootCmd.AddCommand(&cobra.Command{
Use: "generate-config",
Short: "Generate an example config file",
Args: cobra.MaximumNArgs(0),
Run: func(_ *cobra.Command, _ []string) {
fmt.Printf("%s", config.Example)
},
})
rootCmd.AddCommand(loadConfigCmd(&configFile))
// ./gitea-runner generate-config
generateConfigCmd := loadGenerateConfigCmd("generate-config")
generateConfigCmd.Deprecated = "use `config generate` instead."
rootCmd.AddCommand(generateConfigCmd)
// ./gitea-runner cache-server
var cacheArgs cacheServerArgs

116
internal/app/cmd/config.go Normal file
View File

@@ -0,0 +1,116 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package cmd
import (
"fmt"
"os"
"path/filepath"
"slices"
"strings"
"gitea.com/gitea/runner/internal/pkg/config"
"github.com/spf13/cobra"
)
func loadConfigCmd(configFile *string) *cobra.Command {
configCmd := &cobra.Command{
Use: "config",
Short: "Generate, read and edit config files",
Args: cobra.MaximumNArgs(0),
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
}
configCmd.AddCommand(loadGenerateConfigCmd("generate"))
configCmd.AddCommand(&cobra.Command{
Use: "get <key>",
Short: "Print the value of a config key",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
file, err := resolveConfigFile(cmd, configFile)
if err != nil {
return err
}
value, err := config.GetValue(file, args[0])
if err != nil {
return err
}
fmt.Fprintln(cmd.OutOrStdout(), value)
return nil
},
})
for _, sub := range []struct {
use string
short string
edit func(file, key string, values ...string) error
}{
{"set <key> <value>...", "Set the value of a config key", config.SetValue},
{"add <key> <value>...", "Append values to a list config key", config.AddValue},
{"remove <key> <value>...", "Remove values from a list config key", config.RemoveValue},
} {
valueCmd := &cobra.Command{
Use: sub.use,
Short: sub.short,
Args: cobra.MinimumNArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
file, err := resolveConfigFile(cmd, configFile)
if err != nil {
return err
}
return sub.edit(file, args[0], args[1:]...)
},
}
valueCmd.Flags().SetInterspersed(false) // so a value such as `--cpus 2` is not parsed as a flag
configCmd.AddCommand(valueCmd)
}
return configCmd
}
func loadGenerateConfigCmd(use string) *cobra.Command {
return &cobra.Command{
Use: use,
Short: "Generate an example config file",
Args: cobra.MaximumNArgs(0),
Run: func(cmd *cobra.Command, _ []string) {
fmt.Fprintf(cmd.OutOrStdout(), "%s", config.Example)
},
}
}
var defaultConfigFileNames = []string{"config.yaml", "config.yml"}
func resolveConfigFile(cmd *cobra.Command, configFile *string) (string, error) {
if *configFile != "" {
return *configFile, nil
}
var dirs []string
if wd, err := os.Getwd(); err == nil {
dirs = append(dirs, wd)
}
if exe, err := os.Executable(); err == nil {
if dir := filepath.Dir(exe); !slices.Contains(dirs, dir) {
dirs = append(dirs, dir)
}
}
for _, dir := range dirs {
for _, name := range defaultConfigFileNames {
candidate := filepath.Join(dir, name)
if stat, err := os.Stat(candidate); err == nil && !stat.IsDir() {
fmt.Fprintf(cmd.ErrOrStderr(), "using config file %q\n", candidate)
return candidate, nil
}
}
}
return "", fmt.Errorf("no %s found in %s, pass one with --config",
strings.Join(defaultConfigFileNames, " or "), strings.Join(dirs, " or "))
}

View File

@@ -0,0 +1,75 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package cmd
import (
"bytes"
"os"
"path/filepath"
"testing"
"gitea.com/gitea/runner/internal/pkg/config"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func runConfigCmd(t *testing.T, configFile string, args ...string) (string, string, error) {
t.Helper()
out, errOut := &bytes.Buffer{}, &bytes.Buffer{}
cmd := loadConfigCmd(&configFile)
cmd.SetOut(out)
cmd.SetErr(errOut)
cmd.SetArgs(args)
err := cmd.Execute()
return out.String(), errOut.String(), err
}
func TestConfigCmdGeneratePrintsTheExample(t *testing.T) {
out, _, err := runConfigCmd(t, "", "generate")
require.NoError(t, err)
assert.Equal(t, string(config.Example), out)
}
// The subcommands only wire arguments through, so one pass over all of them is enough.
func TestConfigCmdEditsTheFile(t *testing.T) {
file := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(file, []byte("runner:\n labels:\n - self-hosted\n"), 0o600))
_, _, err := runConfigCmd(t, file, "set", "container.options", "--cpus 2")
require.NoError(t, err)
_, _, err = runConfigCmd(t, file, "add", "runner.labels", "ubuntu:docker://node:22")
require.NoError(t, err)
_, _, err = runConfigCmd(t, file, "remove", "runner.labels", "self-hosted")
require.NoError(t, err)
out, _, err := runConfigCmd(t, file, "get", "runner.labels")
require.NoError(t, err)
assert.Equal(t, "ubuntu:docker://node:22\n", out)
out, _, err = runConfigCmd(t, file, "get", "container.options")
require.NoError(t, err)
assert.Equal(t, "--cpus 2\n", out)
}
func TestConfigCmdResolvesTheConfigFile(t *testing.T) {
t.Run("falls back to the working directory", func(t *testing.T) {
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, "config.yaml"), []byte("runner:\n capacity: 2\n"), 0o600))
t.Chdir(dir)
out, errOut, err := runConfigCmd(t, "", "get", "runner.capacity")
require.NoError(t, err)
assert.Equal(t, "2\n", out)
assert.Contains(t, errOut, "using config file")
})
t.Run("reports that none was found", func(t *testing.T) {
t.Chdir(t.TempDir())
_, _, err := runConfigCmd(t, "", "set", "runner.capacity", "4")
require.Error(t, err)
assert.Contains(t, err.Error(), "--config")
})
}

View File

@@ -510,6 +510,7 @@ func (r *Runner) run(ctx context.Context, task *runnerv1.Task, reporter *report.
RunnerUUID: r.uuid,
},
ContainerOptions: r.cfg.Container.Options,
ServiceReadyTimeout: r.cfg.Container.ServiceReadyTimeout,
ContainerDaemonSocket: r.cfg.Container.DockerHost,
Privileged: r.cfg.Container.Privileged,
DefaultActionInstance: r.getDefaultActionsURL(task),
@@ -607,7 +608,8 @@ func (r *Runner) registerExternalCacheJob(token string, cred artifactcache.JobCr
}); err != nil {
log.Warnf("cache external_server register failed (%s): %v", base, err)
if reporter != nil {
reporter.Logf("::warning::cache external_server register failed (%s): %v — cache requests from this job will be unauthenticated and likely return 401", base, err)
reporter.Logf("::warning::%s", runner.EscapeCommandData(fmt.Sprintf(
"cache external_server register failed (%s): %v — cache requests from this job will be unauthenticated and likely return 401", base, err)))
}
} else {
resultsURL, _ = body["results_url"].(string) // absent from a server too old to forward
@@ -617,7 +619,8 @@ func (r *Runner) registerExternalCacheJob(token string, cred artifactcache.JobCr
map[string]any{"token": token}); err != nil {
log.Warnf("cache external_server revoke failed (%s): %v", base, err)
if reporter != nil {
reporter.Logf("::warning::cache external_server revoke failed (%s): %v", base, err)
reporter.Logf("::warning::%s", runner.EscapeCommandData(fmt.Sprintf(
"cache external_server revoke failed (%s): %v", base, err)))
}
}
}, resultsURL

View File

@@ -1,7 +1,7 @@
# Example configuration file, it's safe to copy this as the default config file without any modification.
# You don't have to copy this file to your instance,
# just run `./gitea-runner generate-config > config.yaml` to generate a config file.
# just run `./gitea-runner config generate > config.yaml` to generate a config file.
# Logging for the runner process itself (messages printed to stderr).
# This does not control how workflow step output is streamed to the Gitea UI;
@@ -176,7 +176,7 @@ container:
# network is labelled com.gitea.runner.uuid=<this runner's uuid>, which is how the idle
# cleanup tells its own leftovers apart from those of other runners on the same daemon.
network_create_options:
enable_ipv4: true # Omit to use Docker's default (IPv4 enabled). Set false to disable IPv4.
enable_ipv4: true # Omit to use Docker's default (IPv4 enabled). Set false to disable IPv4.
enable_ipv6: false # Omit to use Docker's default (IPv6 disabled). Enabling it requires dockerd started with --ipv6.
# Whether to use privileged mode or not when launching task containers (privileged mode is required for Docker-in-Docker).
privileged: false
@@ -226,6 +226,9 @@ container:
# from the DinD daemon's filesystem. When enabled, ensure the workspace parent
# directory is also mounted into the runner container and listed in valid_volumes.
bind_workdir: false
# How long a job waits for a service container that declares a healthcheck to become
# healthy. A negative value (e.g. -1s) starts the steps without waiting.
service_ready_timeout: 5m
host:
# The parent directory of a job's working directory.

View File

@@ -92,6 +92,7 @@ type Container struct {
RequireDocker bool `yaml:"require_docker"` // Always require a reachable docker daemon, even if not required by runner
DockerTimeout time.Duration `yaml:"docker_timeout"` // Timeout to wait for the docker daemon to be reachable, if docker is required by require_docker or runner
BindWorkdir bool `yaml:"bind_workdir"` // BindWorkdir binds the workspace to the host filesystem instead of using Docker volumes. Required for DinD when jobs use docker compose with bind mounts.
ServiceReadyTimeout time.Duration `yaml:"service_ready_timeout"` // ServiceReadyTimeout bounds how long a job waits for a service container that declares a healthcheck to report healthy. Negative disables waiting.
}
type ContainerNetworkCreateOptions struct {

530
internal/pkg/config/edit.go Normal file
View File

@@ -0,0 +1,530 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package config
import (
"bytes"
"errors"
"fmt"
"os"
"path/filepath"
"reflect"
"slices"
"strconv"
"strings"
"time"
"go.yaml.in/yaml/v4"
)
type fieldKind int
const (
kindScalar fieldKind = iota
kindSequence
kindSection
)
var durationType = reflect.TypeFor[time.Duration]()
// GetValue renders a flat list or mapping one entry per line, and anything nested as YAML.
func GetValue(file, path string) (string, error) {
session, err := loadForEdit(file, path)
if err != nil {
return "", err
}
node, err := lookupNode(session.root, session.segments)
if err != nil {
return "", err
}
return renderNode(node)
}
func SetValue(file, path string, values ...string) error {
session, err := loadForEdit(file, path)
if err != nil {
return err
}
var replacement *yaml.Node
switch session.field.kind {
case kindSequence:
if len(values) == 0 {
return fmt.Errorf("%q needs at least one value", path)
}
items, err := session.scalars(values)
if err != nil {
return err
}
replacement = &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq", Content: items}
case kindScalar:
if len(values) != 1 {
return fmt.Errorf("%q takes exactly one value", path)
}
items, err := session.scalars(values)
if err != nil {
return err
}
replacement = items[0]
default:
return fmt.Errorf("%q is a section, set one of its keys instead", path)
}
node, err := ensureNode(session.root, session.segments)
if err != nil {
return err
}
replaceNode(node, replacement)
return session.write()
}
func AddValue(file, path string, values ...string) error {
session, err := loadSequenceEdit(file, path, values)
if err != nil {
return err
}
items, err := session.scalars(values)
if err != nil {
return err
}
node, err := ensureNode(session.root, session.segments)
if err != nil {
return err
}
if node.Kind != yaml.SequenceNode {
replaceNode(node, &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq"})
}
for _, item := range items {
if indexOfValue(node, item.Value) >= 0 {
return fmt.Errorf("%s already contains %q", path, item.Value)
}
node.Content = append(node.Content, item)
}
return session.write()
}
func RemoveValue(file, path string, values ...string) error {
session, err := loadSequenceEdit(file, path, values)
if err != nil {
return err
}
items, err := session.scalars(values)
if err != nil {
return err
}
node, err := lookupNode(session.root, session.segments)
if err != nil {
return err
}
if node.Kind != yaml.SequenceNode {
return fmt.Errorf("%s is not a list in %q", path, file)
}
for _, item := range items {
index := indexOfValue(node, item.Value)
if index < 0 {
return fmt.Errorf("%s does not contain %q", path, item.Value)
}
node.Content = slices.Delete(node.Content, index, index+1)
}
return session.write()
}
func indexOfValue(seq *yaml.Node, value string) int {
for i, item := range seq.Content {
if item.Kind == yaml.ScalarNode && item.Value == value {
return i
}
}
return -1
}
// replaceNode assigns field by field, as *node = *with would drop the comments attached to node.
func replaceNode(node, with *yaml.Node) {
node.Kind, node.Tag, node.Style, node.Value, node.Content = with.Kind, with.Tag, with.Style, with.Value, with.Content
}
type editSession struct {
file string
path string
original []byte
root *yaml.Node
field *fieldInfo
segments []string
}
// loadForEdit validates the path and parses the file, so every caller fails before anything is written.
func loadForEdit(file, path string) (*editSession, error) {
if path == "" {
return nil, errors.New("no config key given")
}
segments := strings.Split(path, ".")
field, err := resolvePath(segments)
if err != nil {
return nil, err
}
content, err := os.ReadFile(file)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return nil, fmt.Errorf("config file %q does not exist, create one with `config generate`", file)
}
return nil, err
}
var root yaml.Node
if err := yaml.Unmarshal(content, &root); err != nil {
return nil, fmt.Errorf("parse config file %q: %w", file, err)
}
if root.Kind == 0 || len(root.Content) == 0 {
root = yaml.Node{
Kind: yaml.DocumentNode,
Content: []*yaml.Node{{Kind: yaml.MappingNode, Tag: "!!map"}},
}
}
if root.Content[0].Kind != yaml.MappingNode {
return nil, fmt.Errorf("config file %q is not a YAML mapping", file)
}
return &editSession{file: file, path: path, original: content, root: &root, field: field, segments: segments}, nil
}
func loadSequenceEdit(file, path string, values []string) (*editSession, error) {
session, err := loadForEdit(file, path)
if err != nil {
return nil, err
}
if session.field.kind != kindSequence {
return nil, fmt.Errorf("%q is not a list, use `config set` instead", path)
}
if len(values) == 0 {
return nil, fmt.Errorf("%q needs at least one value", path)
}
return session, nil
}
func (s *editSession) scalars(values []string) ([]*yaml.Node, error) {
nodes := make([]*yaml.Node, 0, len(values))
for _, value := range values {
node, err := scalarNode(s.field.typ, value)
if err != nil {
return nil, fmt.Errorf("%s: %w", s.path, err)
}
nodes = append(nodes, node)
}
return nodes, nil
}
func lookupNode(root *yaml.Node, segments []string) (*yaml.Node, error) {
node := root.Content[0]
for i, segment := range segments {
if node.Kind != yaml.MappingNode {
return nil, fmt.Errorf("%q is not set", strings.Join(segments[:i], "."))
}
value := mappingValue(node, segment)
if value == nil {
return nil, fmt.Errorf("%q is not set", strings.Join(segments[:i+1], "."))
}
node = value
}
return node, nil
}
func ensureNode(root *yaml.Node, segments []string) (*yaml.Node, error) {
node := root.Content[0]
for i, segment := range segments {
if node.Kind != yaml.MappingNode {
if node.Kind == yaml.ScalarNode && node.Tag == "!!null" {
node.Kind, node.Tag, node.Style, node.Value = yaml.MappingNode, "!!map", 0, ""
} else {
return nil, fmt.Errorf("%q is not a section", strings.Join(segments[:i], "."))
}
}
value := mappingValue(node, segment)
if value == nil {
value = &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!null"}
node.Content = append(node.Content,
&yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: segment},
value)
}
node = value
}
return node, nil
}
func mappingValue(mapping *yaml.Node, key string) *yaml.Node {
for i := 0; i+1 < len(mapping.Content); i += 2 {
if mapping.Content[i].Value == key {
return mapping.Content[i+1]
}
}
return nil
}
func renderNode(node *yaml.Node) (string, error) {
if !allScalars(node.Content) {
encoded, err := encodeYAML(node)
if err != nil {
return "", err
}
return strings.TrimRight(string(encoded), "\n"), nil
}
switch node.Kind {
case yaml.SequenceNode:
lines := make([]string, 0, len(node.Content))
for _, item := range node.Content {
lines = append(lines, item.Value)
}
return strings.Join(lines, "\n"), nil
case yaml.MappingNode:
lines := make([]string, 0, len(node.Content)/2)
for i := 0; i+1 < len(node.Content); i += 2 {
lines = append(lines, node.Content[i].Value+"="+node.Content[i+1].Value)
}
return strings.Join(lines, "\n"), nil
default:
return node.Value, nil
}
}
func allScalars(nodes []*yaml.Node) bool {
for _, node := range nodes {
if node.Kind != yaml.ScalarNode {
return false
}
}
return true
}
func encodeYAML(node *yaml.Node) ([]byte, error) {
var buf bytes.Buffer
encoder := yaml.NewEncoder(&buf)
encoder.SetIndent(2)
if err := encoder.Encode(node); err != nil {
return nil, err
}
if err := encoder.Close(); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// restoreBlankLines re-inserts the blank lines between top-level sections that the encoder drops.
func restoreBlankLines(original, generated []byte) []byte {
spaced := map[string]bool{}
blank := false
for line := range strings.Lines(string(original)) {
line = strings.TrimRight(line, "\r\n")
switch {
case strings.TrimSpace(line) == "":
blank = true
case strings.HasPrefix(line, "#"): // the block belongs to the key below it
default:
if key, ok := topLevelKey(line); ok && blank {
spaced[key] = true
}
blank = false
}
}
var out []string
for line := range strings.Lines(string(generated)) {
line = strings.TrimRight(line, "\r\n")
if key, ok := topLevelKey(line); ok && spaced[key] {
start := len(out)
for start > 0 && strings.HasPrefix(out[start-1], "#") {
start--
}
if start > 0 && strings.TrimSpace(out[start-1]) != "" {
out = slices.Insert(out, start, "")
}
}
out = append(out, line)
}
if bytes.HasSuffix(generated, []byte("\n")) {
out = append(out, "")
}
return []byte(strings.Join(out, "\n"))
}
func topLevelKey(line string) (string, bool) {
if line == "" || line[0] == ' ' || line[0] == '\t' || line[0] == '#' || line[0] == '-' {
return "", false
}
key, _, ok := strings.Cut(line, ":")
return key, ok
}
func (s *editSession) write() error {
generated, err := encodeYAML(s.root)
if err != nil {
return err
}
// A file the runner already refused to load stays the user's to fix, only a regression is rejected.
if err := yaml.Unmarshal(generated, &Config{}); err != nil && yaml.Unmarshal(s.original, &Config{}) == nil {
return fmt.Errorf("the edit would produce a config the runner cannot load: %w", err)
}
content := restoreBlankLines(s.original, generated)
if bytes.Contains(s.original, []byte("\r\n")) { // the encoder only ever emits LF
content = bytes.ReplaceAll(content, []byte("\n"), []byte("\r\n"))
}
file := s.file
if resolved, err := filepath.EvalSymlinks(file); err == nil {
file = resolved // keeps a config linked in from elsewhere intact
}
var info os.FileInfo
mode := os.FileMode(0o600)
if stat, err := os.Stat(file); err == nil {
info, mode = stat, stat.Mode().Perm()
}
temp, err := os.CreateTemp(filepath.Dir(file), filepath.Base(file)+".*.tmp")
if err != nil {
return err
}
defer os.Remove(temp.Name())
if _, err := temp.Write(content); err != nil {
temp.Close()
return err
}
if err := temp.Sync(); err != nil {
temp.Close()
return err
}
if err := temp.Close(); err != nil {
return err
}
if info != nil { // before the chmod, as a chown can clear mode bits
if err := preserveOwner(temp.Name(), info); err != nil {
return err
}
}
if err := os.Chmod(temp.Name(), mode); err != nil {
return err
}
return os.Rename(temp.Name(), file)
}
type fieldInfo struct {
kind fieldKind
typ reflect.Type // the element type for a sequence
}
// resolvePath walks the Config struct through the yaml tags of a dotted path.
func resolvePath(segments []string) (*fieldInfo, error) {
typ := reflect.TypeFor[Config]()
for i, segment := range segments {
switch typ.Kind() {
case reflect.Struct:
field, ok := fieldByYAMLName(typ, segment)
if !ok {
return nil, fmt.Errorf("unknown config key %q, valid keys here: %s",
strings.Join(segments[:i+1], "."), strings.Join(yamlNames(typ), ", "))
}
typ = field.Type
case reflect.Map:
// The segment names a user-defined entry, so the walk ends here.
if i != len(segments)-1 {
return nil, fmt.Errorf("%q has no sub-keys", strings.Join(segments[:i+1], "."))
}
return &fieldInfo{kind: kindScalar, typ: typ.Elem()}, nil
default:
return nil, fmt.Errorf("%q is a value, not a section", strings.Join(segments[:i], "."))
}
}
switch typ.Kind() {
case reflect.Slice:
return &fieldInfo{kind: kindSequence, typ: typ.Elem()}, nil
case reflect.Map, reflect.Struct:
return &fieldInfo{kind: kindSection}, nil
default:
return &fieldInfo{kind: kindScalar, typ: typ}, nil
}
}
func fieldByYAMLName(typ reflect.Type, name string) (reflect.StructField, bool) {
for field := range typ.Fields() {
if yamlName(field) == name {
return field, true
}
}
return reflect.StructField{}, false
}
func yamlNames(typ reflect.Type) []string {
names := make([]string, 0, typ.NumField())
for field := range typ.Fields() {
if name := yamlName(field); name != "-" {
names = append(names, name)
}
}
slices.Sort(names)
return names
}
func yamlName(field reflect.StructField) string {
name, _, _ := strings.Cut(field.Tag.Get("yaml"), ",")
if name == "" {
return strings.ToLower(field.Name)
}
return name
}
// scalarNode types the value, so a bad one is reported instead of landing in the file as a string.
func scalarNode(typ reflect.Type, value string) (*yaml.Node, error) {
if typ.Kind() == reflect.Pointer {
typ = typ.Elem()
}
if typ == durationType {
duration, err := time.ParseDuration(value)
if err != nil {
return nil, fmt.Errorf("%q is not a duration such as 30s, 5m or 3h", value)
}
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: duration.String()}, nil
}
switch typ.Kind() {
case reflect.Bool:
parsed, err := strconv.ParseBool(value)
if err != nil {
return nil, fmt.Errorf("%q is not a boolean", value)
}
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!bool", Value: strconv.FormatBool(parsed)}, nil
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
parsed, err := strconv.ParseInt(value, 10, typ.Bits())
if err != nil {
return nil, fmt.Errorf("%q is not a valid %s", value, typ.Kind())
}
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!int", Value: strconv.FormatInt(parsed, 10)}, nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
parsed, err := strconv.ParseUint(value, 10, typ.Bits())
if err != nil {
return nil, fmt.Errorf("%q is not a valid %s", value, typ.Kind())
}
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!int", Value: strconv.FormatUint(parsed, 10)}, nil
case reflect.String:
return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: value}, nil
default:
return nil, fmt.Errorf("unsupported config value type %s", typ)
}
}

View File

@@ -0,0 +1,13 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build windows || plan9
package config
import "os"
// preserveOwner is a no-op where a new file inherits its ownership from the directory.
func preserveOwner(_ string, _ os.FileInfo) error {
return nil
}

View File

@@ -0,0 +1,267 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package config
import (
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const editFixture = `# A leading comment.
log:
# The logging level.
level: info
runner:
capacity: 1
envs:
EXISTING: value
timeout: 3h
labels:
- ubuntu-latest:docker://node:20
- self-hosted
`
func writeEditFixture(t *testing.T) string {
t.Helper()
path := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(path, []byte(editFixture), 0o600))
return path
}
func TestEditValues(t *testing.T) {
tests := []struct {
name string
edit func(file string) error
assert func(t *testing.T, cfg *Config, content string)
}{
{
name: "set scalar",
edit: func(file string) error { return SetValue(file, "runner.capacity", "4") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, 4, cfg.Runner.Capacity)
},
},
{
name: "set duration",
edit: func(file string) error { return SetValue(file, "runner.timeout", "90m") },
assert: func(t *testing.T, cfg *Config, content string) {
assert.Equal(t, 90*time.Minute, cfg.Runner.Timeout)
assert.Contains(t, content, "timeout: 1h30m0s")
},
},
{
name: "set a pointer field in a missing section",
edit: func(file string) error {
return SetValue(file, "container.network_create_options.enable_ipv4", "false")
},
assert: func(t *testing.T, cfg *Config, _ string) {
require.NotNil(t, cfg.Container.NetworkCreateOptions.EnableIPv4)
assert.False(t, *cfg.Container.NetworkCreateOptions.EnableIPv4)
},
},
{
name: "set map entry",
edit: func(file string) error { return SetValue(file, "runner.envs.ADDED", "yes") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, map[string]string{"EXISTING": "value", "ADDED": "yes"}, cfg.Runner.Envs)
},
},
{
name: "set replaces a list",
edit: func(file string) error { return SetValue(file, "runner.labels", "one", "two") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, []string{"one", "two"}, cfg.Runner.Labels)
},
},
{
name: "add appends to a list",
edit: func(file string) error { return AddValue(file, "runner.labels", "ubuntu:docker://node:22") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, []string{"ubuntu-latest:docker://node:20", "self-hosted", "ubuntu:docker://node:22"}, cfg.Runner.Labels)
},
},
{
name: "remove drops a list entry",
edit: func(file string) error { return RemoveValue(file, "runner.labels", "self-hosted") },
assert: func(t *testing.T, cfg *Config, _ string) {
assert.Equal(t, []string{"ubuntu-latest:docker://node:20"}, cfg.Runner.Labels)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
file := writeEditFixture(t)
require.NoError(t, tt.edit(file))
raw, err := os.ReadFile(file)
require.NoError(t, err)
content := string(raw)
cfg, err := LoadDefault(file)
require.NoError(t, err)
tt.assert(t, cfg, content)
assert.Contains(t, content, "# A leading comment.")
assert.Contains(t, content, " # The logging level.")
assert.Contains(t, content, "\n\nrunner:")
})
}
}
func TestEditValuesRejectsBadInput(t *testing.T) {
tests := []struct {
name string
edit func(file string) error
wantErr string
}{
{
name: "unknown key",
edit: func(file string) error { return SetValue(file, "runner.labl", "x") },
wantErr: `unknown config key "runner.labl"`,
},
{
name: "value is not a number",
edit: func(file string) error { return SetValue(file, "runner.capacity", "many") },
wantErr: `"many" is not a valid int`,
},
{
name: "value is not a duration",
edit: func(file string) error { return SetValue(file, "runner.timeout", "soon") },
wantErr: `"soon" is not a duration`,
},
{
name: "value is not a boolean",
edit: func(file string) error { return SetValue(file, "runner.insecure", "maybe") },
wantErr: `"maybe" is not a boolean`,
},
{
name: "set needs a single value",
edit: func(file string) error { return SetValue(file, "runner.capacity", "1", "2") },
wantErr: "takes exactly one value",
},
{
name: "set on a section",
edit: func(file string) error { return SetValue(file, "runner", "x") },
wantErr: "is a section",
},
{
name: "add on a scalar",
edit: func(file string) error { return AddValue(file, "runner.capacity", "4") },
wantErr: "is not a list",
},
{
name: "add a duplicate",
edit: func(file string) error { return AddValue(file, "runner.labels", "self-hosted") },
wantErr: `already contains "self-hosted"`,
},
{
name: "remove a missing entry",
edit: func(file string) error { return RemoveValue(file, "runner.labels", "absent") },
wantErr: `does not contain "absent"`,
},
{
name: "sub-key of a free-form map entry",
edit: func(file string) error { return SetValue(file, "runner.envs.A.B", "x") },
wantErr: "has no sub-keys",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
file := writeEditFixture(t)
err := tt.edit(file)
require.Error(t, err)
assert.Contains(t, err.Error(), tt.wantErr)
content, err := os.ReadFile(file)
require.NoError(t, err)
assert.Equal(t, editFixture, string(content), "a rejected edit must leave the file untouched")
})
}
}
func TestGetValue(t *testing.T) {
file := writeEditFixture(t)
value, err := GetValue(file, "runner.capacity")
require.NoError(t, err)
assert.Equal(t, "1", value)
value, err = GetValue(file, "runner.labels")
require.NoError(t, err)
assert.Equal(t, "ubuntu-latest:docker://node:20\nself-hosted", value)
value, err = GetValue(file, "runner.envs")
require.NoError(t, err)
assert.Equal(t, "EXISTING=value", value)
// A section has no single-line rendering.
value, err = GetValue(file, "runner")
require.NoError(t, err)
assert.Contains(t, value, "labels:\n - ubuntu-latest:docker://node:20")
_, err = GetValue(file, "metrics.addr")
require.Error(t, err)
assert.Contains(t, err.Error(), "is not set")
}
func TestEditValuesFileHandling(t *testing.T) {
t.Run("reports a missing file", func(t *testing.T) {
err := SetValue(filepath.Join(t.TempDir(), "absent.yaml"), "runner.capacity", "4")
require.Error(t, err)
assert.Contains(t, err.Error(), "does not exist")
})
t.Run("writes through a symlink", func(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "real.yaml")
link := filepath.Join(dir, "config.yaml")
require.NoError(t, os.WriteFile(target, []byte(editFixture), 0o600))
require.NoError(t, os.Symlink(target, link))
require.NoError(t, SetValue(link, "runner.capacity", "4"))
info, err := os.Lstat(link)
require.NoError(t, err)
assert.NotZero(t, info.Mode()&os.ModeSymlink, "the symlink must not be replaced by a regular file")
content, err := os.ReadFile(target)
require.NoError(t, err)
assert.Contains(t, string(content), "capacity: 4")
})
t.Run("keeps CRLF line endings", func(t *testing.T) {
file := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(file, []byte(strings.ReplaceAll(editFixture, "\n", "\r\n")), 0o600))
require.NoError(t, SetValue(file, "runner.capacity", "4"))
content, err := os.ReadFile(file)
require.NoError(t, err)
assert.Contains(t, string(content), "capacity: 4\r\n")
assert.NotContains(t, strings.ReplaceAll(string(content), "\r\n", ""), "\n")
})
}
// The example config is the file users edit, so it has to stay written the way
// the encoder emits it, down to the single space before a trailing comment.
func TestEditValuesKeepsExampleConfigIntact(t *testing.T) {
file := filepath.Join(t.TempDir(), "config.yaml")
require.NoError(t, os.WriteFile(file, Example, 0o600))
require.NoError(t, AddValue(file, "runner.labels", "ubuntu:docker://node:22"))
content, err := os.ReadFile(file)
require.NoError(t, err)
withoutAdded := strings.Replace(string(content), " - ubuntu:docker://node:22\n", "", 1)
assert.Equal(t, string(Example), withoutAdded, "only the appended label may differ")
}

View File

@@ -0,0 +1,25 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build !windows && !plan9
package config
import (
"errors"
"os"
"syscall"
)
// preserveOwner keeps a config that root edits owned by the service user it was created for.
func preserveOwner(file string, info os.FileInfo) error {
stat, ok := info.Sys().(*syscall.Stat_t)
if !ok {
return nil
}
// A caller that may replace the file but not chown it is no worse off than before.
if err := os.Chown(file, int(stat.Uid), int(stat.Gid)); err != nil && !errors.Is(err, os.ErrPermission) {
return err
}
return nil
}

View File

@@ -255,6 +255,9 @@ func (r *Reporter) Fire(entry *log.Entry) error {
if step.StartedAt == nil {
step.StartedAt = timestamppb.New(timestamp)
urgentState = true
// The runner's own handler is per step, so an unresumed ::stop-commands:: must not
// leave the reporter suppressed, and no longer registering masks, for the whole job.
r.stopCommandEndToken = ""
}
// Force reporting log errors as raw output to prevent silent failures
@@ -396,10 +399,9 @@ func (r *Reporter) Logf(format string, a ...any) {
func (r *Reporter) logf(format string, a ...any) {
if !r.duringSteps() {
r.logRows = append(r.logRows, &runnerv1.LogRow{
Time: timestamppb.Now(),
Content: fmt.Sprintf(format, a...),
})
// Masked like any other row: these bypass parseLogRow, but a caller can still
// interpolate a secret, such as a configured URL carrying credentials.
r.logRows = append(r.logRows, r.newLogRow(timestamppb.Now(), fmt.Sprintf(format, a...)))
}
}
@@ -700,66 +702,128 @@ func (r *Reporter) parseResult(result any) (runnerv1.Result, bool) {
return ret, ok
}
var cmdRegex = regexp.MustCompile(`^::([^ :]+)( .*)?::(.*)$`)
// A property value never contains a raw ':' (GitHub escapes it as %3A), so excluding ':' ends
// the property list at the first '::' as GitHub does; greedily would swallow a '::' message.
var cmdRegex = regexp.MustCompile(`^::([^ :]+)( [^:]*)?::(.*)$`)
func (r *Reporter) handleCommand(originalContent, command, value string) *string {
if r.stopCommandEndToken != "" && command != r.stopCommandEndToken {
return &originalContent
// handleCommand takes value still escaped, so that the web UI decodes it exactly once. Only
// the branches that consume the payload here decode it.
func (r *Reporter) handleCommand(originalContent, command, properties, value string) *string {
if r.stopCommandEndToken != "" {
if command != r.stopCommandEndToken {
return &originalContent
}
// Resumed here rather than from the switch, because the end token is arbitrary and a
// token naming a real command would otherwise never resume.
r.stopCommandEndToken = ""
return nil
}
switch command {
case "add-mask":
r.addMask(value)
r.addMask(runner.UnescapeCommandData(value))
return nil
case "debug":
if r.debugOutputEnabled {
return &value
return &originalContent // kept as ::debug::, so the web UI labels and decodes it
}
return nil
case "notice":
// Not implemented yet, so just return the original content.
return &originalContent
case "warning":
// Not implemented yet, so just return the original content.
return &originalContent
case "error":
// Not implemented yet, so just return the original content.
return &originalContent
case "group":
// Returning the original content, because I think the frontend
// will use it when rendering the output.
return &originalContent
case "endgroup":
// Ditto
case "notice", "warning", "error":
// Gitea has no annotation store, so the annotation is rendered into the log with
// its source location instead of being dropped: that location is the whole point
// of the command for compiler and linter output.
annotation := formatAnnotation(command, properties, value)
return &annotation
case "group", "endgroup":
// Passed through: the web UI folds the log on these and decodes the payload itself.
return &originalContent
case "stop-commands":
r.stopCommandEndToken = value
return nil
case r.stopCommandEndToken:
r.stopCommandEndToken = ""
r.stopCommandEndToken = runner.UnescapeCommandData(value)
return nil
}
return &originalContent
}
// formatAnnotation folds the file, line, column and title the command carries into its message,
// which the web UI otherwise drops along with the rest of the properties:
//
// ::error file=main.go,line=12,col=5,title=vet::undefined: x
// ::error::main.go:12:5: vet: undefined: x
//
// The ::-form prefix is deliberate, and value is not escaped here because it arrived escaped
// and must stay that way.
func formatAnnotation(level, properties, value string) string {
props := parseCommandProperties(properties)
prefix := props["file"]
if prefix != "" {
if props["line"] != "" {
prefix += ":" + props["line"]
if props["col"] != "" {
prefix += ":" + props["col"]
}
}
prefix += ": "
}
if props["title"] != "" {
prefix += props["title"] + ": "
}
return "::" + level + "::" + prefix + value
}
// parseCommandProperties parses the `file=main.go,line=12` part of a workflow command.
func parseCommandProperties(properties string) map[string]string {
properties = strings.TrimSpace(properties)
if properties == "" {
return nil
}
props := map[string]string{}
for pair := range strings.SplitSeq(properties, ",") {
key, value, ok := strings.Cut(pair, "=")
if !ok {
continue
}
// Only the property-list separators are decoded, the web UI decodes the rest.
value = strings.ReplaceAll(strings.ReplaceAll(value, "%3A", ":"), "%2C", ",")
// GitHub keys its property dictionary case-insensitively, so `File=` works there too.
props[strings.ToLower(strings.TrimSpace(key))] = value
}
// GitHub's toolkit emits `col`; accept `column` as well, which some tools write instead.
if props["col"] == "" {
props["col"] = props["column"]
}
return props
}
func (r *Reporter) parseLogRow(entry *log.Entry) *runnerv1.LogRow {
content := strings.TrimRight(entry.Message, "\r\n")
// cmdRegex only covers the ::cmd:: form, so the ##[add-mask] one would otherwise reach
// the log carrying its own secret. Registered and dropped like its ::add-mask:: twin.
if arg, ok := strings.CutPrefix(content, "##[add-mask]"); ok {
r.addMask(runner.UnescapeCommandData(arg))
return nil
}
matches := cmdRegex.FindStringSubmatch(content)
if matches != nil {
if output := r.handleCommand(content, matches[1], runner.UnescapeCommandData(matches[3])); output != nil {
if output := r.handleCommand(content, matches[1], matches[2], matches[3]); output != nil {
content = *output
} else {
return nil
}
}
content = r.logReplacer.Replace(content)
return r.newLogRow(timestamppb.New(entry.Time), content)
}
// newLogRow applies the masking and validation every row must carry, whatever built it.
func (r *Reporter) newLogRow(t *timestamppb.Timestamp, content string) *runnerv1.LogRow {
return &runnerv1.LogRow{
Time: timestamppb.New(entry.Time),
Content: strings.ToValidUTF8(content, "?"),
Time: t,
Content: strings.ToValidUTF8(r.logReplacer.Replace(content), "?"),
}
}

View File

@@ -72,9 +72,12 @@ func TestReporter_parseLogRow(t *testing.T) {
"Debug enabled", true,
[]string{
"::debug::GitHub Actions runtime token access controls",
// Left escaped: the web UI decodes it, and a real newline would not survive storage.
"::debug::first%0Asecond",
},
[]string{
"GitHub Actions runtime token access controls",
"::debug::GitHub Actions runtime token access controls",
"::debug::first%0Asecond",
},
},
{
@@ -86,31 +89,46 @@ func TestReporter_parseLogRow(t *testing.T) {
"<nil>",
},
},
// The three annotation levels share one code path, so the property shapes are only
// exercised under "error"; notice and warning just prove the level token round-trips.
{
"notice", false,
[]string{
"::notice file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::notice::Gosh, that's not going to work",
},
[]string{
"::notice file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::notice::Gosh, that's not going to work",
},
},
{
"warning", false,
[]string{
"::warning file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::warning::Gosh, that's not going to work",
},
[]string{
"::warning file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::warning::Gosh, that's not going to work",
},
},
{
"error", false,
[]string{
"::error file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::error::Gosh, that's not going to work",
"::error file=file.name,line=42,col=7::Gosh, that's not going to work",
// The message keeps its own '::', the property list ends at the first one.
"::error file=main.cpp,line=12::no member named 'foo' in 'std::vector<int>'",
// GitHub matches property names case-insensitively.
"::error File=file.name,Line=42,Col=7::Gosh, that's not going to work",
// Only the property separators are decoded here, %25/%0A are left for the web UI.
"::error file=a%3Ab.go,title=100%252C::still %25 escaped%0Aand multi-line",
},
[]string{
"::error file=file.name,line=42,endLine=48,title=Cool Title::Gosh, that's not going to work",
"::error::file.name:42: Cool Title: Gosh, that's not going to work",
"::error::Gosh, that's not going to work",
"::error::file.name:42:7: Gosh, that's not going to work",
"::error::main.cpp:12: no member named 'foo' in 'std::vector<int>'",
"::error::file.name:42:7: Gosh, that's not going to work",
"::error::a:b.go: 100%252C: still %25 escaped%0Aand multi-line",
},
},
{
@@ -149,6 +167,24 @@ func TestReporter_parseLogRow(t *testing.T) {
"*** bar baz ***",
},
},
{
// a token naming a real command must still resume
"stop-commands with a command-named token", false,
[]string{
"::stop-commands::add-mask",
"::set-output name=x::suppressed",
"::add-mask::",
"::add-mask::masked",
"masked",
},
[]string{
"<nil>",
"::set-output name=x::suppressed",
"<nil>",
"<nil>",
"***",
},
},
{
"unknown command", false,
[]string{
@@ -179,6 +215,19 @@ func TestReporter_parseLogRow(t *testing.T) {
}
}
// Both add-mask forms must register the secret and drop their own row: the runner forwards
// the raw line, so failing to consume it writes the secret straight to the job log.
func TestReporter_parseLogRowAddMask(t *testing.T) {
for _, line := range []string{"::add-mask::supersecret", "##[add-mask]supersecret"} {
r := &Reporter{logReplacer: strings.NewReplacer()}
assert.Nil(t, r.parseLogRow(&log.Entry{Message: line}), line)
row := r.parseLogRow(&log.Entry{Message: "using supersecret now"})
assert.Equal(t, "using *** now", row.Content, line)
}
}
func TestReporter_Fire(t *testing.T) {
t.Run("ignore command lines", func(t *testing.T) {
client := mocks.NewClient(t)
@@ -1013,7 +1062,7 @@ func TestReporter_Result(t *testing.T) {
}
func TestReporter_SetOutputs(t *testing.T) {
r := &Reporter{state: &runnerv1.TaskState{}}
r := &Reporter{state: &runnerv1.TaskState{}, logReplacer: strings.NewReplacer()}
r.SetOutputs(map[string]string{"foo": "bar"})
got, ok := r.outputs["foo"]