* feat: run jobs in parallel
This changes fixes and restructures the parallel execution of jobs.
The previous changes limiting the parallel execution did break this
and allowed only one job in parallel.
While we run #CPU jobs in parallel now, the jobs added per job-matrix
add to this. So we might over-commit to the capacity, but at least
it is limited.
* fix: correctly build job pipeline
The job pipeline should just append all required pipeline steps.
The parallelism will be handled by the ParallelExecutor and we
shouldn't handle it during building the pipelines.
Also this adds a test, that the ParallelExecutor does run
a limited amount of parallel goroutines.
* test: correct test implementation
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Github sets the outcome and conclusion to skipped
and this PR does the same during the pipeline run for
the StepResults in act.
Co-authored-by: Björn Brauer <[email protected]>
Co-authored-by: Björn Brauer <[email protected]>
* fix: continue jobs + steps after failure
To allow proper if expression handling on jobs and steps (like always,
success, failure, ...) we need to continue running all executors in
the prepared chain.
To keep the error handling intact we add an occurred error to the
go context and handle it later in the pipeline/chain.
Also we add the job result to the needs context to give expressions
access to it.
The needs object, failure and success functions are split between
run context (on jobs) and step context.
Closes#442
Co-authored-by: Björn Brauer <[email protected]>
* style: correct linter warnings
Co-authored-by: Björn Brauer <[email protected]>
* fix: job if value defaults to success()
As described in the documentation, a default value of "success()" is
applied when no "if" value is present on the job.
https://docs.github.com/en/actions/learn-github-actions/expressions#job-status-check-functions
Co-authored-by: Markus Wolf <[email protected]>
* fix: check job needs recursively
Ensure job result includes results of previous jobs
Co-authored-by: Markus Wolf <[email protected]>
* test: add runner test for job status check functions
Co-authored-by: Markus Wolf <[email protected]>
* test: add unit tests for run context if evaluation
Co-authored-by: Björn Brauer <[email protected]>
* refactor: move if expression evaluation
Move if expression evaluation into own function (step context) to
better support unit testing.
Co-authored-by: Björn Brauer <[email protected]>
* test: add unit tests for step context if evaluation
Co-authored-by: Markus Wolf <[email protected]>
* fix: handle job error more resilient
The job error is not stored in a context map instead of a context
added value.
Since context values are immutable an added value requires to keep
the new context in all cases. This is fragile since it might slip
unnoticed to other parts of the code.
Storing the error of a job in the context map will make it more stable,
since the map is always there and the context of the pipeline is stable
for the whole run.
* feat: steps should use a default if expression of success()
* test: add integration test for if-expressions
* chore: disable editorconfig-checker for yaml multiline string
Co-authored-by: Björn Brauer <[email protected]>
Co-authored-by: Björn Brauer <[email protected]>
We should reuse an existing context logger if available.
This will allow test to setup act with a null logger to assert
log messages.
Co-authored-by: Björn Brauer <[email protected]>
Co-authored-by: Björn Brauer <[email protected]>
* Add asset server and upload handling of binary files
Co-authored-by: Björn Brauer <[email protected]>
* Add asset download parts to the asset server
Co-authored-by: Björn Brauer <[email protected]>
* Add artifact-server-path flag
If the flag is not defined, the artifact server isn't started.
This includes the configuration of ACTIONS_RUNTIME_URL and
ACTIONS_RUNTIME_TOKEN which are set if the server is started.
Co-authored-by: Björn Brauer <[email protected]>
* Move ACTIONS_RUNTIME_* vars into the withGithubEnv setup
Co-authored-by: Björn Brauer <[email protected]>
* feat: add artifact server port as flag
This commits adds a flag to define the artifact server port.
If not given, the port defaults to 34567.
Co-authored-by: Björn Brauer <[email protected]>
* test: add artifact server tests
Co-authored-by: Björn Brauer <[email protected]>
* refactor: use fs.FS
This allows to add tests with in-memory file system
* feat: add support for gzip encoded uploads
Co-authored-by: Björn Brauer <[email protected]>
* test: add artifact integration test
* chore: run act tests with asset server path
Co-authored-by: Björn Brauer <[email protected]>
* docs: add new cli flags
Co-authored-by: Björn Brauer <[email protected]>
* test: add test workflow to testdata
* feat: add log output
* refactor: log shutdown error instead of panic
* feat: use outbound ip for the asset server
This change should allow to use the host ip in macos and windows.
Since docker is running in an intermediate vm, localhost is not
sufficient to have the artifacts in the host system.
* fix: do not use canceled context
To shutdown artifact server, we should not use the already canceled
context but the parent context instead.
Co-authored-by: Björn Brauer <[email protected]>
* feat: shutdown artifact server at end of pipeline
When the pipeline is done the asset server should be shut down
gracefully.
Co-authored-by: Björn Brauer <[email protected]>
* fix: close server if graceful shutdown failed
Co-authored-by: Björn Brauer <[email protected]>
* fix: ignore server closed error from listen call
Co-authored-by: Björn Brauer <[email protected]>
Co-authored-by: Björn Brauer <[email protected]>
Co-authored-by: Björn Brauer <[email protected]>
* fix: hide masked value
The ::add-mask:: command output logs the value to be masked.
This does expose critical information which should be hidden from
the output.
* Add test to not output secret in add-mask command
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Add custom docker registry authentication
Uses DOCKER_USERNAME and DOCKER_PASSWORD as secrets provided into
the act cli.
Closes#527
Co-authored-by: Björn Brauer <[email protected]>
* Add test to check if pull authentication is filled in
* Update debug message to be more descriptive
Co-authored-by: Ryan (hackercat) <[email protected]>
Co-authored-by: Björn Brauer <[email protected]>
Co-authored-by: Ryan (hackercat) <[email protected]>
