mirror/catalog
1
0
Fork 0
mirror of https://github.com/tektoncd/catalog.git synced 2024-11-21 05:55:35 +00:00
Code Issues Releases Activity

Add First StepAction git-clone to the catalog

Browse Source 
This PR adds a first StepAction `git-clone` to the
catalog. It is based off the Task but with samples and
tests updated.
This commit is contained in:
Chitrang Patel 2024-03-06 15:51:54 -05:00 committed by tekton-robot
parent 2d62db2482
commit 5168415df3
12 changed files with 1307 additions and 73 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
CONTRIBUTING.md
View File

@ -12,10 +12,10 @@ and the individual `CONTRIBUTING.md` files in each respective project.
PRs are welcome, and will follow PRs are welcome, and will follow
[the tektoncd pull request process](https://github.com/tektoncd/community/blob/main/process.md#pull-request-process). [the tektoncd pull request process](https://github.com/tektoncd/community/blob/main/process.md#pull-request-process).
## How to Contribute a Task or Pipeline ## How to Contribute a Task, StepAction or Pipeline
The Catalog repository is intended to serve as a location where users can find The Catalog repository is intended to serve as a location where users can find
`Task`s and `Pipeline`s that are maintained, useful and follow established `Task`s, `StepAction`s and `Pipeline`s that are maintained, useful and follow established
best practices. best practices.
The process for contributing looks like this: The process for contributing looks like this:
@ -55,7 +55,7 @@ changes.
### Guidelines ### Guidelines
When reviewing PRs that add new `Task`s or `Pipeline`s, maintainers will follow When reviewing PRs that add new `Task`s, `StepAction`s or `Pipeline`s, maintainers will follow
the following guidelines: the following guidelines:
* Submissions should be useful in real-world applications. * Submissions should be useful in real-world applications.

328
stepaction/git-clone/0.1/README.md Normal file
View File

@ -0,0 +1,328 @@
# `git-clone`
**Note: this StepAction is only compatible with Tekton Pipelines versions 0.54.0 and greater!**
**Note: this StepAction is not backwards compatible with the previous versions as it is now run as a non-root user!**
This `StepAction` has two required inputs:
1. The URL of a git repo to clone provided with the `url` param.
2. A path called `output-path` which should be provided by a `Workspace/Volume` in a Task.
The `git-clone` `StepAction` will clone a repo from the provided `url` into the
`output-path` directory. By default the repo will be cloned into the root of
your directory. You can clone into a subdirectory by setting this `StepAction`'s
`subdirectory` param. If the directory where the repo will be cloned is
already populated then by default the contents will be deleted before the
clone takes place. This behaviour can be disabled by setting the
`deleteExisting` param to `"false"`.
**Note**: The `git-clone` StepAction is run as nonroot. The files cloned on to the `output-path`
directory will end up owned by user 65532.
## Workspace-Paths
**Note**: `StepAction`s do not accept `Workspaces`. They only need the path to a directory
which can be passed in via a `param`. The directory can be provided by the `Task` via a
`Workspace`.
**Note**: This stepaction is executed in a Task as a non-root user with UID 65532 and GID 65532.
Generally, the default permissions for storage volumes are configured for the
root user. To make the volumes accessible by the non-root user, you will need
to either configure the permissions manually or set the `fsGroup` field under
`PodSecurityContext` in your TaskRun or PipelineRun.
An example PipelineRun will look like:
```yaml
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: git-clone-
spec:
pipelineRef:
name: git-clone-pipeline
podTemplate:
securityContext:
fsGroup: 65532
...
...
```
An example TaskRun will look like:
```yaml
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
name: taskrun
spec:
taskRef:
name: git-clone
podTemplate:
securityContext:
fsGroup: 65532
...
...
```
* **output**: An example workspace for this StepAction to fetch the git repository in to.
**Note**: The underlying StepAction only needs the path which can be provided via
the param as follows:
```yaml
params:
- name: output-path
value: $(workspaces.output.path)
```
* **ssh-directory**: An optional workspace to provide SSH credentials. At
minimum this should include a private key but can also include other common
files from `.ssh` including `config` and `known_hosts`. It is **strongly**
recommended that this workspace be bound to a Kubernetes `Secret`.
**Note**: The underlying StepAction only needs the path which can be provided via
the param as follows:
```yaml
params:
- name: ssh-directory-path
value: $(workspaces.ssh-directory.path)
```
* **ssl-ca-directory**: An optional workspace to provide custom CA certificates.
Like the /etc/ssl/certs path this directory can have any pem or cert files,
this uses libcurl ssl capath directive. See this SO answer here
https://stackoverflow.com/a/9880236 on how it works.
**Note**: The underlying StepAction only needs the path which can be provided via
the param as follows:
```yaml
params:
- name: ssl-ca-directory-path
value: $(workspaces.ssa-ca-directory.path)
```
* **basic-auth**: An optional workspace containing `.gitconfig` and
`.git-credentials` files. This allows username/password/access token to be
provided for basic auth.
It is **strongly** recommended that this workspace be bound to a Kubernetes
`Secret`. For details on the correct format of the files in this Workspace
see [Using basic-auth Credentials](#using-basic-auth-credentials) below.
**Note**: Settings provided as part of a `.gitconfig` file can affect the
execution of `git` in ways that conflict with the parameters of this StepAction.
For example, specifying proxy settings in `.gitconfig` could conflict with
the `httpProxy` and `httpsProxy` parameters this StepAction provides. Nothing
prevents you setting these parameters but it is not advised.
**Note**: The underlying StepAction only needs the path which can be provided via
the param as follows:
```yaml
params:
- name: basic-auth-path
value: $(workspaces.basic-auth.path)
```
## Parameters
* **output-path**: The git repo will be cloned onto this path
* **ssh-directory-path**: A .ssh directory with private key, known_hosts, config, etc.
* **basic-auth-path**: A directory path containing a .gitconfig and .git-credentials file.
* **ssl-ca-directory-path**: A directory containing CA certificates, this will be used by Git to verify the peer with when fetching or pushing over HTTPS.
* **url**: Repository URL to clone from. (_required_)
* **revision**: Revision to checkout. (branch, tag, sha, ref, etc...) (_default_: "")
* **refspec**: Refspec to fetch before checking out revision. (_default_:"")
* **submodules**: Initialize and fetch git submodules. (_default_: true)
* **depth**: Perform a shallow clone, fetching only the most recent N commits. (_default_: 1)
* **sslVerify**: Set the `http.sslVerify` global git config. Setting this to `false` is not advised unless you are sure that you trust your git remote. (_default_: true)
* **crtFileName**: If `sslVerify` is **true** and `ssl-ca-directory` workspace is given then set `crtFileName` if mounted file name is different than `ca-bundle.crt`. (_default_: "ca-bundle.crt")
* **subdirectory**: Subdirectory inside the `output` workspace to clone the repo into. (_default:_ "")
* **deleteExisting**: Clean out the contents of the destination directory if it already exists before cloning. (_default_: true)
* **httpProxy**: HTTP proxy server for non-SSL requests. (_default_: "")
* **httpsProxy**: HTTPS proxy server for SSL requests. (_default_: "")
* **noProxy**: Opt out of proxying HTTP/HTTPS requests. (_default_: "")
* **verbose**: Log the commands that are executed during `git-clone`'s operation. (_default_: true)
* **sparseCheckoutDirectories**: Which directories to match or exclude when performing a sparse checkout (_default_: "")
* **gitInitImage**: The image providing the git-init binary that this StepAction runs. (_default_: "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:TODO")
* **userHome**: The user's home directory. (_default_: "/tekton/home")
## Results
* **commit**: The precise commit SHA that was fetched by this StepAction
* **url**: The precise URL that was fetched by this StepAction
* **committer-date**: The epoch timestamp of the commit that was fetched by this StepAction
## Platforms
The StepAction can be run on `linux/amd64`, `linux/s390x`, `linux/arm64`, and `linux/ppc64le` platforms.
## Usage
If the `revision` is not provided in the param of the taskrun
then it will auto-detect the branch as specified by the `default`
in the respective git repository.
The following pipelines demonstrate usage of the git-clone StepAction:
- [Cloning a branch](./samples/git-clone-checking-out-a-branch.yaml)
- [Checking out a specific git commit](./samples/git-clone-checking-out-a-commit.yaml)
- [Checking out a git tag and using the "commit" Step Result](./samples/using-git-clone-result.yaml)
## Cloning Private Repositories
This StepAction supports fetching private repositories. There are three ways to
authenticate:
1. The simplest approach is to bind an `ssh-directory` workspace to this
StepAction. The workspace should contain private keys (e.g. `id_rsa`), `config`
and `known_hosts` files - anything you need to interact with your git remote
via SSH. It's **strongly** recommended that you use Kubernetes `Secrets` to
hold your credentials and bind to this workspace.
In a TaskRun that would look something like this:
```yaml
kind: TaskRun
spec:
workspaces:
- name: ssh-directory
secret:
secretName: my-ssh-credentials
```
And in a Pipeline and PipelineRun it would look like this:
```yaml
kind: Pipeline
spec:
workspaces:
- name: ssh-creds
# ...
tasks:
- name: fetch-source
taskSpec:
workspaces:
- name: ssh-creds
steps:
- ref:
name: git-clone
params:
- name: ssh-directory-path
value: $(workspaces.ssh-creds.path)
workspaces:
-name: ssh-creds
# ...
---
kind: PipelineRun
spec:
workspaces:
- name: ssh-creds
secret:
secretName: my-ssh-credentials
# ...
```
The `Secret` would appear the same in both cases - structured like a `.ssh`
directory:
```yaml
kind: Secret
apiVersion: v1
metadata:
name: my-ssh-credentials
data:
id_rsa: # ... base64-encoded private key ...
known_hosts: # ... base64-encoded known_hosts file ...
config: # ... base64-encoded ssh config file ...
```
Including `known_hosts` is optional but strongly recommended. Without it
the `git-clone` StepAction will blindly accept the remote server's identity.
2. Use Tekton Pipelines' built-in credentials support as [documented in
Pipelines' auth.md](https://github.com/tektoncd/pipeline/blob/master/docs/auth.md).
3. Another approach is to bind an `ssl-ca-directory` workspace to this
StepAction. The workspace should contain crt keys (e.g. `ca-bundle.crt`)files - anything you need to interact with your git remote
via custom CA . It's **strongly** recommended that you use Kubernetes `Secrets` to
hold your credentials and bind to this workspace.
In a TaskRun that would look something like this:
```yaml
kind: TaskRun
spec:
workspaces:
- name: ssl-ca-directory
secret:
secretName: my-ssl-credentials
```
And in a Pipeline and PipelineRun it would look like this:
```yaml
kind: Pipeline
spec:
workspaces:
- name: ssl-creds
# ...
tasks:
- name: fetch-source
workspaces:
- name: ssl-creds
taskSpec:
workspaces:
- name: ssl-creds
steps:
- ref:
name: git-clone
params:
- name: ssl-ca-directory-path
value: $(workspaces.ssl-creds.path)
# ...
---
kind: PipelineRun
spec:
workspaces:
- name: ssl-creds
secret:
secretName: my-ssl-credentials
# ...
```
The `Secret` would appear like below:
```yaml
kind: Secret
apiVersion: v1
metadata:
name: my-ssl-credentials
data:
ca-bundle.crt: # ... base64-encoded crt ... # If key/filename is other than ca-bundle.crt then set crtFileName param as explained under Parameters section
```
## Using basic-auth Credentials
**Note**: It is strongly advised that you use `ssh` credentials when the option
is available to you before using basic auth. You can generate a short
lived token from WebVCS platforms (Github, Gitlab, Bitbucket etc..) to be used
as a password and generally be able to use `git` as the username.
On bitbucket server the token may have a / into it so you would need
to urlquote them before in the `Secret`, see this stackoverflow answer :
https://stackoverflow.com/a/24719496
To support basic-auth this StepAction exposes an optional `basic-auth` Workspace.
The bound Workspace must contain a `.gitconfig` and `.git-credentials` file.
Any other files on this Workspace are ignored. A typical `Secret` containing
these credentials looks as follows:
```yaml
kind: Secret
apiVersion: v1
metadata:
name: my-basic-auth-secret
type: Opaque
stringData:
.gitconfig: |
[credential "https://<hostname>"]
helper = store
.git-credentials: |
https://<user>:<pass>@<hostname>
```

223
stepaction/git-clone/0.1/git-clone.yaml Normal file
View File

@ -0,0 +1,223 @@
apiVersion: tekton.dev/v1alpha1
kind: StepAction
metadata:
name: git-clone
labels:
app.kubernetes.io/version: "0.1"
annotations:
tekton.dev/pipelines.minVersion: "0.54.0"
tekton.dev/categories: Git
tekton.dev/tags: git
tekton.dev/displayName: "git clone"
tekton.dev/platforms: "linux/amd64,linux/s390x,linux/ppc64le,linux/arm64"
spec:
params:
- name: output-path
description: The git repo will be cloned onto this path
- name: ssh-directory-path
description: |
A .ssh directory with private key, known_hosts, config, etc. Copied to
the user's home before git commands are executed. Used to authenticate
with the git remote when performing the clone. We recommend providing this
path from a workspace that is bound by a Secret over other volume types.
default: "no-path"
- name: basic-auth-path
description: |
A directory path containing a .gitconfig and .git-credentials file. These
will be copied to the user's home before any git commands are run. Any
other files in this directory are ignored. It is strongly recommended
to use ssh-directory over basic-auth whenever possible and to bind a
Secret to the Workspace providing this path over other volume types.
default: "no-path"
- name: ssl-ca-directory-path
description: |
A directory containing CA certificates, this will be used by Git to
verify the peer with when fetching or pushing over HTTPS.
default: "no-path"
- name: url
description: Repository URL to clone from.
type: string
- name: revision
description: Revision to checkout. (branch, tag, sha, ref, etc...)
type: string
default: ""
- name: refspec
description: Refspec to fetch before checking out revision.
default: ""
- name: submodules
description: Initialize and fetch git submodules.
type: string
default: "true"
- name: depth
description: Perform a shallow clone, fetching only the most recent N commits.
type: string
default: "1"
- name: sslVerify
description: Set the `http.sslVerify` global git config. Setting this to `false` is not advised unless you are sure that you trust your git remote.
type: string
default: "true"
- name: crtFileName
description: file name of mounted crt using ssl-ca-directory workspace. default value is ca-bundle.crt.
type: string
default: "ca-bundle.crt"
- name: subdirectory
description: Subdirectory inside the `output` Workspace to clone the repo into.
type: string
default: ""
- name: sparseCheckoutDirectories
description: Define the directory patterns to match or exclude when performing a sparse checkout.
type: string
default: ""
- name: deleteExisting
description: Clean out the contents of the destination directory if it already exists before cloning.
type: string
default: "true"
- name: httpProxy
description: HTTP proxy server for non-SSL requests.
type: string
default: ""
- name: httpsProxy
description: HTTPS proxy server for SSL requests.
type: string
default: ""
- name: noProxy
description: Opt out of proxying HTTP/HTTPS requests.
type: string
default: ""
- name: verbose
description: Log the commands that are executed during `git-clone`'s operation.
type: string
default: "true"
- name: gitInitImage
description: The image providing the git-init binary that this StepAction runs.
type: string
default: "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.40.2"
- name: userHome
description: |
Absolute path to the user's home directory.
type: string
default: "/home/git"
results:
- name: commit
description: The precise commit SHA that was fetched by this StepAction.
- name: url
description: The precise URL that was fetched by this StepAction.
- name: committer-date
description: The epoch timestamp of the commit that was fetched by this StepAction.
image: "$(params.gitInitImage)"
env:
- name: HOME
value: "$(params.userHome)"
- name: PARAM_URL
value: $(params.url)
- name: PARAM_REVISION
value: $(params.revision)
- name: PARAM_REFSPEC
value: $(params.refspec)
- name: PARAM_SUBMODULES
value: $(params.submodules)
- name: PARAM_DEPTH
value: $(params.depth)
- name: PARAM_SSL_VERIFY
value: $(params.sslVerify)
- name: PARAM_CRT_FILENAME
value: $(params.crtFileName)
- name: PARAM_SUBDIRECTORY
value: $(params.subdirectory)
- name: PARAM_DELETE_EXISTING
value: $(params.deleteExisting)
- name: PARAM_HTTP_PROXY
value: $(params.httpProxy)
- name: PARAM_HTTPS_PROXY
value: $(params.httpsProxy)
- name: PARAM_NO_PROXY
value: $(params.noProxy)
- name: PARAM_VERBOSE
value: $(params.verbose)
- name: PARAM_SPARSE_CHECKOUT_DIRECTORIES
value: $(params.sparseCheckoutDirectories)
- name: PARAM_USER_HOME
value: $(params.userHome)
- name: PARAM_OUTPUT_PATH
value: $(params.output-path)
- name: PARAM_SSH_DIRECTORY_PATH
value: $(params.ssh-directory-path)
- name: PARAM_BASIC_AUTH_DIRECTORY_PATH
value: $(params.basic-auth-path)
- name: PARAM_SSL_CA_DIRECTORY_PATH
value: $(params.ssl-ca-directory-path)
securityContext:
runAsNonRoot: true
runAsUser: 65532
script: |
#!/usr/bin/env sh
set -eu
if [ "${PARAM_VERBOSE}" = "true" ] ; then
set -x
fi
if [ "${PARAM_BASIC_AUTH_DIRECTORY_PATH}" != "no-path" ] ; then
cp "${PARAM_BASIC_AUTH_DIRECTORY_PATH}/.git-credentials" "${PARAM_USER_HOME}/.git-credentials"
cp "${PARAM_BASIC_AUTH_DIRECTORY_PATH}/.gitconfig" "${PARAM_USER_HOME}/.gitconfig"
chmod 400 "${PARAM_USER_HOME}/.git-credentials"
chmod 400 "${PARAM_USER_HOME}/.gitconfig"
fi
if [ "${PARAM_SSH_DIRECTORY_PATH}" != "no-path" ] ; then
cp -R "${PARAM_SSH_DIRECTORY_PATH}" "${PARAM_USER_HOME}"/.ssh
chmod 700 "${PARAM_USER_HOME}"/.ssh
chmod -R 400 "${PARAM_USER_HOME}"/.ssh/*
fi
if [ "${PARAM_SSL_CA_DIRECTORY_PATH}" != "no-path" ] ; then
export GIT_SSL_CAPATH="${PARAM_SSL_CA_DIRECTORY_PATH}"
if [ "${PARAM_CRT_FILENAME}" != "" ] ; then
export GIT_SSL_CAINFO="${PARAM_SSL_CA_DIRECTORY_PATH}/${PARAM_CRT_FILENAME}"
fi
fi
CHECKOUT_DIR="${PARAM_OUTPUT_PATH}/${PARAM_SUBDIRECTORY}"
cleandir() {
# Delete any existing contents of the repo directory if it exists.
#
# We don't just "rm -rf ${CHECKOUT_DIR}" because ${CHECKOUT_DIR} might be "/"
# or the root of a mounted volume.
if [ -d "${CHECKOUT_DIR}" ] ; then
# Delete non-hidden files and directories
rm -rf "${CHECKOUT_DIR:?}"/*
# Delete files and directories starting with . but excluding ..
rm -rf "${CHECKOUT_DIR}"/.[!.]*
# Delete files and directories starting with .. plus any other character
rm -rf "${CHECKOUT_DIR}"/..?*
fi
}
if [ "${PARAM_DELETE_EXISTING}" = "true" ] ; then
cleandir || true
fi
test -z "${PARAM_HTTP_PROXY}" || export HTTP_PROXY="${PARAM_HTTP_PROXY}"
test -z "${PARAM_HTTPS_PROXY}" || export HTTPS_PROXY="${PARAM_HTTPS_PROXY}"
test -z "${PARAM_NO_PROXY}" || export NO_PROXY="${PARAM_NO_PROXY}"
git config --global --add safe.directory "${PARAM_OUTPUT_PATH}"
/ko-app/git-init \
-url="${PARAM_URL}" \
-revision="${PARAM_REVISION}" \
-refspec="${PARAM_REFSPEC}" \
-path="${CHECKOUT_DIR}" \
-sslVerify="${PARAM_SSL_VERIFY}" \
-submodules="${PARAM_SUBMODULES}" \
-depth="${PARAM_DEPTH}" \
-sparseCheckoutDirectories="${PARAM_SPARSE_CHECKOUT_DIRECTORIES}"
cd "${CHECKOUT_DIR}"
RESULT_SHA="$(git rev-parse HEAD)"
EXIT_CODE="$?"
if [ "${EXIT_CODE}" != 0 ] ; then
exit "${EXIT_CODE}"
fi
RESULT_COMMITTER_DATE="$(git log -1 --pretty=%ct)"
printf "%s" "${RESULT_COMMITTER_DATE}" > "$(step.results.committer-date.path)"
printf "%s" "${RESULT_SHA}" > "$(step.results.commit.path)"
printf "%s" "${PARAM_URL}" > "$(step.results.url.path)"

62
stepaction/git-clone/0.1/samples/git-clone-checking-out-a-branch.yaml Normal file
View File

@ -0,0 +1,62 @@
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: cat-branch-readme
spec:
description: |
cat-branch-readme takes a git repository and a branch name and
prints the README.md file from that branch. This is an example
Task demonstrating the following:
- Using the git-clone catalog StepAction to clone a branch
- Passing a cloned repo to subsequent Steps using a Workspace.
git-clone completes before we try to read from the Workspace.
- Using a volumeClaimTemplate Volume as a Workspace.
- Avoiding hard-coded paths by using a Workspace's path
variable instead.
params:
- name: repo-url
type: string
description: The git repository URL to clone from.
- name: branch-name
type: string
description: The git branch to clone.
workspaces:
- name: shared-data
description: |
This workspace will receive the cloned git repo and be accessible
in the next Step for the repo's README.md file to be read.
steps:
- name: fetch-repo
ref:
name: git-clone
params:
- name: url
value: $(params.repo-url)
- name: revision
value: $(params.branch-name)
- name: output-path
value: $(workspaces.shared-data.path)
- name: cat-readme
image: zshusers/zsh:4.3.15
script: |
#!/usr/bin/env zsh
cat $(workspaces.shared-data.path)/README.md
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-checking-out-a-branch
spec:
taskRef:
name: cat-branch-readme
podTemplate:
securityContext:
fsGroup: 65532
workspaces:
- name: shared-data
emptyDir: {}
params:
- name: repo-url
value: https://github.com/tektoncd/pipeline.git
- name: branch-name
value: release-v0.12.x

93
stepaction/git-clone/0.1/samples/git-clone-checking-out-a-commit.yaml Normal file
View File

@ -0,0 +1,93 @@
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: checking-out-a-revision
spec:
description: |
checking-out-a-revision takes a git repository and a commit SHA
and validates that cloning the revision succeeds. This is an example
Task demonstrating the following:
- Using the git-clone catalog StepAction to clone a specific commit
- Passing a cloned repo to subsequent Steps using a Workspace.
git-clone completes before we try to read from the Workspace.
- Using a volumeClaimTemplate Volume as a Workspace.
- Avoiding hard-coded paths by using a Workspace's path
variable instead.
params:
- name: repo-url
type: string
description: The git repository URL to clone from.
- name: commit
type: string
description: The git commit to fetch.
- name: user-home
type: string
description: The home directory of the user performing the git clone.
default: "/tekton/home"
- name: user-uid
type: string
description: The UID of the user performing the git clone.
default: "0"
workspaces:
- name: shared-data
description: |
This workspace will receive the cloned git repo and be passed
to the next Step for the commit to be checked.
steps:
- name: fetch-repo
ref:
name: git-clone
params:
- name: url
value: $(params.repo-url)
- name: revision
value: $(params.commit)
- name: gitInitImage
value: localhost:5000/git-init-4874978a9786b6625dd8b6ef2a21aa70:latest
- name: userHome
value: $(params.user-home)
- name: output-path
value: $(workspaces.shared-data.path)
- name: check-expectations
image: alpine/git:v2.24.3
script: |
#!/usr/bin/env sh
cd "$(workspaces.shared-data.path)"
receivedCommit="$(git rev-parse HEAD)"
if [ "$receivedCommit" != "$(params.commit)" ]; then
echo "Expected commit $(params.commit) but received $receivedCommit."
exit 1
else
echo "Received commit $receivedCommit as expected."
fi
detectedUID="$(ls -l ./README.md | awk '{ print $3 }')"
if [ "$detectedUID" != "$(params.user-uid)" ]; then
echo "Expected README UID of $(params.user-uid) but received $detectedUID."
exit 2
else
echo "Saw README with owner of $detectedUID as expected."
fi
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
generateName: git-clone-checking-out-a-commit-
spec:
taskRef:
name: checking-out-a-revision
podTemplate:
securityContext:
runAsNonRoot: true
fsGroup: 65532
workspaces:
- name: shared-data
emptyDir: {}
params:
- name: repo-url
value: https://github.com/tektoncd/pipeline.git
- name: commit
value: 301b41380e95382a18b391c2165fa3a6a3de93b0 # Tekton Task's first ever commit!
- name: user-home
value: "/home/nonroot"
- name: user-uid
value: "65532"

71
stepaction/git-clone/0.1/samples/git-clone-for-ssl-ca.yaml Normal file
View File

@ -0,0 +1,71 @@
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: cat-readme
spec:
description: |
cat-readme takes a git repository and
prints the README.md file from main branch. This is an example
Task demonstrating the following:
- Using the git-clone catalog StepAction to clone a main branch for the repo which uses custom CAs for HTTPS
- Passing a cloned repo to subsequent StepActions using a Workspace.
git-clone completes before we try to read from the Workspace.
- Using a volumeClaimTemplate Volume as a Workspace.
- Avoiding hard-coded paths by using a Workspace's path
variable instead.
params:
- name: repo-url
type: string
description: The git repository URL to clone from.
workspaces:
- name: shared-data
description: |
This workspace will receive the cloned git repo and be passed
to the next Step for the repo's README.md file to be read.
- name: ssl-ca-dir
description: |
This workspace contains CA certificates, this will be used by Git to
verify the peer with when fetching or pushing over HTTPS.
steps:
- name: fetch-repo
ref:
name: git-clone
params:
- name: url
value: $(params.repo-url)
- name: output-path
value: $(workspaces.shared-data.path)
- name: ssl-ca-directory-path
value: $(workspaces.ssl-ca-dir.path)
- name: cat-readme
image: zshusers/zsh:4.3.15
script: |
#!/usr/bin/env zsh
cat $(workspaces.source.path)/README.md
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-checking-out-a-branch
spec:
taskRef:
name: cat-branch-readme
podTemplate:
securityContext:
fsGroup: 65532
workspaces:
- name: shared-data
emptyDir: {}
- name: ssl-ca-dir
secret:
secretName: my-ssl-credentials
params:
- name: repo-url
value: https://github.com/tektoncd/pipeline.git
---
kind: Secret
apiVersion: v1
metadata:
name: my-ssl-credentials
data:
ca-bundle.crt: jdsfjshfj122w # base64-encoded crt ... If key/filename is other than ca-bundle.crt then set crtFileName param as explained under Parameters section.

63
stepaction/git-clone/0.1/samples/git-clone-sparse-checkout.yaml Normal file
View File

@ -0,0 +1,63 @@
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: sparse-checkout-list-dir
spec:
description: |
sparse-checkout-list-dir takes a git repository and a list of
directory patterns to match and lists all cloned files and directories.
This is an example task demonstrating the following:
- Using the git-clone catalog StepAction to clone a specific set of
files based on directory patterns.
- Passing a cloned repo to subsequent Steps using a Workspace.
git-clone completes before we try to read from the Workspace.
- Using a volumeClaimTemplate Volume as a Workspace.
- Avoiding hard-coded paths by using a Workspace's path
variable instead.
params:
- name: repo-url
type: string
description: The git repository URL to clone from.
- name: sparseCheckoutDirectories
type: string
description: directory patterns to clone
workspaces:
- name: shared-data
description: |
This workspace will receive the cloned git repo and be passed
to the next Step to list all cloned files and directories.
steps:
- name: fetch-repo
ref:
name: git-clone
params:
- name: url
value: $(params.repo-url)
- name: sparseCheckoutDirectories
value: $(params.sparseCheckoutDirectories)
- name: output-path
value: $(workspaces.shared-data.path)
- name: list-dirs
image: zshusers/zsh:4.3.15
script: |
#!/usr/bin/env zsh
ls -R $(workspaces.shared-data.path)/
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-sparse-checkout
spec:
taskRef:
name: sparse-checkout-list-dir
podTemplate:
securityContext:
fsGroup: 65532
workspaces:
- name: shared-data
emptyDir: {}
params:
- name: repo-url
value: https://github.com/tektoncd/pipeline.git
- name: sparseCheckoutDirectories
value: /*,!/*/,/docs/,/cmd/

70
stepaction/git-clone/0.1/samples/using-git-clone-result.yaml Normal file
View File

@ -0,0 +1,70 @@
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: validate-tag-sha
spec:
description: |
validate-tag-sha takes a git repository, tag name, and a commit SHA and
checks whether the given tag resolves to that commit. This example
Task demonstrates the following:
- How to use the git-clone catalog StepAction
- How to use the git-clone Step's "commit" Step Result from another Step.
- How to discard the contents of the git repo when it isn't needed by
passing an `emptyDir` Volume as its "output" workspace.
params:
- name: repo-url
type: string
description: The git repository URL to clone from.
- name: tag-name
type: string
description: The git tag to clone.
- name: expected-sha
type: string
description: The expected SHA to be received for the supplied revision.
workspaces:
- name: output
steps:
- name: fetch-repository
ref:
name: git-clone
params:
- name: url
value: $(params.repo-url)
- name: revision
value: $(params.tag-name)
- name: output-path
value: $(workspaces.output.path)
- name: validate-revision-sha
image: zshusers/zsh:4.3.15
env:
- name: RESULT_COMMIT
value: $(steps.fetch-repository.results.commit)
script: |
#!/usr/bin/env zsh
if [ "$(params.expected-sha)" != "${RESULT_COMMIT}" ]; then
echo "Expected revision $(params.tag-name) to have SHA $(params.expected-sha)."
exit 1
else
echo "Revision $(params.tag-name) has expected SHA $(params.expected-sha)."
fi
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
generateName: using-git-clone-result-
spec:
taskRef:
name: validate-tag-sha
podTemplate:
securityContext:
fsGroup: 65532
workspaces:
- name: output
emptyDir: {} # We don't care about the repo contents in this example, just the "commit" result
params:
- name: repo-url
value: https://github.com/tektoncd/pipeline.git
- name: tag-name
value: v0.12.1
- name: expected-sha
value: a54dd3984affab47f3018852e61a1a6f9946ecfa

300
stepaction/git-clone/0.1/tests/run.yaml Normal file
View File

@ -0,0 +1,300 @@
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-run-noargs
spec:
workspaces:
- name: output
emptyDir: {}
taskSpec:
workspaces:
- name: output
steps:
- ref:
name: git-clone
params:
- name: url
value: $(params.url)
- name: output-path
value: $(workspaces.output.path)
podTemplate:
securityContext:
fsGroup: 65532
params:
- name: url
value: https://github.com/kelseyhightower/nocode
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-run-tag
spec:
workspaces:
- name: output
emptyDir: {}
taskSpec:
workspaces:
- name: output
steps:
- ref:
name: git-clone
params:
- name: url
value: $(params.url)
- name: revision
value: $(params.revision)
- name: output-path
value: $(workspaces.output.path)
podTemplate:
securityContext:
fsGroup: 65532
params:
- name: url
value: https://github.com/kelseyhightower/nocode
- name: revision
value: 1.0.0
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-run-no-submodules
spec:
workspaces:
- name: output
emptyDir: {}
taskSpec:
workspaces:
- name: output
steps:
- ref:
name: git-clone
params:
- name: url
value: $(params.url)
- name: submodules
value: $(params.submodules)
- name: output-path
value: $(workspaces.output.path)
podTemplate:
securityContext:
fsGroup: 65532
params:
- name: url
value: https://github.com/githubtraining/example-dependency
- name: submodules
value: "false"
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-run-no-depth-2
spec:
workspaces:
- name: output
emptyDir: {}
podTemplate:
securityContext:
fsGroup: 65532
taskSpec:
workspaces:
- name: output
steps:
- ref:
name: git-clone
params:
- name: url
value: $(params.url)
- name: depth
value: $(params.depth)
- name: output-path
value: $(workspaces.output.path)
params:
- name: url
value: https://github.com/kelseyhightower/nocode
- name: depth
value: "2"
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-run-sslverify-none
spec:
workspaces:
- name: output
emptyDir: {}
taskSpec:
workspaces:
- name: output
steps:
- ref:
name: git-clone
params:
- name: url
value: $(params.url)
- name: sslVerify
value: $(params.sslVerify)
- name: output-path
value: $(workspaces.output.path)
podTemplate:
securityContext:
fsGroup: 65532
params:
- name: url
value: https://github.com/kelseyhightower/nocode
- name: sslVerify
value: "false"
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-run-ssl-cadirectory-empty
spec:
workspaces:
- name: output
emptyDir: {}
- name: ssl-ca-directory
emptyDir: {}
taskSpec:
workspaces:
- name: output
steps:
- ref:
name: git-clone
params:
- name: url
value: $(params.url)
- name: crtFileName
value: $(params.crtFileName)
- name: output-path
value: $(workspaces.output.path)
- name: ssl-ca-directory-path
value: $(workspaces.ssl-ca-directory.path)
podTemplate:
securityContext:
fsGroup: 65532
params:
- name: url
value: https://github.com/kelseyhightower/nocode
- name: crtFileName
value: ""
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-run-subdirectory
spec:
workspaces:
- name: output
emptyDir: {}
taskSpec:
workspaces:
- name: output
steps:
- ref:
name: git-clone
params:
- name: url
value: $(params.url)
- name: subdirectory
value: $(params.subdirectory)
- name: output-path
value: $(workspaces.output.path)
podTemplate:
securityContext:
fsGroup: 65532
params:
- name: url
value: https://github.com/kelseyhightower/nocode
- name: subdirectory
value: "hellomoto"
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-run-delete-existing
spec:
workspaces:
- name: output
emptyDir: {}
taskSpec:
workspaces:
- name: output
steps:
- ref:
name: git-clone
params:
- name: url
value: $(params.url)
- name: deleteExisting
value: $(params.deleteExisting)
- name: output-path
value: $(workspaces.output.path)
podTemplate:
securityContext:
fsGroup: 65532
params:
- name: url
value: https://github.com/kelseyhightower/nocode
- name: deleteExisting
value: "true"
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-run-without-verbose
spec:
workspaces:
- name: output
emptyDir: {}
taskSpec:
workspaces:
- name: output
steps:
- ref:
name: git-clone
params:
- name: url
value: $(params.url)
- name: verbose
value: $(params.verbose)
- name: output-path
value: $(workspaces.output.path)
podTemplate:
securityContext:
fsGroup: 65532
params:
- name: url
value: https://github.com/kelseyhightower/nocode
- name: verbose
value: "false"
---
apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
name: git-clone-sparse
spec:
workspaces:
- name: output
emptyDir: {}
taskSpec:
workspaces:
- name: output
steps:
- ref:
name: git-clone
params:
- name: url
value: $(params.url)
- name: sparseCheckoutDirectories
value: $(params.sparseCheckoutDirectories)
- name: output-path
value: $(workspaces.output.path)
podTemplate:
securityContext:
fsGroup: 65532
params:
- name: url
value: https://github.com/kelseyhightower/nocode
- name: sparseCheckoutDirectories
value: "CONTRIBUTING.md,STYLE.md"

100
test/e2e-common.sh
View File

@ -36,7 +36,7 @@ fi
# Default registry image value to use in this script # Default registry image value to use in this script
REGISTRY_IMAGE="registry" REGISTRY_IMAGE="registry"
# Do the tasks modifications if special PLATFORM value is specified. By default nothing happens. # Do the resources modifications if special PLATFORM value is specified. By default nothing happens.
if [[ -n ${PLATFORM} ]] && [[ -f "$(dirname $0)/$(echo ${PLATFORM}| tr / -).sh" ]]; then if [[ -n ${PLATFORM} ]] && [[ -f "$(dirname $0)/$(echo ${PLATFORM}| tr / -).sh" ]]; then
# Load script specific to platform. File name should follow the pattern "os-arch.sh", for instance "linux-s390x.sh". # Load script specific to platform. File name should follow the pattern "os-arch.sh", for instance "linux-s390x.sh".
source $(dirname $0)/$(echo ${PLATFORM}| tr / -).sh source $(dirname $0)/$(echo ${PLATFORM}| tr / -).sh
@ -113,7 +113,7 @@ function add_sidecar_secure_registry() {
--from-file=ca.crt="${TMD}"/ca.crt --from-file=ca.key="${TMD}"/ca.key --from-file=ca.crt="${TMD}"/ca.crt --from-file=ca.key="${TMD}"/ca.key
# Add a secure internal registry as sidecar # Add a secure internal registry as sidecar
kubectl create -n "${tns}" -f ${taskdir}/tests/internal-registry/internal-registry.yaml kubectl create -n "${tns}" -f ${resourcedir}/tests/internal-registry/internal-registry.yaml
} }
function add_task() { function add_task() {
@ -141,11 +141,15 @@ function install_pipeline_crd() {
latestreleaseyaml="https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml" latestreleaseyaml="https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml"
fi fi
[[ -z ${latestreleaseyaml} ]] && fail_test "Could not get latest released release.yaml" [[ -z ${latestreleaseyaml} ]] && fail_test "Could not get latest released release.yaml"
echo "Installing ${latestreleaseyaml}"
${KUBECTL_CMD} apply -f ${latestreleaseyaml} || ${KUBECTL_CMD} apply -f ${latestreleaseyaml} ||
fail_test "Build pipeline installation failed" fail_test "Build pipeline installation failed"
# Make sure thateveything is cleaned up in the current namespace. echo "Enabling StepActions on the cluster"
for res in tasks pipelines taskruns pipelineruns; do ${KUBECTL_CMD} patch cm feature-flags -n tekton-pipelines -p '{"data":{"enable-step-actions":"true"}}'
# Make sure that eveything is cleaned up in the current namespace.
for res in stepactions tasks pipelines taskruns pipelineruns; do
${KUBECTL_CMD} delete --ignore-not-found=true ${res}.tekton.dev --all ${KUBECTL_CMD} delete --ignore-not-found=true ${res}.tekton.dev --all
done done
@ -154,19 +158,19 @@ function install_pipeline_crd() {
} }
function test_yaml_can_install() { function test_yaml_can_install() {
# Validate that all the Task CRDs in this repo are valid by creating them in a NS. # Validate that all the StepAction/Task CRDs in this repo are valid by creating them in a NS.
readonly ns="task-ns" ns="task-stepaction-ns"
all_tasks="$*" all_tasks="$*"
${KUBECTL_CMD} create ns "${ns}" || true ${KUBECTL_CMD} create ns "${ns}" || true
local runtest local runtest
for runtest in ${all_tasks}; do for runtest in ${all_tasks}; do
# remove task/ from beginning # remove task/ or stepaction/ from beginning
local runtestdir=${runtest#*/} local runtestdir=${runtest#*/}
# remove /0.1/tests from end # remove /0.1/tests from end
local testname=${runtestdir%%/*} local testname=${runtestdir%%/*}
runtest=${runtest//tests} runtest=${runtest//tests}
# in case a task is being removed then it's directory # in case a stepaction/task is being removed then it's directory
# doesn't exists, so skip the test for YAML # doesn't exists, so skip the test for YAML
[ ! -d "${runtest%%/*}/${testname}" ] && continue [ ! -d "${runtest%%/*}/${testname}" ] && continue
@ -176,15 +180,15 @@ function test_yaml_can_install() {
[[ ${ignore} == "${testname}" ]] && skipit=True [[ ${ignore} == "${testname}" ]] && skipit=True
done done
# don't test the tasks which are deprecated # don't test the stepactions/tasks which are deprecated
cat ${runtest} | grep 'tekton.dev/deprecated: \"true\"' && skipit=True cat ${runtest} | grep 'tekton.dev/deprecated: \"true\"' && skipit=True
# In case if PLATFORM env variable is specified, do the tests only for matching tasks # In case if PLATFORM env variable is specified, do the tests only for matching stepactions/tasks
[[ -n ${PLATFORM} ]] && [[ $(grep "tekton.dev/platforms" ${runtest} 2>/dev/null) != *"${PLATFORM}"* ]] && skipit=True [[ -n ${PLATFORM} ]] && [[ $(grep "tekton.dev/platforms" ${runtest} 2>/dev/null) != *"${PLATFORM}"* ]] && skipit=True
[[ -n ${skipit} ]] && continue [[ -n ${skipit} ]] && continue
echo "Checking ${testname}" echo "Checking ${testname}"
${KUBECTL_CMD} -n ${ns} apply -f <(sed "s/namespace:.*/namespace: task-ns/" "${runtest}") ${KUBECTL_CMD} -n ${ns} apply -f <(sed "s/namespace:.*/namespace: task-stepaction-ns/" "${runtest}")
done done
} }
@ -192,14 +196,18 @@ function show_failure() {
local testname=$1 tns=$2 local testname=$1 tns=$2
echo "FAILED: ${testname} task has failed to comeback properly" ; echo "FAILED: ${testname} task has failed to comeback properly" ;
echo "--- Task Dump" ${KUBECTL_CMD} api-resources
${KUBECTL_CMD} get -n ${tns} task -o yaml echo "Namespace: ${tns}"
echo "--- Pipeline Dump"
${KUBECTL_CMD} get -n ${tns} pipeline -o yaml
echo "--- PipelineRun Dump"
${KUBECTL_CMD} get -n ${tns} pipelinerun -o yaml
echo "--- TaskRun Dump" echo "--- TaskRun Dump"
${KUBECTL_CMD} get -n ${tns} taskrun -o yaml ${KUBECTL_CMD} get --ignore-not-found=true -n ${tns} taskrun -o yaml
echo "--- Task Dump"
${KUBECTL_CMD} get --ignore-not-found=true -n ${tns} task -o yaml
echo "--- PipelineRun Dump"
${KUBECTL_CMD} get --ignore-not-found=true -n ${tns} pipelinerun -o yaml
echo "--- Pipeline Dump"
${KUBECTL_CMD} get --ignore-not-found=true -n ${tns} pipeline -o yaml
echo "--- StepAction Dump"
${KUBECTL_CMD} get --ignore-not-found=true -n ${tns} stepaction -o yaml
echo "--- Container Logs" echo "--- Container Logs"
for pod in $(${KUBECTL_CMD} get pod -o name -n ${tns}); do for pod in $(${KUBECTL_CMD} get pod -o name -n ${tns}); do
echo "----POD_NAME: ${pod}---" echo "----POD_NAME: ${pod}---"
@ -208,19 +216,19 @@ function show_failure() {
exit 1 exit 1
} }
function test_task_creation() { function test_resource_creation() {
local runtest local runtest
declare -A task_to_wait_for declare -A resource_to_wait_for
for runtest in $@;do for runtest in $@;do
# remove task/ from beginning # remove stepaction/ or task/ from beginning
local runtestdir=${runtest#*/} local runtestdir=${runtest#*/}
# remove /0.1/tests from end # remove /0.1/tests from end
local testname=${runtestdir%%/*} local testname=${runtestdir%%/*}
# get version of the task # get version of the task or stepaction
local version=$(basename $(basename $(dirname $runtest))) local version=$(basename $(basename $(dirname $runtest)))
# check version is in given format # check version is in given format
[[ ${version} =~ ^[0-9]+\.[0-9]+$ ]] || { echo "ERROR: version of the task is not set properly"; exit 1;} [[ ${version} =~ ^[0-9]+\.[0-9]+$ ]] || { echo "ERROR: version of the resource is not set properly"; exit 1;}
# replace . with - in version as not supported in namespace name # replace . with - in version as not supported in namespace name
version="$( echo $version | tr '.' '-' )" version="$( echo $version | tr '.' '-' )"
local tns="${testname}-${version}" local tns="${testname}-${version}"
@ -231,37 +239,37 @@ function test_task_creation() {
done done
# remove /tests from end # remove /tests from end
local taskdir=${runtest%/*} local resourcedir=${runtest%/*}
# check whether test folder exists or not inside task dir # check whether test folder exists or not inside stepaction/task dir
# if not then run the tests for next task (if any) # if not then run the tests for next stepaction/task (if any)
[ ! -d $runtest ] && skipit=True [ ! -d $runtest ] && skipit=True
ls ${taskdir}/*.yaml 2>/dev/null >/dev/null || skipit=True ls ${resourcedir}/*.yaml 2>/dev/null >/dev/null || skipit=True
cat ${taskdir}/*.yaml | grep 'tekton.dev/deprecated: \"true\"' && skipit=True cat ${resourcedir}/*.yaml | grep 'tekton.dev/deprecated: \"true\"' && skipit=True
# In case if PLATFORM env variable is specified, do the tests only for matching tasks # In case if PLATFORM env variable is specified, do the tests only for matching stepactions/tasks
[[ -n ${PLATFORM} ]] && [[ $(grep "tekton.dev/platforms" ${taskdir}/*.yaml 2>/dev/null) != *"${PLATFORM}"* ]] && skipit=True [[ -n ${PLATFORM} ]] && [[ $(grep "tekton.dev/platforms" ${resourcedir}/*.yaml 2>/dev/null) != *"${PLATFORM}"* ]] && skipit=True
[[ -n ${skipit} ]] && continue [[ -n ${skipit} ]] && continue
# In case of rerun it's fine to ignore this error # In case of rerun it's fine to ignore this error
${KUBECTL_CMD} create namespace ${tns} >/dev/null 2>/dev/null || : ${KUBECTL_CMD} create namespace ${tns} >/dev/null 2>/dev/null || :
# Install the task itself first. We can only have one YAML file # Install the stepaction/task itself first. We can only have one YAML file
yaml=$(printf ${taskdir}/*.yaml) yaml=$(printf ${resourcedir}/*.yaml)
started=$(date '+%Hh%M:%S') started=$(date '+%Hh%M:%S')
echo "${started} STARTING: ${testname}/${version} " echo "${started} STARTING: ${testname}/${version} "
# dry-run this YAML to validate and also get formatting side-effects. # dry-run this YAML to validate and also get formatting side-effects.
${KUBECTL_CMD} -n ${tns} create -f ${yaml} --dry-run=client -o yaml >${TMPF} ${KUBECTL_CMD} -n ${tns} create -f ${yaml} --dry-run=client -o yaml >${TMPF}
[[ -f ${taskdir}/tests/pre-apply-task-hook.sh ]] && source ${taskdir}/tests/pre-apply-task-hook.sh [[ -f ${resourcedir}/tests/pre-apply-task-hook.sh ]] && source ${resourcedir}/tests/pre-apply-task-hook.sh
function_exists pre-apply-task-hook && pre-apply-task-hook function_exists pre-apply-task-hook && pre-apply-task-hook
[[ -d ${taskdir}/tests/fixtures ]] && { [[ -d ${resourcedir}/tests/fixtures ]] && {
# Create a configmap to make every file under fixture # Create a configmap to make every file under fixture
# available to the sidecar. # available to the sidecar.
${KUBECTL_CMD} -n ${tns} create configmap fixtures --from-file=${taskdir}/tests/fixtures ${KUBECTL_CMD} -n ${tns} create configmap fixtures --from-file=${resourcedir}/tests/fixtures
# The task may already have a volumes section and in that case, we # The task may already have a volumes section and in that case, we
# need to append fixtures volume. # need to append fixtures volume.
if [[ -n $(grep "^[[:space:]]\{2,\}volumes:$" ${TMPF}) ]]; then if [[ -n $(grep "^[[:space:]]\{2,\}volumes:$" ${TMPF}) ]]; then
@ -284,7 +292,7 @@ EOF
env: env:
- name: CONFIG - name: CONFIG
value: | value: |
$(cat ${taskdir}/tests/fixtures/*.yaml|sed 's/^/ /') $(cat ${resourcedir}/tests/fixtures/*.yaml|sed 's/^/ /')
EOF EOF
} }
@ -297,7 +305,7 @@ EOF
# Install resource and run # Install resource and run
for yaml in ${runtest}/*.yaml;do for yaml in ${runtest}/*.yaml;do
cp ${yaml} ${TMPF} cp ${yaml} ${TMPF}
[[ -f ${taskdir}/tests/pre-apply-taskrun-hook.sh ]] && source ${taskdir}/tests/pre-apply-taskrun-hook.sh [[ -f ${resourcedir}/tests/pre-apply-taskrun-hook.sh ]] && source ${resourcedir}/tests/pre-apply-taskrun-hook.sh
function_exists pre-apply-taskrun-hook && pre-apply-taskrun-hook function_exists pre-apply-taskrun-hook && pre-apply-taskrun-hook
# Make sure we have deleted the content, this is in case of rerun # Make sure we have deleted the content, this is in case of rerun
@ -307,7 +315,7 @@ EOF
${KUBECTL_CMD} -n ${tns} create -f ${TMPF} ${KUBECTL_CMD} -n ${tns} create -f ${TMPF}
done done
task_to_wait_for["$testname/${version}"]="${tns}|$started" resource_to_wait_for["$testname/${version}"]="${tns}|$started"
done done
# I would refactor this to a function but bash limitation is too great, really need a rewrite the sooner # I would refactor this to a function but bash limitation is too great, really need a rewrite the sooner
@ -320,20 +328,20 @@ EOF
set +x set +x
while true;do while true;do
# If we have timed out then show failures of what's remaining in # If we have timed out then show failures of what's remaining in
# task_to_wait_for we assume only first one fails this # resource_to_wait_for we assume only first one fails this
[[ ${cnt} == "${maxloop}" ]] && { [[ ${cnt} == "${maxloop}" ]] && {
for testname in "${!task_to_wait_for[@]}";do for testname in "${!resource_to_wait_for[@]}";do
target_ns=${task_to_wait_for[$testname]} target_ns=${resource_to_wait_for[$testname]}
show_failure "${testname}" "${target_ns}" show_failure "${testname}" "${target_ns}"
done done
} }
[[ -z ${task_to_wait_for[*]} ]] && { [[ -z ${resource_to_wait_for[*]} ]] && {
break break
} }
for testname in "${!task_to_wait_for[@]}";do for testname in "${!resource_to_wait_for[@]}";do
target_ns=${task_to_wait_for[$testname]%|*} target_ns=${resource_to_wait_for[$testname]%|*}
started=${task_to_wait_for[$testname]#*|} started=${resource_to_wait_for[$testname]#*|}
# sometimes we don't get all_status and reason in one go so # sometimes we don't get all_status and reason in one go so
# wait until we get the reason and all_status for 5 iterations # wait until we get the reason and all_status for 5 iterations
for tektontype in pipelinerun taskrun;do for tektontype in pipelinerun taskrun;do
@ -361,7 +369,7 @@ EOF
done done
if [[ ${breakit} == True ]];then if [[ ${breakit} == True ]];then
unset task_to_wait_for[$testname] unset resource_to_wait_for[$testname]
[[ -z ${CATALOG_TEST_SKIP_CLEANUP} ]] && ${KUBECTL_CMD} delete ns ${target_ns} >/dev/null [[ -z ${CATALOG_TEST_SKIP_CLEANUP} ]] && ${KUBECTL_CMD} delete ns ${target_ns} >/dev/null
echo "${started}::$(date '+%Hh%M:%S') SUCCESS: ${testname} testrun has successfully executed" ; echo "${started}::$(date '+%Hh%M:%S') SUCCESS: ${testname} testrun has successfully executed" ;
fi fi

32
test/e2e-tests.sh
View File

@ -17,7 +17,7 @@
# Configure the number of parallel tests running at the same time, start from 0 # Configure the number of parallel tests running at the same time, start from 0
MAX_NUMBERS_OF_PARALLEL_TASKS=7 # => 8 MAX_NUMBERS_OF_PARALLEL_TASKS=7 # => 8
export RELEASE_YAML=https://github.com/tektoncd/pipeline/releases/download/v0.51.0/release.yaml export RELEASE_YAML=https://github.com/tektoncd/pipeline/releases/download/v0.57.0/release.yaml
source $(dirname $0)/../vendor/github.com/tektoncd/plumbing/scripts/e2e-tests.sh source $(dirname $0)/../vendor/github.com/tektoncd/plumbing/scripts/e2e-tests.sh
source $(dirname $0)/e2e-common.sh source $(dirname $0)/e2e-common.sh
@ -59,8 +59,8 @@ TEST_RUN_ALL_TESTS=${TEST_RUN_ALL_TESTS:-""}
set -ex set -ex
set -o pipefail set -o pipefail
all_stepactions=$(echo stepaction/*/*/tests)
all_tests=$(echo task/*/*/tests) all_tests=$(echo task/*/*/tests)
function detect_changed_e2e_test() { function detect_changed_e2e_test() {
# detect for changes in e2e tests dir # detect for changes in e2e tests dir
git --no-pager diff --name-only "${PULL_BASE_SHA}".."${PULL_PULL_SHA}"|grep "^test/[^/]*" git --no-pager diff --name-only "${PULL_BASE_SHA}".."${PULL_PULL_SHA}"|grep "^test/[^/]*"
@ -68,44 +68,50 @@ function detect_changed_e2e_test() {
[[ -z ${TEST_RUN_ALL_TESTS} ]] && [[ ! -z $(detect_changed_e2e_test) ]] && TEST_RUN_ALL_TESTS=1 [[ -z ${TEST_RUN_ALL_TESTS} ]] && [[ ! -z $(detect_changed_e2e_test) ]] && TEST_RUN_ALL_TESTS=1
function detect_new_changed_tasks() { function detect_new_changed_resources() {
# detect for changes in tests dir of the task # detect for changes in tests dir of the task
git --no-pager diff --name-only "${PULL_BASE_SHA}".."${PULL_PULL_SHA}"|grep 'task/[^\/]*/[^\/]*/tests/[^/]*'|xargs -I {} dirname {}|sed 's/\(tests\).*/\1/g' git --no-pager diff --name-only "${PULL_BASE_SHA}".."${PULL_PULL_SHA}"|grep 'task/[^\/]*/[^\/]*/tests/[^/]*'|xargs -I {} dirname {}|sed 's/\(tests\).*/\1/g'
# detect for changes in the task manifest # detect for changes in the task manifest
git --no-pager diff --name-only "${PULL_BASE_SHA}".."${PULL_PULL_SHA}"|grep 'task/[^\/]*/[^\/]*/*[^/]*.yaml'|xargs -I {} dirname {}|awk '{print $1"/tests"}' git --no-pager diff --name-only "${PULL_BASE_SHA}".."${PULL_PULL_SHA}"|grep 'task/[^\/]*/[^\/]*/*[^/]*.yaml'|xargs -I {} dirname {}|awk '{print $1"/tests"}'
# detect for changes in tests dir of the stepaction
git --no-pager diff --name-only "${PULL_BASE_SHA}".."${PULL_PULL_SHA}"|grep 'stepaction/[^\/]*/[^\/]*/tests/[^/]*'|xargs -I {} dirname {}|sed 's/\(tests\).*/\1/g'
# detect for changes in the stepaction manifest
git --no-pager diff --name-only "${PULL_BASE_SHA}".."${PULL_PULL_SHA}"|grep 'stepaction/[^\/]*/[^\/]*/*[^/]*.yaml'|xargs -I {} dirname {}|awk '{print $1"/tests"}'
} }
if [[ -z ${TEST_RUN_ALL_TESTS} ]];then if [[ -z ${TEST_RUN_ALL_TESTS} ]];then
all_tests=$(detect_new_changed_tasks|sort -u || true) all_tests=$(detect_new_changed_resources|sort -u || true)
[[ -z ${all_tests} ]] && { [[ -z ${all_tests} ]] && {
echo "No tests has been detected in this PR. exiting." echo "No tests has been detected in this PR. exiting."
success success
} }
fi fi
test_yaml_can_install "${all_stepactions}"
test_yaml_can_install "${all_tests}" test_yaml_can_install "${all_tests}"
function test_tasks { function test_resources {
local cnt=0 local cnt=0
local task_to_tests="" local resource_to_tests=""
for runtest in $@;do for runtest in $@;do
task_to_tests="${task_to_tests} ${runtest}" resource_to_tests="${resource_to_tests} ${runtest}"
if [[ ${cnt} == "${MAX_NUMBERS_OF_PARALLEL_TASKS}" ]];then if [[ ${cnt} == "${MAX_NUMBERS_OF_PARALLEL_TASKS}" ]];then
test_task_creation "${task_to_tests}" test_resource_creation "${resource_to_tests}"
cnt=0 cnt=0
task_to_tests="" resource_to_tests=""
continue continue
fi fi
cnt=$((cnt+1)) cnt=$((cnt+1))
done done
# in case if there are some remaining tasks # in case if there are some remaining resources
if [[ -n ${task_to_tests} ]];then if [[ -n ${resource_to_tests} ]];then
test_task_creation "${task_to_tests}" test_resource_creation "${resource_to_tests}"
fi fi
} }
test_tasks "${all_tests}" test_resources "${all_stepactions}"
test_resources "${all_tests}"
success success

32
test/run-test.sh
View File

@ -37,26 +37,36 @@ if [[ -z ${@} || ${1} == "-h" ]];then
This script will run a single task to help developers testing directly a This script will run a single task to help developers testing directly a
single task without sending it to CI. single task without sending it to CI.
You need to specify the task name as the first argument and the task version as You need to specify the resource kind as the first argument, resource name
the second argument. For example : as the second argument and the resource version as the second argument.
${0} git-clone 0.1 For example :
${0} task git-clone 0.1
will run the tests for the git-clone task
while
${0} stepaction git-clone 0.1
will run the tests for the git-clone stepaction.
will run the tests for git-clone
EOF EOF
exit 0 exit 0
fi fi
TASK=${1} RESOURCE=${1}
VERSION=${2} NAME=${2}
VERSION=${3}
taskdir=task/${TASK}/${VERSION} resourcedir=${RESOURCE}/${NAME}/${VERSION}
kubectl get ns ${TASK}-${VERSION//./-} >/dev/null 2>/dev/null && kubectl delete ns ${TASK}-${VERSION//./-} kubectl get ns ${RESOURCE}-${NAME}-${VERSION//./-} >/dev/null 2>/dev/null && kubectl delete ns ${RESOURCE}-${NAME}-${VERSION//./-}
if [[ ! -d ${taskdir}/tests ]];then if [[ ! -d ${resourcedir}/tests ]];then
echo "No 'tests' directory is located in ${taskdir}" echo "No 'tests' directory is located in ${resourcedir}"
exit 1 exit 1
fi fi
test_task_creation task/${TASK}/${VERSION}/tests test_resource_creation ${RESOURCE}/${NAME}/${VERSION}/tests