mirror/catalog
1
0
Fork 0
mirror of https://github.com/tektoncd/catalog.git synced 2024-11-23 06:08:46 +00:00
Code Issues Releases Activity

feat: added support for golang and python ecosystems

Browse Source 
Signed-off-by: Ilona Shishov <Ilona.Shishov@gmail.com>
This commit is contained in:
Ilona Shishov 2023-09-26 11:26:02 +03:00 committed by tekton-robot
parent 962bb64bdc
commit 4df486f198
10 changed files with 509 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
task/redhat-dependency-analytics/0.1/README.md
View File

@ -27,7 +27,7 @@ kubectl apply -f samples/workspace.yaml -n <NAMESPACE>
### 2. Secret
The redhat-dependency-analytics task uses the `EXHORT_SNYK_TOKEN` token to authenticate with Snyk (vulnerability data provider).
This Token must be saved in a secret by the name of `exhort`.<br />
This Token must be saved in a secret by the name of `exhort`.
To generate a new Snyk token please visit the following [link](https://app.snyk.io/login?utm_campaign=Code-Ready-Analytics-2020&utm_source=code_ready&code_ready=FF1B53D9-57BE-4613-96D7-1D06066C38C9).
This [sample](samples/secret.yaml) file can be referred to in order to create a secret, replace `{{ EXHORT_SNYK_TOKEN }}` with the generated Snyk token before running.
@ -43,13 +43,13 @@ kubectl apply -f samples/secret.yaml -n <NAMESPACE>
- **project-directory-path**: Path to directory within workspace where all project files are located or where project has been cloned to. `(default: project-package)`
- **output-file-path**: Path to file within workspace where the Red Hat Dependency Analytics report will be saved. `(default: redhat-dependency-analytics-report.json)`
- **image**: Image where Exhort Javascript API and required dependencies are installed. `(default: quay.io/ecosystem-appeng/exhort-javascript-api:0.7.0-alpha)`.
<br />
List of images for different ecosystem versions can be found [here](https://github.com/RHEcosystemAppEng/exhort-javascript-api/tree/main/docker-image)
## Output
The complete response of Red Hat Dependency Analytics is saved in JSON format within the workspace directory under file name defined by parameter `output-file-name`. <br />
This response provides both a summary and a comprehensive report detailing all discovered vulnerabilities. <br />
The provided response may be used by a subsequent task for decision making, such as Passing or Failing a build.
The complete response of Red Hat Dependency Analytics is saved in JSON format within the workspace directory under file name defined by parameter `output-file-name`.
This response provides both a summary and a comprehensive report detailing all discovered vulnerabilities.
The provided response may be used by a subsequent task for decision making, such as Passing or Failing a build.
In the logs, a simplified report summary will be displayed, example:
```
@ -139,7 +139,8 @@ An example PipelineRun and TaskRun are provided in the `samples` directory in or
#### For PipelineRun Example:
1. Deploy the [git-clone](https://hub.tekton.dev/tekton/task/git-clone) Tekton Task to your environment. Refer to the `git-clone` documentation for instructions on setting up the pipeline with the appropriate parameters to align with your GitHub repository.
<br >**NOTE** that the sample pipeline has been pre-configured to facilitate the cloning of public repositories in a straightforward manner. In this setup, simply providing an HTTPS URL for a public repository is adequate to ensure the functionality of the pipeline.
**NOTE** that the sample pipeline has been pre-configured to facilitate the cloning of public repositories in a straightforward manner. In this setup, simply providing an HTTPS URL for a public repository is adequate to ensure the functionality of the pipeline.
1. Deploy the pipeline with [pipeline.yaml](samples/pipeline.yaml), run:
```
@ -162,5 +163,5 @@ An example PipelineRun and TaskRun are provided in the `samples` directory in or
kubectl apply -f samples/task-run.yaml -n <NAMESPACE>
```
<small>**NOTE:** The redhat-dependency-analytics task expects to have a secret by the name of `exhort` configured with the `EXHORT_SNYK_TOKEN` key,
as well as an attached workspace with the target manifest file stored within.</small>
**NOTE:** The redhat-dependency-analytics task expects to have a secret by the name of `exhort` configured with the `EXHORT_SNYK_TOKEN` key,
as well as an attached workspace with the target manifest file stored within.

156
task/redhat-dependency-analytics/0.2/README.md Normal file
View File

@ -0,0 +1,156 @@
# `Red Hat Dependency Analytics`
**Please Note: this Task is only compatible with Tekton Pipelines versions 0.37.5 and greater!**
## Overview
The redhat-dependency-analytics task is an interface between Tekton and Red Hat Dependency Analytics (RHDA) platform.
It provides vulnerability and compliance analysis for your applications dependencies in your software supply chain.
The redhat-dependency-analytics task for Tekton Pipelines utilizes the [Exhort JavaScript API](https://github.com/RHEcosystemAppEng/exhort-javascript-api), mirroring the functionality of the [VSCode Red Hat Dependency Analytics plugin](https://marketplace.visualstudio.com/items?itemName=redhat.fabric8-analytics).
**Note: Currently this Task only supports Maven (`mvn`), Node (`npm`), Golang (`go mod`) and Python (`pip`) ecosystems, support for other ecosystems will be provided very soon.**
## Prerequisites
Prior to executing the redhat-dependency-analytics task, ensure that you have set up the two necessary components.
### Workspace
Workspace is used as a common filesystem between tasks. It provides a designated area for the input, output, and intermediate files used during the execution of the pipeline by the redhat-dependency-analytics task.
This [sample](samples/workspace.yaml) file can be referred to in order to create a workspace.
The following command can be used to create a workspace from the sample file.
```
kubectl apply -f samples/workspace.yaml -n <NAMESPACE>
```
## Parameters
- **manifest-file-path**: Path to target manifest file (ex. pom.xml, package.json, go.mod, requirements.txt) within workspace to perform analysis upon.
- **output-file-path**: Path to file within workspace where the Red Hat Dependency Analytics report will be saved. `(default: redhat-dependency-analytics-report.json)`
- **rhda-image**: Image where Exhort Javascript API and required dependencies are installed. `(default: quay.io/ecosystem-appeng/exhort-javascript-api:0.1.1-ea.26)`.
- **python-image**: Image with installed Python interpreter and associated tools (such as pip, pip3, etc.). `(default: python:3.11)`.
- **use-go-mvs-logic**: indicate whether to use the Minimal version selection (MVS) algorithm to select a set of module versions to use when building Go packages. Relevant for Go ecosystem only. `(default: false)`.
List of images for different ecosystem versions can be found [here](https://github.com/RHEcosystemAppEng/exhort-javascript-api/tree/main/docker-image)
## Output
The complete response of Red Hat Dependency Analytics is saved in JSON format within the workspace directory under file name defined by parameter `output-file-name`.
This response provides both a summary and a comprehensive report detailing all discovered vulnerabilities.
The provided response may be used by a subsequent task for decision making, such as Passing or Failing a build.
In the logs, a simplified report summary will be displayed, example:
```
Red Hat Dependency Analytics Report
==================================================
Dependencies
Total Scanned : 8
Total Direct : 7
Total Transitive : 1
Provider: Osv-nvd
Provider Status : OK
Source: Osv-nvd
Vulnerabilities
Total : 8
Direct : 8
Transitive : 0
Critical : 2
High : 4
Medium : 2
Low : 0
==================================================
Full report is saved into file: redhat-dependency-analytics-report.json
Task is completed.
```
In case of failure, please refer to the `error.log` file within workspace for more information.
## Installation
### Install task on environment using kubectl
```
kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/redhat-dependency-analytics/0.2/redhat-dependency-analytics.yaml -n <NAMESPACE>
```
### Install task on environment using tkn
```
tkn hub install task redhat-dependency-analytics -n <NAMESPACE>
```
## Platforms
The Task can be run on `linux/amd64` platform.
## Usage
You can apply the specified task to resources such as TaskRun, Pipeline, and PipelineRun using the following configuration:
```
...
...
- name: redhat-dependency-analytics
taskRef:
name: redhat-dependency-analytics
workspaces:
- name: output
workspace: output
params:
- name: manifest-file-path
value: /path/to/manifest/file/in/workspace
- name: output-file-path
value: /path/to/output/file/in/workspace
- name: rhda-image
value: your-image-name:tag
- name: python-image
value: your-image-name:tag
- name: use-go-mvs-logic
value: false
...
...
```
## Demo
An example PipelineRun and TaskRun are provided in the `samples` directory in order to demonstrate the usage of the redhat-dependency-analytics task.
### Deployment Instructions:
1. Deploy a new workspace with [workspace.yaml](samples/workspace.yaml), run:
```
kubectl apply -f samples/workspace.yaml -n <NAMESPACE>
```
1. Deploy the redhat-dependency-analytics task by utilizing the [redhat-dependency-analytics.yaml](redhat-dependency-analytics.yaml) configuration file. You can initiate it by using the following command:
```
kubectl apply -f redhat-dependency-analytics.yaml -n <NAMESPACE>
```
Alternatively, you can consult the [installation guidelines](#installation) for the task setup instructions.
#### For PipelineRun Example:
1. Deploy the [git-clone](https://hub.tekton.dev/tekton/task/git-clone) Tekton Task to your environment. Refer to the `git-clone` documentation for instructions on setting up the pipeline with the appropriate parameters to align with your GitHub repository.
**NOTE** that the sample pipeline has been pre-configured to facilitate the cloning of public repositories in a straightforward manner. In this setup, simply providing an HTTPS URL for a public repository is adequate to ensure the functionality of the pipeline.
1. Deploy the pipeline with [pipeline.yaml](samples/pipeline.yaml), run:
```
kubectl apply -f samples/pipeline.yaml -n <NAMESPACE>
```
1. In [pipeline-run.yaml](samples/pipeline-run.yaml), first replace `{{ GITHUB_URL }}` with the Github URL to the project repository where the target manifest file resides, next replace `{{ MANIFEST_FILE_PATH }}` with the path to the target manifest file within workspace (e.g., "pom.xml" or "path/to/my/project/pom.xml").
Additionally, if you are operating within a Python environment, you have the flexibility to substitute the default value of the `python-image` parameter with a base image that incorporates the specific Python version you prefer. If you are operating within a Go environment, you might prefer to use the Minimal version selection (MVS) algorithm to select a set of module versions to use when building Go packages, in that case, set `use-go-mvs-logic` parameter to true.
Finally create the pipelinerun, run:
```
kubectl apply -f samples/pipeline-run.yaml -n <NAMESPACE>
```
#### For TaskRun Example:
1. Store the target manifest file into a desired location inside workspace.
1. In [task-run.yaml](samples/task-run.yaml), replace `{{ MANIFEST_FILE_PATH }}` with the path to the target manifest file within workspace (e.g., "pom.xml" or "path/to/my/project/pom.xml"). Additionally, if you are operating within a Python environment, you have the flexibility to substitute the default value of the `python-image` parameter with a base image that incorporates the specific Python version you prefer. If you are operating within a Go environment, you might prefer to use the Minimal version selection (MVS) algorithm to select a set of module versions to use when building Go packages, in that case, set `use-go-mvs-logic` parameter to true.
Then create the taskrun, run:
```
kubectl apply -f samples/task-run.yaml -n <NAMESPACE>
```

137
task/redhat-dependency-analytics/0.2/redhat-dependency-analytics.yaml Normal file
View File

@ -0,0 +1,137 @@
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: redhat-dependency-analytics
labels:
app.kubernetes.io/version: "0.2"
annotations:
tekton.dev/categories: Security
tekton.dev/pipelines.minVersion: "0.37.5"
tekton.dev/tags: Security, Vulnenrability, CVE
tekton.dev/displayName: "Red Hat Dependency Analytics"
tekton.dev/platforms: "linux/amd64"
spec:
description: >-
The Red Hat Dependency Analytics task is an interface between Tekton and Red Hat Dependency Analytics (RHDA) platform.
It provides vulnerability and compliance analysis for application dependencies in your software supply chain.
workspaces:
- name: output
description: Volume backing this workspace is used for input/output of the task.
params:
- name: manifest-file-path
description: Path to target manifest file within workspace.
- name: output-file-path
description: Path to file within workspace, where the analysis report is saved.
default: redhat-dependency-analytics-report.json
- name: rhda-image
description: Image where Exhort Javascript API and required dependencies are installed.
default: quay.io/ecosystem-appeng/exhort-javascript-api:0.1.1-ea.26
- name: python-image
description: Image with installed Python interpreter and associated tools (such as pip, pip3, etc.).
default: python:3.11
- name: use-go-mvs-logic
description: Uses the Minimal version selection (MVS) algorithm to select a set of module versions to use when building Go packages.
default: false
sidecars:
- name: python
image: $(params.python-image)
volumeMounts:
- mountPath: /shared
name: shared-data
script: |
#!/bin/sh
# Function to handle errors
handle_error() {
cp error.log /shared/error.log
exit 1
}
# Wait for a Step to supply the sidecar with requirements file.
while [ ! -f /shared/requirements.txt ] ; do
if [ -f /shared/notPython ]; then
exit 0
fi
sleep 1
done
python -m venv /tmp 2>error.log || handle_error
/tmp/bin/pip3 install -r /shared/requirements.txt 2>error.log || handle_error
/tmp/bin/pip3 freeze --all > /shared/pip_freeze.txt 2>error.log || handle_error
SHOW_LIST=$(awk -F '==' '{print $1}' < /shared/pip_freeze.txt)
/tmp/bin/pip3 show $(echo "$SHOW_LIST") > /shared/pip_show.txt 2>error.log || handle_error
touch /shared/pythonComplete
steps:
- name: redhat-dependency-analytics
image: $(params.rhda-image)
workingDir: $(workspaces.output.path)
env:
- name: RHDA_SOURCE
value: tekton
volumeMounts:
- mountPath: /shared
name: shared-data
script: |
#!/bin/sh
# Function to handle errors
handle_error() {
# Save exit code into output file.
jq -n {} | jq --arg exit_code "1" '. + {exit_code: $exit_code}' > $OUTPUT_FILE_PATH
# Print stderr message to console
error_message=$(sed -n '/^ERROR:/p' error.log)
printf "\n[ERROR] Failed to install dependencies from requirements.txt.\n$error_message"
exit 1
}
# Set the timeout duration in seconds
timeout_duration=60
elapsed_time=0
# get task parameter values
MANIFEST_FILE_PATH="$(params.manifest-file-path)"
OUTPUT_FILE_PATH="$(params.output-file-path)"
# install dependencies for requirements.txt file
MANIFEST_FILE_BASENAME=$(basename "$MANIFEST_FILE_PATH")
if [ "$MANIFEST_FILE_BASENAME" = "requirements.txt" ]; then
cp $MANIFEST_FILE_PATH /shared/requirements.txt
# Wait for sidecar to provide dependency data.
while [ ! -f /shared/pip_show.txt ] || [ ! -f /shared/pip_freeze.txt ] || [ ! -f /shared/pythonComplete ] ; do
# Check if error occured
if [ -f /shared/error.log ]; then
cp /shared/error.log .
handle_error
fi
# Check if the timeout is reached
if [ "$elapsed_time" -ge "$timeout_duration" ]; then
echo "Timeout reached." > error.log
handle_error
fi
sleep 1
((elapsed_time++))
done
export EXHORT_PIP_FREEZE=$(cat /shared/pip_freeze.txt | base64 -w 0)
export EXHORT_PIP_SHOW=$(cat /shared/pip_show.txt | base64 -w 0)
else
touch /shared/notPython
fi
export EXHORT_GO_MVS_LOGIC_ENABLED=$(echo "$(params.use-go-mvs-logic)")
# execute RHDA
sh /rhda.sh "$MANIFEST_FILE_PATH" "$OUTPUT_FILE_PATH"
volumes:
- name: shared-data
emptyDir: {}

34
task/redhat-dependency-analytics/0.2/samples/pipeline-run.yaml Normal file
View File

@ -0,0 +1,34 @@
# PipelineRun for starting pipeline.
# Prior to running the pipeline, ensure that you have attached the Workspace and deployed the 'git-clone' task within your environment.
# Refer to https://hub.tekton.dev/tekton/task/git-clone for guidance on implementing the 'git-clone' task.
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: redhat-dependency-analytics-pipelinerun
spec:
pipelineRef:
name: redhat-dependency-analytics-pipeline
workspaces:
# Volume backing this workspace is used for input/output of the task.
- name: output
persistentvolumeclaim:
claimName: output
params:
# Replace with Github URL to project repository.
- name: github-url
value: {{GITHUB_URL}}
# Replace with path to target manifest file within workspace (e.g., "pom.xml" or "path/to/my/project/pom.xml")
- name: manifest-file-path
value: {{MANIFEST_FILE_PATH}}
- name: output-file-path
value: redhat-dependency-analytics-report.json
# Replace with base image where Exhort Javascript API and required dependencies are installed.
- name: rhda-image
value: quay.io/ecosystem-appeng/exhort-javascript-api:0.1.1-ea.26
# Replace with base image including the desired Python version.
- name: python-image
value: python:3.11
# Set to true if using Minimal version selection (MVS) algorithm for building Go packages is preferred.
- name: use-go-mvs-logic
value: false

49
task/redhat-dependency-analytics/0.2/samples/pipeline.yaml Normal file
View File

@ -0,0 +1,49 @@
# End to end pipeline.
# Prior to running the pipeline, ensure that you have attached the Workspace and deployed the 'git-clone' task within your environment.
# Refer to https://hub.tekton.dev/tekton/task/git-clone for guidance on implementing the 'git-clone' task.
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: redhat-dependency-analytics-pipeline
spec:
workspaces:
- name: output
params:
- name: github-url
- name: manifest-file-path
- name: output-file-path
- name: rhda-image
- name: python-image
- name: use-go-mvs-logic
tasks:
# git-clone-project task
- name: git-clone-project
taskRef:
name: git-clone
workspaces:
- name: output
workspace: output
params:
- name: url
value: $(params.github-url)
# redhat-dependency-analytics task
- name: redhat-dependency-analytics
taskRef:
name: redhat-dependency-analytics
runAfter:
- git-clone-project
workspaces:
- name: output
workspace: output
params:
- name: manifest-file-path
value: $(params.manifest-file-path)
- name: output-file-path
value: $(params.output-file-path)
- name: rhda-image
value: $(params.rhda-image)
- name: python-image
value: $(params.python-image)
- name: use-go-mvs-logic
value: $(params.use-go-mvs-logic)

30
task/redhat-dependency-analytics/0.2/samples/task-run.yaml Normal file
View File

@ -0,0 +1,30 @@
# Stand alone redhat-dependency-analytics TaskRun.
# Make sure that you have attached the Workspace containing the manifest file within your environment.
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
name: redhat-dependency-analytics-taskrun
spec:
taskRef:
name: redhat-dependency-analytics
workspaces:
# Volume backing this workspace is used for input/output of the task.
- name: output
persistentvolumeclaim:
claimName: output
params:
# Replace with path to target manifest file within the worspace (e.g., "pom.xml" or "path/to/my/project/pom.xml")
- name: manifest-file-path
value: {{MANIFEST_FILE_PATH}}
- name: output-file-path
value: redhat-dependency-analytics-report.json
# Replace with base image where Exhort Javascript API and required dependencies are installed.
- name: rhda-image
value: quay.io/ecosystem-appeng/exhort-javascript-api:0.1.1-ea.26
# Replace with base image including the desired Python version.
- name: python-image
value: python:3.11
# Set to true if using Minimal version selection (MVS) algorithm for building Go packages is preferred.
- name: use-go-mvs-logic
value: false

10
task/redhat-dependency-analytics/0.2/samples/workspace.yaml Normal file
View File

@ -0,0 +1,10 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: output
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi

4
task/redhat-dependency-analytics/0.2/tests/pre-apply-task-hook.sh Normal file
View File

@ -0,0 +1,4 @@
#!/bin/bash
# Add git-clone
add_task git-clone latest

11
task/redhat-dependency-analytics/0.2/tests/resources.yaml Normal file
View File

@ -0,0 +1,11 @@
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: output
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi

69
task/redhat-dependency-analytics/0.2/tests/run.yaml Normal file
View File

@ -0,0 +1,69 @@
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: redhat-dependency-analytics-pipeline
spec:
workspaces:
- name: output
params:
- name: github-url
- name: manifest-file-path
- name: output-file-path
- name: rhda-image
- name: python-image
- name: use-go-mvs-logic
tasks:
- name: git-clone-project
taskRef:
name: git-clone
workspaces:
- name: output
workspace: output
params:
- name: url
value: $(params.github-url)
- name: redhat-dependency-analytics
taskRef:
name: redhat-dependency-analytics
runAfter:
- git-clone-project
workspaces:
- name: output
workspace: output
params:
- name: manifest-file-path
value: $(params.manifest-file-path)
- name: output-file-path
value: $(params.output-file-path)
- name: rhda-image
value: $(params.rhda-image)
- name: python-image
value: $(params.python-image)
- name: use-go-mvs-logic
value: $(params.use-go-mvs-logic)
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: redhat-dependency-analytics-pipelinerun
spec:
pipelineRef:
name: redhat-dependency-analytics-pipeline
workspaces:
- name: output
persistentvolumeclaim:
claimName: output
params:
- name: github-url
value: https://github.com/githubtraining/example-maven.git
- name: manifest-file-path
value: pom.xml
- name: output-file-path
value: redhat-dependency-analytics-report.json
- name: rhda-image
value: quay.io/ecosystem-appeng/exhort-javascript-api:0.1.1-ea.26
- name: python-image
value: docker.io/library/python@sha256:61d662f6d52206ab2290af4258257b5369573b6a4bbd904896699cc909221334
- name: use-go-mvs-logic
value: false