mirror/catalog
1
0
Fork 0
mirror of https://github.com/tektoncd/catalog.git synced 2024-11-24 06:15:46 +00:00
Code Issues Releases Activity

Add a more generic jenkins task

Browse Source 
We had previouslly the triggers-jenkins-job task, this task is more generic and
allows more operations, it currently support starting a jenkins job and getting
the log of a build. It allows as well to wait that the job had started or that
the job has finished.

With this more generic task it makes it easier to have other jenkins operations
in there.

Signed-off-by: Chmouel Boudjnah <chmouel@redhat.com>
This commit is contained in:
Chmouel Boudjnah 2020-10-13 22:29:20 +02:00 committed by tekton-robot
parent e6fcc20450
commit cd1eeae5d6
6 changed files with 544 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

113
task/jenkins/0.1/README.md Normal file
View File

@ -0,0 +1,113 @@
# Jenkins Task
The following task can be used to interact with Jenkins using the Jenkins REST API.
More details on Remote Access API can be found [here](https://www.jenkins.io/doc/book/using/remote-access-api/)
## Install the Task
```bash
kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/master/task/jenkins/0.1/jenkins.yaml
```
## Parameters
- **JENKINS_HOST_URL**: The URL on which Jenkins is running (**Required**)
- **JENKINS_SECRETS**: The name of the secret containing the username and API token for authenticating the Jenkins (_Default_: jenkins-credentials) (**Required**)
- **ARGS**: Extra arguments to add to the control script. (**Required**)
The arguments are :
1. `start`: Start a new Jenkins job.
```
start [--file-to-upload FILE_TO_UPLOAD] [--wait-started] [--wait-finished] job [job_parameters [job_parameters ...]]
positional arguments:
job - The job name
job_parameters - Optional: The parameters to add i.e: key=value
optional arguments:
--file-to-upload FILE_TO_UPLOAD
The path of the file to upload to job.
--wait-started Wether to wait for the job to be started.
--wait-finished Wether to wait for the job to be finished.
```
`log`: Get log of a jenkins build.
```
log [-h] [--output-file OUTPUT_FILE] job [build_number]
positional arguments:
job - The job name
build_number - The build number, use 'lastBuild' to get the latest build
optional arguments:
--output-file OUTPUT_FILE
The location where to save logs on the filesystem. (i.e: a workspace location)
```
## Results
- **build_number**: This will output the current jenkins build_number of the job.
## Workspaces
- **source**: In case any file needs to be provided or saved by the Jenkins Job. (_Default_: `emptyDir: {}`)
## Secrets
Secrets containing `username`,`API token` that are used in the task for making the REST API request.
```yaml
apiVersion: v1
kind: Secret
metadata:
name: jenkins-credentials
type: Opaque
stringData:
username: username
apitoken: api-token
```
## Usage
1. Start a job without parameters
```yaml
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
name: jenkins-job-run
spec:
taskRef:
name: jenkins
params:
- name: JENKINS_HOST_URL
value: "http://localhost:8080"
- name: ARGS
value: ["start", "job"]
workspaces:
- name: source
emptyDir: {}
```
1. Start job with the parameters `param=value` and wait that it finishes.
```yaml
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
name: jenkins-job-run
spec:
taskRef:
name: jenkins
params:
- name: JENKINS_HOST_URL
value: "http://localhost:8080"
- name: ARGS
value: ["start", "--wait-finished", "test", "param=value"]
workspaces:
- name: source
emptyDir: {}
```

244
task/jenkins/0.1/jenkins.yaml Normal file
View File

@ -0,0 +1,244 @@
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: jenkins
labels:
app.kubernetes.io/version: "0.1"
annotations:
tekton.dev/pipelines.minVersion: "0.12.1"
tekton.dev/tags: jenkins, build
tekton.dev/displayName: "jenkins operation"
spec:
description: >-
The following task can be used to interact with the the Jenkins REST API.
workspaces:
- name: source
description: >-
The workspace which can be used to mount files which can be
send via the API to the Jenkins job.
results:
- name: build_number
params:
- name: JENKINS_HOST_URL
type: string
description: Server URL on which Jenkins is running
- name: JENKINS_SECRETS
type: string
description: Jenkins secret containing credentials
default: jenkins-credentials
- name: ARGS
type: array
description: The argument to send to the control script, see README of this task for details.
default: ["--help"]
steps:
- name: jenkins-control
image: registry.access.redhat.com/ubi8:8.2
workingDir: $(workspaces.source.path)
args:
- $(params.ARGS)
env:
- name: BUILD_ID_RESULTS_PATH
value: $(results.build_number.path)
- name: USERNAME
valueFrom:
secretKeyRef:
name: $(params.JENKINS_SECRETS)
key: username
- name: API_TOKEN
valueFrom:
secretKeyRef:
name: $(params.JENKINS_SECRETS)
key: apitoken
script: |
#!/usr/libexec/platform-python
import argparse
import base64
import http.cookiejar
import json
import os
import sys
import time
import urllib.request
COOKIE_JAR = http.cookiejar.CookieJar()
JENKINS_URL = """$(params.JENKINS_HOST_URL)"""
USERNAME = os.getenv("USERNAME")
PASSWORD = os.getenv("API_TOKEN")
BUILD_ID_RESULTS_PATH = os.getenv("BUILD_ID_RESULTS_PATH")
def parse_args(args):
parser = argparse.ArgumentParser(
description='Tekton jenkins task control CLI', )
actions = parser.add_subparsers(title="Actions")
start = actions.add_parser("start", help="Start a Jenkins Job")
start.add_argument("--file-to-upload",
type=str,
help="The path of the file to upload to job.")
start.add_argument("--wait-started",
action='store_true',
default=False,
help="Wether to wait for the job to be started.")
start.add_argument("--wait-finished",
action='store_true',
default=False,
help="Wether to wait for the job to be finished.")
start.add_argument("job", type=str)
start.add_argument("job_parameters", nargs="*")
log = actions.add_parser("log", help="Get log of a JOB")
log.add_argument("--output-file",
type=str,
help="The location where to save logs on the filesystem.")
log.add_argument("job", type=str)
log.add_argument("build_number", nargs="?")
if len(args) == 0:
parser.print_help()
sys.exit(1)
args = parser.parse_args(args)
args.action = sys.argv[1]
return args
def build_parameters(parameters):
data = {}
for params in parameters:
if "=" in params:
key_value = params.split("=")
data[key_value[0]] = key_value[1]
if data:
data = urllib.parse.urlencode(data).encode("utf-8")
return data
def get_crumb(headers, cookiejar):
url = f"{JENKINS_URL}/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,%22:%22,//crumb)"
opener = urllib.request.build_opener(
urllib.request.HTTPCookieProcessor(cookiejar))
opener.addheaders = headers
return opener.open(url)
def build_req(url, data=None, debug=False, file_to_upload=None):
base64string = base64.b64encode(f"{USERNAME}:{PASSWORD}".encode("utf-8"))
headers = [("Authorization", f"Basic {base64string.decode('utf-8')}")]
crumb = get_crumb(headers, COOKIE_JAR).read().decode().replace(
"Jenkins-Crumb:", "")
headers.append(("Jenkins-Crumb", crumb))
# TODO(chmou):This doesn't seem to work or i can't make it to work :\
if file_to_upload:
headers.append(("Content-Type", "multipart/form-data"))
headers.append(("Content-Length", os.stat(file_to_upload).st_size))
request = urllib.request.Request(url, open(file_to_upload, "rb"))
else:
request = urllib.request.Request(url, data=data)
opener = urllib.request.build_opener(
urllib.request.HTTPCookieProcessor(COOKIE_JAR))
opener.addheaders = headers
if debug:
print(f"Requesting {url} with data={repr(data)}")
return (opener, request)
def get_job_info(job, build_number='lastBuild'):
url = f"{JENKINS_URL}/job/{job}/{build_number}/api/json?depth=0"
opener, request = build_req(url)
return json.loads(opener.open(request).read())
def wait_for_build_started(job_name, _info_current):
try:
_info = get_job_info(job_name)
except urllib.error.HTTPError as err:
if not hasattr(err, "code") or err.code != 404:
raise
_info = {'id': 0}
max_loop = 100
current = 0
print(f"Waiting for job '{job_name}' to start.")
while _info_current['id'] == _info['id']:
try:
_info = get_job_info(job_name)
except urllib.error.HTTPError as err:
if not hasattr(err, "code") or err.code != 404:
raise
if current >= max_loop:
raise Exception(
f"It took to long to wait for the job '{job_name}' to start")
current += 1
time.sleep(5)
print(f"Job '{job_name}' has started with build number: {_info['id']}")
return _info
def wait_for_build_completed(job_name, build_number):
job_info = get_job_info(job_name, build_number)
max_loop = 100
current = 0
print(f"Waiting for job '{job_name}' to be completed")
while True:
if 'building' in job_name and job_info['building']:
job_info = get_job_info(job_name)
if current >= max_loop:
raise Exception(
f"It took to long to wait for the job '{job_name}' to complete"
)
current += 1
time.sleep(5)
continue
break
print(f"Job '{job_name}' has completed: {job_info['url']}")
return job_info
def main():
args = parse_args(sys.argv[1:])
data = {}
if args.action == "log":
query_type = f"{args.build_number}/consoleText"
elif args.action == "start":
if args.job_parameters:
data = build_parameters(args.job_parameters)
query_type = "buildWithParameters"
else:
query_type = "build"
url = f"{JENKINS_URL}/job/{args.job}/{query_type}"
# We catch the error in case the job has never been started yet.
try:
_info_current = get_job_info(args.job)
except urllib.error.HTTPError:
_info_current = {'id': 0}
opener, request = build_req(url, data=data, debug=True)
with opener.open(request) as handle:
output = handle.read().decode("utf-8")
if output:
print(output)
if 'output_file' in args:
open(args.output_file, 'w').write(output)
if args.action == "start":
print(f"Job {args.job} has started.")
if args.wait_started or args.wait_finished:
_info_current = wait_for_build_started(args.job, _info_current)
if args.wait_finished:
wait_for_build_completed(args.job, _info_current['id'])
if BUILD_ID_RESULTS_PATH and _info_current['id'] != 0:
open(BUILD_ID_RESULTS_PATH, 'w').write(_info_current['id'])
if __name__ == '__main__':
main()

20
task/jenkins/0.1/samples/run.yaml Normal file
View File

@ -0,0 +1,20 @@
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
name: jenkins-run
spec:
taskRef:
name: jenkins
params:
- name: JENKINS_HOST_URL
value: http://localhost:8080
- name: JENKINS_SECRETS
value: jenkins-credentials
- name: ARGS
value:
- start
- test
- id=123
workspaces:
- name: source
emptyDir: {}

8
task/jenkins/0.1/samples/secrets.yaml Normal file
View File

@ -0,0 +1,8 @@
apiVersion: v1
kind: Secret
metadata:
name: jenkins-credentials
type: Opaque
stringData:
username: username
apitoken: api-token

98
task/jenkins/0.1/tests/pre-apply-task-hook.sh Executable file
View File

@ -0,0 +1,98 @@
#!/bin/bash
# This will create a Jenkins deployment, a jenkins service, grab the password,
# gets a 'crumb' and generate a secret out of it for our task to run against to.
cat <<EOF | kubectl apply -f- -n "${tns}"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins
spec:
selector:
matchLabels:
run: jenkins
replicas: 1
template:
metadata:
labels:
run: jenkins
spec:
containers:
- name: jenkins
image: jenkins/jenkins:lts
ports:
- containerPort: 8080
volumeMounts:
- name: jenkins-home
mountPath: /var/jenkins_home
volumes:
- name: jenkins-home
emptyDir: {}
EOF
kubectl -n "${tns}" wait --for=condition=available --timeout=600s deployment/jenkins
kubectl -n "${tns}" expose deployment jenkins --target-port=8080
set +e
lock=0
while true;do
apitoken="$(kubectl -n "${tns}" exec deployment/jenkins -- cat /var/jenkins_home/secrets/initialAdminPassword 2>/dev/null)"
[[ -n "${apitoken}" ]] && break
sleep 5
lock=$((lock+1))
if [[ "${lock}" == 60 ]];then
echo "Error waiting for jenkins to generate a password"
exit 1
fi
done
set -e
# We need to execute the script on the pods, since it's too painful with direct exec the commands
cat <<EOF>/tmp/script.sh
#!/bin/bash
set -x
cookiejar=\$(mktemp)
apitoken=\$(cat /var/jenkins_home/secrets/initialAdminPassword)
while [[ -z "\${crumb}" ]];do
crumb=\$(curl --fail -s -u "admin:\${apitoken}" --cookie-jar "\${cookiejar}" 'jenkins:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,%22:%22,//crumb)')
sleep 2
done
set -e
cat <<XXMLOF>/tmp/job.xml
<?xml version='1.1' encoding='UTF-8'?>
<project>
<actions/>
<description></description>
<keepDependencies>false</keepDependencies>
<properties>
<hudson.model.ParametersDefinitionProperty>
<parameterDefinitions>
<hudson.model.StringParameterDefinition>
<name>Word</name>
<description></description>
<defaultValue>Bird it is!</defaultValue>
<trim>false</trim>
</hudson.model.StringParameterDefinition>
</parameterDefinitions>
</hudson.model.ParametersDefinitionProperty>
</properties>
<builders>
<hudson.tasks.Shell>
<command>echo &quot;Hello \\\${Word}&quot;</command>
<configuredLocalRules/>
</hudson.tasks.Shell>
</builders>
<publishers/>
<buildWrappers/>
</project>
XXMLOF
curl -f -s -X POST -H "\${crumb}" --cookie "\${cookiejar}" -u "admin:${apitoken}" -X POST -H "Content-Type:application/xml" -d @/tmp/job.xml "jenkins:8080/createItem?name=test"
echo \${crumb}|sed 's/Jenkins-Crumb://'
EOF
tar cf - /tmp/script.sh|kubectl -n "${tns}" exec -i deployment/jenkins -- tar xf - -C /
crumb=$(kubectl -n "${tns}" exec -i deployment/jenkins -- /bin/bash /tmp/script.sh)
kubectl delete secret jenkins-credentials 2>/dev/null || true
kubectl create secret generic -n "${tns}" jenkins-credentials --from-literal=apitoken="${apitoken}" --from-literal=username="admin"

61
task/jenkins/0.1/tests/run.yaml Normal file
View File

@ -0,0 +1,61 @@
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: jenkins-run
spec:
pipelineSpec:
results:
- name: build_number
description: The run build number
value: $(tasks.start.results.build_number)
workspaces:
- name: source
tasks:
- name: start
taskRef:
name: jenkins
workspaces:
- name: source
workspace: source
params:
- name: JENKINS_HOST_URL
value: http://jenkins:8080
- name: ARGS
value: ["start", "--wait-finished", "test", "Word=World"]
- name: log
runAfter:
- start
taskRef:
name: jenkins
workspaces:
- name: source
workspace: source
params:
- name: JENKINS_HOST_URL
value: http://jenkins:8080
- name: ARGS
value: ["log", "test", "$(tasks.start.results.build_number)", "--output-file=$(workspaces.source.path)/output.log"]
- name: check
runAfter:
- log
taskSpec:
workspaces:
- name: source
steps:
- name: check-it-baby
image: registry.access.redhat.com/ubi8/ubi-minimal:8.2
script: |
ls $(workspaces.source.path)
grep "Hello World" $(workspaces.source.path)/output.log
workspaces:
- name: source
workspace: source
workspaces:
- name: source
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Mi