Trigger multi Jenkins jobs in parallel – Pipeline project

This post helps you understand how to parallelize multiple Jenkins jobs in a single run. If you need a basic understanding of how pipeline Jenkins job works, please follow this post

Approach #1

• All the stages declared under a parallel block will be executed in parallel; let’s say,

parallel {
    // the below 3 stages execute at the same time
    stage('Parallel Test 1') {
        build(job: "jenkins_job_1")
    }
    stage('Parallel Test 2') {
        // the below jobs run sequentially one after the other
        build(job: "jenkins_job_1")
        build(job: "jenkins_job_2")
        build(job: "jenkins_job_3")
    }
    stage('Parallel Test 3') {
        echo "executing at last"
    }
}

• The below image depicts how the multiple Jenkins jobs were triggered at the same time

• Copy the below example Jenkinsfile for practice

	pipeline {
	agent any
	stages {
	stage('Non-Parallel Stage') {
	steps {
	echo "Executing this stage first"
	}
	}
	stage('Parallel Stage') {
	parallel {
	stage('Parallel Test 1') {
	steps {
	echo "Here trigger job: jenkins_job_1. Triggered at time:"
	sh(script: "date -u")
	build(job: "jenkins_job_1")
	}
	}
	stage('Parallel Test 2') {
	steps {
	echo "Here trigger job: jenkins_job_2. Triggered at time:"
	sh(script: "date -u")
	build(job: "jenkins_job_2")
	echo "Here trigger job: jenkins_job_3. Triggered at time:"
	sh(script: """date -u""")
	build(job: "jenkins_job_3")
	}
	}
	}
	}
	}
	}

view raw

pipeline_parallel

hosted with ❤ by GitHub

• Blue Ocean view after completion of the pipeline job

 

Approach #2

In this approach, we will see how to group multiple jobs and run them in parallel (this is the only extended approach for parallelization in Jenkins pipeline project)

• Custom syntax to group multiple jobs to run in parallel

stages {
    stage('single run') {
        parallel {
            stage('Parallel Test 1') {
                steps {
                    script {
                        def group1 = [:]
                        group1['test_1'] = {
                            build(job: 'jenkins_job_1')
                        }
                        group1['test_2'] = {
                            build(job: 'jenkins_job_2')
                        }
                        parallel group1
                    }
                }
            }
            stage('Parallel Test 2') {
                steps {
                    script {
                        def group2 = [:]
                        group2['test_3'] = {
                            build(job: 'jenkins_job_3')
                        }
                        group2['test_4'] = {
                            build(job: 'jenkins_job_4')
                        }
                        parallel group2
                    }
                }
            }
        }
    }
}

• Copy the below example Jenkinsfile for practice

	pipeline {
	agent any
	stages {
	stage('single run') {
	parallel {
	stage('Parallel Test 1') {
	steps {
	script {
	def group1 = [:]
	group1["test_1"] = {
	echo "test_1"
	sh(script: "date -u")
	build(job: 'jenkins_job_1')
	}
	group1["test_2"] = {
	echo "test_2"
	sh(script: "date -u")
	build(job: 'jenkins_job_2')
	}
	parallel group1
	}
	}
	}
	stage('Parallel Test 2') {
	steps {
	script {
	def group2 = [:]
	group2["test_3"] = {
	echo "test_3"
	sh(script: "date -u")
	build(job: 'jenkins_job_3')
	}
	group2["test_4"] = {
	echo "test_4"
	sh(script: "date -u")
	build(job: 'jenkins_job_4')
	}
	parallel group2
	}
	}
	}
	}
	}
	}
	}

view raw

pipeline_parallel_groups

hosted with ❤ by GitHub

• Blue Ocean view after completion of the pipeline job

Jenkins Pipeline project

• Create a Jenkins Pipeline job

• Select Pipeline script from SCM in the Pipeline section
• Mention Jenkinsfile with the steps involved for pipeline project

• Make sure the Jenkinsfile in the project repo

pipeline {

    agent any
    stages {
        stage('Build') {
            steps {
                sh 'pip install -r requirements.txt'
            }
        }
        
        stage('Test') {
            steps {
                echo 'Stage 2'
                sh 'behave -f allure_behave.formatter:AllureFormatter -o allure-results features/scenarios/**/*.feature'
            }
        }
        
        stage('Publish reports') {
           steps {
                script {
                    allure([
                        includeProperties: false,
                        jdk: '',
                        properties: [],
                        reportBuildPolicy: 'ALWAYS',
                        results: [[path: 'allure-results']]
                    ])
                }
            }
        }
        
    }
}

• Checkout dashboard after the job completion

• Here is the Blue Ocean view