In SnapLogic development, automating the build, release, and deployment of Snap Packs is crucial for efficient software delivery. In this blog post, we’ll walk you through a GitLab CI/CD pipeline setup that automates the process of building and deploying a SnapLogic Snap Pack to a specific organization and project within a SnapLogic space. We’ll break down each pipeline stage and explain the steps involved.
Overview of Pipeline Stages
The pipeline consists of the following stages:
- Build – This stage compiles the code and packages the application into a deployable artifact.
- Release – In this stage, we manage versioning, create Git tags, upload the artifact to the GitLab package registry, and generate release notes.
- Deploy – This final stage deploys the application to a SnapLogic instance. The deployment in our case is triggered manually on the Gitlab UI but can be modified for your use case.
The pipeline is written in GitLab CI/CD YAML syntax. We are using a self-hosted custom GitLab runner on a VM. We’ll break down each pipeline stage and explain the steps involved.
1. Build Stage – Compiling and Packaging the Application
The build job is the first step in the CI/CD pipeline, and its goal is to compile the code and package the application.
YAML Configuration for Build Job:
build_job:
stage: build
image: maven:3.8.8-jdk-11
script:
- mvn clean package
artifacts:
paths:
- target/*.zip Key Details:
- Maven Command – The command mvn clean package removes any existing compiled artifacts and packages the application.
- Artifacts – The .zip file generated from the mvn package step is stored as an artifact. This file will be used later in the release and deploy stages.
2. Release Stage – Tagging, Uploading Artifacts, and Creating Releases
The release job is responsible for managing versioning, creating Git tags, uploading the artifact to the GitLab package registry, and creating a GitLab release, making it easy to keep track of clean, separate versions of the same Snap Pack.
YAML Configuration for Release Job:
release_job:
stage: release
dependencies:
- build_job
script:
- |
# Extract version and artifactId from pom.xml
VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
ARTIFACT_ID=$(mvn help:evaluate -Dexpression=project.artifactId -q -DforceStdout)
RELEASE_NAME="${ARTIFACT_ID}-${VERSION}"
TAG_NAME="v${VERSION}"
ZIP_FILE="target/${ARTIFACT_ID}-${VERSION}.zip"
echo "Release Name: ${RELEASE_NAME}"
echo "ZIP File: ${ZIP_FILE}"
echo "Tag Name: ${TAG_NAME}"
# Extract commit author information to set the correct git user
AUTHOR_EMAIL=$(git log -1 --pretty=format:'%ae')
AUTHOR_NAME=$(git log -1 --pretty=format:'%an')
git config --global user.email "${AUTHOR_EMAIL}"
git config --global user.name "${AUTHOR_NAME}"
# Check if the tag already exists, and if so, exit the pipeline
if git rev-parse "${TAG_NAME}" >/dev/null 2>&1; then
echo "Tag ${TAG_NAME} already exists. Exiting..."
exit 1
fi
# Create and push the tag
git tag -a "${TAG_NAME}" -m "Release ${RELEASE_NAME}"
git push https://gitlab-ci-token:${GITLAB_TAG_TOKEN}@${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git ${TAG_NAME}
# Here we upload the zip file to GitLab Package Registry
PACKAGE_URL="${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/${ARTIFACT_ID}/${VERSION}/${ARTIFACT_ID}-${VERSION}.zip"
echo "Uploading ${ZIP_FILE} to ${PACKAGE_URL}"
UPLOAD_RESPONSE=$(curl -k --header "JOB-TOKEN: ${CI_JOB_TOKEN}" \
--upload-file "${ZIP_FILE}" \
"${PACKAGE_URL}")
echo "Upload response: ${UPLOAD_RESPONSE}"
# Create the release in GitLab
echo "Creating release..."
RELEASE_RESPONSE=$(curl -k --header "JOB-TOKEN: ${CI_JOB_TOKEN}" \
--header "Content-Type: application/json" \
--data @- \
"${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/releases" << EOF
{
"name": "${RELEASE_NAME}",
"tag_name": "${TAG_NAME}",
"description": "Release ${RELEASE_NAME}",
"assets": {
"links": [
{
"name": "${RELEASE_NAME}.zip",
"url": "${PACKAGE_URL}"
}
]
}
}
EOF
)
echo "Release creation response: ${RELEASE_RESPONSE}"
# Check for errors during the release creation process
if echo "${RELEASE_RESPONSE}" | jq -e '.message' > /dev/null; then
echo "Error creating release: $(echo ${RELEASE_RESPONSE} | jq -r '.message')"
exit 1
else
echo "Release created successfully"
fi
only:
- main
Key Details:
- Version and Artifact ID – We extract the version and artifactId from the pom.xml file to use them in the release name, tag, and artifact.
- Git Tagging – We check if the version tag already exists. If not, we create and push the tag to GitLab.
- Artifact Upload – The .zip file is uploaded to the GitLab package registry, making it available for download.
- Release Creation – We create a release on GitLab, attaching the .zip artifact to the release.
- Conditional Execution – The job only runs for commits pushed to main branch.
3. Deploy Stage – Deploying to SnapLogic
The final stage of the pipeline is the deploy job, which is responsible for deploying the artifact to SnapLogic. This job is triggered manually, providing flexibility to control when deployment happens. For the purposes of our use case, we are triggering the https://cdn.elastic.snaplogic.com/api/1/rest/admin/snappack/binary/${ZIP_NAME?duplicate_check=False endpoint but note that this can be different for your use case. The duplicate_check query parameter signifies that the deployment should overwrite the Snap Pack if it already exists in that organization and project. Of course, this can be removed if we want the opposite scenario. Additionally, we are using the credentials needed for signing in to SnapLogic from the Gitlab Runner environment specified directly on the server where this runner is configured. These values could also be stored as Gitlab Secrets and used in the pipeline.
YAML Configuration for Deploy Job:
deploy_job:
stage: deploy
dependencies:
- build_job # Ensure build job completes before deploying
script:
- |
# Extract version and artifactId
VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
ARTIFACT_ID=$(mvn help:evaluate -Dexpression=project.artifactId -q -DforceStdout)
ZIP_FILE="target/${ARTIFACT_ID}-${VERSION}.zip"
ZIP_NAME=$(basename "${ZIP_FILE}")
# Encode credentials for SnapLogic
AUTH_HEADER=$(echo -n "${SL_USER}:${SL_PASSWORD}" | base64)
# Deploy to SnapLogic
echo "Deploying to SnapLogic..."
DEPLOY_RESPONSE=$(curl --location \
'https://cdn.elastic.snaplogic.com/api/1/rest/admin/snappack/binary/${ZIP_NAME}?duplicate_check=False ' \
--header "accept: application/json, text/javascript, */*; q=0.01" \
--header "accept-language: en-US,en;q=0.9" \
--header "Authorization: Basic ${AUTH_HEADER}" \
--form '_method="put"' \
--form 'path="/YOUR-ORGANIZATION/projects/YOUR-PROJECT"' \
--form 'org_path="/ORGANIZATION-PATH"' \
--form "file=@${ZIP_FILE}")
echo "SnapLogic Deploy Response: ${DEPLOY_RESPONSE}"
when: manual
only:
- main
Key Details:
- Manual Trigger – The deploy job has the “when”: manual directive, meaning it will only be executed when triggered manually via the GitLab UI. Of course, you can modify this entry to automate the deployment to your SnapLogic organization fully.
- Deployment to SnapLogic – The .zip file is uploaded to SnapLogic using the SnapLogic API with basic authentication.
Conclusion
This GitLab CI/CD pipeline automates the process of building, releasing, and deploying custom Snap Pack to SnapLogic. The build and release stages are fully automated, while the deploy stage is triggered manually, providing a controlled and flexible approach to test and production deployments. This setup can easily be adapted to other systems, repositories, or environments, and it helps improve the efficiency of your development cycle.
By using GitLab CI/CD, teams can ensure that their applications are consistently built, versioned, and deployed with minimal human intervention, allowing faster iterations and more reliable software delivery.
Authors: Petar Popovski, Bojan Velevski and Bojan Manchev
