Setup an Automated Build using Jenkins and Artifactory

Description

Such environment setup facilitates continuous integration (CI) and continuous delivery (CD), which improves productivity across your development ecosystem, by automatically:

building modules when source code changes
saving build results
reproducing builds
archiving binary modules

This guide should take 1 hour to complete.

Intended Audience

The audience for this document is engineers who are in charge of integrating MicroEJ Module Manager (MMM) to their continuous integration environment.

In addition, this document should be of interest to all developers wishing to understand how MicroEJ works with headless module builds.

For those who are only interested in command line module build, consider using the MMM Command Line Interface.

Prerequisites

MICROEJ SDK 5 5.8.1 or higher.
Docker and Docker Compose V2 on Linux, Windows or Mac
Git 2.x installed, with Git executable in path. We recommend installing Git Bash if your operating system is Windows (https://gitforwindows.org/).

This training was tested with Jenkins 2.426.1, Artifactory 7.71.5 and Gitea 1.21.1.

Note

For SDK versions before 5.4.0, please refer to this MicroEJ Documentation Archive.

Introduction

The overall build and deployment flow of a module can be summarized as follows:

Some event triggers the build process (i.e module source changed, user action, scheduled routine, etc.)
The module source code is retrieved from the Source Control System
The module dependencies are imported from the Repository Manager
The Automation Server then proceeds to building the module
If the build is successful, the module binary is deployed to the Repository Manager

Overview

The next sections describe step by step how to setup the build environment and build your first MicroEJ module.

The steps to follow are:

Run and setup Jenkins, Artifactory and Gitea
Create a simple MicroEJ module (Hello World)
Create a new Jenkins job for the Hello World module
Build the module

In order to simplify the steps, this training will be performed locally on a single machine.

Artifactory will host MicroEJ modules in 3 repositories:

microej-module-repository: repository initialized with pre-built MicroEJ modules, a mirror of the Central Repository
custom-modules-snapshot: repository where custom snapshot modules will be published
custom-modules-release: repository where custom release modules will be published

Prepare your Docker environment

This section assumes the prerequisites have been properly installed.

Create a new directory, inside create a file named docker-compose.yaml and copy this content:

version: '3'
services:
  artifactory:
    image: releases-docker.jfrog.io/jfrog/artifactory-oss:7.71.5
    container_name: artifactory
    environment:
      - JF_ROUTER_ENTRYPOINTS_EXTERNALPORT=8082
    ports:
      - 8082:8082 # for router communication
      - 8081:8081 # for artifactory communication
      - 8085:8085 # for artifactory federation communication
    volumes:
      - artifactory:/var/opt/jfrog/artifactory
      - /etc/localtime:/etc/localtime:ro
    restart: always
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "10"
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000

  gitea:
    image: gitea/gitea:1.21.1
    container_name: gitea
    environment:
      - USER_UID=1000
      - USER_GID=1000
    restart: always
    volumes:
      - gitea:/data
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "3000:3000"
      - "222:22"

  jenkins:
    image: jenkins_master
    container_name: jenkins
    build:
      dockerfile: Dockerfile
    restart: always
    ports:
      - 50000:50000
      - 8080:8080
    volumes:
      - jenkins:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
    links:
      - gitea
      - artifactory

volumes:
  gitea:
  artifactory:
  jenkins:

Create another file named Dockerfile and copy this content:

FROM jenkins/jenkins:2.426.1-lts
USER root
RUN apt-get update -qq \
    && apt-get install -qqy apt-transport-https ca-certificates curl gnupg2 software-properties-common
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
RUN add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/debian \
   $(lsb_release -cs) \
   stable"
RUN apt-get update  -qq \
    && apt-get -y install docker-ce
RUN usermod -aG docker jenkins

In this directory, launch the command docker compose up -d. After a few moments you should have three running containers (named jenkins, gitea and artifactory).

Using docker compose ps will show if containers started properly. Logs can be viewed with docker compose logs.

Get a Module Repository

A Module Repository is a portable ZIP file that bundles a set of modules for extending the MicroEJ development environment. Please consult the Module Repository section for more information.

This training uses the MicroEJ Central Repository, which is the Module Repository used by MICROEJ SDK to fetch dependencies when starting an empty workspace. It bundles Foundation Library APIs and numerous Add-On Libraries.

Next step is to download a local copy of this repository:

Visit the Central Repository on the MicroEJ Developer website.
Navigate to the Production Setup section.
Click on the offline repository link. This will download the Central Repository as a ZIP file.

Setup Artifactory

Configure Artifactory

For demonstration purposes we will allow anonymous users to deploy modules in the repositories:

Once Artifactory container is started, go to http://localhost:8082/.
Login to Artifactory for the first time using the default admin account (Username: admin, Password: password).
Skip the installation wizard if it appears.
Go to Administration > User Management > Settings.
In the User Security Configuration section, check Allow Anonymous Access.
Click on Save.
Go to Administration > User Management > Permissions.
Click on Anything entry (do not check the line), then go to Users tab
Click on anonymous and check Deploy/Cache permission in the Selected Users Repositories category.
Click on Save.

Next steps will involve uploading large files, so we have to increase the file upload maximum size accordingly:

Go to Administration > Artifactory > General > Settings.
In the General Settings section, change the value of File Upload In UI Max Size (MB) to 1024 then click on Save.

Create Repositories

We will now create and configure the repositories. Let’s start with the repository for the future built snapshot modules:

Go to Administration > Repositories > Repositories in the left menu.
Click on Add Repositories > Local Repository
Select Maven.
Set Repository Key field to custom-modules-snapshot and click on Create Local Repository.

Repeat the same steps for the other repositories with the Repository Key field set to custom-modules-release and microej-module-repository.

Import MicroEJ Repositories

In this section, we will import MicroEJ repositories into Artifactory repositories to make them available to the build server.

Go to Administration > Artifactory > Import & Export > Repositories.
Scroll to the Import Repository from Zip section.
As Target Local Repository, select microej-module-repository in the list.
Click on Select file and select the MicroEJ module repository zip file (central-repository-[version].zip) that you downloaded earlier (please refer to section Get a Module Repository).
Click Upload. At the end of upload, click on Import. Upload and import may take some time.

Artifactory is now hosting all required MicroEJ modules. Go to Application > Artifactory > Artifacts and check that the repository microej-module-repository does contain modules as shown in the figure below.

../_images/tuto_microej_cli_artifactory_module_preview.PNG

Setup Gitea

Install Gitea

Once the Gitea container is started, go to http://localhost:3000/.
Don’t change anything on the Initial Configuration, click on Install Gitea
Click on Register account and create one. The first created user become the administrator.

Configure Gitea

At the top right click on the arrow then New Repository
As Repository Name set helloworld, leave the other options as default.
Click Create Repository.

Setup Jenkins

Install Jenkins

Once Jenkins container is started, go to http://localhost:8080/.
To unlock Jenkins, copy/paste the generated password that has been written in the container log. Click on Continue.
Select option Install suggested plugins and wait for plugins installation.
Fill in the Create First Admin User form. Click Save and continue.
Click on Save and finish, then on Start using Jenkins.

Configure Jenkins

Go to Manage Jenkins > Plugins.
Add Docker Pipeline plugin:
1. Go to Available plugins section.
2. Search Docker Pipeline.
3. Install it and restart Jenkins

Build a new Module using Jenkins

Since your environment is now setup, it is time to build your first module from Jenkins and check it has been published to Artifactory. Let’s build an “Hello World” Sandboxed Application project.

Create a new MicroEJ Module

In this example, we will create a very simple module using the Sandbox Application buildtype (build-application) that we’ll push to a Git repository.

Note

For demonstration purposes, we’ll create a new project and share it on a local Git bare repository. You can adapt the following sections to use an existing MicroEJ project and your own Git repository.

Start MICROEJ SDK.
Go to File > New > Sandboxed Application Project.
Fill in the template fields, set Project name to com.example.hello-world.
Click Finish. This will create the project files and structure.
Right-click on source folder src/main/java and select New > Package. Set a name to the package and click Finish.
Right-click on the new package and select New > Class. Set Main as name for the class and check public static void main(String[] args), then click Finish.
Add the line System.out.println("Hello World!"); to the method and save it.
Locate the project files
1. In the Package Explorer view, right-click on the project then click on Properties.
2. Select Resource menu.
3. Click on the arrow button on line Location to show the project in the system explorer.

Note

For more details about Applications development, refer to the Application Developer Guide.

Upload to your Git repository

Note

We need the IP address of the Docker Bridge Network, here we consider that it’s 172.17.0.1 but you can check with the command ip addr show docker0 on the Docker host.

Open the project directory, create a file named Jenkinsfile and copy this content inside:

pipeline {
    agent {
        docker { 
            image 'microej/sdk:5.8.1' 
            args '-e ACCEPT_MICROEJ_SDK_EULA_V3_1B=YES'
        }
    }
    stages {
        stage('Publish') {
            steps {
                sh 'mmm publish shared -r ivy/ivysettings-artifactory.xml'
            }
        }
    }
}

Create a directory named ivy, create a file named ivysettings-artifactory.xml and copy this content inside:

<?xml version="1.0" encoding="UTF-8"?>
<ivy-settings>
  
	<property name="artifactory.repository.url" value="http://172.17.0.1:8082/artifactory" override="false"/>
	<property name="local.repository.dir" value="${user.home}/.ivy2/repository/" override="false"/>
  
	<!--
		Map MMM resolvers (*.resolver) to custom resolver
		
		Kinds of repositories:
		- release: used when publishing a released module.
		- snapshot: used when publishing a snapshot module.
		- local: used when publishing a snapshot module locally.
	-->
	<property name="release.resolver" value="modulesReleaseRepository" override="false"/>
	<property name="shared.resolver" value="modulesSnapshotRepository" override="false"/>
	<property name="local.resolver" value="localRepository" override="false"/>
 
	<property name="modules.resolver" value="fetchAll" override="false" />
	<property name="request.cache.dir" value="${user.home}/.ivy2/cache" override="false"/>
	<property name="default.conflict.manager" value="latest-compatible" override="false"/>
  
	<settings defaultResolver="${modules.resolver}" defaultConflictManager="${default.conflict.manager}" defaultResolveMode="dynamic"/>
	<caches defaultCacheDir="${request.cache.dir}"/>
  
  	<resolvers>
	 
		<url name="modulesReleaseRepository" m2compatible="true">
			<artifact pattern="${artifactory.repository.url}/custom-modules-release/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
			<ivy pattern="${artifactory.repository.url}/custom-modules-release/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
		</url>
		
		<url name="modulesSnapshotRepository" m2compatible="true" checkmodified="true">
			<artifact pattern="${artifactory.repository.url}/custom-modules-snapshot/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
			<ivy pattern="${artifactory.repository.url}/custom-modules-snapshot/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
		</url>
		
		<url name="microejModulesRepository" m2compatible="true">
			<artifact pattern="${artifactory.repository.url}/microej-module-repository/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
			<ivy pattern="${artifactory.repository.url}/microej-module-repository/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
		</url>
		
		<filesystem name="localRepository" m2compatible="true" checkmodified="true">
			<artifact pattern="${local.repository.dir}/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
			<ivy pattern="${local.repository.dir}/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
		</filesystem>
		
		<chain name="fetchRelease">
			<resolver ref="modulesReleaseRepository"/>
			<resolver ref="microejModulesRepository"/>
		</chain>
		
		<chain name="fetchSnapshot">
			<resolver ref="modulesSnapshotRepository"/>
			<resolver ref="fetchRelease"/>
		</chain>

		<chain name="fetchLocal">
			<resolver ref="localRepository"/> 
			<resolver ref="fetchSnapshot"/> 
		</chain>

		<chain name="fetchAll">
			<resolver ref="fetchLocal"/>
		</chain>
	
	</resolvers>
</ivy-settings>

This file configures the MicroEJ Module Manager to import and publish modules from the Artifactory repositories described in this training. Please refer to the Settings File section for more details.

Note

At this point, the content of the directory com.example.hello-world should look like the following:

com.example.hello-world
├── bin
│   └── ...
├── ivy
│   └── ivysettings-artifactory.xml
├── src
│   └── ...
├── src-adpgenerated/
│   └── ...
├── CHANGELOG.md
├── Jenkinsfile
├── LICENSE.txt
├── README.md
└── module.ivy

Open a terminal from the directory com.example.hello-world and type the following commands:

git init
git checkout -b main
git add *
git commit -m "Add Hello World application"
git remote add origin http://localhost:3000/<admin_user>/helloworld.git
git push -u origin main

Create a New Jenkins Job

Start by creating a new job for building our application.

Go to Jenkins dashboard.
Click on New Item.
Set item name to Hello World.
Select Multibranch Pipeline.
Validate with Ok button.
In General tab set Display Name to Hello World
In Branch Sources, click on Add Source > Git.
Add Project Repository http://172.17.0.1:3000/<admin_user>/helloworld.git
Click on Save.

Build the “Hello World” Application

Let’s run the job!

In Jenkins Hello World dashboard, click on main branch, then click on Build Now.

Note

You can check the build progress by clicking on the build progress bar and showing the Console Output.

At the end of the build, the module is published to http://localhost:8082/artifactory/list/custom-modules-snapshot/com/example/hello-world/.

Congratulations!

At this point of the training:

Artifactory is hosting your module builds and MicroEJ modules.
Jenkins automates the build process using MicroEJ Module Manager.

The next recommended step is to adapt MMM/Jenkins/Artifactory configuration to your ecosystem and development flow.

Appendix

This section discusses some of the customization options.

Customize Jenkins

Jenkins jobs are highly configurable, following options and values are recommended by MicroEJ, but they can be customized at your convenience.

In General tab:

Check Discard old builds and set Max # of builds to keep value to 15.
Click on Advanced button, and check Block build when upstream project is building.

In Build triggers tab:

Check Poll SCM, and set a CRON-like value (for example H/30 * * * * to poll SCM for changes every 30 minutes).

In Post-build actions tab:

Add post-build action Publish JUnit test result report:
Set Test report XMLs to **/target~/test/xml/**/test-report.xml, **/target~/test/xml/**/*Test.xml.

Note

The error message ‘**/target~/test/xml/**/test-report.xml’ doesn’t match anything: ‘**’ exists but not ‘**/target~/test/xml/**/test-report.xml’ will be displayed since no build has been executed yet. These folders will be generated during the build.

Check Retain long standard output/error.
Check Do not fail the build on empty test results

Customize `target~` path

Some systems and toolchains don’t handle long path properly. A workaround for this issue is to move the build directory (that is, the target~ directory) closer to the root directory.

To change the target~ directory path, set the build option target.

In Advanced, expand Properties text field and set the target property to the path of your choice. For example:

target=C:/tmp/

Setup an Automated Build using Jenkins and Artifactory

Description

Intended Audience

Prerequisites

Introduction

Overview

Prepare your Docker environment

Get a Module Repository

Setup Artifactory

Configure Artifactory

Create Repositories

Import MicroEJ Repositories

Setup Gitea

Install Gitea

Configure Gitea

Setup Jenkins

Install Jenkins

Configure Jenkins

Build a new Module using Jenkins

Create a new MicroEJ Module

Upload to your Git repository

Create a New Jenkins Job

Build the “Hello World” Application

Appendix

Customize Jenkins

Customize target~ path

Customize `target~` path