Setup an Automated Build using Jenkins and Artifactory

This tutorial explains how to setup an environment for automating MicroEJ Module build and deployment using Jenkins and JFrog Artifactory.

Such environment setup facilitates continuous integration (CI) and continuous delivery (CD), which improves productivity across your development ecosystem, by automatically:

  • building modules when source code changes
  • saving build results
  • reproducing builds
  • archiving binary modules

The tutorial should take 2 hours to complete.

Intended Audience

The audience for this document is engineers who are in charge of integrating MicroEJ Module Manager (MMM) to their continuous integration environment.

In addition, this tutorial should be of interest to all developers wishing to understand how MicroEJ works with headless module builds.

For those who are only interested by command line module build, consider using the MMM Command Line Interface.

Introduction

The overall build and deployment flow of a module can be summarized as follows:

  1. Some event triggers the build process (i.e module source changed, user action, scheduled routine, etc.)
  2. The module source code is retrieved from the Source Control System
  3. The module dependencies are imported from the Repository Manager
  4. The Automation Server then proceeds to building the module
  5. If the build is successful, the module binary is deployed to the Repository Manager
../_images/tuto_microej_cli_flow.PNG

Prerequisites

This tutorial was tested with Jenkins 2.277.4 and Artifactory 7.24.3.

Note

For SDK versions before 5.4.0, please refer to this MicroEJ Documentation Archive.

Overview

The next sections describe step by step how to setup the build environment and build your first MicroEJ module.

The steps to follow are:

  1. Install and setup MicroEJ build tools, Jenkins and Artifactory
  2. Create a Jenkins job template for MMM builds
  3. Create a simple MicroEJ module (Hello World)
  4. Create a new Jenkins job for the Hello World module
  5. Build the module

In order to simplify the steps, this tutorial will be performed locally on a single machine.

Artifactory will host MicroEJ modules in 3 repositories:

  • microej-module-repository: repository initialized with pre-built MicroEJ modules, a mirror of the Central Repository
  • custom-modules-snapshot: repository where custom snapshot modules will be published
  • custom-modules-release: repository where custom release modules will be published

Install the Build Tools

This section assumes the prerequisites have been properly installed.

  1. Locate your JDK installation directory (typically something like C:\Program Files\Java\jdk1.8.0_[version] on Windows).
  2. Set the environment variable JAVA_HOME to point to this directory.
  3. Set the environment variable JRE_HOME to point to the jre directory (for example C:\Program Files\Java\jdk1.8.0_[version]\jre).
  4. Create a directory named buildKit.
  5. Export the MicroEJ build kit from your MicroEJ SDK version to the buildKit directory, by following the steps described here.
  6. Edit the file buildKit/microej-module-repository/ivysettings.xml and replace its content by:
<?xml version="1.0" encoding="UTF-8"?>
<ivy-settings>
  
	<property name="artifactory.repository.url" value="http://localhost:8081/artifactory" override="false"/>
	<property name="local.repository.dir" value="${user.home}/.ivy2/repository/" override="false"/>
  
	<!--
		Map MMM resolvers (*.resolver) to custom resolver
		
		Kinds of repositories:
		- release: used when publishing a released module.
		- snapshot: used when publishing a snapshot module.
		- local: used when publishing a snapshot module locally.
	-->
	<property name="release.resolver" value="modulesReleaseRepository" override="false"/>
	<property name="shared.resolver" value="modulesSnapshotRepository" override="false"/>
	<property name="local.resolver" value="localRepository" override="false"/>
 
	<property name="modules.resolver" value="fetchAll" override="false" />
	<property name="request.cache.dir" value="${user.home}/.ivy2/cache" override="false"/>
	<property name="default.conflict.manager" value="latest-compatible" override="false"/>
  
	<settings defaultResolver="${modules.resolver}" defaultConflictManager="${default.conflict.manager}" defaultResolveMode="dynamic"/>
	<caches defaultCacheDir="${request.cache.dir}"/>
  
  	<resolvers>
	 
		<url name="modulesReleaseRepository" m2compatible="true">
			<artifact pattern="${artifactory.repository.url}/custom-modules-release/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
			<ivy pattern="${artifactory.repository.url}/custom-modules-release/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
		</url>
		
		<url name="modulesSnapshotRepository" m2compatible="true" checkmodified="true">
			<artifact pattern="${artifactory.repository.url}/custom-modules-snapshot/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
			<ivy pattern="${artifactory.repository.url}/custom-modules-snapshot/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
		</url>
		
		<url name="microejModulesRepository" m2compatible="true">
			<artifact pattern="${artifactory.repository.url}/microej-module-repository/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
			<ivy pattern="${artifactory.repository.url}/microej-module-repository/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
		</url>
		
		<filesystem name="localRepository" m2compatible="true" checkmodified="true">
			<artifact pattern="${local.repository.dir}/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
			<ivy pattern="${local.repository.dir}/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
		</filesystem>
		
		<chain name="fetchRelease">
			<resolver ref="modulesReleaseRepository"/>
			<resolver ref="microejModulesRepository"/>
		</chain>
		
		<chain name="fetchSnapshot">
			<resolver ref="modulesSnapshotRepository"/>
			<resolver ref="fetchRelease"/>
		</chain>

		<chain name="fetchLocal">
			<resolver ref="localRepository"/> 
			<resolver ref="fetchSnapshot"/> 
		</chain>

		<chain name="fetchAll">
			<resolver ref="fetchLocal"/>
		</chain>
	
	</resolvers>
</ivy-settings>

This file configures MicroEJ Module Manager to import and publish modules from the Artifactory repositories described in this tutorial. Please refer to Settings File section for more details.

Note

At this point, the content of the directory buildKit should look like the following:

buildKit
├── bin
│   ├── mmm
│   └── mmm.bat
├── conf
│   └── easyant-conf.xml
├── lib
│   ├── ant.jar
│   └── ...
├── microej-build-repository
│   ├── ant-contrib
│   ├── be
│   └── ...
├── microej-module-repository
│   └── ivysettings.xml
└── release.properties

Get a Module Repository

A Module Repository is a portable ZIP file that bundles a set of modules for extending the MicroEJ development environment. Please consult the Module Repository section for more information.

This tutorial uses the MicroEJ Central Repository, which is the Module Repository used by MicroEJ SDK to fetch dependencies when starting an empty workspace. It bundles Foundation Library APIs and numerous Add-On Libraries.

Next step is to download a local copy of this repository:

  1. Visit the Central Repository on the MicroEJ Developer website.
  2. Navigate to the Working Offline section.
  3. Click on the offline repository link. This will download the Central Repository as a ZIP file.

Setup Artifactory

Install and Start Artifactory

  1. Download Artifactory here: https://jfrog.com/fr/open-source/ and select the appropriate package for your operating system.
  2. Unzip downloaded archive, then navigate to app/bin directory (by default artifactory-oss-[version]/app/bin).
  3. Run artifactory.bat or artifactory.sh depending on your operating system.
  4. Once Artifactory is started, go to http://localhost:8081/.
  5. Login to Artifactory for the first time using the default admin account (Username: admin, Password: password).
  6. On the Welcome wizard, set the administrator password, then click Next,
  7. Configure proxy server (if any) then click Next, or click Skip.
  8. Click on Finish.

Artifactory is up and running.

Configure Artifactory

For demonstration purposes we will allow anonymous users to deploy modules in the repositories:

  1. Go to Administration > Security > Settings.
  2. In the General Security Settings section, check Allow Anonymous Access.
  3. Click on Save.
  4. Go to Administration > Identity and Access > Permissions.
  5. Click on Anything entry (do not check the line), then go to Users tab
  6. Click on anonymous and check Deploy/Cache permission in the Repositories category.
  7. Click on Save and finish.

Next steps will involve uploading large files, so we have to augment the file upload maximum size accordingly:

  1. Go to Administration > Artifactory.
  2. In the General Settings section, change the value of File Upload In UI Max Size (MB) to 1024 then click on Save.

Create Repositories

We will now create and configure the repositories. Let’s start with the repository for the future built snapshot modules:

  1. Go to Administration > Repositories > Repositories in the left menu.
  2. Click on Add Repositories > Local Repository
  3. Select Maven.
  4. Set Repository Key field to custom-modules-snapshot and click on Save and Finish.

Repeat the same steps for the other repositories with the Repository Key field set to custom-modules-release and microej-module-repository.

Import MicroEJ Repositories

In this section, we will import MicroEJ repositories into Artifactory repositories to make them available to the build server.

  1. Go to Administration > Artifactory > Import & Export > Repositories.
  2. Scroll to the Import Repository from Zip section.
  3. As Target Local Repository, select microej-module-repository in the list.
  4. Click on Select file and select the MicroEJ module repository zip file (microej-[MicroEJ version]-[version].zip) that you downloaded earlier (please refer to section Get a Module Repository).
  5. Click Upload. At the end of upload, click on Import. Upload and import may take some time.

Artifactory is now hosting all required MicroEJ modules. Go to Administration > Artifactory > Artifacts and check that the repository microej-module-repository does contain modules as shown in the figure below.

../_images/tuto_microej_cli_artifactory_module_preview.PNG

Setup Jenkins

Install Jenkins

  1. Download Jenkins here: https://www.jenkins.io/download/. In this tutorial we will use the WAR (Web Archive), but you can use any other installation package (Docker, Ubuntu/Debian, …).
  2. Open a terminal and type the following command: java -jar [path/to/downloaded/jenkinswar]/jenkins.war. After initialization, the terminal will print out Jenkins is fully up and running.
  3. Go to http://localhost:8080/.
  4. To unlock Jenkins, copy/paste the generated password that has been written in the terminal log. Click on Continue.
  5. Select option Install suggested plugins and wait for plugins installation.
  6. Fill in the Create First Admin User form. Click Save and continue.
  7. Click on Save and finish, then on Start using Jenkins.

Configure Jenkins

First step is to configure the JDK and MMM paths:

  1. Go to Manage Jenkins > Global Tool Configuration.
  2. Add JDK installation:
    1. Scroll to JDK section.
    2. Click on Add JDK.
    3. Set Name to JDK [jdk_version] (for example JDK 1.8).
    4. Uncheck Install automatically.
    5. Set JAVA_HOME to the absolute path of your JDK installation (for example C:\Program Files\Java\jdk1.8.0_[version] on Windows).
  3. Click on Save.
  4. Go to Manage Jenkins > Configure System.
    1. Scroll to Global properties section.
    2. Check Environment variables.
    3. Click on Add.
    4. Set Name to MICROEJ_BUILD_KIT_HOME.
    5. Set Value to the absolute path of the buildKit folder.
  5. Click on Save.

Create a Job Template

  1. Go to Jenkins dashboard.
  2. Click on New item to create a job template.
  3. Set item name to Template - MMM from Git.
  4. Select Freestyle project.
  5. Click on Ok.

In General tab:

  1. Check This project is parameterized and add String parameter named easyant.module.dir with default value to $WORKSPACE/TO_REPLACE. This will later point to the module sources.

In Source Code Management tab:

  1. Select Git source control:
  2. Set Repository URL value to TO_REPLACE,
  3. Set Branch Specifier value to origin/master,
  4. In Additional Behaviours, click on Add, select Advanced sub-modules behaviors, then check Recursively update submodules.

In Build tab:

  • For Windows, add build step Execute Windows batch command:

    • In Command, set the following content:
    cd "%easyant.module.dir%"
"%MICROEJ_BUILD_KIT_HOME%\\bin\\mmm.bat" publish shared"

  • For Linux, add build step Execute shell:

    • In Command, set the following content:
    cd "${easyant.module.dir}"
"${MICROEJ_BUILD_KIT_HOME}/bin/mmm" publish shared"

Finally, click on Save.

Build a new Module using Jenkins

Since your environment is now setup, it is time to build your first module from Jenkins and check it has been published to Artifactory. Let’s build an “Hello World” Sandboxed Application project.

Create a new MicroEJ Module

In this example, we will create a very simple module using the Sandbox Application buildtype (build-application) that we’ll push to a Git repository.

Note

For demonstration purposes, we’ll create a new project and share it on a local Git bare repository. You can adapt the following sections to use an existing MicroEJ project and your own Git repository.

  1. Start MicroEJ SDK.

  2. Go to File > New > Sandboxed Application Project.

  3. Fill in the template fields, set Project name to com.example.hello-world.

    ../_images/tuto_microej_cli_module_creation.PNG

  4. Click Finish. This will create the project files and structure.

  5. Right-click on source folder src/main/java and select New > Package. Set a name to the package and click Finish.

  6. Right-click on the new package and select New > Class. Set a name to the class and check public static void main(String[] args), then click Finish.

    ../_images/tuto_microej_cli_module_files.PNG
  7. Locate the project files
    1. In the Package Explorer view, right-click on the project then click on Properties.
    2. Select Resource menu.
    3. Click on the arrow button on line Location to show the project in the system explorer.
    ../_images/tuto_microej_cli_module_location.PNG

  8. Open a terminal from this directory and type the following commands:

    git init --bare ~/hello_world.git
git init
git remote add origin ~/hello_world.git
git add com.example.hello-world
git commit -m "Add Hello World application"
git push --set-upstream origin master

Note

For more details about MicroEJ Applications development, refer to the Application Developer Guide.

Create a New Jenkins Job

Start by creating a new job, from the job template, for building our application.

  1. Go to Jenkins dashboard.
  2. Click on New Item.
  3. Set item name to Hello World.
  4. In Copy from field, type Template - MMM from Git (autocomplete enabled).
  5. Validate with Ok button.

The job configuration page opens, let’s replace all the TO_REPLACE placeholders from the job template with correct values:

  1. In General tab, set easyant.module.dir to value $WORKSPACE/com.example.hello-world.

    ../_images/tuto_microej_cli_jenkins_parameter.PNG

  2. In Source Code Management, edit Repository URL to ~/hello_world.git.

    ../_images/tuto_microej_cli_jenkins_git_hello.PNG

  3. Click on Save.

Build the “Hello World” Application

Let’s run the job!

In Jenkins’ Hello World dashboard, click on Build with Parameters, then click on Build.

Note

You can check the build progress by clicking on the build progress bar and showing the Console Output.

At the end of the build, the module is published to http://localhost:8081/artifactory/list/custom-modules-snapshot/com/example/hello-world/.

Congratulations!

At this point of the tutorial:

  • Artifactory is hosting your module builds and MicroEJ modules.
  • Jenkins automates the build process using MicroEJ Module Manager.

The next recommended step is to adapt MMM/Jenkins/Artifactory configuration to your ecosystem and development flow.

Appendix

This section discusses some of the customization options.

Customize Jenkins

Jenkins jobs are highly configurable, following options and values are recommended by MicroEJ, but they can be customized at your convenience.

In General tab:

  1. Check Discard old builds and set Max # of builds to keep value to 15.
  2. Click on Advanced button, and check Block build when upstream project is building.

In Build triggers tab:

  1. Check Poll SCM, and set a CRON-like value (for example H/30 * * * * to poll SCM for changes every 30 minutes).

In Post-build actions tab:

  1. Add post-build action Publish JUnit test result report:
  2. Set Test report XMLs to **/target~/test/xml/**/test-report.xml, **/target~/test/xml/**/*Test.xml.

Note

The error message ‘**/target~/test/xml/**/test-report.xml’ doesn’t match anything: ‘**’ exists but not ‘**/target~/test/xml/**/test-report.xml’ will be displayed since no build has been executed yet. These folders will be generated during the build.

  1. Check Retain long standard output/error.
  2. Check Do not fail the build on empty test results

Customize target~ path

Some systems and toolchains don’t handle long path properly. A workaround for this issue is to move the build directory (that is, the target~ directory) closer to the root directory.

To change the target~ directory path, set the build option target.

In Advanced, expand Properties text field and set the target property to the path of your choice. For example:

target=C:/tmp/