Setup an Automated Build using Jenkins and Artifactory¶

This tutorial explains how to setup an environment for automating MicroEJ Module build and deployment using Jenkins and JFrog Artifactory.

Such environment setup facilitates continuous integration (CI) and continuous delivery (CD), which improves productivity across your development ecosystem, by automatically:

building modules when source code changes
saving build results
reproducing builds
archiving binary modules

The tutorial should take 2 hours to complete.

Intended Audience¶

The audience for this document is engineers who are in charge of integrating MicroEJ Module Manager (MMM) to their continuous integration environment.

In addition, this tutorial should be of interest to all developers wishing to understand how MicroEJ works with headless module builds.

For those who are only interested by command line module build, consider using the MMM Command Line Interface.

Introduction¶

The overall build and deployment flow of a module can be summarized as follows:

Some event triggers the build process (i.e module source changed, user action, scheduled routine, etc.)
The module source code is retrieved from the Source Control System
The module dependencies are imported from the Repository Manager
The Automation Server then proceeds to building the module
If the build is successful, the module binary is deployed to the Repository Manager

Prerequisites¶

MicroEJ SDK 5.4.0 or higher.
Git 2.x installed, with Git executable in path. We recommend installing Git Bash if your operating system is Windows (https://git-for-windows.github.io/).
Java Development Kit (JDK) 1.8.x.

This tutorial was tested with Jenkins 2.277.4 and Artifactory 6.23.13.

Note

For SDK versions before 5.4.0, please refer to this MicroEJ Documentation Archive.

Overview¶

The next sections describe step by step how to setup the build environment and build your first MicroEJ module.

The steps to follow are:

Install and setup MicroEJ build tools, Jenkins and Artifactory
Create a Jenkins job template for MMM builds
Create a simple MicroEJ module (Hello World)
Create a new Jenkins job for the Hello World module
Build the module

In order to simplify the steps, this tutorial will be performed locally on a single machine.

Artifactory will host MicroEJ modules in 2 repositories:

microej-module-repository: repository initialized with pre-built MicroEJ modules, a mirror of the Central Repository
libs-snapshot-local: repository where custom modules will be published

Install the Build Tools¶

This section assumes the prerequisites have been properly installed.

Locate your JDK installation directory (typically something like C:\Program Files\Java\jdk1.8.0_[version] on Windows).
Set the environment variable JAVA_HOME to point to this directory.
Set the environment variable JRE_HOME to point to the jre directory (for example C:\Program Files\Java\jdk1.8.0_[version]\jre).
Create a directory named buildKit.
Export the MicroEJ build kit from your MicroEJ SDK version to the buildKit directory, by following the steps described here.
Edit the file buildKit/microej-module-repository/ivysettings.xml and replace its content by:

<?xml version="1.0" encoding="UTF-8"?>
<ivy-settings>
  
	<property name="artifactory.repository.url" value="http://localhost:8081/artifactory" override="false"/>
	<property name="local.repository.dir" value="${user.home}/.ivy2/repository/" override="false"/>
  
	<!--
		Map MMM resolvers (*.resolver) to custom resolver
		
		Kinds of repositories:
		- release: used when publishing a released module.
		- snapshot: used when publishing a snapshot module.
		- local: used when publishing a snapshot module locally.
	-->
	<property name="release.resolver" value="modulesReleaseRepository" override="false"/>
	<property name="shared.resolver" value="modulesSnapshotRepository" override="false"/>
	<property name="local.resolver" value="localRepository" override="false"/>
 
	<property name="modules.resolver" value="fetchAll" override="false" />
	<property name="request.cache.dir" value="${user.home}/.ivy2/cache" override="false"/>
	<property name="default.conflict.manager" value="latest-compatible" override="false"/>
  
	<settings defaultResolver="${modules.resolver}" defaultConflictManager="${default.conflict.manager}" defaultResolveMode="dynamic"/>
	<caches defaultCacheDir="${request.cache.dir}"/>
  
  	<resolvers>
	 
		<url name="modulesReleaseRepository" m2compatible="true">
			<artifact pattern="${artifactory.repository.url}/libs-release-local/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
			<ivy pattern="${artifactory.repository.url}/libs-release-local/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
		</url>
		
		<url name="modulesSnapshotRepository" m2compatible="true" checkmodified="true">
			<artifact pattern="${artifactory.repository.url}/libs-snapshot-local/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
			<ivy pattern="${artifactory.repository.url}/libs-snapshot-local/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
		</url>
		
		<url name="microejModulesRepository" m2compatible="true">
			<artifact pattern="${artifactory.repository.url}/microej-module-repository/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
			<ivy pattern="${artifactory.repository.url}/microej-module-repository/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
		</url>
		
		<filesystem name="localRepository" m2compatible="true" checkmodified="true">
			<artifact pattern="${local.repository.dir}/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
			<ivy pattern="${local.repository.dir}/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
		</filesystem>
		
		<chain name="fetchRelease">
			<resolver ref="modulesReleaseRepository"/>
			<resolver ref="microejModulesRepository"/>
		</chain>
		
		<chain name="fetchSnapshot">
			<resolver ref="modulesSnapshotRepository"/>
			<resolver ref="fetchRelease"/>
		</chain>

		<chain name="fetchLocal">
			<resolver ref="localRepository"/> 
			<resolver ref="fetchSnapshot"/> 
		</chain>

		<chain name="fetchAll">
			<resolver ref="fetchLocal"/>
		</chain>
	
	</resolvers>
</ivy-settings>

This file configures MicroEJ Module Manager to import and publish modules from the Artifactory repositories described in this tutorial. Please refer to Settings File section for more details.

Note

At this point, the content of the directory buildKit should look like the following:

buildKit
├── bin
│   ├── mmm
│   └── mmm.bat
├── conf
│   └── easyant-conf.xml
├── lib
│   ├── ant.jar
│   └── ...
├── microej-build-repository
│   ├── ant-contrib
│   ├── be
│   └── ...
├── microej-module-repository
│   └── ivysettings.xml
└── release.properties

Get a Module Repository¶

A Module Repository is a portable ZIP file that bundles a set of modules for extending the MicroEJ development environment. Please consult the Module Repository section for more information.

This tutorial uses the MicroEJ Central Repository, which is the Module Repository used by MicroEJ SDK to fetch dependencies when starting an empty workspace. It bundles Foundation Library APIs and numerous Add-On Libraries.

Next step is to download a local copy of this repository:

Visit the Central Repository on the MicroEJ Developer website.
Navigate to the Working Offline section.
Click on the offline repository link. This will download the Central Repository as a ZIP file.

Setup Artifactory¶

Install and Start Artifactory¶

Download Artifactory here: https://jfrog.com/fr/open-source/ and select the appropriate package for your operating system.
Unzip downloaded archive, then navigate to app/bin directory (by default artifactory-oss-[version]/app/bin).
Run artifactory.bat or artifactory.sh depending on your operating system. After initialization, the terminal should print the message Artifactory successfully started. In case an error occurs, check that JAVA_HOME and JRE_HOME environment variables are correct.
Go to http://localhost:8081/.
Login to Artifactory for the first time using the default admin account (Username: admin, Password: password).
On the Welcome wizard, set the administrator password, then click Next,
Configure proxy server (if any) then click Next, or click Skip.
On Create Repositories page, select Maven then click on Create.
Click on Finish.

Artifactory is up and running.

Configure Artifactory¶

For demonstration purposes we will allow anonymous users to deploy modules in the repositories.

Go to Admin > Security > Security Configuration.
In the General Security Settings section, check Allow Anonymous Access. Click Save.
Go to Admin > Security > Permissions.
Click on Anything entry (do not check the line), then go to Users tab, click on Anonymous and check Deploy/Cache permission. Click Save and finish.

Next steps will involve uploading large files, so we have to augment the file upload maximum size accordingly:

Go to Admin > General Configuration.
In the General Settings section, change the value of File Upload Max Size (MB) to 1024 then click on Save.

Configure Repositories¶

We will now create and configure the repositories. First step is to configure to pre-defined repository for the future snapshot modules built.

Go to Admin > Repositories > Local.
Click on libs-snapshot-local repository, then check Handle Releases and uncheck Handle Snapshots. Click Save and finish.

Next step is to create the repositories that will hold the MicroEJ modules.

Go to Admin > Repositories > Local.
Click on New, and select Maven.
Set Repository Key field to microej-module-repository, then uncheck Handle Snapshots. Click on Save and finish.
Make this repository accessible by default:
1. Go to Admin > Security > Permissions.
2. Click on Anything entry (do not check the line)
3. On the Resources tab, drag repository microej-module-repository from the Available repositories area to the Included Repositories area.
4. Click on Save & Finish.

../_images/tuto_microej_cli_artifactory_permissions.PNG

Import MicroEJ Repositories¶

In this section, we will import MicroEJ repositories into Artifactory repositories to make them available to the build server.

Go to Admin > Import & Export > Repositories.
Scroll to the Import Repository from Zip section.
Import the MicroEJ Module Repository:
1. As Target Local Repository, select microej-module-repository in the list.
2. As Repository Zip File, select MicroEJ module repository zip file (microej-[MicroEJ version]-[version].zip) that you downloaded earlier (please refer to section Get a Module Repository).
3. Click Upload. At the end of upload, click on Import. Upload and import may take some time.

Artifactory is now hosting all required MicroEJ modules. Go to Artifacts and check that the repository microej-module-repository does contain modules as shown in the figure below.

../_images/tuto_microej_cli_artifactory_module_preview.PNG

Setup Jenkins¶

Install Jenkins¶

Download Jenkins here: https://www.jenkins.io/download/. In this tutorial we will use the WAR (Web Archive), but you can use any other installation package (Docker, Ubuntu/Debian, …).
Open a terminal and type the following command: java -jar [path/to/downloaded/jenkinswar]/jenkins.war. After initialization, the terminal will print out Jenkins is fully up and running.
Go to http://localhost:8080/.
To unlock Jenkins, copy/paste the generated password that has been written in the terminal log. Click on Continue.
Select option Install suggested plugins and wait for plugins installation.
Fill in the Create First Admin User form. Click Save and continue.
Click on Save and finish, then on Start using Jenkins.

Configure Jenkins¶

First step is to configure the JDK and MMM paths:

Go to Manage Jenkins > Global Tool Configuration.
Add JDK installation:
1. Scroll to JDK section.
2. Click on Add JDK.
3. Set Name to JDK [jdk_version] (for example JDK 1.8).
4. Uncheck Install automatically.
5. Set JAVA_HOME to the absolute path of your JDK installation (for example C:\Program Files\Java\jdk1.8.0_[version] on Windows).
Click on Save.
Go to Manage Jenkins > Configure System.
1. Scroll to Global properties section.
2. Check Environment variables.
3. Click on Add.
4. Set Name to MICROEJ_BUILD_KIT_HOME.
5. Set Value to the absolute path of the buildKit folder.
Click on Save.

Create a Job Template¶

Go to Jenkins dashboard.
Click on New item to create a job template.
Set item name to Template - MMM from Git.
Select Freestyle project.
Click on Ok.

In General tab:

Check This project is parameterized and add String parameter named easyant.module.dir with default value to $WORKSPACE/TO_REPLACE. This will later point to the module sources.

In Source Code Management tab:

Select Git source control:
Set Repository URL value to TO_REPLACE,
Set Branch Specifier value to origin/master,
In Additional Behaviours, click on Add, select Advanced sub-modules behaviors, then check Recursively update submodules.

In Build tab:

For Windows, add build step Execute Windows batch command:
- In Command, set the following content:
cd "%easyant.module.dir%" "%MICROEJ_BUILD_KIT_HOME%\\bin\\mmm.bat" publish shared"

For Linux, add build step Execute shell:

In Command, set the following content:

cd "${easyant.module.dir}"
"${MICROEJ_BUILD_KIT_HOME}/bin/mmm" publish shared"

Finally, click on Save.

Build a new Module using Jenkins¶

Since your environment is now setup, it is time to build your first module from Jenkins and check it has been published to Artifactory. Let’s build an “Hello World” Sandboxed Application project.

Create a new MicroEJ Module¶

In this example, we will create a very simple module using the Sandbox Application buildtype (build-application) that we’ll push to a Git repository.

Note

For demonstration purposes, we’ll create a new project and share it on a local Git bare repository. You can adapt the following sections to use an existing MicroEJ project and your own Git repository.

Start MicroEJ SDK.
Go to File > New > Sandboxed Application Project.
Fill in the template fields, set Project name to com.example.hello-world.
Click Finish. This will create the project files and structure.
Right-click on source folder src/main/java and select New > Package. Set a name to the package and click Finish.
Right-click on the new package and select New > Class. Set a name to the class and check public static void main(String[] args), then click Finish.
Locate the project files
1. In the Package Explorer view, right-click on the project then click on Properties.
2. Select Resource menu.
3. Click on the arrow button on line Location to show the project in the system explorer.

Open a terminal from this directory and type the following commands:

git init --bare ~/hello_world.git
git init
git remote add origin ~/hello_world.git
git add com.example.hello-world
git commit -m "Add Hello World application"
git push --set-upstream origin master

Note

For more details about MicroEJ Applications development, refer to the Application Developer Guide.

Create a New Jenkins Job¶

Start by creating a new job, from the job template, for building our application.

Go to Jenkins dashboard.
Click on New Item.
Set item name to Hello World.
In Copy from field, type Template - MMM from Git (autocomplete enabled).
Validate with Ok button.

The job configuration page opens, let’s replace all the TO_REPLACE placeholders from the job template with correct values:

In General tab, set easyant.module.dir to value $WORKSPACE/com.example.hello-world.
In Source Code Management, edit Repository URL to ~/hello_world.git.
Click on Save.

Build the “Hello World” Application¶

Let’s run the job!

In Jenkins’ Hello World dashboard, click on Build with Parameters, then click on Build.

Note

You can check the build progress by clicking on the build progress bar and showing the Console Output.

At the end of the build, the module is published to http://localhost:8081/artifactory/list/libs-snapshot-local/com/example/hello-world/.

Congratulations!

At this point of the tutorial:

Artifactory is hosting your module builds and MicroEJ modules.
Jenkins automates the build process using MicroEJ Module Manager.

The next recommended step is to adapt MMM/Jenkins/Artifactory configuration to your ecosystem and development flow.

Appendix¶

This section discusses some of the customization options.

Customize Jenkins¶

Jenkins jobs are highly configurable, following options and values are recommended by MicroEJ, but they can be customized at your convenience.

In General tab:

Check Discard old builds and set Max # of builds to keep value to 15.
Click on Advanced button, and check Block build when upstream project is building.

In Build triggers tab:

Check Poll SCM, and set a CRON-like value (for example H/30 * * * * to poll SCM for changes every 30 minutes).

In Post-build actions tab:

Add post-build action Publish JUnit test result report:
Set Test report XMLs to **/target~/test/xml/**/test-report.xml, **/target~/test/xml/**/*Test.xml.

Note

The error message ‘**/target~/test/xml/**/test-report.xml’ doesn’t match anything: ‘**’ exists but not ‘**/target~/test/xml/**/test-report.xml’ will be displayed since no build has been executed yet. These folders will be generated during the build.

Check Retain long standard output/error.
Check Do not fail the build on empty test results

Customize `target~` path¶

Some systems and toolchains don’t handle long path properly. A workaround for this issue is to move the build directory (that is, the target~ directory) closer to the root directory.

To change the target~ directory path, set the build option target.

In Advanced, expand Properties text field and set the target property to the path of your choice. For example:

target=C:/tmp/

Setup an Automated Build using Jenkins and Artifactory¶

Intended Audience¶

Introduction¶

Prerequisites¶

Overview¶

Install the Build Tools¶

Get a Module Repository¶

Setup Artifactory¶

Install and Start Artifactory¶

Configure Artifactory¶

Configure Repositories¶

Import MicroEJ Repositories¶

Setup Jenkins¶

Install Jenkins¶

Configure Jenkins¶

Create a Job Template¶

Build a new Module using Jenkins¶

Create a new MicroEJ Module¶

Create a New Jenkins Job¶

Build the “Hello World” Application¶

Appendix¶

Customize Jenkins¶

Customize target~ path¶

Customize `target~` path¶