Setup an Automated Build using Jenkins and Artifactory

This tutorial explains how to setup an environment for automating MicroEJ Module build and deployment using Jenkins and JFrog Artifactory.

Such environment setup facilitates continuous integration (CI) and continuous delivery (CD), which improves productivity across your development ecosystem, by automatically:

  • building modules when source code changes
  • saving build results
  • reproducing builds
  • archiving binary modules

The tutorial should take 2 hours to complete.

Intended Audience

The audience for this document is engineers who are in charge of integrating MicroEJ Module Manager (MMM) to their continuous integration environment.

In addition, this tutorial should be of interest to all developers wishing to understand how MicroEJ works with headless module builds.

For those who are only interested by command line module build, consider using the CommandLineBuild tool.

Introduction

The overall build and deployment flow of a module can be summarized as follows:

  1. Some event triggers the build process (i.e module source changed, user action, scheduled routine, etc.)
  2. The module source code is retrieved from the Source Control System
  3. The module dependencies are imported from the Repository Manager
  4. The Automation Server then proceeds to building the module
  5. If the build is successful, the module binary is deployed to the Repository Manager
../_images/tuto_microej_cli_flow.PNG

Prerequisites

This tutorial was tested with Jenkins 2.235.3 and Artifactory 6.20.1.

Overview

The next sections describe step by step how to setup the build environment and build your first MicroEJ module.

The steps to follow are:

  1. Install and setup MicroEJ build tools, Jenkins and Artifactory
  2. Create a Jenkins job template for MMM builds
  3. Create a simple MicroEJ module (Hello World)
  4. Create a new Jenkins job for the Hello World module
  5. Build the module

For the purposes of simplifying the steps, this tutorial will be performed locally on a single machine.

Artifactory will host MicroEJ modules in 3 repositories:

  • microej-module-repository: repository initialized with pre-built MicroEJ modules, a mirror of the Central Repository
  • microej-build-repository: repository initialized with build scripts and tools exported from MicroEJ SDK
  • libs-snapshot-local: repository where custom modules will be published

Install the Build Tools

This section assumes the prerequisites have been properly installed.

  1. Locate your JDK installation directory (typically something like C:\Program Files\Java\jdk1.8.0_[version] on Windows).

  2. Set the environment variable JAVA_HOME to point to the bin directory (for example C:\Program Files\Java\jdk1.8.0_[version]\bin).

  3. Set the environment variable JRE_HOME to point to the jre directory (for example C:\Program Files\Java\jdk1.8.0_[version]\jre).

  4. Download the pre-configured settings file by cloning the following git repository:

    git clone --recursive https://github.com/MicroEJ/Tool-CommandLineBuild.git

  5. Create a directory named buildKit in the Tool-CommandLineBuild directory.

  6. Export the MicroEJ build kit from your MicroEJ SDK version to the buildKit directory, by following the steps described here.

  7. Set the environment variable MICROEJ_BUILD_TOOLS_HOME to point to the Tool-CommandLineBuild directory

Note

At this point, the content of the directory Tool-CommandLineBuild should look like the following:

Tool-CommandLineBuild
├── buildKit
│   ├── ant
│   │   └── lib
│   │       ├── ant.jar
│   │       ├── ant-launcher.jar
│   │       └── ...
│   └── microej-build-repository.zip (or is2t_repo.zip)
├── easyant
│   └── ...
├── ivy
│   ├── ivysettings-artifactory.xml
│   └── ...
└── ...

Get a Module Repository

A Module Repository is a portable ZIP file that bundles a set of modules for extending the MicroEJ development environment. Please consult the Module Repository section for more information.

This tutorial uses the MicroEJ Central Repository, which is the Module Repository used by MicroEJ SDK to fetch dependencies when starting an empty workspace. It bundles Foundation Library APIs and numerous Add-On Libraries.

Next step is to download a local copy of this repository:

  1. Visit the Central Repository on the MicroEJ Developer website.
  2. Navigate to the Working Offline section.
  3. Click on the offline repository link. This will download the Central Repository as a ZIP file.

Setup Artifactory

Install and Start Artifactory

  1. Download Artifactory here: https://api.bintray.com/content/jfrog/artifactory/jfrog-artifactory-oss-$latest.zip;bt_package=jfrog-artifactory-oss-zip.
  2. Unzip downloaded archive, then navigate to bin directory (by default artifactory-oss-[version]/bin).
  3. Run artifactory.bat or artifactory.sh depending on your operating system. After initialization, the terminal should print the message Artifactory successfully started. In case an error occurs, check that JAVA_HOME and JRE_HOME environment variables are correct.
  4. Go to http://localhost:8081/.
  5. Login to Artifactory for the first time using the default admin account (Username: admin, Password: password).
  6. On the Welcome wizard, set the administrator password, then click Next,
  7. Configure proxy server (if any) then click Next, or click Skip.
  8. On Create Repositories page, select Maven then click on Create.
  9. Click on Finish.

Artifactory is up and running.

Configure Artifactory

For demonstration purposes we will allow anonymous users to deploy modules in the repositories.

  1. Go to Admin > Security > Security Configuration.
  2. In the General Security Settings section, check Allow Anonymous Access. Click Save.
  3. Go to Admin > Security > Permissions.
  4. Click on Anything entry (do not check the line), then go to Users tab, click on Anonymous and check Deploy/Cache permission. Click Save and finish.

Next steps will involve uploading large files, so we have to augment the file upload maximum size accordingly:

  1. Go to Admin > General Configuration.
  2. In the General Settings section, change the value of File Upload Max Size (MB) to 1024 then click on Save.

Configure Repositories

First step is to configure to pre-defined repository for the future snapshot modules built.

  1. Go to Admin > Repositories > Local.
  2. Click on libs-snapshot-local repository, then check Handle Releases and uncheck Handle Snapshots. Click Save and finish.

Next step is to create the repositories that will hold the MicroEJ modules.

  1. Go to Admin > Repositories > Local.
  2. Click on New, and select Maven.
  3. Set Repository Key field to microej-module-repository, then uncheck Handle Snapshots. Click on Save and finish.
  4. Click on New, and select Maven.
  5. Set Repository Key field to microej-build-repository, then uncheck Handle Snapshots. Click on Save and finish.
  6. Make these two repositories accessible by default:
    1. Go to Admin > Security > Permissions.
    2. Click on Anything entry (do not check the line)
    3. On the Resources tab, drag repositories microej-module-repository and microej-build-repository from the Available repositories area to the Included Repositories area.
    4. Click on Save & Finish.
../_images/tuto_microej_cli_artifactory_permissions.PNG

Import MicroEJ Repositories

In this section, we will import MicroEJ repositories into Artifactory repositories to make them available to the build server.

  1. Go to Admin > Import & Export > Repositories.
  2. Scroll to the Import Repository from Zip section.
  3. Import the MicroEJ Module Repository:
    1. As Target Local Repository, select microej-module-repository in the list.
    2. As Repository Zip File, select MicroEJ module repository zip file (microej-[MicroEJ version]-[version].zip) that you downloaded earlier (please refer to section Get a Module Repository).
    3. Click Upload. At the end of upload, click on Import. Upload and import may take some time.
  4. Import the MicroEJ Build Repository:
    1. As Target Local Repository, select microej-build-repository in the list.
    2. As Repository Zip File, select MicroEJ Build Repository zip file (microej-build-repository.zip or is2t_repo.zip) that you exported from MicroEJ SDK earlier (please refer to section Install the Build Tools).
    3. Click Upload. At the end of upload, click on Import. Upload and import may take some time.

Artifactory is now hosting all required MicroEJ modules. Go to Artifacts and check that repositories microej-module-repository and microej-build-repository do contain modules as shown in the figure below.

../_images/tuto_microej_cli_artifactory_preview.PNG

Setup Jenkins

Install Jenkins

  1. Download Jenkins WAR (Web Archive) here: http://mirrors.jenkins.io/war-stable/latest/jenkins.war
  2. Open a terminal and type the following command: java -jar [path/to/downloaded/jenkinswar]/jenkins.war. After initialization, the terminal will print out Jenkins is fully up and running.
  3. Go to http://localhost:8080/.
  4. To unlock Jenkins, copy/paste the generated password that has been written in the terminal log. Click on Continue.
  5. Select option Install suggested plugins and wait for plugin installation.
  6. Fill in the Create First Admin User form. Click Save and continue.
  7. Click on Save and finish, then on Start using Jenkins.

Configure Jenkins

First step is to configure JDK and Ant installations:

  1. Go to Manage Jenkins > Global Tool Configuration.
  2. Add JDK installation:
    1. Scroll to JDK section.
    2. Click on Add JDK.
    3. Set Name to JDK [jdk_version] (for example JDK 1.8).
    4. Uncheck Install automatically.
    5. Set JAVA_HOME to path/to/jdk[jdk_version] (for example C:\Program Files\Java\jdk1.8.0_[version] on Windows).
  3. Add Ant installation:
    1. Scroll to Ant section.
    2. Click on Add Ant.
    3. Set Name to Ant 1.9.
    4. Uncheck Install automatically.
    5. Set ANT_HOME to path/to/apache-ant-1.9.[version].
  4. Click on Save.

Create a Job Template

  1. Go to Jenkins dashboard.
  2. Click on New item to create a job template.
  3. Set item name to Template - MMM from Git.
  4. Select Freestyle project.
  5. Click on Ok.

In General tab:

  1. Check This project is parametrized and add String parameter named easyant.module.dir with default value to $WORKSPACE/TO_REPLACE. This will later point to the module sources.

In Source Code Management tab:

  1. Select Git source control:
  2. Set Repository URL value to TO_REPLACE,
  3. Set Branch Specifier value to origin/master,
  4. In Additional Behaviours, click on Add, select Advanced sub-modules behaviors, then check Recursively update submodules.

In Build tab:

  1. Add build step Invoke Ant:
    • As Ant version, select Ant 1.9.
    • Set Targets to value -lib ${MICROEJ_BUILD_TOOLS_HOME}/buildKit/ant/lib.
    • In Advanced, set Build file to value $MICROEJ_BUILD_TOOLS_HOME/easyant/build-module.ant.
    • In Advanced, expand Properties text field then add the following Ant properties:
    personalBuild=false
jenkins.build.id=$BUILD_ID
jenkins.node.name=$NODE_NAME
user.ivysettings.file=$MICROEJ_BUILD_TOOLS_HOME/ivy/ivysettings-artifactory.xml
../_images/tuto_microej_cli_jenkins_build.PNG

Finally, click on Save.

Build a new Module using Jenkins

Since your environment is now setup, it is time to build your first module from Jenkins and check it has been published to Artifactory. Let’s build an “Hello World” Sandboxed Application project.

Create a new MicroEJ Module

In this example, we will create a very simple module using the Sandbox Application buildtype (build-application) that we’ll push to a Git repository.

Note

For demonstration purposes, we’ll create a new project and share it on a local Git bare repository. You can adapt the following sections to use an existing MicroEJ project and your own Git repository.

  1. Start MicroEJ SDK.

  2. Go to File > New > MicroEJ Sandboxed Application Project.

  3. Fill in the template fields, set Project name to com.example.hello-world.

    ../_images/tuto_microej_cli_module_creation.PNG

  4. Click Finish. This will create the project files and structure.

  5. Right-click on source folder src/main/java and select New > Package. Set a name to the package and click Finish.

  6. Right-click on the new package and select New > Class. Set a name to the class and check public static void main(String[] args), then click Finish.

    ../_images/tuto_microej_cli_module_files.PNG
  7. Locate the project files
    1. In the Package Explorer view, right-click on the project then click on Properties.
    2. Select Resource menu.
    3. Click on the arrow button on line Location to show the project in the system explorer.
    ../_images/tuto_microej_cli_module_location.PNG

  8. Open a terminal from this directory and type the following commands:

    git init --bare ~/hello_world.git
git init
git remote add origin ~/hello_world.git
git add com.example.hello-world
git commit -m "Add Hello World application"
git push --set-upstream origin master

Note

For more details about MicroEJ Applications development, refer to the Application Developer Guide.

Create a New Jenkins Job

Start by creating a new job, from the job template, for building our application.

  1. Go to Jenkins dashboard.
  2. Click on New Item.
  3. Set item name to Hello World.
  4. In Copy from field, type Template - MMM from Git (autocomplete enabled).
  5. Validate with Ok button.

The job configuration page opens, let’s replace all the TO_REPLACE placeholders from the job template with correct values:

  1. In General tab, set easyant.module.dir to value $WORKSPACE/com.example.hello-world.

    ../_images/tuto_microej_cli_jenkins_parameter.PNG

  2. In Source Code Management, edit Repository URL to ~/hello_world.git.

    ../_images/tuto_microej_cli_jenkins_git_hello.PNG

  3. Click on Save.

Build the “Hello World” Application

Let’s run the job!

In Jenkins’ Hello World dashboard, click on Build with Parameters, then click on Build.

Note

You can check the build progress by clicking on the build progress bar and showing the Console Output.

At the end of the build, the module is published to http://localhost:8081/artifactory/list/libs-snapshot-local/com/example/hello-world/.

Congratulations!

At this point of the tutorial:

  • Artifactory is hosting your module builds and MicroEJ modules.
  • Jenkins automates the build process using MicroEJ Module Manager.

The next steps recommended are:

  • Adapt Jenkins/Artifactory configuration to your ecosystem and development flow.

Appendix

This section discusses some of the customization options.

Customize Jenkins

Jenkins jobs are highly configurable, following options and values are recommended by MicroEJ, but they can be customized at your convenience.

In General tab:

  1. Check Discard old builds and set Max # of builds to keep value to 15.
  2. Click on Advanced button, and check Block build when upstream project is building.

In Build triggers tab:

  1. Check Poll SCM, and set a CRON-like value (for example H/30 * * * * to poll SCM for changes every 30 minutes).

In Post-build actions tab:

  1. Add post-build action Publish JUnit test result report:
  2. Set Test report XMLs to **/target~/test/xml/**/test-report.xml, **/target~/test/xml/**/*Test.xml.
  3. Check Retain long standard output/error.
  4. Check Do not fail the build on empty test results

Add a Self-Signed Certificate

In case your Artifactory instance uses a self-signed SSL certificate, you might fall into this error when fetching dependencies:

HttpClientHandler: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target url=[artifactory address]

The authority has to be added to the trust store of the JRE/JDK that is running Artifactory. Here is a way to do it:

  1. Install Keystore Explorer.
  2. Start Keystore Explorer, and open file [JDK home]/jre/lib/security/cacerts with the password changeit. You may not have the right to modify this file. Edit rights if needed before opening it.
  3. Click on Tools, then Import Trusted Certificate.
  4. Select your certificate.
  5. Save the cacerts file.

Customize target~ path

Some systems and toolchains don’t handle long path properly. A workaround for this issue is to move the build directory (that is, the target~ directory) closer to the root directory.

To change the target~ directory path, set the Build Options target.

In Advanced, expand Properties text field and set the target property to the path of your choice. For example:

target=C:/tmp/