Setup an Automated Build using Jenkins and Artifactory¶
This tutorial explains how to setup an environment for automating MicroEJ Module build and deployment using Jenkins and JFrog Artifactory.
Such environment setup facilitates continuous integration (CI) and continuous delivery (CD), which improves productivity across your development ecosystem, by automatically:
- building modules when source code changes
- saving build results
- reproducing builds
- archiving binary modules
The tutorial should take 2 hours to complete.
Intended Audience¶
The audience for this document is engineers who are in charge of integrating MicroEJ Module Manager (MMM) to their continuous integration environment.
In addition, this tutorial should be of interest to all developers wishing to understand how MicroEJ works with headless module builds.
For those who are only interested by command line module build, consider using the MMM Command Line Interface.
Introduction¶
The overall build and deployment flow of a module can be summarized as follows:
- Some event triggers the build process (i.e module source changed, user action, scheduled routine, etc.)
- The module source code is retrieved from the Source Control System
- The module dependencies are imported from the Repository Manager
- The Automation Server then proceeds to building the module
- If the build is successful, the module binary is deployed to the Repository Manager
Prerequisites¶
- MicroEJ SDK
5.4.0
or higher. - Git
2.x
installed, with Git executable in path. We recommend installing Git Bash if your operating system is Windows (https://git-for-windows.github.io/). - Java Development Kit (JDK)
1.8.x
.
This tutorial was tested with Jenkins 2.277.4
and Artifactory 7.24.3
.
Note
For SDK versions before 5.4.0, please refer to this MicroEJ Documentation Archive.
Overview¶
The next sections describe step by step how to setup the build environment and build your first MicroEJ module.
The steps to follow are:
- Install and setup MicroEJ build tools, Jenkins and Artifactory
- Create a Jenkins job template for MMM builds
- Create a simple MicroEJ module (Hello World)
- Create a new Jenkins job for the Hello World module
- Build the module
In order to simplify the steps, this tutorial will be performed locally on a single machine.
Artifactory will host MicroEJ modules in 3 repositories:
microej-module-repository
: repository initialized with pre-built MicroEJ modules, a mirror of the Central Repositorycustom-modules-snapshot
: repository where custom snapshot modules will be publishedcustom-modules-release
: repository where custom release modules will be published
Install the Build Tools¶
This section assumes the prerequisites have been properly installed.
- Locate your JDK installation directory (typically something like
C:\Program Files\Java\jdk1.8.0_[version]
on Windows). - Set the environment variable
JAVA_HOME
to point to this directory. - Set the environment variable
JRE_HOME
to point to thejre
directory (for exampleC:\Program Files\Java\jdk1.8.0_[version]\jre
). - Create a directory named
buildKit
. - Export the MicroEJ build kit from your MicroEJ SDK version to the
buildKit
directory, by following the steps described here. - Edit the file
buildKit/microej-module-repository/ivysettings.xml
and replace its content by:
<?xml version="1.0" encoding="UTF-8"?>
<ivy-settings>
<property name="artifactory.repository.url" value="http://localhost:8081/artifactory" override="false"/>
<property name="local.repository.dir" value="${user.home}/.ivy2/repository/" override="false"/>
<!--
Map MMM resolvers (*.resolver) to custom resolver
Kinds of repositories:
- release: used when publishing a released module.
- snapshot: used when publishing a snapshot module.
- local: used when publishing a snapshot module locally.
-->
<property name="release.resolver" value="modulesReleaseRepository" override="false"/>
<property name="shared.resolver" value="modulesSnapshotRepository" override="false"/>
<property name="local.resolver" value="localRepository" override="false"/>
<property name="modules.resolver" value="fetchAll" override="false" />
<property name="request.cache.dir" value="${user.home}/.ivy2/cache" override="false"/>
<property name="default.conflict.manager" value="latest-compatible" override="false"/>
<settings defaultResolver="${modules.resolver}" defaultConflictManager="${default.conflict.manager}" defaultResolveMode="dynamic"/>
<caches defaultCacheDir="${request.cache.dir}"/>
<resolvers>
<url name="modulesReleaseRepository" m2compatible="true">
<artifact pattern="${artifactory.repository.url}/custom-modules-release/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
<ivy pattern="${artifactory.repository.url}/custom-modules-release/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
</url>
<url name="modulesSnapshotRepository" m2compatible="true" checkmodified="true">
<artifact pattern="${artifactory.repository.url}/custom-modules-snapshot/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
<ivy pattern="${artifactory.repository.url}/custom-modules-snapshot/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
</url>
<url name="microejModulesRepository" m2compatible="true">
<artifact pattern="${artifactory.repository.url}/microej-module-repository/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
<ivy pattern="${artifactory.repository.url}/microej-module-repository/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
</url>
<filesystem name="localRepository" m2compatible="true" checkmodified="true">
<artifact pattern="${local.repository.dir}/[organization]/[module]/[branch]/[revision]/[artifact]-[revision](-[classifier]).[ext]" />
<ivy pattern="${local.repository.dir}/[organization]/[module]/[branch]/[revision]/ivy-[revision].xml" />
</filesystem>
<chain name="fetchRelease">
<resolver ref="modulesReleaseRepository"/>
<resolver ref="microejModulesRepository"/>
</chain>
<chain name="fetchSnapshot">
<resolver ref="modulesSnapshotRepository"/>
<resolver ref="fetchRelease"/>
</chain>
<chain name="fetchLocal">
<resolver ref="localRepository"/>
<resolver ref="fetchSnapshot"/>
</chain>
<chain name="fetchAll">
<resolver ref="fetchLocal"/>
</chain>
</resolvers>
</ivy-settings>
This file configures MicroEJ Module Manager to import and publish modules from the Artifactory repositories described in this tutorial. Please refer to Settings File section for more details.
Note
At this point, the content of the directory buildKit
should look like the following:
buildKit
├── bin
│ ├── mmm
│ └── mmm.bat
├── conf
│ └── easyant-conf.xml
├── lib
│ ├── ant.jar
│ └── ...
├── microej-build-repository
│ ├── ant-contrib
│ ├── be
│ └── ...
├── microej-module-repository
│ └── ivysettings.xml
└── release.properties
Get a Module Repository¶
A Module Repository is a portable ZIP file that bundles a set of modules for extending the MicroEJ development environment. Please consult the Module Repository section for more information.
This tutorial uses the MicroEJ Central Repository, which is the Module Repository used by MicroEJ SDK to fetch dependencies when starting an empty workspace. It bundles Foundation Library APIs and numerous Add-On Libraries.
Next step is to download a local copy of this repository:
- Visit the Central Repository on the MicroEJ Developer website.
- Navigate to the Working Offline section.
- Click on the offline repository link. This will download the Central Repository as a ZIP file.
Setup Artifactory¶
Install and Start Artifactory¶
- Download Artifactory here: https://jfrog.com/fr/open-source/ and select the appropriate package for your operating system.
- Unzip downloaded archive, then navigate to
app/bin
directory (by defaultartifactory-oss-[version]/app/bin
). - Run
artifactory.bat
orartifactory.sh
depending on your operating system. - Once Artifactory is started, go to
http://localhost:8081/
. - Login to Artifactory for the first time using the default
admin
account (Username:admin
, Password:password
). - On the Welcome wizard, set the administrator password, then click Next,
- Configure proxy server (if any) then click Next, or click Skip.
- Click on Finish.
Artifactory is up and running.
Configure Artifactory¶
For demonstration purposes we will allow anonymous users to deploy modules in the repositories:
- Go to Administration > Security > Settings.
- In the General Security Settings section, check Allow Anonymous Access.
- Click on Save.
- Go to Administration > Identity and Access > Permissions.
- Click on Anything entry (do not check the line), then go to Users tab
- Click on anonymous and check Deploy/Cache permission in the Repositories category.
- Click on Save and finish.
Next steps will involve uploading large files, so we have to augment the file upload maximum size accordingly:
- Go to Administration > Artifactory.
- In the General Settings section, change the value of File Upload In UI Max Size (MB) to
1024
then click on Save.
Create Repositories¶
We will now create and configure the repositories. Let’s start with the repository for the future built snapshot modules:
- Go to Administration > Repositories > Repositories in the left menu.
- Click on Add Repositories > Local Repository
- Select Maven.
- Set Repository Key field to
custom-modules-snapshot
and click on Save and Finish.
Repeat the same steps for the other repositories with the Repository Key field set to custom-modules-release
and microej-module-repository
.
Import MicroEJ Repositories¶
In this section, we will import MicroEJ repositories into Artifactory repositories to make them available to the build server.
- Go to Administration > Artifactory > Import & Export > Repositories.
- Scroll to the Import Repository from Zip section.
- As Target Local Repository, select
microej-module-repository
in the list. - Click on Select file and select the MicroEJ module repository zip file (
microej-[MicroEJ version]-[version].zip
) that you downloaded earlier (please refer to section Get a Module Repository). - Click Upload. At the end of upload, click on Import. Upload and import may take some time.
Artifactory is now hosting all required MicroEJ modules.
Go to Administration > Artifactory > Artifacts and check that the repository microej-module-repository
does contain modules as shown in the figure below.
Setup Jenkins¶
Install Jenkins¶
- Download Jenkins here: https://www.jenkins.io/download/. In this tutorial we will use the WAR (Web Archive), but you can use any other installation package (Docker, Ubuntu/Debian, …).
- Open a terminal and type the following command:
java -jar [path/to/downloaded/jenkinswar]/jenkins.war
. After initialization, the terminal will print out Jenkins is fully up and running. - Go to
http://localhost:8080/
. - To unlock Jenkins, copy/paste the generated password that has been written in the terminal log. Click on Continue.
- Select option Install suggested plugins and wait for plugins installation.
- Fill in the Create First Admin User form. Click Save and continue.
- Click on Save and finish, then on Start using Jenkins.
Configure Jenkins¶
First step is to configure the JDK and MMM paths:
- Go to Manage Jenkins > Global Tool Configuration.
- Add JDK installation:
- Scroll to JDK section.
- Click on Add JDK.
- Set Name to
JDK [jdk_version]
(for exampleJDK 1.8
). - Uncheck Install automatically.
- Set JAVA_HOME to the absolute path of your JDK installation (for example
C:\Program Files\Java\jdk1.8.0_[version]
on Windows).
- Click on Save.
- Go to Manage Jenkins > Configure System.
- Scroll to Global properties section.
- Check Environment variables.
- Click on Add.
- Set Name to
MICROEJ_BUILD_KIT_HOME
. - Set Value to the absolute path of the
buildKit
folder.
- Click on Save.
Create a Job Template¶
- Go to Jenkins dashboard.
- Click on New item to create a job template.
- Set item name to
Template - MMM from Git
. - Select Freestyle project.
- Click on Ok.
In General tab:
- Check This project is parameterized and add String parameter named
easyant.module.dir
with default value to$WORKSPACE/TO_REPLACE
. This will later point to the module sources.
In Source Code Management tab:
- Select Git source control:
- Set Repository URL value to
TO_REPLACE
, - Set Branch Specifier value to
origin/master
, - In Additional Behaviours, click on Add, select Advanced sub-modules behaviors, then check Recursively update submodules.
In Build tab:
For Windows, add build step Execute Windows batch command:
- In Command, set the following content:
cd "%easyant.module.dir%" "%MICROEJ_BUILD_KIT_HOME%\\bin\\mmm.bat" publish shared"
For Linux, add build step Execute shell:
- In Command, set the following content:
cd "${easyant.module.dir}" "${MICROEJ_BUILD_KIT_HOME}/bin/mmm" publish shared"
Finally, click on Save.
Build a new Module using Jenkins¶
Since your environment is now setup, it is time to build your first module from Jenkins and check it has been published to Artifactory. Let’s build an “Hello World” Sandboxed Application project.
Create a new MicroEJ Module¶
In this example, we will create a very simple module using the Sandbox Application buildtype (build-application
) that we’ll push to a Git repository.
Note
For demonstration purposes, we’ll create a new project and share it on a local Git bare repository. You can adapt the following sections to use an existing MicroEJ project and your own Git repository.
Start MicroEJ SDK.
Go to File > New > Sandboxed Application Project.
Fill in the template fields, set Project name to
com.example.hello-world
.Click Finish. This will create the project files and structure.
Right-click on source folder
src/main/java
and select New > Package. Set a name to the package and click Finish.Right-click on the new package and select New > Class. Set a name to the class and check
public static void main(String[] args)
, then click Finish.- Locate the project files
- In the Package Explorer view, right-click on the project then click on Properties.
- Select Resource menu.
- Click on the arrow button on line Location to show the project in the system explorer.
Open a terminal from this directory and type the following commands:
git init --bare ~/hello_world.git git init git remote add origin ~/hello_world.git git add com.example.hello-world git commit -m "Add Hello World application" git push --set-upstream origin master
Note
For more details about MicroEJ Applications development, refer to the Application Developer Guide.
Create a New Jenkins Job¶
Start by creating a new job, from the job template, for building our application.
- Go to Jenkins dashboard.
- Click on New Item.
- Set item name to
Hello World
. - In Copy from field, type
Template - MMM from Git
(autocomplete enabled). - Validate with Ok button.
The job configuration page opens, let’s replace all the TO_REPLACE
placeholders from the job template with correct values:
In General tab, set
easyant.module.dir
to value$WORKSPACE/com.example.hello-world
.In Source Code Management, edit Repository URL to
~/hello_world.git
.Click on Save.
Build the “Hello World” Application¶
Let’s run the job!
In Jenkins’ Hello World
dashboard, click on Build with Parameters, then click on Build.
Note
You can check the build progress by clicking on the build progress bar and showing the Console Output.
At the end of the build, the module is published to http://localhost:8081/artifactory/list/custom-modules-snapshot/com/example/hello-world/
.
Congratulations!
At this point of the tutorial:
- Artifactory is hosting your module builds and MicroEJ modules.
- Jenkins automates the build process using MicroEJ Module Manager.
The next recommended step is to adapt MMM/Jenkins/Artifactory configuration to your ecosystem and development flow.
Appendix¶
This section discusses some of the customization options.
Customize Jenkins¶
Jenkins jobs are highly configurable, following options and values are recommended by MicroEJ, but they can be customized at your convenience.
In General tab:
- Check Discard old builds and set Max # of builds to keep value to
15
. - Click on Advanced button, and check Block build when upstream project is building.
In Build triggers tab:
- Check Poll SCM, and set a CRON-like value (for example
H/30 * * * *
to poll SCM for changes every 30 minutes).
In Post-build actions tab:
- Add post-build action Publish JUnit test result report:
- Set Test report XMLs to
**/target~/test/xml/**/test-report.xml, **/target~/test/xml/**/*Test.xml
.
Note
The error message ‘**/target~/test/xml/**/test-report.xml’ doesn’t match anything: ‘**’ exists but not ‘**/target~/test/xml/**/test-report.xml’
will be displayed since no build has been executed yet. These folders will be generated during the build.
- Check Retain long standard output/error.
- Check Do not fail the build on empty test results
Customize target~
path¶
Some systems and toolchains don’t handle long path properly. A
workaround for this issue is to move the build directory (that is, the
target~
directory) closer to the root directory.
To change the target~
directory path, set the
build option target
.
In Advanced, expand Properties text field and
set the target
property to the path of your choice. For example:
target=C:/tmp/