MicroEJ Module Manager¶

Introduction¶

Modern electronic device design involves many parts and teams to collaborate to finally obtain a product to be sold on its market. MicroEJ encourages modular design which involves various stake holders: hardware engineers, UX designers, graphic designers, drivers/BSP engineers, software engineers, etc.

Modular design is a design technique that emphasizes separating the functionality of an application into independent, interchangeable modules. Each module contains everything necessary to execute only one aspect of the desired functionality. In order to have team members collaborate internally within their team and with other teams, MicroEJ provides a powerful modular design concept, with smart module dependencies, controlled by the MicroEJ Module Manager (MMM). MMM frees engineers from the difficult task of computing module dependencies. Engineers specify the bare minimum description of the module requirements.

The following schema introduces the main concepts detailed in this chapter.

MMM Overview¶

MMM is based on the following tools:

Apache Ivy (http://ant.apache.org/ivy) for dependencies resolution and module publication;
Apache EasyAnt (https://ant.apache.org/easyant/history/trunk/reference.html) for module build from source code.

Specification¶

MMM provides a non ambiguous semantic for dependencies resolution. Please consult the MMM specification available on https://developer.microej.com/packages/documentation/TLT-0831-SPE-MicroEJModuleManager-2.0-E.pdf.

Module Project Skeleton¶

In MicroEJ SDK, a new MicroEJ module project is created as follows:

Select File > New > Project…,
Select MicroEJ > Module Project [1],
Fill the module information (project name, module organization, name and revision),
Select one of the suggested skeletons depending on the desired module nature,
Click on Finish.

The project is created and a set of files and directories are generated from the selected skeleton.

Note

When an empty Eclipse project already exists or when the skeleton has to be created within an existing directory, the MicroEJ module is created as follows:

In the Package Explorer, click on the parent project or directory,
Select File > New > Other…,
Select EasyAnt > EasyAnt Skeleton.

Module Description File¶

A module description file is an Ivy configuration file named module.ivy, located at the root of each MicroEJ module project. It describes the module nature (also called build type) and dependencies to other modules.

<ivy-module version="2.0" xmlns:ea="http://www.easyant.org" xmlns:m="http://ant.apache.org/ivy/extra"
                          xmlns:ej="https://developer.microej.com" ej:version="2.0.0">
    <info organisation="[organisation]" module="[name]" status="integration" revision="[version]">
        <ea:build organisation="com.is2t.easyant.buildtypes" module="[buildtype_name]" revision="[buildtype_version]">
            <ea:property name="[buildoption_name]" value="[buildoption_value]"/>
        </ea:build>
    </info>

    <configurations defaultconfmapping="default->default;provided->provided">
        <conf name="default" visibility="public"/>
        <conf name="provided" visibility="public"/>
        <conf name="documentation" visibility="public"/>
        <conf name="source" visibility="public"/>
        <conf name="dist" visibility="public"/>
        <conf name="test" visibility="private"/>
    </configurations>

    <publications>
    </publications>

    <dependencies>
      <dependency org="[dep_organisation]" name="[dep_name]" rev="[dep_version]"/>
    </dependencies>
</ivy-module>

Enable MMM Semantic¶

The MMM semantic is enabled in a module by adding the MicroEJ XML namespace and the ej:version attribute in the ivy-module node:

<ivy-module xmlns:ej="https://developer.microej.com" ej:version="2.0.0">

Note

Multiple namespaces can be declared in the ivy-module node.

MMM semantic is enabled in the module created with the Module Project Skeleton.

Module Dependencies¶

Module dependencies are added to the dependencies node as follow:

<dependencies>
  <dependency org="[dep_organisation]" name="[dep_name]" rev="[dep_version]"/>
</dependencies>

When no matching rule is specified, the default matching rule is compatible.

Dependency Matching Rule¶

The following matching rules are specified by MMM:

Name	Range Notation	Semantic
compatible	[M.m.p-RC, (M+1).0.0-RC[	Equal or up to next major version. Default if not set.
equivalent	[M.m.p-RC, M.(m+1).0-RC [	Equal or up to next minor version
greaterOrEqual	[M.m.p-RC, ∞[	Equal or greater versions
perfect	[M.m.p-RC, M.m.(p+1)-RC[	Exact match (strong dependency)

Set the matching rule of a given dependency with ej:match="matching rule". For example:

<dependency org="[dep_organisation]" name="[dep_name]" rev="[dep_version]" ej:match="perfect" />

Dependency Visibility¶

A dependency declared public is transitively resolved by upper modules. The default when not set.
A dependency declared private is only used by the module itself, typically for:
- Bundling the content into the module
- Testing the module

The visibility is set by the configurations declared in the configurations node. For example:

<configurations defaultconfmapping="default->default;provided->provided">
    <conf name="[conf_name]" visibility="private"/>
</configurations>

The configuration of a dependency is specified by setting the conf attribute, for example:

<dependency org="[dep_organisation]" name="[dep_name]" rev="[dep_version]" conf="[conf_name]->*" />

Automatic Update Before Resolution¶

The Easyant plugin ivy-update can be used to automatically update the version (attribute rev) of every module dependencies declared.

<info organisation="[organisation]" module="[name]" status="integration" revision="[version]">
    <ea:plugin org="com.is2t.easyant.plugins" name="ivy-update" revision="1.+" />
</info>

When the plugin is enabled, for each module dependency, MMM will check the version declared in the module file and update it to the highest version available which satisfies the matching rule of the dependency.

Build Options¶

MMM build options can be set with:

<ea:property name="[buildoption_name]" value="[buildoption_value]"/>

The following build options are globally available:

Build Options¶
Property Name	Description	Default Value
`target`	Path of the build directory `target~`.	`${basedir}/target~`

Refer to the documentation of Module Natures for specific build options.

MicroEJ Module Manager Configuration¶

By default, when starting an empty workspace, MicroEJ SDK is configured to import dependencies from MicroEJ Central Repository and to publish built modules to a local directory. The repository configuration is stored in a settings file (ivysettings.xml), and the default one is located at $USER_HOME\.microej\microej-ivysettings-[VERSION].xml

Preferences Page¶

The MMM preferences page in the MicroEJ SDK is available at Window > Preferences > MicroEJ > Module Manager [1].

MMM Preferences Page¶

This page allows to configure the following elements:

Settings File: the file describing how to connect module repositories. See the settings file section.
Options: files declaring MMM options. See the Options section.
Use Module repository as Build repository: the settings file for connecting the build repository in place of the one bundled in MicroEJ SDK. This option shall not be enabled by default and is reserved for advanced configuration.
Build repository Settings File: the settings file for connecting the build repository in place of the one bundled in MicroEJ SDK. This option is automatically initialized the first time MicroEJ SDK is launched. It shall not be modified by default and is reserved for advanced configuration.
Set verbose mode: to enable advanced debug traces when building a module.
Runtime JRE: the Java Runtime Environment that executes the build process.
Max build history size: the maximum number of previous builds available in Build Module shortcut list:

Settings File¶

The settings file is an XML file that describes how MMM connects local or online module repositories. The file format is described in Apache Ivy documentation.

To configure MMM to a custom settings file (usually from an offline repository):

Set Settings file to a custom ivysettings.xml settings file [1],
Click on Apply and Close button

If the workspace is not empty, it is recommended to trigger a full resolution and rebuild all the projects using this new repository configuration:

Clean caches
- In the Package Explorer, right-click on a project;
- Select Ivy > Clean all caches.
Resolve projects using the new repository

To resolve all the workspace projects, click on the Resolve All button in the toolbar:

To only resolve a subset of the workspace projects:
- In the Package Explorer, select the desired projects,
- Right-click on a project and select Ivy > Clean all caches.
Trigger Add-On Library processors for automatically generated source code
- Select Project > Clean…,
- Select Clean all projects,
- Click on Clean button.

Options¶

Options can be used to parameterize a module description file or a settings file. Options are declared as key/value pairs in a standard Java properties file, and are expanded using the ${my_property} notation.

A typical usage in a settings file is for extracting repository server credentials, such as HTTP Basic access authentication:

Declare options in a properties file
Register this property file to MMM options

Use this option in a settings file

A typical usage in a module description file is for factorizing dependency versions across multiple modules projects:

Declare an option in a properties file
Register this property file to MMM options
Use this option in a module description file

Module Build¶

In MicroEJ SDK, the build of a MicroEJ module project can be started as follows:

In the Package Explorer, right-click on the project,

Select Build Module.

Module Build¶

The build of a module can take time depending on

the module nature to build,

the number and the size of module dependencies to download,

the repository connection bandwidth, …

The module build logs are redirected to the integrated console.

Alternatively, the build of a MicroEJ module project can be started from the build history:

Module Build History¶

Build Kit¶

The Module Manager Build Kit is a consistent set of tools, scripts, configuration and artifacts required for building modules in command-line mode. Starting from MicroEJ SDK 5.4.0, it also contains a Command Line Interface (CLI). The Build Kit allows to work in headless mode (e.g. in a terminal) and to build your modules using a Continuous Integration tool.

The Build Kit is bundled with MicroEJ SDK and can be exported using the following steps: [2]

Select File > Export > MicroEJ > Module Manager Build Kit,
Choose an empty Target directory,
Click on the Finish button.

Once the Build Kit is fully exported, the directory content shall look like:

/
├─ bin
│  ├─ mmm
│  └─ mmm.bat
├─ conf
├─ lib
├─ microej-build-repository
│  ├─ ant-contrib
│  ├─ com
│  ├─ ...
│  └─ ivysettings.xml
├─ microej-module-repository
│  └─ ivysettings.xml
└─ release.properties

Add the bin directory of the Build Kit directory to the PATH environment variable of your machine.
Make sure the JAVA_HOME environment variable is set and points to a JRE/JDK installation or that java executable is in the PATH environment variable (Java 8 is required)
Confirm that the installation works fine by executing the command mmm --version. The result should display the MMM CLI version.

The mmm tool can run on any supported Operating Systems:

on Windows, either in the command prompt using the Windows batch script mmm.bat or in MinGW environments such as Git BASH using the bash script mmm.
on Mac OS X and Linux distributions using the bash script mmm.

The build repository (microej-build-repository directory) contains scripts and tools for building modules. It is specific to a MicroEJ SDK version and shall not be modified by default.

The module repository (microej-module-repository directory) contains a default Settings File for importing modules from MicroEJ Central Repository and this local repository (modules that are locally built will be published to this directory). You can override with custom settings or by extracting an offline repository.

To go further with headless builds, please consult the next chapter for command line builds, and this tutorial to setup MicroEJ modules build in continuous integration environments.

[1]	(1, 2, 3) If using MicroEJ SDK versions lower than `5.2.0`, please refer to the following section.

[2]	If using MicroEJ SDK versions lower than `5.4.0`, please refer to the following section.

Command Line Interface¶

Starting from version 5.4.0, MicroEJ SDK provides a Command Line Interface (CLI). Please refer to the Build Kit section for installation details.

The following operations are supported by the MMM CLI:

creating a module project
cleaning a module project
building a module project
running a MicroEJ Application project on the Simulator
publishing a module in a module repository

Usage¶

In order to use the MMM CLI for your project:

go to the root directory of your project
run the following command
```
mmm [OPTION]... [COMMAND]
```

where COMMAND is the command to execute (for example mmm build). The available commands are:

help: display help information about the specified command
init: create a new project
clean: clean the project
build: build the project
publish: build the project and publish the module
run: run the MicroEJ Application project on the Simulator

The available options are:

--help (-h): show the help message and exit
--version (-V): print version information and exit
--build-repository-settings-file (-b): path of the Ivy settings file for build scripts and tools. Defaults to ${CLI_HOME}/microej-build-repository/ivysettings.xml.
--module-repository-settings-file (-r): path of the Ivy settings file for modules. Defaults to ${CLI_HOME}/microej-module-repository/ivysettings.xml.
--ivy-file (-f): path of the project’s Ivy file. Defaults to ./module.ivy.
--verbose (-v): verbose mode. Disabled by default. Add this option to enable verbose mode.
-Dxxx=yyy: any additional option passed as system properties.

When no command is specified, MMM CLI executes Easyant with custom targets using the --targets (-t) option (defaults to clean,package).

Shared configuration¶

In order to share configuration across several projects, these parameters can be defined in the file ${user.home}/.microej/.mmmconfig. This file uses the TOML format. Parameters names are the same than the ones passed as system properties, except the character _ is used as a separator instead of -. The parameters defined in the [options] section are passed as system properties. Here is an example:

build_repository_settings_file = "/home/johndoe/ivy-configuration/ivysettings.xml"
module_repository_settings_file = "/home/johndoe/ivy-configuration/ivysettings.xml"
ivy_file = "ivy.xml"

[options]
my.first.property = "value1"
my.second.property = "value2"

Warning

TOML values must be surrounded with double quotes
Backslash characters (\) must be doubled (for example a Windows path C:\\Users\\johndoe\\ivysettings.xml)

Command line options take precedence over those defined in the configuration file. So if the same option is defined in both locations, the value defined in the command line is used.

Commands¶

init

The command init creates a new project (executes Easyant with skeleton:generate target). The skeleton and project information must be passed with the following system properties:

skeleton.org: organisation of the skeleton module. Defaults to com.is2t.easyant.skeletons.
skeleton.module: name of the skeleton module. Mandatory, defaults to microej-javalib.
skeleton.rev: revision of the skeleton module. Mandatory, defaults to + (meaning the latest released version).
project.org: organisation of the project module. Mandatory, defaults to com.mycompany.
project.module: name of the project module. Mandatory, defaults to myproject.
project.rev: revision of the project module. Defaults to 0.1.0.
skeleton.target.dir: relative path of the project directory (created if it does not exist). Mandatory, defaults to the current directory.

For example

mmm init -Dskeleton.org=com.is2t.easyant.skeletons -Dskeleton.module=microej-javalib -Dskeleton.rev=4.2.8 -Dproject.org=com.mycompany -Dproject.module=myproject -Dproject.rev=1.0.0 -Dskeleton.target.dir=myproject

If one of these properties is missing, it will be asked in interactive mode:

$ mmm init -Dskeleton.org=com.is2t.easyant.skeletons -Dskeleton.module=microej-javalib -Dskeleton.rev=4.2.8 -Dproject.org=com.mycompany -Dproject.module=myproject -Dproject.rev=1.0.0

...

-skeleton:check-generate:
   [input] skipping input as property skeleton.org has already been set.
   [input] skipping input as property skeleton.module has already been set.
   [input] skipping input as property skeleton.rev has already been set.
   [input] The path where the skeleton project will be unzipped [/home/tdelhomenie/microej/working/skeleton]

To force the non-interactive mode, the property skeleton.interactive.mode must be set to false. In non-interactive mode the default values are used for missing non-mandatory properties, and the creation fails if mandatory properties are missing.

$ mmm init -Dskeleton.org=com.is2t.easyant.skeletons -Dskeleton.module=microej-javalib -Dskeleton.rev=4.2.8 -Dproject.org=com.mycompany -Dskeleton.target.dir=myproject -Dskeleton.interactive.mode=false

...

* Problem Report:

expected property 'project.module': Module name of YOUR project

clean

The command clean cleans the project (executes Easyant with clean target). For example

mmm clean

cleans the project.

build

The command build builds the project (executes Easyant with clean,package targets). For example

mmm build -f ivy.xml -v

builds the project with the Ivy file ivy.xml and in verbose mode.

publish

The command publish builds the project and publishes the module. This command accepts the publication target as a parameter, amongst these values:

local (default value): executes the clean,publish-local Easyant target, which publishes the project with the resolver referenced by the property local.resolver in the Settings File.
shared: executes the clean,publish-shared Easyant target, which publishes the project with the resolver referenced by the property shared.resolver in the Settings File.
release: executes the clean,release Easyant target, which publishes the project with the resolver referenced by the property release.resolver the Settings File.

For example

mmm publish local

builds the project and publishes the module using the local resolver.

run

The command run runs the application on the Simulator (executes Easyant with compile,simulator:run targets). It has the following requirements:

to run on the Simulator, the project must be configured with one of the following Module Natures:
the property application.main.class must be set to the Fully Qualified Name of the application main class (for example com.mycompany.Main)
a MicroEJ Platform must be provided (see Platform Selection section)
Application Options must be defined using properties file under in the build directory (see Using a Properties File section)
the module must have been built once before running the Simulator. So the mmm build command must be executed before running the Simulator the first time or after a project clean (mmm clean command).

Note

The next times, it is not required to rebuild the module if source code files have been modified. The content of src/main/java and src/main/resources folders are automatically compiled by mmm run command before running the Simulator.

For example

mmm run -D"platform-loader.target.platform.file"="/path/to/the/platform.zip"

runs the application on the given platform.

The Simulator can be launched in debug mode by setting the property execution.mode of the application file build/commons.properties to debug:

execution.mode=debug

The debug port can be defined with the property debug.port. Go to Simulator Debug options section for more details.

help

The command help displays the help for a command. For example

mmm help run

displays the help of the command run.

Troubleshooting¶

Unresolved Dependency¶

If the following message appears when resolving module dependencies:

:: problems summary ::
:::: WARNINGS
   module not found: com.mycompany#mymodule;[M.m.p-RC,M.m.(p+1)-RC[

             ::::::::::::::::::::::::::::::::::::::::::::::

             ::          UNRESOLVED DEPENDENCIES         ::

             ::::::::::::::::::::::::::::::::::::::::::::::

             :: com.mycompany#mymodule;[M.m.p-RC,M.m.(p+1)-RC[: not found

             ::::::::::::::::::::::::::::::::::::::::::::::

First, check that either a released module com.mycompany/mymodule/M.m.p or a snapshot module com.mycompany/mymodule/M.m.p-RCYYYYMMDD-HHMM exists in your module repository.

If the module does not exist,
- if it is declared as a direct dependency, the module repository is not compatible with your source code. You can either check if an other module version is available in the repository or add the missing module to the repository.
- otherwise, this is likely a missing transitive module dependency. The module repository is not consistent. Check the module repository settings file and that consistency check has been enabled during the module repository build (see Configure Consistency Check).
If the module exists, this may be either a configuration issue or a network connection error. We have to find the cause in the resolution logs with the verbose mode option enabled:

For URL repositories, find:
```
trying https://[MY_REPOSITORY_URL]/[MY_REPOSITORY_NAME]/com.mycompany/mymodule/
tried https://[MY_REPOSITORY_URL]/[MY_REPOSITORY_NAME]/com.mycompany/mymodule/
```
For filesystem repository, find:
```
trying [MY_REPOSITORY_PATH]/com.mycompany/mymodule/
tried [MY_REPOSITORY_PATH]/com.mycompany/mymodule/
```
If your module repository URL or filesystem path does not appear, check your settings file. This is likely a missing resolver.

Otherwise, if your module repository is an URL, this may be a network connection error between MMM (the client) and the module repository (the server). First, check for Invalid Certificate issue.

Otherwise, the next step is to debug at the HTTP level:
```
HTTP response status: [RESPONSE_CODE] url=https://[MY_REPOSITORY_URL]/com.mycompany/mymodule/
CLIENT ERROR: Not Found url=https://[MY_REPOSITORY_URL]/com.mycompany/mymodule/
```
Depending on the HTTP error code:
- 401 Unauthorized: check your settings file credentials configuration.
- 404 Not Found: add the following options to log raw HTTP traffic:
```
-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog -Dorg.apache.commons.logging.simplelog.showdatetime=true -Dorg.apache.commons.logging.simplelog.log.org.apache.http=DEBUG -Dorg.apache.commons.logging.simplelog.log.org.apache.http.wire=ERROR
```
  Particularly, Ivy requires the HTTP HEAD request which may be disabled by some servers.

Invalid Certificate¶

If the following message appears when resolving module dependencies:

HttpClientHandler: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target url=[artifactory address]

The server may use a self-signed certificate that has to be added to the JRE trust store that is running MicroEJ Module Manager. Here is a way to do it:

Install Keystore Explorer,
Start Keystore Explorer, and open file [JRE_HOME]/lib/security/cacerts or [JDK_HOME]/jre/lib/security/cacerts with the password changeit. You may not have the right to modify this file. Edit rights if needed before opening it,
Click on Tools, then Import Trusted Certificate,
Select your certificate,
Save the cacerts file.

If the problem still occurs, add the following option to enable SSL protocol traces:

-Djavax.net.debug=all

This is useful to detect advanced errors such as:

invalid certificate chain: one of root or intermediate certificate may be missing in the JRE/JDK truststore.
TLS protocol negotiation issues.

Target “simulator:run” does not exist¶

If the following message appears when executing the mmm run command:

* Problem Report:

Target "simulator:run" does not exist in the project "my-app".

it means that the command run is not supported by the build type declared by your module project. Make sure it is one of the following ones:

build-application, with version 7.1.0 or higher
build-microej-javalib, with version 4.2.0 or higher
build-firmware-singleapp, with version 1.3.0 or higher

Meta Build¶

A Meta Build is a module allowing to build other modules. It is typically used in a project containing multiple modules. The Meta Build module serves as an entry point to build all the modules of the project.

Meta Build creation¶

In the MicroeEJ SDK, select File > New > Module Project.

New Meta Build Project¶
Fill in the fields Project name, Organization, Module and Revision, then select the Skeleton named microej-meta-build
Click on Finish. A template project is automatically created and ready to use.

Meta Build configuration¶

The main element to configure in a meta build is the list of modules to build. This is done in 2 files, located at the root folder:

public.modules.list which contains the list of the modules relative paths to build and publish.
private.modules.list which contains the list of the modules relative paths to build. These modules are not published but only stored in a private and local repository in order to be fetched by the public modules.

The format of these files is a plain text file with one module path by line, for example:

module1
module2
module3

These paths are relative to the meta build root folder, which is set by default to the parent folder of the meta build module (..). For this reason, a meta build module is generally created at the same level of the other modules to build. Here is a typical structure of a meta build:

/
├─ module1
│  ├─ ...
│  └─ module.ivy
├─ module2
│  ├─ ...
│  └─ module.ivy
├─ module3
│  ├─ ...
│  └─ module.ivy
└─ metabuild
   ├─ private.modules.list
   ├─ public.modules.list
   └─ module.ivy

The modules build order is calculated based on the dependency information. If a module is a dependency of another module, it is built first.

For a complete list of configuration options, please refer to Meta Build Module Nature section.

Former MicroEJ SDK Versions (lower than 5.2.0)¶

This section describes MMM configuration elements for MicroEJ SDK versions lower than 5.2.0.

New MicroEJ Module Project¶

The New MicroEJ Module Project wizard is available at File > New > Project…, EasyAnt > EasyAnt Project.

Preferences Pages¶

MMM Preferences Pages are located in two dedicated pages. The following pictures show the options mapping using the same options numbers declared in Preferences Page.

Ivy Preferences Page¶

The Ivy Preferences Page is available at Window > Preferences > Ivy > Settings.

../_images/mmm_preferences_up_to_5-1_ivy_settings_annotated.png

Easyant Preferences Page¶

The Easyant Preferences Page is available at Window > Preferences > EasyAnt4Eclipse.

../_images/mmm_preferences_up_to_5-1_ea4eclipse_annotated.png

Build Kit¶

Create an empty directory (e.g. mmm_sdk_[version]_build_kit),
Locate your SDK installation plugins directory (by default, C:\Program Files\MicroEJ\MicroEJ SDK-[version]\rcp\plugins on Windows OS),
Open the file com.is2t.eclipse.plugin.easyant4e_[version].jar with an archive manager,
Extract the directory lib to the target directory,
Open the file com.is2t.eclipse.plugin.easyant4e.offlinerepo_[version].jar with an archive manager,
Navigate to directory repositories,
Extract the file named microej-build-repository.zip for MicroEJ SDK 5.x or is2t_repo.zip for MicroEJ SDK 4.1.x to the target directory.

Former MicroEJ SDK Versions (from 5.2.0 to 5.3.x)¶

Build Kit¶

The Build Kit is bundled with MicroEJ SDK and can be exported using the following steps:

Select File > Export > MicroEJ > Module Manager Build Kit,
Choose an empty Target directory,
Click on the Finish button.

Once the Build Kit is fully exported, the directory content shall look like: