VEE Port Qualification

Introduction

A VEE Port integrates one or more Foundation Libraries with their respective Abstraction Layers.

VEE Port Qualification is the process of validating the conformance of the Abstraction Layer that implements the Low Level APIs of a Foundation Library.

../_images/qualification-overview.png

VEE Port Qualification Overwiew

For each Low Level API, an Abstraction Layer implementation is required. The validation of the Abstraction Layer implementation is performed by running tests at two-levels:

  • In C, by calling Low Level APIs (usually manually).
  • In Java, by calling Foundation Library APIs (usually automatically using VEE Port Test Suite).

The following figure depicts an example for the FS Pack:

../_images/qualification-test-suite-fs.png

VEE Port Qualification Example for FS Pack

MicroEJ provides a set of tools and pre-defined projects aimed at simplifying the steps for validating VEE Ports in the form of the VEE Port Qualification Tools (PQT).

VEE Port Qualification Tools Overview

The VEE Port Qualification Tools provide the following components:

Please refer to the VEE Port Qualification Tools README for more details and the location of the components.

VEE Port Test Suite

The purpose of a VEE Port Test Suite is to validate the Abstraction Layer that implements the Low Level APIs of a Foundation Libraries by automatically running Java tests on the device.

The MicroEJ Test Suite Engine is used for building, running a Test Suite, and providing a report.

A Test Suite contains one or more tests. For each test, the Test Suite Engine will:

  1. Build an Executable for the test.
  2. Run the Executable onto the device.
  3. Retrieve the execution traces.
  4. Analyze the traces to determine whether the test has PASSED or FAILED.
  5. Append the result to the Test Report.
  6. Repeat until all tests of the Test Suite have been executed.
VEE Port Test Suite on Device Overview

VEE Port Test Suite on Device Overview

Create a VEE Port Test Suite

A VEE Port Test Suite is composed of two projects:

  • the Test Suite module: the project that contains test cases. Test cases are written in Junit. When this project is built, it produces a versionned library. See Test Suite Versioning for available Test Suite modules for the most common Packs provided by MicroEJ Corp.
  • the Test Suite runner: the project that contains the configuration for its execution on a VEE Port. When this project is built, it runs the Test Suite on a Device and generates the Test Suite report.

Note

Creating a VEE Port Test Suite requires SDK 5.6.0 or higher.

Create the Test Suite Module

The Test Suite module contains the tests of the Foundation Library to be qualified.

Create the Test Suite Module Project

A new Test Suite module is created using the microej-javaimpl Skeleton (see Foundation Library Implementation).

To create the Test Suite module, click on: File > New > Project… then select MicroEJ > Module Project

Fill up the following fields of the form:

  • Project name (e.g: myFoundationLib-testsuite).
  • Organization (e.g: com.mycompany).
  • Module (e.g: myFoundationLib-testsuite).
  • Revision (version of your Test Suite module).
  • Select the Skeleton: microej-javaimpl.

Then, create two test source folders:

  • Right-click on your project.
  • Click on: New > Source Folder.
  • Fill up the Folder name field of the form with: src/test/java and for the second folder: src/test/resources.

You should get a Foundation Library Test Suite project that looks like:

Foundation Library Test Suite Project Skeleton

Foundation Library Test Suite Project Skeleton

Your Test Suite module project is created and ready to be setup.

Configure the Test Suite Module Project

Open the module.ivy file and follow steps below:

  • Edit the module ivy-module > info > ea:build node to update rip.printableName:

    <ea:build organisation="com.is2t.easyant.buildtypes" module="build-microej-javaimpl" microej.lib.name="myFoundationLib-testsuite-1.0" rip.printableName="myFoundationLib Test Suite Impl" revision="5.2.+">
    
  • Add the following properties in the ivy-module > info node:

    <ea:property name="skip.test" value="set"/>
    <ea:property name="target.main.classes" value="${basedir}/target~/test/classes"/>
    <ea:property name="addon-processor.src.test.java.path.ref.name" value="src.java.path"/>
    
  • Update the JUnit dependency to:

    <dependency org="ej.library.test" name="junit" rev="1.7.1" conf="default;test->*"/>
    
  • Add a module.ant file at the root of the Test Suite project with the following content:

    <project>
            <target name="BuildTestTarget" extensionOf="abstract-compile:compile-ready" depends="resources-std:copy-test-resources">
                    <augment id="src.java.path">
                            <path location="${basedir}/src/test/java" />
                            <path location="${target}/adpgenerated/src-adpgenerated/junit/java"/>
                    </augment>
            </target>
    </project>
    

Note

An error on module.ant file can occurred with message Target resources-std:copy-test-resources does not exist in this project. Please ignore it.

Create a New Test Case

Right click on src/test/java, then click on New > Class. Fill Name: with the MyTest and then click on Finish. Copy/paste the following example in MyTest.java file:

import org.junit.Assert;
import org.junit.Test;

public class MyTest {

        @Test
        public static void Test() {
                Assert.assertTrue(true);
        }
}

The console output on the Simulator for this test should be:

=============== [ Initialization Stage ] ===============
=============== [ Launching on Simulator ] ===============
OK: Test
PASSED: 1
=============== [ Completed Successfully ] ===============

SUCCESS

Build the Test Suite Module

Once the test cases are implemented, you can build the module. The next step is to create a Test Suite Runner. The Test Suite Runner will fetch the Test Suite Module dependency.

Create the Test Suite Runner

The Test Suite runner project contains configuration files for running a Test Suite module on a Device using a VEE Port.

Create the Test Suite Runner Project

  • To create the Test Suite runner project, click on: File > New > Other… > MicroEJ > Module Project.

  • Fill up the following fields of the form:

    • Project name
    • Organization
    • Module
    • Revision (version of your Test Suite module)
    • Select the Skeleton: microej-testsuite
  • Inside the module.ivy file, add the dependency to the Test Suite module as following:

    <dependency org="com.mycompany" name="myFoundationLib-testsuite" rev="0.1.0" conf="test->default;provided->provided"/>
    
  • Inside the module.ant, add the following ANT target to configure trace redirection options :

    <target name="tracefile:init" extensionOf="abstract-test:test-ready">
            <!-- Set the launch.test.trace.file when the testsuite.trace.ip properties is not set -->
            <condition property="microej.testsuite.properties.launch.test.trace.file">
                    <not>
                            <isset property="microej.testsuite.properties.testsuite.trace.ip" />
                    </not>
            </condition>
    </target>
    
  • Create the file override.module.ant at the root of the project. Add the following content to configure the load of testsuite options:

    <project name="myFoundationlib.testsuite.override" xmlns:ac="antlib:net.sf.antcontrib">
            <!-- Load options from 'local.properties' beside this file -->
            <ac:if>
                    <available file="local.properties" type="file"/>
                    <ac:then>
                            <property file="local.properties"/>
                    </ac:then>
            </ac:if>
            <!-- Load options from 'config.properties' beside this file -->
            <property file="config.properties"/>
    </project>
    
  • Create the following .properties files:

Note

{PROJECT_LOC} refers here to the location of your Test Suite runner project.

Configure and Run the Test Suite

Follow the Run a Test Suite on a Device tutorial to configure your VEE Port and run the Test Suite on your Device.

Test Suite Versioning

Foundation Libraries are integrated in a VEE Port using Packs (see Pack Import). Use the Test Suite version compliant with the API version provided by the Foundation Library to validate the Abstraction Layer implementation. For example, the Test Suite FS module 3.0.3 should be used to validate the Abstraction Layer implementation of the Low Level API FS provided by the FS Pack 5.1.2.

Note

A Pack can provide several Foundation Libraries.

Core Engine

Core Engine Validation
Architecture Test Suite
7.0.0 or higher Core Engine Test Suite

UI Pack

UI Validation
UI Pack C Test Suite
13.0.0 or higher (UI3) Graphical User Interface Test Suite
[6.0.0-12.1.5] (UI2) Graphical User Interface Test Suite

FS Pack

FS API Implementation and Validation
FS Pack FS API Java Test Suite
[6.0.0-6.1.0[ 2.1.1 3.0.8
[5.1.2-5.2.0[ 2.0.6 3.0.3
[4.0.0-4.1.0[ 2.0.6 On demand [1]

BLUETOOTH Pack

BLUETOOTH API Implementation and Validation
BLUETOOTH Pack BLUETOOTH API Java Test Suite
2.1.0 2.1.0 2.0.0
2.0.1 2.0.0 2.0.0

NET Pack

NET, SSL and SECURITY APIs Implementations and Validations
NET Pack NET API SSL API SECURITY API NET Java Test Suite SSL Java Test Suite SECURITY Java Test Suite
[8.1.2-8.2.0] 1.1.0 2.1.0 N/A 3.4.0 (On demand [1]) 3.0.1 (On demand [1]) N/A
9.0.0 1.1.0 2.2.0 1.3.1 3.4.0 (On demand [1]) 3.1.4 (On demand [1]) 1.1.0 (On demand [1])
[9.0.1-9.4.1] 1.1.1 2.2.0 1.3.1 3.5.2 (On demand [1]) 3.1.4 (On demand [1]) 1.1.0 (On demand [1])
[10.0.0-10.5.0] 1.1.4 2.2.3 1.4.2 4.1.1 4.0.1 1.3.1
[11.0.1-11.0.2] 1.1.4 2.2.3 1.4.2 4.1.1 4.0.2 1.3.1

EVENT QUEUE Pack

EVENT QUEUE API Implementation and Validation
EVENT QUEUE Pack EVENT QUEUE API Java Test Suite
2.0.1 2.0.0 2.0.0
[1](1, 2, 3, 4, 5, 6, 7, 8, 9) Test Suite available on demand, please contact MicroEJ Support.