tycho-surefire:integration-test
Full name:
org.eclipse.tycho:tycho-surefire-plugin:2.7.5:integration-test
Description:
Executes integration-tests in an OSGi runtime.
The goal launches an OSGi runtime and executes the project's
integration-tests (default patterns are PluginTest*.class,
*IT.class
) in that runtime. The "test runtime" consists of
the bundle built in this project and its transitive dependencies,
plus some Equinox and test harness bundles. The bundles are
resolved from the target platform of the project. Note that the
test runtime does typically not contain the entire target
platform. If there are implicitly required bundles (e.g.
org.apache.felix.scr
to make declarative services work),
they need to be added manually through an
extraRequirements
configuration on the
target-platform-configuration
plugin.
This goal adopts the maven-failsafe paradigm, that works in the following way:
pre-integration-test
phase could be used to prepare any prerequisite (e.g. starting web-server, files, ...)integration-test
phase does not fail the build if there are test failures but a summary file is writtenpost-integration-test
could be used to cleanup/tear down any resources from thepre-integration-test
phase- test outcome is checked in the
verify
phase that might fail the build if there are test failures
summary files are generated according to the default maven-surefire-plugin for integration with tools that already work with maven-surefire-plugin (e.g. CI servers)
Attributes:
- Requires a Maven project to be executed.
- Requires dependency resolution of artifacts in scope:
test
. - The goal is thread-safe and supports parallel builds.
- Binds by default to the lifecycle phase:
integration-test
.
Required Parameters
Name | Type | Since | Description |
---|---|---|---|
<reportDirectory> |
File |
- |
(no description) Default value is: ${project.build.directory}/failsafe-reports . |
<summaryFile> |
File |
- |
(no description) Default value is: ${project.build.directory}/failsafe-reports/failsafe-summary.xml . |
Optional Parameters
Name | Type | Since | Description |
---|---|---|---|
<appArgLine> |
String |
- |
Arbitrary applications arguments to set on the command line. |
<application> |
String |
- |
Eclipse application to be run. If not specified, default
application org.eclipse.ui.ide.workbench will be used. Application
runnable will be invoked from test harness, not directly from
Eclipse. Note that you need to ensure that the bundle which defines
the configured application is included in the test runtime. |
<argLine> |
String |
- |
Arbitrary JVM options to set on the command line. User property is: tycho.testArgLine . |
<bundleStartLevel> |
BundleStartLevel[] |
- |
Bundle start level and auto start configuration used by the test
runtime. Ignored if testRuntime is
p2Installed . Example:
<bundleStartLevel> <bundle> <id>foo.bar.myplugin</id> <level>6</level> <autoStart>true</autoStart> </bundle> </bundleStartLevel> |
<debugOptions> |
String |
- |
if set to a non-null value, the platform is put in debug mode. If
the value is a non-empty string it is interpreted as the location
of the .options file. This file indicates what debug points are
available for a plug-in and whether or not they are enabled. for a
list of available options see https://git.eclipse.org/c/equinox/rt.equinox.framework.git/plain/bundles/org.eclipse.osgi/.options User property is: osgi.debug . |
<debugPort> |
int |
- |
Set this parameter to suspend the test JVM waiting for a client to
open a remote debug session on the specified port. If further
customization of JVM debug parameters is required then
argLine can be used instead.User property is: debugPort . |
<defaultStartLevel> |
BundleStartLevel |
- |
The default bundle start level and auto start configuration used by
the test runtime for bundles where the start level/auto start is
not configured in bundleStartLevel . Ignored if
testRuntime is p2Installed . Example:
<defaultStartLevel> <level>6</level> <autoStart>true</autoStart> </defaultStartLevel> |
<deleteOsgiDataDirectory> |
boolean |
- |
Whether to recursively delete the directory
osgiDataDirectory before running the tests.Default value is: true . |
<dependencies> |
Dependency[] |
- |
Additional dependencies to be added to the test runtime. Ignored if
testRuntime is p2Installed . Note: This
parameter has only limited support for dependencies to artifacts
within the reactor. Therefore it is recommended to specify
extraRequirements on the
target-platform-configuration plugin instead. Example:
<plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>target-platform-configuration</artifactId> <version>${tycho-version}</version> <configuration> <dependency-resolution> <extraRequirements> <requirement> <type>eclipse-feature</type> <id>example.project.feature</id> <versionRange>0.0.0</versionRange> </requirement> </extraRequirements> </dependency-resolution> </configuration> </plugin> extraRequirements are -
together with the dependencies specified in the
MANIFEST.MF of the project - transitively resolved against
the target platform. The resulting set of bundles is included in
the test runtime. |
<enableAssertions> |
boolean |
1.5.0 |
By default, Tycho Surefire disables JVM assertions for the
execution of your test cases. To enable the assertions, set this
flag to "true". Default value is: false .User property is: enableAssertions . |
<environmentVariables> |
Map |
- |
Additional environments to set for the forked test JVM. |
<excludedGroups> |
String |
- |
(junit47 provider with JUnit4.8+ only) Excluded groups/categories
(comma-separated). Any methods/classes/etc with one of the
groups/categories specified in this list will specifically not be
run. For JUnit, this parameter forces the use of the junit47
provider User property is: excludedGroups . |
<excludes> |
List |
- |
List of patterns (separated by commas) used to specify the tests
that should be excluded in testing. When not specified and when the
test parameter is not specified, the default excludes
will be **/*$* (which excludes all inner classes). |
<explodedBundles> |
String[] |
- |
List of bundles that must be expanded in order to execute the
tests. Ignored if testRuntime is
p2Installed . |
<forkedProcessTimeoutInSeconds> |
int |
- |
Kill the forked test process after a certain number of seconds. If
set to 0, wait forever for the process, never timing out. User property is: surefire.timeout . |
<frameworkExtensions> |
Dependency[] |
- |
List of framework extension bundles to add. Note: The goal does not
automatically detect which bundles in the test runtime are
framework extensions, but they have to be explicitly specified
using this parameter. Ignored if testRuntime is
p2Installed . |
<groups> |
String |
- |
(junit47 provider with JUnit4.8+ only) Groups/categories for this
test (comma-separated). Only classes/methods/etc decorated with one
of the group/category specified here will be included in test run,
if specified. For JUnit, this parameter forces the use of the
junit47 provider User property is: groups . |
<includes> |
List |
- |
List of patterns (separated by commas) used to specify the tests
that should be included in testing. When not specified and when the
test parameter is not specified, the default includes
will be **/Test*.java **/*Test.java **/*Tests.java
**/*TestCase.java |
<osgiDataDirectory> |
File |
- |
OSGi
data directory (osgi.instance.area , aka the
workspace) of the Equinox runtime used to execute tests.Default value is: ${project.build.directory}/work/data/ . |
<parallel> |
ParallelMode |
0.16.0 |
(JUnit 4.7 provider) Supports values "classes"/"methods"/"both" to
run in separate threads, as controlled by threadCount. User property is: parallel . |
<perCoreThreadCount> |
boolean |
0.16.0 |
(JUnit 4.7 provider) Indicates that threadCount is per cpu core. Default value is: true .User property is: perCoreThreadCount . |
<pluginArtifacts> |
List |
- |
(no description) User property is: plugin.artifacts . |
<printBundles> |
boolean |
- |
prints all loaded bundles Default value is: false .User property is: tycho.printBundles . |
<product> |
String |
- |
Eclipse product to be run, i.e. -product parameter passed to test
Eclipse runtime. |
<profileName> |
String |
0.19.0 |
p2 profile
name of the installation under test. Only relevant if
testRuntime is p2Installed . If tests are
installed on top of an already existing installation in
work , this must match the name of the existing
profile.Default value is: DefaultProfile . |
<providerHint> |
String |
0.16.0 |
Normally tycho will automatically determine the test framework
provider based on the test project's classpath. Use this to force
using a test framework provider implementation with the given role
hint. Tycho comes with providers
"junit3","junit4","junit47","junit5". Note that when specifying a
providerHint, you have to make sure the provider is actually
available in the dependencies of tycho-surefire-plugin. |
<providerProperties> |
Properties |
0.16.0 |
Use this to specify surefire provider-specific properties. |
<redirectTestOutputToFile> |
boolean |
- |
Set this to "true" to redirect the unit test standard output to a
file (found in reportsDirectory/testName-output.txt). Default value is: false .User property is: maven.test.redirectTestOutputToFile . |
<rerunFailingTestsCount> |
Integer |
- |
Flaky tests will re-run until they pass or the number of reruns has
been exhausted. See surefire documentation for details.
Note: This feature is supported only for JUnit 4.x Default value is: 0 .User property is: surefire.rerunFailingTestsCount . |
<runOrder> |
String |
0.19.0 |
Defines the order the tests will be run in. Supported values are
"alphabetical", "reversealphabetical", "random", "hourly"
(alphabetical on even hours, reverse alphabetical on odd hours) and
"filesystem". Default value is: filesystem . |
<showEclipseLog> |
boolean |
- |
Enables -consolelog for the test OSGi runtime Default value is: false .User property is: tycho.showEclipseLog . |
<skip> |
Boolean |
- |
Same as skipTests User property is: maven.test.skip . |
<skipAfterFailureCount> |
Integer |
- |
Skips the remaining tests after the Nth failure or error. See
surefire documentation for details. Default value is: 0 .User property is: surefire.skipAfterFailureCount . |
<skipExec> |
boolean |
- |
Deprecated. Use skipTests instead. User property is: maven.test.skipExec . |
<skipITs> |
boolean |
- |
(no description) User property is: skipITs . |
<skipTests> |
Boolean |
- |
Set this to "true" to skip running tests, but still compile them.
Its use is NOT RECOMMENDED, but quite convenient on occasion.
Default: false User property is: skipTests . |
<suiteXmlFiles> |
List |
- |
Only supported by the TestNG test provider. The values specified
are passed to TestNG as test suite files. The suite files will
overwrite the includes and excludes
patterns. The path to the suite file(s) could be relative (test
bundle classpath) or an absolute path to xml files outside the test
bundle.
<configuration> <suiteXmlFiles> <suiteXmlFile>myTestSuite.xml</suiteXmlFile> </suiteXmlFiles> </configuration> |
<surefireProperties> |
File |
- |
(no description) Default value is: ${project.build.directory}/surefire.properties . |
<systemProperties> |
Map |
- |
Additional system properties to set for the forked test JVM. |
<test> |
String |
- |
Specify this parameter if you want to use the test pattern matching
notation, Ant pattern matching, to select tests to run. The Ant
pattern will be used to create an include pattern formatted like
**/${test}.java When used, the includes
and excludes patterns parameters are ignoredUser property is: test . |
<testClass> |
String |
- |
Identifies a single test (suite) class to run. This is useful if
you have a single JUnit test suite class defining which tests
should be executed. Will be ignored if test is
specified. Example:
<testClass>foo.bar.FooTest</testClass> User property is: testClass . |
<testClassesDirectory> |
File |
- |
The directory containing generated test classes of the project
being tested. User property is: project.build.testOutputDirectory . |
<testRuntime> |
String |
0.19.0 |
How to create the OSGi test runtime. Allowed values are
default and p2Installed . Mode
p2Installed is EXPERIMENTAL - only works when
installing products under test (see below).
<plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>tycho-surefire-plugin</artifactId> <version>${tycho-version}</version> <configuration> <testRuntime>p2Installed</testRuntime> </configuration> </plugin> <plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>target-platform-configuration</artifactId> <version>${tycho-version}</version> <configuration> <dependency-resolution> <extraRequirements> <!-- product IU under test --> <requirement> <type>p2-installable-unit</type> <id>example.product.id</id> <versionRange>0.0.0</versionRange> </requirement> </extraRequirements> </dependency-resolution> </configuration> </plugin> Default value is: default . |
<threadCount> |
int |
0.16.0 |
(JUnit 4.7 provider) The attribute thread-count allows you to
specify how many threads should be allocated for this execution.
Only makes sense to use in conjunction with the parallel parameter. User property is: threadCount . |
<trimStackTrace> |
boolean |
1.3.0 |
When true , stack traces are trimmed to only show lines
within the test.Default value is: true .User property is: trimStackTrace . |
<useJDK> |
ToolchainProvider$JDKUsage |
- |
Which JDK to use for executing tests. Possible values are:
SYSTEM , BREE .
In META-INF/MANIFEST.MF :
Bundle-RequiredExecutionEnvironment: JavaSE-1.7 <toolchains> <toolchain> <type>jdk</type> <provides> <id>JavaSE-1.7</id> </provides> <configuration> <jdkHome>/path/to/jdk/1.7</jdkHome> </configuration> </toolchain> </toolchains> Default value is: SYSTEM . |
<useUIHarness> |
boolean |
- |
Run tests using UI (true) or headless (false) test harness. Default value is: false . |
<useUIThread> |
boolean |
- |
Run tests in UI (true) or background (false) thread. Only applies
to UI test harness. Default value is: true . |
<useUnlimitedThreads> |
boolean |
0.16.0 |
(JUnit 4.7 provider) Indicates that the thread pool will be
unlimited. The parallel parameter and the actual number of
classes/methods will decide. Setting this to "true" effectively
disables perCoreThreadCount and threadCount. Default value is: false .User property is: useUnlimitedThreads . |
<work> |
File |
- |
Root directory (osgi.install.area)
of the Equinox runtime used to execute tests. Default value is: ${project.build.directory}/work . |
Parameter Details
<appArgLine>
- Type:
java.lang.String
- Required:
No
<application>
- Type:
java.lang.String
- Required:
No
<argLine>
- Type:
java.lang.String
- Required:
No
- User Property:
tycho.testArgLine
<bundleStartLevel>
testRuntime
is
p2Installed
. Example:
<bundleStartLevel> <bundle> <id>foo.bar.myplugin</id> <level>6</level> <autoStart>true</autoStart> </bundle> </bundleStartLevel>
- Type:
org.eclipse.sisu.equinox.launching.BundleStartLevel[]
- Required:
No
<debugOptions>
- Type:
java.lang.String
- Required:
No
- User Property:
osgi.debug
<debugPort>
argLine
can be used instead.- Type:
int
- Required:
No
- User Property:
debugPort
<defaultStartLevel>
bundleStartLevel
. Ignored if
testRuntime
is p2Installed
. Example:
<defaultStartLevel> <level>6</level> <autoStart>true</autoStart> </defaultStartLevel>
- Type:
org.eclipse.sisu.equinox.launching.BundleStartLevel
- Required:
No
<deleteOsgiDataDirectory>
osgiDataDirectory
before running the tests.- Type:
boolean
- Required:
No
- Default:
true
<dependencies>
testRuntime
is p2Installed
. Note: This
parameter has only limited support for dependencies to artifacts
within the reactor. Therefore it is recommended to specify
extraRequirements
on the
target-platform-configuration
plugin instead. Example:
<plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>target-platform-configuration</artifactId> <version>${tycho-version}</version> <configuration> <dependency-resolution> <extraRequirements> <requirement> <type>eclipse-feature</type> <id>example.project.feature</id> <versionRange>0.0.0</versionRange> </requirement> </extraRequirements> </dependency-resolution> </configuration> </plugin>
extraRequirements
are -
together with the dependencies specified in the
MANIFEST.MF
of the project - transitively resolved against
the target platform. The resulting set of bundles is included in
the test runtime.- Type:
org.apache.maven.model.Dependency[]
- Required:
No
<enableAssertions>
- Type:
boolean
- Since:
1.5.0
- Required:
No
- User Property:
enableAssertions
- Default:
false
<environmentVariables>
- Type:
java.util.Map
- Required:
No
<excludedGroups>
- Type:
java.lang.String
- Required:
No
- User Property:
excludedGroups
<excludes>
test
parameter is not specified, the default excludes
will be **/*$*
(which excludes all inner classes).- Type:
java.util.List
- Required:
No
<explodedBundles>
testRuntime
is
p2Installed
.- Type:
java.lang.String[]
- Required:
No
<forkedProcessTimeoutInSeconds>
- Type:
int
- Required:
No
- User Property:
surefire.timeout
<frameworkExtensions>
testRuntime
is
p2Installed
.- Type:
org.apache.maven.model.Dependency[]
- Required:
No
<groups>
- Type:
java.lang.String
- Required:
No
- User Property:
groups
<includes>
test
parameter is not specified, the default includes
will be **/Test*.java **/*Test.java **/*Tests.java
**/*TestCase.java
- Type:
java.util.List
- Required:
No
<osgiDataDirectory>
osgi.instance.area
, aka the
workspace) of the Equinox runtime used to execute tests.- Type:
java.io.File
- Required:
No
- Default:
${project.build.directory}/work/data/
<parallel>
- Type:
org.eclipse.tycho.surefire.ParallelMode
- Since:
0.16.0
- Required:
No
- User Property:
parallel
<perCoreThreadCount>
- Type:
boolean
- Since:
0.16.0
- Required:
No
- User Property:
perCoreThreadCount
- Default:
true
<pluginArtifacts>
- Type:
java.util.List
- Required:
No
- User Property:
plugin.artifacts
<printBundles>
- Type:
boolean
- Required:
No
- User Property:
tycho.printBundles
- Default:
false
<product>
- Type:
java.lang.String
- Required:
No
<profileName>
testRuntime
is p2Installed
. If tests are
installed on top of an already existing installation in
work
, this must match the name of the existing
profile.- Type:
java.lang.String
- Since:
0.19.0
- Required:
No
- Default:
DefaultProfile
<providerHint>
- Type:
java.lang.String
- Since:
0.16.0
- Required:
No
<providerProperties>
- Type:
java.util.Properties
- Since:
0.16.0
- Required:
No
<redirectTestOutputToFile>
- Type:
boolean
- Required:
No
- User Property:
maven.test.redirectTestOutputToFile
- Default:
false
<reportDirectory>
- Type:
java.io.File
- Required:
Yes
- Default:
${project.build.directory}/failsafe-reports
<rerunFailingTestsCount>
Note: This feature is supported only for JUnit 4.x
- Type:
java.lang.Integer
- Required:
No
- User Property:
surefire.rerunFailingTestsCount
- Default:
0
<runOrder>
- Type:
java.lang.String
- Since:
0.19.0
- Required:
No
- Default:
filesystem
<showEclipseLog>
- Type:
boolean
- Required:
No
- User Property:
tycho.showEclipseLog
- Default:
false
<skip>
skipTests
- Type:
java.lang.Boolean
- Required:
No
- User Property:
maven.test.skip
<skipAfterFailureCount>
- Type:
java.lang.Integer
- Required:
No
- User Property:
surefire.skipAfterFailureCount
- Default:
0
<skipExec>
- Type:
boolean
- Required:
No
- User Property:
maven.test.skipExec
<skipITs>
- Type:
boolean
- Required:
No
- User Property:
skipITs
<skipTests>
false
- Type:
java.lang.Boolean
- Required:
No
- User Property:
skipTests
<suiteXmlFiles>
includes
and excludes
patterns. The path to the suite file(s) could be relative (test
bundle classpath) or an absolute path to xml files outside the test
bundle.
<configuration> <suiteXmlFiles> <suiteXmlFile>myTestSuite.xml</suiteXmlFile> </suiteXmlFiles> </configuration>
- Type:
java.util.List
- Required:
No
<summaryFile>
- Type:
java.io.File
- Required:
Yes
- Default:
${project.build.directory}/failsafe-reports/failsafe-summary.xml
<surefireProperties>
- Type:
java.io.File
- Required:
No
- Default:
${project.build.directory}/surefire.properties
<systemProperties>
- Type:
java.util.Map
- Required:
No
<test>
**/${test}.java
When used, the includes
and excludes
patterns parameters are ignored- Type:
java.lang.String
- Required:
No
- User Property:
test
<testClass>
test
is
specified. Example:
<testClass>foo.bar.FooTest</testClass>
- Type:
java.lang.String
- Required:
No
- User Property:
testClass
<testClassesDirectory>
- Type:
java.io.File
- Required:
No
- User Property:
project.build.testOutputDirectory
<testRuntime>
default
and p2Installed
. Mode
p2Installed
is EXPERIMENTAL - only works when
installing products under test (see below).
- In
default
mode, all necessary files to define the test runtime likeconfig.ini
are generated by tycho. This installation mode has the advantage that the test runtime is minimal (defined by the transitive dependencies of the test bundle plus and the test harness) and existing bundle jars are referenced rather than copied for the installation - In
p2Installed
mode, use p2 director to install test bundle, test harness bundles and respective dependencies. This installation mode can be used for integration tests that require a fully p2-provisioned installation. To install a product IU, add it as extra requirement to the test bundle (see example below). Note that this installation mode comes with a certain performance overhead for executing the provisioning operations otherwise not required.
<plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>tycho-surefire-plugin</artifactId> <version>${tycho-version}</version> <configuration> <testRuntime>p2Installed</testRuntime> </configuration> </plugin> <plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>target-platform-configuration</artifactId> <version>${tycho-version}</version> <configuration> <dependency-resolution> <extraRequirements> <!-- product IU under test --> <requirement> <type>p2-installable-unit</type> <id>example.product.id</id> <versionRange>0.0.0</versionRange> </requirement> </extraRequirements> </dependency-resolution> </configuration> </plugin>
- Type:
java.lang.String
- Since:
0.19.0
- Required:
No
- Default:
default
<threadCount>
- Type:
int
- Since:
0.16.0
- Required:
No
- User Property:
threadCount
<trimStackTrace>
true
, stack traces are trimmed to only show lines
within the test.- Type:
boolean
- Since:
1.3.0
- Required:
No
- User Property:
trimStackTrace
- Default:
true
<useJDK>
SYSTEM
, BREE
.
- SYSTEM: Use the currently running JVM (or from toolchain if configured in pom.xml)
- BREE: use MANIFEST header
Bundle-RequiredExecutionEnvironment
to lookup the JDK from toolchains.xml. The value of BREE will be matched against the id of the toolchain elements in toolchains.xml.
In
META-INF/MANIFEST.MF
:
Bundle-RequiredExecutionEnvironment: JavaSE-1.7
<toolchains> <toolchain> <type>jdk</type> <provides> <id>JavaSE-1.7</id> </provides> <configuration> <jdkHome>/path/to/jdk/1.7</jdkHome> </configuration> </toolchain> </toolchains>
- Type:
org.eclipse.tycho.core.maven.ToolchainProvider$JDKUsage
- Required:
No
- Default:
SYSTEM
<useUIHarness>
- Type:
boolean
- Required:
No
- Default:
false
<useUIThread>
- Type:
boolean
- Required:
No
- Default:
true
<useUnlimitedThreads>
- Type:
boolean
- Since:
0.16.0
- Required:
No
- User Property:
useUnlimitedThreads
- Default:
false
<work>
- Type:
java.io.File
- Required:
No
- Default:
${project.build.directory}/work