tycho-surefire:test
Full name:
org.eclipse.tycho:tycho-surefire-plugin:3.0.3:test
Description:
Executes tests in an OSGi runtime.
The goal launches an OSGi runtime and executes the project's tests in that runtime. The "test runtime" consists of the bundle built in this project and its transitive dependencies, plus some Equinox and test harness bundles. The bundles are resolved from the target platform of the project. Note that the test runtime does typically not contain the entire target platform. If there are implicitly required bundles (e.g. org.apache.felix.scr
to make declarative services work), they need to be added manually through an extraRequirements
configuration on the target-platform-configuration
plugin.
Attributes:
- Requires a Maven project to be executed.
- Requires dependency resolution of artifacts in scope:
runtime
. - The goal is thread-safe and supports parallel builds.
- Binds by default to the lifecycle phase:
integration-test
.
Optional Parameters
Name | Type | Since | Description |
---|---|---|---|
<appArgLine> |
String |
- |
Arbitrary applications arguments to set on the command line. |
<application> |
String |
- |
Eclipse application to be run. If not specified, default application org.eclipse.ui.ide.workbench will be used. Application runnable will be invoked from test harness, not directly from Eclipse. Note that you need to ensure that the bundle which defines the configured application is included in the test runtime. |
<argLine> |
String |
- |
Arbitrary JVM options to set on the command line. User property is: tycho.testArgLine . |
<bundleStartLevel> |
BundleStartLevel[] |
- |
Bundle start level and auto start configuration used by the test runtime. Ignored if testRuntime is p2Installed . Example:
<bundleStartLevel> <bundle> <id>foo.bar.myplugin</id> <level>6</level> <autoStart>true</autoStart> </bundle> </bundleStartLevel> |
<debugOptions> |
String |
- |
if set to a non-null value, the platform is put in debug mode. If the value is a non-empty string it is interpreted as the location of the .options file. This file indicates what debug points are available for a plug-in and whether or not they are enabled. for a list of available options see https://git.eclipse.org/c/equinox/rt.equinox.framework.git/plain/bundles/org.eclipse.osgi/.options User property is: osgi.debug . |
<debugPort> |
int |
- |
Set this parameter to suspend the test JVM waiting for a client to open a remote debug session on the specified port. If further customization of JVM debug parameters is required then argLine can be used instead.User property is: debugPort . |
<defaultStartLevel> |
BundleStartLevel |
- |
The default bundle start level and auto start configuration used by the test runtime for bundles where the start level/auto start is not configured in bundleStartLevel . Ignored if testRuntime is p2Installed . Example:
<defaultStartLevel> <level>6</level> <autoStart>true</autoStart> </defaultStartLevel> |
<deleteOsgiDataDirectory> |
boolean |
- |
Whether to recursively delete the directory osgiDataDirectory before running the tests.Default value is: true . |
<dependencies> |
Dependency[] |
- |
Additional dependencies to be added to the test runtime. Ignored if testRuntime is p2Installed . Note: This parameter has only limited support for dependencies to artifacts within the reactor. Therefore it is recommended to specify extraRequirements on the target-platform-configuration plugin instead. Example:
<plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>target-platform-configuration</artifactId> <version>${tycho-version}</version> <configuration> <dependency-resolution> <extraRequirements> <requirement> <type>eclipse-feature</type> <id>example.project.feature</id> <versionRange>0.0.0</versionRange> </requirement> </extraRequirements> </dependency-resolution> </configuration> </plugin> extraRequirements are – together with the dependencies specified in the MANIFEST.MF of the project – transitively resolved against the target platform. The resulting set of bundles is included in the test runtime. |
<enableAssertions> |
boolean |
1.5.0 |
By default, Tycho Surefire disables JVM assertions for the execution of your test cases. To enable the assertions, set this flag to "true". Default value is: false .User property is: enableAssertions . |
<environmentVariables> |
Map<String,String> |
- |
Additional environments to set for the forked test JVM. |
<excludedGroups> |
String |
- |
(junit47 provider with JUnit4.8+ only) Excluded groups/categories (comma-separated). Any methods/classes/etc with one of the groups/categories specified in this list will specifically not be run. For JUnit, this parameter forces the use of the junit47 provider User property is: excludedGroups . |
<excludes> |
List<String> |
- |
List of patterns (separated by commas) used to specify the tests that should be excluded in testing. When not specified and when the test parameter is not specified, the default excludes will be **/*$* (which excludes all inner classes). |
<explodedBundles> |
String[] |
- |
List of bundles that must be expanded in order to execute the tests. Ignored if testRuntime is p2Installed . |
<failIfNoTests> |
boolean |
- |
If set to "false" the test execution will not fail in case there are no tests found. Default value is: true .User property is: failIfNoTests . |
<forkedProcessTimeoutInSeconds> |
int |
- |
Kill the forked test process after a certain number of seconds. If set to 0, wait forever for the process, never timing out. User property is: surefire.timeout . |
<frameworkExtensions> |
Dependency[] |
- |
List of framework extension bundles to add. Note: The goal does not automatically detect which bundles in the test runtime are framework extensions, but they have to be explicitly specified using this parameter. Ignored if testRuntime is p2Installed . |
<groups> |
String |
- |
(junit47 provider with JUnit4.8+ only) Groups/categories for this test (comma-separated). Only classes/methods/etc decorated with one of the group/category specified here will be included in test run, if specified. For JUnit, this parameter forces the use of the junit47 provider User property is: groups . |
<includes> |
List<String> |
- |
List of patterns (separated by commas) used to specify the tests that should be included in testing. When not specified and when the test parameter is not specified, the default includes will be **/Test*.java **/*Test.java **/*Tests.java **/*TestCase.java |
<osgiDataDirectory> |
File |
- |
OSGi data directory (osgi.instance.area , aka the workspace) of the Equinox runtime used to execute tests.Default value is: ${project.build.directory}/work/data/ . |
<parallel> |
ParallelMode |
0.16.0 |
(JUnit 4.7 provider) Supports values "classes"/"methods"/"both" to run in separate threads, as controlled by threadCount. User property is: parallel . |
<perCoreThreadCount> |
boolean |
0.16.0 |
(JUnit 4.7 provider) Indicates that threadCount is per cpu core. Default value is: true .User property is: perCoreThreadCount . |
<pluginArtifacts> |
List<Artifact> |
- |
(no description) User property is: plugin.artifacts . |
<printBundles> |
boolean |
- |
prints all loaded bundles Default value is: false .User property is: tycho.printBundles . |
<printWires> |
boolean |
- |
prints all loaded bundles Default value is: false .User property is: tycho.printWires . |
<product> |
String |
- |
Eclipse product to be run, i.e. -product parameter passed to test Eclipse runtime. |
<profileName> |
String |
0.19.0 |
p2 profile name of the installation under test. Only relevant if testRuntime is p2Installed . If tests are installed on top of an already existing installation in work , this must match the name of the existing profile.Default value is: DefaultProfile . |
<providerHint> |
String |
0.16.0 |
Normally tycho will automatically determine the test framework provider based on the test project's classpath. Use this to force using a test framework provider implementation with the given role hint. Tycho comes with providers "junit3","junit4","junit47","junit5". Note that when specifying a providerHint, you have to make sure the provider is actually available in the dependencies of tycho-surefire-plugin. |
<providerProperties> |
Properties |
0.16.0 |
Use this to specify surefire provider-specific properties. |
<reactorConcurrencyLevel> |
int |
- |
Configures the overall concurrency level. Important Note: If there are multiple mojo configurations this will choose the lowest configured number! So for example if in the same reactor there is one configuration with a concurrency level of 5 and one with a concurrency level of 3 then then only three parallel runs will be possible! |
<redirectTestOutputToFile> |
boolean |
- |
Set this to "true" to redirect the unit test standard output to a file (found in reportsDirectory/testName-output.txt). Default value is: false .User property is: maven.test.redirectTestOutputToFile . |
<reportsDirectory> |
File |
- |
Base directory where all reports are written to. Default value is: ${project.build.directory}/surefire-reports . |
<rerunFailingTestsCount> |
Integer |
- |
Flaky tests will re-run until they pass or the number of reruns has been exhausted. See surefire documentation for details.
Note: This feature is supported only for JUnit 4.x Default value is: 0 .User property is: surefire.rerunFailingTestsCount . |
<runOrder> |
String |
0.19.0 |
Defines the order the tests will be run in. Supported values are "alphabetical", "reversealphabetical", "random", "hourly" (alphabetical on even hours, reverse alphabetical on odd hours) and "filesystem". Default value is: filesystem . |
<showEclipseLog> |
boolean |
- |
Enables -consolelog for the test OSGi runtime Default value is: false .User property is: tycho.showEclipseLog . |
<skip> |
Boolean |
- |
Same as skipTests User property is: maven.test.skip . |
<skipAfterFailureCount> |
Integer |
- |
Skips the remaining tests after the Nth failure or error. See surefire documentation for details. Default value is: 0 .User property is: surefire.skipAfterFailureCount . |
<skipExec> |
boolean |
- |
Deprecated. User property is: maven.test.skipExec . |
<skipTests> |
Boolean |
- |
Set this to "true" to skip running tests, but still compile them. Its use is NOT RECOMMENDED, but quite convenient on occasion. Default: false User property is: skipTests . |
<suiteXmlFiles> |
List<String> |
- |
Only supported by the TestNG test provider. The values specified are passed to TestNG as test suite files. The suite files will overwrite the includes and excludes patterns. The path to the suite file(s) could be relative (test bundle classpath) or an absolute path to xml files outside the test bundle.
<configuration> <suiteXmlFiles> <suiteXmlFile>myTestSuite.xml</suiteXmlFile> </suiteXmlFiles> </configuration> |
<surefireProperties> |
File |
- |
(no description) Default value is: ${project.build.directory}/surefire.properties . |
<systemProperties> |
Map<String,String> |
- |
Additional system properties to set for the forked test JVM. |
<test> |
String |
- |
Specify this parameter if you want to use the test pattern matching notation, Ant pattern matching, to select tests to run. The Ant pattern will be used to create an include pattern formatted like **/${test}.java When used, the includes and excludes patterns parameters are ignoredUser property is: test . |
<testClass> |
String |
- |
Identifies a single test (suite) class to run. This is useful if you have a single JUnit test suite class defining which tests should be executed. Will be ignored if test is specified. Example:
<testClass>foo.bar.FooTest</testClass> User property is: testClass . |
<testClassesDirectory> |
File |
- |
The directory containing generated test classes of the project being tested. User property is: project.build.outputDirectory . |
<testFailureIgnore> |
boolean |
- |
Set this to true to ignore a failure during testing. Its use is NOT RECOMMENDED, but quite convenient on occasion. Default value is: false .User property is: maven.test.failure.ignore . |
<testRuntime> |
String |
0.19.0 |
How to create the OSGi test runtime. Allowed values are default and p2Installed . Mode p2Installed is EXPERIMENTAL - only works when installing products under test (see below).
<plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>tycho-surefire-plugin</artifactId> <version>${tycho-version}</version> <configuration> <testRuntime>p2Installed</testRuntime> </configuration> </plugin> <plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>target-platform-configuration</artifactId> <version>${tycho-version}</version> <configuration> <dependency-resolution> <extraRequirements> <!-- product IU under test --> <requirement> <type>p2-installable-unit</type> <id>example.product.id</id> <versionRange>0.0.0</versionRange> </requirement> </extraRequirements> </dependency-resolution> </configuration> </plugin> Default value is: default . |
<threadCount> |
int |
0.16.0 |
(JUnit 4.7 provider) The attribute thread-count allows you to specify how many threads should be allocated for this execution. Only makes sense to use in conjunction with the parallel parameter. User property is: threadCount . |
<trimStackTrace> |
boolean |
1.3.0 |
When true , stack traces are trimmed to only show lines within the test.Default value is: true .User property is: trimStackTrace . |
<useJDK> |
ToolchainProvider$JDKUsage |
- |
Which JDK to use for executing tests. Possible values are: SYSTEM , BREE .
In META-INF/MANIFEST.MF :
Bundle-RequiredExecutionEnvironment: JavaSE-1.7 <toolchains> <toolchain> <type>jdk</type> <provides> <id>JavaSE-1.7</id> </provides> <configuration> <jdkHome>/path/to/jdk/1.7</jdkHome> </configuration> </toolchain> </toolchains> Default value is: SYSTEM . |
<useUIHarness> |
boolean |
- |
Run tests using UI (true) or headless (false) test harness. Default value is: false . |
<useUIThread> |
boolean |
- |
Run tests in UI (true) or background (false) thread. Only applies to UI test harness. Default value is: true . |
<useUnlimitedThreads> |
boolean |
0.16.0 |
(JUnit 4.7 provider) Indicates that the thread pool will be unlimited. The parallel parameter and the actual number of classes/methods will decide. Setting this to "true" effectively disables perCoreThreadCount and threadCount. Default value is: false .User property is: useUnlimitedThreads . |
<work> |
File |
- |
Root directory (osgi.install.area) of the Equinox runtime used to execute tests. Default value is: ${project.build.directory}/work . |
Parameter Details
<appArgLine>
- Type:
java.lang.String
- Required:
No
<application>
- Type:
java.lang.String
- Required:
No
<argLine>
- Type:
java.lang.String
- Required:
No
- User Property:
tycho.testArgLine
<bundleStartLevel>
testRuntime
is p2Installed
. Example:
<bundleStartLevel> <bundle> <id>foo.bar.myplugin</id> <level>6</level> <autoStart>true</autoStart> </bundle> </bundleStartLevel>
- Type:
org.eclipse.sisu.equinox.launching.BundleStartLevel[]
- Required:
No
<debugOptions>
- Type:
java.lang.String
- Required:
No
- User Property:
osgi.debug
<debugPort>
argLine
can be used instead.- Type:
int
- Required:
No
- User Property:
debugPort
<defaultStartLevel>
bundleStartLevel
. Ignored if testRuntime
is p2Installed
. Example:
<defaultStartLevel> <level>6</level> <autoStart>true</autoStart> </defaultStartLevel>
- Type:
org.eclipse.sisu.equinox.launching.BundleStartLevel
- Required:
No
<deleteOsgiDataDirectory>
osgiDataDirectory
before running the tests.- Type:
boolean
- Required:
No
- Default:
true
<dependencies>
testRuntime
is p2Installed
. Note: This parameter has only limited support for dependencies to artifacts within the reactor. Therefore it is recommended to specify extraRequirements
on the target-platform-configuration
plugin instead. Example:
<plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>target-platform-configuration</artifactId> <version>${tycho-version}</version> <configuration> <dependency-resolution> <extraRequirements> <requirement> <type>eclipse-feature</type> <id>example.project.feature</id> <versionRange>0.0.0</versionRange> </requirement> </extraRequirements> </dependency-resolution> </configuration> </plugin>
extraRequirements
are – together with the dependencies specified in the MANIFEST.MF
of the project – transitively resolved against the target platform. The resulting set of bundles is included in the test runtime.- Type:
org.apache.maven.model.Dependency[]
- Required:
No
<enableAssertions>
- Type:
boolean
- Since:
1.5.0
- Required:
No
- User Property:
enableAssertions
- Default:
false
<environmentVariables>
- Type:
java.util.Map<java.lang.String, java.lang.String>
- Required:
No
<excludedGroups>
- Type:
java.lang.String
- Required:
No
- User Property:
excludedGroups
<excludes>
test
parameter is not specified, the default excludes will be **/*$*
(which excludes all inner classes).- Type:
java.util.List<java.lang.String>
- Required:
No
<explodedBundles>
testRuntime
is p2Installed
.- Type:
java.lang.String[]
- Required:
No
<failIfNoTests>
- Type:
boolean
- Required:
No
- User Property:
failIfNoTests
- Default:
true
<forkedProcessTimeoutInSeconds>
- Type:
int
- Required:
No
- User Property:
surefire.timeout
<frameworkExtensions>
testRuntime
is p2Installed
.- Type:
org.apache.maven.model.Dependency[]
- Required:
No
<groups>
- Type:
java.lang.String
- Required:
No
- User Property:
groups
<includes>
test
parameter is not specified, the default includes will be **/Test*.java **/*Test.java **/*Tests.java **/*TestCase.java
- Type:
java.util.List<java.lang.String>
- Required:
No
<osgiDataDirectory>
osgi.instance.area
, aka the workspace) of the Equinox runtime used to execute tests.- Type:
java.io.File
- Required:
No
- Default:
${project.build.directory}/work/data/
<parallel>
- Type:
org.eclipse.tycho.surefire.ParallelMode
- Since:
0.16.0
- Required:
No
- User Property:
parallel
<perCoreThreadCount>
- Type:
boolean
- Since:
0.16.0
- Required:
No
- User Property:
perCoreThreadCount
- Default:
true
<pluginArtifacts>
- Type:
java.util.List<org.apache.maven.artifact.Artifact>
- Required:
No
- User Property:
plugin.artifacts
<printBundles>
- Type:
boolean
- Required:
No
- User Property:
tycho.printBundles
- Default:
false
<printWires>
- Type:
boolean
- Required:
No
- User Property:
tycho.printWires
- Default:
false
<product>
- Type:
java.lang.String
- Required:
No
<profileName>
testRuntime
is p2Installed
. If tests are installed on top of an already existing installation in work
, this must match the name of the existing profile.- Type:
java.lang.String
- Since:
0.19.0
- Required:
No
- Default:
DefaultProfile
<providerHint>
- Type:
java.lang.String
- Since:
0.16.0
- Required:
No
<providerProperties>
- Type:
java.util.Properties
- Since:
0.16.0
- Required:
No
<reactorConcurrencyLevel>
- Type:
int
- Required:
No
<redirectTestOutputToFile>
- Type:
boolean
- Required:
No
- User Property:
maven.test.redirectTestOutputToFile
- Default:
false
<reportsDirectory>
- Type:
java.io.File
- Required:
No
- Default:
${project.build.directory}/surefire-reports
<rerunFailingTestsCount>
Note: This feature is supported only for JUnit 4.x
- Type:
java.lang.Integer
- Required:
No
- User Property:
surefire.rerunFailingTestsCount
- Default:
0
<runOrder>
- Type:
java.lang.String
- Since:
0.19.0
- Required:
No
- Default:
filesystem
<showEclipseLog>
- Type:
boolean
- Required:
No
- User Property:
tycho.showEclipseLog
- Default:
false
<skip>
skipTests
- Type:
java.lang.Boolean
- Required:
No
- User Property:
maven.test.skip
<skipAfterFailureCount>
- Type:
java.lang.Integer
- Required:
No
- User Property:
surefire.skipAfterFailureCount
- Default:
0
<skipExec>
- Type:
boolean
- Required:
No
- User Property:
maven.test.skipExec
<skipTests>
false
- Type:
java.lang.Boolean
- Required:
No
- User Property:
skipTests
<suiteXmlFiles>
includes
and excludes
patterns. The path to the suite file(s) could be relative (test bundle classpath) or an absolute path to xml files outside the test bundle.
<configuration> <suiteXmlFiles> <suiteXmlFile>myTestSuite.xml</suiteXmlFile> </suiteXmlFiles> </configuration>
- Type:
java.util.List<java.lang.String>
- Required:
No
<surefireProperties>
- Type:
java.io.File
- Required:
No
- Default:
${project.build.directory}/surefire.properties
<systemProperties>
- Type:
java.util.Map<java.lang.String, java.lang.String>
- Required:
No
<test>
**/${test}.java
When used, the includes
and excludes
patterns parameters are ignored- Type:
java.lang.String
- Required:
No
- User Property:
test
<testClass>
test
is specified. Example:
<testClass>foo.bar.FooTest</testClass>
- Type:
java.lang.String
- Required:
No
- User Property:
testClass
<testClassesDirectory>
- Type:
java.io.File
- Required:
No
- User Property:
project.build.outputDirectory
<testFailureIgnore>
- Type:
boolean
- Required:
No
- User Property:
maven.test.failure.ignore
- Default:
false
<testRuntime>
default
and p2Installed
. Mode p2Installed
is EXPERIMENTAL - only works when installing products under test (see below).
- In
default
mode, all necessary files to define the test runtime likeconfig.ini
are generated by tycho. This installation mode has the advantage that the test runtime is minimal (defined by the transitive dependencies of the test bundle plus and the test harness) and existing bundle jars are referenced rather than copied for the installation - In
p2Installed
mode, use p2 director to install test bundle, test harness bundles and respective dependencies. This installation mode can be used for integration tests that require a fully p2-provisioned installation. To install a product IU, add it as extra requirement to the test bundle (see example below). Note that this installation mode comes with a certain performance overhead for executing the provisioning operations otherwise not required.
<plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>tycho-surefire-plugin</artifactId> <version>${tycho-version}</version> <configuration> <testRuntime>p2Installed</testRuntime> </configuration> </plugin> <plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>target-platform-configuration</artifactId> <version>${tycho-version}</version> <configuration> <dependency-resolution> <extraRequirements> <!-- product IU under test --> <requirement> <type>p2-installable-unit</type> <id>example.product.id</id> <versionRange>0.0.0</versionRange> </requirement> </extraRequirements> </dependency-resolution> </configuration> </plugin>
- Type:
java.lang.String
- Since:
0.19.0
- Required:
No
- Default:
default
<threadCount>
- Type:
int
- Since:
0.16.0
- Required:
No
- User Property:
threadCount
<trimStackTrace>
true
, stack traces are trimmed to only show lines within the test.- Type:
boolean
- Since:
1.3.0
- Required:
No
- User Property:
trimStackTrace
- Default:
true
<useJDK>
SYSTEM
, BREE
.
- SYSTEM: Use the currently running JVM (or from toolchain if configured in pom.xml)
- BREE: use MANIFEST header
Bundle-RequiredExecutionEnvironment
to lookup the JDK from toolchains.xml. The value of BREE will be matched against the id of the toolchain elements in toolchains.xml.
In
META-INF/MANIFEST.MF
:
Bundle-RequiredExecutionEnvironment: JavaSE-1.7
<toolchains> <toolchain> <type>jdk</type> <provides> <id>JavaSE-1.7</id> </provides> <configuration> <jdkHome>/path/to/jdk/1.7</jdkHome> </configuration> </toolchain> </toolchains>
- Type:
org.eclipse.tycho.core.maven.ToolchainProvider$JDKUsage
- Required:
No
- Default:
SYSTEM
<useUIHarness>
- Type:
boolean
- Required:
No
- Default:
false
<useUIThread>
- Type:
boolean
- Required:
No
- Default:
true
<useUnlimitedThreads>
- Type:
boolean
- Since:
0.16.0
- Required:
No
- User Property:
useUnlimitedThreads
- Default:
false
<work>
- Type:
java.io.File
- Required:
No
- Default:
${project.build.directory}/work