MSBuild Tasks for Running Tests

Official Content
This documentation is valid for:

Running tests in batches is easy using MSBuild tasks. You can do it using the RunTests task.

Note: Instead of using this task by itself, we recommend starting using the GXtest.msbuild file that should be present on your GeneXus installation root folder, and modifying it to fit your own needs. You will see an example in the next article. 

1 - UpdateTestRefs (MSBuild Task)

This task must be used before executing RunTests, and RecordMockingData tasks. Tests to run must exist at the moment this task is used. After this task, the TestsRunner procedure must be built (see examples below).

Parameters:
Type: {All, Unit, Web, UI, SD} 
Specifies the type of tests to build before the execution. (before RunTests task only). Set:
All: to run all Unit tests first, then web UI tests, and at last all SD UI tests.
Unit: to run all Unit tests.
Web: to run all web UI tests.
SD: to run all SD tests.
UI: to run all web and SD UI tests.

TestObjects: List of test object names separated by semicolons: Unit tests, UI tests, and Test Suites
Specifies the tests to build before the execution.

ServerUserName: string
Specifies the server user name (local or gxtechnical)

ServerPasswordstring
Specifies the server password.

Examples of use in an MSBuild file:

<UpdateTestRefs Type="All" ServerUserName="gxtechnical\username" ServerPassword="password"/>
<BuildOne BuildCalled="true" ObjectName="TestsRunner"/>
<RunTests Type="All" DetailedResults="true" ServerUserName="gxtechnical\username" ServerPassword="password"/>
<UpdateTestRefs TestName="testExample" ServerUserName="local\admin" ServerPassword="password"/>
<BuildOne BuildCalled="true" ObjectName="TestsRunner"/>
<RecordMockingData TestName="testExample" ServerUserName="local\admin" ServerPassword="password">
<UpdateTestRefs TestObjects="MyUnitTest;MyWebUITest" DetailedResults=”true” ServerUserName="gxtechnical\username" ServerPassword="password"/>
<BuildOne BuildCalled="true" ObjectName="TestsRunner"/>
<RunTests TestObjects="MyUnitTest;MyWebUITest" ServerUserName="gxtechnical\username" ServerPassword="password"/>

2 - RunTests (MSBuild Task)

This task allows running all the tests by their type (Unit, UI, or both) in a specific Knowledge Base, or listed by their names using the TestObjects parameter.

Type: {All, Unit, Web, UI, SD} 
Specifies the type of test to run. Set:
All: to run all enabled Unit tests first, then enabled web UI tests, and at last, all enabled SD UI tests. Default value.
Unit: to run all enabled Unit tests.
Web: to run all enabled web UI tests.
SD: to run all enabled SD tests.
UI: to run all enabled web and SD UI tests.

TestObjects: List of test object names separated by semicolons(;): Unit tests, UI tests, and Test Suites
Specifies the tests to build before the execution.

Note: when used from an MSBuild step in Jenkins semicolons (;) must be escaped so they are correctly passed from Jenkins to MSBuild. There are two options:

1- Surround parameters with single quotes e.g.:

'/p:TestObjects="Test1;Test2;Test3"'

2- Use %3B instead of ; e.g.:

/p:TestObjects="Test1%3BTest2%3BTest3"

DetailedResults (optional): bool
A boolean parameter to set the verbosity of test results in the console output. 
If true, detailed results will be displayed on the console output.
If false, only a summary is shown. Default value.

LogLevel (optional): string
Specifies how much information is logged when executing web UI tests.
The default value is the one set in the Log Level property on the selected environment.

Browser (optional): {Chrome, IE, Firefox, Edge }
Specifies the browser name where web UI tests will run.
The default value is the one set in the Default Browser property on the selected KB.

FullPageScreenshot (optional): boolean
Specifies the default screenshot command behavior.
The default value is the one set in Full Page Screenshot property on the selected environment.

VerifyStopsExecution (optional): boolean
Specifies whether a failed Verify check will stop the test run.
The default value is the one set in Verify Stops Execution property on the selected environment.

BaseURL (optional): string
Specifies the base URL name where web UI tests will run on go commands, except when a full URL is used.
The default value is the one set in Base URL property on the selected environment.

FileUploadBasePath (optional): string
Specifies the base directory path to where files for web UI tests will be looked for in FileUploadBy commands.
The default value is the one set in the File Upload Base Path property on the selected environment.

Arguments (optional): string
Useful for setting properties for the browser at the moment it is opened. Each browser has its own set of properties defined although there are some common arguments for most of them: for example --headless to run tests without visible interface.
The default value is the one set in the Arguments property on the selected environment.

ScreenshotMode (optional): string
Specifies when the screenshots are taken and saved during web UI tests execution. 
The default value is the one set in the Screenshot saving mode property on the selected environment.

HtmlMode (optional): string
Specifies when the page's HTML is saved during web UI tests execution
The default value is the one set in the Html saving mode property on the selected environment.

ExecutedCount (output): numeric
Contains the total number of executed tests

SuccessCount (output): numeric
Contains the total number of successful tests (includes WarningCount)

WarningCount (output): numeric
Contains the total number of warned tests

SkippedCount (output): numeric
Contains the total number of skipped tests

ErrorCount (output): numeric
Contains the total number of failed tests (exception + error results).

ResultsFile (input/output) (optional): string 

A path where to save the GXtest XML results file. As output contains the path of the mentioned file.

ServerUserName: string
Specifies the server user name (local\username or gxtechnical\username)

ServerPasswordstring
Specifies the server password.

AllowFailedTests: boolean
true: the task will succeed even if a test failed.
false: if there are any failed tests, this task is set as failed. Default value.

Examples of use in an MSBuild file:

<RunTests Type="All" Browser="Chrome" BaseURL="http://mytestingenvironment/myApp/" FileUploadBasePath="/testingDir/" ServerUserName="gxtechnical\username" ServerPassword="password"/>
<RunTests Type="Unit" DetailedResults="true" ServerUserName="local\admin" ServerPassword="password"/>
<RunTests Type="UI" Browser="Firefox" BaseURL="http://mytestingenvironment/myApp/" ServerUserName="gxtechnical\username" ServerPassword="password" AllowFailedTests="true"/>

<RunTests TestObjects="BasicTestsSuite;SmokeTestsHomePage;LooseTest2" Browser="Firefox" Arguments="--headless" AllowFailedTests="true">
      <Output TaskParameter="ExecutedCount" PropertyName="ExecutedCount"/>
      <Output TaskParameter="SuccessCount" PropertyName="SuccessCount"/>
      <Output TaskParameter="ErrorCount" PropertyName="ErrorCount"/>
      <Output TaskParameter="WarningCount" PropertyName="WarningCount"/>
      <Output TaskParameter="SkippedCount" PropertyName="SkippedCount"/>
      <Output TaskParameter="ResultsFile" PropertyName="ResultsFile"/>
</RunTests>

3 - RunTestsList (MSBuild Task) DEPRECATED

Warning: This task was deprecated and will be removed in future versions.

4 - RecordMockingData (MSBuild task)

Mocking works by recording data (SQLs / results) used for a test. This information is stored (as mocking data) and will be used in all future executions of the test in different environments. 

Parameters:
TestName: string
Specifies the name of the test to record the mocking data for.

MockDataFilePath (output): string
The absolute path where the mocking data file was saved.

ServerUserName: string
Specifies the server user name (local or gxtechnical)

ServerPasswordstring
Specifies the server password.

Examples of use in an MSBuild file:

<RecordMockingData TestName="testExample"  ServerUserName="gxtechnical\username" ServerPassword="password">
    <Output TaskParameter="MockDataFilePath" PropertyName="MockPath"/>
</RecordMockingData>
<RecordMockingData TestName="testExample" ServerUserName="local\admin" ServerPassword="password">
    <Output TaskParameter="MockDataFilePath" PropertyName="MockPath"/>
</RecordMockingData>

5 - JUnitExportTests (MSBuild task)

Converts the last execution test results to JUnit format.

Parameters:

SourceResultsFile: string (optional)
A source GXtest XML results file to convert to JUnit XML format.


JUnitTestFilePath (input/output): string (optional)
An absolute directory path where the JUnit file will be saved / As output contains the JUnit file's absolute path.
If the directory path does not exist then GXtest will try to create it.
Passing an absolute file path where test results will be exported is supported.

ServerUserName: string
Specifies the server user name (local or gxtechnical).

ServerPasswordstring
Specifies the server password.

Examples of use in an MSBuild file:

<JUnitExportTests JUnitTestFilePath="C:/path" ServerUserName="gxtechnical\username" ServerPassword="password">  
    <Output TaskParameter="JUnitTestFilePath" PropertyName="JUnitTestFilePathOutput"/>
</JUnitExportTests>
<JUnitExportTests JUnitTestFilePath="C:/non-existingPath" ServerUserName="local\admin" ServerPassword="password"> 
    <Output TaskParameter="JUnitTestFilePath" PropertyName="JUnitTestFilePathOutput"/>
</JUnitExportTests>
<JUnitExportTests JUnitTestFilePath="C:/path/customName.xml" ServerUserName="local\admin" ServerPassword="password"> 
    <Output TaskParameter="JUnitTestFilePath" PropertyName="JUnitTestFilePathOutput"/>
</JUnitExportTests>

6 - ExportResultsToHTML (MSBuild task)

Converts the last execution test results to custom GXtest HTML format.

Parameters:
HTMLFilePath (input/output): string
An absolute directory path where the HTML file will be saved / As output contains the HTML file's absolute path.
If the directory path does not exist then GXtest will try to create it.
Passing an absolute file path where test results will be exported is supported.
If empty, the target directory used is a folder 'TestResults' located inside the KB path.

ExportType: string
All: all tests will be exported.
Failed: only failed test results (result is Error or Exception) will be exported.
Passed: only passed test results (result is OK or Warning) will be exported.
If empty, all test results are exported.

ServerUserName: string
Specifies the server user name (local or gxtechnical).

ServerPasswordstring
Specifies the server password.

Examples of use in an MSBuild file:

<ExportResultsToHTML ExportType="All" HTMLFilePath="C:\folder\" >
      <Output TaskParameter="HTMLFilePath" PropertyName="HTMLFilePath" />
</ExportResultsToHTML>
<ExportResultsToHTML HTMLFilePath="myPath\" >
      <Output TaskParameter="HTMLFilePath" PropertyName="HTMLFilePath" />
</ExportResultsToHTML>
<ExportResultsToHTML ExportType="Failed" HTMLFilePath="$(FilePath)" >
      <Output TaskParameter="HTMLFilePath" PropertyName="HTMLFilePath" />
</ExportResultsToHTML>

7 - ExportResultsToAllure (MSBuild task)

Converts GXtest execution results to Allure format. The goal is to have integration with tools that allow us to see execution results including successful steps and screenshots for UI commands.

Parameters:

SourceFile: string (optional)
Source GXtest formatted results file to be converted to Allure format. Default value: the path of the current opened KB.

TargetDir (input/output): string (optional)
An absolute directory path where the Allure file will be saved.
If empty, the target directory used is the same directory as the SourceFile parameter value.
If the directory path does not exist then GXtest will try to create it.
It is NOT supported by passing an absolute file path where test results will be exported, since Allure requires a UUID base file name.

Note: this directory is cleaned before the export of results so everything works fine at the Allure report level. Removed files are the ones that end on '-testsuite.xml' or contains '-attachment.'

Properties: string (optional)
Properties to be shown as environment variables on Allure reports. Typical values can be the KB path, KB version, and environment name, among others.

The format for this is Properties="var1=value1;var2=value2;var3=value3"

Note: when used from an MSBuild step in Jenkins semicolons (;) must be escaped so they are correctly passed from Jenkins to MSBuild. There are two options:

1- Surround parameters with single quotes e.g.:

'/p:OtherVariables="var1=value1;var2=value2;var3=value3"'

2- Use %3B instead of ; e.g.:

/p:OtherVariables="var1=value1%3Bvar2=value2%3Bvar3=value3"

ServerUserName: string
Specifies the server user name (local or gxtechnical).

ServerPasswordstring
Specifies the server password.

Examples of use in an MSBuild file:

<ExportResultsToAllure Properties="KB=$(KBPath);Environment=$(EnvironmentName);CustomFixedVar=customValue;$(OtherVariables)" >
      <Output TaskParameter="OutputFile" PropertyName="OutputFile" />
</ExportResultsToAllure>
<ExportResultsToAllure SourceFile="customPath/myFile.xml" >
      <Output TaskParameter="OutputFile" PropertyName="OutputFile" />
</ExportResultsToAllure>
// Jenkins example: invoked as /p:AllureResultsPath="$WORKSPACE\allure-results"
<ExportResultsAllure TargetDir="$(AllureResultsPath)" Properties="KB=$(KBPath);Environment=$(EnvironmentName);CustomFixedVar=customValue;$(OtherVariables)" >
      <Output TaskParameter="OutputFile" PropertyName="OutputFile" />
</ExportResultsToAllure>