Menu

Integrate TestNG with Universal Agent

In this article we will walk you through an example how to integrate, execute and schedule test execution for your TestNG test project with Universal Agent.

Notes

Prerequisites

  1. Activate Automation Integration
  2. Install and Register the Automation Host
  3. You machine needs to have Java installed. The Java version should be the one compatible with what you use to run your TestNG project
  4. This example use Maven to execute the tests. Make sure you install maven to the automation host machine
  5. Your machine needs to have git installed.
  6. Along with Maven, this example also uses maven-compiler-plugin and maven-surefire-plugin to execute tests and generate report. Refer to the IMPORTANT NOTES section to configure the pom.xml file when you integrate your actual JUnit for Java project with Universal Agent

Step 1: Create a new Universal Agent for your TestNG project

Access to your Automation Host UI. Click on Add button to create a new agent. The New Agent dialog will be shown.

create-agent.png

Next, enter the following to the New Agent dialog:

General Information

  • Agent Name: enter name of the agent, e.g. TestNG
  • qTest Manager Project: select a qTest Manager project from which the agent is going to execute scheduled tests, e.g. qConnect Sample Project
  • Agent Type: select Universal Agent

Pre-Execute Script

We are going to configure a pre-execute script to either clone the sample TestNG project from github or pull the latest updates of that project if it already exists to the host machine.

Enter the script below to the Pre-Execute Script editor, which is specific to the Operating System that the automation host is running:

Windows

if not exist "D:\testng-sample" (
 cd /d D:\
 git clone https://github.com/QASymphony/testng-sample
) else (
 cd /d "D:\testng-sample"
 git pull --all
)

Linux or Mac

#!/bin/bash
if [ ! -d "/usr/local/var/testng-sample" ]
then
 cd /usr/local/var
 git clone https://github.com/QASymphony/testng-sample
else
 cd /usr/local/var/testng-sample
 git pull --all
fi

Execute Command

Executor: select node as the script executor

Working Directory

  • If your host machine is running on Linux or Mac, enter /usr/local/var/testng-sample
  • If your host machine is running on Windows, enter D:\testng-sample

Execute Command

We are going to create a script written in NodeJS to:

  • obtain the automation content(s) of scheduled test runs by resolving the value of magic variable TESTCASES_AC. Refer to this article to learn more about MAGIC VARIABLES in Universal Agent
  • if the value of TESTCASES_AC variable is empty, which means there is no test runs being scheduled to be executed (e.g. when this Universal Agent is executed in the first time), we will let the agent execute the whole tests in our TestNG project then create test run logs to qTest Manager as the execution result
  • if TESTCASES_AC variable has value, which are the automation contents of scheduled test runs, we will instruct Universal Agent to only execute tests that match those automation contents

Notes:

  • make sure you install maven to your host machine for the command to be executed successfully by Universal Agent.
  • the execute command will need to specify full path to maven executable. You can execute the following command to find the Maven executable path from either Terminal (on Linux/Mac) or Command Prompt (Windows), in bold.
    $ mvn -v
    Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T14:41:47-04:00)
    Maven home: /usr/local/opt/apache-maven-3.5.4
    Java version: 10.0.2, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk-10.0.2.jdk/Contents/Home
    Default locale: en_US, platform encoding: UTF-8
    OS name: "mac os x", version: "10.14.2", arch: "x86_64", family: "mac"
    

Enter the code snippet below to the Execute Command editor dependent on the OS that the automation host is running.

 

Windows

const fs = require("fs");
const { execSync } = require("child_process");

// NOTE: change the value to reflect the actual path to maven executable in your host machine let mavenExePath = 'C:\\apache-maven-3.5.4\\bin\\mvn'; // get automation content from magic variable TESTCASES_AC let testcases_AC = $TESTCASES_AC; // print automation content(s) to the execution log console.log ('*** testcases_AC: ' + testcases_AC); // we are executing the test with maven // the default command will include 'clean' goal to make sure test result folder are clean let command = `"${mavenExePath}" clean test`; // if the universal agent execution is kicked off by a job scheduled, // update the command to execute the test matching automation content(s) if (testcases_AC && testcases_AC.length > 0) { command = `"${mavenExePath}" -Dtest="${testcases_AC}" clean test`; } console.log('*** executing command: ' + command); // execute the test execSync(command, {stdio: 'inherit'}); console.log('*** execution completed ***');

Linux or Mac

const fs = require("fs");
const { execSync } = require("child_process");

// NOTE: change the value to reflect the actual path to maven executable in your host machine let mavenExePath = '/usr/local/opt/apache-maven-3.5.4/bin/mvn'; // get automation content from magic variable TESTCASES_AC let testcases_AC = $TESTCASES_AC; // print automation content(s) to the execution log console.log ("*** testcases_AC: " + testcases_AC); // we are executing the test with maven // the default command will include 'clean' goal to make sure test result folder are clean let command = `"${mavenExePath}" clean test`; // if the universal agent execution is kicked off by a job scheduled, // update the command to execute the test matching automation content(s) if (testcases_AC && testcases_AC.length > 0) { command = `"${mavenExePath}" -Dtest="${testcases_AC}" clean test`; } console.log('*** executing command: ' + command); // execute the test execSync(command, {stdio: "inherit"}); console.log('*** execution completed ***');

Path to Results

Enter the value below to specify the path to the test results generated when the execution completes.

  • If your host machine is running on Linux or Mac: /usr/local/var/testng-sample/target/surefire-reports/junitreports
  • If your host machine is running on Windows: D:\testng-sample\target\surefire-reports\junitreports

Result Parser

Select JUnit for Java as the Result Parser.

Select SAVE to finish creating the agent. Below is how TestNG agent looks like, on Mac.

testng-universal-agent.png

Step 2: Execute the Agent

From Agents list, locate the TestNG agent and click on Run Now button.

run-now.png

The TestNG agent will display.

testng-execution-dialog.png

Click on the Execute button to execute the agent. Once execution completes, the Console Log shows all the test results have been submitted to qTest Manager, as highlighted in the screenshot below.

testng-execution-complete-1.png

Next, access to qTest Manager. Select qConnect Sample Project then go to Test Execution module. You'll see the test results submitted to qTest Manager as Test Runs under a Test Suite naming Automation 2018-12-18, as below.

submitted-test-runs.png

Now we are going to schedule test execution for some specific test runs with the TestNG agent and verify it only executes tests that match the scheduled test runs.

Step 3: Schedule Test Execution for specific Test Runs

From qTest Manager, select the project qConnect Sample Project then click on Test Execution tab. Do the followings:

  1. Select the newly created test suite, in our example, it is Automation 2018-12-18.
  2. From the test run list on the right, select the first three (3) test runs
  3. Click on MORE button, then select Schedule

schedule-tests.png

On SCHEDULE AUTOMATION TEST EXECUTION dialog, enter the followings:

  1. Name: name of the schedule, e.g. Execute Specific Tests with TestNG
  2. Agent: select the TestNG agent we created previously

Click OK button to complete test execution scheduling for the selected three test runs.

schedule-tests-dlg.png

Now go back to Automation Host UI. Click Poll Now button.

poll-now.png

At this stage, the Automation Host does the followings:

  • immediately polls to qTest Manager to load schedule jobs and
  • execute the job execution for the three tests in TestNG project that match the automation content of the three scheduled test runs

When the execution completes, click on View Log icon to view execution log.

view-log.png

A log screen will display that shows the last execution logs. Verify that the log reported only three (3) tests have been executed.

view-log-dlg.png

Now get back to Test Execution of qConnect Sample Project in qTest Manager. Click on each test run that was scheduled to be executed, you'll see the Execution History of each test run was updated with 2 test run logs: one test log created when the agent was executed in the first time, and one test log for the scheduled job execution.

execution-history.png

You have successfully scheduled test execution for specific tests in your TestNG project using Universal Agent, and have it report the test execution to qTest Manager.

IMPORTANT NOTES

When integrating your actual TestNG project with Universal Agent and you choose to execute it with Maven (highly recommended), make sure to add the plugin maven-sure-fire-plugins and maven-compiler-plugin to your pom.xml file, as shown below.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    ... 
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.19.1</version>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>testNgSuite.xml</suiteXmlFile>
                    </suiteXmlFiles>
                    <testFailureIgnore>true</testFailureIgnore>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <encoding>iso-8859-1</encoding>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <!-- End configuration -->
</project>

You can also refer to the pom.xml file used in our sample here: https://github.com/QASymphony/cucumber-sample/blob/master/pom.xml 

 

Powered by Zendesk