Automated Test Reports

Every time you run an automated test on the Continuous Testing cloud platform, a shareable HTML report is generated automatically.

The report includes the following data:

Test Capabilities
Test Steps
Screenshots
Command Parameters and Output
Debug Data
Error Messages (including exceptions)

You can use the report to analyze failures and adjust test cases. The report can be shared with other developers and testers and can really boost collaboration and productivity.

Obtaining The Report

Whenever you run an automated test on the grid, a report URL and a report ID are added to the test capabilities. You can obtain the report URL by printing it to the log or console. The capabilities names are reportURL and reportTestId.

The best way is to print the report URL and report ID in the teardown stage of the test.

Getting Report URL in Java

@After
public void tearDown() {
    if (driver != null)
    {
        driver.quit();
        System.out.println("Report URL : " + driver.getCapabilities().getCapability("reportUrl"));
		System.out.println("Report ID: " + driver.getCapabilities().getCapability("reportTestId"));
    }
}

Getting Report URL in Python

 def tearDown(self):
        if self.driver is not None:
            print(self.driver.capabilities.get("reportUrl"))
            print(self.driver.capabilities.get("reportTestId"))
            self.driver.quit()

Getting Report URL in C Sharp

 [TearDown()]
        public void TearDown()
        {
            if (driver != null)
            {
                Console.WriteLine(driver.Capabilities.GetCapability("reportUrl"));
                Console.WriteLine(driver.Capabilities.GetCapability("reportTestId"));
                driver.Quit();
            }
        }

Getting Report URL in Ruby

 def teardown
    puts @driver.capabilities['reportUrl']
    puts @driver.capabilities['reportTestId']
    @driver.quit
  end

Understanding The Report

Watch the video to see what the report looks like and how you can browse the report to see the relevant information.

The first tab in the report is the execution summary. It lists information related to test execution, device, and capabilities.

The next three tabs are the init steps. The init steps include initializing test capabilities, test type, and test context (Native or Web)

All steps following the steps listed above are the steps of the test itself. These include:

Screenshots of the device at the time the step took place (when a command was executed) - screenshots correspond to each and every step of the test. For every step there is a screenshot of the device at the time a specific command was executed. For every text that is sent to an input field, for every click or any other gesture, a screenshot is taken to display the device state at the time of carrying out the command. The screenshots add a visual dimension that really helps you understand the cause the of failures if such occur.

Automation Test Summary Report.mp4

Command parameters - For each step, under the Debug Properties tab, there is a collapsible section called Command Parameters. You can see under this section the parameters that were passed to the command as well as additional information.
Command output and result - For each step, under the Result Details tab, there is a section titled Result Data. If the command that was used returns some value, you will see the returned value under Result Data. It could the text of an element, the page source (XML of node hierarchy) and more.

Disabling The Report

If you don't want to generate a report, you can do so by passing a capability to disable the report.

report.disable = true

Disable Report using Java

dc.setCapability("report.disable", true);

Disable Report using Python

self.dc['report.disable'] = True

Disable Report using C Sharp

dc.SetCapability("report.disable", true);

Disable Report using Ruby

 desired_caps = {
         caps: {
             # other caps
             report.disable: true,
             # other caps
         appium_lib: {
              server_url: 'https://cloud.seetest.io:443/wd/hub',
            }
     }

Page tree

Automated Test Reports

Obtaining The Report

Understanding The Report

Disabling The Report