Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
SEO Metadata
titleAutomated Test Reports

Every time you run an automated test on the Experitest cloud platform, a shareable HTML report is generated automatically. 

SEO Metadata
titleAutomated Test Reports

Every time you run an automated test on the Experitest cloud platform, a shareable HTML report is generated automatically. 


Every time you run an automated test on the Experitest cloud platform, a shareable HTML report is generated automatically. 

The report includes the following data:

  1. Test Capabilities
  2. Test Steps
  3. Screenshots
  4. Command Parameters and Output
  5. Debug Data
  6. Error Messages (including exceptions)

You can use the report to analyze failures and adjust test cases. The report can be shared with other developers and testers and can really boost collaboration and productivity.





Obtaining The Report

Whenever you run an automated test on the grid, a report URL is and a report ID are added to the test capabilities. You can obtain the report URL by printing it to the log or console. The capability name is reportURL capabilities names are reportURL and reportTestId. 

The best way is to print the report URL and report ID in the teardown stage of the test.

Code Block
languagejava
titleGetting Report URL in Java
@After
public void tearDown() {
    if (driver != null)
    {
        driver.quit();
        System.out.println("Report URL : " + driver.getCapabilities().getCapability("reportUrl"));
		System.out.println("Report ID: " + driver.getCapabilities().getCapability("reportTestId"));
    }
}
Code Block
languagepy
titleGetting Report URL in Python
 def tearDown(self):
        if self.driver is not None:
            print(self.driver.capabilities.get("reportUrl"))
            print(self.driver.capabilities.get("reportTestId"))
            self.driver.quit()
Code Block
languagec#
titleGetting Report URL in C Sharp
 [TearDown()]
        public void TearDown()
        {
            if (driver != null)
            {
                Console.WriteLine(driver.Capabilities.GetCapability("reportUrl"));
                Console.WriteLine(driver.Capabilities.GetCapability("reportTestId"));
                driver.Quit();
            }
        }
Code Block
languageruby
titleGetting Report URL in Ruby
 def teardown
    puts @driver.capabilities['reportUrl']
    puts @driver.capabilities['reportTestId']
    @driver.quit
  end


Understanding The Report

Watch the video to see how what the report looks like and how you can browse the report to see the relevant information.

The first tab in the report is the execution summary. It lists information related to test execution, device, and capabilities.

The next three tabs are the init steps. The init steps include initializing test capabilities, test type, and test context (Native or Web) 

All steps following the steps listed above are the steps of the test itself. These include:

  • Screenshots of the device at the time the step took place (when a command was executed) - screenshots correspond to each and every step of the test. For every step there is a screenshot of the device at the time a specific command was executed. For every text that is sent to an input field, for every click or any other gesture, a screenshot is taken to display the device state at the time of carrying out the command. The screenshots add a visual dimension that really helps you understand the cause the of failures if such occur.

View file
nameAutomation Test Summary Report.mp4
height400

  • Command parameters - For each step, under the Debug Properties tab, there is a collapsible section called Command Parameters. You can see under this section the parameters that were passed to the command as well as additional information.
  • Command output and result - For each step, under the Result Details tab, there is a section titled Result Data. If the command that was used returns some value, you will see the returned value under Result Data. It could the text of an element, the page source (XML of node hierarchy) and more.


Disabling The Report

If you don't want to generate a report, you can do so by passing a capability to disable the report.

Code Block
report.disable = true
Code Block
languagejava
titleDisable Report using Java
dc.setCapability("report.disable", true);
Code Block
languagepy
titleDisable Report using Python
self.dc['report.disable'] = True
Code Block
languagec#
titleDisable Report using C Sharp
dc.SetCapability("report.disable", true);
Code Block
languageruby
titleDisable Report using Ruby
 desired_caps = {
         caps: {
             # other caps
             report.disable: true,
             # other caps
         appium_lib: {
              server_url: 'https://cloud.seetest.io:443/wd/hub',
            }
     }