Page tree
Skip to end of metadata
Go to start of metadata


Continuous Testing Cloud exposes Rest APIs to execute Espresso and XCUI based Tests.

This page provides the details of the API.

The role of the user performing the Rest Operation is specified by the Authorization header.

For more information please visit How To Execute Rest API, for a detailed example.

All actions are also available for project admins and users

Status of the API Run

This API enabled us to get the status of the Espresso/XCUI test run.

GET /api/v1/test-run/{id}/status

Where id is Test run id.

Response
Response Status: 200 OK
[
    {
  		"status": "SUCCESS",
  		"data": {
    		"Test Run Id": "3520316",
    		"Test Run State": "Finished",
            "Number of tests in file" : "11",
            "Number of requested tests" : "11",
    		"Total number of tests": "11",
    		"Number of passed tests": "3",
    		"Number of failed tests": "8",
   			"Number of skipped tests": "0",
    		"Number of running tests": "0",
    		"Number of queued tests": "0",
			"Number of ignored tests" : "0",
    		"Link to Reporter": "https://CLOUD_SERVER/reporter/reporter/tests?testView={%22byKey%22:%22test.run.id%22,%22byKeyValue%22:%223520316%22}"
  		},
  	"code": "OK"
	}
]

Number of tests in file - the number of tests in the uploaded file

Number of requested tests - the number of tests that were selected for execution after applying includeTestsignoreTestFile.

"Number of skipped tests" - tests that were not executed because of timeout/cancellation.

"Number of ignored tests" - tests that were not executed because they were ignored by the user (ignoreTestFile parameter)

Running Async Espresso/XCUITest

Enables users to run Espresso or XCUITest asynchronously, the response received is immediate.

POST /api/v1/test-run/execute-test-run-async

Parameters

Name

Type

Mandatory

Description

executionTypeStringYes

Specified if it is Espresso or XCUITest

runningType StringYes

coverage / fastFeedback

  1. Coverage - all of the tests will be executed on all of the selected devices.

  2. Fast feedback - all of the tests will be spread across the selected devices.

appFile

Yes (either app or appUrl or CloudApp

The application is under test. 
appUrlStringThe URL location of the application is under test. 
cloudAppLongThe ID of the application saved in cloud application service
testAppFileYes (either testApp or testAppUrl or CloudTestApp)

espresso / XCUI tests(.zip extension only). 

testAppUrlStringThe URL Location of espresso / XCUI tests. (.zip extension only)
cloudTestAppLongThe ID of the test app saved in cloud file repository service
deviceQueriesStringNo

For coverage mode:

List of strings (XPath queries). The test run should run on devices that match provided queries (device per query).

Multiple queries should be passed in the standard way by repeating the parameter multiple times.

For Fast Feedback:

Only one device query must be provided on the list.

The number of devices used is decided by maxDevices and minDevices parameters.

If the maxDevices parameter is not provided then the execution will try to reserve the maximum number of devices according to license restriction.

It is recommended to use maxDevices parameter always

uuidStringNo

Provision profile UUID to sign the Application and the test application.

Relevant to iOS execution-only

runTagsStringNoMap as JSON - wanted tags to upload to the reporter.
overallTimeoutLongNo

The number of milliseconds.

Timeout for the overall test run. If not provided - default value (14,400,000) - 4 hours - should be used.

creationTimeoutLongNo

The number of milliseconds.

Timeout for test requests in queued status. If not provided - default value (14,400,000) - 4 hours - should be used.

reservationDurationLongNo

The reservation time of each session in minutes.

The default time is 120 minutes

  • the reservationTime cannot exceed the max reservation time configured for the cloud or project.
includeTestsList of StringsNo

Run only the specified tests. The supported formats:

  • fullClassName, methodName
  • fullClassName.methodName
  • fullClassName 
  • methodName
  • packageName

For cucumber/Cucumberish tags, run only the matching tags. Each tag will be provided separately:

  • @tag1
  • @tag2
ignoreTestFileFileNo

This file specifies the tests to exclude.

iOS

  • xcscheme file

Android

  • the file containing full class and method name, separated with comma, e.g.
com.example.app.SomeTest, testFoo
com.example.app.AnotherTest, testBar
maxDevicesInteger No 

The maximum number of devices to use during the execution. default value - max available devices license.

Note: Only use in case of fastFeedback

minDevicesInteger No

The minimum number of devices to use during the execution.

Default value - 1.

Note: Only use in case of fastFeedback

retry Integer No

An integer from 0 to 5 -

Specifies how many times failed test should be retried

Default Value -  0 (means no retry)

additionalAppIdsList of LongsNo

List of application ids that should be installed on the device/emulator before starting the Espresso/XCUI test.

Application id is returned when uploading application with Rest API or using the Get Applications REST API.

retryDifferentDevice

boolean

No

A boolean value specifies whether or not to run a failed test on different devices according to the "retry" value.

Default Value -  false (means will not try to run a failed test on different devices).

If "maxDevices" is specified,  then "retry" might be limited to the min between ("maxDevices" - 1) and "retry". 

useUIAutomatorbooleanNoFor Android only
A boolean value decides whether or not to disable Continuous Testing's UIAutomator.
Default Value = false - UIAutomator will not be disabled.
True - UiAutomator will be disabled on Test Start and will be enabled after it ends
useTestOrchestratorbooleanNoFor Android only
A boolean value decides whether or not to run the test in Test Orchestrator Mode.
Default Value = false - test will not run in Test Orchestrator Mode
true - test will run in Test Orchestrator Mode
clearPackageDatabooleanNoFor Android only
If useTestOrchestrator is true, then clearPackageData makes sure to remove all shared states from the device's CPU and memory after each test. 
Default Value = false - don't clear shared state data
true - clear shared state data
Response
{
    "status": "SUCCESS",
    "data": {
        "Test Run Id": "3520546",
        "Link to Reporter": "https://CLOUD_SERVER/reporter/reporter/tests?testView={%22byKey%22:%22test.run.id%22,%22byKeyValue%22:%223520546%22}"
    },
    "code": "OK"
}

Running Sync Espresso/XCUITest

Enables users to run Espresso or XCUITest synchronously, the response received is after the test run.

POST /api/v1/test-run/execute-test-run

Parameters

Name

Type

Mandatory

Description

executionTypeStringYes

Specified if it is Espresso or XCUITest

runningType StringYes

coverage / fastFeedback

  1. Coverage - all of the tests will be executed on all of the selected devices.

  2. Fast feedback - all of the tests will be spread across the selected devices.

appFileYes (either app or appUrl or cloudAppApplication under test
appUrlStringThe URL location of the application is under test. 
cloudAppLongThe ID of the application saved in cloud application service
testAppFileYes (either testApp or testAppUrl or cloudTestApp)espresso / XCUI tests.
testAppUrlStringThe URL Location of espresso / XCUI tests(.zip extension only).
cloudTestAppLongThe ID of the test app saved in cloud file repository service
deviceQueries StringYes

For coverage mode:

List of strings (XPath queries). The test run should run on devices that match provided queries (device per query).

Multiple queries should be passed in the standard way by repeating the parameter multiple times.

For Fast Feedback:

Only one device query must be provided on the list.

The number of devices used is decided by maxDevices and minDevices parameters.

If the maxDevices parameter is not provided then the execution will try to reserve the maximum number of devices according to license restriction.

It is recommended to use maxDevices parameter always

uuidStringNoProvision profile UUID to sign the Application and the test application.
runTagsStringNoMap as JSON - wanted tags to upload to the reporter.
overallTimeoutLongNo

The number of milliseconds.

Timeout for the overall test run. If not provided - default value (14,400,000) - 4 hours - should be used.

creationTimeoutLongNo

The number of milliseconds.

Timeout for test requests in queued status. If not provided - default value (14,400,000) - 4 hours - should be used.

reservationDurationLongNo

The reservation time of each session in minutes.

The default time is 120 minutes

  • the reservationTime cannot exceed the max reservation time configured for the cloud or project.
includeTestsList of StringsNo

Run only the specified tests. If no matching tests were found an error is reported. The supported formats:

  • fullClassName, methodName
  • fullClassName.methodName
  • fullClassName 
  • methodName
  • packageName

For cucumber tags, run only the matching tags. Each tag will be provided separately:

  • @tag1
  • @tag2
ignoreTestFileFileNo

This file specifies the tests to exclude.

iOS

  • xcscheme file

Android

  • the file containing full class and method name, separated with comma, e.g.
com.example.app.SomeTest, testFoo
com.example.app.AnotherTest, testBar
maxDevicesInteger No 

The maximum number of devices to use during the execution. default value - max available devices license.

Note: Only use in case of fastFeedback

minDevicesInteger No

The minimum number of devices to use during the execution.

Default value - 1.

Note: Only use in case of fastFeedback

retry Integer No

An integer from 0 to 5 -

Specifies how many times failed test should be retried

Default Value -  0 (means no retry)

additionalAppIdsListNo

List of application ids that should be installed on the device/emulator before starting the Espresso/XCUI test.

Application id is returned when uploading the application with Rest API.

retryDifferentDevicebooleanNo

A boolean value specifies whether or not to run a failed test on different devices according to the "retry" value.

Default Value -  false (means will not try to run a failed test on different devices).

If "maxDevices" is specified,  then "retry" might be limited to the min between ("maxDevices" - 1) and "retry". 

useUIAutomatorboolean No

For Android only
A boolean value decides whether or not to disable Continuous Testing's UIAutomator.
Default Value = false - UIAutomator will not be disabled.
True - UiAutomator will be disabled on Test Start and will be enabled after it ends

useTestOrchestratorbooleanNoFor Android only
A boolean value decides whether or not to run the test in Test Orchestrator Mode.
Default Value = false - test will not run in Test Orchestrator Mode
true - test will run in Test Orchestrator Mode
clearPackageDatabooleanNo

For Android only
If useTestOrchestrator is true, then clearPackageData makes sure to remove all shared states from the device's CPU and memory after each test. 
Default Value = false - don't clear shared state data
true - clear shared state data


Response
{
  "status": "SUCCESS",
  "data": {
    "Test Run Id": "3520409",
    "Test Run State": "Finished",
    "Number of tests in file" : "11",
    "Number of requested tests" : "11",
    "Total number of tests": "11",
    "Number of passed tests": "3",
    "Number of failed tests": "8",
    "Number of skipped tests": "0",
    "Number of running tests": "0",
    "Number of queued tests": "0",
    "Link to Reporter": "https://CLOUD_SERVER/reporter/reporter/tests?testView={%22byKey%22:%22test.run.id%22,%22byKeyValue%22:%223520409%22}"
  },
  "code": "OK"
}

Cancel all tests of the executed suite

Cancel all tests of the executed suit.

POST /api/v1/test-run/{id}/cancel

Where id is Test run id.

Java Code Examples

The examples below using the Unirest HTTP library To compile then use the following dependencies in your Java Project.

Maven dependency:

<dependency>
    <groupId>com.mashape.unirest</groupId>
    <artifactId>unirest-java</artifactId>
    <version>1.4.9</version>
</dependency>


Gradle dependency:

dependencies {
	compile group: 'com.mashape.unirest', name: 'unirest-java', version: '1.4.9'
}

Following are the code examples:

executeTestRunAsync
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;
import com.mashape.unirest.http.exceptions.UnirestException;
import java.io.File;
import org.junit.Test;


public class ExecuteTestRunAsync {

    String urlBase = "http://cloudHost:cloudPort"; // modify hostname and port as applicable your environment
    String user = "admin";
    String password = "password"; // modify the admin password as applicable your environment
    String url = urlBase + "/api/v1/test-run/execute-test-run-async";
    String pathToTestApp = "pathtoTestApp"; // path to XcuiTests or espresso tests
    String pathToApp = "pathToFIle";

    @Test
    public void executeTestRunAsync() throws UnirestException {
        File app = new File(pathToApp);      
        File testApp =  new File(pathToTestApp);
        HttpResponse<String> response = Unirest.post(url)
            .basicAuth(user, password)
            .queryString("executionType", "espresso")
			.queryString("runningType", "coverage")
			.queryString("deviceQueries", "@os='android'") // can be repeated if multiple queries
            .field("app", app)
            .field("testApp", testApp)
            .asString();
        System.out.println(response.getBody());
    }
}


executeTestRun
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;
import com.mashape.unirest.http.exceptions.UnirestException;
import java.io.File;
import org.junit.Test;


public class ExecuteTestRun {

    String urlBase = "http://cloudHost:cloudPort"; // modify hostname and port as applicable your environment
    String user = "admin";
    String password = "password"; // modify the admin password as applicable your environment
    String url = urlBase + "/api/v1/test-run/execute-test-run";
    String pathToTestApp = "pathtoTestApp"; // path to XcuiTests or espresso tests
    String pathToApp = "pathToFIle";

    @Test
    public void executeTestRun() throws UnirestException {
        File app = new File(pathToApp);      
        File testApp =  new File(pathToTestApp);
		Unirest.setTimeouts(0, 0); //set infinity timeout for post request
        HttpResponse<String> response = Unirest.post(url)
            .basicAuth(user, password)
            .queryString("executionType", "espresso")
			.queryString("runningType", "coverage")
			.queryString("deviceQueries", "@os='android'") // can be repeated if multiple queries
            .field("app", app)
            .field("testApp", testApp)
            .asString();
        System.out.println(response.getBody());
    }
}



cancelTestRun
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;
import com.mashape.unirest.http.exceptions.UnirestException;
import org.junit.Test;

public class TestRun {
    String urlBase = "http://cloudHost:cloudPort";   // modify hostname and port as applicable your environment
    String user = "admin";
    String password = "password";  // modify the admin password as applicable your environment
	Long testRunId = testRunId; // need to put here the test run id
	String url = urlBase + "/api/v1/test-run/" + testRunId +"/cancel";



    @Test
    public void cancelTestRun() throws UnirestException {
        
        HttpResponse<String> response = Unirest.get(url)
            .basicAuth(user, password)
            .asString();

        System.out.println(response.getBody());

    }

}


getTestRunStatus
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;
import com.mashape.unirest.http.exceptions.UnirestException;
import org.junit.Test;

public class TestRun {
    String urlBase = "http://cloudHost:cloudPort";   // modify hostname and port as applicable your environment
    String user = "admin";
    String password = "password";  // modify the admin password as applicable your environment
	Long testRunId = testRunId; // need to put here the test run id
	String url = urlBase + "/api/v1/test-run/" + testRunId +"/status";



    @Test
    public void getTestRunStatus() throws UnirestException {
        
        HttpResponse<String> response = Unirest.get(url)
            .basicAuth(user, password)
            .asString();

        System.out.println(response.getBody());

    }

}


cURL Command
curl "https://Cloud_URL/api/v1/test-run/execute-test-run-async?deviceQueries=@os='android'" -H "Authorization: Basic YOUR_CREDENTIALS" -F "executionType=espresso" -F "runningType=coverage" -F "app=@APP_PATH" -F "testApp=@TEST_APP_PATH"

# here "deviceQueries" parameter is passed via query string, as it contains "@", which curl treats as a file name indicator.
# multiple device queries can be specified by adding the parameter multiple times: "https://Cloud_URL/api/v1/test-run/execute-test-run-async?deviceQueries=...&deviceQueries=...".
# if device query contains special characters (e.g. spaces) they should be URL encoded.
  • No labels