SeeTest 12.11 Release Notes
Released 28 November 2019 ,  Release notes home   



The Highlights:

 Performance: Transaction for Web, Performance API

Since version 12.7 we have been continuously improving on what was the first installment of our much anticipated UI Performance Testing offering. In version 12.7 we introduced UI Performance Testing - Manual testing, in version 12.8 we significantly expanded this offering to include automation, transaction video reports and a report repository in Reporter. 
Version 12.9 announced the third installment which included HAR files as part of transaction reports and improved UX and filtering capabilities for the transaction report repository. In version 12.10 we announced our fourth installment which included automatic Speed Index score calculation as part of a transaction along with much-improved reporting analytics with Grouping. In this version 12.11 we are happy to announce our fifth installment of which includes the much anticipated performance transaction for web for automation, as well as REST API for performance measurements comparison.

Transaction Support for Web / Selenium Tests:   
In version 12.11, we have added the support for performance transaction for your automated Web / Selenium tests. You can implement performance transaction into your Selenium tests, see a report and analyse it. For each transaction we collect all the raw data (Browser name, OS, URL etc...), measure the duration time and calculate the Speed Index. This enables large scale performance transaction analytics for Web directly from your CI/CD pipeline.

REST API to Analyze Performance Results as Part of your CI pipeline:
Version 12.11 includes REST API for performance measurements comparison. You can specify the transaction name and baseline measurements you want to compare to and define a condition upon which the tests are to fail. An example for such condition can be greater than x% deviation in performance in parameter or parameters as duration time, CPU, memory or network usage as well as Speed Index score. You can now add the condition to your CI/CD pipeline and trigger alerts to know in real time of any performance degradation

                                

 Video Report for Web

Version 12.11 includes detailed video report also for automated Selenium tests. As in automated Appium tests, for each Selenium test executed, we generate a report that includes all the test steps, Selenium session capabilities and test video. It also includes test status, date and duration. By watching the video together with the test steps and the other data provided in this report, users can quickly identify issues and their root cause.

 App Version Improvement

In version 12.11 we enhanced the granularity of version property of an application to better identify application version.  If in the past application version was represented by one property (called 'version'), it is now represented by  two types of versions fields:

Build Version - indicates the build number of the application (used for debug/testing)

Release Version - indicates public version of the application 

This helps you better identify your application version and correlate bugs or issues to the right version of an application. It also enables uniformity of installation between iOS and Android applications.

For more information please see: Manage Project Applications 
















 Browser and OS Version Support

  • iOS 13.2.3, 13.3 beta 3
  • iPadOS 13.2.3, 13.3 beta 3
  • Android 10

  • Chrome: 78

  • Firefox: 71
  • Edge: 79

 Important notes:

  • Starting from previous version (12.10) we have deprecated support for iOS 10.
  • Starting from previous version (12.10) we have deprecated support for Android 5.0 and below.


 Additional New Features 

 Appium OSS - Support Appium  v1.15 

 Espresso - Video default bit-rate decreased to 4 Mb/s

 Espresso - Test without any UI interactions will have an added step to the report to notify the user.

 Appium Grid - Be able to work from Appium Desktop against a cloud device

 Selenium Grid - Windows - Support for AutoIT script execution for popup handling

 Reporter - Transactions Group By - replace dragging with drop-down

 Reporter - Add "speed index" to transactions filtering abilities

 Performance Transactions - Set "3G - average" as default profile for both manual and automated transactions

 Video Report - Support setShowReport command - skip steps from Video report

 Cloud Management

This section is relevant only for Cloud Administrators.

To access Cloud Management documentation, you should log in.

In case you do not have credentials please contact sales@experitest.com

Notes and Known Issues

  • Cloud Deployment - Stopping the support for macOS - from 12.12 we will not continue to support the deployment on macOS for the following components:

    • Cloud Server

    • Region Proxy Manger

    • Application Signer 

    • File Storage 

  • We will continue to support Device Host Machine and Selenium Agent on macOS and Windows operating systems


  • Please see the updated System Requirements (update the requirements for Selenium Agent)

  • Cloud Deployment on Linux - We plan to support from 20.1 version, the deployment of most of the cloud components on Linux (CentOS / Red Hat Enterprise )

    The supported components on Linux (planned for 20.1) are:

    • Cloud Server

    • Region Proxy Manger

    • Application Signer 

    • File Storage 

    • SeeTest Reporter

    • Network Virtualization 
       

  • macOS Catalina Support - In order to deploy Device Host Machine on macOS Catalina you need to manually approve the installer (from the Security and Privacy settings)

 Bug Fixes and Improvements

  • iOS - Particular Web Application doesn't allow actions such as sendKeys - "running JavaScript on device is not possible due to code blocking" - Fixed

  • Espresso - Report - Failed tests steps are marked as passed in a failed report - Fixed

  • iOS 13 - Simulate Capture QR code does not send the decoded string to application code - Fixed

  • Android application fails to load ScriptEngine after rebuilding with APKTool - Fixed

  • iOS Simulators - Shows up as Available in Devices List View when they should not - Fixed

  • Android - Device Language - Changing device region and language doesn't work the same as changing it in the device UI - Fixed

  • Android - Cloud - Permission Restricted is not enforced on some applications - Fixed

  • iOS 13 - Simulate Capture QR code does not send the decoded string to application code - Fixed

  • Remote Debugging - iOS - should be created only for logged in user - Fixed

  • Remote Debugging - iOS - fails to generate symbols - cannot create directory /var/empty - Fixed

  • Espresso - Report - "Incomplete" test mark as "Passed"- Fixed

  • Web Studios - iOS - CTRL + Mousewheel scroll does not zoom in / out on Maps Application - Fixed

  • iOS - getApplicationInfo(releaseVersion) returns "unknown" values - Fixed

  • iOS - Failing to execute getPickerValues on specific application - Fixed

  • iOS - Web elements are not found during test run (while object spy shows them) in instrumented app - Fixed

  • Ruby - Cannot debug tests with listener since @clientID is null for sessionID = false - Fixed

  • iOS - iPad Pro (11-inch) - Unable to launch Safari - "No application with label 'Safari' was found" - Fixed

  • Reporter - Custom test view link filters doesn't display properly in UI after creation

  • Reporter - The login page keeps on reloading and users can't login - Fixed

  • Selenium Stability - IE & Edge browsers stays open after long run - Fixed

  • Manual Selenium session - Ctrl + a and copy selects the outer browser characters - Fixed

  • Selenium Manual Session- Edge - Space bar does not work in the address bar - Fixed

  • Selenium Stability - OutOfMemoryError: unable to create new native thread on multiple long runs - Fixed

  • Reporter - Grouping / un-grouping transactions cause selected columns to go back to default - Fixed

  • Cloud Cleanup - Improve cleanup stability and performance  for Android devices - Fixed

  • Users page UI/UX - List of projects listed by Alphabetical order - Fixed

  • Cloud Installer does not work on Windows Server 2012 R2 - Fixed

  • Login to Cloud gives error 'license packages are missing' - Fixed

  • Cloud events - disable Selenium agent doesn't appear as an event - Fixed

  • Execution page - LDAP users with at least one uppercase letter were not able to see their own execution - Fixed

  • Android Application Restrictions - When activity name doesn't contain app bundle name, "Restrict App" doesn't work- Fixed



 Next Release 

  • Due date*:  30.12.2019 or prior to

  • Highlights*:  HAR File Viewer from performance transaction report, New Device UI Page, Automated accessibility testing on iOS


*Please note that due date and highlights of next release are as anticipated by Experitest on the date of release of current version, and are forward looking statements. Actual due dates and highlights may vary.