Introduction

The run options are accessible using the menu Run > Run Test Case

RunTestCaseMenu

The different sections let the user select the test case(s) to launch and choose its execution parameters.

Selection type
Environment & Country
Robot settings
Execution settings

Selection Type

RunTestSelection

Manual selection let the user select one or several test cases using the filters.

ExtendedTestCaseFilters

Campaign selection let the user select a campaign from a dropdown menu. Clicking 'load' button display the selected test cases in the section below.

CampaignSelection

Environment & Country

By default, in automatic mode, you can select the preconfigured environment.

Warning: The environment and countries must be configured for the related applications. Besides, the testcase attributes override the configuration (e.g. Active in PROD)

EnvironmentCountries

You can also define yourself the url of the application to test. In that case, the environment still needs to be defined in order to determine which data for the property calculation to use. url can have different format :

simple string : http://host
Composed string for multiple applications using : and ; separators : APP1:qa.cerberus-testing.org;APP2:toto.cerberus-testing.org
Composed string for multiple applications using JSON format : {"APP1":"https://qa.cerberus-testing.org:443","APP2":"http://toto.cerberus-testing.org"}

EnvironmentCountries

Last you need to select one or more countries.

EnvironmentCountries

Robots settings

RunRobotSettings

The robots settings can be linked to a campaign. If so, the robot(s) will be automatically selected when the corresponding campaign is loaded.

However, if the user changes this value, it will be overriden by his selection.

'Save Robot Preferencies' button save these settings in the browser cache of the device.

The selected robot can be edited using the left button. A new robot can be created using the right one.

Robot Capabilities

When creating or updating a robot, you can specify capabilities in the 'Capabilities' tab of the robot modal window. Appium 2 and Selenium 4 use the W3C protocol to apply the capabilities to the robot. Therefore, it’s essential to follow the correct syntax to ensure compliance with the W3C protocol.

Basic WebDriver Capabilities

The syntax remains mostly the same, with the exceptions of 'version' and 'platform', which have been replaced by 'browserVersion' and 'platformName' respectively. You can find the list of W3C WebDriver standard capabilities here: https://www.w3.org/TR/webdriver1/#capabilities.

Appium Capabilities

With Appium 2, each Appium capability name must be prefixed with the appium prefix. When defining Appium capabilities in the Capabilities tab, you must prefix the capability name like this: <prefix>:<capability>.

For example:

appium:app (instead of just app)
appium:appActivity (instead of appActivity)
…

You can find the official documentation provided by Appium:

General capabilities: https://appium.io/docs/en/2.0/guides/caps/
UIAutomator2 driver capabilities: https://github.com/appium/appium-uiautomator2-driver
XCUITest driver capabilities: https://appium.github.io/appium-xcuitest-driver/latest/reference/capabilities/

Cloud Provider Capabilities

Cerberus supports the following 3 cloud testing providers:

BrowserStack
LambdaTest
Kobiton

To define specific cloud capabilities, in addition to those generated by Cerberus when launching the test, you must prefix the capability name as follows: <prefix>:<capability>.

Prefixes are specific to each testing cloud provider:

bstack for BrowserStack (e.g., bstack:buildName; bstack:sessionName; …)
lt for LambdaTest (e.g., lt:build; lt:name; …)
kobiton for Kobiton (e.g., kobiton:sessionName; kobiton:sessionDescription; …)

Each of these testing cloud providers provides documentation on capabilities:

BrowserStack:
- Web Automate capabilities: https://www.browserstack.com/docs/automate/capabilities
- App Automate capabilities: https://www.browserstack.com/docs/app-automate/capabilities
LambdaTest:
- Capabilities Generator: https://www.lambdatest.com/capabilities-generator/
Kobiton:
- Launching an Appium 2 session on Kobiton: https://docs.kobiton.com/automation-testing/basic-appium-server/launch-a-basic-appium-2-session
- Available Kobiton capabilities: https://docs.kobiton.com/automation-testing/capabilities/available-capabilities#_kobiton_capabilities

Capabilities File

You can view the capabilities used in your tests by navigating to the Robot tab in the Execution Report. Two files are available:

Robot caps: The list of capabilities provided by you and Cerberus to the robot.
Robot server caps: The capabilities generated by the server (Selenium or Appium). This file combines Robot caps and additional server-specific capabilities.

Execution settings

RunExecutionSettings

Tag : regroup all the execution in the same tag. It help identify this execution for reporting sake. If no value, the tag will be a timestamp. you can use %USER% and %TIMESTAMP% variable in this field.
Verbose : Verbose define how much trace information will be output on the execution. The higher the level is from 0 to 2 the more information will be and the slower the testcase will be executed.

Trace \ Verbose Level

Chrome & Firefox display

no (headless)

yes

JSON INFO on Cerberus Application Server logs

yes

Save Selenium log

only when result <> OK

yes

Full HAR file (when using Cerberus proxy executor) (1)

yes

HAR summarized stat file (when using Cerberus proxy executor) (1)

yes

Browserstack log activation

yes

Browserstack Save HAR and Selenium log

yes

(1) HAR and stat data is only collected if cerberus_networkstatsave_active is active.

Screenshot : define if screenshots and video have to be taken and the frequency.
Page Source : define if Page source have to be taken and the frequency.
Robot Log : define if robot (Selenium or Appium) log will be taken.
Timeout : define the maximum time (in ms) to spent in an action or control. Correspond to the time Cerberus will wait until it consider that a field or object is not available.
Retries : Defined the number of retries for each test cases. A retry is a re submission of a testcase (in the queue) if its status is different from OK.
Priority : define the priority for all the test cases (reminder: the lowest (=0) the most important). Default value is 1000. Tips: Use 0 priority only when relaunching manually a test case.
Manual Execution : define the test case execution type: Y : manual, N : Automatic, A : Automaticly determined from test case. default value is 'N'

Execution parameter \ Test Case Group

AUTOMATED

MANUAL

PRIVATE

Automated execution

Manual execution

Automated execution

Manual Execution

Automated execution

Execution result status

ExecutionStatus

Here is a summary of all execution status with their corresponding meanings.

OK : The execution was performed correctly and all controls were OK.
KO : The execution was performed correcly and at least one control failed resulting a global KO. That means that a bug needs to be reported to development teams.
FA : The execution did not performed correctly and needs a correction from the team that is in charge of managing the testcases. It couls be a failed SQL or action during the test.
NA : Test could not be executed as a data could not be retreived. That probably means that the test is not possible in the current environment/status.
NE : Test Case was not executed (following a condition execution).
WE : Test Case was not yet executed (waiting a manual execution).
PE : Test Case is currently running.
QU : Test Case is still waiting in the queue for ressources or constrains to be released.
QE : Test Case failed in the queue and will not be triggered.
CA : Test Case has been cancelled by user.