Applitools FAQs

Several important enhancements will be introduced in Applitools Eyes 10.9. These changes will make tests run faster and more efficiently. The changes require users to install an upgraded SDK version to be compatible with the upcoming new Eyes version and without it future enhancements and new Eyes versions will not be available.

Notice if the SDK that you are using is not on that list - there is no need to upgrade.

SDKs that support 10.9 Eyes:

JS :

SDK

Minimum Version

Selenium 3

3.8.4

Selenium 4

4.33.6

WDIO 4

2.11.6

WDIO 4 Service

1.0.3

WDIO 5

5.9.5

WDIO 5 Service

1.3.1

Images

4.12.34

Applitools for Selenium IDE

1.10.0

Java:

SDK

Minimum Version

Selenium Java 3

3.160.0

Appium Java 4

4.5.0

Java 3 Images

3.160.0

.Net SDKs:

SDK

Minimum Version

Selenium C#

2.22.31

Appium C#

4.0.6

C# Images

2.4.5

Eyes.Windows

2.5.6

Coded UI

2.4.4

LeanFT

2.6.4

Other SDKs:

SDK

Minimum Version

Selenium Python 3

3.16.4

Python Images

3.16.4

Selenium Python 4

4.1.10

Selenium Ruby

3.16.1

Appium Ruby

3.16.1

Ruby Images

3.16.1

Espresso

4.2.0

EyesXCUI

8.2.0

XCTest

4.6.0

EyesEarlyGrey

4.6.0

ASTR

3.45.0

ImageTester

1.2.5

UFT

1.11

Each team has a unique API key that will ensure that all of your team's tests are listed together and linked to the same team.

The API key can be found in the Applitools Test Manager (dashboard) or in the Admin Panel.

In the Test Manager:

In the Admin Panel

The API key can be set hardcoded in the test by using the Eyes setApiKey(<API_KEY>) method.

The best practice is to set the API key as a system environment variable. Then, there is no need to set the key in each test, Eyes will use the system variable by default.

How to set the API key as a system environment variable:

MacOS

Open Terminal window

Open the .bash_profile file as follow:

open ~/.bash_profile

Add the following command to the .bash_profile: (Replace <YOUR_API_KEY> with your key)

export APPLITOOLS_API_KEY=<YOUR_API_KEY>

Save (command+S)

Windows

In Search, search for and then select:System(Control Panel)

Click theAdvanced system settingslink

ClickEnvironment Variables

In the sectionSystem Variables, clickNew

Set the variable name toAPPLITOOLS_API_KEY

Set the variable value to your API key

ClickOK

Motivation:

A few performance enhancements will be introduced as part of Applitools Eyes version 10.9; these changes require you to use one of those SDK versions or higher, and an additional whitelisting might be needed due to that.

Please check if your network requires additional whitelisting of the new Azure storage and perform the whitelisting prior to upgrading the SDKs (your company’s storage address was sent to you over email, you can also reach out to support if you don’t have it).

In case you’re not sure if this action is needed according to your organization’s policy, you can follow the below instructions in order to test it yourself.

What should you do?

The below commands can be executed with either curl OR powershell OR wget which can be run from Mac (bash), Linux (bash) and Windows (cmd / powershell).

Step #1: GET the Azure blob URL

curl:

curl "https://<eyes-server>. applitools.com/api/sessions/renderinfo?apiKey =<YOUR_API_KEY>"

curl via Power shell:

curl -Method GET https://<eyes-server>. applitools.com/api/sessions/renderinfo?apiKey =<YOUR_API_KEY>

wget:

wget --verbose \

--method GET \

--output-document \

- 'https://<eyes-server>. applitools.com/api/sessions/renderinfo?apiKey =<YOUR_API_KEY>'

Step #2: in the output of the previous command - copy the value of the field "resultsUrl", the format should look like:

https://eyesexample.blob.core.windows.net/temp/__random__?sv=xxxxxxx&sr=xxx&sig=xxxxxxxx&se=xxxxx&sp=xx&accessKey=xxxxxx

Please replace __random__ with something else (this is just a name, so anything can work).

Step #3: Create a PUT request using the URL we just changed. You will also have to create a Date header with the current time.

curl:

curl --verbose --request PUT \

--url ' https://eyesexample.blob.core.windows.net/temp/test_cloud?sv=xxxxxxx&sr=xxx&sig=xxxxxxxx&se=xxxxx&sp=xx&accessKey=xxxxxx ' \

--header 'Date: Mon, 2 Feb 2020 09:42:00 GMT' \

--header 'x-ms-blob-type: BlockBlob' \

--header 'x-ms-version: 2015-02-21' \

--form 'image=""'

curl via PowerShell:

curl -Method PUT -Body "" -Headers @{'x-ms-date' = 'Mon, 2 Feb 202009:42:00 GMT'; 'x-ms-blob-type' = 'BlockBlob';'x-ms-version' = '2015-02-21'} " https://eyesexample.blob.core.windows.net/temp/test_cloud?sv=xxxxxxx&sr=xxx&sig=xxxxxxxx&se=xxxxx&sp=xx&accessKey=xxxxxx "

wget:

wget --verbose \

--method PUT \

--header 'x-ms-version: 2015-02-21' \

--header 'Date: Mon, 2 Feb 2020 09:42:00 GMT' \

--header 'x-ms-blob-type: BlockBlob' \

--body-data '------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name="image"\r\n\r\n""\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW--' \

--output-document \

- ' https://eyesexample.blob.core.windows.net/temp/test_cloud?sv=xxxxxxx&sr=xxx&sig=xxxxxxxx&se=xxxxx&sp=xx&accessKey=xxxxxx '

What should you expect to see?

If command #3 was executed successfully, you should see in the console / terminal window - HTTP/1.1 201 Created - this means that whitelisted is not needed or that the whitelisting was successful.

If command #3failed, you should see in the console / terminal window a network error such as FORBIDDEN - this means that you need to whitelist the new Azure storage address.

At the end of each test, the TestResultsHandler allows you to download directly from the SDK the following :

(i)the test baseline images,

(ii) the actual images, and

(iii) images with the differences highlighted.

Java TestResultsHandler Github Repo

C# TestResultsHandler Github Repo

Javascript TestResultsHandler Github Repo

Python TestResultsHandler Github Repo

Ruby TestResultsHandler Github Repo

Java

TestResults testResult= eyes.close(false);

ApplitoolsTestResultsHandler testResultHandler= new ApplitoolsTestResultsHandler(testResult,viewKey);

Note: The following dependency needs to be added to your pom.xml file:

<dependency>

<groupId>org.json</groupId>

<artifactId>org.json</artifactId>

<version>RELEASE</version>

</dependency>

.NET

TestResultsresults=eyes.Close(false);

vartestResultHandler=newApplitoolsTestResultsHandler.ApplitoolsTestResultsHandler("ViewKey",results);

Ruby

results= eyes.close(false);

test_results_handler= ApplitoolsTestResultHandler.New(results,viewKey)

Python

test_results = eyes.close(False)

test_results_handler = ApplitoolsTestResultsHandler.ApplitoolsTestResultsHandler(test_results, "ViewKey")

Java Script

let results = await eyes.close(false);

const handler= new ApplitoolsTestResultHandler(results, applitoolsViewKey);

Note: For getting your account View Key please contact Applitools support.

The TestResultsHandler extends the capabilitiesof TestResult by adding the following methods:

1) downloadDiffs - Downloading the test imageswith the highlighted detected differences to a given directory. In case of New, Missing or passed step no image will be downloaded.

Java

testResultHandler.downloadDiffs(Path_to_directory);

.NET

testResultHandler.downloadDiffs(Path_to_directory);

Ruby

test_results_handler.download_diffs(Path_to_directory);

Python

test_results_handler.download_diffs(Path_to_directory)

Javascript

handler.downloadImages(downloadDir, 'diff')

2) downloadBaselineImages-Downloading the test baseline images to a given directory

Java

testResultHandler.downloadBaselineImages(Path_to_directory);

.NET

testResultHandler.downloadBaselineImages(Path_to_directory);

Javascript

handler.downloadImages(downloadDir, 'baseline');

3)downloadCurrentImages-Downloading the test current image to a given directory.

Java

testResultHandler.downloadCurrentImages(Path_to_directory);

.NET

testResultHandler.downloadCurrentImages(Path_to_directory);

Javascript

handler.downloadImages(downloadDir, 'current');

4) downloadImages - Downloading the test baseline image and current image to a given directory.

Java

testResultHandler.downloadImages(String Path_to_directory);

.NET

testResultHandler.downloadImages(Path_to_directory);

Ruby

test_results_handler.download_images(Path_to_directory);

Python

test_results_handler.download_images(Path=Path_to_directory)

5)setPathPrefix-Setting this path prefix will determine the structure of the repository for the download images

Java

testResultHandler.setPathPrefixStructure("TestName/AppName/Viewport/hostingOS/hostingApp");

.NET

testResultHandler.setPathPrefixStructure("TestName/AppName/Viewport/hostingOS/hostingApp");

In addition to downloading the images of the test, TestResultHandler also gives access through code to the visually comparison result per step. It returns an array of elements called RESULT_STATUS which can be one of the following four options: PASS, UNRESOLVED, NEW or MISSING

Java

RESULT_STATUS[] stepsResultsArray = testResultHandler.calculateStepResults();

.NET

RESULT_STATUS[] = testResultHandler.calculateStepResults();

Ruby

steps_results_array = test_results_handler.calculate_step_results()

Python

steps_results_array = test_results_handler.calculate_step_results()

Java Script

handler.setDirStructure(['myImages', appName]);

* Notice that this method receive an array of strings.

The TestResuletHandler class is currently available in Java, Ruby, .NET and Python, and it will soon be available in other SDK as well.

Further regarding:

# Getting Diff ImagesManually.

# Getting Current/Baseline Images Manually.

# Extend API features with EyesUtilities.

Recent versions of Chrome displaynotification bar at the top of the browser saying "Chrome is being controlled by automated test software", as follows:

http://127.0.0.1:4444/wd/hub'

Previously, passing the "disable-infobars” ChromeOption to the WebDriver prevented Chrome from displaying this notification. Recently, the "disable-infobars" option has been deprecated and no longer removes the notification. The current workaround for this is to pass in an option called "excludeSwitches" and then exclude the "enable_automation" switch.

CAUTION: Please review what the "enable_automation" switch does via the documentation and ensure your test suite does not rely on any features listed there before disabling it.

Here is the relevant syntax to do so:

When using ChromeDriver

Java:

ChromeOptions options = new ChromeOptions();

options.setExperimentalOption("excludeSwitches", new String[]{"enable-automation"});

WebDriver driver = new ChromeDriver(options);

Protractor:

exports.config = {

capabilities: {

'browserName': 'chrome',

'chromeOptions': {

'excludeSwitches': ['enable-automation']

}

},

specs: ['ProtractorExample.js']

};

JavaScript:

var chromeCapabilities=webdriver.Capabilities.chrome()

var chromeOptions = {

'excludeSwitches': ['enable-automation']

};

chromeCapabilities.set('chromeOptions', chromeOptions);

var driver = new webdriver.Builder()

.withCapabilities(chromeCapabilities)

.build();

Python:

chrome_options = webdriver.ChromeOptions();

chrome_options.add_experimental_option("excludeSwitches", ['enable-automation']);

driver = webdriver.Chrome(options=chrome_options);

Ruby:

options = Selenium::WebDriver::Chrome::Options.new(options: {"excludeSwitches" => ["enable-automation"]})

driver = Selenium::WebDriver.for :chrome, options: options

Java:

ChromeOptions options = new ChromeOptions();

options.setExperimentalOption("excludeSwitches", new String[]{"enable-automation"});

DesiredCapabilities caps = DesiredCapabilities.chrome();

caps.setCapability(ChromeOptions.CAPABILITY, options);

WebDriver driver = new RemoteWebDriver(caps);

Protractor:

exports.config = {

capabilities: {

'browserName': 'chrome',

'chromeOptions': {

'excludeSwitches': ['enable-automation']

}

},

seleniumAddress: ' http://localhost:4444/wd/hub',

specs: ['ProtractorExample.js']

};

JavaScript:

var chromeCapabilities=webdriver.Capabilities.chrome()

var chromeOptions = {

'excludeSwitchest': ['enable-automation']

};

chromeCapabilities.set('chromeOptions', chromeOptions);

var driver = new webdriver.Builder().forBrowser('chrome').usingServer(' http://localhost:4444/wd/hub' ).withCapabilities(chromeCapabilities).build();

Python:

options = webdriver.ChromeOptions()

options.add_experimental_option("excludeSwitches", ["enable-automation"])

caps = options.to_capabilities()

driver = webdriver.Remote(command_executor=',

desired_capabilities=caps)

Ruby:

caps = Selenium::WebDriver::Remote::Capabilities.chrome("chromeOptions" => {"excludeSwitches" => [ "enable-automation" ]})

driver = Selenium::WebDriver.for :remote, url: 'http://localhost:4444/wd/hub', desired_capabilities: caps

The SetLogHandler method of Eyes activates the trace logs of Applitools Eyes SDK. Please find below an example of the syntax of this method in each of the languages. Note that this syntax writes the logs to the standard output.

If you want finer control on how the logs are processed, you can provide your own LogHandler implementation instead of StdoutLogHandler.

Java:

Print logs to console:

eyes.setLogHandler(new StdoutLogHandler(true));

Export logs to file:

eyes.setLogHandler(new FileLogger("path/to/file.log",true,true));

.Net:

Print logs to console:

eyes.SetLogHandler(new StdoutLogHandler(true));

Export logs to file:

eyes.SetLogHandler(new FileLogHandler("path/to/file.log",true,true));

Python:

Print logs to console:

from applitools.common import logger, StdoutLogger

logger.set_logger(StdoutLogger())

Export logs to file:

from applitools.common import logger, FileLogger

logger.set_logger(FileLogger("path/to/file.log"))

Ruby:

Print logs to console:

require 'logger'

eyes.log_handler = Logger.new(STDOUT)

Export logs to file:

eyes.log_handler = Logger.new('path/to/logfile.log')

JavaScript-Selenium:

Print logs to console:

var ConsoleLogHandler = require('eyes.selenium').ConsoleLogHandler;

eyes.setLogHandler(new ConsoleLogHandler(true));

Export logs to file:

var FileLogHandler = require('eyes.selenium').FileLogHandler;

eyes.setLogHandler(new FileLogHandler(true));

Note: The eyes.log file will appear under the root folder.

WebdriverIO 4 & 5:

Print logs to console:

const { ConsoleLogHandler } = require ('@applitools/eyes-sdk-core')

eyes.setLogHandler(new ConsoleLogHandler(true));

Export logs to file:

var { FileLogHandler } = require('@applitools/eyes-sdk-core');

eyes.setLogHandler(new FileLogHandler(true));

Note: The eyes.log file will appear under the root folder.

Espresso:

Print logs to Logcat:

import com.applitools.eyes.android.common.logger.LogHandler;

import com.applitools.eyes.android.common.logger.StdoutLogHandler;

LogHandler mLogHandler = new StdoutLogHandler(true);

eyes.setLogHandler(mLogHandler);

UFT:

Export logs to a file (add this before eyes.open):

eyes.LogFile = "eyes.log"

Storybook:

Export logs to a file (run this from the terminal/Console to start the test with logs):

Mac:

APPLITOOLS_SHOW_LOGS=true npx eyes-storybook >file.log

Windows:

setAPPLITOOLS_SHOW_LOGS=true

npx eyes-storybook >file.log

Cypress:

Export logs to a file (run this from the terminal/Console to start the test with logs):

APPLITOOLS_SHOW_LOGS=true npx cypress open >file.log

PHP:

Print logs to console:

$eyes->setLogHandler(new PrintLogHandler(true));

Export logs to file:

$eyes->setLogHandler(new FileLogger("path/to/logfile.log", true, true));

This article will explain how to configure a proxy server in the Applitools Eyes SDK to address issues such as connection refused or connection timeout when using Applitools Eyes behind a proxy server.

Please find below the instructions for setting a proxy server in the relevant languages:

Java:

eyes.setProxy(new ProxySettings("http://YOUR-PROXY-URI"));

or

eyes.setProxy(new ProxySettings("http://YOUR-PROXY-URI", YOUR_USER, YOUR_PASSWORD));

Please make sure to add the commands beforecalling eyes.open(...) and also import the below package:

import com.applitools.eyes.ProxySettings;

Ruby:

eyes.set_proxy(" https://proxy ", "user", "password").

Note that the user and password parameters are optional.

Python:

Configure the environment variable HTTPS_PROXY:

import os

os.environ['HTTPS_PROXY'] = "http://localhost:8888"

.Net (C#):

var proxyURI = new Uri(string.Format("{0}:{1}", proxyURL, port));

WebProxy proxy = new WebProxy(proxyURI, true, null);

eyes.Proxy = proxy;

Or:

var proxyURI = new Uri(string.Format("{0}:{1}", proxyURL, port));

ICredentials credentials = new NetworkCredential(username, password);

WebProxy proxy = new WebProxy(proxyURI, true, null, credentials);

eyes.Proxy = proxy;

JavaScript (@applitools/eyes-selenium)

var proxyInfo = {

url: pHost,

username: pUser,

password: pPass,

isHttpOnly: isHttpOnly

};

eyes.setProxy(proxyInfo);

JavaScript (eyes.selenium)

eyes.setProxy("http://YOUR-PROXY-URI");

or

eyes.setProxy("http://YOUR-PROXY-URI", YOUR_USER, YOUR_PASSWORD);

or

1. npm install tunnel-agent

2. add 'applitools-http-proxy.js' to your project :

var tunnel = require('tunnel-agent');

var https = require('https');

var __request = https.request;

var proxy = { host: '', port: 0 };

https.request = function (options, callback) {

if (options.host.indexOf('applitools') > -1) {

var tunnelingAgent = tunnel.httpsOverHttp({

proxy: {

host: proxy.host,

port: proxy.port

}

});

options.agent = tunnelingAgent;

}

return __request(options, callback);

};

module.exports = function (host, port) {

proxy.host = host;

proxy.port = port;

};

3. in your app in the beginning insert the line:

require('./applitools-http- proxy.js')("proxy_ip", port);

Full example:

var webdriver = require('selenium-webdriver');

var driver = new webdriver.Builder().

withCapabilities(webdriver.Capabilities.chrome()).

build();

require('./applitools-http-proxy.js')("84.42.3.33", 3128);

var Eyes = require('eyes.selenium').Eyes;

var eyes = new Eyes();

eyes.setApiKey(APPLITOOLS_APIKEY);

try {

eyes.open(driver, 'Applitools', 'Test Web Page', {width: 800, height: 600}).then(function (driver) {

driver.get(' http://www.applitools.com ');

/* Visual validation point #1 */

eyes.checkWindow('Test Web Page');

driver.findElement(webdriver.By.css('.read_more')).click();

/* Visual validation point #2 */

eyes.checkWindow('Features Page');

eyes.close();

});

} finally {

eyes.abortIfNotClosed();

}

driver.quit();

Version 4 of Eyes-Selenium for Python is a completely new SDK, and has a few changes that are important to know, ifyou are migrating from v3 to v4, or even if it is your first time using Eyes-Selenium for Python.

Packages

Version 3

eyes-selenium

Version 4

eyes-selenium

eyes-common

eyes-core

Import statements

Version 3

from applitools.eyes import Eyes, BatchInfo

from applitools import logger, StdoutLogger

Version 4

from applitools.selenium import Eyes, Target

from applitools.common import logger, StdoutLogger, BatchInfo

Check API

Version 3

Classic API: check_window(), check_region()

Version 4

Classic API: check_window(), check_region()

Fluent API: check()

Click here to learn more about the Fluent API.

Sometimes, when attempting to capture a full-page screenshot, the webpage does not scroll down, and only the visible viewport is captured.

When capturing a full-page screenshot, Eyes is trying to scroll the body of the page (or the document in some cases) in order to capture the whole page. However, sometimes within a specific page, there is a different scrollable element that needs to be scrolled which Eyes has not found automatically.

Sometimes this element needs to be found and specified in order to capture a full-page screenshot.

In order to determine which element is scrollable, we can investigate using the dev tools. In firefox, this is especially simple. After right-clicking on the webpage we want to investigate, and selecting "Inspect", the HTML of the webpage will be shown. In firefox, if an element is scrollable, the element will be marked with a "scroll" tag:

C#.

In other browsers, there is no scroll tag, but we can always select an element to check, and within the dev tools console, enter:

$0.scrollTop = 300

Meaning that if the selected element is scrollable, the element should move down by 300 (or any number preferred) pixels down. If nothing happens, the element is not scrollable.

An example of this method is shown below:

After finding the scrollable element and its selector, we simply insert it to the "By selector". We then capture a screenshot of the element by checking the region, and by adding the "fully" function, which tells Applitools to perform the scrolling mechanism. In Java, the code should look like this:

eyes.check("test", Target.Region(By.cssSelector("ScrollableElement")).fully());

This is supported in other SDKs as well.

Some examples of the fully interface within the Fluent API are: javascript and

Sometimes when looking at full screenshots, it's easy to miss small marks. For this purpose we’ve created a tool called ‘Radar’ and you can use it using the below icon:

You can use the right\left errors to shift between the radar's targets.

To see even smaller changes click on the ‘magnifyingglasses’ icon in the toolbox and red circles will pop up from the centers of the areas which include changes/differences.

You can double click on the area to zoom in, or use the zoom buttons from the toolbox:

In your tests, you might havescrollable elements within the main content, which you wish to visually validate.

For example, "step 4" in the Applitools tutorial contains a scrollable element:

checkRegion in nested Frames article.

As you can see in the example above, the viewport doesn't reveal the full element. In order to solve this issue, and visually validate the scrollable elements, Applitools Eyes can automatically scroll and stitch the element to capture the entire contents of the region.

In order to determine which element is scrollable, we can investigate using the dev tools. In firefox, this is especially simple. After right-clicking on the webpage we want to investigate, and selecting "Inspect", the HTML of the webpage will be shown. In firefox, if an element is scrollable, the element will be marked with a "scroll" tag:

In other browsers, there is no scroll tag, but we can always select an element to check, and within the dev tools console, enter:

$0.scrollTop = 300

Meaning that if the selected element is scrollable, the element should move down by 300 (or any number preferred) pixels down. If nothing happens, the element is not scrollable.

After finding the requested nested element and its selector, we simply insert it to the "By selector". In the above example, the code should look like this:

eyes.check("test", Target.Region(By.id("step4Content")).fully());

Note:

The"fully" function tells Applitools to perform the scrolling and stitching mechanism.

For further reading, please find the

The Image Tester is a semi-codeless stand-alone solution with which you can perform visual validations while using frameworks which are not yet supported by Applitools.

With Image Tester, the visual validation process is divided into two sections:

1.Taking the screenshots and saving them as files (user's responsibility).

After taking the relevant images, it is necessary to organize them in the following hierarchy:

Batch -> Test -> Step

Batch:

Batch is used to gathering multiple tests together. This will be a folder containing the tests which you want to gather under the same batch, and the batch name will be the name of that folder.

Test:

This will be a folder containing the images which you want to gather under the same test, and the test name will be the name of that folder.

Steps/Screenshots:

These are the images which you want to validate as steps, and the steps names will be the name of the image files.

2. Visual validating the screenshots with Image Tester.

After creating the folders hierarchy, download the Image Tester tool (attached).

Next, specify which tests/steps you wish to visually validate:

Open your CMD/Terminal and go to the ImageTester.jar location

To run a test, execute the following command:

Java -jar ImageTester.jar-k <YOUR_API_KEY> -f <BATCH_FOLDER_FULL_PATH>

For optional parameters see the following list or execute Java -jar ImageTester.jar

Optional Parameters:

-a [app-name] - Set the name of the application under test (Default: ImageTester)

-p [http://proxy{;user;pass}] - Set proxy and optional username and password

-s [server] - Set Applitools server URL

-ml [match-level] - Set the comparison level: Strict/Content/Layout (Default: Strict)

-br [branch] -Set the branch

-pb [parent-branch] -Set the parent branch

-bn [baseline-env-name] -Set custom baseline environment name

-vs [WidthxHeight] -Set the viewport size identifier

-lf [log-file] -Set log fine name to enable logging

-os [OS-name] -Set custom os name

-ap [browser-name] - Set custom browser name or host application name

-as - Set automatic save on failure

Note: Same as the fully coded solution, the first run will create your baseline, and from the second run, it will check for mismatches.

The Image Tester can also be used for validation of PDF files. Click HERE to learn how.

The Image Tester is a semi-codeless stand-alone solution with which you can perform visual validations for PDF files.

With Image Tester, the visual validation process is divided into two sections:

1. Organize the PDF files(user's responsibility).

It is necessary to organize the PDF files in the following hierarchy:

Batch -> Test -> Step

Batch:

Batch is used to gathering multiple tests together. This will be a folder containing the tests which you want to gather under the same batch, and the batch name will be the name of that folder.

Test:

This will be a folder containing the PDF files which you want to validate, each file is a test, and the test name will be the name of that file.

Steps/Pages:

These are the pages of the PDF file, and the steps names will be the index of the pages.

2. Visual validating the PDF files with Image Tester.

After creating the folders hierarchy, download the Image Tester tool (attached).

Next, specify which tests/steps you wish to visually validate:

Open your CMD/Terminal and go to the ImageTester.jar location

To run a test, execute the following command:

Java -jar ImageTester.jar-k <YOUR_API_KEY> -f <BATCH_FOLDER_FULL_PATH>

For optional parameters see the following list or execute Java -jar ImageTester.jar

Optional Parameters:

-a [app-name] - Set the name of the application under test (Default: ImageTester)

-p [http://proxy{;user;pass}] - Set proxy and optional username and password

-s [server] - Set Applitools server URL

-ml [match-level] - Set the comparison level: Strict/Content/Layout (Default: Strict)

-br [branch] -Set the branch

-pb [parent-branch] -Set the parent branch

-bn [baseline-env-name] -Set custom baseline environment name

-vs [WidthxHeight] -Set the viewport size identifier

-lf [log-file] -Set log fine name to enable logging

-os [OS-name] -Set custom os name

-ap [browser-name] - Set custom browser name or host application name

-as - Set automatic save on failure

-di [dpi] -Set the quality of the conversion on PDF files

-sp [pages] -Comma separated page numbers to include in PDF testing (e.g. 1,2,5,7)

-pp [password] -The password if the PDF file is protected

-pn -Preserve original test names when specifying pages

Note: Same as the fully coded solution, the first run will create your baseline, and from the second run, it will check for mismatches.

The Image Tester can also be used for validation of image files. Click HERE to learn how.

Comparing two different URLs (I.e: Prod URL and Pre-prod URL) can be done automatically using Applitools. To do so, you would need to create a test that will run twice with the same baseline parameters (same parameters in eyes.open), but open a different URL in each run.

As a result, the first test run will generate a baseline from the first URL and the second test run will be compared against it. Setting eyes.setSaveDiffs(true)in the first tests is done to ensure that when you run it in the future, the first test will always override the baseline and will ignore previous results.

Please find below a code example that will enable you to save the first URL as baseline and the second URL as a comparison version (a helper class in Java, as well as an example on how to use it, is attached below):

[]

Eyes eyes = new Eyes();

// This is your api key, make sure you use it in all your tests:

eyes.setApiKey(“API_KEY”);

// Makes sure that the 1st URL is saved as the baseline

eyes.setSaveDiffs(true);

driver = eyes.open(driver, “App Name”, “Test Name”, new RectangleSize(1024, 768));

driver.get(“First URL”);

// Visual validation point #1

eyes.checkWindow(“PageName”);

// End visual testing. Validate visual correctness.

eyes.close(false);

// Consecutive test run with the 2nd URL:

The following command will make sure that the 2nd URL does not override the baseline.

eyes.setSaveDiffs(false);

driver = eyes.open(driver, “App Name”, “Test Name”, new RectangleSize(1024, 768));

driver.get(“Second URL”);

// Visual validation point #1

eyes.checkWindow(“PageName”);

// End visual testing. Validate visual correctness.

eyes.close();

This feature came as a request by many of our customers who asked for an easy way to perform functional test scenarios using Applitools Eyes without requiring additional coding.

Thefloating regionis structured from 2 parts:

1) An inner rectangle - this is the area which you would like to validate (and can move).

2) An outer rectangle - this is the area where the inner rectangle can move in, and as long as the inner rectangle is within this area and the content is the same, no differences should be flagged. Note that the content within the outer rectangle will be ignored in terms of visual validation.

In order to define a floating region, all you need to do is click on the floating region icon (see image below) and mark the region that you want to validate.

After creating the region, the size of the boundary where this region is allowed to appear (or "float") can be specified as appears below.

Here are a few examples where floating regions are useful:

Running a test to validate credit card transactions, where during the test a specific transaction is generated and the test expects to find this specific transaction in a table, but all other transactions are random, and the order of the transactions might be different every time you run the test.

Running a Google search for fixed term for which a certain result is expected, but may appear at a different position in each test run (see image below).

A page displays random data which causes another fixed part of the page to appear at a different position on each test run.

Note:As when using a floating region, a content comparison ismade to validate that the same element is shared between thebaseline and current images (with some movement tolerance) - a floating region has no relevance while comparing with the Layout match level, which ignores the content and focuses only on validating the structure of the page.

When using Applitools Eyes JavaScript SDK and Jest, there is sometimes slower than normal performance.

To resolve this, add the following to the package.json file:

{

"jest": {

"extraGlobals": ["Math"],

"testEnvironment": "node"

}

}

The Eyes SDK needs the Math package to do some image processing before uploading screenshots to Applitools. Since Math is available on the shared memory by default, this causes performance to dramatically slow down, as now Jest needs to synchronize the VMs' access to a shared resource (locking it and adding access checks). This could cause the Eyes SDK to slow down by a factor of 2 to N times, depending on the number of VMs accessing Math at the same time.

You can read more about this in Jest's documentation here.

Note: This isnot relevant to tests run on the Applitools Ultrafast Grid.

Applitools Eyes provides you the option to set different match levels within the test setup. The different match levels refers to different comparison methods between the current screenshot and its baseline. For further info regarding Applitools different match levels click here>>>

The Match regions feature provides you the ability to define a specific region in an image in which you can set a different match level from the global one. This featurecomes in handy when having to validate several different content types on the same image.

Note that except for setting the region match level to "Ignore", setting a specific match level for a region is available only via Appltools dashboard.

For example, let's take a look this Amazon page. This page is static but contains a changing ad.

click here>>>

To run a visual test on this page with a certain (which is not layout) match level, and overlook the ad changes, you have two options:

a. Use "Ignore region" If you'd like to Ignore this region completely

b. Set the ad region match levelto LAYOUT, and the global match level of the page to STRICT. or .CONTENT In that case, the test would not ignore the region completely, but will compare the type of element in that region to its parallel in the baseline.

To do so, you should follow a few easy steps:

Set the global match level to the desired one. In this case, it would be STRICT (for further info about setting the global match level ).

Click on "Layout region", you can find it on the upper toolbar as can be seen in the following image:

Define the relevant region to which you want to set a different match level.

Optionally, set the match level of this region as you wish.

Important: you can set a different match level for a specific region only when the global match level is not LAYOUT.

This article will explain How to Create an Ignorable Region on a SpecificScreen

Sometimes, different parts of the screen need to be tested ina different manner.

One of the ways to perform an advanced analysis of a window is with the"Ignore Region" Function.

Users can mark aspecific area of the screen that Eyes will not assess - Changes within this area will be ignored.For example, areas that have more than one expected result can be ignored and therefore will not cause Failed tests.

This Option can be critical when working with a dynamic app or website. There may be areas that are supposed to change every few seconds or minutes and therefore should never be identical to your baseline version.

One of the most useful features of the ignore region tool is that a defined region on one baseline will be automatically assigned as ignorable on all the other baselines that include the same region.

Areas that areespecially relevant:

Areas that have been designated for advertisements

Media Players

Animations

Date Fields

Areas displaying specific user details

document

To add an Ignore region, simply select it from the regions drop-down list of the Annotations section, draw the ignore region on top of the area that you would like to ignore and then click the save button.

See also this about automated test maintenance to reduce your test maintenance overhead when using regions and more from the test manager.

In Visual Testing, highlighted differences can be the result of either intended change in the UI of the application, which is a natural evolution of the product, or as a result of a bug. The first type of differences is handled by updating the baseline with new captured screenshots. If the differences are a result of a UI bug, there are two use cases that should be addressed:

a) Inadditionto the highlighted UI bug, there isalsoan intended change that should be included in the baseline for future test runs. In this case, ideally, we would like to have the option to accept the new screenshot as the baseline and still have that bug fail our future tests.

b) The UI bug is theonlyhighlighted difference and is a known bug, which is currently being worked on. This situation causes unnecessary time consumption of testers, who are required to manually validate the failed Applitools results due to that bug. In this case, ideally, we would like to have a "snooze" button on the failure caused by the bug, which could be set until the estimated time for fixing that bug.

In order to handle the aforementioned situations, Applitools added the option to add aBug Regionin theTest Result Editor.

Bug Regionis enables marking regions as bugs, addressing the two scenarios above, and allowing collaboration between all relevant team members (testers, developers, product managers, etc.).

Remark Regions

How to add a Bug Region:

1.Click on the exclamation mark.

2.Drag the mouse over the region of the bug to select it, as seen in the figure below:

3.Insert a title for this region, describing the bug.

4.Click the "Create" button.

As can be seen in the figure above, all collaboration regarding the bug, between the different team members, appears in the chat box.

5. In the first scenario (a) in whichinadditionto the highlighted UI bug there isalsoan intended change,adding a bug region andchecking the option'Fail tests'in the checkbox prior to accepting the baseline willupdate the new screenshot as the baseline image. It will also cause all future runs to fail until the bug region is removed or the "Fail tests" checkbox is unchecked.

In the second scenario (b) in whichthe UI bug is theonlyhighlighted difference and is a known bug,adding a bug region,checking the Fail testsand setting theSnoozeoption, as seen below, will not cause future tests to fail due to that buguntilthe selected date. (Other visualdiscrepancies may still fail the test.)

Removing, copying, and sharingthe bug region is done using the three icons on the top right corner of the region.

In the dashboard page, filtering Batches containingBug Regions orcan be done using the "steps annotation" view mode as shown below.

When passing viewport size to eyes.open(...) the size should be referred to the inner area of the browser (The viewport), Our SDK calculates the desired outer window size according to the given params and resizes it accordingly.

There are four main reasons for when setting the viewport size can fail:

The calculated window size is bigger than the available size of the screen.

The size is too small for the browser (each of the browser has a minimum size that is supported).

When using appium, don’t try to pass a viewport size, otherwise a failure will occur, as in mobile devices the window is always maximized.

On windows machines, the zoom of the display must be set to 100\%, otherwise the calculation of the browser size, according to the viewport size will fail and throw an error.

In this article you will learn how to copy baselines from one branch to another.

Step 1:

Navigate to the Dashboard and go to Branch baselines

Step 2:

In the top right corner of the page, select the branch that you want to copy from

Step 3:

Mark all the baselines that you want to copy and click on the "copy baselines to branch" button

Step 4:

Put the branch name that you want to copy to and click on the "copy to branch" button

The next article will explain how to adjust your test's Match Level in your code.

For your convenience please find the video guide below:

When writing an automated visual test, sometimes we will want to change the comparison method between our test and its baseline, especially when dealing with applications that consist of dynamic content.

Applitools Eyes provides you the option to set different match levels within the test setup:

here

There are a few match levels that can be selected. All theoptions can be found under the MatchLevel enumerable. When not specified, the default match level is "STRICT" which is designed to mimic the human eye.

The match level is defined at the test level (in future versions you will be able to set it in the step level or set it directly from the UI for specific regions). To set the comparison level use the following syntax:

C# (set before eyes.open)

eyes.MatchLevel = MatchLevel.<selected_level>

Java (set before eyes.open)

eyes.setMatchLevel(MatchLevel.<selected_level>)

Javascript (set before eyes.open)

eyes.setMatchLevel("<selected_level>");

Python

from applitools.eyes import MatchLevel

eyes.match_level = MatchLevel.<selected_level>

Ruby

eyes.match_level = :

eyes.match_level = Applitools::MATCH_LEVEL<selected_level>

UFT\QTP (Before open, either one of the lines according to the desired match level)

eyes.MatchLevel(eyes.Match<selected_level>)

Applitools Eyes can test the UI in 4 different comparison levels:

Exact (MatchLevel.EXACT) - pixel-to-pixel comparison, not recommended. Exact performs a pixel-to-pixel comparison with predefined threshold. This mode was added mainly for demos and is likely to cause false positive results, therefore, we wouldn't recommend using it.

Strict (MatchLevel.STRICT) - Strict compares everything including content (text), fonts, layout, colors and position of each of the elements. Strict knows to ignore rendering changes that are not visible to the human (anti-aliasing changes, small pixel movements and various other changes that are typically caused when running tests on different machines with different graphic cards, etc.). Strict is the recommended match level when running regression tests on the same browser/OS(strict is not designed to do cross browser comparison).

Strict comparison will detect differences in the content/text, so although it does not do OCR by default, it can still detect character changes. There is an option to activate OCR on elements in the .Net SDK, but in most cases we don't recommend using it.

Content (MatchLevel.CONTENT) - Content works in a similar way to Strict except for the fact that it ignores colors (the content level can be useful ifyour website includes different colors which are not relevant for your tests).

Layout (MatchLevel.LAYOUT) - Layout, as its name implies, compares the layouts (i.e. structure) of the baseline and actual images. It validates the alignment and relative position of all elements on the page, such as: buttons, menus, text areas, paragraphs, images, and columns. It ignores the content, colour and other style changes between the pages.

The Layout algorithm reverse engineers the structure of the compared images by applying complex image processing algorithms resulting with the logical page structure of each image that includes all the components of the page. Then, it proceeds to map each component with its counter part in the other image which allows detection of missing or new elements as well as misplaced ones. All such elements are considered different and are highlighted as such in the comparison result.

Layout will have greater benefits than other match levelin certain scenarios, which most notably include: dynamic content, language localisation, responsive design, and cross-environment testing (including: operating systems, browsers, devices, and form factors) sinceit is more flexible approach of visual testing will have greater benefits.

Note: As when using a floating region, a content comparison ismade to validate that the same element is shared between thebaseline and current images (with some movement tolerance) - a floating region has no relevance while comparing with the Layout match level, which ignores the content and focuses only on validating the structure of the page.

Applitools dashboard allows you to examine what the result of the test would have been had you run it using the different match levels.

For a full match level explanation video, please click .

The Applitools Eyes Admin Panelpage allows you to view your account's details, and manage users and teams in your account.

Access to the admin panel

The Admin panel is accessible only for the account and team admins. Admins can go to"Admin" option in the dropdown menu:

article

In the Admin panel you will find three options

Account- View the account settings - account admins, usage andconcurrency limits.

Users- View and manage users in theaccount - users details, and privileges.

Teams- View and manage teams in theaccount - teams are a shared workspace to see and manage tests results.

Note:Some options might be available only in certain plans and deployment modes. For further information see the comparison in our pricing page.

Adding users to a team

Go toTeamspage.

Click on the team that you want to add a user to from the Teams list.

InMEMBERSsection, type the user email address or select from the drop-down lists (displays all users of the account) and click the "Add" button.

Click the "Save" button to save changes.

If the user does not exist in the account yet, you will be prompted to approve its addition via the "Add New User" dialog.

In this dialog box, you can edit the proposed username, full name and email of the new user.

Once added, the user will receive a "Welcome" email that includes a link to the team's dashboard in Applitools Eyes.

Note:First time users might be required to go through the signup process to set a few details such as password.

Set user privileges

There are three modes of privileges for team members: Admin, Regular and View only

* A team admin can only add users to the teams they are an admin on, an account admin can add users to the users list and to any team under that account.

To change user privileges go to Teams page.

Click on the team that you want to change it's member's privileges.

In the MEMBERS section, check the requested checkbox next to the user name and save.

Note:users who are Account admins already granted with the highest privileges and don't need further assignment on team's level.

Remove user from a team

Go toTeamspage.

Click on the team that you want to remove a user from.

Hover with the mouse over the user that you want to remove, under theMEMBERSsection, and click on the "X" next to the user's details.

Click the "Save" button to save changes.

Add new team

Go to Teams page.

Click the "Add new team" button.

In the "Add New Team" dialog enter a name for the new team and click the "Add" button.

Disable a team

Go toTeamspage.

Click on the team that you want to disable.

InGeneralsection, uncheck the "Enabled" checkbox.

Click the "Save" button to save changes.

To enable the team, simply check the checkbox and save again.

Note:Disabling a team will prevent access to the test results of the team.

Delete a team

Warning: deleting a team will result in permanent deletion of all the results and baselines associated with this team.

Go toTeamspage.

Check the checkbox next to the team's details.

Click the "Delete" button.

Delete user from the account

Go to Users page.

Check the checkbox next to the user's details.

Click the "Delete" button.

Note:Deleting a user will remove it from the account as well as from all the teams it is a member of.

API keys

Each team has a distinct set of API access keys to perform different tasks.

The Run key - This is the key used by the team to execute tests.

The View key - Provides the ability to read results and test information via the API. (See Results Handler and EyesUtilities articles)

The Update key - Allows API operations that require updates such as Branch merging.(See the Branches and EyesUtilities articles)

You can find those keys in Teams page under API KEYS section.

Integration with Jira

Applitools integrates with Jira and allowing you to mark areas in steps as " Bug Regions " and open/associate Jira issues with them copying test details to Jira.

To set up the integration with Jira, in the Admin Panelgo to Teams page.

Click on the desired team that you want to integrate with Jira.

In the JIRA section, check the "Enabled" checkbox.

Set your Jira server URL and additional mandatory fields.

You can also add Jira settings in Default Settings area for all tests, or for specific application by using the "Add" button (optional).

To learn more about Jira integration in Applitools Eyes see this .

The next article will explain how to adjust your viewport size.

For your convenience, please find the following video guide:

In order to get consistent results in your automated visual tests, it's important to make sure that your browser's size is the same on each test run.

Selenium provides the option to set your window size, which includes the entire browser window, as you can see below:

http://viewportsizes.mattstow.com/mine/

Applitools eyes provides a method based on Selenium, and addsthe ability to resize the window according to thedesiredviewport size, which consist the actual content frame, as you can see below:

This method considers tool bars,plugins, and other elements that can resize the content frame.

In order to adjust the viewport size, we need to specify it in our test, as mentioned below:

following this line, we need to specify the desiredwidthand height parameters.

How can we get the current viewport size of our browser?

There are 2 different ways to get this information:

1. JavaScript console, by following the next steps:

and writing instant commends:

2. alternatively, we can go into a website that provides the requested information, for example:

After getting the viewport size information, by either of the methods, we can use it to recompose the "rectangle size" parameter in our test:

After defining the viewport size and running our test, we can see that the new parameters took effect in the new results, instead of the browser's default size parameters.

The next article will explain how to create a batch which includes several tests.

For your convenience please find the following video guide below:

setBatch is a helpful Applitools Eyes command, that provides us the ability to gather multiple tests and run them together.

After setting your API key, you can go ahead and add the next commands:

eyes.setBatch (new batchInfo("login")

After creating a new batch info command, you can go ahead and specify the string with the name of the batch (e.g "login").

Note that by using this method the results of your tests will be displayed as two separate batches:

The reason the tests appear separated has to do with the method written before each test:

In order to cluster the tests together, we need to change a few commands in our test code:

First, we need to create a batch info object before each of the tests, and reference the object to each of our tests.

To do that, we need to create a BeforeClassmethod, that will run before the TestClass, and set it as a static method called setBatch.

Next, instead of just naming the string, as we did before, we need to store it in a batch field variable, which we need to create following this step:

Next, we can go down to the previous setBatch command and create a few little changes:

Before the last change: After the last change:

After making those changes and running the test, we will see the next results in our test results:

As you can see, instead of having two separate test results as before, the tests are now batched together under the same result.

In unique cases, where you need to aggregate tests from different instances, processes or machines (with potentially different batchInfo objects) ApplitoolsBatch ID command can be used.

The batch ID is one of the batch's optional parameters (other than batch name for example). Setting the same Batch ID to each of the relevant tests which you wish to aggregate together will group them under the same batch.

Note: Make sureto generate a uniquebatch ID for each of your batches, which will prevent irrelevant tests to be grouped under the same batch.

Java

/* Create the batch object and set the ID */

BatchInfo b = new BatchInfo("BATCH_NAME_HERE");

b.setId("SOME_UNIQUE_ID");

/* Set the batch to Eyes */

eyes.setBatch(b);

Javascript

/* Set a new batch to eyes. */

eyes.setBatch("BATCH_NAME_HERE", "SOME_UNIQUE_ID");

Python

''' You need to make sure to import the BatchInfo class '''

from applitools.common.config import BatchInfo

''' Create the batch object and set the ID'''

b = BatchInfo("BATCH_NAME_HERE")

b.id_ = "SOME_UNIQUE_ID"

''' Set the batch to Eyes '''

eyes.batch = b

Ruby

# Create the batch object and set the ID

b = BatchInfo.new "BATCH_NAME_HERE"

b.id = "SOME_UNIQUE_ID"

# Set the batch to Eyes

eyes.batch = b

XCUI - Swift

/* Create the batch object and set the Name */

let my_batch = BatchInfo(name: "my_batch")

/* Set the batch to Eyes */

eyes.batch = my_batch;

C#

/* Create the batch object and set the ID */

BatchInfo b = new BatchInfo("BATCH_NAME_HERE");

b.Id = "SOME_UNIQUE_ID";

/* Set the batch to Eyes */

eyes.Batch = b;

On-premise and Private cloud deployment modes allow customers to work with their own private Applitools Eyes server (as opposed to Applitool's public server).

In order to use your private server when running tests, youneed to explicitly declare which server you are working with (usually in the Initialization.

Following are syntax examples in all available languages:

Espresso:

Eyes eyes = new Eyes(URI.create("https://<org>eyesapi.applitools.com"));

FYI - You may need to add the below dependency to your build.gradle

implementation 'com.fasterxml.jackson.core:jackson-databind:2.9.7'

Java:

eyes.setServerUrl(new URI("https://<org>eyesapi.applitools.com"));

JS/Protractor:

var eyes = new Eyes("https://<org>eyesapi.applitools.com");

Ruby:

eyes = Applitools::Eyes.new(server_url:'https://<org>eyesapi.applitools.com')

Python:

eyes = Eyes('https://<org>eyesapi.applitools.com')

C#:

var eyes = new Eyes(new URI("https://<org>eyesapi.applitools.com));

QTP:

eyes.ServerUrl = "https://<org>eyesapi.applitools.com"

Applitools Eyes provides automatic maintenance, which is designed to simplify and to expedite the visual validation process.

Applitools provides three different methods of automatic maintenance:

1.Automatic Region lookup:In some cases, adding different regions to your test (e.g: floating region, ignore region) is necessary in order to deal with dynamic content. In tests that contain multiple steps with the same elements, enabling automatic region lookup (which is disabled by default) adds the same regions to each of the familiar steps by searching and copying the relevant regions to the content. Make sure to turn it on in order to enable it in your test.

2.Automatic accept/reject:This automatic function allows you to accept or reject all of the test's steps that contain the same set of changes.

Disabling automatic maintenance:In order to disable both automatic region lookup and the accept/reject, you can simply addauto=falseparameter to the end of the results URL.

3.Region propagation for new baselines:When working with different environments (different browsers, OS, etc), the automatic propagation copies the regions we specified in our test to the baseline of the new environments.

This article was removed.

See our new documentation about how to set baseline environment name here: https://applitools.com/docs/api/selenium-ide/method-eyes-setbaselineenvname-seleniumide-command.html?Highlight=baseline\%20environment\%20name

See our new documentation about cross-environment tests here: https://applitools.com/docs/topics/general-concepts/working-withcross-environment-tests.html

This article has moved to our new documentation section. Please click here for information about Applitools integration withAzure DevOps.

Applitools Eyes Admin Panel allows you to manage users and teams in your account. It is only accessible for account and team admins.

You can access the admin panel by clicking the "Admin" menu item:

this

To learn more about the tasks you can accomplish with the admin panel, go to Applitools Wiki page.

Transport Layer Security (TLS) is a widely deployed security protocol used today for web browsers and other applications that require data to be securely exchanged over the Internet. TLS aims primarily to provide privacy and data integrity between two or more communicating computer applications.

Eyes SDKs are using this protocol to communicate with the Applitools servers and currently support TLS1.2 or higher.

If the framework that you are using is not supporting TLS1.2 by default, please try to use the following code to force it to use this version.

Use the following code before calling to eyes.open()

C#:

System.Net.ServicePointManager.SecurityProtocol |= System.Net.SecurityProtocolType.Tls12;

Java:

System.setProperty("jdk.tls.client.protocols", "TLSv1.2");

UFT/QTP:

Dim servicePointMgr, secProtocolType

set servicePointMgr = DotNetFactory.CreateInstance("System.Net.ServicePointManager", "C:\Windows\Microsoft.NET\Framework\v4.0.30319\System.dll")

Set secProtocolType = DotNetFactory.CreateInstance("System.Net.SecurityProtocolType", "C:\Windows\Microsoft.NET\Framework\v4.0.30319\System.dll")

servicePointMgr.SecurityProtocol = secProtocolType.Tls12

In order to integrate Applitools test results Dashboard into your TeamCity Build Report, you'll need to use Batches, a feature that allows you to aggregate several tests together.

To learn more about the implementation, please visit the following Github page.

* Notice, if you are using a Docker container, the APPLITOOLS_BATCH_ID environment variable would have to be set in the "Additional Docker parameters" section.

There are 3 deployment modes availablefor Applitools Eyes:

Enterprise Cloud In this mode, each customer gets his own accounts in our enterprise cloud infrastructure with a separate container that contains the results of his tests. In this case, the baseline images will be stored in the cloud.

Private Cloud Dedicated Server In this mode, we allocate dedicated machines in our cloud infrastructure that are only accessible by a specific customer with a 2 layer authentication. This modeis also based on certificate and network isolation in addition to the user/password/API key, so only people from your organization which were granted specific access permissions/certs will be able to access it.

On-Premise deployment In this mode, we install the Applitools Eyes server locally inside the customers' organization network and the images will be stored locally. This option involves some overhead from our side (remote upgrades, service packs, etc.) and therefore the cost of this option is higher.

Note that both Private Cloud and On-Premise deployment modes allows Single Sign-on (SSO) using SAML and LDAP authorization protocols.

In this article, we will explain how to get Eyes logs for Selenium IDE.

1. After running your test -> open the browser and navigate to the following URL:

2. Enable the developer mode:

3. Click on the background page link - It will open a DevTools window.

4. You can see the logs in the console pane inside the dev tools windows.

Grouping steps by similar differences feature is an innovative clustering option, that aggregates the different steps of your tests by similar differences, and makes the process of analysing and maintaining large-scale automated test suite quick and simple.

To use the “Grouping” feature, click on the following button, found on the top menu of the dashboard:

and wait a couple of seconds

The images will be grouped by exact similar differences throughout all the steps (e.g. a change in the header across the entire batch), so you can quickly analyze and accept/reject numerous steps with a single click.

In the example below, you can see that the test has detected the same mismatch (visual difference) across 76 steps in the batch and found 2 groups of mismatches; those steps failed due to the same mismatch, so instead of reviewing each of the steps individually and accepting the same change over and over again, the new grouping feature automatically detects that the same mismatch was found in all 76 steps and allows you to accept all at once.

here

Slack integration tutorial can be found

Or watch this video:

This article will explain:

Why setting the viewport size is crucial for automated visual tests

How to setup the browser viewport size in your Applitools Eyes tests.

Please note that you can also learn how to set you viewport size here.

Setting a fixed resolution for automated visual tests is mandatory to ensure consistent results. The UI of the application under test can behave differently in different sizes, and therefore, we recommend using the option to configure the viewport size at the beginning of the test to ensure the test results are repeatable on various configurations. If you do not configure the viewport size, the next time you run the test, the browser might open up in a different size which may cause failures in your visual test. There are twoways to setthe browser'ssize: usingthe selenium web-driver or usingthe Eyes.open method.

Note that this results in changing the browser size to have its viewport (viewable content frame size) to match the parameter and not the entire browser window. If failed to do so, the SDK will throw an exception (e.g., setting 1024x768 on a mobile device).

Settings the viewport size using Eyes.open is preferable,since it setsthe size of the client area of the browser (which is the actual size of your web page)rather than that of the browser window (which is affected by the browser,window borders and caption).

Also, if Eyes.openfails to set the required viewport size, the test will be aborted before starting. This will not happen when failing to set the browser window size using the web-driver.

See the SDK guide for examples how to set the viewport size using Eyes.open in your preferred programming language.

Investigating failures to set ViewPort size:

The main reason for failures in this step is when the selected size is too large for the current size of the window (note that Applitools Eyes tries to set theinnerbrowser window size to the selected size while ignoring toolbars and status bars while Selenium sets the size of the entire window, so there are some cases where Selenium setSize will work while Applitools setSize will fail). In order to determine the maximum size that can be set for theinnerbrowser window on your machine, you can open your browser in its maximum size and open the browser's developer tools,browse to the console tab and use the commands:

window.innerWidthand window.innerHeight (as shown in the following image).

These commands will show thewidth and height values respectivelyof the current viewport size(which arethe maximum possible values, if the browser is maximized).It is recommended touse a number that is a bit smaller than the result(i.e. if the result is 700, use 690) for your tests.

See example screenshot below:

Tip: When opening a batch of tests in Applitools Eyes Tests Dashboard,you can see for each test the OS, browser and viewport size. This combination defines the baseline. So if you see a new test that should not be "new" take a look at these values. You can also see these values at the top right corner of the screen when you open a test.

In this article you will learn how to integrate the Applitools test results Dashboard into your Jenkins Build Report. This integration requires using Batches, a feature that allows you to group several tests together.

Behind the scenes, Applitools Jenkins plugin generates a unique batch-id passed via env. variable to the tests and tracks the batch in job's build report. Every Jenkins job will show you the results of a specific batch of tests in Applitools.

Installing the Applitools eyes Jenkins plugin:

Go to the project in Jenkins that you want to integrate. Then click on "Manage Jenkins" (1) and then go to "Manage Plugins" (2).

official Jenkins plugin documentation

Then go to the"Available" plugin list (3) andsearch for "Applitools" (4). After the search,install the "Applitools Eyes Plugin" (5) without restart (6).

Project configuration - Free style project

Go back to your Jenkins project, and click on "Configure" (7).

Now you can find under "Build Environment" an option titled "Applitools support" (8).If you're using the public cloud, leave the URL as it is, but if you're using a private cloud, change the URL according to your server's address and press "Save".

Project configuration - Pipeline project

To use the Applitools plugin in a pipeline project, you need to add the Applitools() directive and put your run code in a block. Following is a script example:

node {

stage('Applitools build') {

Applitools() {

sh 'mvn clean test'

}

}

}

If you are using a dedicated Applitools Eyes server, you should update the Applitools URL accordingly inside the Applitools directive. For example:

node {

stage('Applitools build') {

Applitools('https://myprivateserver.com') {

sh 'mvn clean test'

}

}

}

Linking the Applitools batch ID with the relevant Jenkins Job:

The batch-ID parameter is passed through an environment variablecalled APPLITOOLS_BATCH_ID to the executed tests, and the build name is passed through anenvironment variable called APPLITOOLS_BATCH_NAME.

To set the batch name and the batch-ID in your test code, use the following syntax before starting your tests:

JAVA:

//Set only once per Jenkins job

BatchInfo mybatch = new BatchInfo(System.getenv("APPLITOOLS_BATCH_NAME"));

mybatch.setId(System.getenv("APPLITOOLS_BATCH_ID"));

//End of - Set only once per Jenkins job

eyes.setBatch(mybatch);

Ruby:

#Set only once per Jenkins job

batchInfo = Applitools::BatchInfo.new(ENV["APPLITOOLS_BATCH_NAME"])

batchInfo .id = ENV["APPLITOOLS_BATCH_ID"]

#End of - Set only once per Jenkins job

eyes.batch = batchInfo

Python:

import os

from applitools.eyes import BatchInfo

#Set only once per Jenkins job

batchInfo = BatchInfo(os.environ['APPLITOOLS_BATCH_NAME’])

batchInfo.id_ = os.environ['APPLITOOLS_BATCH_ID’]

#End of - Set only once per Jenkins job

eyes.batch = batchInfo

.NET:

///Set only once per Jenkins job

batchInfo = new BatchInfo(Environment.GetEnvironmentVariable("APPLITOOLS_BATCH_NAME"));

batchInfo.Id = Environment.GetEnvironmentVariable("APPLITOOLS_BATCH_ID");

///End of - Set only once per Jenkins job

eyes.Batch = batchInfo;

JavaScript:

eyes.setBatch(process.env.APPLITOOLS_BATCH_NAME, process.env.APPLITOOLS_BATCH_ID);

This is it - you're done!

The next time you build a project with Jenkins, you will be able to get the Applitools test results integrated into your Jenkins Build Report, as can be seen below:

For a video tutorial about the Jenkins integration, please visit our blog post, and for further detailsplease visit our .

Bobcat is an automated testing framework for functional testing of web applications. It wraps Selenium, so anything possible in raw Selenium can be done with Bobcat, including running it on all supported browsers, Selenium Grid and cloud providers You can also utilize Appium for mobile testing. Bobcat comes with a handy set of utilities and accelerators that will help you develop your tests faster.

Assuming you followed our Selenium Java SDK Tutorial,here are the additional steps you need to take to add Applitools to your Bobcat framework:

Add this Maven dependency to your project pom file

<dependency>

<groupId>com.applitools</groupId>

<artifactId>eyes-selenium-java3-jersey1x</artifactId>

<version>RELEASE</version>

<exclusions>

<exclusion>

<groupId>org.seleniumhq.selenium</groupId>

<artifactId>selenium-java</artifactId>

</exclusion>

<exclusion>

<groupId>io.appium</groupId>

<artifactId>java-client</artifactId>

</exclusion>

<exclusion>

<groupId>org.seleniumhq.selenium</groupId>

<artifactId>selenium-api</artifactId>

</exclusion>

</exclusions>

</dependency>

Get the (wrapped) webDriver from Bobcat to be used when calling eyes.open :

WebDriver wrappedDriver = ((EventFiringWebDriver)webDriver).getWrappedDriver();

Run your script.

In this article wewill learn how to create a new Jira issue directly from the Applitools Eyes test result

There are three main steps in the process

Retrieve Jira project details

Configure Applitools and Jira server

Create a new Jira issue directly from the Applitools Eyes test result.

Retrieve Jira project details (pid and issueType) from Jira dashboard

Open JIRA dashboard

(Top-Right corner) JIRA Administration -> Projects

issue regions

Click on the project you wish to connect

Click on Details and grab the PID from the URL and put it in Applitools settings

Click on Bug (example issue type) and grab the issuetype ID from the URL and put it in Applitools settings

Configure Jira next-gen:

Step 1: go tohttps://<YOUR_JIRA_SERVER>/rest/api/3/project/

Step 2: in the JSON, search for the project and find its ID field

Step 3: go tohttps://<YOUR_JIRA_SERVER>/rest/api/3/project/<PROJECT_ID>

step 4: in the JSON note the IDs of the issue types that you’d like to use

Set the extracted values in the issuetype text box in applitools admin panel.

Configure Applitools and Jira server

The first step is to set up the integration between Applitools Eyes and Jira:

To enable the integration, navigate to Admin panel and locate your team.

Under the Jira section, enable the integration and enter the following details:

JIRA Server URL

pid (Project ID) - in which new issue will be created

issuetype - id of the type of issue you create as defined in your Jira project

Optionally - adding custom issue fields can be achieved by clicking on the add button, use the dropdown to select built in fields or directly type the name of custom fields in the text box.

To learn more about how to identify Jira issue fields and theirs possible values read the link.

# You can click the preview link at any time to test your configuration.

If your team is developing different application that required different Jira settings click the "Add application specific settings" button to add an application specific configuration that will override the default one.

Once you are done click the save button to save the configuration.

After the integration is configured, you can easily create Jira issues directly from Applitools Eyes .

For example: Create a "missing"issue :

To create a matching Jira issue, click the "Create new Jira issue" which open the Jira create issue form in a new tab with the details of the issue, the test and any others fields you configured in the Admin panel.

At this point if you are ready to create the Jira issue, you can click on the create button.

The resulting Jira issue includes:

* The name of the test

* The batch name

* The steps description

* Thecreated time

* The execution environment details

* Andany custom properties you define for the test.

Clicking the "View in Applitools Eyes" open the corresponding Applitools Eyes issue when you can zoom in on changes, see the issue in the context of the entire UI page and compare it with the baseline.

You can also linked the new Jira issue with your Applitools Eyes issue by entering its id in the "Link in Jira issue" - this will allow you to load the Jira issue by clicking the "View in Jira" button directly from the issueregion.

If needed you can replace the current link of Jira issue with another or disconnect an existing link.

The next article will explain how to deal with shifting content while using an automated visual test.

For your convenience, please find the following video guide:

When writing visual automated tests, we can encounter an issue caused by a shifting content\elements.

In a visual test, shifts of an element (e.g: image) will cause the test to fail:

Thefloat regionwas designed to deal with this exact problem, in which the test fails, even though no actual anomaly was taken place.

The floating region allows us to select an area in the page, and create a "tolerance" area, in which the image is able to move without failing the test.

After defining a floating region, we can go ahead and accept and save it as our baseline.

Another example of the necessity of the floating region is when we have an element in our application, that we want to visually verify, but the location of the element constantly changes.

In the following example, we want to verify the text line "important information you're looking for" appears, regardless it's position.

In cases of pages where elements tend to switch places, the tests are more likely to fail. This can be solved by applying a floating region. The floating region is able to ignore random and insignificant location changes in our visual test. The difference between those two examples is the fact that now we will adjust a bigger region manually, in which the element can shift, instead of just enclosing the item that we want.

You can find a video explaining the different uses of the floating region at the beginning of the article.

Note:As when using a floating region, a content comparison ismade to validate that the same element is shared between thebaseline and current images (with some movement tolerance) - a floating region has no relevance while comparing with the Layout match level, which ignores the content and focuses only on validating the structure of the page.

In your tests, you might have nested DIVs within the main content, which you wish to visually validate.

For example:

As you can see in the example above, the "SQL" element is nested inside an iframe.

In order to solve this issue, Applitools Eyes helps you identify and test regions within nested frames quickly and painlessly by automatically changing the webdriver context to the target frame, scrolling and stitching to capture the entire contents of the region and reverting to the original context all with a single CheckRegion command.

After finding the requested nested element and it's selector, we simply insert it to the By selector. In the above example, the code should look like this:

eyes.checkRegionInFrame(FrameNameOrId, By.className("SQLelement"), "MyRegion");

Match level per step provides you a coded ability toset a different match level for a specific step of the test (a reminder: a step in Applitools visual tests includes a comparison of a single image).

For example, you might have a test match level of STRICT and you want to change the match level of the first step to LAYOUT. (To learn how to set a test match level click here )

Note thatthis option is only available via code, and not via the Applitools dashboard.

To change the match level of a certain stepplease find the following syntax:

Ruby:

target = Applitools::Selenium::Target.region(:css, 'CSS_SELECTOR').match_level(:layout)

eyes.check "StepName", target

Javascript:

eyes.check("StepName", Target.region(By.css("CSS_SELECTOR")).matchLevel(MatchLevel.Layout));

Java:

eyes.check("StepName", Target.region(By.cssSelctor("CSS_SELECTOR")).layout());

C#:

eyes.Check("StepName", Target.Region(By.CssSelector("CSS_SELECTOR")).Layout());

While using Applitools Eyes, you might encounter issues with the eyes.open() command.

For example, you were in theprocess of implementing Eyes for HP QTP (UFT), and the following error message appeared:

Open(): No baseline info was set!

Function file: C:\Eyes\Eyes.qfl

Line (260): " eyes_.Throw "Open(): No baseline info was set!"".

The reason for this error message might be a problem with the command setting.It is important to make sure that the command SetBaselineInfoFromBrowser is properly set in your test code, for example:

Set br = Browser(“Google”)

br.Navigate(“ http://www.google.com” )

eyes.ApiKey = “YOUR_API_KEY”

Call eyes.SetBatch(“10”, “MyBatch”)

call eyes.SetBaselineInfoFromBrowser(br, 1200, 600)

eyes.MatchLevel= eyes.MatchLayout

call eyes.Open(“google”, “GoogleIE11”)

’browser.Refresh

call eyes.CheckObject(br, “Google”)

eyes.Close()

Applitools is a visual validation tool that compares images from any source against a pre-existing baseline.

As such tool Applitools can support comparing web pages against mockup images.

To achieve such a comparison first the mockup image should be set as the baseline, second, the actual web site should be compared against it.

Here is a quick and easy way to conduct such a comparison.

Prepare the mockup image as part of an HTML File

First, create an html page with the image as follow:

<!DOCTYPE html>

<html lang="en-US">

<head>

<meta charset="UTF-8">

<title>Mockup</title>

</head>

<style type="text/css">

body{

margin: 0;

padding: 0;

border: 0;

}

</style>

<body>

<script>

function addImage(path, width){

document.body.innerHTML = "<center><img src='"+path+"' width='"+width+"'></center>";

}

</script>

</body>

</html>

Conduct the Comparison using Selenium and Applitools

Next, Use selenium SDK to capture the mockup image and save it as baseline and then validate the web page against this baseline:

import com.applitools.eyes.BatchInfo;

import com.applitools.eyes.MatchLevel;

import com.applitools.eyes.RectangleSize;

import com.applitools.eyes.selenium.StitchMode;

import com.applitools.eyes.selenium.fluent.Target;

import org.openqa.selenium.By;

import org.openqa.selenium.JavascriptExecutor;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;

import java.rmi.UnexpectedException;

import java.util.ArrayList;

public class Mockup {

private static WebDriver driver;

private static CompareEyes eyes;

private static String appName = "My Application Name";

private static String testName = "My Test Name";

private static RectangleSize viewPortSize = new RectangleSize(800, 600);

private static String htmlFileLocation = "file:///Users/john/Desktop/mockup.html";

private static ArrayList<String[]> dataBase = new ArrayList<>();

public static void main(String[] args) {

try {

driver = new ChromeDriver();

initEyes();

eyes.setEnableComparison(true);

initDataBase();

eyes.open(driver, appName, testName, viewPortSize);

uploadMockups();

eyes.switchToComparisonMode(driver);

runTest();

eyes.close();

driver.quit();

} catch (UnexpectedException e) {

e.printStackTrace();

}finally {

driver.quit();

eyes.abortIfNotClosed();

}

}

public static void initEyes(){

/* Create CompareEyes instance */

eyes = new CompareEyes();

/* Set API key */

eyes.setApiKey(System.getenv("APPLITOOLS_API_KEY"));

/* Set batch */

eyes.setBatch(new BatchInfo("My Batch"));

/* Set match level to Layout */

eyes.setMatchLevel(MatchLevel.LAYOUT);

/* Enable full page screenshot with CSS stitching */

eyes.setForceFullPageScreenshot(true);

eyes.setStitchMode(StitchMode.CSS);

}

public static void initDataBase(){

/* Creating a database of URL<->Mockup pairs. */

dataBase.add(new String[]{"http://www.applitools.com", "/Users/john/Desktop/Applitools.png"});

dataBase.add(new String[]{"https://help.applitools.com", "/Users/john/Desktop/Support.png"});

}

public static void uploadMockups(){

String width;

String getWidthCommand = "function getWidth() { var scrollWidth = document.documentElement.scrollWidth; var bodyScrollWidth = document.body.scrollWidth; return Math.max(scrollWidth, bodyScrollWidth);}; return getWidth();";

String hideScrollbarCommand = "document.body.style.overflow = 'hidden';";

/* Taking screenshots of the mockups */

for (String entry[]:dataBase) {

driver.get(entry[0]);

eyes.setHideScrollbars(true);

((JavascriptExecutor) driver).executeScript(hideScrollbarCommand);

width = ((JavascriptExecutor) driver).executeScript(getWidthCommand).toString();

driver.get(htmlFileLocation);

((JavascriptExecutor) driver).executeScript("addImage('" + entry[1] + "','" + width + "');");

eyes.check(entry[0], Target.region(By.cssSelector("img")).timeout(0));

}

}

public static void runTest(){

/* Taking screenshots of the URLs */

for (String entry[]:dataBase) {

driver.get(entry[0]);

eyes.check(entry[0], Target.window());

}

}

}

The above example is using CompareEyes class (Download it from here ) and is able to validate multiple URLs/mockups in a single run.

To run this example:

1. Download the following files:

Test.java

CompareEyes.java (Download from here ).

mockup.html

2. Place the html file on your machine.

3. Edit the htmlFileLocation parameter in the Test.java file.

4. Edit the URL<->mockup pairs database in Test.java file with your data.

5. Edit the app name, test name, viewport size in Test.java. (optional)

5. Run the test.

This article will explain the screenshot adjustment that Applitools Eyes provides, using Ruby examples.

For your convenience, please find the following video guide:

In automated visual tests, it's crucial to make sure that your screenshots are reliableand sound.

With a dependency of the browser, sometimes making the screenshot resemble the reality might be a bit more challenging,especially when using functions that include dynamic behavior, such asfloating menusorinfinite scroll bar.

Most browsersdon't automatically get a full page screenshot: the browser grabs only what is visible in the viewport of the browser's window while the test is running.

In order to get a full page screenshot, we have to enable it with Applitools Eyes.

In order to do so, there are two available methods:

1. Force full page screenshot:

2. Use CSS transition:

If you don't enable CSS transition, the floating bar will appear several times, due to the fact that without using CSS transition, Applitools will use Selenium to scroll the page, capture screenshots, and patch them together, which will trigger a visual anomaly that will fail the test, as you can see below:

With the two options enabled, the menu bar only appears once, and no scrollbar events will be triggered. In addition, the infinitescrollbar problem will not occur, since the only thing that appears on the screen is the content of the viewport.

In case you still see a scrollbar, you can hide it using the following method:

The following lines contain the syntax for the different programming languages:

Java:

eyes.setForceFullPageScreenshot(true);

Python:

eyes.force_full_page_screenshot = True

Ruby:

eyes.force_fullpage_screenshot = true

.Net:

eyes.ForceFullPageScreenshot = true

Javascript:

eyes.setForceFullPageScreenshot(true);

A baseline is defined by 5 parameters as can be seen in this article:

1) Application name

2) Test name

3) OS

4) Viewport Size

5) Browser

The first two are the test parameters, and parameters 3-5 are the environment parameters.

When running a test, Applitools Eyes checks if there is an existing baseline. If there is, then it compares the test to it, and if not it createsone.

A baseline is always saved within its environment (combination of OS, Viewport Size andBrowser).

There is an option to compare testsacross environments - for that you set aBaselineEnvironment Name.

Setting a Baseline Environment name allows you to compare and validate your application across different OS, Browsers and Viewport sizes.

A Baseline Environment name can only be associated withone environment, you can see all the environments that you ran tests against and their associated names under the environment view.

In order to run test across environments, we will need to define a Baseline Name, that will link between the different environments, using the following syntax:

Java

eyes.setBaselineEnvName("BaselineName");

Python

eyes.baseline_name="BaselineName"

Ruby

eyes.baseline_name='BaselineName'

JavaScript - Selenium Web Driver

eyes.setBaselineEnvName("BaselineName");

C#

eyes.BaselineEnvName = "BaselineName";

Note: Locate the above method before the eyes.open(); method.

When running a test for the first time with a givenBaseline Name,the Baseline Name "points" tothe environment of that test, and any future tests that will run with thisBaseline Namewill be compared to the corresponding test within the environment to which theBaseline Nameis "pointing" to.

Accept / Reject

Accepting / rejecting and saving in cross environment test changes the Status (passed / failed) of the current test results but it is NOT changing the baseline of the environment to which the Baseline Name is pointing to or the baseline of the checkpoint (if such exist).

Same as in version control forcode, but slightly different,Applitools supports a branching mechanism.

The main difference is that branches in Applitools contain visual baselines instead of code.

The branching mechanism in Applitools allows the user to create, maintain and merge branches easily.

This article covers the following topics:

Creating anew branch via the Test Manager

Running tests in a specific branch

Merging branches via the Test Manager

Resolving conflicts

Creating a new branch via the Test Manager

Go toBranch baselinespage by selecting it from the drop-down menu next to the Applitools logo.

Choose a branch which you want tocopy baselines from.

You can do so by selecting the branch from the drop-down menu on the right side of the screen.

Select the baselines which you want to copy to the new branch by checking the checkbox next to their name and click the "Copy baselines to branch..."button.

In the "Copy baselines to branch" dialog, type a name for a new branch or select an existing one from the list and click the "Copy to branch" button.

Running tests in a specific branch

To run a test in a specific branch add the following line before eyes.open():

Java:

eyes.setBranchName(<branch name>);

Ruby:

eyes.branch_name = <branch name>

Python:

eyes.branch_name = <branch name>

DotNet:

eyes.BranchName = <branch name>;

JS:

eyes.setBranchName(<branch name>)

A "parent branch" can be configured in order to compare against an existing baseline in a different branch, in case a baseline cannot be found in the requested branch.

If not set, the parent branch will be "default". Explicitly setting an alternative parent branch can be done by adding the following line before eyes.open():

Java:

eyes.setParentBranchName(<branch name>);

Ruby:

eyes.parent_branch_name = <branch name>

Python:

eyes.parent_branch_name = <branch name>

DotNet:

eyes.ParentBranchName = <branch name>

JS:

eyes.setParentBranchName(<branch name>);

The Behavior of a Comparison When Running a Test

If the requested branch does not exist, it will be created.

If thebaseline exists in this branch, the testwill be compared against it.

If thebaseline does not exist in this branch, it will look up for the baseline in theparent branch and compare against it (note that in order to create a baseline in the requested branch the test result must be resolved - either accepted or rejected - and saved).

If thebaseline does not exist in this branch or in the parent branch,a new baseline will be created in this branch.

Note:If you set a parent branch which does notexist, an exception will be thrown.

Merging branches via the Test Manager

Go toCompare & mergebranchespage by selecting it from the drop-down menu next to the Applitools logo.

Select the source and target branches and click the "Compare" button.

If changes exist and there are no conflicts (discussed later in this article), choose the baselines you would like to merge (by checking the relevant checkboxes) and click the "Merge" button to merge the baselines from the source branch into the target branch.

A confirmation dialog will appear, click the "Yes" button to confirm merging.

Resolving conflicts

There are two reasons for conflicts:

A conflict occurs when a mismatching test in the target branch was saved after running that test in the source branch.

A conflict can also occur when successfully merging a sourcebranch"B1" into the target branch, and then attempting to merge another source branch "B2" into the same targetbranch(since the target branch waschanged due to the first merge).

All conflicts must be resolved before merging the source branch to the target branch.

To resolve a conflict a decision needs to be made - whether the source image should be taken as the baseline or the target image.

You can do so by clicking on one of the followingbuttons on the right side of the baselinerow:

Take source baseline

Take target baseline

In addition to the screenshot, a baseline can also have an annotation (i.e., regions and remarks) attached to it.

In case the source image was decided as the baseline, another decision is required before merging - keeping the source annotations, replacing them with the annotations from the target or to keeping both annotations.

To do so, click on the "Compare and edit source baseline" button, located on the right side of the baseline row.

This will lead to theCompare baselinepage.

To add the target annotations to source baseline click on the "Add target annotations to source" button.

To replace the target annotations with the source annotations, click the arrow next to"Add target annotations to source" button to open the menu and click on "Replace source annotations with target's".

Click the "Save" button to save changes.

This feature was rated high on the wish list by many of our customers and we are happy to announce that its now available. We would like to use this opportunity to thank you for helping us improve our service by providing us with excellent feedback. Keep it coming!

The check region feature allows you to handle cases where the position of some of the UI elements in your page might change based on some dynamic content that appears around it.

For example, if you are showing the user name of the logged in user on the top bar of your website, and the position of all the UI elements that appears to the right of the user name is relative and changes based on the length of the user name, you can run 'check window' to validate the entire window, mark this dynamic region (the user name and the moving elements on the right) as ignore region and then use 'check region' to validate the UI elements of interest to the right of the user name, that might appear in a different position the next time you run the test.

Check region method works in a similar way to check window, except for the fact that it takes a screenshot of the HTML element specified as its input (by means of a WebDriver "By" selector) instead of a screenshot of the entire web page. The check region command will take a snapshot of the specific object, regardless of where it appears in the page, and will perform smart visual comparison of that region with the baseline..In case this is a new region that does not exist in the baseline, this region will be added as a new region to the baseline.

Syntax:

Java:

eyes.checkRegion(selector, "Optional validation tag or description" );

.Net:

eyes.CheckRegion(selector, "Optional validation tag or description" );

Python:

eyes.check_region(by=By.CSS_SELECTOR,".someclass", "Optional validation tag or description")

Ruby:

eyes.check_region_by_selector(By.CSS_SELECTOR, ".someclass", "Optional validation tag or description")

Protractor:

eyes.checkRegionBy(By.CSS_SELECTOR, “Region by By”);

eyes.checkRegionByElement(element(by.className(‘form-inline’)), “Region by Element”);

eyes.checkRegion({left: 20, top: 50, width: 450, height: 120}, “Region by coordinates”);

where "selector" is the “By” selector (of Selenium Webdriver) that identifies DOM element to check.

You can see how Applitools eye different features handle with a dynamic content here.

This article embeds the full documentation regarding the Applitools Simple Test Runner, which is an XML-basedstand-alone tool for Visual Testing - No Coding skills required.

See full documentation here.