Sauce Labs FAQs

In keeping with industry standards, Sauce Labs will be fully deprecating support for TLS v1.0 and TLS v1.1 protocol versions as of February 1st, 2020. This will affect all https traffic, including:

Test runners

API clients

Web browsers that access the Sauce Labs UI

As of November 2019, the great majority of https traffic to the Sauce Labs service uses TLS 1.2 (99.9% or higher). Most customers will not need to take any advance action; where possible, customers known by Sauce Labs to be using older versions of TLS have already been notified directly. Nonetheless, customers should verify the following in advance of February 1st:

Test Runners

Any test runner that uses the Sauce Labs cloud (the app.saucelabs.com or app.testobject.com application or REST interfaces at saucelabs.com/rest/* path) must use versions of Java that support TLS 1.2. This may affect Bamboo, Jenkins, Jira Server, Confluence Server, or any other Java-based systems that connect to Sauce Labs. Customers will need to check the underlying version of Java used in these programs and, if necessary, upgrade them to versions that support TLS 1.2.

Unaffected Java versions:

JDK 8 and later

JDK 7 versions 1.7.0_131-b31 and higher

Affected Java versions:

JDK 7 versions earlier than 1.7.0_131-b31

JDK 6 and earlier

Similar steps should be taken for any programs making requests to the above URIs that rely on OpenSSL or Pythons ssl module.

API Clients

Any API client (such as curl) that queries a Sauce Labs API endpoint must support TLS 1.2. Check the libraries they use to connect to our rest API at api.saucelabs.com or saucelabs.com/rest/* and be sure these libraries support TLS 1.2.

Error Messages

Connection requests that are not in compliance with the TLS 1.2 protocol version or higher will result in an HTTP response code of 406 or an SSL handshake protocol version failure.

Web Browsers

Browsers that load the Sauce Labs UI ( www.saucelabs.com and www.testobject.com) must use TLS 1.2 or higher. A full list of such browsers: https://en.wikipedia.org/wiki/Transport_Layer_Security#Web_browsers

Loading this page in a browser will report if the browser supports TLS 1.2: https://www.ssllabs.com/ssltest/viewMyClient.html

Some browsers also make the TLS version visible in their developer tools or after clicking the padlock icon in the address bar.

Testing internal systems with Sauce Labs requires that our service is able to establish a connection. Some companies accomplish this by whitelisting our external IP addresses. However, we recommend instead that you use Sauce Connect, for these reasons:

Configurability

Sauce Connect provides many tools designed to improve the testing experience, from selective traffic routing to bandwidth restriction.

Each Sauce Connect instance can be assigned different network proxies, run on different hosts, allocated to different users and allowed access to different systems under test. This allows infrastructure teams great control and flexibility over system access.

Whitelisting provides no controls over which systems can be accessed by which users; Anyone with the correct URL can access any service.

Portability

Sauce Connect can be used in any part of your corporate infrastructure, with relatively minimal setup. It allows customers to test systems in different environments, different locations and different physical hosts. With a flexible Sauce Connect setup, a developer can run tests on their local machine, their staging service, their build server, and anywhere else required.

Whitelisting, in comparison, requires different routes and URLs for each environment under test. This places a maintenance burden on the networking team and leads to configuration problems for testers.

Reliability & Ease of Maintenance

Once Sauce Connect is running, the only maintenance is occasional updates of the software itself. Sauce Connect can be set up for multiple corporate users, and multiple systems under test, at the same time.

With Sauce Connect, our support team can provide greater assistance debugging connectivity and reliability problems, thanks to improved logging and network tracking.

Sauce Connect's underlying network tools improve reliability over bad network connections.

Security

Sauce Labs' service operates from a wide range of IP addresses. Any of these addresses may be allocated to any test environment or Sauce Labs customer at any time.

When whitelisting, this means any Sauce Labs customer could conceivably send requests to your system under test, and your network infrastructure would allow it. Whitelisting provides no tools for time-based access controls or easy disconnection.

By terminating the Sauce Connect client, the connection between our service and your infrastructure is severed. Sauce Connect thus allows fine-grained control of access.

Sauce Labs provides a way for you to choose the language for a specific real device project.This setting provides two different experiences on iOS and Android.

AndroidOn Android the language set in the Device Settings menu will change the language on the entire device.This means that not only the application will change language, but also the Operating System.iOSOn iOS, the application locale is changed, and not the system locale.This means that the OS's Language & Region settings will still be set to their default (English - UK)

Sauce Labs' service is intended to be set up as closely to a stock customer environment as possible. We make limited changes to browser & registry settings, Apple defaults and other configurable options. We don't generally install extra video codecs, browser plugins, .Net frameworks, or similar products.

There are several customization options available to customers requiring changes to the standard environments. Please note that Sauce Labs support does not extend to customization options, and having these in your tests may limit the support we can provide with other issues as well.

If you need to make changes to Browser settings, some browsers allow you to do this directly with Selenium, as described in this article from Selenium HQ. Our KB articles Customizing Firefox with FirefoxProfile and Customizing Chrome with ChromeOptions also has someexamples of ways to customize browser settings and functions.

For making changes to the environment, you may be able to use a prerun executable, as described Setting Up Pre-Run Executables.

If your changes are applicable to multiple customers and can't be achieved through prerun executables or Selenium profiles, consider raising a feature request.

Sauce Labs has reserved the following IP blocks for our services. These services include Automated and Manual Testings on Virtual Desktops, Simulators, Emulators and Real Devices. You can whitelist these IP addresses to allow traffic in a restricted network setting. These IP addresses can also be deliberately blocked to stop traffic from us.

Please note that not all of the IP addresses are actively in use, and some will not response to ICMP requests (for example, ping).

United States Data Center

162.222.72.0/21(this is equivalent to the range 162.222.72.0 162.222.79.255)

66.85.48.0/21(this is equivalent to the range 66.85.48.0 - 66.85.55.255)

United States Headless Data Center - US-EAST-1

Endpoints for outbound traffic from your organization. These are Sauce Labs servers our REST API, Sauce Connect services, and our Web UI.

35.227.75.136

35.237.73.165

34.73.236.203

34.73.75.126

Inbound traffic from Sauce Labs test VMs to your organization:

34.74.73.147

34.73.78.33

34.74.9.21

34.73.160.152

European Data Center

185.94.24.0/22(this is equivalent to the range 185.94.24.0 -185.94.27.255)

Sauce Labs posts information about any service interruptions on our Status Page located at https://status.saucelabs.com. As a benefit to our users, we also offer the ability to access this data via an API.

Sauce Labs US West Data Center

https://kd2w7ghdk56w.statuspage.io/api/v2/<ENDPOINT>

You can browse a full range of our endpoints at https://status.us-west-1.saucelabs.com/api

Here are a few examples of some commonly used endpoints:

Endpoint

Returns

https://kd2w7ghdk56w.statuspage.io/api/v2/summary.json

Summary of the status page

https://kd2w7ghdk56w.statuspage.io/api/v2/incidents/unresolved.json

List of current incidents

https://kd2w7ghdk56w.statuspage.io/api/v2/scheduled-maintenances/upcoming.json

List of upcoming maintenances

Sauce Labs European Data Center

https://78dz91ktdsmp.statuspage.io/api/v2/<ENDPOINT>

You can browse a full range of our endpoints at https://status.eu-central-1.saucelabs.com/api

Here are a few examples of some commonly used endpoints:

Endpoint

Returns

https://78dz91ktdsmp.statuspage.io/api/v2/summary.json

Summary of the status page

https://78dz91ktdsmp.statuspage.io/api/v2/incidents/unresolved.json

List of current incidents

https://78dz91ktdsmp.statuspage.io/api/v2/scheduled-maintenances/upcoming.json

List of upcoming maintenances

Sauce Labs US East Data Center

https://gr36cm5yj7vw.statuspage.io/api/v2/<ENDPOINT>

You can browse a full range of our endpoints at https://status.us-east-1.saucelabs.com/api

Here are a few examples of some commonly used endpoints:

Endpoint

Returns

https://gr36cm5yj7vw.statuspage.io/api/v2/summary.json

Summary of the status page

https://gr36cm5yj7vw.statuspage.io/api/v2/incidents/unresolved.json

List of current incidents

https://gr36cm5yj7vw.statuspage.io/api/v2/scheduled-maintenances/upcoming.json

List of upcoming maintenances

heres an integration between HockeyApp and TestObject. You can upload your Apps on HockeyApp and get them to TestObject through an API Token. For the integration to work, the API Token needs to have full access to the App. Usually this is sufficient, but sometimes it's not.

If the user who is exporting the token is a tester, the integrationwon'twork. Neither will it work if the user sets themselves as a developer. The developer who uploaded the build (the App) needs to also export the API Token, and needs to share it with the second user with developer rights.

These are the exact steps to share the App with developer rights, to be followed by the developer who uploaded the build:

Go to https://rink.hockeyapp.net/manage/dashboard.

Click on the App you want to share.

Click on Invite user.

Enter the email.

Set the role as Developer.

Save.

Now the newly invited user can log in and go their Settings page and create the API Token for the application.

User must login with the Developer account created above.

The user will go to Account Settingsfrom https://rink.hockeyapp.net/manage/dashboard.

Click API Tokens tab

Follow the steps to generate an API token for your project

Now the user can copy-paste the newly created API Token into the Real Device Cloud form and click "Validate".

esting internal systems with Sauce Labs requires that our service is able to establish a connection. Some companies accomplish this by whitelisting our external IP addresses. However, we recommend instead that you use Sauce Connect, for these reasons:

Configurability

Sauce Connect provides many tools designed to improve the testing experience, from selective traffic routing to bandwidth restriction.

Each Sauce Connect instance can be assigned different network proxies, run on different hosts, allocated to different users and allowed access to different systems under test. This allows infrastructure teams great control and flexibility over system access.

Whitelisting provides no controls over which systems can be accessed by which users; Anyone with the correct URL can access any service.

Portability

Sauce Connect can be used in any part of your corporate infrastructure, with relatively minimal setup. It allows customers to test systems in different environments, different locations and different physical hosts. With a flexible Sauce Connect setup, a developer can run tests on their local machine, their staging service, their build server, and anywhere else required.

Whitelisting, in comparison, requires different routes and URLs for each environment under test. This places a maintenance burden on the networking team and leads to configuration problems for testers.

Reliability & Ease of Maintenance

Once Sauce Connect is running, the only maintenance is occasional updates of the software itself. Sauce Connect can be set up for multiple corporate users, and multiple systems under test, at the same time.

With Sauce Connect, our support team can provide greater assistance debugging connectivity and reliability problems, thanks to improved logging and network tracking.

Sauce Connect's underlying network tools improve reliability over bad network connections.

Security

Sauce Labs' service operates from a wide range of IP addresses. Any of these addresses may be allocated to any test environment or Sauce Labs customer at any time.

When whitelisting, this means any Sauce Labs customer could conceivably send requests to your system under test, and your network infrastructure would allow it. Whitelisting provides no tools for time-based access controls or easy disconnection.

By terminating the Sauce Connect client, the connection between our service and your infrastructure is severed. Sauce Connect thus allows fine-grained control of access.

Running a Chrome test with W3C enabled on our virtual device loud might lead to anUnsupportedCommandException error when using the Actions class.This is due to the Chromium team not having implemented the complete set of Actions methods on Chrome at the time of writing.

This error might also be presented as an unknown commandwhen looking at the chromedriver logs for your test.For the latest status on chromedriver's status on WebDriver implementation, see here.

Once the methods have been implemented and a new chromedriver version is released, you will be able to run Chrome tests with the Action class on Sauce.

The only known workaround is to not use the W3C capabilities and revert to older JSON Wire Protocol capabilities.NOTE: This is not an issue specific to Sauce Labs, and attempting W3C mode with Chrome will result in the same error on any Chrome instance.

Starting with Android 7.0, Google set stricter rules on how applications and emulators trust user/admin added certificates.Before, adding your custom SSL certificate to the emulator (which in Sauce's case we did by adding the CyberVillains certificate. which is used by Sauce Connect) would be enough to ensure a successful connection.

This is no longer the case. Your application itself needs to have this certificate for the connection to be trusted.This is discussed a bit more in detail in this post from the Android Developer's blog. .

Because the Sauce Labs CyberVillains certificate (the certificate used when re-encrypting traffic sent via Sauce Connect) is not trusted as a public CA, your app will not trust this certificate by default.

What does this mean for you? You have two options depending on your setup, and the original certificate of your server.

Option 1

If the server that your app connects to has a valid CA-trusted certificate you can simply add the flag `-B all` to your Sauce Connect tunnel.This will disable SSL bumping and won't re-encrypt traffic.

NOTE: If you do need SSL bumping for other types of tests (for example, Android web tests or desktop browser tests) you should use a separate tunnel for those tests, and avoid using `-B all` on them to have re-encryption work in that situation.

Option 2

If the server that your app connects to does not have a valid CA-trusted certificate, then you will need to add your server's certificate onto your application.The post in the Developer's Blog linked above explains how to do this.The process involves adding the certificate file to your app by creating a network configuration file, and telling your app to trust it.As an example, this is what a network_security_config.xmlfile would look like:

<?xml version="1.0" encoding="utf-8"?><network-security-config> <debug-overrides> <trust-anchors> <certificates src="@raw/selfsigned_ca" /> </trust-anchors> </debug-overrides></network-security-config>

....where @raw/selfsigned_ca is the path to res/raw/selfsigned_ca,which contains my site's certificate.

You then add this file to your AndroidManifest.xml with the android:networkSecurityConfig attribute, as explained in this page on Network Security Configuration in the Android Developers documentation.

Firefox and Chrome's dev and beta releases are automatically updated when new releases are made by each project. However, as is the nature of betas, sometimes the beta breaks compatibility with Selenium, making it impossible for Sauce Labs to use. Because Firefox, Chrome, and Selenium are all Open Source projects, not under the control of Sauce Labs, we're unable to set project direction and ensure this functionality never breaks.

End users rarely install beta or dev. Those who do tend to understand that functionality may be missing or broken in these versions. In addition, the beta and dev releases are made available mostly to assist with testing the Selenium functionality of said releases.

Sauce Labs makes very minimal changes to the browsers we offer. We also take care to offer exactly what we say we do. Making custom patches to Firefox, Chrome, or Selenium would make them no longer reflect reality. If Sauce Labs can't spin up beta or dev with Selenium, it's likely that this functionality doesn't work locally on all platforms. We also don't want to offer a beta or dev release that isn't the same as the project's current beta or dev release, because that would just lead to confusion.

We recommend customers testing commercial web applications for end user consumption stick to released versions of browsers. These have passed testing by their authors, the Selenium project and Sauce Labs, and are known to have, at the very least, passed the Selenium and Sauce Labs test suites.

Firefox Release Process

Sauce Labs pulls from these three release channels.

The official "Release" version, a stable and commonly used version that most consumers will use. Mozilla has more information here: https://www.mozilla.org/en-US/firefox/channel/desktop/

The Beta version:

From Mozilla's site: "Beta is an unstable testing and development platform. By default, Beta sends data to Mozilla and sometimes our partners to help us handle problems and try ideas."

The Dev version:

From Mozilla's site: "Developer Edition is an unstable testing and development platform. By default, Developer Edition sends data to Mozilla and sometimes our partners to help us handle problems and try ideas."

Firefox Release

Firefox Beta

Firefox Dev Edition

Chrome Release Process

The Chrome Project follows a similar release process. t uses a Stable, Dev, and Beta version described on their official project page https://www.chromium.org/getting-involved/dev-channel :

Stable channel: This channel has gotten the full testing and blessing of the Chrome test team, and is the best bet to avoid crashes and other issues. It's updated roughly every two-three weeks for minor releases, and every 6 weeks for major releases.

Beta channel: If you are interested in seeing what's next, with minimal risk, Beta channel is the place to be. It's updated every week roughly, with major updates coming every six weeks, more than a month before the Stable channel will get them.

Dev channel: Want to see what's happening quickly, then you want the Dev channel. The Dev channel gets updated once or twice weekly, and it shows what we're working on right now. There's no lag between major versions, whatever code we've got, you will get. While this build does get tested, it is still subject to bugs, as we want people to see what's new as soon as possible.

There are instances when our Sauce OnDemand Plugin for Jenkins does not behave as expected.In order for our Support team to get to the bottom of this, it's useful to not only provide the Jenkins build console output, but also the debug log that can be created by the plugin itself. This is not turned on by default, so you'll need to set it up, and then re-run the problem build to generate the debug log.

Create a new Log Recorder following the instructions in the official Jenkins documentation. This screenshot shows an example of the Logger classes needed for the OnDemand plugin.

Once Log Recorder is created, re-run your build where the issue occured.

Go back to the Logger page, and copy and send the new logs that were generated.NOTE:If using a Jenkins Slave, your log page may minimize the logs produced inside the slave, which need to be expanded before being sent to Sauce Labs Support.

On April 24, 2018 SauceLabs OPS deployed a new SSL certificate for its OnDemand service:

Djavax.net

The reasons for the new certificate were two-fold:

The original certificate was about to expire.

The original certificate was issued by GeoTrust CA, which is no longer widely trusted and it was imperative to move away from this CA.

All browsers and up-to-date non-browser clients recognized this new certificate, and its deployment was transparent and without negative impact. However, some older non-browser clients (e.g. JDK) had connectivity issues with Sauce because the new root certificate was not in their trust store.

When running tests using an affected client, the attempt to connect to ondemand.saucelabs.com:443 will throw this exception:

javax.net.ssl.SSLHandshakeException:

sun.security.validator.ValidatorException: PKIX path building failed:

sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

To correct the problem, the trust store used by the client needs to be updated so it includes theDigicert Global Root G2 root certificate.

There are several options for customers running into this problem.All three mean that the client's trust store needs to be updated to the latest root certificates.

Option 1:Upgrade the JDK version on affected machinesUpdate the JDK version to the latest minor version (e.g. JDK 8u172 for Java 8). This will update the trust stores automatically and include desired certificates. Most customers will find this to be simplest option.

Option 2:Add the DigiCert CA cert manually to the Java Trust StoreInstall the Digicert Global Root G2 cert (Root certificate type) to ensure a successful handshake with Sauce's ondemand service. The Digicert Root Certificate can be found here and installation notes can be found here.

Option 3:Update the JDK version on one machine and then copy the JDK trust store certs from that machine onto the other affected machines.

This method is explained in detail here.

If none of the options above help, it is likely that you are using a custom trust store. You can find out what trust store is used byrunning a test with -.debug="ssl,handshake",which will print out which trust stores are in use. Once that is known, adding the DigiCert cert to that trust store will fix the issue. Alternatively, you could move from the custom trust store in favor of the default one.

Summary

When using Safari 11 or 12 for a manual testing and you click on an element, you might encounter a pop-up that reads "This Safari window is remotely controlled by an automated test."

This is caused by the new SafariDriver on Safari 11 and 12, and was implemented as a security precaution. This can also be reproduced locally, and is not a Sauce-specific issue.

Workaround

The current workaround for this is to clickStop session,and you can continue your manual test as normal.

There may be situations where you wantto connect to the Sauce Labs browser cloud through a proxy. For tests written in Java, you can use the proxyhost settings to specify the URL and port for both http and https connections, as shown in this example:

System.getProperties().put("http.proxyHost",proxyHost);System.getProperties().put("http.proxyPort",proxyPort);System.getProperties().put("https.proxyHost",proxyHost);System.getProperties().put("https.proxyPort",proxyPort);

Note that this will route all traffic coming from your test framework through the proxy. This may not be a desired condition for your testing environment if, for example, you are calling internal services from your test script which should not be routed through the proxy. In these cases, you can exclude such internal services by specifying their domain name as follows:

System.getProperties().put("http.nonProxyHosts",proxyHost);

System.getProperties().put("https.nonProxyHosts",proxyHost);

If excluding specific hosts is not enough, then and additional solution is to proxy exclusively the WebDriver traffic. This can be achieved by overriding the Java HTTP client library with the SeleniumHttpCommandExecutor. The StackOverflow article Using Selenium RemoteWebdriver behind corporate proxy provides an example of how to create a Java class, and then use it in your test script, to set this up. These examples are taken from that article, with values for Sauce Labs added.

Java ClassMyHttpClientFactory

import java.net.URL;

import org.apache.http.impl.client.HttpClientBuilder; import org.openqa.selenium.remote.internal.ApacheHttpClient;

public class MyHttpClientFactory implements org.openqa.selenium.remote.http.HttpClient.Factory {

final HttpClientBuilder builder;

public MyHttpClientFactory(HttpClientBuilder builder) {

this.builder = builder;

}

@Override

public org.openqa.selenium.remote.http.HttpClient createClient(URL url) {

return new ApacheHttpClient(builder.build(), url);

}

}

connectViaProxy Method

Add this method to your Java test script to route only WebDriver traffic through the proxy.

public RemoteWebDriver connectViaProxy(DesiredCapabilities caps) {

String proxyHost = "?";

int proxyPort = 8080;

String proxyUserDomain = "?";

String proxyUser = "?";

String proxyPassword = "?";

String host="ondemand.saucelabs.com";

URL url;

try {

url = new URL("https://" + host + "/wd/hub");

} catch (MalformedURLException e) {

throw new RuntimeException(e.getMessage(), e);

}

HttpClientBuilder builder = HttpClientBuilder.create();

HttpHost proxy = new HttpHost(proxyHost, proxyPort);

CredentialsProvider credsProvider = new BasicCredentialsProvider();

credsProvider.setCredentials(new AuthScope(proxyHost, proxyPort), new NTCredentials(proxyUser, proxyPassword, getWorkstation(), proxyUserDomain));

if (url.getUserInfo() != null && !url.getUserInfo().isEmpty()) {

credsProvider.setCredentials(new AuthScope(url.getHost(), (url.getPort() > 0 ? url.getPort() : url.getDefaultPort())), new UsernamePasswordCredentials(url.getUserInfo()));

}

builder.setProxy(proxy);

builder.setDefaultCredentialsProvider(credsProvider);

Factory factory = new MyHttpClientFactory(builder);

HttpCommandExecutor executor = new HttpCommandExecutor(new HashMap<String, CommandInfo>(), url, factory);

return new RemoteWebDriver(executor, caps);

}

private String getWorkstation() {

Map<String, String> env = System.getenv();

if (env.containsKey("COMPUTERNAME")) {

// Windows

return env.get("COMPUTERNAME");

} else if (env.containsKey("HOSTNAME")) {

// Unix/Linux/MacOS

return env.get("HOSTNAME");

} else {

// From DNS

try

{

return InetAddress.getLocalHost().getHostName();

}

catch (UnknownHostException ex)

{

return "Unknown";

}

}

}

When using Protractor behind a proxy, you may see an error like:

Error updating Sauce pass/fail status: 'Could not send request: connect ETIMEDOUT 162.222.75.243:443'

This can be caused by Protractor not having thecorrectproxy configuration forcommunicatingwith the Sauce Labs REST API or with our OnDemand endpoint. To resolve it, make sure thatexports.configsection of your Protractor configuration file has these entries:

exports.config = {

sauceUser: YOUR_SAUCE_USERNAME,

sauceKey: YOUR_SAUCE_ACCESS_KEY,

webDriverProxy: 'http://your_proxy_address:port', // Proxy for Commands (e.g. ondemand.saucelabs.com)

sauceProxy: 'http://your_proxy_address:port', // Proxy for the REST API

// The rest of your config is omitted

}

ThewebDriverProxyentry ensures that your test commands are sent from WebDriver, through the proxy, to our Selenium address. ThesauceProxyentry ensures that the success or failure status of your tests is sent from Protractor, through the proxy, to the Sauce Labs REST API.

Starting with Chrome 62 the new default is to disable Flash for websites.

Sauce Labs attempted to change this default Chrome setting on its virtual machines; however, there seems to be no programmatic way to enable Flash across all OS images that support Chrome. For this reason, Flash is not enabled on Chrome version 62 and higher on any OS.

If you are running tests that require Flash, you will likely see a pop-up requesting you to allow Flash. If this causes an issue during your test, you will need to add an additional step in your script to select 'Allow' when the pop-up appears. T his article from StackOverflow includes a number of examples of how to accomplish this.

Here is Google Chrome's Flash Roadmap: https://www.chromium.org/flash-roadmap

f you experience issues finding UI elements for an Android app, it might be that the developer has used an Android security feature in their app to block taking screenshots, so the UI elements are protected and not accessible.

If this is the case, you'll see this in the device log when trying to take a screenshot during a test:

W SurfaceFlinger : FB is protected: PERMISSION_DENIED

Prior to Android 9, apps using this security feature did not stop the app from being displayed in the video of live tests, or the video recording of automated tests. In Android 9 however, protected views in an app will display as a blank, black screen invideo of live tests, or the video recording of automated tests. This is not a bug - this is the Android security feature working as designed.

Safari has an option that lets you use the Tab key to highlight items on a web page:

Test Configuration Options

This article covers how to use a pre-run executable to configure this option in Safari so you can use the Tab key in this way in your tests.

The Script

Save this script as a .sh file, and either upload it to Sauce Storage, or to a local publicly accessible over the internet, such as a GitHub Gist.

#!/bin/bash

defaults write com.apple.Safari WebKitTabToLinksPreferenceKey -bool truedefaults write com.apple.Safari com.apple.Safari.ContentPageGroupIdentifier.WebKit2TabsToLinks -bool true

Setting theprerunCapability

You will need to specify the location of your script within the prerun capability of your tests as described in thetopic. This Python example refers to a Sauce Storage location:

desired_capabilities['prerun'] = {

'executable':'sauce-storage:safaritabkey.sh',

'background':'false'

}

Why do I need to download my job data?

Sauce Labs does not retain customer job assets forever. We keep your data for 30 days, but eventually we remove your assets. Once these assets are removed, they are gone forever and Sauce Labs can not recover them.

What data do you remove?

Job videos

Job screenshots

Job logs

Job commands

Can I download this data to keep it myself?

Yes! Our REST API gives you some tools to download job data, giving you the flexibility to retrieve and store your data however you need.While you can do this on a job-by-job basis using the command line, you'll save a lot of time by using one of our REST API client libraries and your favorite programming language.

How do I download all my jobs?

Depending on how many jobs you've run, there are two ways to fetch your job data. Sauce Labs may occasionally limit some API methods to only return a limited number of jobs in order to improve system performance. If there are jobs within the last 30 days which you can't get with Method 1, try Method 2. If that doesn't work, lodge a ticket with Support.

Method 1

Use the GET FULL JOBS method to grab all the relevant information about your jobs, including their Job IDs

Method 2

Use the GET JOBS TO AND FROM TIME method to grab all the job IDs

Fetching assets

Now you have the job IDs, you can fetch the assets like the logs, videos and screenshots.

For each job call the GET JOB ASSET NAMES method to grab all of the asset filenames

For each job asset name, call the GET JOB ASSET FILES method to download the relevant assets

UPDATE June 2018

You can avoid the bug described in this article by using Safari 11.1 on MacOS 10.13. Because of this, specifying Safari '11.0' AND 'MacOS 10.13' as desired capabilities is no longer an available combination. Specifying Safari '11', '11.1', or not specifying a Safari version along with the MacOS 10.13 platform, will now default to Safari '11.1'.

--------------------------------------------

When testing with Safari 11 and Selenium 3.x, you may find that click events on option elements (such as dropdown menus) are not handled correctly, making it appear as if the click does not occur or register correctly.

This is a WebKit bug, not a Sauce Labs bug. You can find details at https://bugs.webkit.org/show_bug.cgi?id=174710. This bug has been fixed, but Apple have yet to fix this in SafariDriver. This has been reported to Apple under bug report number33459305.

In June 2018, Sauce Labs will change the list of our service components you can receive status notifications about, so you can receive more focused information about the specific services that are of interest to you. We'll notify you when the list is updated, and at that time you should also update your notification subscriptions to make sure you're receiving the notifications you want. If you don't update your subscription, you will only receive notifications about the status of our Automated VM Testing services.

New Service Components

When you update your notification subscriptions on the Sauce Labs Status page, you will be asked to select components based on this list.

Automated VM Testing

Automated PC Testing

Automated Mac Testing

Automated IOS Simulator Testing

Automated Android Emulator Testing

Automated RDC Testing

Automated RDC Testing, US Data Center

Automated RDC Testing, EU Data Center

Manual Testing

Manual VM Testing

Manual RDC Testing

Sauce Connect Proxy

Sauce Connect VM

Sauce Connect RDC

REST API

REST API VMs

REST API RDC

Web Interface

Sauce UI

Real Device Cloud UI

Analytics

saucelabs.com

Documentation Wiki

Updating Your Service Notification Subscriptions

If you are already subscribed to status notifications, it's easy to update your options by logging into your account on the Sauce Labs Status page. If you don't already have notification subscriptions set up, follow the first two steps for updating, and then provide the contact information to use in your subscription.

Go to https://status.saucelabs.com.

Click Subscribe to Updates in the upper-right corner of the page.

Select the type of notification are using, such as email or SMS, by clicking on the appropriate icon.

Enter the contact information you used for that type of notification, such as your email address or phone number, in the text field.

Click the Subscribe button.The notification system will recognize the contact information you entered, and load the form with the notification subscription options.

Click on the name of the components you want to receive notifications about.The name of the component will turn grey when you select it.

Click Update Preferences.

So, you've got your app server running on localhost port 8888, and you're running Sauce Connect so that our browsers can access it. This works just fine in Firefox and Chrome, but for some reason, Safari and IE utterly fail to load the site. Why the difference?

A little background on Sauce Connect: it runs a tunnel endpoint in our cloud, and one in the machine running Connect, such that requests made from our machines will appear to come from your computer. When Firefox and Chrome run in our cloud, the way they interact with this tunnel enables them to "see" your localhost machine as their own. That means that all ports are available to them.

By contrast, Safari and IE refuse to acknowledge your machine as "localhost". Their heads are stuck in the cloud. So, when they see a request for port 8888, they look for it in the Sauce VM.

To get around this, we run a process that grabs these requests and actively forwards them through the tunnel. However, this takes processing power, and we have to manually enable each port that is to be proxied in this way. So, not every port is covered.

Here's the list of proxied ports, which will work for any browser. We do our best to support those that are used by popular frameworks, such as port 3000 for Rails servers, so if you have a request, just let us know (via [email protected]).

We're working to fully remove this localhost proxy as a superior solution. In the meantime,you can either choose to run your server on one of those ports, or alternatively, direct your tests to a non-localhost URL.

To do that, modify the hosts file in the machine running Sauce Connect. Just add a fake domain, such as "myname.local", which points to 127.0.0.1 . Then, point your tests to that URL instead of 'localhost'. This will skip the localhost proxy entirely and prevent the issue.

Testing Time Limits for Real Devices

Manual tests for free users have a 10 minute limit from session start

Manual tests for all other users are limited to six hours

Automated tests for all other users are limited to six hours

Testing Timeouts for Real Devices

Manual tests for paid users will time out after 15 minutes of inactivity

Automated tests time out by default after 60 seconds of inactivity. You canincrease this to 90 seconds with thenewCommandTimeout desired capability.

Device caching: When running automated tests and specifying the desired capabilitytestobject_cache_device:true,the device is kept allocated to you for 10 seconds after a test finishes, so you can then re-use the same device in your next test. This means that if you are sending over testobject_cache_device:true as a capability but taking longer than 10 seconds to reach TestObject on the next test, you will not be using device caching.

If you send Sauce Labs more tests at once than your concurrency allows (for example, you have a concurrency of 100 but send 200 tests at once), the "extra" tests will be queued and run as the first tests finish. Using the previous example, when the number of running tests drops to 99, 1 test from the queue will start running, and when the next test finishes, another test from the queue will start running. This continues until all the queued tests have been run. However, there are some caveats to this queuing process worth knowing.

We have a limit to the number of jobs that can be queued up. The number of queued tests must be less then <your concurrency limit> + 150.For example, if your concurrency limit is 100, there can be no more than 250 tests (100 + 150) in the queue. Once this queue limit is reached,further session requests are rejected right away. Thus, if you send 351 tests, 100 will run right away, 250 will be queued, and 1 will be rejected immediately.

A test is only allowed to remain in the queue for 10 minutes. If it has not been moved out of the queue to run before 10 minutes elapse, the test is removed from the queue and not run at all.

If a test is queued and run later, there is a chance that, by the time the test is run, your test runner will have already timed-out the test. In this case, Sauce Labs will start a virtual machine to run the test and then wait for commands from the test runner. If the test runner has already timed-out, our VM will receive no commands. After 90 seconds, Sauce will end the session and shut down the VM. The error message of those tests will be "Test did not see a new command for 90 seconds. Timing out." and no commands will have been run (the Commands tab for your test will be empty). To accommodate this, you can raise the idle timeout from 90 seconds to reduce the likelihood of timeouts from your test runner. This is done through the Desired Capability called "idleTimeout". For example: ["idleTimeout": 300] would increase the timeout to 300 seconds. See the topic Test Configuration Options in our Sauce Labs Cookbook.

There seems to be a bug on Chrome 64 that makes certain websites load very slowly.You can see the following on Sauce's chromedriver logs:

[14.252][INFO]: COMMAND Navigate {

"url": "https://ops.qa.heal.com/dashboard#CV-UHRFW"

}

[14.252][INFO]: Waiting for pending navigations...

[14.969][INFO]: Done waiting for pending navigations. Status: ok

[314.253][SEVERE]: Timed out receiving message from renderer: 299.283

[314.253][INFO]: Waiting for pending navigations...

[314.253][SEVERE]: Timed out receiving message from renderer: -0.001

[314.253][INFO]: Timed out. Stopping navigation...

[314.254][SEVERE]: Timed out receiving message from renderer: -0.002

[314.255][INFO]: Done waiting for pending navigations. Status: timeout

[314.255][INFO]: RESPONSE Navigate timeout

(Session info: chrome=64.0.3282.119)

This can be reproduced locally as well, which means this is not Sauce-related.

We believe that this is related to this bug: https://bugs.chromium.org/p/chromedriver/issues/detail?id=2273

The good news is that this seems to be resolved with a combination of Chrome 65 and chromedriver 2.35. At the time of writing, Chrome 65 is still in Beta. Running tests with the following desired capabilities appears to alleviate the problem:

'browserName': "chrome", 'version': "beta", 'chromedriverVersion':'2.35'

Due to a change made in Selenium 3.9.0, you may see an error when trying to start tests using Sauce Labs (or any other cloud provider) that requires basic authentication. This only happens when using Selenium 3.9.0 Java client bindings.

The error looks like this:

org.openqa.selenium.WebDriverException: Unable to parse remote response: Sauce Labs Authentication Error.

You used username 'None' and access key 'None' to authenticate, which are not valid Sauce Labs credentials.

This problem started when the default http client was changed from Apache's HttpClient to OkHttp. The solution is to update the Java client bindings to 3.9.1.

On Mac OS 10.12/Safari 10.1, Safari loses focus to GridLauncher, which is the Selenium server process, and Selenium commands fail with:

00:45:24.395 WARN - Exception thrown

org.openqa.selenium.NoSuchWindowException: The driver is not focused on a window. You must switch to a window before proceeding. (WARNING: The server did not provide any stacktrace information)

Similarly, on Mac OS 10.12/Safari 11.0, Safari loses focus to GridLauncherV3 (V3 because Safari 11 is using Selenium 3 by default). This is causing issues such as being unable to find an element when Safari is not in focus:

22:11:23.315 INFO - Executing: [find element: By.xpath: (//div[contains(@class, 'Select-option')])[position()=7]])

22:11:23.399 WARN - Exception thrown

org.openqa.selenium.NoSuchElementException: An element could not be located on the page using the given search parameters. (WARNING: The server did not provide any stacktrace information)

This is an Apple bug, not a Sauce Labs bug. It was reported to Apple in the bug report at http://www.openradar.me/radar?id=4982823136002048, which was closed as a duplicate of another bug. The problem has been addressed in the Safari Technology Preview, so when Apple releases Safari 11.1 as part of an OS update, this issue should be resolved on Sauce Labs.

When executing a mobile test with Appium,the automation framework that the test is running with is the one the receives and executes the driver commands sent by the test.

For iOS 9.2 and below, the only existent automation framework that Apple had available was the UIAutomation framework. With the release of iOS 10, Apple deprecated UIAutomation in favor of a new XCUITest automation framework.

iOS 9.3 is the only iOS version for which one can use either UIAutomation or XCUITest. For example, with the desired capability "automationName": "XCUITest" you can tell the Appium driver to use the XCUITest automation framework instead of using the default UIAutomation framework.

Although it is possible to test with XCUITest on iOS 9.3, we don't recommend it. In practice, we've found that iOS 9.3 doesnt have the complete infrastructure required to reliably support all the new behaviors that come with XCUITest. For this reason, testing with XCUITest is something that we recommend doing only with iOS 10.0 and above.

There are many differences between these two automation frameworks, some of which might require modifications to your tests scripts. You can find more information through the documentation available at the Appium GitHub repo.

Here are two ways to determine if a test was running using the Sauce Connect Proxy.

In the Test Detail page, clickon more... The lightening symbol will begreen if aSauce Connect Proxy was enabled during the test. You will also be able to see the ID of theSauce Connect Proxy tunnel.

Sauce Connect Enabled

Sauce Connect Disabled

Sauce Connect Maintenance and Troubleshooting

In theAutomated Testssection, tests that used a Sauce Connect Proxy will show a black lightening symbol. To see the details of the active tunnels, click Tunnels.

If you expected your test to be running through the Sauce Connect Proxy and it is not, check the configuration of your Sauce Connect Proxy tunnel and your test's desired capabilities. You may be missing the "tunnelIdentifier" or "parentTunnel" desired capabilities.

For more troubleshooting, check out the Sauce Connect Command Line Reference flags and the guidelines.

In the Test Details page,under the Metadatatab, in the Base Configrow, you can findthe exact desired capabilities that Sauce Labs received for the test.

On Windows 10 with the IE 11 browser, the result of hovering or performing a "moveTo" action in an automated test fails to appear in the application. For example, after hovering over the first user in image 1, the "name : user1" appears. In image 2 it didn't appear because we usedWindows 10 and IE 11.

Image 1

Image 2

RequireWindowFocus" : True

Our investigations suggest this happens because the IE driver doesn't have its focus on the browser window. Using the " desired capability in the automated test is a way to work around the problem.

The Safari 10.1 browser version is onlyavailable in Sauce Labs on Mac OS 10.12 for automated and manual tests. Use our Platform Configurator to obtain the correct desired capabilities for your automated tests.

Note: If youonly specify "10" as your browser version instead of "10.1", Sauce Labs will automatically default to providing Safari version "10.1" withMac OS 10.12. For example,sending in the these desired capabilities will default them to Safari 10.1:

DesiredCapabilities caps = DesiredCapabilities.safari(); caps.setCapability("version", "10");

To learn about the new changes in Safari 10.1, check out the Safari documentation in Apple.

Pre-run executable scripts can be used to alter browser settings by making changes to registry keys. For some settings, the change will not take effect until the browser or OS is restarted.Unfortunately there is not a way to restart the browser or OS using Selenium during a test session so these new settings will not take effect for your tests.One example is changing the Enable Enhanced Protected Mode in Internet Explorer, another is disabling the popup blocker in the Microsoft Edge browser.

You can find more information here:

https://answers.microsoft.com/en-us/windows/forum/windows_7-desktop/windows-registry-changes-is-a-restart-always/e131b560-1d03-4b12-a32c-50df2bf12752?auth=1

Sauce Labs bills customers for every minute of time used by one of your test sessions.

When your tests are running in parallel, they are using multiple test sessions. This means we bill you for each minute used by each of those sessions.

For instance, say you have 6 tests, each of which takes two minutes. If you run tests singularly, they will be finished in 12 minutes, and you will have had 12 minutes of Sauce Labs session time: 1 session x 6 tests x 2 minutes = 12 minutes.

If you run two tests at once, your tests will be finished in 6 minutes. However, Sauce Labs will still have allocated you 12 minutes of session time: 2 sessions x 3 tests each x 2 minutes = 12 minutes.

We don't formally offer a way to upload files to test file upload functionally, but there are several ways this can be done.

Use the LocalFileDetector function that is native to Selenium

Here are some example tests:

JAVA

import junit.framework.Assert;

import junit.framework.TestCase;

import org.openqa.selenium.*;

import org.openqa.selenium.remote.*;

import java.net.URL;

import java.util.concurrent.TimeUnit;

public class TestingUploadSauce extends TestCase {

private RemoteWebDriver driver;

public void setUp() throws Exception {

DesiredCapabilities capabillities = DesiredCapabilities.firefox();

capabillities.setCapability("version", "7");

capabillities.setCapability("platform", Platform.WIN10);

capabillities.setCapability("name", "Remote File Upload Test");

driver = new RemoteWebDriver(

new URL("https://<username>:<api-key>@ondemand.saucelabs.com:443/wd/hub"),

capabillities);

driver.setFileDetector(new LocalFileDetector());

driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);

}

public void testSauce() throws Exception {

driver.get("http://sl-test.herokuapp.com/guinea_pig/file_upload");

WebElement upload = driver.findElement(By.id("myfile"));

upload.sendKeys("/Users/sso/the/local/path/to/darkbulb.jpg");

driver.findElement(By.id("submit")).click();

driver.findElement(By.tagName("img"));

Assert.assertEquals("darkbulb.jpg (image/jpeg)", driver.findElement(By.tagName("p")).getText());

}

public void tearDown() throws Exception {

driver.quit();

}

}

RUBY

require 'rubygems'

require "test/unit"

require 'selenium-webdriver'

class ExampleTest < Test::Unit::TestCase

def setup

caps = Selenium::WebDriver::Remote::Capabilities.firefox

caps.version = "54"

caps.platform = :WIN10

caps[:name] = "Remote File Upload with Ruby"

@driver = Selenium::WebDriver.for(

:remote,

:url => "https://<username>:<api-key>@ondemand.saucelabs.com:443/wd/hub",

:desired_capabilities => caps)

@driver.file_detector = lambda do |args|

# args => ["/path/to/file"]

str = args.first.to_s

str if File.exist?(str)

end

end

def test_sauce

@driver.navigate.to "http://sl-test.herokuapp.com/guinea_pig/file_upload"

element = @driver.find_element(:id, 'myfile')

element.send_keys "/Users/sso/SauceLabs/sauce/hostess/maitred/maitred/public/images/darkbulb.jpg"

@driver.find_element(:id, "submit").click

@driver.find_element(:tag_name, "img")

assert "darkbulb.jpg (image/jpeg)" == @driver.find_element(:tag_name, "p").text

end

def teardown

@driver.quit

end

end

Use a pre-run executable to download a file from the internet

There is some information on this here.

NOTE: If you want to download a file that is only accessible on your network, this won't work even if you have Sauce Connect running. This is because the connection from the VM to your network is limited only to the browsers and doesn't work on all outbound connections

As part of your test, navigate to a site that contains your file and download it automatically to the Downloads file

The downloaded files can be found here:

Windows = C:\Users\Administrator\DownloadsMac = /Users/chef/DownloadsLinux = /home/chef/DownloadsNOTE: The expected behavior can vary between OS/browser combination and file type (for example, the file may be displayed on the browser instead of downloading).

As of early November, 2017, if you use Selenium 3.6.0 and above with Chrome 61+, and try to add a Chrome Option on Sauce, the test will return a Cannot find Chrome binary error.

This is because, starting with Selenium 3.6.0, the client bindings send W3C-compliant Chrome Options. These aren't being handled correctly yet by our backend, a limitation that will be fixed soon.

There are three known workarounds:

Use a Selenium client binding version lower than 3.6.0.

Specify the Chrome binary together with your arguments(Windows only))

"goog:chromeOptions":{ "binary": "D:\\Program Files\\Chrome 61\\61.0.3163.79\\chrome.exe", "args": [ ] }

or

"goog:chromeOptions":{ "binary": "D:\\Program Files\\Chrome 62\\62.0.3202.62\\chrome.exe", "args": [ ] }

Avoid using Selenium-given capabilities and force DesiredCapability as "chromeOptions" rather than "goog:chromeOptions". This means, for example, not using the ChromeOptions.CAPABILITY that will return "goog:chromeOptions".

Editing the Host file of the virtual machine will not work if Sauce Connect Proxy is in use. If you are using Sauce Connect Proxy, the Host file of the machine running Sauce Connect Proxy will be referenced and you can make the desired changes there.

Tests often include http requests to various domains/endpoints. DNS will, of course, resolve those domain names to an IP address.

Sometimes tests need to redirect their normal requests to a development server. These domains will not be found by DNS. A solution to this problem is to add an entry to the local hosts file of the computer running the test, thus making the domain resolve to the IP address of the development server. For example, one might want www.fonts.comto resolve to the IP address 196.118.45.9, because that is the IP address of a server that holds a development version of the fonts.

This can be done on a Sauce Labs VM with the help of a pre-run executable.

To make this happen, follow these steps:

Write a host file script with the URL redirect to the new IP address.

Upload the script to a publicly accessible location, like GitHub or Sauce Storage

Set thepreruncapability in your test script to load the script as the host file in the Sauce Labs virtual machine.

Below are examples of steps (1) and (3).

The Host File Script

Here are examples of the host file script,EditDNS,in both OS X/Linux and Windows versions.

#!/bin/bash

echo "196.118.45.9 www.fonts.com" >> /etc/hosts

@echo off

echo 196.118.45.9 www.fonts.com > %temp%\temphosts.txt

type C:\WINDOWS\system32\drivers\etc\hosts >> %temp%\temphosts.txt

copy /Y %temp%\temphosts.txt C:\WINDOWS\system32\drivers\etc\hosts

Setting the prerun Capability in Your Test Script

Having created and uploaded your host file script, you now need to refer to it using the prerun capability in your test script, as shown in this Python example.

#!/usr/bin/python

# -*- coding: UTF-8 -*-

import unittest

import time

from selenium import webdriver

class Selenium2OnSauce(unittest.TestCase):

def setUp(self):

self.desired_capabilities = webdriver.DesiredCapabilities.CHROME

self.desired_capabilities['version'] = '56'

self.desired_capabilities['platform'] = 'OS X 10.11'

self.desired_capabilities['name'] = 'Editing the DNS'

self.desired_capabilities['prerun'] = {'executable':'[LINK_TO_SCRIPT]', 'background': False }

self.driver = webdriver.Remote(command_executor = 'http://SAUCE_USERNAME:[email protected]:80/wd/hub', desired_capabilities = self.desired_capabilities)

self.driver.implicitly_wait(30)

def test_sauce(self):

self.driver.get('http://www.fonts.com')

title = self.driver.title

self.assertEquals("Cross Browser Testing, Selenium Testing, and Mobile Testing | Sauce Labs", title)

def tearDown(self):

self.driver.quit()

if __name__ == '__main__':

unittest.main()

Which browser versions are supported in SauceLabs?

SauceLabs supports many major versions, as well as dev and beta versions of most widely used browsers. We do not support specific minor versions, and you cannot select specific minor version to use for your tests.

Why doesn't Sauce Labs provide all minor versions?

Our service is designed for speedy and efficientprovisioning of desired platform-browser combinations for your tests. It is impractical for us to maintain virtual machine images with every minor browser version, and hosting many different browser versions would result in far too many idle, unused VMs. The only way Sauce Labs can spin up the test VMs quickly is by using a finite number of images. Downloading and installing user specified versions of browsers with each test is not a feasible solution either. Doing so would dramatically slow down tests across our whole cloud.

Are the browser versions offered by SauceLabs relevant?

To satisfy needs of our customers, we put active research into what versions are used the most in the world and support those. Obviously, usage changes and new versions come out. Sauce Labs adapts to these new usage patterns and adds support for new versions based on that. Most customers are happy with this because the browser versions we offer reflect what their own customers are most likely to use day-to-day.

In the end, no one can control which browser versions users will use to access your website, and therefore it is not crucial that tests are run on a specific minor version of a browser.

In order to stay on top of latest supported versions, please refer to Product Announcements & Release Notes.

Our feature request portal is managed directly by our Product Management team, giving you a direct line to ask for changes and new features. You can subscribe, comment, and vote for existing requests to show your interest.

To Make a Feature Request

Open the portal at https://saucelabs.ideas.aha.io.

Search the existing ideas. If someone else has requested a feature, adding a vote and/or comment makes it clearer that many customers are interested in that feature.

Click Add a new idea if no one has requested this feature.

Choose a product. If you're not sure what product the feature relates to, choose General Ideas.

Give your idea a brief summary under Your idea.

Add more details in the Please add more details field. The more clearly we understand what a feature is for and who it helps, the better we can help. You can find an optional, suggested template below.

Enter your email address under Tell us who you are.

Click Share Idea.

Feature Request Template Suggestion

A short paragraph explaining why the feature is needed and why the current product is lacking

A short paragraph explaining who the feature benefits, for example "All mobile testers" or "Developers fixing UI bugs"

A short paragraph explaining how you think the idea should work (if relevant)

Feature Request Example

Product: General Ideas

Your Idea: Capture HAR files automatically

More Details:

HAR Files are a standard format for recording HTTP traffic. They are used to debug page loading problems and track request and response times. Locally, they can be captured using an intercepting proxy or the Developer tools in a browser, but when using Sauce Labs there is no easy way to capture this information.

This feature would be useful for anyone trying to debug a page loading problem, check that proper requests were being made, or figure out why certain assets don't load. It would be useful for customers and Sauce support staff.

It would work by automatically capturing a HAR file, just like all the other logs currently captured. By visiting the Logs tab, users can view and download the HAR file for a given test. It should also be accessible via the REST API.

Sauce Labs requires Identity Provider (IdP) initiated Single Sign-On (SSO), which means you will need to log in to your identity provider instance (for example, Okta) to access the Sauce Labs web application for the first time. This will either log you into a newly created Sauce account (withJust-in-Time provisioning, if enabled) or log you into your existing Sauce account.

After this initial login, you will no longer need to go through youridentity provider instance again, as your Sauce logged-in session will remain active for the next 30 days. After 30 days, you will need to refresh you Sauce Labs login by logging in through you IdP again.

Selenium WebDrivers screenshot implementation varies depending on the browsers WebDriver and browser version. This is not to be confused with the automated screenshots Sauce Labs takes of tests as they are run. Check out the information on our implementation of the Selenium screenshot testing option in the Test Configuration Options topic in our wiki.

Browser/WebDriver Combinations that Take Full Page Screenshots

Full Page means that the entire HTML page is captured, regardless of what is rendered as visible on the browser itself. Firefox version 47 and below (which all use FirefoxDriver) and all versions of Internet Explorer take full page screenshots. However, with IEDriver version 2.41.0, screenshots will sometimes have black bars on the image. If you have to use this older version of the IE Driver, you can work around the problem by specifying a 64 bit version of the IE driver in the capabilities, like this:

capabillities.setCapability("iedriver-version", "x64_2.41.0");

This workaround is not necessary for IE 9 on Windows 7, as the screenshots work as expected on 2.41.0.

Browser/WebDriver Combinations that TakePartial Page Screenshots

Partial Page means that the screenshot captures only the currently visible portion of the viewport. This will vary with the resolution of the VM and how the browser renders the page. It's more likely the screenshots will be partial for these browser/webdriver combinations:

Firefox 48 and above (which all use Geckodriver).Known issues with screenshots: https://github.com/mozilla/geckodriver/issues/265

Chrome(all versions)Known issues with screenshots: https://code.google.com/p/chromedriver/issues/detail?id=294

Safari (all versions)Known issues with screenshots: https://code.google.com/p/selenium/issues/detail?id=3752

When tests are run using Sauce Connect and the browser is not able to connect, or times out when trying to connect to a particular site or server, the browser on the test VM will display a Zero Sized Reply error. This is the message that our Squid cache, which the Sauce Connect Proxy application is built on, gives when something is not right.

This error message can occur for a number of reasons known to the Squid project. If you encounter this error in either your automated or manual tests,please open a new support ticket at [email protected] and include the following information:

Command line flags used to start the Sauce Connect instance

Version and build number of the Sauce Connect instance

Operating System used to run Sauce Connect

The logfile produced by Sauce Connect, ideally when run with the '-v' flag (verbose mode)

While we know that having an resolution ETA for defect corrections can be reassuring and helpful for planning purposes, Sauce Labs Support is unable to provide ETAs.

How Our Teams are Organized

Sauce Labs' technical teams work on groups of related functionality. For instance, we have a team working on our Web UI, another on our Emulator support, and a third on Desktop environments. Our teams are holistic in that ach team accomplishes all tasks for their functionality, such as:

Product improvements from strategic planning

Defects reported by customers

Security concerns

3rd party updates (for example, new browser versions)

Tasks required by other teams

How Work is Prioritized

Planning for teams is done as part of an Agile process. Tasks are ordered by Product Management and technical leadership according to a number of competing factors:

3rd party release schedules (for example, new Selenium versions)

Seriousness of defect (customer reported or otherwise)

Impact on other teams

Legal and contractual obligations

Security obligations

Release goals

A defect reported by a customer is one of many tasks a product team will deal with, and is effectively "competing" for time against all the other priorities of that team (including other customer reported defects). This means a team may not be able to address a defect immediately.

Tasks are re-prioritised regularly as defects become more serious or impact more customers, as releases are announced, or external obligations arise.

How Work is Planned

Our development methodology means that the planning of tasks occurs immediately before each two week "sprint." Teams have a meeting with their leadership and product management, and select enough of the top priority tasks to fit in two weeks of work (a "sprint"). Tasks are only allocated to sprints immediately before that sprint begins. This means that tasks without a sprint haveno ETA. They may be allocated to the next sprint, but they may not.

Estimating Complexity

The biggest problem when it comes to providing an ETA for complex problems is that determining why a problem is occurring is often the bulk of the work. This is something that can't be estimated. The only way to know how long it takes is to do it.

When something is new, such as a new version of Windows or the iOS emulator, estimation becomes harder again. Often, Sauce Labs is among the first in the world to attempt something. This means there are no other resources to assist, and the problems that arise are completely novel.

This uncertainty is amplified when a problem is occurring in the interaction between systems Sauce Labs did not develop. For instance, determining why a version of Selenium is misbehaving with a version of Internet Explorer. We have the source code for Selenium, but not for Internet Explorer. We're also didn't construct either, so much of our work has to rely on educated guesses

Summary

We don't provide ETAs as it's usually close to impossible to provide a meaningful estimate. Even if we can provide one, we can only do so shortly before the ETA itself, and even that is subject to change.

We do not recommend testing PDF download functionality with Sauce. There are two main issues that come up when performing such a test using Selenium Webdriver:

Clicking on the link to the PDF file may automatically open a PDF plugin to view the file, which Selenium can't access.

Download dialogs are native browser UI elements, so they they are not supported by Selenium.

You could try to work around these issues by changing your browser configuration to download PDF files automatically, as described in this article from Elemental Selenium. However, users don't have access to the filesystem on the virtual machines where the browser is running. Essentially, file validation isn't something that can be done with a remote Selenium grid such as Sauce Labs.

To test file download functionality of the site, you should consider downloading it locally. First, get the URL of the PDF link using Selenium. Then download the file using wget, cURL, or your favorite HTTP library to your local drive. Finally, you can validate the file by checking the MD5 values of the downloaded files.

This blog post has more information regarding file downloads with Selenium: http://ardesco.lazerycode.com/testing/webdriver/2012/07/25/how-to-download-files-with-selenium-and-why-you-shouldnt.html

Often an administrator will want to start Sauce Connect Proxy as a Windows service. While Sauce Connect is not currently set up to be a Windows service, it can be made to run as one with the help of some external tools. This article describes how to set this up, but note that our example uses a third party tool. We provide this only as an example rather than a supported product from Sauce Labs.

What you will need

A copy of the latest version of Sauce Connect Proxy from https://wiki.saucelabs.com/display/DOCS/Sauce+Connect+Proxy

A Windows Service manager. In the example here we use NSSM.

Steps to Implement

Download NSSM ( http://nssm.cc/download ) and extract the .zip file.

Open the command prompt as Administrator, then navigate to the NSSM directory via the command line.

Create the service with this command:nssm install SauceConnect1 "C:\<path_to_sauceconnect_client>\bin\sc.exe" "-u <username>" "-k <access key>" "--tunnel-identifier TUNNELNAME" "--no-remove-colliding-tunnels" "--pidfile C:\Windows\Temp\pid1.log" "-P 4447"

Open up the Windows Services Manager and make sure the new services (named SauceConnect1) are set to start manually.

Create a batch file (for example restartSC.bat) that contains these lines:NET STOP SauceConnect1sleep 30NET START SauceConnect1sleep 30

Open the Windows Task Scheduler and set it to call restartSC.bat once a day or at any other period of time.

Once you've set all these up, the tunnel should restart itself daily at the time of your choosing.

What is a Concurrency Limit?

Your account's concurrency limit is the maximum number of tests you can start at once, including automated and manual. Once you have used all your concurrency slots, additional tests will not start until an existing test finishes.

Concurrency slots are shared across all accounts in an organization. Each subaccount has a limit that specifies how many concurrency slots are available from the organization's total.

Real Devices have a separate concurrency limit.

Checking Your Concurrency Limit

Each Sauce Labs account has a set maximum number of concurrent sessions. You can find your concurrency limit on the My Account page (at https://saucelabs.com/beta/users/ username). If this number does not match your subscription or invoiced contract, please contact Support.

Subaccounts may have had their concurrency limit lowered by their parent account. To access higher concurrency levels, you will need to ask the person responsible for the parent account to increase your limit.

How Concurrency Slots Are Used

Every independent Selenium or Appium session you start uses a single concurrency slot. Slots remain in use until the associated test has completely stopped running. This includes any time required for:

Transmission of a quit command from your test runner

Timing out due to lack of commands

Timing out due to errors

Encoding and storing assets

The number of concurrent tests your tests request is controlled by your test runner.

Requesting Tests in Excess of Your Concurrency

Test requests made when all your concurrency slots are in use may result in an error, or in the test being queued.

As tests complete, queued tests are allocated to concurrency slotsin the order they were queued.

Queuing is intended to deal with small mistakes in concurrency management. Sauce Labs does not recommend queuing tests intentionally.

Tests that are queued for too long will time out with an error. Tests may also time out if too many tests are already queued. Sauce Labs reserves the right to alter queue size, time and availability at will.

Queued tests are marked as "Queued" on your Dashboard.

Reasons Concurrency is Lower than Expected

Concurrency is in Use by Other accounts

Subaccounts use concurrency slots from their organization's total pool. Because of this, it's possible to have other users consuming slots you are trying to use.

For example: the Organization has a concurrency limit of 10. Subaccount A has a concurrency limit of 5. Subaccount B has a concurrency limit of 7.

When Subaccount A is using all 5 concurrency slots, that only leaves 5 of the Organization's slots open. Even though Subaccount B has a concurrency limit of 7, it will only be able to use the 5 remaining slots until Subaccount A's tests finish.

Your Test Runner has Not Requested Your Full Concurrency

The Selenium/Appium test lifecycle is controlled entirely by your test runner. Sauce Labs does not start a test session until it has been requested.

Most test runners require additional configuration in order to run tests in parallel. Many require the user to configure the desired number of parallel tests. If you're not seeing the correct number of tests run, check that your test runner is set up to run in parallel, and that it's configured to use your full concurrency.

One way to check if your test runner is running in parallel is to check your Dashboard. The gauge on the left shows your concurrent session count. Check it before and during a test run to see if it's increased by more then 1. If not, your tests are running serially.

status.saucelabs.com

Sauce Labs Service Issues

Tests may queue up if there is a service issue preventing Sauce Labs from allocating tests to VMs. First, check for an incident at . That site is the comprehensive source of truth for our service status.

If there is no incident and no reason to assume your concurrency slots are being used by colleagues, or if you aren't seeing any tests run (either on specific platforms or across the board), contact support.

If you need to test the Chrome browser with certain browser options, you can useChromeOptions to customize browser settings for your tests. You can find more information on the official Chrome website. This article includes some examples of customizations that you may find useful.

Default Chrome Browser Settings Used by Sauce Labs

By default, Sauce Labs uses a set of default ChromeOptions. If you use custom options, those defaults will be entirely overriden. It's important to retain these default options for your tests to work properly, so if you want to use custom options, you will also want to include these options that Sauce normally provides:

"args": [

"start-maximized",

"disable-webgl",

"blacklist-webgl",

"blacklist-accelerated-compositing",

"disable-accelerated-2d-canvas",

"disable-accelerated-compositing",

"disable-accelerated-layers",

"disable-accelerated-plugins",

"disable-accelerated-video",

"disable-accelerated-video-decode",

"disable-gpu",

"disable-infobars",

"test-type"

]

Example Chrome Browser Settings

Setting Browser Language

DesiredCapabilities caps = new DesiredCapabilities();

ChromeOptions options = new ChromeOptions();

options.addArguments("--lang=es"); -> Changing the language to spanish

caps.setCapability(ChromeOptions.CAPABILITY, options);

Forcing Chrome to Allow Push Notifications

DesiredCapabilities caps = new DesiredCapabilities();

ChromeOptions options = new ChromeOptions();

Map<String, Object> prefs = new HashMap<String, Object>();

prefs.put("profile.default_content_settings.popups", 0);

prefs.put("profile.default_content_setting_values.notifications", 1);

options.setExperimentalOption("prefs", prefs);

caps.setCapability(ChromeOptions.CAPABILITY, options);

If you need to test the Firefox browser with certain browser options, you can use FirefoxProfile to customize browser settings for your tests. You can find more information about setting FirefoxProfile i n the official Selenium documentation.Here are a few examples of what you can do with changing browser settings.

Setting Browser Language

DesiredCapabilities caps = new DesiredCapabilities();FirefoxProfile profile = new FirefoxProfile();profile.setPreference("intl.accept_languages", "es");caps.setCapability(FirefoxDriver.PROFILE, profile);

Installing a Firefox Extension, such as Modify Headers

You will need to download the .xpi file to your local machine before creating the profile settings.

DesiredCapabilities caps = new DesiredCapabilities();FirefoxProfile profile = new FirefoxProfile();profile.addExtension(new File("path\of\Modify Headers xpi file"));profile.setPreference("general.useragent.override", "UA-STRING");profile.setPreference("extensions.modify_headers.currentVersion", "0.7.1.1-signed");profile.setPreference("modifyheaders.headers.count", 1);profile.setPreference("modifyheaders.headers.action0", "Add");profile.setPreference("modifyheaders.headers.name0", "X-Forwarded-For");profile.setPreference("modifyheaders.headers.value0", "161.76.79.1");profile.setPreference("modifyheaders.headers.enabled0", true);profile.setPreference("modifyheaders.config.active", true);profile.setPreference("modifyheaders.config.alwaysOn", true);profile.setPreference("modifyheaders.config.start", true);caps.setCapability(FirefoxDriver.PROFILE, profile);

The Problem

By default, Android emulators are not able to accept Unicode characters. This means that foreign languages, extended characters, and Emoji are all unsupported by default.

The Solution

Thankfully, you can turn Unicode support back on! For Appium tests, simply set the unicodeKeyboard desired capability to true.

unicodeKeyboard: true