Distil Networks FAQs

We apologize that you are experiencing issues while browsing a site protected by Distil Networks.The issue you are experiencing is often tied to a third-party browser plugin blocking the Distil Networks JavaScript snippet from being executed.

Why is a third-party plugin causing me to complete a CAPTCHA or complete an Unblock Request form?As an option with our service, we empower site owners to protect (and choose how to protect) their websites from automated agents that don't run JavaScript. When a third-party plugin blocks the Distil Networks JavaScript from executing, it causes your browser to look like the millions of malicious bots that are actively attacking websites every day. When the JavaScript cannot be executed, it causes Distil to respond with the content protection setting chosen by the site owner.

Most plugins that block Distil Networks are related to user privacy. A few notes on our JavaScript/cookies:

Distil Networks uses first-party cookies on websites. This means that the cookie is only accessible on the website you’re currently visiting, much like a login cookie would be.

Distil Networks does not require cookies to be enabled in order to complete our JavaScript test.

Distil Networks does not collect any personally identifiable information in our JavaScript test.

This Distil Networks JavaScript code is processed after all other JavaScript execution, meaning it will never slow down the page you're requesting.

We hope that all users feel comfortable allowing the Distil Networks JavaScript to run. We are currently working with security plugin providers to revise their classification of our service. If you agree, please consult the documentation of the specific plugin that you utilize for information on how to specifically whitelist the Distil Networks JavaScript.

Distil’s software development kit (SDK) helps protect your API servers by placing advanced bot detection directly inside your mobile app. Once integrated with it, the Distil SDK generates a unique mobile token that identifies the device on the server side using encryption and tamper proofing.

In order for your mobile app to perform a request to your API server, a token must be submitted along with the request as an HTTP header. To obtain such a token, the SDK performs a challenge-response authentication with the Distil server. The token is valid for a limited time. After it expires, the token authentication must take place again before successfully accessing your API endpoint.

Prerequisites

Distil mobile app security with the DistilSDK requires:

A Distil instance (on-premise Distil appliance, on-premise customer appliance, cloud CDN, private cloud, or a Connector type integration) protecting your API endpoints.

A mobile application that has integrated the Distil SDK.

Supported Operating Systems

The Distil SDK currently supports the following operating systems:

iOS v10.3.1+

Android v4.2+ and applications built with API level 17

Deployment Architecture

As outlined in the diagram below, the SDK-integrated application calls getToken to initiate the authentication. In case a valid (non-expired) token is available in the local SDK cache, it can be returned immediately to the application. If not, the SDK performs a GET /challenge request.

The Distil server provides the SDK with a challenge and the client sends a response back to the Distil server. This response may also contain device-specific parameters. The Distil server verifies the response and issues a token, valid or invalid, depending on validity of the challenge response. The validity of the token is opaque to the client.

On subsequent API calls, the Distil proxy intercepts the request along with the token added by the application and the request is either allowed or denied depending on the token.

Users browsing a site protected by Distil Networks may experience an error due to a plugin, such as Google Chrome’s Data Saver extension. This extension routes your network traffic through a third-party data center or other proxy.

Why is a plugin causing me to get blocked?

This type of plugin works by routing unencrypted requests from your browser through a third-party service, while still allowing your browser to directly send other requests. Such third-party requests often come from data centers that are geographically dispersed throughout the world, making a single request appear as though it is simultaneously coming from multiple browsers and locations. This ultimately leads to browser validation issues that result in receiving the “Unable To Identify Your Browser” page.

How do I resolve this issue?

We recommend either disabling the plugin extension before accessing the site or visiting the website over a secure (HTTPS) connection.

For disabling the plugin in the Chrome browser on your computer:

Click the Data Saver icon in the menu bar.

Choose Turn Off Data Saver.

For disabling the plugin in the Chrome browser on your Android device:

Open the Chrome app.

Tap and then Settings.

Under Advanced, tap Data Saver.

Switch the setting to Off.

Distil Networks' Help Center is your knowledge base forhow-toarticles, FAQs, feature announcements, and other general information regarding Distil. The best way to stayup to date with the latest featureannouncements, article updates, and system maintenance notifications is by subscribing to sections.

For example, make sure you never miss a scheduled or unscheduled maintenance window by subscribing to theNotifications section. Simply select the section name, click Follow, and set your subscription preference. From now on, you will receive a notification as soon as we publish anew Notification article.

You can also subscribe to individual articles and receive analert any time an article is updated or a comment is posted to the article. Simply access the article you'd like to follow and clickFollow.

The values that are used for ACLs on the legacy Apollo platform must be standard english unicode. Special non-english characters are not supported as values for ACLs. Using non-standard unicode characters may prevent the ACLs from being updated at the edge instances. This can result in a difference between what is configured in the Portal and what is active at the edge instances.

Click your username in the top-right corner of the banner menu and then select User Settings from the dropdown menu for access. From this page, you can:

Modify user contact information and password settings

Regenerate your Distil authentication token

Configure additional two-factor authentication settings

The table below lists all possible actions Distil can take for a request, along with the definition of each action.

Action

Definition

@block

Either the default Distil or Custom Block page was served to the user.

@captcha

Either the default Distil or custom CAPTCHA page was served to the user.

@captcha_correct

The user correctly completed a CAPTCHA.

@captcha_incorrect

The user completed a CAPTCHA incorrectly and was then served another CAPTCHA.

@captcha_update_fail

The user correctly completed a CAPTCHA, but an internal process failed updating their traffic record. The user was then served another CAPTCHA.

@domain_not_found

The user made a request to a domain that was not configured in the Distil Portal.

@drop

Either the default Distil or Custom Drop page was served to the user.

@error

The user encountered an error generated by Distil. This could be related to an HTTP or TCP timeout.

@force_identify

The user was served the Distil identification page.

@force_ssl

A request was made over HTTP when all traffic should be HTTPS. A 301 redirect to the HTTPS version of the website was served.

@hpl

A request was made to the Distil honeypot link, which then triggered violation 32 for a GET request and 65536 for a POST request.

@identify_block

A request was served the Distil identification page. If JavaScript was not running, the user was shown the Distil Block page or Custom Block page.

@identify_captcha

A request served the Distil identification page. If JavaScript was not running, the user was shown the default Distil or custom CAPTCHA page.

@identify_cookie

A request was made to the Distil identify cookie page, which validate that a user was running JavaScript and using a valid cookie (if available).

@identify_drop

A request served the Distil identification page. If JavaScript was not running, the user was shown the Distil Drop page or Custom Drop page.

@invalid_domain

The user made a request to an invalid domain.

@js_validate

The user was served the default Distil Validation page or Custom JavaScript Validation page.

@jst

The user was served the full JavaScript test.

@jst_head

The user was served the JavaScript AJAX Token Refresh.

@jst_invalid

The user submitted invalid data in the JavaScript POST that prevents clearing of JavaScript.

@jst_miss

The user made a request to a page that matches the pattern set by the Distil JavaScript URL path, however it was no longer an active path.

@jst_post

The user completed the Distil JavaScript test.

@pjst

The user was served the partial JavaScript test.

@proxy

A request was made that was proxied to origin/cache. It was not subject to Distil injections either because it was found to be non-injectable based on Content-Type or was whitelisted.

@proxy_ajax

A request was made that includes the active Distil AJAX tokens, disabling injections.

@proxy_inject

A request was made to a page that was subject to Distil's injections.

@test_block

A request was made to the test Block page and the user was served either the default Distil Block page or Custom Block page.

@test_captcha

A request was made to the test CAPTCHA page and the user was served either the default Distil or custom CAPTCHA page.

@test_drop

A request was made to the test Drop page and the user was served either the default Distil Drop page or Custom Drop page.

@test_error

A request was made to the test Error page and the user was served either the default Distil Error or Custom Error page.

@test_js_validate

A request was made to the test JavaScript Validate page and the user was served either the default Distil JavaScript Validation or Custom JavaScript Validation page.

@test_unable_to_identify

A request was made to the test Unable to Identify page and the user was served either the default Distil Unable to Identify page or Custom Unable to Identify page.

@unable_to_identify

Either the default Distil Unable to Identify page or Custom Unable to Identify page was served to the user.

@upstream_not_found

A request was made to Distil, however we are unable to find an appropriate upstream to forward the request.

@www_redirect

A request was made to a root domain (e.g., domain.com) and Distil returned a 301 redirect to the the www hostname (e.g., www.domain.com)

This article is intended to go over a few scenarios to illustrate how caching, Force Identification (Force ID), Single Page Apps, and whitelisting can interact with each other.

Assumptions

Below will be the workflow for the following scenarios:

Client --> CDN --> Legacy Distil Proxy --> Origin

All requests go to the URI https://www.foo.bar/login.html. The basic application has the browser send an AJAX request to POST credentials to the login endpoint /submit.json when attempting to authenticate. In all of the following examples, the setting Automated Browsers is set to CAPTCHA and Force Identification is set to enabled. It is important to consider that the behavior of Force ID can be different with Automated browser set to monitor vs CAPTCHA/Block/Drop. However, that is a topic for another article.

Scenario 1

Further Assumptions

Caching enabled at CDN

False

Some whitelisting is applied

False

Above is the most straight forward example. All users from all locations will have the same experience. If the user's browser has not fingerprinted yet, then the client will go through the identification process. Below is the Force identification process.

The browser sends a GET /login.html without having fingerprinted or setting the Distil identifiers within an existing session. Instead of Distil proxying the request to the origin Distil will perform the following:

Distil will return the identification page

The browser will fingerprint

The browser will be directed to /distil_identify_cookie.html

The browser will be redirected via a 302 to URI that the client originally requested. From this example, the URI would be /login.html

Once the user's browser has successfully gone through the above workflow, the client will be able to submit credentials via AJAX to /submit.json with Distil cookies.

Scenario 2

Further Assumptions

Caching enabled at CDN

True

Some whitelisting is applied

False

In this example, different users can have different experiences when interacting with the website. We will assume that the origin returns cacheable content and that the CDN honors cache control headers and directives. The first user to the site will go through the identification process mentioned in scenario 1. However, when the 2nd user visits the site shortly after the 1st user, the 2nd user will see the cached version of the of /login.html resource. The /login.html page should still have the Distil JS injected into the page so that the 2nd user may fingerprint properly. As long as the 2nd user is able to execute the Distil JS prior to sending the AJAX call to /submit.json, then the user workflow will not be disrupted. However, if the 2nd user submits their credentials to the endpoint /submit.json prior to completing the Distil fingerprinting process, then this will result in a violation. Distil recommends to key on the browser load event to disable the site login event until the load event has completed. Below a link to the browser load event.

https://developer.mozilla.org/en-US/docs/Web/API/Window/load_event

Scenario 3

Further Assumptions

Caching enabled at CDN

True

Some whitelisting is applied

True

This example will be similar to scenario 2 as long as the 1st user does not have the requests to /login.html whitelisted. However, if requests to /login.html are whitelisted for the 1st user and not the 2nd, then the experience for the 2nd user will be different.

If the whitelisted version of the resource is cached for /login.html, then the Distil JS will not be injected in the response HTML. This means that while the 2nd user will be able fetch the resource /login.html from the CDN cache, the browser will not be able to fingerprint. When the 2nd user attempts to authenticate to /submit.json, the 2nd user will not have a unique session and will receive a captcha response.

Why Would I Need to do This?

Your local hosts file serves as an authoritative DNS data store for your local computer. You can edit your hosts file to map any domain or subdomain to a specific IP address. Modifying your local host is an easy way to control what IP address your browser will use to send HTTP traffic for a particular domain or subdomain.

Test Before Onboarding

Before onboarding your production domain to Distil, it is advisable to send a bit of test traffic through the Distil network to verify that everything will work as expected. In most cases, everything works as expected, but finding out before making a DNS change is certainly best practice.

Test When Troubleshooting

After moving your production domain to Distil, you may find the need to test your site directly, bypassing Distil. This is a key step in troubleshooting when trying to isolate the cause of site performance or server errors.

How do Modify My Hosts File?

The quickest way to test your site on the Distil service is to modify your local HOSTS file to resolve DNS requests against Distil. This will ensure your domain routes to Distil then to your origin server. This will accurately simulate what traffic will look like against Distil.

Editing HOSTS in Windows 10 and Windows 8

Click the Start button and search for Notepad.

Right-click the Notepad icon and select Run as Administrator.

Select Continue and grant permission on the Windows Needs Your Permission User Access Control (UAC) window.

In Notepad, select File > Open.

Enter “C:\Windows\System32\Drivers\etc\hosts” in the filename field and click Open.

Add an entry for [yourdomain.com] and [www.yourdomain.com] pointing at the Distil IP provided under the Show DNS Configuration Instructions page in the Distil Portal (Domains > [yourdomain.com] > Settings > Show DNS Configuration Instructions).

Editing HOSTS in Windows Vista and Windows 7

Navigate to Windows Start menu > All Programs > Accessories.

Right-click the Notepad icon and select Run as Administrator.

Select Continue and grant permission on the Windows Needs Your Permission User Access Control (UAC) window.

In Notepad, select File > Open.

Enter “C:\Windows\System32\Drivers\etc\hosts” in the filename field and click Open.

Add an entry for [yourdomain.com] and [www.yourdomain.com] pointing at the Distil IP provided under the Show DNS Configuration Instructions page in the Distil Portal (Domains > [yourdomain.com] > Settings > Show DNS Configuration Instructions).

Editing HOSTS in Mac OSX

Navigate to Applications > Utilities > Terminal.

Enter “sudo nano /private/etc/hosts” to open the HOSTS file.

Enter your user password, when prompted.

Add an entry for [yourdomain.com] and [www.yourdomain.com] pointing at the Distil IP provided under the Show DNS Configuration Instructions page in the Distil Portal (Domains > [yourdomain.com] > Settings > Show DNS Configuration). This should go at the bottom of the file.

Save the HOSTS file by pressing Control+X.

Refresh DNS values by entering “sudo dscacheutil -flushcache”

Migrating Your Website to Distil Using PipeDNS:

Pipe DNS requires you to submit updates to them to change their DNS. In that case, ask them to change yourwww CNAMEto point to www.yourwebsite.com.distil.us. Have them point your root domain at the IP address given to you by Distil in either the Add Domain wizard in the portal or in your deployment email.

The tutorial video below covers the same topics as this article.

Distil Networks' wildcard feature is designed for customers having a large volume of domains or subdomains. It eliminates having to enter every domain and subdomain in the Distil Portal. Using wildcards lets you manage a single set of traffic routing rules, protection settings, and distribution settings no matter what domain or subdomain your users are attempting to access.

For example, let’s say your site is sectioned by category (photos.yourwebsite.com, videos.yourwebsite.com, audio.yourwebsite.com, etc.). Rather than entering each subdomain one by one, use the wildcard:

*.yourwebsite.com

All requests made to any yourwebsite.com subdomain are filtered through Distil using the same criteria. All reporting data is summarized within the wildcard, as well.

Within the Distil Portal, you can create a wildcard when adding a domain or add one to an already-existing domain.

Customers having on-premise Distil deployments, can create a wildcard for the top-level domain (.com, .net, .org, etc.), so that any any request for a domain with having matching top-level domains (e.g., yourwebsite.com, yoursecondwebsite.com, yourthirdwebsite.com, etc.) is treated as a wildcard.

Creating a Wildcard for a New Domain

To create a wildcard with a new domain:

Log in to the Distil Networks Portal to access the Domains Dashboard.

Select Add Domain.

Enter the Fully Qualified Domain Name.

Enter the Origin Server (IP/CNAME).

Select the Wildcard? checkbox.

Select Add Domain.

Migration (DNS Changes) articles

You will also need to update your DNS to complete the process (for more information about this, check out the Migration (DNS Changes) articles in our Help Center). Once you have completed this, your traffic is then routed through the Distil Content Protection Network.

That’s it!

To add new subdomains to a wildcard, simply update your DNS to route the new subdomain traffic through your wildcard settings. For more information, see the section "Adding New Domains and Subdomains to an Existing Wildcard" below.

Adding a Wildcard to an Existing Domain

To add a wildcard subdomain to a domain that’s already in the Distil Portal:

Log in to the Distil Networks Portal.

Within the Domains Dashboard, select the associated domain.

Select Settings from the top banner menu.

Within the Subdomains section, select Add Subdomain.

Select the Wildcard? checkbox.

NOTE: The Origin Server (IP/CNAME) field is auto-populated with the origin server of the associated domain.

Verify the Origin Server (IP/CNAME) is correct.

Click Add Subdomain.

All done!

To add new subdomains to a wildcard, simply update your DNS to route the new subdomain traffic through your wildcard settings. For more information, see the section "Adding New Domains and Subdomains to an Existing Wildcard" below.

Creating a Wildcard for a Top-Level Domain

This option is available only to customers with on-premise deployments. To create a wildcard for a top-level domain:

Log in to the Distil Networks Portal.

Within the Domains Dashboard, select Add Domain.

Enter a top-level domain (com, net, org, etc.) in the Fully Qualified Domain Name field.

NOTE: Do not include a period in this field. For example, enter “com” rather than “.com”.

Select the Wildcard? checkbox.

Enter the Origin Server (IP/CNAME) in the corresponding field.

Click Add Domain.

You will also need to update your DNS to complete the process (for more information about this, check out the Migration (DNS Changes) articles in our Help Center). Once you have completed this, your traffic is then routed through the Distil Content Protection Network.

Congratulations!

To add new subdomains to a wildcard, simply update your DNS to route the new subdomain traffic through your wildcard settings. For more information, see the section "Adding New Domains and Subdomains to an Existing Wildcard" below.

Adding New Domains and Subdomains to an Existing Wildcard

Now that you’ve created wildcards, you can add new domains and subdomains to a wildcard without any additional updates in the Distil Portal.

NOTE: The settings and reporting data of an exact domain or subdomain will take priority over any wildcard settings and reporting data.

To add new domains and subdomains to a wildcard:

Select the domain record associated with the wildcard from the Domains Dashboard.

Select Settings.

Select Show DNS Configuration Instructions.

Follow the generated instructions to update your DNS (for more information about updating your DNS, see the in our Help Center.)

Google and other good bots may end up crawling the Distil captcha, block, or drop page. When this happens the request is recorded as an Error, as the HTTP status code served back on these pages is a 405, 416, or 456 respectively.

An example of how Google webmaster tools records a request for the Distil threat response pages is shown below:

To avoid errors such as these you can modify your robots.txt with the directive to Disallow for the path /distil. Testing this with robots.txt Tester on the Google webmasters tool is a quick and easy way to find out how the Status changes.

The status shows Blocked as seen from the screenshot above. With the robots.txt change live on your site Google bot/other good bots will no longer try to crawl the page. If you test via the option Fetch as Google, you will see a status of Blocked and it will no longer be reported as an error.

Welcome to the Distil Networks Help Center. The Help Center will provide you with the ability to:

Open, update, and close cases.

View all your cases and case history with our support team.

Browse our knowledge base content.

NOTE: Some contentis restricted to signed in registered users.

Participate in our online support community.

There are two ways to register:

If you have already emailed us, chances are we know who you are. If so, all you have to do is set a password.

If you are new to the Help Center and have yet to email us, please signup.

Thanks for registering with us!

This is an error page that Distil displays to client browsers when we are having issues connecting to your server.

What is this page showing me?

There was an error connecting to your origin servers,

Information about the request thatfailed, and

Tracking numbers necessary forDistil's Support Team to investigate the root cause of the error.

What is the most likely cause of this error?

In almost all cases, a 502 error indicates that the origin server failed to respond to a request within a reasonable amount of time (a minimum of 60 seconds). The most common causes and solutions include:

Your server may be rate limiting or blocking requests from Distil.

Requests sent to your servers from Distil will appear to have originated from a limited set of IP addresses. As such, your firewall, load balancer, or application server may be configured to detect this as malicious behavior and attempt to block it. Make sure the Distil IP range is not subject to rate limiting or blocking.

There is no route to your server.

Make sure the configuration for the impacted domain is correct in the Distil Portal. In particular, verify that the IP/CNAME is correct.

Things you should try before contacting Distil

First, determine if the issue exists when bypassing the Distil network. To do so, edit the hosts file on your computer to override normal DNS lookups for the impacted domain. You can read more about testing with a hosts file here.

If you are able to reproduce the issue when bypassing Distil, then the issue is not caused by Distil. Contact your server administrator and/or hosting provider to investigate.

Make sure you have whitelisted all of Distil’s IP ranges in your origin firewall.

Make sure your origin IP/CNAME is correct inside the Distil Portal.

Make sure your DNS settings are correct.

Make sure your origin server(s) and load balancer(s) are all online.

Check the resources on your origin server(s) and load balancer(s). Make sure CPU and RAM is not being maxed out by other processes, making you server(s) unable to handle new request.

Check if any scripts are failing or taking a very long time to execute.

Run a traceroute or MTR to your origin IP and check for packet loss somewhere in the network.

What do I need to send to Distil Support?

When submitting a support request to Distil to investigate this error, it is important to provide the trace information from the error page. Please don't send a screenshot, rather, copy and paste the trace information. When copied and pasted, the text will look like this (from item 3 above):

You reached this page when attempting to access http://www.example.com/ from 192.168.0.5 on 2015-12-25 01:23:45 GMT. Trace: BB1EDE86-8FA5-11E4-B9B7-AC009C5BE403 (502) via b3a86554-de04-4db6-bf5a-ae890e529127; .454 / 673

The tutorial video below covers the same topics as this article.

The CAPTCHA Requests report gives you an overview of CAPTCHA activity related to your site, including the number of times CAPTCHA pages were solved, failed, or not attempted. This report helps you to determine how effective CAPTCHA pages are as litmus tests in relation to letting human users in and blocking out malicious bots.

Typically, solved CAPTCHA forms are a sign of human users that venture outside of “normal” user behavior for your site. They solve the CAPTCHA and move along to your site without further blocks or issues. Failed and unattempted CAPTCHA forms generally signify a malicious bot (or automated browser) that 1) behaves beyond “normal” user behavior, 2) receives the CAPTCHA, and 3) is either unable to complete the form or ditches the effort and moves to another site or page.

CAPTCHA forms aren’t always the answer. They can be defeated by malicious bots and can lead to poor user experience and lost leads.

In light of this, Distil serves CAPTCHA pages according to your Content Protection settings, which you can modify at any time in the Distil Portal.

Accessing the CAPTCHA Requests Report

Follow these steps to access the CAPTCHA Requests report:

Log in to the Distil Networks Portal to access the Domains dashboard.

Select a domain.

Click Reports on the banner menu.

Click TrafficOverview to expand the Reports dropdown menu.

Click CAPTCHA Requests.

Reviewing the CAPTCHA Requests Report

The CAPTCHA Requests report shows a snapshot of CAPTCHA activity surrounding your site, including:

Date Filter Specific date range highlighted by the CAPTCHA Requests report.

Total Traffic vs CAPTCHA Served Total requests to access your site compared with number of times a CAPTCHA page was served.

Attempts Solved vs Failed Breakdown of all CAPTCHA attempts, including solved, failed, and no attempt made.

Daily CAPTCHA Requests Graphical breakdown of total traffic versus CAPTCHAs served, solved, and failed.

Distil maintain a distinct traffic records for a given fingerprint per domain configured in the Distil Portal. A fingerprint is a Distil cookie or SDK token typically. The sub-domains for a given Distil domain will all share the same traffic record for a fingerprint for the domain. The traffic record includes information about what fingerprint has successfully completed a captcha. This means that if api.domain.com and www.domain.com are separate domains in the Distil Portal and the API calls from the browser are going to api.domain.com, then the www and api domains will have distinct traffic records for a given client even if the same identifier is used. This scenario is very important when designing single page web applications and the captcha workflow. See the following article for more information on designing single page web applications with Distil, https://help.distilnetworks.com/hc/en-us/articles/360036574073-Setting-up-CAPTCHA-with-Single-Page-Apps.

Let's assume that www and api are configured as sub-domains to domain.com within the Distil Portal.If the user clears captcha at www.domain.com in this scenario, then Distil acknowledge that the captcha was cleared for the users fingerprint at api.domain.com. If www and api are configured as sub-domains to domain.com, then the Distil traffic record is available for both sub-domains, www and api.

Now let's assume thatwww.domain.comandapi.domain.comare both distinct domains in the Distil Portal. If the user is in violation for AJAX calls toapi.domain.com, but the captcha clear event occurs atwww.domain.com,then Distil will never mark the client as clearing captcha for requests going toapi.domain.com. This behavior may appear to be a captcha loop to the client. In this type of workflow, the client must be instructed to solve the captcha atapi.domain.com.

Onboarding to the Distil network is a simple process but it requires some preparation. Before you migrate to the Distil network, please follow this short checklist.

Verify Origin Server IP / Hostname After you log in to the Distil Portal for the first time, verify that we correctly identified your origin server's IP address.

NOTE:if you do nothave a Distil Portal account yet, contact your account executive and continue with the list.

To verify that your origin server's IP address is correct, select your domainfrom the main domain list and clickSettingson the top left of the next page. You can find your origin server IP address on the top right of the page. ClickEditto change this setting. You can change this entry to either an IP address or a hostname.

Whitelist Distil IPs on your Firewall Since all your traffic will route through the Distil network, your firewall will see only connections from Distil IP ranges. Many firewalls will misinterpret this high traffic volume from a narrow IP range as an attack and begin to drop traffic. If your firewall uses rate limiting or throttling, it is important to whitelist Distil IP ranges.

For a full list of Distil's current IP addresses, please visit our support article, What Distil IP Range Do I Need to Whitelist?

Request Client IP As a reverse proxy, Distil relays requests to your origin server from our own IP addresses. By default, Distil passes the requesting client IP back to your origin server in theX-Forwarded-ForandX-Real-IPheaders. If your web server requires the requesting client IP in a different header (for exampleX-True-Client-IP), you can configure this in the Distil Portal underSettings->Content Distribution->Custom Headers.

Whitelist Monitoring Tools Since Distil's technology looks for automation, Distil often will flag monitoring tools such as Pingdom, Catchpoint, New Relic, and similar services as "bad bots" unless you whitelist these services. If you utilize these tools or any custom monitoring scripts, please add their IP ranges to theIP Access Listunder theSettingstab for your domain.

For more information about adding Distil IP ranges, please visit our support article, I Have A Third-Party Monitoring Tool, How Do I Add It?

Prepare DNS SettingsBefore you change your DNS to point to the Distil network, reduce the time-to-live (TTL) values on the A and CNAME records for your domain on your DNS provider. We recommend you lower these values to 5 minutes (300 seconds). This allows traffic to move on to the Distil network quickly, rather than wait for propagation due to a long TTL.

Please visit our collection of DNS Migration articles at the Distil Support Center for provider-specific instructions on updating your DNS. Distil only proxies HTTP and HTTPS traffic. Distil does not proxy email or FTP traffic. Email traffic includes SMTP, POP3, IMAP, and other e-mail protocols. Configure any email or FTP-oriented subdomains (such asmail.,smtp., orftp.) to point to your origin server IP and not to the Distil network. If you point your email traffic at Distil, it will interfere with your ability to send and receive emails.

Request SSL Distil does not enable SSL or HTTPS traffic for websites by default. If your site requires SSL, youmustcontact Distil support at [email protected] before you onboard. In your request, please include all the specific hostnames you want to use with HTTPS. Distil support will add your domain to our network's SSL certificate and confirm with you that we completed this process. If you currently use SSL, please do not attempt toonboardbefore you request SSL.

You can find more information on the Distil shared SSL certificate in our support article, How Does Distil Implement SSL (TLS) Certificates For My Site? If you do not use SSL, please skip this step.

Conduct Functional Tests We recommend performing basic functional tests to address any unexpected false positives. Configure all threat responses to CAPTCHA mode and use a generic browser to make a request to your website. Your Distil solutions engineer can help troubleshoot any problems that may arise.

Find more information about conducting functional tests in our support article, Functional Testing Before Onboarding with Distil.

Take advantage of the following optional checklist items to further optimize your Distil deployment:

Prepare Logging- If you use on-server logging or require the IP address of the user making the request, you'll want to integrate Distil'sX-Forwarded-Forheader into your logs. You can find more information in the Logging Integration and How Can I Set Distil To Only Monitor Bots? articles.

Adjust Upload Size Limit- Distil allows users to upload file sizes up to 4MB. If you or your users need to the ability to upload files larger than 4MB, please contact Distil support at [email protected].

Prepare Long-Running Processes- Distil allows HTTP requests that receive a response within 60 seconds. If you have any requests that may exceed this limit (i.e. a page that displays a list after a long database query), the request will timeout and a 504 error will be served. If you haverequests that will take longer than 60seconds, please inform Distil via [email protected]. They can provide you with possible solutions to prevent any potential issues.

That's it! Once your domain is ready for migration, log in to the Distil Portal, click your domain, clickSettings,and clickShow DNS Configuration Instructionsfor tailored migration steps. You are well on your way to a bot-free web.

If you have any questions at all, feel free to reach out to us at [email protected].

The purpose of this article is to go over the browser requirements for sending cookies in XMLHttpRequests (XHR) or Fetch requests. These types of requests may also be known more broadly as AJAX requests. It is important to review that browsersrequire certain attributes to be set with XHR or fetch objects to send cookies.

If the browser loads the main document from one domain, and then submits AJAX calls to another domain, then you will need to refer to browser guidelines around CORS. For example, if you the main browser document is loaded from www.foo.com and AJAX calls are sent to api.foo.com, then you will need to refer to the CORS documentation.If the browser loads the main document from www.foo.com and the AJAX calls are made to the same domain, but a different path like www.foo.com/api/v1, then it is most likely not necessary to follow the CORS guidelines. Below is a link to the CORS documentation.

https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

Below are links and examples from the links on the most basic ways to send cookies in a given HTTP request.

https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/withCredentials

var xhr = new XMLHttpRequest();

xhr.open('GET', 'http://example.com/', true);

xhr.withCredentials = true;

xhr.send(null);

https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch

fetch('https://example.com', {

credentials: 'include'

});

By editing a domain’s default settings, you can configure automated responses to thwart attacks against your entire site and all of its content. You can also tailor specific settings for individual paths.

To access default content protection settings for a domain:

Log in to the Distil Portal.

Select a domain from your Domains dashboard.

Click Settings on the banner menu.

Click Edit Default Settings in the Content Protection section.

Content protection settings are organized by tabs, including:

Automated Threats Policy Known violators, known violator data centers, identities, aggregator user agents, and automated browsers.

JavaScript Injection Configuration JS delay, threshold, location, prefix, and generated encoding.

Machine Learning Policy Estimated graph and threshold slider.

Rate Limiting Policy Pages per minute, pages per session, and session length.

You can activate multiple threat responses for Distil to use in automatically mitigating threats.

NOTE: All of these settings default to monitor-only mode for new customers.

How does the SDK impact jailbroken devices?

Jailbroken devices are not necessarily bad actors. We indicate a jailbroken device in our logs, but we do not block them from accessing API servers.

How is the Distil SDK actually deployed and what do I need to do from my side?

The SDK uses the existing Distil platform and is deployed in our existing deployment options. You only need an instance of Distil protecting your domain, in addition to integration of the Distil SDK library with your app.

How is the token requested?

Token requests are completely transparent and are handled by the Distil SDK.

What happens if the SDK fails to receive the token? What if the SDK fails to retrieve the token?

The SDK uses the same channel as the mobile app. As long as the Distil edge node is online, it will retrieve the token. If the edge node goes offline, all traffic is sent directly to the customer’s origin server.

Is the edge node location embedded in the SDK? Or is it something that can be configurable?

The edge node is not embedded. The SDK sends a request under your URL, which is captured by our edge node.

How do you distribute your SDK releases?

We distribute the SDK on request. Simply ask your AE for your unique Distil SDK version.

What is the estimated latency?

The SDK requests a new token every five (5) minutes to prevent each API request from requesting a token. Therefore, there is no latencyunless an API request happens when a new token is required. The entire challenge<>response and token request process takes well below one (1) second.

Does the SDK check IP addresses for VPN?

We check traffic against our list of known threats. Known threats include a mix of known violators, data centers, identities, aggregator user agents, and automated browsers. For example, if we have detected a known violator on another site, your own site is automatically protected from that threat. VPN exit nodes usually come from data centers, so we would detect those with this check.

Does the SDK work when a user clicks a certain link, or can it run in "stealth" mode all the time?

The SDK is in constant operation. Every few minutes, the SDK automatically does a full check of the device it’s running on and reports back to the Distil instance on the threats it’s detected (emulators, device farms, jailbroken devices, etc). We then track violations by issuing a temporary token, which is included on each request back to origin for each API call.

What is the size of the Distil SDK?

The Distil SDK for Android is 322kB. The Distil SDK for iOS is 379 kB.

Integrating the Distil SDK is a simple four-step process:

Request the SDK

Install the SDK

Test the SDK

Release Your App

1. Request the SDK

Request the Distil SDK from your account executive.You’ll receive a .ZIP file that includes:

Detailed instructions regardinghow to integrate the SDK into your application

The library to use within your application

An example application to see code examples on how to integrate the SDK library

The SDK API documentation describing each function that should be used

2. Install the SDK

After receiving the Distil SDK, follow the detailed instructions regarding how to integrate it with your application project. The instructions are specific to your account and the OS (Android, iOS) of the app you’re developing.

You must also add the delivered binaries of the Distil SDK as a library in your application. Once initialized, the library should be called for every HTTP request to your protected API.

SDK Interface

Once the SDK library is included with the app, you must ensure that:

The SDK init() function is called at application startup.

A token returned from the getToken() function is added as a header toeach HTTP request made to the API endpoint.

3. Test the SDK

After installation of the Distil SDK, test your application using your normal testing scenarios. Additionally, you should:

Ensure all calls from your mobile app to protected API have a token appended.

Test your app without a Distil server.

Test your app when a Distil server is blocking calls. This requires a test domain protected by Distil.

4. Release Your App

With the Distil SDK now fully integrated and all functionality tested, release your app by publishing it to its respective app store.

Distil provides your application with the token through the getToken call. Your application must always perform the getToken call immediately before any request to the API and the returned token should be appended as an HTTP header. Your application should never store the token, since when the token expires the SDK will transparently perform the authentication steps again upon the next getToken call.

Handle Error States

The getToken method can fail to fetch a token due to various reasons such as lack of network connectivity, internal errors etc. The application must handle these errors with a similar strategy as for the connections to the API server. For example, the application shows a dialog to the user requesting them to check their connectivity status.

For non-network related errors, the application should continue as normal, without a token. This ensures proper function in the case of the Distil server is bypassed for some reason. We also recommend using a circuit-breaker that backs off from calling getToken when in prolonged error state.

Response Codes

When your application makes API calls with the token added as a header, the Distil server responds with either a 200 (OK) or a 456 (BLOCKED) code. The 200 response means the request is allowed by Distil. Your API server sends back the payload to your application. The 456 response means the request triggered a treat violation and the security setting configured in the Distil Portal is set to drop the request for the specific threat violation.

Violation Types

The table below lists the possible violations you can customize behavior for in the Distil Portal.

Violation

Description

No Distil Identifier

The request did not have a token added as a HTTP header.

The Distil Portal reports this threat as "Missing Unique ID".

Bad Client

The application issuing the request was executed under suspicious circumstances. This typically occurs when the request is made from an application simulator/emulator or an automation tool.

The Distil Portal reports this threat as "Bad Client Automation Tools".

Invalid or Expired Token

The request’s authentication token is invalid or expired. This typically occurs when a request fails the SDK challenge or fails the token integrity check.

The Distil Portal reports invalid token violations as "Invalid Identifier Test Results" and reports expired tokens as "Identifier Tampering or Expiration".

Single page application are web pages that are populated with content that is returned via AJAX or Fetch. Using a captcha with single page applications with Distil can be a challenge because the content that is returned in the AXAJ or Fetch calls is typically JSON and Distil returns a text/html page for the captcha or Drop pages. The client side browser application will then return an error in this situation instead of render the captcha page. This article is intended to provide guidance for rendering a captcha page in the browser when a single page application is used.

Below are the high level modification.

Customize the HTML on the single page application

Use a Custom CAPTCHA page

Customize the HTML on the Single Page Application

The single page application needs to be customized to display the Distil CAPTCHA page on top of the main browser canvas to avoid disrupting the existing user session and to allow the user to solve the CAPTCHA. Below is the full code snippet for the customized single page app page.

<!DOCTYPE html>

<head>

<style type="text/css">

iframe {

position: absolute;

width: 100\%;

height: 100\%;

top: 0px;

left: 0px;

}

</style>

<script type="text/javascript">

window.addEventListener('message', function(text){

try {

console.log('Got message');

console.log(text);

document.getElementById('captchaFrame').style.visibility = 'hidden';

} catch (e){

console.log(e);

}

}, false);

window.onload = function(){

document.getElementById('button').addEventListener('click', function(){

var xhr = new XMLHttpRequest();

xhr.open('GET', "http://api.badbotjail.com/ajax_info.txt", true);

xhr.withCredentials = true;

xhr.onreadystatechange = function(){

console.log(xhr.readyState + ' ' + xhr.status)

if ((xhr.readyState == 4) && (xhr.status == 456)){

alert("Received a 456 from origin. Show CAPTCHA form");

if (document.getElementById('captchaFrame')){

document.getElementById('captchaFrame').parentNode.removeChild(document.getElementById('captchaFrame'))

}

var iframe = document.createElement('IFRAME');

iframe.id = 'captchaFrame'

iframe.src = "http://api.badbotjail.com/distil_r_captcha.html"

document.body.appendChild(iframe)

} else if (xhr.readyState == 4){

alert('Response: ' + xhr.status)

}

}

xhr.send("p=abcdefg");

})

}

</script>

</head>

<body>

<input type="submit" id="button" name="Click me" value="Click me" style="width: 400px; height: 100px">

<br><br>

<input type="text" name="input_headers"/>

</body>

</html>

Use a Custom Captcha page

The customized captcha page has a before unload event listener. This event listener will sent a message to the parent page which will then tell the parent page to hide the iFrame containing the captcha page. Below is the full code snippet for the customized captcha page.

<!DOCTYPE html>

<html>

<head>

<title>Pardon Our Interruption</title>

<link rel="stylesheet" type="text/css" href="//cdn.distilnetworks.com/css/distil.css" media="all">

<meta http-equiv="content-type" content="text/html; charset=UTF-8" />

<meta name="viewport" content="width=1000" />

<META NAME="robots" CONTENT="noindex, nofollow">

<meta http-equiv="cache-control" content="max-age=0" />

<meta http-equiv="cache-control" content="no-cache" />

<meta http-equiv="expires" content="0" />

<meta http-equiv="expires" content="Tue, 01 Jan 1980 1:00:00 GMT" />

<meta http-equiv="pragma" content="no-cache" />

<script type="text/javascript">

window.addEventListener('beforeunload', function(){

window.parent.postMessage('Hello!!!', '*')

}, false)

</script>

</head>

<body class='block-page'>

<div class='container'>

<div class='row'>

<div class='sidebar col-lg-4 col-sm-5'>

<img src="//cdn.distilnetworks.com/images/anomaly-detected.png" alt="0">

</div>

<div class='content col-lg-8 col-sm-7'>

<h1>Pardon Our Interruption...</h1>

<p>

As you were browsing <strong><!-- DISTIL HTTP HOST --></strong> something about your browser made us think you were a bot. There are a few reasons this might happen:

</p>

<ul>

<li>You're a power user moving through this website with super-human speed.</li>

<li>You've disabled JavaScript in your web browser.</li>

<li>A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this <a title='Third party browser plugins that block javascript' href='http://ds.tl/help-third-party-plugins' target='_blank'>support article</a>.</li>

</ul>

<p>After completing the CAPTCHA below, you will immediately regain access to <!-- DISTIL HTTP HOST -->.</p>

<!-- DISTIL CAPTCHA FORM -->

</div>

</div>

</div>

</body>

</html>

Once you’ve created paths, arrange their priority. Paths having a higher priority (closer to 1) take precedence over those with a lower priority.

There are two ways to set path priority:

Drag and Drop Hover over a path table entry to enable the row selector (). Drag and drop the row up or down to position it.

Edit Priorities Click Edit Priorities on the path table to manually edit the Priority fields. Change the priority of one or multiple paths at a time, and then clickSave.

This is an error page that Distil displays to client browsers when we are having issues serving the current request.

What is this page showing me?

There was an error servicing the request,

Information about the request thatfailed, and

Tracking numbers necessary forDistil's Support Team to investigate the root cause of the error.

What is the most likely cause of this error?

In almost all cases, a 400 error is related to an issue with the request being made. Issues include headers or cookies that or too large for the server to parse, as well as request URLs that are longer than a server can parse.

Things you should try before contacting Distil

Try clearing any cookies in your browser for the requested domain. Over time, cookie size can grow when visiting websites, causing issues in some cases.

What do I need to send to Distil Support?

When submitting a support request to Distil to investigate this error, it is important to provide the trace information from the error page. Please don't send a screenshot. Rather, copy and paste the trace information. When copied and pasted, the text will look like this (from item 3above):

You reached this page when attempting to access http://www.example.com/ from 192.168.0.5 on 2015-12-25 01:23:45 GMT. Trace: BB1EDE86-8FA5-11E4-B9B7-AC009C5BE403 (400) via b3a86554-de04-4db6-bf5a-ae890e529127; .454 / 673

Access to Dynamic Reporting Engine (DRE) in the Distil Portal is based on a user’s role. An admin user must grant access to DRE via the Account Management Settings page.

To grant access to a user:

Log in to the Distil Portal.

Click the down arrow to the right of your login name, and then click Account Management Settings.

Select a user from the User Management box.

Select the applicable Dynamic Reporting Engine Permission, such as:

Admin User is able to create, view, and edit reports.

Stats ONLY User is only able to view reports but cannot edit reports.

No Access User is not able to access the reports.

Click Update User to save the user permission.

This article walks through a typical investigation of a suspicious visitor which triggered a Captcha page.

Upon detecting the threat, Distil serves a Captcha page to the visitor.

Beyond the Turing test, each Captcha page includes the IP address and trace information associated with the request.

As an analyst, you can extract the information and then explore it using Dynamic Reporting Engine (DRE):

IP Address: 71.94.45.130

Trace: 6ce2681d-e5a8-4469-9f16-f69e30ff930e

Via: 809d12b6-952f-41fc-abe9-8f1075cba0cb

Let’s take that request information and access the Request Investigation dashboard.

Expand the filters list and paste the trace value (6ce2681d-e5a8-4469-9f16-f69e30ff930e) in the Request ID field. Then, click Run.

We can view the threat category, the Captcha results, the response Distil took, and the complete rundown of forensic data associated with the malicious attempting all in a single view.

Next, we can filter by the request’s IP address (71.94.45.130) to drill down into any other past appearances of the abusive source in your web, API, and mobile traffic.

We can see our original Captcha’ed request was part of a series of malicious automated requests.

Click the settings icon to access management options, including:

Save the explore as a Look

Save the explore to a Dashboard

CAUTION: Saving the explore to the dashboard you are working in will not alter the chart you explored from. Instead, it will add an additional tile for the explore.

Merge Results

Remove Fields and Filters

Clear Cache and Refresh

NOTE: This option ensures you are viewing the latest data. By default, most results are cached for 1 hour.

Zuid

The Traffic Overview dashboard provides an in-depth look into your request data. The dashboard contains eight mini-reports, from which you can explore data even further.

Accessing the Traffic Overview Dashboard

To access the Traffic Overview dashboard:

Log in to the Distil Portal.

Click Dynamic Reporting Engine on the top banner menu.

NOTE: You can click Dynamic Reporting Engine on the banner menu to return to the Traffic Overview dashboard from any other page.

Reviewing the Traffic Overview Dashboard

Use the top date range menu to adjust and isolate the data for a specific period of time.

By default, each dashboard displays data in UTC time. Change the time zone using the Time Zone dropdown selector.

Traffic Overview

The Traffic Overview tile shows the total request volume for a selected period, segmented by category. Each request is identified as either a human, a bad bot, a good bot (search engines, such as Google, Bing, and Yahoo!, as well as social media, such as Facebook, LinkedIn, Twitter), or a bad bot you have added to your whitelist.

Traffic by Category (\%)

Similar to the Traffic Overview tile, the Traffic by Category (\%) tile shows a percentage breakdown of the total request volume for a selected period, segmented by category. Each request is identified as either a human, a bad bot, a good bot (search engines, such as Google, Bing, and Yahoo!, as well as social media, such as Facebook, LinkedIn, Twitter), or a bad bot you have added to your whitelist.

Threat Traffic Overview

The Threat Traffic Overview tile shows categorized breakdown of the ways Distil detected bad bots threatening your site.

Bad Bot Traffic by Country

The Bad Bot Traffic by Country tile provides a breakdown of countries having the highest number of threats against your site.

Traffic by Distil Action

The Traffic by Distil Action tile displays the various ways Distil Networks has trapped threats to your site and its content from incoming bad bots. Using this tile data, you can get a high-level overview of the most active traps.

CAPTCHA Overview

The CATPTCHA Overview tile gives you an overview of CAPTCHA activity related to your site, including the number of times CAPTCHA pages were served, attempted, solved, and failed. This tile helps you to determine how effective CAPTCHA pages are as litmus tests in relation to letting human users in and blocking out malicious bots.

Top 1000 Bad Bot Details

The Top 1000 Bad Bot Details tile provides a dynamic picture of unwanted traffic targeting your site. It’s comprised of a table that lists the IP address, IP organization, user agent, request URL, and total number of bad bots.

Top 100 Malicious Fingerprints

The Top 100 Malicious Fingerprints tile shows a list of hi-def fingerprints () and the total number of associated bad bots.

Filters Panel

The Explore tool provides a dynamic data console for running new or ad hoc queries. Use it to build a suite of dashboards for your common, recurring queries or as a starting place for investigations.

Accessing near-real time, log-level data, the Explorer tool allows you to filter, group, and measure data while leveraging more than 100 dimensions.

Convert the results into a visualization that can be saved to a dashboard, downloaded, shared with others, and more.

Accessing Explore

The Explore window is accessed from the dropdown within DRE.

Alternatively, you can launch the Explore window directly from within a dashboard. Investigate a specific data set in a dashboard, then click Explore from Here.

Fields Panel

The Fields panel displays the fields available within each log. Fields are comprised of:

Dimensions (specific log fields, such as counts, sums, avg, etc)

Measures (aggregates based on the logs)

You can use the search field or scroll through the list to find the desired dimension or measure. The explore window has four main components:

Fields Panel

Results Panel

Visualization Panel

Dimensions and Measures are grouped together into similar buckets.

You can also create custom dimensions, custom measures, and table calculations.

You can use fields as columns, pivoted rows, or filters.

Additionally, you can select a column to show up in the results by clicking the field name. Once selected, the field name will be highlighted.

Hovering over a field allows you to select if for Pivot or as a Filter.

Filters Panel

View the filters that are currently expected in the report and edit the operators, leveraging common operators (=,<>,in,not null ect).

Add, duplicate, or remove filters.

Additionally, create a custom filter using more complex search logic.

Visualization Panel

Expand the Visualization panel to view a variety of data graphics based on the data shown in the Results table.

You can select from various visualization types like table view, histogram, pie chart, line chart, or single number displays.

NOTE: Each visualization requires results in a specific format. If the search result data is not in the proper format, then the visualization will not load. Rather, it shows the correct format needed to load the visualization. For example, ...

Once you’ve selected the correct visualization, edit the visualization options to tailor labels, axis, color, and more.

Results Panel

Once you have configured the fields, click Run to initiate the search. The Results Table displays the first 500 rows of results, with the option to raise or lower the row limit.

Reorder the columns by dragging or dropping them. You can reorder the results by clicking on the column header.

Additionally, view the raw SQL query used to pull the report data.

You can tailor the data shown by your dashboards by adding or editing filters. Filters can be either values like numbers, strings or dates that are mapped to specific fields. Additionally, you can choose explicit fields to filter on.

Filters can be applied to all or some of the tiles.

Filters can also be configured to apply to other filters.

To edit a dashboard filter:

Log in to the Distil Portal.

Click Dynamic Reporting Engine on the top banner menu.

Select a dashboard for the list of available dashboards.

Configure the filters according to your business’ needs using numbers, strings, or dates in the applicable fields.

Configure the filters according to your business’ needs using numbers, strings, or dates in the applicable fields.

Add multiple values to a filter by clicking the add (“+”) icon, and remove them by clicking the remove (“X”) icon.

When you have editing all desired filters, click RUN to apply the filters and regenerate the dashboard.

Dynamic Reporting Engine (DRE) provides a useful way to drill down and explore your web, API, and mobile traffic, including:

Dashboards Filter, generate visualizations, and share custom data sets to dig deep into your request data. Leverage Distil-managed out-of-the-box reports and tailor unique dashboards for your organization.

Explore Sift through near real-time request data and leverage more than 100 various fields to view and filter on, then save filter sets and visualizations as custom dashboards.

grant access to DRE

Accessing DRE

Access to DRE in the Distil Portal is based on a user’s role. An admin user must via the Account Management Settings page.

To access DRE

Log in to the Distil Portal.

Click Dynamic Reporting Engine on the top banner menu.

Combining a mix of filtered data sets, charts, and other visualizations, Dashboards provide versatile views into your request traffic. DRE provides Distil-managed and maintained dashboards, as well as the ability to create custom dashboards specific to your organization.

Dashboards are automatically updated and refreshed to ensure you are investigating with the most up-to-date data.

explore the data

Tiles

Dashboards are built using tiles. Tiles can be either static content (such as text and headings) or dynamic charts.

Charts

Charts are the visualization output of a saved Explore. Additionally you can open a dashboard chart in an Explore window to investigate the data even further.

Looks

Accessing DRE Dashboards

To access DRE dashboards:

Log in to the Distil Portal.

Click Dynamic Reporting Engine on the top banner menu.

Navigating Dashboards

View the dashboards you have access to by clicking the Dashboards dropdown menu.

NOTE: Dashboards are organized by Global, Account, and then Domain level access.

Hover your cursor over individual data points to view additional details and values.

Click an individual data point to drill deeper into the information behind each one. Drill downs offer a customized, granular view per field to provide relevant and useful information.

While viewing a drill down, click Explore from Here to . Sift through near real-time request data and leverage more than 100 various filters.

The Explore window displays the query, the filters applied on the dashboard, and a variety of charts and other visualizations.

Refine the results by adding more filters, changing the fields, and editing the visualization.

Once you have completed an investigation, you can:

Save the explore as a Look

Save the explore to a Dashboard

CAUTION: Saving the explore to the dashboard you are working in will not alter the chart you explored from. Instead, it will add an additional tile for the explore.

Merge Results

Remove Fields and Filters

Clear Cache and Refresh

NOTE: This option ensures you are viewing the latest data. By default, most results are cached for 1 hour.

This updated guide now contains two sections: one for the Distil Connector deployment type and another for the Distil Reverse Proxy / Cloud / Appliance deployment type. If you are unsure of which section applies to your deployment, please double check with your account executive or solution architect.

Billable Page Requests for Connector

Distil counts each request sent by your Connector integration to the Distil Analysis API, including static assets. By default, the Distil-provided Connector reference implementations pass all requests and all HTTP methods to the Distil analysis API.

Should you wish to ignore specific content, simply do not send that content through the integration (e.g. do not on-board a static asset subdomain). You may also modify the reference integration to not send specific file types to the analysis API.

Please see the "Estimating Your Billable Requests" section below for help in estimating your traffic volumes. Note that filtering out the static content only applies if you plan to modify your integration to not send static content to us for analysis.

Billable Page Requests for Reverse Proxy / Cloud / Appliance

Distil counts all HTTP page requests as GET or POST requests made to a web server's content pages and AJAX endpoints. Essentially, they’re all requests made to your web servers for files that are not an image, JavaScript, CSS, or other static content.

Distil defines page requests differently than most client-side analytics libraries, including Google Analytics and Omniture. Therefore it isn’t uncommon for the total number of page requests to be different from that reported by your analytics tool.

File Types

File types having the following file extensions are included as page requests:

.htm, .html, .shtml, .xhtml

.asp, .aspx

.php

.pl

.cgi

.cfm

.do

.jsp

/ (pretty URLs)

Static file types having the following file extensions are not included as billable page requests:

.bmp

.class

.css

.csv

.doc

.docx

.ejs

.eot

.eps

.gif

.ico

.jar

.jpg

.jpeg

.js

.mid

.midi

.otf

.pdf

.pict

.pls

.png

.ppt

.pptx

.ps

.svg

.svgz

.swf

.tif

.tiff

.ttf

.webp

.woff

.woff2

.xls

.xlsx

Estimating Your Billable Requests

Using standard macOS and Linux commands, you can also use the following four steps to determine a correct estimate of your billable requests outside of the portal. You can use a similar technique for SIEM tools (e.g., Splunk, ELK stack).

1. Count the number of lines in your log file(s).

BASH example:

$ wc -l access_log

23998651 access_log

2. Count the number of lines in the log file that are not billable by Distil (refer to the static file types listed above).

BASH example:

$ awk -F"\"" '{ print $2 }' access_log | grep -E '\.jpg |\.js |\.css |\.jpeg |\.png |\.gif |\.pdf |\.axd ' | wc -l

10622213

NOTE: This example assumes the access_log file has a LogFormat with the request string in double quotes and no other quoted strings before it. For example:

66.249.66.186 - - [01/Jan/1970:01:01:01 -0500] "GET / HTTP/1.0" 200 96 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

3. Subtract the number of non-billable lines from the line total.

[Total lines in log file (step 1)] - [Non-billable lines (step 2)] = [line total (used in step 4, below)]

23998651 - 10622213 = 13376438

4. Extrapolate based on the time period for the inputs (Distil invoicing is based on billable requests per month).

For example, your access_log was for a one week period and came to 13,376,438 billable page requests (found in step 3). Multiply the weekly billable page requests (13,376,438) by 4.3 to extrapolate the number to one month results.

13,376,438 x 4.3 = 57,518,683

Distil Page Counter Tools

Distil has created the following scripts to help estimatethe volume of page requests for a subset of your log files. You can use these, along with your known variances, to extrapolate the data to determine likely monthly page request volume.

NOTE: These tools are provided without warranty and their output does not constitute an agreement between you and Distil on the billable volume of requests for your site or sites. Use of non-standard file extensions mayimpact the ability of this tool to accurately count your page requests.

Mac

Mac Page Counter

Windows

Windows Page Counter

Linux

Linux Page Counter

Usage

Download the file to a known location and change the permissions on the file to make it executable.

From the command line, run ./distil_log_analyzer.1.0.darwin.amd64 -h to show the help screen:

Examples

To count the page requests in one file:

./distil_log_analyzer.1.0.darwin.amd64 <filename>

To count the page requests in a directory of files:

./distil_log_analyzer.1.0.darwin.amd64 <directory>

Below is the current Distil Networks IP range:

192.225.208.0/20

103.226.68.0/22

45.250.72.0/22

185.154.244.0/22

52.220.156.244/32

13.228.137.235/32

13.251.73.82/32

13.229.14.41/32

13.251.119.111/32

13.251.134.71/32

18.139.215.232/32

18.139.251.125/32

18.139.75.84/32

3.0.154.191/32

User Settings

The Distil portal gives you the ability to view and manage your account detailsincluding your company’s contact, reporting, billing, and plan information in addition to your own user settings.

Click your username in the top-right corner of the banner menu and then select Account Management from the dropdown menu to access Account Management and pages.

When opening a ticket, there are3severity levels:

Severity 1: Critical Production outage affecting all or the majority of requests or major issue impacting service.

Severity 2: High Persistent or irregular issues affecting some requests; System usage is limited.

Severity 3: Normal Inquires about routine technical issues; information requests; configuration.

If your request does not match the proper severity level, our support staff reserves the right to upgrade or downgrade your ticket severity rating.

The tutorial video below covers the same content as this article.

The Upstream HTTP Errors report details the number of errors (4xx and 5xx) returned by your origin server to Distil. Use it to correlate events and identify problems at precise moments in time.

Upstream error responses either:

Return directly from your origin server,

Indicate no response was received from your origin, or

Indicate an invalid response was received from your origin server.

NOTE: This report is based on UTC time. Data is summarized by week, day, and hour.

Accessing the Upstream HTTP Errors Report

Follow these steps to access the Upstream HTTP Errors report:

Log in to the Distil Networks Portal to access the Domains dashboard.

Select a domain.

Click Reports on the banner menu.

Click Traffic Overview to expand the Reports dropdown menu.

Click Upstream HTTP Errors.

Reviewing the Upstream HTTP Errors Report

The Upstream HTTP Errors report includes:

Time Table An organized breakdown of HTTP error codes returned by your origin server to Distil.

Breakdown of Errors by HTTP Error Code Number of requests associated with each HTTP error code.

4xx and 5xx Errors by Day Total number of HTTP errors returned on a specific date.

Select a day from the Time Table to drill down even deeper, breaking out HTTP errors on a specific day hour by hour.

Clearing your entire cache or select cache items is a very simple task and can be done in multiple ways:

Log in to the Distil Portal.

Search for and select the parent Domain or Account you are configuring.

Select Sign Into Account.

Select the specific domain you are configuring.

Navigate to Settingsand selectClear Cache by URL.

From this area, you are able to either:

Clear AllCached Resources

Clear Specific Files

Clear AllCached Resources

If you click Clear All Cached Resources, then all of your cached items will be cleared instantly.

Clear Specified Files

Click Select Which Cached Resources to Clear to clear individual files.To clear a specific file, copy and paste the URL path of the item in the text box. To clearmultiple items, then separate them out by new lines. For example, to clear 1.jpg, 2.jpg, 3.jpg,enter the following list:

/images/1.jpg

/images/2.jpg

/images/3.jpg

Account Admins are able to copy settings from existing domains they manage when they create a new domain. This allows for quicker creation and assurance that a mistake isn't made.

Be aware that only some settings, the ones Distil calls the "Proxy Settings" are being copied over with this functionality. To see which fields are going to be copied over, check here.

How to Copy Settings Over:

Log in as an Account Admin.

Click on the Add Domain button.

Fill out the fields as you normally would and when you get to the end of the form, select the domain whose settings you would like to copy over from the dropdown.

Press the Add Domain button to finish creating the new domain.

Matt's Traceroute (aka My Trace Route or MTR) is an application that combines traceroute and ping. With MTR, you can send packets from your local machine to a remote server and observe both the routing and network performance between the machines. MTR is an invaluable network diagnostic tool, particularly when trying to isolate the source of issues like latency or connectivity issues. When submitting a ticket to Distil Support, you may be asked to provide an MTR.

Install and run MTR on Windows

Unlike traceroute, MTR is not installed on Windows by default. Luckily, installing and running MTR on Windows is very easy!

Download and run WinMTR

Type the domain in the appropriate input field and press Start

Once the 'Sent' column reaches at least 1000 (a few minutes), click 'Stop'

Click 'Copy Text to clipboard' and send the paste to the support thread

Install and run MTR on Linux

Open a terminal window

Install mtr (via apt, yum, or other depending on your platform)

Run

mtr example.com -c 1000 -r

Copy and paste the output to the support thread

Install and run MTR on Mac OS

There are a few different ways to get MTR installed on a Mac, but here is the simplest.

Download and install MTR

Open a terminal window

Run

mtr example.com -c 1000 -r

Copy and paste the output to the support thread.

Running your site through Distil can impact server-based analytics packages that gather information from scraping your web access logs. This is because as bad traffic is mitigated from reaching origin, your numbers will reflect human usage versus human-and-bot usage.

Caching

If you disable caching, Distil will pass all requests, based on your threat responses, back to the origin server. When caching is enabled, Distil will cache all images, by default, at our regional endpoints. In addition, we will cache any page with the extensions specified in your portal settings.

NOTE:If HTTP cache headers are explicitly defined, Distil's edge nodes will respect and override any Portal settings.

Headers and Cookies

The Distil nodes will pass back all associated headers and cookies related to the request. Please email [email protected] for more detailed descriptions of these objects.

Use these headers and cookies to:

Track bot clients and cookies.

Count how many bot devices share a common IP address.

Measure how much bot traffic (number and percent) originate from a particular IP address.

Identify users that are served CAPTCHA or Block pages.

Analyze all requests from a particular bot device or a specific IP address.

We will cease posting maintenance and incident notices on this page. We’ve launched a new status page located at status.distilnetworks.com. We encourage all customers to subscribe to the page in order to receive notifications regarding status of services and maintenance activity. We recommend adding your emergency and maintenance contact information for email and SMS notification in order to stay current on all Distil activity. Once subscribed, you will receive a text and/or email notification to confirm your subscription, along with a confirmation text/email thereafter.Emails will come from [email protected], so be sure to whitelist the domain.

If you have any questions, please don't hesitate to contact us [email protected].

Below is a list of the common types of reports Distil Support receives, organized by severity level. Severity levels are listed only as a guide. All cases are reviewed and assigned a true severity level based on the specifics of your report. Being as thorough as possible with your report will ensure that your case is given the correct severity level and priority.

If you have verified that Distil is causing your site to be down, the first thing to do is to move your site off of Distil.

Sev 1 - Global Impact

My site is down

Required: Domain, screenshot of the error, verification of site availability when bypassing Distil using hosts file testing to origin.

Users are seeing a Distil 504 or Distil 502 error on every page and for all users

See the special support pages for these Distil Error pages for more information.

My site is under a DDoS attack which is impacting site availability

Required: Domain, time of attack start, example IP, country of attack origin.

My site is not rendering properly on Distil

Required: Domain, exact steps to reproduce the issue.

Sev 2 - Localized Impact

Intermittent reports of seeing a Distil 504 or Distil 502 error

See the special support pages for these Distil Error pages for more information.

My site is Performing Slowly and I think it's caused by Distil

Required: Traceroute to your website

Optional: MTR to your website

Sev 3 - Usage and Configuration

A user has reported seeing a Block or CAPTCHA when accessing my site

Required: Domain, Source IP address, Trace information from Block Page.

I need help configuring integration between my CDN and Distil

See this support article for more information and if you still have questions, give us as much detail as possible.

I want to onboard a new domain to Distil and would like help

Onboarding help is available Monday through Friday during normal support hours.

My origin makes decisions based on IP or location of the client and that isn't working

See this support article for more information about Distil HTTP headers including X-Forwarded-For and if you still have questions, give us as much detail as possible about the problem you are having.

My website monitoring, SEO, or other tool is being blocked by Distil

Required: Domain, screenshot from SEO tool management console, specific details on any errors received, name of the bot that you believe should be whitelisted.

Real user monitoring indicates Distil is making my site slow

Required: Domain, name of RUM tool, waterfall graph from monitoring tool, detailed overview.

Ticket Submission Methods

There are three ways to get help from Distil Support:

Distil Portal

Email

Phone*

NOTE: Phone support is availablewith VIP and Gold SLAs.

Submitting Support Requests via the Distil Portal

The best way to submit support requests is via the Distil Portal. Doing so ensures that your request is properly associated to your account and Support SLA. To submit a support request in the Portal, select theHelpbutton in the bottom right of the browser window. The Help button opens a search box which is linked to Distil's knowledge base, allowing you to research existing support articles and guides. Enter a relevant phrase to see results in our knowledge base,or selectContact Usto submit a support request.

Submitting Support Requests via Email

You can submit support requests to Distil by sending an email to . It is important that you use the same email address that is associated with your Distil Portal login account in order to properly associate the case to your account and Support SLA.

Submitting Support Requests by Phone

Phone support is available for VIP and GoldSLA customers Monday-Friday, 4am to 8pm Eastern US Time, excluding holidays (consult your support agreement or contact your account executivefor phone contact information). VIP and GoldSLA customers should still submit urgent requests via the Distil Portal as described above.

Submitting Urgent Requests

For customers with an VIP and GoldSLA, Distil offers 24x7 Support for criticalissues (consult your Support SLA for definitions). In order to trigger an escalation between 8pm and 8am Eastern US Time, you must log in to the Distil Portal and submit your support request and mark the request Critical.

Migrating Your Website to Distil Using RackspaceDNS:

Rackspace provides an easy to use and understand DNS system. Migrating your Rackspace domain is a simple process that involves two steps:

Change your CNAME record for www.website.comto point to a CNAME provided by Distil

Change your A Record entry forwebsite.com to point to an IP provided by Distil

Editing yourwwwCNAME to point to Distil's Network

log in to the Rackspace Cloud Server Control Panel

Navigate to your Cloud Server overview page, by selecting Hosting

Click Cloud Servers.

Click on the link for your Cloud Server.

Click on the DNS tab at the top. This is where you will modify your DNS records.

Click on the domain in the Domain Management screen.

Click on the Add button to add our first record a new window should appear.

In the Name field enter www

In the Content field enter www.yourwebsite.com.distil.us

Rackspace recommends you set the TTL to 60 seconds.

Set the Type to CNAME.

Click OK

Editing your root domainyourwebsite.comA Recordto point to Distil's Network

log into the Rackspace Cloud Server Control Panel

Navigate to your Cloud Server overview page, by selecting Hosting

Click Cloud Servers.

Click on the link for your Cloud Server.

Click on the DNS tab at the top. This is where you will modify your DNS records.

Click on the domain in the Domain Management screen.

Click on the Add button to add our first record a new window should appear.

In the Name field be sure your base domain is entered

In the Content field enter the IP address given to you by Distil in either the Add Domain wizard in the portal or in your deployment email

Rackspace recommends you set the TTL to 60 seconds.

Set the Type to A.

Press the OK button to save your record.

If you need to reset your password to theDistil Networks Help Center, please follow this link: Password Reset

Comprehensive platform testing has been performed to ensure functionality using the following desktop web browsers:

Chrome

Safari

Firefox

Internet Explorer 11+

NOTE: The Distil portal also has limited support on mobile devices.

To protect your sites with Distil Networks, you must first add and configure your domains within our Portal. Adding them to the Portal lets you review detailed traffic information about your sites and manage their protection settings.

Are you managing a large volume of subdomains? Try using wildcards.

Are you adding a domain that uses HTTPS? Follow these steps to enable HTTPS with Distil.

Adding a Domain

The steps to add a domain in the Portal vary between cloud or on-premise deployments.

Cloud Deployments

To add a domain to an account by way of a cloud deployment:

Log in to the Distil Networks Portal to access the Domains Dashboard.

setting up a wildcard subdomain

Click Add Domain.

Enter your domain in the Hostname field.

Click Add Domain.

Distil automatically performs a DNS lookup to validate if the domain exists. The returned IP address is added as the IP/CNAME (destination) value to your domain configuration.

NOTE: You’ll receive an error if there isn’t a valid public DNS configuration for the domain you’re trying to add.

To complete this process, you’ll also need to update your DNS. Traffic is then routed through the Distil Content Protection Network.

For more information on updating your DNS, check out the Migration (DNS Changes) articles in our Help Center.

That’s it!

On-Premise Deployments

To add a domain to an account using an on-premise deployment:

Log in to the Distil Networks Portal to access the Domains Dashboard.

Click Add Domain.

Enter your Fully Qualified Domain Name and Origin Server (IP/CNAME) in the provided fields.

Select the Wildcard? checkbox if you are setting up a wildcard subdomain.

Click Add Domain.

To complete this process, you’ll also need to update your DNS. Traffic is then routed through the Distil Content Protection Network.

For more information on updating your DNS, check out the Migration (DNS Changes) articles in our Help Center.

Adding a Subdomain

If your site uses subdomains, you can track and manage the protection of your subdomains in the Distil Portal by one of two methods:

Add each subdomain as its own domain using the above steps (On-Premise Deployments). Subdomain traffic information and protection settings are individually tracked on your Domains dashboard.

Follow the steps above (On-Premise Deployments)to add each subdomain as a nested subdomain. Subdomain traffic information and protection settings are tracked and summarized within the parent domain on your Domains dashboard.

Cloud Deployments

To add a nested subdomain to an account by way of a cloud deployment:

Select the associated domain from the Domains Dashboard.

Click Settings on the top banner menu.

Click Add Subdomain within the Subdomains section (highlighted in red).

Enter your subdomain(e.g., yoursubdomain.domain.com) in the Hostname field.

Verify that the Origin Server IP/CNAME (auto-populated from the associated domain) is correct.

Click Add Subdomain.

Congratulations!

On-Premise Deployments

To add a nested subdomain to an account using an on-premise deployment:

Select the associated domain from the Domains Dashboard.

Click Settings on the top banner menu.

Click Add Subdomain within the Subdomains section (highlighted in red).

Enter your subdomain(e.g., yoursubdomain.domain.com) in the Hostname field.

Select the Wildcard? checkbox, if you’re .

Verify that the Origin Server IP/CNAME (auto-populated from the associated domain) is correct.

Click Add Subdomain.

All done!

The Billable Requests report shows the number of requests for all domains associated with your account that are billable by Distil.

Understanding your account’s usage of the requests allotted per the plan is a great way to:

Figure out if an overage charge is likely to occur

Figure out if the plan should be upgraded to avoid recurring overage charges

Distil’s billing report is a great way to stay up to date on the above topics.

To view the Billing Report,

Click on Account Management

Click on the “View Billable Requests” button

How to analyze the report:

Current Billing Period:

This section is always for the just the current billing period.

If your plan is monthly, this will show requests from the beginning of your plan (first of the month) until the current date.

If your plan is annual, this will show requests from the beginning of your plan until the current date.

If the Percentage Plan Usage is close to 100\%, you are likely to get an overage charge.

Billable Requests Usage Trend:

This section shows the last 12 months of requests.

If the plan is monthly, the Guideline Per Month line shows you the # of requests in the plan. If a month has a red bar over the Guideline line, it means that an overage charge was accrued that month.

If the plan is annual, the Guideline Per Month line shows you the # of requests you should get on average each month to not accrue an overage charge at the end of the annual plan. So if you see the red bar go above the Guideline line, it doesn’t necessarily mean an overage charge will occur, but it is likely if many months go above the Guideline.

Aside from letting Distil manage your certificate, we support existing SSL certificates for establishing HTTPS traffic. In this scenario, you securely upload your existing certificates by encrypting them using GPG and emailing them to Distil. The dedicated SSL certificate can be a standard certificate, an Extended Validation (EV) certificate, a wildcard certificate, or a Unified Communications Certificate (UCC).

To use your own established SSL certificates:

Speak to a Distil support engineer who will provide their GPG public key.

Encrypt your certificate using the public key.

Email the certificate to the support engineer who provided their public key. We will add the certificate on our side and confirm that it is working as expected, as well as confirm with the customer that the certificate looks good.

Direct your traffic to Distil.

Please note there may be additional charges for this option, please talk to your Account Manager at Distil for more information.