Rigor FAQs
Rigor's Frequently Asked Questions page is a central hub where its customers can always go to with their most common questions. These are the 128 most popular questions Rigor receives.
Frequently Asked Questions About Rigor
Benchmark checks compare the performance of many pages, typically our homepage compared to the homepages of our competitors.
To create a Benchmark Check:
Go to Benchmark > + New
Name the check, enter a company name, provide a site URL
Use the + Add competitor to add names and URLs for company sites to compare against the initial site URL
Create
Performance Metrics Glossary
Once created, the Benchmark check will update automatically once per day.
Benchmark checks provide the following metrics*:
Benchmark Score
A Rigor-generated score from 0-100 that relates performance relative to the competition. It's calculated based on performance across various metrics. Rank is based on this score. Higher is better.
Lighthouse Performance Score
This is based on the Google Lighthouse Performance score with mobile settings, which rates sites on a scale of 1 - 100. Learn more about it here.
First Byte Time
Time from the startof the first request until receivingthe first byte ofthe first non-redirect request. 3xx redirects willincrease this time.
DOM Load Time
Time until the document has loaded, andtheinitial markup has been parsed. This corresponds to the browser DOMContentLoaded event.
Onload Time
Time until the page has loaded. This corresponds to the browser load event.
Fully Loaded Time
Time until there is 1.5 seconds of network inactivity after onload, waiting up to a maximum of 5 seconds. If onload is never reached, it is the time from the start of the first request to the end of the last request to finish. The last request to finish isnot always the last request started.
Request Count
The number of elements on the page or the number of requests it takes to load the page. Lower is better.
Content Size
Total size (bytes) of all content loaded.
Start Render
Time until thefirst pixel of content is drawn.
Visually Complete
Time until all above-the-fold content has finished rendering.
Speed Index
A calculated metric that represents how quickly the page renders above-the-fold content.
*Please refer to our for more definitions
View ArticleRigor partners with leading infrastructure and connectivity providers to establish a global network of checkpoints. We can configure Rigor to monitor sites from any or all locations. Heres a complete list of Rigors current monitoring stations and their DNS addresses.
The list is available in the Rigor Monitoring app under Monitoring Network. It's also available in JSON, RSS, XML, or CSV format. You can also download a list of just theIPs.
All IP Addresses*DNS can be used for excluding our locations from analytics and whitelisting.
**Region Code can beused for specifying Check location(s) viafeatures such as the Bulk Import.
DNS*
Location
Region Code**
ap-jp-tokyo.services.rigor.com
Tokyo
ap-jp-tokyo
ap-kr-seoul.services.rigor.com
Seoul
ap-kr-seoul
ap-in-mumbai.services.rigor.com
Mumbai
ap-in-mumbai
ap-sg-singapore.services.rigor.com
Singapore
ap-sg-singapore
ap-au-sydney.services.rigor.com
Sydney
ap-au-sydney
ap-hk-hong-kong.services.rigor.com
Hong Kong
ap-hk-hong-kong
na-ca-montreal.services.rigor.com
Montreal
na-ca-montreal
na-us-texas.services.rigor.com
Texas
na-us-texas
eu-de-frankfurt.services.rigor.com
Frankfurt
eu-de-frankfurt
eu-ie-dublin.services.rigor.com
Dublin
eu-ie-dublin
eu-it-milan.services.rigor.com
Italy
eu-it-milan
eu-gb-london.services.rigor.com
London
eu-gb-london
eu-nl-amsterdam.services.rigor.com
The Netherlands
eu-nl-amsterdam
na-us-chicago.services.rigor.com
Chicago
na-us-chicago
sa-br-sao-paulo.services.rigor.com
So Paulo
sa-br-sao-paulo
na-us-virginia.services.rigor.com
N. Virginia
na-us-virginia
na-us-ohio.services.rigor.com
Ohio
na-us-ohio
na-us-california.services.rigor.com
N. California
na-us-california
na-us-oregon.services.rigor.com
Oregon
na-us-oregon
na-us-miami.services.rigor.com
Miami
na-us-miami
sa-cl-chile.services.rigor.com
Chile
sa-cl-chile
sa-ar-argentina.services.rigor.com
Argentina
sa-ar-argentina
na-mx-mexico-city.services.rigor.com
Mexico City
na-mx-mexico-city
eu-fr-paris.services.rigor.com
Paris
eu-fr-paris
na-ca-toronto.services.rigor.com
Toronto
na-ca-toronto
na-ca-quebec-city.services.rigor.com
Quebec City
na-ca-quebec-city
eu-se-stockholm.services.rigor.com
Stockholm, Sweden
eu-se-stockholm
na-us-illinois.services.rigor.com
Illinois
na-us-illinois
na-us-washington.services.rigor.com
Washington
na-us-washington
na-us-iowa.services.rigor.com
Iowa
na-us-iowa
ap-au-new-south-wales.services.rigor.com
New South Wales, Australia
ap-au-new-south-wales
ap-au-victoria.services.rigor.com
Victoria, Australia
ap-au-victoria
ap-jp-osaka.services.rigor.com
Osaka, Japan
ap-jp-osaka
ap-kr-busan.services.rigor.com
Busan, South Korea
ap-kr-busan
eu-gb-cardiff.services.rigor.com
Cardiff, Wales
eu-gb-cardiff
ap-in-chennai.services.rigor.com
Chennai, India
ap-in-chennai
ap-in-pune.services.rigor.com
Pune, India
ap-in-pune
me-ae-dubai.services.rigor.com
Dubai, United ArabEmirates
me-ae-dubai
ap-my-kuala-lumpur.services.rigor.com
Kuala Lumpur,Malaysia
ap-my-kuala-lumpur
ap-id-jakarta.services.rigor.com
Jakarta,Indonesia
ap-id-jakarta
af-za-johannesburg.services.rigor.com
Johannesburg, South Africa
af-za-johannesburg
ap-cn-shanghai.services.rigor.com
Shanghai, China
ap-cn-shanghai
ap-cn-beijing.services.rigor.com
Beijing, China
ap-cn-beijing
View ArticleScripted Tests are a crucial aspect of Rigors platform.They are made up of steps, with each step performing some action or navigation on a page. Scripted Tests can be used to measure a user flow, record business transactions across a page, and more. In order to write useful scripts it is important to understand how Rigor Steps, Actions, Wait Times and Navigations work.
The Steps
Rigors steps are comprised of a Name, Action and corresponding value or selector, Business Transaction, and settings. Each Script is made up of one or more steps and steps denote actions to be taken by the script. These actions are then recorded, executed, and completed or failed.
filmstrip
Each action performs a specific function in the step. For example: a Click action will click on whatever you direct it to and a Fill in Field action will direct the script to fill in the desired text in the field that is directed. Whichever action is selected for the the step there will be a corresponding selector, value, and value to point that step to.
In this example, this step is configured to perform a click action on a component with an id of 12345. When this step runs the recorder will look for the corresponding id and will make a click action.
If multiple steps are configured for a Real Browser Check, then Rigor will record each step into a that will show the progression of your steps just as you have scripted. However, your filmstrip may show a different story than you may want if you havent properly configured your script.
Waiting for Navigation
One key aspect for writing scripts properly is accounting for page navigation. Page navigation occurs when a step has resulted in the navigation from one page to another. This could be the result of clicking a link, executing some JavaScript code, explicitly navigating to a specific URL, or less frequently, filling in a text field, or choosing an option in a dropdown. Accounting for this page navigation is done at the step level and is as easy as checking a box.
Rigor has pre-configured the Wait for Navigation box to appear for certain actions that may result in page navigation, and has checked the box for steps that will certainly result in page navigation; such as a Go to URL step. If this checkbox is misconfigured, your script will not run properly, so it is important to carefully script each step taking page navigation into account.
When the Wait for Navigation checkbox is selected, Rigor will wait a short duration after each of these steps to see if navigation occurs. If this is selected for steps that are known to not cause navigation, your check will experience a short delay between that step and the next, unnecessarily. But by unselecting the Wait for Navigation checkbox, Rigor will not wait to detect page navigations, resulting in no delays between steps in your check.
It is important to note that if you expect a step to cause navigation, then this setting must be enabled. Otherwise Rigor cannot detect that navigation has occurred and the check will likely fail.
View ArticlePlease note that prior to March 30th, 2019, Rigor collected a metric called First Interactive Time. This has been replaced with Time to Interactive. We have updated the name and the algorithm to collect this metric for improved accuracy and to be consistent with tools like Google Lighthouse. You may see timing differences for Time to Interactive due to this update.
There are two metrics that Rigor offers that measure interactivity: Time to Interactive and First CPU Idle.
Time to Interactive
Time to Interactive indicates the amount of time until a page is first expected to be usable and will respond to user input quickly. It can be a great metric to determine when users can reasonably expect to interact with a page, but it is important to understand how Time to Interactive is calculated to know if it is the right metric to monitor for your application.
Unlike performance and paint timings, there is no browser-supported API for interactive timings, and there is no standard definition for when a page is considered "interactive". Rigor tested multiple approaches for monitoring interactivity and settled on a solution for Time to Interactive time that yields consistent results without affecting check timings.*
*While there are multiple definitions of the metrics from different tools, Rigors definition of Time to Interactive is consistent with Google Lighthouse.
Calculating TTI
To calculate Time to Interactive, Rigor creates a PerformanceObserver to collect long task entries that can block user interactivity while your page is loading. After the page load is complete, we analyze the long taskentries as well as network activity following the steps below:
Find interactive windows where there is nolong task that takes more than 50 ms* to complete, and no more than two successful in-flight non-GET requests. *Tasks shorter than 50 mswon'tbe perceived by users so they are ignored.
Starting at First Contentful Paint, begin searching for a 5 second interactive (quiet) window.
Time to Interactive is the end of the last long task before the interactive window.
Advanced settings on the check edit page
While Rigor's approach to calculate Time to Interactive works for most applications, you may want a measurement of interactivity that is customized for you. To do that, you can use the User Timing API to log an event when a specific element appears or when a certain resource has loaded, and Rigor will monitor it. User timings appear on run result pages and in the Performance KPIs report.
First CPU Idle
First CPU Idle indicates when a page is minimally interactive. It is defined as the start of the first interactive window or DOM Content Loaded, whichever is first. A page is considered minimally interactive when:
Most, but not all, UI elements on the screen are interactive.
The page responds, on average, to most user input in a reasonable amount of time.
This metric can stand on its own, but is also important because it is a factor in the Performance Score. The industry standards for what interactivity means is constantly evolving, but as it stands now, First CPU Idle has a more forgiving definition for interactivity when compared to Time to Interactive.
Calculating FCI
To calculate First CPU Idle, Rigor follows the same basic approach we use for Time to Interactive. Rigor creates a PerfromanceObserver to collectlong task entries that block user interactivity.
However, the interactive window for First CPU Idle is defined differently from Time to Interactive. Interactivity is determined by task clusters (periods of time where there may be many long tasks) which can be either heavy ( >= 250 ms in duration) or light (< 250 ms).
After the page load is complete we analyze these heavy and light task clusters following these steps:
Find interactive windows where there are no large groups of "densely packed" long tasks. Long tasks must be grouped in a "cluster" < 250 ms long and there must be at least 1 s of padding between these clusters.
Starting at First Meaningful Paint, find an interactive window that is 5 seconds* long
First CPU Idle is the start of this first interactive window.
*The required window size shrinks over time. It starts at 5000 ms and shrinks to 3000 ms 15 seconds after First Meaningful Paint.
FAQ
Why don't I see interactive timing data for my Firefox Real Browser check?
Currently Time to Interactive and First CPU Idle are only available for Chrome checks. You can update your check to use Chrome at any time via the .
Why do my interactive timings look different in Rigor than in another performance applications?
Our approach is consistent with other performance applications that report interactive timings, but there is no standard definition or method to calculate interactive timings, so implementation details can vary and lead to different results. Time to Interactive and First CPU Idle are also dependent on multiple other metrics (First Contentful and Meaningful Paint, DOM Load, etc), so any difference in those values can amplify differences in interactive timings.
Why don't I see interactive timing data for my Chrome Real Browser check?
Both Time to Interactive and First CPU Idle are dependent on other metrics. Time to Interactive requires First Contentful Paint and DOM Load, and First CPU Idle requires First Meaningful Paint and DOM Load. If any of those metrics are missing (e.g. if the page is blank and no Paint events occur), there will be no value for the corresponding interactive metric. If you see missing or null values for your interactive timings, make sure the required metrics are present.
View ArticleRigor features over 40 metrics to help you better understand your web performance data. These metrics will help find, fix, and prevent defects and will give you a greater insight into how to shape your web performance.
Performance Timings
Connection Timings
Resource and Error Counts
Content Sizes
First byte time
DNS time
Request count
Content size
DOM interactive time
TCP Connect time
HTML count
HTML size
First paint time
SSL time
Image count
Image size
First contentful paint time
Send time
JavaScript count
JavaScript size
First meaningful paint time
Wait time
CSS count
CSS size
Start render time
Receive time
Video count
Video size
DOM load time
Font count
Font size
DOM complete time
Other count
Other size
First CPU Idle
Client error count
Time to Interactive
Connection error count
Onload time
Server error count
Visually complete
Error count
Speed index
Fully loaded time
Other Metrics
Response Time
Uptime
Availability
Performance Timings
First ByteTime:Time from the startof the first request until receivingthe first byte ofthe first non-redirect request. 3xx redirects willincrease this time.
DOM InteractiveTime:Time until the DOM is fully loaded and processed.
First Paint Time:Time until the browser renders anything other than the default background.
First Contentful Paint Time:Time until the browser first renders any content.
First Meaningful Paint Time:Time until the biggest above-the-fold layout change has happened, and web fonts have loaded.
Start Render Time:Time until thefirst pixel of content is drawn.
DOM LoadTime:Time until the document has loaded, andtheinitial markup has been parsed. This corresponds to the browser DOMContentLoaded event.
DOM Complete Time: Time until the page and all of its subresources are ready.
First CPU Idle:Time until the page is minimally interactive and will respond to user input in a reasonable amount of time. Click here to learn more about how this metric is calculated.
Time to Interactive:Time until the page is first expected to be usable and will respond to user input quickly. Click here to learn more about how this metric is calculated.
Onload Time:Time until thepage has loaded. This corresponds to the browser load event.
Visually Complete Time: Time until all above-the-fold content has finished rendering.
Speed index: A calculated metric that represents how quickly the page renders above-the-fold content. Read more about how it works here.
Fully Loaded Time:Time until there is 1.5 seconds of network inactivity after onload, waiting up to a maximum of 5 seconds. If onload is never reached, it is the time from the start of the first request to the end of the last request to finish. The last request to finish isnot always the last request started.
Connection Timings
DNS Time:Time required to resolve a host name from the DNS server
TCP Connect Time: Time to create a TCP connection.
SSL Time:Time required for SSL/TLS negotiation.
Send Time:Time required to send HTTP data to the server.
Wait Time:Time from when a request is finished until the time the first byte of the response is received for the first request in a page.
Receive Time: Time required to read the entire response from the server.
Resource and Error Counts
Request count:Number of requests made.
HTML count:Number of requests for HTML documents.
Image count:Number of requests for images.
JavaScript count:Number of requests for JavaScript files.
CSS count:Number of requests for CSS files.
Video count:Number of requests for videos.
Font count:Number of requests for fonts.
Other count:Number of requests for all other resources that are not HTML, image, JavaScript, CSS, video, or font requests.
Client error count:Number of responses with a status code between 400 and 499.
Connection error count:Number of responses where the status code is 504 or 0 (a browser-aborted request).
Server error count:Number of responses where the status code is 500 or higher (excluding 504).
Error count:Total count of responses with status codes greater than or equal to 400.
Content Sizes
Content size:Total size (bytes) of all content loaded.
HTML size:Total size (bytes) of all HTML content loaded.
Image size:Total size (bytes) of all images loaded.
JavaScript size:Total size (bytes) of all JavaScript files loaded.
CSS size:Total size (bytes) of all CSS files loaded.
Video size:Total size (bytes) of all videos loaded.
Font size:Total size (bytes) of all fonts loaded.
Other size:Total size (bytes) of all other resources loaded.
Other Metrics
Response Time:The Response Time for a single page Real Browser Check is the same as the Load Time. If a Real Browser Check has multiple steps then the Response Time equals the sum of Load Times for each page accessed during the user flow. The Response Time for an Uptime Check is equal to the Server Time.
Uptime:For an Uptime Check, the Uptime metric represents the percentage uptime of an endpoint for a set time frame.
For Real Browser Checks the Uptime percentage represents the percentage of time that the check passed within a set time frame.
Availability:Percentage of total number of successful runs divided by the total amount of runs.
View ArticleTags are short, user-defined text strings that can be assigned to any performance test or snapshot in the Rigor Optimization application. Tags can be any string of length 80 characters or less and include a priority level of Low, Medium, or High.
Tags can help you prioritize your workflow and share results with your team. For example, your team could filter by high priority tags to address recent performance regressions and then dismiss or reclassify those tags with a lower priority once issues have been addressed.
Any Rigor Optimization user with a User permissions role can create and apply tags to organize noteworthy results or filter test and snapshot results to match your workflow.
How to Assign Tags
Tags can be assigned both at the Performance Test or Performance Snapshot level.
From a Test Results page or from a Performance Snapshot page, we can see a table of our performance snapshots. If we select the snapshot by clicking the checkbox on the left of the Test Name then we will see different icons pop up on the header.
Rigor Optimization API
Click the tag icon to open the Add Tag modal:
Tag names can include spaces or punctuation and must be less than 80 characters. As a best practice, choose a tag name thats concise, yet descriptive. For example, some good names for tags could be Site Redesign v2 or Introduced New Compression Bug.
When you create a tag, assign a priority of Low, Medium, or High. Tags are color-coded based on priority (Grey for Low, Yellow for Medium, and Red for High) which will help us easily identify tags by priority. We ma also filter tags by priority.
Click Add Tag to close the modal and apply the tags to selected defects.
We can assign multiple, different tags to the same tests or snapshots.
Note: Tags can be assigned at the Test or Snapshot level, but tags assigned at one level will not automatically be added to the other. If we assign a tag on the results for one test it will only be visible when were viewing the test and not when were viewing snapshots for that test. Similarly, if we set tags on a snapshot for a test these will only be visible when were viewing other snapshots.
How to Remove Tags
There are two ways to remove tags:
Individually: Click the x icon to remove any specific tag, or
Clear all tags: Use the Remove Tags button to remove all tags set on a specific test or snapshot.
Searching and Filtering by Tags
Both the Test Results and Snapshot Results views have searching and filtering options for tags. To search for items with a tag, simply type the tag name (or a substring thereof) in the search box of the toolbar to see results for all matching tags.
To filter tags by priority click the dropdown box.
Tagging and the API
The has full support for creating, removing and searching for tags both at the Performance Test and Snapshot level. Using tags with the API can be an effective way of automatically capturing specific milestones in your software development lifecycle, for example tagging a snapshot with a build number, or tagging performance tests against different testing environments.
View ArticleIgnore Known Defects, Filter Defects by Page and Customize Results with Powerful Optimization Features
Were happy to announce several new enhancements to the Optimization platform to help you tailor results to show only the findings that you care about most.
Mute Irrelevant Defects
We listened to Optimization users and learned that different teams care about taking action on different defects. Your team may know about an ongoing issue, and wish to hide that known defect from a report.
The new Mute command allows you to hide defects that you deem irrelevant or low priority. Muted defects will not display in any tables, charts, or reports in snapshot results. If priorities change you can use the Unmute option to restore hidden defects.
To Mute a defect:
Select the Defects tab for any snapshot
Check the defects you wish to hide, and
Use the Mute command from the toolbar.
Muted defects will then be hidden from this snapshot and from all future snapshots for any test using the selected Defect Check Policy that was used when creating this test.
Performance Knowledge Base
Filter Defects by Page
Optimization allows users to filter defects by the page that that is being affected. When using optimization scans for multiple pages you can now filter though all pages and view the defects that are caught on each page.*
* The pages in the filter dropdown are not named by the URL but by the <title> tag in the HTML document for the page.
Sometimes the same defects are caught on multiple pages, in this case Optimization will allow you to see all the relevant pages that it affects.
To Filter Defect Occurrences:
Select the Defect you want to view
Scroll dow to the 'Affected URL's' Table
Select the 'All Pages' dropdown next to the 'Filter URL' search box
Click on the page where you want to view the occurrences of the defect
Customize Defect Severity
Optimization ranks defect results by severity. This severity is defined by our algorithms, where more severe defects are generally those that offer the greatest optimization benefit for the least effort.
But we know that each website is different. You may decide any image optimization is critical, while optimizations to your CSS are less important.
With our new defect severity customizations, you can change the assigned severity for any detected defect, and those settings will persist to all future snapshots for all tests using that check policy.
To change the severity of a check:
Select the appropriate defect from the Defects view
Select the Severity option
Set a new Severity for the check
Any defects with modified severity will be tagged in your results with a reminder about the change, and you can easily undo any changes later by hitting Reset.
Change How Defects Flag with Configurable Defect Thresholds
Many of the Optimization defects alert only when a certain threshold is met. For example, we can alert when compression savings are larger than a certain number of bytes or when cache durations are longer than a certain number of seconds. Each check is different based on what is being examined.
Until now, alerting thresholds were pre-configured in the application based on the best practices weve encountered from experience and industry research. Again, we know that each website is different. Some website owners may care if even the smallest lossless image optimizations are possible, while others may only want to focus on the truly extreme cases.
To help with this, we now offer configurable Defect Alerting Thresholds. Defect Thresholds can be set at two levels: a general level across the entire Defect Check Policy, or a specific level for a specific Defect Check (for example, Content without HTTP Compression).
Default Policy Thresholds
To set at the defect check policy level, go to Settings, then select Defect Check Policies and edit the policy of interest. Remember, Defect Check Policies are assigned to a Performance Test when you create that test. Each test has exactly one policy, although the same policy can apply to multiple tests. (More information about Defect Check Policies ).
The default level is Standard, which is most similar to the behavior you are already used to. To see more defects, you can reset this value to Low. This is useful if you want to see as much detail as possible so you can optimize every last millisecond of speed from your site. If youdon'thave a lot of time to make performance optimizations, you may instead want to use the High level. High will only alert on the most severe defects, which reduces the noise but may cause you to miss some of the smaller optimizations.
Disclaimer: Changes made here will only apply to future snapshots. Existing snapshots, will not be updated.
Override Defect Check Thresholds
While configuring a broad policy level threshold is quick and easy, it may not give you the level of control you are looking for. Now Optimization gives us the ability to override thresholds at the Defect Check level.
Not all Defects have thresholds, for example a Resource Not Found error is either there or its not; theres no middle ground. To spot defects with thresholds, use the Has Threshold filter in the defects view or look for those defect rows with the Threshold icon displayed.
To customize a defect check:
Hit the Threshold icon on any defect row, and
Choose the appropriate new value.
A preview window will be displayed that shows the current threshold levels, and you can even supply a Custom level with your own specific values.
You can always review all the customizationsyou'vemade later by visiting your Defect Check Policy settings page, and you can undo any changesyou'vemade using the Reset command on and all checks.
Note that when you change a threshold, its possible some defects that alerted earlier will no longer display. For example, if an image can be optimized 200kb and your threshold changed from Savings > 100kb to Savings > 300kb, that opportunity for 200kb savings is no longer considered a defect.
Defects in this state are called unflagged. Unflagged defects are simply defects that were found by Optimization but that are not reported in your results due to your threshold configuration. Still, we think it is important that you be aware of those defects not reported (especially if you wanted to fix them later), so any view that hides defects due to the unflagged status will show a link on that view to the number of unflagged defects hidden from that view. If you follow that link, you can then choose to further customize the threshold level to a value that causes that defect to reappear in your results.
Improved and Updated Defect Checks
One of the Optimizations key advantages is the huge repository of performance defect checks that are analyzed, as showcased in our .
Having a large depth of rules to check for defects helps us identify and correct problems that other performance systems would never find, and we must also continuously focus to deliver relevant check results.
Because technologies and priorities change and evolve over time, we recently audited Optimizations repository of performance defect checks. We removed obsolete checks, consolidated a number of related checks, and added new checks (especially around video).
Now you will see new check rules and results in future snapshots. Some checks will be marked as Deprecated in historical snapshots.
Disclaimer: As a necessary result of the defect check improvements, you are likely to see different results when comparing performance snapshots made before this release to those after this release. The Optimization score and number of reported issues could be different. While we endeavor to keep results as consistent as possible over time, we hope you also understand the ongoing need to keep our results accurate and timely with the latest advances in website performance recommendations. All snapshots made after this release will show consistent and comparable results.
We hope you find these new customization features helpful in personalizing your Optimization reports to the exact level of detail you expect. Over time well continue to improve these features based on user suggestions.
Enjoy!
View ArticleMany corporate environments monitor and secure traffic to/from the internet by using proxy servers. Depending on your network configuration, running our agent in these environments may require some additional configuration steps, especially when using Docker Desktop since it uses an intermediary Linux virtual machine that may not automatically be configured to use your proxy servers and may not share certificates installed by network policies with Docker containers.
This will largely be an issue when your corporate proxy server inspects SSL/TLS traffic and uses a self-signed CA Certificate that is trusted within your network, as our agent will fail when verifying secure communications while connecting to Rigor's services.
Below we'll cover two distinct solutions: using a secondary proxy that does not inspect SSL/TLS traffic and injecting a custom CA Certificate into the Docker container.
Knowing exactly what configuration is most appropriate is for you to decide after consulting with your network administrator.
Configuring Docker to use a secondary proxy
In cases where the system-wide proxy configuration will inspect SSL/TLS traffic, but a secondary proxy server is available, you can provide the secondary proxy connection information by configuring the Docker Engine or by specifying proxy configuration only to the Rigor Agent by specifying environment variables. Docker has great documentation on how you can configure the Docker engine on their website: https://docs.docker.com/network/proxy/
This approach will require a secondary proxy server that is configured to allow SSL/TLS traffic to pass through it without being inspected.
Using a custom Certificate Authority
For cases when a secondary proxy server is not available and your proxy server has a custom CA Certificate distributed to workstations, it's possible to inject your certificate into our agent Docker container and add it to the trusted list of certificates.
While it's possible for you to create your own Docker container based upon Rigor's agent container, we don't recommend it as it will require some automated process to detect new versions of the Rigor agent, re-build the derived Docker image, distribute the image, and restart all applicable Docker containers.
Instead, we recommend that you follow these general steps:
Create a folder on your host machine and place the CA Certificate (in CRT format) in the folder
Mount that folder as volume to the container
Modify the command used when launching the container to update the CA Certificate cache before starting the agent software
Docker Compose Example
Here is a simple docker-compose.yml file example of starting the Rigor agent container with a mounted volume and registering the certificates on startup:
version:'3'services:agent:image:docker.rigor.com/agent:stablecommand:bash-c"sudoupdate-ca-certificates&&bundleexecshoryuken-r./config/boot_shoryuken.rb"environment:RUNNER_TOKEN: your_runner_token_hereDISABLE_NETWORK_SHAPING:"true"volumes:-./certs:/usr/local/share/ca-certificates/my_certs/
You can see that we included the default COMMAND instruction from the Rigor agent's Dockerfile and simply prepended the `sudoupdate-ca-certificates` terminal command.
Docker run command
To execute the same example using the `docker run` command, we would use the following:
dockerrun-eDISABLE_NETWORK_SHAPING=true-eRUNNER_TOKEN=your_runner_token_here--volume=`pwd`/certs:/usr/local/share/ca-certificates/my_certs/docker.rigor.com/agent:stablebash-c"sudoupdate-ca-certificates&&bundleexecshoryuken-r./config/boot_shoryuken.rb"
View ArticleWe can use the Optimization Connector to connect Monitoring to Optimization, a web app that provides actionable intelligence for improving web performance. Once connected, we can send Rigor Monitoring data to Optimization to receive step-by-step remediation plans for fixing performance defects.
Log in to Optimization and go to Settings
Real Browser check
Click API Credentials in the left menu
Copy the API key
In Monitoring, go to Admin > Optimization Connector
Paste the Optimization API key copied in step 3 in the Optimization API Key field
ClickSave
Analyzing Monitoring Data with Optimization
With the Optimization Connector set-up, we can now send Monitoring waterfall data to Optimization for analysis. To analyze data with Optimization:
Navigate to a
Click on an individual run
Click the 'Click Here to Analyze with Optimization' button
Clicking Analyze will send the waterfall data from all pages from Monitoring to Optimization and open the Optimization analysis. Heres an example of a completed Optimization analysis:
Once a waterfall has been analyzed, the Analyze button will change to View Optimization Analysis and link to the existing Optimization analysis for reference.
View ArticleDifferent types of users have different permissions within Rigor Monitoring and Optimization.
Rigor Monitoring
There are three types of User Roles:
Administrator: creates new users, has access to Admin panel
Manager: can create and edit checks and reports
Report Viewer: can view checks and reports
Report Viewer
Manager
Administrator
Dashboard
View Admin panel
x
x
View checks
Create checks
x
Edit checks
x
Pause checks
x
View reports
Create reports
x
Edit reports
x
Acknowledge alerts
x
View check information details
x
Rigor Optimization
There are three types of User Roles:
Client Admin: creates new users, has access to change account settings
User: can create and edit tests and defect check policies
Read Only: can view results and reports
Read Only
User
Client Admin
View and export results
Compare results
Create tests
x
Create snapshots
Whitelist first party content
Configure integrations
Create Defect Check Policies
x
API access
Manage tags
x
Create users
x
x
View account usage
View ArticleThe attached guide is your one-stop-shop for getting up and running in Rigor Monitoring. It walks through every type of check and introduces your Implementation Specialist and Client Success Manager. If you have any questions please don't hesitate to reach out to [email protected].
View ArticleThe Rigor Platform offers two products: Monitoring and Optimization. Both of these require that the user setup an account. Adding users to both Monitoring and Optimization is not complicated and is detailed below:
Adding Users in Monitoring
Users with Administrator roles have the ability to invite new users to Rigor. There is no limit to the number of users that can be added to a Rigor account.
To create a new user:
Admin > Users and Groups > New > +User
https://optimization.rigor.com/
Enter a name, email address, and select a user role for the new user:
You could optionally configure other settings such as the users phone number, timezone, and how they will receive alerts and reports from Rigor.
Click Create
After clicking Create, Rigor will send a welcome email with the subject: [Rigor] Please set your password to the new users email address, prompting them to set a password and log into the Rigor app.
Adding Users in Optimization
Just like with Monitoring users with Admin access have the ability to invite new users to Optimization. To add a user in Optimization:
Go toand click the three dots icon in the top right corner
Click 'Manage Users'
Then click 'New User'
Enter a name, email address, and select a permission level for the new user:
Click 'Create'
After clicking Create, Rigor will send a welcome email with the subject:[Rigor Optimization] Welcome to Rigor Optimization.
Please set your password to the new users email address, prompting them to set a password and log into the Rigor app.
Groups
After more than one user has been created, you can create a Group with several users. Groups are used to configure alerts to send to several emails at once, so it's best to create Groups based on users that should be alerted together.
To create a new group:
Admin > Users and Groups > New > +Group
View ArticleWe can use Alert Webhooks to notify an HTTP endpoint when a Real Browser check or Uptime check fails. This endpoint could be a third-party alerting service, a company chat application, or even a custom API. To use Alert Webhooks, we need to:
Create a new Alert Webhook
Edit a check to notify the webhook
We can select from the following pre-configured services:
Datadog
HipChat
OpsGenie
PagerDuty
Slack
ServiceNow
Edit a Rigor check Configure a webhook to send to a custom endpoint
Send alerts to Datadog
Create a new Alert Webhook under Admin > Alert Webhooks
Select the pre-built webhook for Datadog
Select the Triggers you want to activate. There are two types: When failed and When back online
Insert your Datadog API key into the Trigger endpoint URLs
OPTIONAL: If you want, you can customize the JSON data sent in the notification via the Payload section. This allows you to change the text and alert format to something that better suits your workflow.
Once you have saved and named your webhook, it can be added to any Rigor monitoring check via the check notifications page in the check edit menu. From that point forward, your Datadog events stream will receive a POST request with the specified JSON payload any time your check fails. Example failures would look like this:
Send alerts to HipChat
Log in to HipChat and go to Rooms
Select the room to receive Rigor alert notifications and click Integrations in the left menu
Click Create on the Build Your Own! option
Name the integration and click Create
Copy the integration URL and click Save
In Rigor, go to Admin > Alert Webhooks > + New and create a HipChat Alert Webhook
Paste the HipChat integration URL copied in step 5 in the Webhook URL field for each trigger
(optional) Customize the JSON data to be sent to HipChat
Save the webhook
Edit a Rigor check and set it to notify the webhook
With the webhook set-up, Rigor alerts will post notifications to HipChat:
Send alerts to OpsGenie
Log in to OpsGenie and go to Integrations
Find the Rigor integration and click Add (or click here )
Copy the API Key in the integration settings
In Rigor, go to Admin > Alert Webhooks > + New and create an OpsGenie Alert Webhook
Paste the OpsGenie API key copied in step 3 in the apiKey field in the Payload section for each trigger
(optional) Customize the JSON data to be sent to OpsGenie
Save the webhook
Edit a Rigor check and set it to notify the webhook
With the webhook set-up, Rigor alerts will trigger alerts in OpsGenie:
When the check comes back online, the associated OpsGenie alert will get closed:
For more, see the OpsGenie integration documentation.
Send alerts to PagerDuty
Log in to PagerDuty and go to Configuration > Services
Click Add Services, name the Service, and select Rigor as the integration type
Click Add Service
Copy the Service API Key of the new PagerDuty Service
In Rigor, go to Admin > Alert Webhooks > + New and create a PagerDuty Alert Webhook
Paste the PagerDuty Service API Key copied in step 4 in the service_key field in the Payload section for each trigger
(optional) Customize the JSON data to be sent to PagerDuty
Save the webhook
Edit a Rigor check and set it to notify the webhook
With the webhook set-up, Rigor alerts will trigger incidents in PagerDuty:
When the check comes back online, the associated PagerDuty incident will get resolved:
Send alerts to Slack**You need to have a paid, 'Standard' Slack plan to integrate as a webhook.
Log in to Slack and go to Integrations
Click the Incoming WebHooks option (or click here )
Select the Slack channel to receive alerts and click Add Incoming WebHooks Integration
Copy the Webhook URL for the new Incoming WebHook
In Rigor, go to Admin > Alert Webhooks > +New> [Slack]
Paste the Webhook URL copied in step 4 in the Webhook URL field for each trigger
(optional) Customize the JSON data to be sent to Slack
Save the webhook
Edit a Rigor check and set it to notify the webhook
With the webhook set-up, Rigor alerts will post notifications to Slack:
Send Alerts to ServiceNow
Create a new Alert Webhook under Admin > Alert Webhooks
Select the icon for ServiceNow
Provide the credentials for the ServiceNow user that will be used to create an incident. Click "Generate Header" button to generate the encoded authorization header that will be used with this webhook. Note: Make sure the ServiceNow user has the permissions to create, modify and close incidents.
Select the 'Triggers' you want to activate. There are two types: 'When failed' and 'When back online'. 'When failed' will open a new incident in ServiceNow when a check fails and 'When back online' will close that same incident when that check resumes functioning. Note: you should not use the 'When back online' trigger without also using a 'When failed' trigger.
Find the base URL of your ServiceNow instance. In the new webhook form, edit the URLs for each trigger's endpoint, replacing "<<YOUR_SERVICENOW_BASE_URL>>" with the base URL for your ServiceNow instance.
OPTIONAL: If you want, you can customize the JSON data sent in the notification via the 'Payload' section. This allows you to change the text and alert format to something that better suits your workflow. Note: When customizing a 'When back online' trigger, the state and close_code fields should not be changed.
Once you have saved and named your webhook, it can be added to any Rigor monitoring check via the check notifications page in the check edit menu. From that point forward, new ServiceNow incidents will be created with the specified details any time a check fails. In turn, the incident will be closed automatically when the check is back online.
Send alerts to a custom endpoint
In Rigor, go to Admin > Alert Webhooks > + New and create a Custom Alert Webhook
(optional) Add Request Headers
In the Webhook URL field for each trigger, provide the endpoint to receive Rigor alerts
(optional) Customize the JSON data to be sent in the webhook
Testthe webhook (check API endpoint to verify results)
Save the webhook
and set it to notify the webhook
With the webhook set-up, Rigor alerts will trigger POST requests to the specified endpoints.
View ArticleRigor Real Browser checks have a Changelog. The Changelog provides a detailed history of different changes made to checks. It includes:
Rigor system events (new browsers, check name change, viewports, network throttling etc.)
Custom System Events
Alerts
Notes/Comments
System Events
The Changelog tracks System Events, such as which user was last to run or edit a check. This helps us audit changes to check configurations.
System Events appear automatically whenever a user:
Creates a check
Edits a check
Changes check steps
Pauses or unpauses a check
Mutes or unmutes a check
Runs a check manually, or
Takes or releases ownership of an issue
Changelog Audit Page
Users can also click on the system event line or the View details link to be taken to the changelog page.
this blog post
This page gives a side-by-side view of the previous changes made to a check and the current settings. Users will be able to see changes made to steps, advanced configuration settings, and updates to basic settings, and alerts. This high level viewport into check configurations will help users revert changes and stay on track of their checks.
Alerts will also be in the changelog, however theywon'tappear in the Changelog Audit Page. Only the 15 most recent alerts will be displayed.
Adding Notes
We can also add Notes to help teams stay in sync about the status of issues.
For example, we might add a note to record a specific instance that triggered an alert.
To add a note from a check read screen:
Click the Changelog icon
Add your comment in the text field
Click the blue message bubble or hit enter
We can also edit or delete any of our own notes by hovering and clicking Edit or Delete(the X in the top right corner):
Read more about System Events in .
Note: The Changelog doesnt currently log changes to:
Tags
Configuration Templates
View ArticleRigors Changelog shows logged changes that we can reference for reporting on changes to performance before and after events.
Changes to System Events automatically update the Changelog. We can also make annotations to the Changelog in the Rigor monitoring apps UI or programmatically via the API using Control Groups.
Annotations via Control Groups allow us to add events to the Performance History and Changelog of Rigor Checks. These annotations can then serve as a record of what changes were made and when they occurred, which makes it easy to find the cause of performance improvements or regressions. Read more about reporting on performance before and after changes here.
How To Annotate Check History Via The API
First, navigate to Admin > Control Groups and create a group for the checks youd like to annotate.
On the Control Group read screen, find the endpoint and post token by clicking the Help button. Copy the API endpoint and post token for your Control Group.
Using cURL (or your favorite HTTP client), send a request to the API endpoint with the Control Group post token, the annotate command, and a message and title (optional). Rigor will automatically append a timestamp to the annotation.
Visit the Performance History or Changelog for any Check in your Control Group to view your annotation. Click the flag icon to view annotations in the Performance History graph.
View ArticleRigors Changelog shows logged changes that we can reference for reporting on changes to performance before and after events. We can use custom date range filters to view the Performance History of checks before and after changes.
Here are some things to consider when planning to analyze performance changes before and after a deploy or change to your site:
Know exactly when to expect a major change to a site? Schedule a blackout period. A blackout period helps keep checks from running and triggering alerts while changes are being pushed to a live site.
Changing something on the fly? Use a Control Group and the Rigor API to make an annotation on any checks that will be affected by the change.
Make any needed changes to Rigor check steps or configured settings for checks monitoring the site where changes will take place.
Decide how you will measure success.
Reporting On Change In Average Response Time
On 1 February 2016, we scheduled a change to move the Rigor Knowledge Base from our Help Desks CMS to a site that we manage ourselves and host on Github pages.
We had a pre-existing Rigor Real Browser check configured to monitor the performance of visiting help.rigor.com and clicking on the first article link named Rigor KB (homepage > article). The check did not run, capture data, or alert while we were making changes over a scheduled period of time.
We can use the custom date range view to see the performance of the check for a week before and after the change. The blue vertical line represents changes made to the check. We updated the check steps to use selectors that matched the HTML and CSS of our new site.
After we made changes to the site and the Rigor check configuration we resumed the check and began collecting data again.
Comparing one week before and one week after the change we can see that our changes reduced the average response time of the user flow from 3 seconds to about 750 ms.
The average response time of the check over a week was 3 seconds with the slowest response time equalling 30 seconds.
After changing to the new site the average response time of the check over a week was 752 ms (2.2 seconds faster) with the slowest response time equalling 3 seconds (27 seconds faster).
View ArticleAttached is an onboarding guide that we created for Administrators to fully set up their Rigor accounts. Please refer to this for any questions. If you can't find something you need, please don't hesitate to reach out to [email protected].
View ArticleWhen creating or editing an existing Real Browser check, we will find settings for locations and other customizations on the Advanced tab.
Default Settings
User Agent
Run checks with the default Firefox user agent string, select from a list of preset user agent strings, or use a custom string. Read more about how to use User Agent strings with the Rigor app here.
Viewport Size
this article
Run checks from a default browser size, select from a list of preset viewports for common devices, or set a custom viewport by height and width. Custom viewports are especially helpful for monitoring the performance of mobile or responsive web pages.
Optional Settings
Security
With this option checked, Rigor will fail a run if an SSL certificate is expired.
Connection
Select a network connection setting to simulate various types of connections, such as Mobile LTE or. New checks default to 'Cable' connections.
Response Time Monitor
By default Rigor will count a run as a failure if the response time of the run is greater than 60,000 ms. You may set a custom response time monitor lower than 60,000 ms to make alerting more aggressive. For more ideas about using response time monitors see this article.
Threshold Monitors
For multi-step checks that monitor several pages, you can set lower response time thresholds for specific pages.
DNS Overrides
You can add DNS override rules to reroute requests from one host to another. This is most helpful when testing an existing production site against page resources loaded from a development site or a specific CDN edge node.
Cookies
You can preload cookies in the browser before the run starts. Simply add the cookie name and the value into the fields. Any cookies that are set will apply to the domain of the starting URL of the check. We use the public suffix list to determine what the domain is.
Excluded Files
Specify files to avoid requesting when running the check. Select from preset options of commonly excluded files or set custom configurations to exclude specific resources. Read more about best practices for excluding files or using exclusions to keep Rigor from affecting your analytics.
Secure Basic Auth
Use this to enable Rigor to talk to sites with basic auth. The username and password is scoped to the domain of the start URL. Credentialswon'tbe shared with 3rd parties.
Custom Headers
Specify custom HTTP headers to send with every request.
JavaScript Files
Include custom JavaScript functions to reference in your steps. More about using custom JavaScript in checks here.
Tags
Optionally add tags to checks for an organization or to compare checks by tag on a dashboard. Read more about using tags in the Rigor app here.
Configuration Template
Select an existing configuration to use for this check. Read more about configuration templates in .
View ArticleHTTP Request Type
By default, Rigor checks run as GET requests. If youd like to change the request type you can select from the dropdown list.
this article
POST Data
If you selected the HTTP Request Type POST you can enter any POST data in this field. This field accepts posts data in XML/JSON format.
User Agent
Run checks with the default Firefox user agent string, select from a list of preset user agent strings, or use a custom string. Read more about how to use User Agent strings with the Rigor app here.
Secure Basic Auth
Use this to enable Rigor to talk to sites with basic auth. The username and password is scoped to the domain of the start URL. Credentialswon'tbe shared with 3rd parties.
Request Headers
Optionally modify the Request Headers for the check. This can be helpful to set a cookie or send over authorization credentials.
Connection
Select a network connection setting to simulate various types of connections, such as Mobile LTE or. New checks default to 'Cable' connections.
Success Criteria
Success Criteria are conditions for returning the status success. We can use settings here to customize Uptime checks to only return status of success if certain text is present or absent, if the server returns a specific HTTP response code, or if the results match a regular expression. If we want to test a redirect we can use the criteria Goes to URL and supply the URL where the check should redirect.
Response Time Monitor
By default Rigor will count a run as a failure if the response time of the run is greater than 60,000 ms. You may set a custom response time monitor lower than 60,000 ms to make alerting more aggressive. For more ideas about using response time monitors see this article.
Tags
Optionally add tags to checks for an organization or to compare checks by tag on a dashboard. Read more about using tags in the Rigor app here.
Configuration Template
Select an existing configuration to use for this check. Read more about configuration templates in .
View ArticleUsing Share Links Via UI
You can share links to specific pages in monitoring, allowing Rigor users to share pages with people that don't have Rigor accounts. This is a secure way to show the uptime as well as specific KPIs to key clients, team members, and others without having to add them to Rigor's platform.
monitoring-api.rigor.com
1. Navigate to the Real Browser Check homepage or KPI page.
2. Configure the datepicker, metrics, and view as you'd like the client to see!
3. Click the 'Share this Page' icon from the top right corner as depicted in the screenshot.
4. Click the 'Create a Share Link' that appears from the right-hand side panel.
5. Share the generated link to allow anyone to access a specific page. This panel can be used to create, manage, and track views. You can share the same page (i.e. KPI or homepage) as many times as you need!
Using Share Links Via API
1. Navigate to
2. Enter the API Key found in the Rigor Platform under Settings.
3. Once an API endpoint has been hit a share link will be automatically created.
View ArticleIn the Rigor Platform, we offer over 40 metrics to help you better understand your web performance. Along with these metrics we offer two scores: the Best Practices Score and The Performance Score, which directly is the Google Lighthouse Performance Score. This article will focus on the Performance Score and the way that Rigor calculates it.
The Performance Score, like Google Lighthouse, is a weighted average score that takes in 5 metrics: Time to Interactive, Speed Index, First Contentful Paint, First CPU Idle, and First Meaningful Paint and outputs a score from 0-100 that helps you understand your overall performance. It can be difficult to understand how to measure performance given so many metrics; the Performance score is intended to help in this area by combining the three important areas of web performance: speed, paint, and interactivity.
FAQ
Why is my Rigor Performance Score different than Lighthouse?
There is really no difference between the makeup of Rigor's Performance Score and the Google Lighthouse Performance Score. The same math is used for both scores. However, you may see a difference in the output of the score. Where Lighthouse is limited to only running on your local machine with limited network configurations, The Rigor Performance Score has the ability to run with network throttling, different locations, different browsers and an array of viewport sizes. The difference in these inputs is what will change the output. As these inputs remain the same so will the score.
Why is there a "No Data" value given for the Performance Score?
This can be broken into three different situations:
1) One or more components may be missing.
Since the performance score is a composite score made of interdependent metrics it is possible that there will be no value shown for your performance score. This may be due to not having a timing for one of the 5 metrics listed above. For example:
By running a scan in Optimization of example.com we see that the Performance Score shows a "No data" value.
If we dig a little deeper into the Page Load Details we see that there is no timing for Speed Index. Since Speed Index is a required metric for calculating the Performance Score there will be a "no data" value given for that score. In Monitoring this will hold true for any of the other metrics related to the Performance Score as well.
2) The Optimization Scan/RBC Check is old and is not updated to reflect our new metrics.
It is important to keep in mind that since the Performance Score is a brand new score that takes in new metrics like: Time to Interactive and First CPU Idle, old scans/RBC checks which don't have these metrics will yield a "No data" value for the scan.
We can see this holds true for the monitoring dashboard. With these older RBCs, it is possible that they do not contain the necessary components in order to calculate the Performance Score.
3) By uploading a HAR file straight to Optimization.
If you upload a HAR file to the Optimization scan you will also see a "No data" value given for the performance score, because all the components are not there in the HAR.
View ArticleIn the Rigor Optimization product, you can view a historical archive of numerous performance metrics for a performance test over time including the best practices score, content size, and more in the Snapshot History view. This archive of data is automatically maintained as you continue to run new performance snapshots for any of your performance tests. No setup required.
Accessing Your Snapshot History
To access the Snapshot History view, visit the View Results page off the Rigor Optimization home page, then click the snapshot history link next to the performance test of interest.
Alternatively, you can also click Snapshot History directly from any existing snapshot as shown below.
Snapshot History Graphs
Within the Snapshot History view youll immediately see a detailed graph with numerous filtering options available for customization.
In addition to different available charts for selection, you can also customize the time range for available snapshots (based on when the snapshot was created) for display, as well as filter the results based on whether they apply to first party content, third party content, or both. All time ranges use the local timezone and locale settings configured for your Rigor Optimization user account.
Available Charts
The following charts are available for selection:
Defects By Severity
The distinct number of defects, by severity, over time. Ideally youll want the number of higher severity defects to decrease over time.
Possible Savings
This chart shows the original size of the selected content for each snapshot over time, as well as the potential size of all recommended lossy and lossless optimizations were applied. Think of this as the theoretical best case content size you could achieve for your existing content by fixing all the defects identified by Rigor Optimization. As your site becomes more optimized, the original size should decrease and move closer to the optimized size values.
Best Practices Score
The best practices score for each snapshot over time. The best practices score measures how well a site is implementing performance best practices, and how much potential there is to optimize its performance. This metric can not be filtered for first or third party content. Ideally you want your best practices score to increase over time.
Total Content Size
The total original size of all content over time, as well as how much of that original size was first party vs third party content. This chart is useful for monitoring how much of your content is pulled in from external third parties, such as ad providers or hosted javascript libraries. Ideally your overall and third party content size should decrease over time.
Content By Host
This chart totals all content identified by hostname over time. This is useful for spotting large external dependencies you may not be aware of, for example a large dependency on a particular CDN or font library. To keep this chart readable, only the top 30 hosts, sorted by size, will be displayed for each snapshot.
Content By Type
This chart totals your content by file type over time. For example total image size, total font size, etc. This is especially useful in monitoring your usage of media such as images and video, as well as dependencies on specific resource types like Javascript, CSS and Fonts. Ideally you want all sizes to decrease over time, but pay particular attention to images and video since those resources often offer the greatest optimization potential.
Resources Loaded
This chart shows the total number of requests made over time. Specifically, the number of first party requests, third party requests, and both. Generally (but not always) the less requests you make, the faster your site will load.
Customizing Your Results
As mentioned above, you can further customize your results by date range and first vs. third party content in the chart filters. In addition, you can drag your mouse over any range in the chart to zoom in.
Click Reset Zoom to zoom back out.
You can also click on the chart legend to show or hide specific ranges. This is especially useful in the Content by Host chart where many ranges will be simultaneously displayed.
Use the Show All and Hide All links as a convenient shortcut to select or deselect all ranges.
Downloading charts
For convenience, a download image link is provided in the top-right corner of the snapshot history view. Simply click to download a PNG image version of the currently visible chart.
Custom Time Ranges and Paging
You can use the date picker to select pre-built ranges as well as a custom range of snapshots, all in the local time of your user account.
Use the previous and next buttons to page through your custom range. Each click will move the results up or down one-half the total selected range distance.
Sharing and Deep Linking
You may notice whenever you change any of the filter options for the charts, the URL in your browser will also change. This is called deep linking and means you can copy/paste the URL from your browser to preserve the exact chart and applied filters currently used. This is especially useful for sharing a link with your peers so they can see the exact chart you are viewing on their own local workstation, even at a later point in time.
The deep links will also preserve the exact selected time range at the point in time the URL was copied. So for example, if you selected the Last 7 Days date filter, the link will preserve the specific 7 day range at the point in time the URL was copied, rather then resetting to a new 7 day range at a later date. In other words, you should always see the same result every time you click the link, even if clicked several days later.
In addition to deep linking, you can click the Share button at the top-right corner of the page to generate a guest-accessible link. Share links allow access to this specific result even for people who do not have a login to the Rigor Optimization product. This is useful for sharing your results with others who may not be actively using the product. Users clicking share-links will have read only access, meaning they can not create or launch new performance snapshots, but they can view all the results from this page and all linked snapshots from this page. (They can not, however, view snapshots from other performance tests unless those are shared seperately).
Click to View Snapshot
You can click any point in the displayed chart to view the specific performance snapshot detail that generated that data point. These links will also work if accessed via share link.
Live Mode (Dashboard Mode)
Live Mode is a feature designed for those who like to view continuously updating dashboards on a dedicated display, for example in a DevOps team room. To enable Live Mode, simply click the Live Mode button in the chart button bar, and then optionally click the expander button in the top-right corner to expand the graph to full screen.
Alternatively, you can also apply custom sizing to the displayed chart using the splitter bar located below the charting area.
Once enabled, the currently selected chart will automatically update every 30 seconds. Only relative time ranges are allowed for selection in Live Mode, and live mode will always end at the current time. To suspect live mode, simply click the disable button in the chart button bar.
Live mode will also update the snapshot table results found below the charting area.
Snapshot List Updates
The snapshot list table has also been extended to show new data columns used in the charts, including Original Size, Lossy Optimized Size, Losslessly Optimized Size, and inline sparkline charts showing the defect counts by severity for each snapshot.
If Live Mode was enabled above, this table will also update with each refresh to the chart.
View ArticleJenkins is a popular and extensible open-source platform for managing continuous integration builds. You can use Rigors Optimization plugin for Jenkins to test your website performance as a part of your Jenkins build process.
This article assumes you already have Jenkins installed and youre familiar with how to manage Jenkins freestyle projects. You can learn more about getting started with Jenkins on the Jenkins Website.
How to Set Up the Rigor Optimization Plugin for Jenkins
The Rigor Optimization plugin is freely available for download and can be installed via the Jenkins plugin manager. From your Jenkins installation homepage, select Manage Jenkins and then Manage Plugins. Then select the Available tab and search for Rigor Optimization. Once you locate the plugin, install through the interface.
For more details about this plugin, check out the plugin wiki on the Jenkins site.
Add a New Build Step
Once the plugin is installed, select your Jenkins freestyle project from the root page and click Configure. Scroll down to the Build section and click Add build step. Find the option labeled Test website performance using Rigor Optimization.
Rigor Optimization Performance Knowledge base
Select this and you will see a custom build step to use Rigor Optimization.
Configure Your API Key
You will need to grant API access to Jenkins to call into the Rigor Optimization API to initiate performance snapshots as part of the build. To do so, first, locate your API key in your Rigor Optimization settings.
Note the API key must belong to a user with *User* level permissions or higher. (e.g. not a read-only user).
In Jenkins click Add next to the API key setting of the build step, then select Rigor API Key under the Kind dropdown.
Enter your credentials and then select them on the return screen.
Configure the Performance Tests
When this build step executes, Jenkins will call into the Rigor Optimization API to initiate new performance snapshots for one or more performance tests. So the next step of our set-up is to identify which existing performance tests we want to execute.
To locate these, navigate to the View Test Results page of the Rigor Optimization application, and locate one or more tests to run with each build. To find the Performance Test ID, hover over the i icon and locate the Performance Test ID in the popup.
Add this to the Performance Test IDs field in your Jenkins configuration and repeat for any other tests you would like to launch. You can separate multiple test IDs with commas.
Then click the Test Connection button to confirm your API key and Test IDs are valid.
If you see errors, double-check that your API key was pastedcorrectly, the account has User level or higher permissions, and the listed Performance Test IDs were properly listed.
You are now ready to go! When you save the build step, the next time your build runs it will launch new snapshots for the configured tests as part of the build. Over time you will see historical records of how your website performance changed after each change was committed.
How to Customize Build Failures
You may want to optionally fail the build if certain performance problems are detected. This next section will help you further configure different parameters for failing the build.
First, check the option Fail build based on test results and a number of new options will be displayed.
If one or more of these settings are enabled, the build will wait until all initiated performance snapshots complete, then test the results of each against these parameters. If any one of these parameters fails for any one of the configured tests, the build will be marked as failed.
Lets walk through each option.
Fail if below this Best Practices Score
This refers to the overall Best Practices Score assigned to each snapshot when it completes. This is perhaps the easiest metric to track. Assign a minimum value you would like to see (ideally based on your existing historical results), and if any snapshots from this build score under that number, the build will be marked as failed.
Fail if above this number of critical defects
Like the name implies, if any snapshot has more than this number of critical defects reported, the build will fail. Whats interesting here is you can configure which defects get marked as Critical using the Rigor Optimization Customizations feature.
For example, you can assign specific thresholds to specific defect types, and re-assign the priority of those defects to Critical when those defects flag. This could be useful for alerting you when images above a certain size are introduced, or lossless optimization savings are beyond a certain value, for example.
Fail if defect IDs found
This field takes a comma separated list of defect IDs that, if found, will fail the build. For example, perhaps you want to fail the build if any compression defects are introduced.
To locate the defect IDs, visit any defect page in the Rigor Optimization application (or in the and look for the ID label towards the top of the defect description:
Timeout waiting for test completion
If one or more metrics are configured in the Fail build based on test results section, the build will wait for all snapshots to complete before proceeding to the next build step. Depending on the tests configured, this could delay your build from 30 seconds to several minutes. This timeout value allows you to set an upper limit on how long to wait. If the timeout occurs waiting for completion of the snapshots, the build step terminates and proceeds to the next step. This may or may not fail the Jenkins build based on the next setting.
Fail if tests encounter errors
This controls how the Jenkins build will be marked if any Rigor performance tests encountered errors (including a timeout, per the above section). If true, the build will always be marked as failed if errors are encountered. This is appropriate if you want immediate visibility into potential configuration errors. If false, the build will not be marked as failed if test errors occur. This option is less disruptive to your build process but may mask configuration errors and hence puts the onus on you to monitor your logs from time to time.
Linking Builds to Snapshots
Whenever a Rigor performance test is run from a Jenkins build, that test is tagged inside Rigor Optimization with the name of the Jenkins build project that executed the test. For example, the tests below were run from the ZoompfCore Jenkins project.
In addition, each snapshot executed is also tagged with the project name, build number and build pass/fail details. For example:
this association allows you to quickly spot which tests and snapshots belong to which Jenkins builds, and quickly filter your results in Rigor Optimization for any errors that may have occurred.
Debugging Build Failures
The Rigor Optimization plugin logs verbose details of all performance tests run in the console output of the Jenkins build. If a build failure occurs (or even if it succeeds), simply view the Console Output for that build number and scroll down to the Rigor Optimization section. You should see something like this:
If failures occur, verbose details of what failed and why can be found in the log.
View ArticleThere are a number of performance optimizations that can be made or best practices that should be followed when building or configuring a website to improve the performance and user experience.Rigor's Best Practices Score is a high-level numeric score that measures how well or how poorly a site is implementing these performance optimizations.
The Best Practices Score is based on the impact that different optimization have.Some optimizations can have a larger impact on the performance and user experience than others.For example, an unapplied optimization that reduces a page's critical rendering path will impact the overall score more than an unapplied optimization that reduce the number of requests that must be made.
Rigor classifies how an optimization impacts a site into different impact categories, such as those that reduce the number of requests or those that reduce the size of responses. Rigor measures how well a site is implementing each category on a scale from 0-100,the higher the score the better optimized your site is for that category. These scores are weights and combined to create the total Best Practices Score.
This combined approach helps you better understand where a site is doing well, and where the is room for improvement. Let's look at an example:
In the example above, the site has an overall Best Practices score of 44. The score for the Request Countimpact category of optimization is 33 out of 100, indicating there is a lot of opportunity for improvement. This category represents 40% of the total score, so improving this would be very important. We can also see that this site has a Latency score of 100, meaning the site has succeeded in implementing optimizations or best practices which reduce latency.
To know what improvements can be made to increase the score of an impact category, you can filter the list of Defects by impact category, as shown below:
View ArticleRigor Optimization is an external front-end performance platform that enables web developers and content managers to identify bugs, prioritize fixes, and optimize content that negatively impacts performance. Optimization enables tech teams to test throughout the development lifecycle and measure the impact of changes on site performance and functionality.
In this article, well cover some things to consider as we get started, and well walk through the basics of creating tests and analyzing results in the Optimization app.
Logging into Rigor Optimization
Rigor is a web application, so there is no installation necessary. A member of the team at Rigor can provision an account for your companys use, or someone on your team may invite you to join an existing Rigor account.
When an account has been created for you Rigor will send you an email with instructions to set a password. Once a password has been set you will be able to log into the Rigor app at https://optimization.rigor.com.
If you have not yet received an email invitation from Rigor to set a password, please ask the Rigor Admin for your organization to create a new user for you.
Account Settings
Users with GroupAdmin permissions can create users by navigating to Profile > Settings > Account Settings or by following this link.
GroupAdmin users can also view their account license and limits from the Account Settings page.
User Settings
When you log into the Rigor app you should see your settings in the top right corner of the user interface:
From the settings page, users can edit their profile, enable integrations, access API credentials, and more.
Install the plugin directly from the Jenkins site
Creating a Test
Basic Performance Test
Once logged into Rigor Optimization, users can create a new test by clicking New Test.
In order to create a basic Single Page performance test, specify three values:
URL of page to test
Device type
Test name (optional)
Click Start Test to begin the performance analysis.
Basic scans usually take between 1-5 minutes to complete depending on the size and complexity of the page.
Advanced Performance Tests
Bulk Pages
Website Crawl
Scripted
HAR File
Rigor Optimization makes it possible to run tests in local developer environments or sites behind a firewall. Learn more in this blog post.
Advanced test types are not included in all packages. If one of these options isn't available in your account, contact your Rigor account manager for more information.
Defining and Enforcing Performance Best Practices
Rigor Optimization scans your site for approximately 300 performance defects. Before diving into the results, it's best to define a small list of internal performance best practices that can be referenced when implementing Optimization.
Read this post from the Rigor blog to learn more.
Defect Policies
Defect Check Policies govern which performance defects the Optimization engine looks for when you run a test. By default, each test uses the All Defects Policy. This policy includes the most common performance checks used by our top customers. Follow the instructions on Step 2 of this blog post to create your own Defect Check Policy.
Analyzing Test Results
Segmenting First and Third Party Content
When analyzing the performance of your site, there are two classifications of hosts that can serve content:
First party: content that you create, develop, or have direct control to manage (i.e. images, video, CSS)
Third party: content linked from your webpage over which you do not have direct control (i.e. ads, social plugins, analytics tags)
Rigor Optimization makes it simple to classify first and third party hosts so that you can ignore content you don't control. We recommend either defining first party hosts for each snapshot, or globally whitelisting first party hosts and blacklisting known third parties. Learn more in this blog post.
Personalizing Results
Similar to defining a global Defect Check Policy, Rigor users can personalize all results within an Optimization snapshot. This includes:
Muting defects that aren't relevant to your organization or team
Defining severity to prioritize most important defects first
Specifying a threshold for which a defect is flagged to help enforce internal best practices
These customizationsdon'tchange the scope of how Rigor tests for performance issues. They simply act as filters that determine whether we show you a problem or how a problem impacts your overall results.
Learn more on our in this article.
Understanding the Results
The Optimization engine runs a performance analysis checking for over 300 different performance defects. The Optimization Knowledge Base defines each of these checks and includes an overview, technical details, instructions on how to resolve, and external references to learn more.
Active users can view the full list of defects and descriptions on our here. Don't see the defect you're looking for? Let us know at [email protected].
Optimization API
Each customer has access to our comprehensive REST API for Rigor Optimization. The API allows you to extend the functionality of Optimization to:
Create, Read, Update, and Delete Performance Tests
Create, view, and delete existing Snapshots
View performance defect details inside a Snapshot
View site content details inside a Snapshot
API documentation can be accessed at optimization-api.rigor.com. This includes instructions for accessing the API, retrieving your API key, and using the interactive API Explorer.
Case Study
See how Rigor Engineering is using the Optimization API to integrate performance testing with continuous deployment.
Integrations
Many of our clients use Optimization in sync with their development, QA, and operations workflows. We've built integrations that enable our customers to integrate performance testing with their internal processes to enhance automated testing and increase efficiency.
JIRA
Performance defects are bugs. When new issues are introduced to your site, it's important to log these bugs in a way that's seamless with your current issue resolution workflow. If you use JIRA to track bugs and defects on your web application, you can now log these bugs directly from Rigor Optimization.
To get started, log int your Optimization account and then select Settings > Tools from the top right corner.
Click Enable under the JIRA section and you'll find instructions and more information about the integration. For more detailed instructions, see our this article.
Jenkins
Treating performance defects as bugs and automating performance optimization as part of your workflow makes it much easier to manage performance issues over time.
If you use Jenkins for your continuous integration builds, the Optimization plugin makes it easy to automate performance testing as a discrete step in your build process while allowing you to specify success criteria for a passed or failed build.
For full details about how to install and configure the Rigor Optimization plugin for Jenkins, check out this post in this article.
Want to skip the documentation? .
View ArticleHow Does Rigor Collect Data?
Continuous Data Collection
To ensure maximum accuracy and precision, Rigor collects performance data continuously while testing a website. This means that while every step of a script is running, Rigor is collecting metrics, recording custom user timings, and capturing screenshots. When a check runs, Rigor collects and reports network data and filmstrips during the entire scripted flow. This ensures that the user can see a filmstrip and video of all the user actions, all of the requests/responses that occurred in the waterfall, and all of the browser, timing, or UX events that occur during the run of the script. The results of this data can be trended on a Performance KPI graph, alerted as a threshold, and can be viewed in the Run Results page.
Performance Metrics Glossary
Understanding Run Results
For any check run result that has screen captures, both a filmstrip and video will be available. You can scroll through the filmstrip, or scrub the video at different speeds. Full-sized screen captures are also available for download from the Filmstrip widget.
Click the video button to watch a real time, slow motion or accelerated playback of the rendered content loaded by that runner result. You can also download any video for sharing with your teammates.
If a run visited multiple pages, a page selector dropdown will be available to jump to any single page in the results to view metrics, filmstrips and a waterfall specific to that page.
Click any of the screenshots to open the Filmstrip Screenshot Viewer to see a larger image. Inside this viewer, navigate forward and back to more easily view any visual changes.
Changing the Interval Between Frames
By default, only visual changes are shown in the filmstrip, but you can use the interval selector in the filmstrip header to adjust the period between frames. Available intervals are Every 100 ms, Every 500 ms, and Every 1 s.
Any interval that is set after page load will persist across different pages, but will always reset to "Every Visual Change" on page load.
Page Transitions
You may notice in filmstrips for muiti-page flows that images from the prior page show up at the start of the next page, as is shown below.
While this may seem counter-intuitive, this is actually how the browser renders pages from one to the next. When navigating away from the current page, the existing page is not replaced with a white frame, but instead remains rendered as-is until the first paint of the new page beings. You can see that in your own day to day experience in any modern web browser (and more importantly, that's what your users see as well), and inevitably leads to a better user experience since you still have the ability to read rendered content while waiting for the next page.
The Rigor filmstrips begin loading each page at a 0ms ticker (reset with each new page) and begin transition to the next page when the fully loaded event is fired.
Sharing Page-Specific Views
Navigation between pages is persisted in the URL, so sharing a page is as easy as copying and pasting the current URL. By default, the All Pages summary will load if a specific page has not been selected.
Viewing Multiple Pages
On the All Pagesview, these modules display timing data across all the pages visited during that run, so you can quickly analyze your check results and find outliers. Hover over a single bar to view all other values for that single page, or click on a bar to drill down for more page-specific details.
For more information about these metrics, click check out our .
Select a page from the dropdown to view page-specificdata. Click on a timing metric to auto-scroll the filmstrip to that point in time. This allows you to easily correlate your timing data with what a user would see when visiting your site.
In Closing
Beyond the addition of new metrics and visualization features, we've integrated all these results into one easy to navigate and share page to streamline your analysis workflow. And as before, you can click the Analyze button on any page in the workflow to run a detailed Rigor Optimization performance analysis of that page.
We hope these updates are helpful in identifying and resolving the causes of slow performance on your site.
View ArticlePerformance KPIs allows to you to track and trend key metrics over time, providing insights into business-relevant metrics about your site and it's performance.
In Rigor, every Real Browser check hasa Performance KPIs report, which can be used to compare a variety of metrics from multiple pages. The data powering this report is fully accessible via our API.
Rigor Monitoring API
To get to the Performance KPIs report, click the link in the reports dropdown menu.
Use cases
With the Performance KPIs report, you can:
Assess how your user timings and other Real Browser metrics trend over time
See how a KPI trends over time, using aggregators like median, average, maximum, minimum, and more
Compare data for multiple KPIs from a single page or from the check as a whole
Compare data for one KPI overmultiple pages visited by a check
Compare multiple Real Browser checks to one another
Filter out data based on success/failure status
Trackperformance by location, country, or world region
Available KPIs
Change the KPIdisplayed on the graph using the '+ Add Metrics'dropdown filter.Metrics are grouped by their type: timing, count, size, or user timings.
Time Intervals
You can view run-level data or aggregate by the following time intervals.
Note: the API allows you to set whatever interval you want (not just the preset ones here). For example, you could do every 42 minutesif you want.
Segments
Pages
You can filter the data to show one or more pages, or click "Total" to see how the check performs as a whole. The Total series sums the data from all of the pages visited during a check run.
Compare Checks
You can select any other Real Browser checks to compare. This could be helpful when comparing different network configurations, using Rigor for A/B testing, or comparing staging vs. production.
Aggregate
You may want to change the aggregation method depending on the type of information you're looking for.
Median and Mean values both provide good baselines for your performance, although Median datais more resistant to outliers
Maximum and Minimum values can help you findoutliers in your data that indicate poor performance or broken functionality
90th, 95th and 99th percentile are good for excluding outliers on the extreme ends of the spectrum
Filters
Exclude failures
Location
You can view data from any number of the locations it runs from in a single graph. You can customize which you'd like to see.
You can also choose to group your data by location, country, or world region.
Share and Export
After you configure the chart to view the KPIs you want, you can share the report in two ways:
Copy and paste the URL from your browser to share with any Rigor user. The graph will load for that user with the same timeframe, pages, and metrics you have selected
Export an image of the chart by clicking on the image export button. This is useful if you want to share this data with someone without a Rigor login.
Export the data in the graph as a CSV
Pull the data from the
View ArticleWhat Is the IP Address for APItester.com?
Rigor's free API Testing product, apitester.com, runs in a cloud environment from the following IP addresses:
54.208.109.193
50.16.249.180
50.17.187.84
Depending on the configuration of your test, you may need to whitelist these IP addresses for them to run successfully.
Can I Share My Results?
Yes! After clicking on a test to view the test's results, click the "Share" icon in the top right to create a share link that you can send to your team. After creating the share link, you have the option to delete the link by following the same steps.
here
Where Can I Find Documentation About How to Use Some of API Tester's More Advanced Features?
API Tester is built on top of the same technology that powers Rigors commercial API monitoring product. You can learn more about how to setup tests or use the JavaScript step .
Have Feedback or Another Question?
Please reach out to the Rigor team at:[email protected]
View ArticleSetup
Full documentation on the API can be found here
To get your API key, go here
Use the "User API-Key" value for the calls below. Some steps may also need the "Client Key" value (see below).
For each request to the API, you will want to pass these 2 headers:
Accept: application/jsonAPI-KEY: (your api key from above)
FAQ
To create a new account
You'll need your "client key", which can be found in your settings here: https://optimization.rigor.com/Settings/API
Creating a user
Specifically, you'll want to do an HTTP post to https://optimization-api.rigor.com/v2/users
With a JSON payload that defines your user details, like this:
{"account_keys": ["account_key1", "account_key2", ...],"name": "James Holden","email": "[email protected]",}
where account_key1,2,3... is from your list of accounts from step 2. This is the list of accounts to which this user will have initial access. If you want the user to have access to ALL accounts, instead use the "client_keys" parameter, like this:
{"client_keys": ["(your client key from step 1)"],"name": "James Holden","email": "[email protected]",}
Updating a user
If you want to change permissions of a user, you'll first need that user's id. You can get that from the "Get Users" call.
You can then use the "Update User" call to change their permissions and details.
View ArticleOn the Basic tab for Real Browser checks and Uptime checks, well find a setting called Round Robin.
The Round Robin setting relates to the Frequency setting on the Basic tab and the Locations settings.
When Round Robin is enabled Rigor will cycle check runs through the selected locations at the specified frequency, running from one location at a time.
When Round Robin is disabled Rigor will run the check from all selected locations at the specified frequency, running from all locations at once.
This feature comes in handy, especially if our multi-step check:
Monitors a user login requiring unique browser sessions, where the user can only be logged in from one location at once, or
Monitors a user flow where a site dynamically identifies the users zip code for use in a form
By default, Rigor checks run Round Robin through locations. As a best practice, we should only disable round-robin if we need maximum run coverage for more aggressive alerting.
Note: Not all Rigor accounts have access to disable the Round Robin setting. If youd like to disable Round Robin anddon'thave the ability to un-check the box in the app, email [email protected] and well work with you to add this option to your account plan.
View ArticleHere are some common metrics defined in the Rigor app:
Server Time
The time it takes for the server to respond to the initial request.
Render Time
The time it takes for the user to first see the website.
Load Time
The time it takes for the website to completely load all required page elements.
Response Time
The Response Time for a single page Real Browser Check is the same as the Load Time. If a Real Browser Check has multiple steps then the Response Time equals the sum of Load Times for each page accessed during the user flow. The Response Time for an Uptime Check is equal to the Server Time.
Uptime
For an Uptime Check, the Uptime metric represents the percentage uptime of an endpoint for a set time frame. For Real Browser Checks the Uptime percentage represents the percentage of time that the check passed within a set time frame.
Page Size
The total size of all the files on the page (KB/MB).
Requests
The number of elements on the page, or the number of requests it takes to load the page.
Errors
Client, Server, or Connection errors that are fired during the page load (HTTP Response Code).
Total Load Time
The time it takes for the page to load all resources, including those that fire after the onload event.
Note: Rigor displays the total load time on each waterfall chart, but does not save this metric for reporting or trending.
View ArticleGlobal Variables can be defined in a single place and reused across checks check here for API usage.
The Global Variables form is accessible from the administrator menu at the top right of the screen.
Description (optional): A description can be useful to explain the purpose of the variable for future reference.
Key: The name of the variable. You will use this to access your variable within a check.
Value: The value that will replace the variable when the check is run.
Using Global Variables
Global Variables can be used like any other variables in a Real Browser Check. They can be accessed via the env namespace. For example, if you have a Global Variable with key rigor_url, you can access it within a check with {{env.rigor_url}}.
If you need a reminder of what Global Variables are available while creating or editing an Real Browser Check, hints are available by clicking the i icon at the top right of the check. You can view all available built-in variables and Global Variables. Experienced users might find the keyboard shortcut useful: [SHIFT] + [i]. Please note that this help panel does not show request variables or custom variables that have been defined in the check.
View ArticleBlackout Periods (Basic)
Specify times when the check should not run. For more information about setting blackout periods, see this article.
Locations (Basic)
By default Rigor will run new Real Browser checks from all available United States locations. Use this setting to disable locations or select additional multiple locations from Rigors global monitoring network. For a full list of locations and their IP addresses, see this article.
Notify of location-specific failures
Enable this option to receive failure notifications regardless of location-specific issues. Not sure whether to enable this feature? See this article.
Escalations
Use escalations to send alerts to new users or groups if an alert has been unacknowledged for a specified amount of time.
We can add multiple escalation emails or repeat escalation emails until a check is back online. Read more about setting escalations here.
View ArticleIn Rigor, each Real Browser Check also has a Domain Performance Report. Domain Performance Reports help us see how specific domains contribute to the checks total response time over time.
Rigor API
This view can be helpful for identifying which domains may be loading in more assets or slow assets that impact the overall response time for a page or check. We can use the metric selector to sort by time spent in a specific request phase, total time, or total size and we can use the domain selector to include or exclude specific domains from the report view.
This chart is a stacked area chart, so the areas for the domains stack on top of each other as we include more domains in the chart.
To navigate to a Real Browser Checks Domain Performance Report:
Navigate to the read screen for the Real Browser Check
Click Reports > Domain Performance
Once viewing the Domain Performance Report we can sort by locations, select a specific time frame to review, select specific domains, select different metrics, or click and drag on the report chart to zoom into a smaller time frame.
Notes:
Updating the locations or time frame will cause the chart to re-load and will clear any customized filters on the domain or metric level.
Domain Performance Reports are excellent for viewing trends in the Rigor app, but right now we can only visualize these trends for limited time ranges depending on the number of data points associated with a specific Real Browser Check. If we need to look at data over a long period of time like a month or a quarter, we can use a Custom Report or pull data via the .
View ArticleThis guide contains best practices, common conventions, and helpful tips for utilizing multi-step Real Browser Checks in Rigor Monitoring. Testing mission critical user flows on your site to ensure the optimal user experience is a necessity. Use this guide to assist with creating and maintaining these scripts.
Real Browser Check Scripting Guide - Rigor Monitoring
Download the PDF
View ArticleA Real Browser check allows you to monitor the user experience around performance for a single page or a multi-step user flow. For each page checked in a Real Browser check, Rigor captures a HAR file, or waterfall chart, that helps us see the performance of specific resources within the page.
You can use this type of check to monitor conversion paths or any path that requires multiple steps and must execute JavaScript.
Here are some ways to create Real Browser Checks in Rigor:
From Scratch (single page)
From Scratch (multi-step)
Via Import (multi-step from Selenium IDE source code)
From Scratch (single page)
To create a new Real Browser Check from Scratch:
Navigate to Real Browser + New > From ScratchORFrom the side navigation, click + New > Real Browser > From Scratch
On the Basic tab:
Name the check. Be sure to pick a unique, descriptive name so that you can determine the checks function at a glance.
Supply a starting URL. The check will begin by going to this URL and the first step in the check will be within the context of this page.
Frequency. Define how often the check should run.
Check Active / Inactive. Use this slider to activate or pause the check.
here
On the Notifications tab:
Select who is notified.Tell Rigor who, how, and when to alert users or groups if the check fails.
Location-specificalerts.Enable this to send additional alerts if a check fails consecutively from the same location multiple times. Alerts will send at the same frequency as configured in the main notifications settings.
Group alerts. Create groups to alert teams simultaneously.
Notification window.Select a specific timeframe to be notified of alerts. Some users set this to limit noise outside of business hours.
Escalations. Notify additional users if an alert goes unacknowledged.
From Scratch (multi-step)
You can easily add multiple steps to a Real Browser check. If you'd like to build a multi-step check, you should first open the initial URL (the same URL that you set on the Basic tab) in your browser.
For each step you would like to add to the user flow, you can Inspect Element in your browser to identify an id, class, name, CSS Path, or XPATH to target each element. Once you know the correct way to target a specific element you can + Add Step in Rigor to configure the Real Browser check to interact with the element or wait for the element to be present on a page.
Rigor Real Browser checks can go to URLs, click links, fill out form fields, and select values from dropdown lists. They can also accept or dismiss alerts, run JavaScript and save values from JavaScript, and switch to an iframe or switch back to the main source code for a page.
On the Steps tab:
Use + Add Step to create steps for your check that simulate user flow. There are two types of steps: actions and waits.
Use the hamburger icon to drag and re-order steps; use the trashcan icon to delete steps.
Click the Test button to queue up a test run in the Rigor system. Testswon'taffect reporting or trigger alerts but they can help diagnose whether test steps need to be edited to successfully simulate a path or user flow.
When youTesta before and after screenshot of each step will be created.
When a test comes back with a green Test Result: OK you know that you can save the Real Browser check and expect to see available results on runs unless some performance issue prevents the check steps from executing successfully.
Download Rigor Scripting Guide for more help creating multi-step Real Browser Checks from scratch.
Via Import (multi-step from Selenium IDE source code) *
If Selenium IDE is able to replay all of the steps without any errors (steps will be highlighted in green) then we can save & import. Please reference our full scripting guide here if first time!
Save the Selenium IDE test case (command + S) or by clicking the disk icon in the upper right-hand corner.
Checks > Real Browser + New > From File
Select your file to import > Import Rigor will automatically create the steps that correspond with the Selenium test and remove any steps that are duplicated by our system.
Note: Please make sure to hit the test button at the end all steps, this will be another way to make sure this will run within the Rigor ecosystem. After all the steps run without issue, we recommend that you save and manually trigger a live run as well. For further troubleshooting steps please refer !
View ArticleWhat is a Rigor Private Location?
Rigor's Private Location is a software package that enables quick and easy deployment of Rigor Monitoring and Optimization solutions beyond Rigor's public networkso you can find, fix, and prevent web performance defects on any internal web application, in any environment - whether inside or outside of your firewalls. This allows Rigor users to test earlier in the development cycle and against internal sites or applications that are not available to the public.
Customers can, through our web interface, create new Private Locations and launch the agent to perform any checks that are assigned to them.
Who should use Private Locations?
Customers who need:
to test private applications that are not exposed to the public
to test pre-production applications which don't have public staging sites
a higher level of flexibility when it comes to providing Rigor access to applications
to test from locations not currently supported by Rigor's public locations
Requirements
Rigor's Private Locations are currently distributed as Docker Containers and are able to be run in nearly all hardware and OS configurations. Our agent will run in Docker installed on Linux, Windows 1, or OSX 1 machines.
Since the Rigor agent is running a web browser there is a high degree of freedom you have when deciding on what kind of hardware to use for the agent. You will find that our agent will run on nearly all hardware:
Required:
Docker
Linux, Windows 1, or OSX 1
An internet connection
Recommended for Real Browser checks:
Linux
2.3 GHz Dual-Core Intel Xeon (or equivalent) processor
8 GB RAM
(Note: As a point of reference, we use m4 Large (8 gig RAM, 2 cores) instances in our AWS locations (and similar specs elsewhere); we also have some Private Locations up and running for testing in our local environments. That said, the closer you get to the m4 specs the more consistent the timings will be from across other Rigor locations. Regardless of that, though, consistent specs (no matter what they are) across all of the private locations is the most important component to ensure consistency in your data.)
Getting Started
Administrators within your Rigor account can navigate to the Private Location management section of the interface by selecting "Locations" in the sidebar on the left side of Rigor's Monitoring interface. Once there, a new Private Location can be created.
When creating a Private Location, you must specify a name and can optionally provide geographic information. The provided name will be displayed to all Rigor users in your account when selecting locations for new and existing checks. The optional geographic information is used for seamless integration with Rigor's own locations (e.g. region-based reporting) and is available to checks running in that location as global variables.
Once a Private Location is created, you'll see that we are "Building your Private Location..." While this is happening please navigate to the top right corner and click "Instructions".
You will be directed to a screen with instructions for launching your Private Location. We provide instructions for a few common scenarios: Docker for Linux, Docker Compose, AWS Elastic Container Service (ECS), and Docker for Mac/Windows 1.
Each Private Location can have multiple agents running by repeating the setup instructions or scaling up your infrastructure using the same configuration. You are free to scale up each of your Private Locations as you need in order to handle the number of checks that you have configured to use it.
Configuring Checks
Once your Agent is up and running for a Private Location, the location will appear for all users in your organization in the location selection drop-down when configuring a check. If you have provided geographical information for a location, it will be listed within the World Region (North America, Europe, Asia, etc.) you specified when creating your location. All Private Locations are indicated with a pin icon in the locations drop-down. In this way they are distinguished from Rigor's public global locations.
You can select a combination of public and Private Locations and Rigor will schedule work across them following the same behavior as you expect using Rigor's public locations.
Checking the health of your Private Location
With the Private Location, your organization is responsible for the health and stability of the location. Visiting the Private Locations page, administrators will find an overview of all of the Private Locations for your organization which will indicate the health of each Private Location. Clicking on a Private Location will show you additional information about that particular location as well as a list of checks that are currently configured to run from that Location.
Managers will be able to see the health of Private Locations; however, administrators are the only ones that will be able to create and view granular details on these locations.
Engineering Note: Due to technical limitations of network shaping -- and to minimize noisy-neighbor issues -- each agent processes a single check at a time.
Disabling Network Shaping
Disabling network shaping will instruct the Rigor agent to not perform throttling of the network connection, which is used to emulate different types of mobile and desktop connections.
You may need to disable network shaping if you:
are hosting Docker on Windows or OSX
are uncomfortable with the security implications of Docker's NET_ADMIN capability
In order to disable network shaping, you will need to provide an additional environment variable to the Docker container. Provide the "DISABLE_NETWORK_SHAPING" variable with a value of "true". With network shaping disabled, you may remove the NET_ADMIN capability from your launch commands, as it is no longer required.
For the Docker CLI,insert the following into the run command provided in the location setup instructions immediately following the "run" keyword.
-e "DISABLE_NETWORK_SHAPING=true"
Additional Reading
Here is some additional documentation about our Private Location feature and agent:
Security Considerations for Private Locations
Automatically Update Private Location Runners
1 Windows and OSX Docker hosts will not support Network Shaping
View ArticleOutbound Internet Traffic
Rigor's private location test agent will need outbound access to the following resources via HTTPS:
qapi.rigor.com -- Rigor's agent API for orchestration and status reporting
AWS S3 in us-east-1 -- Test resultpayloads will be uploaded directly to S3 storage and Docker image binary data
AWS SQS in us-east-1 -- Runners obtain units of work via SQS
docker.rigor.com -- Rigor's private Docker registry to pull the Docker image
api.rollbar.com -- We use the Rollbar API for error logging and reporting
Discovering IP address ranges for AWS services
AWS services have dynamic IP addresses, but the ranges of IPs for each service is published and can be automated. For more information, see AWS IP Ranges in the AWS documentation.
The relevant services in the list of IP ranges are "AMAZON" and "S3" in the us-east-1 region.
Inbound Internet Traffic
Rigor's test agent does not require any inbound connections nor requires any ports to be exposed.
Docker's NET_ADMIN Capability
In order to enable network traffic shaping, our private location docker container must have elevated access to the host system's network stack. This is set by adding the NET_ADMIN capability. Typically, this grants the container process the CAP_NET_ADMIN Linuxcapability.
Linux's capabilities MAN page provides this information:
CAP_NET_ADMINPerform various network-related operations:
interface configuration;
administration of IP firewall, masquerading, and accounting;
modify routing tables;
bind to any address for transparent proxying;
set type-of-service (TOS)
clear driver statistics;
set promiscuous mode;
enabling multicasting;
use setsockopt(2) to set the following socket options:SO_DEBUG, SO_MARK, SO_PRIORITY (for a priority outside therange 0 to 6), SO_RCVBUFFORCE, and SO_SNDBUFFORCE.
It is possible to run our private location agent without the NET_ADMIN capability, but this will result in loss of network shaping functionality and all checks will run unthrottled regardless of the configuration settings in the Rigor UI.
Data Sent to Rigor
As of this time,ALL web content the test agent sees for Real Browser checks are bundled up and sent back to Rigor-- including screenshots, full HTML documents, CSS, and images. It is transferredvia HTTPS and encrypted in S3 using the AES256 specification.
While Rigor takes significant steps to secure all of our customer's content, please consider carefully the kind of information visible to the test agents.
View ArticleRigor will often publish updates to our private location docker images and it is critical that your organization automates the upgrade of your locations.
Watchtower is entirely optional, but we do require that you havean auto-upgrade solution in place for the Rigor runner. Failing to automatically update our Docker container will result in inconsistentdata and loss of functionality.
Using Watchtower
Watchtower is a 3rd party open source Docker container that will connect to remote Docker repositories on a schedule and check for updates. When an updated image is found Watchtower will instruct your Docker host to pull the newest image from the repository, stop the container, and start it again. It will ensure that environment variables, network settings, and links between containers are intact.
On your Docker host, you can simply launch the Watchtower container via command line:
docker run -d \
--name watchtower \
-v /var/run/docker.sock:/var/run/docker.sock \
v2tec/watchtower --label-enable --cleanup
Using the "label-enable" flag will ensure that only containers with the correct label, like the Rigorrunner, will be auto-updated.
There are additional options available in the Watchtower documentation that you should explore, including auto-cleanup of old images to ensure that your Docker host does not hold on to outdated images.
It is important to note that in order for Watchtower to issue commands to the Docker host, it requires the docker.sock volume or TCP connection and this provides Watchtower with full administrative access to your Docker host. This level of access should not be taken lightly and it is one of the reasons Rigor decided to separate the auto-update procedure from the Rigor agent. If you are uncomfortable with providing Watchtower with this level of access you are encouraged to explore other options.
Using Amazon Elastic Container Service
You can still use Watchtower with Amazon's Elastic Container Service. Here is an same task definition that you can run as aDAEMON within your cluster.
{
"requiresCompatibilities": [
"EC2"
],
"containerDefinitions": [
{
"command": [
"--label-enable",
"--cleanup"
],
"name": "watchtower",
"image": "v2tec/watchtower:latest",
"memory": "512",
"essential": true,
"environment": [],
"linuxParameters": null,
"cpu": "256",
"mountPoints": [
{
"readOnly": null,
"containerPath": "/var/run/docker.sock",
"sourceVolume": "dockerHost"
}
]
}
],
"volumes": [
{
"name": "dockerHost",
"host": {
"sourcePath": "/var/run/docker.sock"
},
"dockerVolumeConfiguration": null
}
],
"networkMode": null,
"memory": "512",
"cpu": "256",
"placementConstraints": [],
"family": "watchtower"
}
Using Another Solution
Due to the complexity and variability of our customers' infrastructure configurations and the security requirements for managing Docker containers, Rigor does not require any specific solution for automatically discovering and deploying updated Docker images.
If your operations team already has a mechanism established for deploying updates to Docker images you are able to use it without making any configuration changes to the Rigor runner.
Rigor recommends that your upgrade mechanism discovers and deploys new images at least once every 24 hours.
View ArticleGetting started with Selenium IDE
Find the latest version of Selenium IDE and click + Add to Chrome or Firefox Firefox Chrome
here
Restart the browser and go to Selenium IDE to open the recording window.
Youll be prompted to Record a new test in new project or Open an existing project. Choose the applicable action.
Navigate to Base URL and input the URL youd like to record via Selenium IDE for the user flow to record and monitor.
In the browser window, start interacting with the page, clicking and stepping through the user flow to record and monitor. Notice that Selenium IDE records actions as steps.
When were done with the steps wed like to record and monitor we can click the red circle icon on the Selenium IDE recorder to stop the recording.
Reading The Selenium IDE Recording Results
The Selenium IDE records steps and displays the steps in a table with three columns: Command, Target, and Value.
Command:represents the actionTarget:represents the selector type (ID, CSS, XPath) andthe specific selector that Selenium can use to target the element that corresponds to the actionValue: represents any value that should be entered into a fieldFor example, if we want to record entering the value Zoompf into a search field with the ID search-input then the row in the Selenium IDE recorder for this action would look like:
Command
Target
Value
type
id=search-input
Zoompf
Playing Back The Selenium IDE Script & Troubleshooting
In the Selenium IDE recorder window, click the Play triangle to Play current test case. We may want to adjust the speed settings from Fast to Slow.
This will allow you to quickly know if the user flow is capturing what you want. A common issue youll want to check for is that the Target is selecting static elements versus dynamic. If a step fails in playback it will be come up in red. We recommend checking the steps target first and seeing if different XPath, id, or CSS element fixes the step. If after doing the above has issues and Selenium IDE is unable to play back all of the steps without any errors (even after checking the target) we may need to edit steps in the IDE or troubleshoot further before creating a check in Rigor. There is some helpful information about debugging Selenium IDE cases here. Still stuck? Email [email protected] for help.
Saving Selenium Script and Importing into Rigor
If Selenium IDE is able to replay all of the steps without any errors (steps will be highlighted in green) then we can save & import.
Save the Selenium IDE test case (command + S) or by clicking the disk icon in the upper right-hand corner.
Checks > Real Browser + New > From File
Select your file to import > Import Rigor will automatically create the steps that correspond with the Selenium test and remove any steps that are duplicated by our system.
Note: Please make sure to hit the test button at the end all steps, this will be another way to make sure this will run within the Rigor ecosystem. After all steps run without issue, we recommend that you save and manually trigger a live run as well. For further troubleshooting steps please refer !
View ArticleUnderthe 'Advanced' tab for Real Browser checks, there is a setting labeled Enforce SSL/TLS certificate validation. When enabled, this feature is used to enforce the validation of Expired, Invalid Hostname, or Untrusted Issuer on SSL/TLS certificates.
Note: When testing pre-production environments that may have self-signed or invalid certificates, it's best to leave this feature disabled.
View ArticleRigors Real Browser Checks step builder helps us build user flows or multi-step checks to test. While many basic interactions such as Click or Fill in field are available out-of-the-box, we may sometimes want to include custom steps that could require running JavaScript or saving custom variables from JavaScript that can be re-used in later steps. JavaScript is versatile and allows us to manipulate elements on a web page, send data, or make use of conditional logic.
In Rigors Real Browser Checks there are two ways to run JavaScript as part of a user flow:
Run JavaScript: this check step type simply runs a snippet of JavaScript code, or
Save return value from JavaScript: this check step type runs JavaScript code AND saves a variable returned from that code
Saving JavaScript Return Values to Custom Variables
Sometimes, it may be necessary to save values from JavaScript functions so that these values can be used for other steps in a check. Examples of such values include dates, timestamps, randomized data, or unique identifiers.
To do so, select Save return value from javascript.
In the input field with the placeholder text value enter the JavaScript youd like to run that includes a line of code to return a value.
Then, this custom variable can be accessed in other steps using this format: **
Common Use Cases for JavaScript in Real Browser Checks
Here are some examples of how Javascript can be used in your checks:
Clicking on an element that is not directly accessible by the browser.
document.getElementById("hiddenBtn").click();
Executing instructions based on conditional statements.
if(document.getElementById("overlay") != undefined){
document.getElementById("overlayCloseBtn").click();
}
Sending data via a POST request.
$.post("demo.asp", //jQuery
{
first: "John",
last: "Doe"
}
);
Running custom functions on a webpage
pageInit();
Tip:
Many Javascript functions like $.ajax and setTimeout are run asynchronously, which means that they may not return data before the browser moves on to the next step. If you make use of such functions when setting custom variables, your custom variables may not return the data you expect.
View ArticleGlobal Variables can be defined in a single place and reused across checks.
Form
The Global Variables form is accessible from the administrator menu at the top right of the screen.
Description (optional): A description can be useful to explain the purpose of the variable for future reference.
Key: The name of the variable. You will use this to access your variable within a check.
Value: The value that will replace the variable when the check is run.
Using Global Variables
Global Variables can be used like any other variables in an API Check. They can be accessed via the env namespace. For example, if you have a Global Variable with key rigor_url, you can access it within a check with {{env.rigor_url}}.
If you need a reminder of what Global Variables are available while creating or editing an API Check, hints are available by clicking the i icon at the top right of the check. You can view all available built-in variables and Global Variables. Experienced users might find the keyboard shortcut useful: [SHIFT] + [i]. Please note that this help panel does not show request variables or custom variables that have been defined in the check.
View ArticleIf were using web analytics applications like Google Analytics or Omniture we may want to ensure Rigors automated testsdon'timpact analytics data.
There are the main ways to exclude Rigor from analytics:
Exclude Rigors IPs or DNS from our third-party analytics with filter settings (e.g. Exclude internal traffic from Google Analytics with view filters).
Exclude analytics files from loading when Rigor runs Real Browser checks
We can select excluded files from Rigors preset options or add custom exclusion rules.
Uptime Checks, Content Checks, and Benchmark Checks do not run JavaScript and will not affect web analytics that relies on JavaScript.
Note: If wed like to see the impact of our web analytics service on our sites performance we should use IP addresses to filter within our analytics settings instead of adding Excluded Files in the Rigor app.
DNS exclusion in Google Analytics
Follow Google's instructions to create a new filter for your view.
Leave the Filter Type as Predefined.
Click the Select filter type drop-down menu and select Exclude.
Click the Select source or destination drop-down menu and select traffic from the IP addresses.
Click the Select expression drop-down menu and select the appropriate expression.
Enter Rigor's regular expression rigor\.com. See the screenshot below to confirm all settings.
View ArticleThe Rigor Optimization product can post a notification of up to 10 distinct URLs when a performance snapshot has completed processing. Snapshot complete webhook notifications can be configured both at the account level and per-snapshot.
Account Level Webhooks
Webhooks enabled at the account level will fire uniformly for all snapshots created in the account by any user. To enable, you must be have Client Administrator access to your account.
To configure:
Visit the Settings page, then select Account Settings.
On the top-right corner, you will see a section titled Account Configuration. Click the Edit button.
Get Snapshot Detail API
Inside the edit page, click Add URL to add a new webhook notification. You can add up to 10 distinct URLs to receive notifications.
You can also optionally specify a Custom Key value to be returned in the payload of every webhook notification (See the Get Snapshot Detail API for the format of the postback payload). (Note: if you can also override this value on a per-snapshot basis to reconcile with your internal systems using the API, see the Per-Snapshot Webhooks section below).
Finally, you can add any number of HTTP headers to be included in the response payload, including any authorization tokens you may require for secure access.
Per-Snapshot Webhooks
If you require a finder degree of control over when and how your webhook notifications are sent, per-snapshot webhooks are the way to go. To utilize, you will need to use the Create New Snapshot API call to initiate a new snapshot.
Inside the POST payload to the Create New Snapshot call, you can add an optional webhook element as shown below.
{ "webhook": { "urls": [ "url1", "url2", ... ], "custom_key": "snapshot_custom_key", "headers": { "header1": "value1", "header1": "value2", ... } }}
All fields are optional and can be omitted, although you will need to supply at least one valid URL to activate the webhook.
Per-snapshot webhooks will override and replace any account level webhooks for just that one snapshot. Additionally, you can use per-snapshot webhooks without configuring account level webhooks.
Per-snapshot webhooks are especially useful if you want to reconcile a unique identifier in your internal system to a specific snapshot in the Rigor Optimization product. Additionally, per-snapshot webhooks allow you to enforce a stronger degree of security by supplying time expired authentication tokens in your postback payloads.
Webhook Payload
For both types of webhooks, a POST notification will be sent to the URL(s) specified in the webhook configuration when the snapshot completes (success or failure). The body posted will be in JSON format according to the specification described in the call.
The custom_key property returned in that payload will be set to the Custom Key value specified according to your webhook configuration, or null if no value was set.
View ArticleTroubleshooting User Timings
There are a few reasons why a user timing may not appear in Rigor. Test in your own browser to make sure that the mark or measure is logged as you expect before continuing:
The user timing you're looking for occurs after Fully Loaded Time
Rigor collects data for user timings at Fully Loaded Time, so metrics that are logged afterare not recorded. This also means user timings that are logged as the result of a user interaction like clicking a button or filling in a form are not recorded.
To check if this is the reason why your metric is not appearing in Rigor, you can download the HAR file for the check in question via the run results.
If the HAR does not include the metric you're looking for, Rigor was not able to record that metric. If the HAR does include the metric, continue reading for other possible explanations.
Your check has reached the limit of 100 user timings
Rigor stores the first 100 user timings logged per check for trending, alerting, and aggregating. If a user timing is missing in Rigor, your check may have reached this limit.
This can happen when:
Your site logs more than 100 unique user timings during a typical page load
Your CDN logs additional metrics. Some CDNs add timestamps to metric names, which can quickly add towards the limit
You have changed your check configuration to monitor a new site with different metrics, or you have changed the names of your user timings. This means the list of metrics Rigor is looking for is out of date
To learn more about how to create user timings for Rigor to collect, check out this documentation from Mozilla.
If you would like to reset or replace the user timings Rigor is logging, please reach out to [email protected].
View ArticleEvery Real Browser, Uptime, and API check requires setting a primary user or group to receive notifications.
In some cases, it becomes necessary to escalate alerts to other users or groups if the initial alert to the primary user goes unacknowledged.
You can configure Escalations on the Notifications tab:
Edit check > Notifications
Scroll down to Escalations > + Add
Select an amount of time (e. if you'd like to notify a Next Level Ops Team when the initial alert is unacknowledged for 15 minutes, select After 15 minutes)
Check the box for Repeat only if youd like to resend alerts at the set frequency until the alert is acknowledged by a user
Specify escalations to send at any time or only during a specific time window
Select a user or group to receive the escalated alert
Define how to send the alert - via email, phone, or text message
Notes:
Escalations are based on the original alert time and not cumulative. If, for example, you set two escalations to different users or groups for the same timeframe, those two users or groups will receive escalated alerts at the same time.
Escalations only receive Failure alerts, not 'Back Online' ones
View ArticleIf a check has multiple steps and monitors multiple pages, you can use Threshold Monitors to set thresholds on any metrics that Rigor collects on individual pages within a user flow. If any of these pages do not meet the set threshold, the whole check will be marked as a failure and alert accordingly.
Thresholds can be set based on timing, resource count and size, and error count.
By default, you will automatically receive an alert for any check that takes longer than 118 seconds. Threshold Monitors enable you to be proactive and implement checks based on thresholds you create to align your site performance with key performance indicators (KPIs) for your specific business needs.
To Set a Threshold Monitor:
Create or edit a Real Browser Check
Go to the 'Advanced' tab and add a Threshold Monitor
Select the metric, the threshold type, and the value you want to set
Specify what URL of the page you want to monitor
Best Practices
Avoid adding too many Threshold Monitors to one check. Instead, create a new check for each set of Threshold Monitors
Make sure that you do not add Threshold Monitors that are mutually exclusive. For example, it's best not to create one that ensures that the page load time is greater than 200 milliseconds and one that ensures it is always less than 100 milliseconds
Proactively check on how resources are causing page bloat with size & count thresholds
Set Total errors to 0 to be alerted whenever there is a resource that does not load
View ArticleReal Browser and Uptime Checks monitor:
Average Response Time
Slowest Response Time
Fastest Response Time
When creating an Uptime or Real Browser Check, visit 'Advanced' settings to set a response time monitor.
If the Real Browser or Uptime check has multiple steps, the response time represents the sum of the Fully Loaded time for each page accessed during the check.So for a single-page Real Browser check, Response Time is the same as Fully Loaded time.
By default, Rigor deems any check that takes longer than 118 seconds to load as a failure and alerts according to the specified notification settings.
You can use a custom Response Time Monitor to define a threshold. If a checks response time exceeds the time specified in the Response Time Monitor, the check will be marked as a failure and alert accordingly.
Should I set a response time monitor?
Response time monitors affect alerting, making alerts and notifications more aggressive and proactive based on the time thresholds we set. Alerts then affect the Uptime percentage reported in Daily, Weekly, or Monthly performance reports.
As a best practice, you should only enable custom Response Time monitors if you have a good idea of your baseline Response Time and if you want to be notified any time the site takes longer than that baseline
For example, if youre operating around an SLA that your site must load in less than 4 seconds at all times you should consider enabling a custom Response Time monitor of 4000 ms
If you know your average Response Time is 15 seconds but youdon'tneed to take action if the site loads slower than 15 seconds, you should not enable a Response Time monitor. You can rely on reports in Rigor to see how often your sites Response Time is above average
View ArticleComparison Reports are a great way to compare multiple Real Browser checks against a baseline so that you can track and trend over time. You can compare any of the metrics We capture for Real Browser checks, such as Onload, Visually Complete, or Start Render. Some popular use cases for this report would be comparing third party vs. no third party, production vs. staging, or desktop vs. mobile.
In the report, you can export everything as a CSV and view your comparisons in a variety of ways, such as:
By location, country, world region
Over a designated time period
By specific pages (i.e. all homepages)
enter the Performance KPIs report
How to Create a Comparison Report
In the side menu, click Reports > New > +Comparison Report
Name the report
Click +Add Real Browser Check to select as many checks as you want to compare
Optional: Select a check to be the baseline (recommended)
Optional: If you have long check names, you can give them "nicknames" to make for easier viewing in the report(recommended)
Select a Date Range
Select a Metric to compare. You can find our metrics glossary here
Select an aggregate (i.e. Median, Average, Max, etc.)
Optional: Select locations specific to show. By default, well show all locations from which the checks ran.
How to Read a Comparison Report
Metrics Overview table: A high-level overview of your data
Daily Overall Performance:Compare checks by time. If a baseline is selected, youll see the % differences below the chart.
In the below example, excluding 3rd parties reduces the visually complete time by 69.21%.The calculation for % difference is:(value - baseline_value) / baseline_value * 100
Click into a data point on the chart to : From there, you can drill down to a specific run or waterfall chart and download a CSV of run-level data.
View Article
