Contrast Security FAQs

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

.NET Agent

Sub Category

Installation

Question

Is it possible to prevent an IIS restart following installation of the Contrast .NET agent?

Answer

By default, IIS will be restarted following installation of the Contrast .NET agent to allow the agent to instrument the relevant application pools. However, it's possible to prevent this restart, by passing the following property to the installer: SUPPRESS_SERVICE_START=1.

Your resultant command to run the installer would look similar to the following for an attended install:

ContrastSetup.exe SUPPRESS_SERVICE_START=1

Alternatively, it would be similar to the following for an unattended install:

ContrastSetup.exe -s -norestart SUPPRESS_SERVICE_START=1

You can find more information on this and the .NET agents other command line options in the following OpenDocs article: .NET Installation > Command Line Options

License Type

On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Configuration

Objective

In the event when Contrast Teamserver property files need to be updated there are two ways to accomplish this.

For simple changes to the properties files, it is best to use the encrypted properties editoras outlined in our Open docs article.

When using theencrypted properties editoris not feasible due to the sheer amount of updates needed. For instance, automating changes with multiple Teamserver Nodes in a cluster, you can create .cleartext versions of the .properties files and place them in the /data/conf/ folder. When Contrast is started it will ingest these .cleartext files, by replacing the current .properties file with the new .cleartext version and then removing the .cleartext file. The following example illustrates this method using the saml.properties file. Steps: 1. Use the encrypted properties editor to extract the values out of the properties file.

authenticator.saml.keystore.path : /path/to/jks.jksauthenticator.saml.secret.url : authenticator.saml.keystore.default.key : default_keystoreauthenticator.saml.keystore.passwordMap : keystore=passauthenticator.saml.keystore.password : keystore_passauthenticator.saml.url : http://acme.local:8080/Contrast

2. Create a new file with the same name +.cleartext saml.properties.cleartext 3. Paste in the contents of those value. 4. Modify the values as needed and save the file. 5. Make sure to set the correct ownership on the .cleartext files to match the rest in the folder contents. Typically these are owned by the contrast_service group and user. chowncontrast_service:contrast_service *.cleartext 6. Restart the contrast server All the .cleartext files will start to disappear as the properties files are updated&encrypted. You can check the logs to see if this was done by looking for the following entry.

INFO (DataInitializationListener.java:671) Updated /opt/contrast/data/conf/saml.properties with /opt/contrast/data/conf/saml.properties.cleartext

Note! Teamserver requires a functional database.properties to start. Before updates to this cleartext file are made, Tomcat checks that it has a valid connection to the MySQL database.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Ruby Agent

Sub Category

Configuration & Connectivity

There are two primary components of the Ruby agent: theagentand aserviceused to communicate to the Contrast Server. For more information on the Ruby agent, see our Open Docs site.

If the Contrast Ruby agent is having trouble connecting with the service verify the following details:

The service must be running

The port for the service and the ruby agent must match

The ruby agent is loading its configuration file

The connection is actually established, but the logging level is set too high.

Details:

The ruby agent runs within the Ruby on Rails application's thread so all interactions with the Contrast UI require that the Ruby agent to forward information about events to the Contrast Service. In addition, the service is responsible for reading the server features and application settings from Contrast UI and forwarding them to the agent. If the communication between the service and the agent is not established the web application will still run but it will not be protected from attack or report vulnerabilities.

Diagnosing that the Agent cannot connect to the Service:

If the agent is correctly installed but unable to connect to the service, when the application starts there will be an error in the contrast log file.

2017-10-30 15:43:32,892 [Contrast Security] DEBUG - Sending 1 messages (current=Contrast::Api::Dtm::ApplicationCreation)2017-10-30 15:43:32,892 [Contrast Security] DEBUG - Marshaled data length 186 bytes2017-10-30 15:43:32,892 [Contrast Security] ERROR - Client threw exception communicating with service :: Connection refused - connect(2)for"127.0.0.1"port 30556

Verifying that the Service is running:

The ruby service should be identifiable as Contrast-Service in the process list. The Ruby agent is bundled with the service. The Ruby agent attempts to start the service when the application starts up. The agent adds three Rake tasks to the application:

rake contrast:service:statusthis checks for a process named Contrast-Service running on the local server.

rake contrast:service:startif the service is not found, this tasks attempts to start the service that was bundled with the agent. It uses the common config entries that the agent has so it will use the host and port values that the agent expects.

rake contrast:service:startthis attempts to kill a running Contrast-Service process. Note that the agent will attempt to restart the service on a subsequent request so this is primarily useful as a restart mechanism.

Verifying Service and Agent port numbers:

Both the agent and the service need the following sections in their configuration file:

agent:service:host: 127.0.0.1port: 30555 # any valid port number is allowed

Verifying the Service Logging level:

The default logging level for the service is ERROR. If you see a log filecontrast_service.logthat is empty it may be because the log level is too high. To update the service log level:

Contrast Service Logging

agent:service:logging:path: contrast_service.loglevel: DEBUG# DEBUG, INFO, WARN, ERROR

Verifying that the Agent is loading the Configuration YAML file:

If the agent can not find a configuration file it will not prevent the application from running but it will output a warning to STDOUT.

~/testing/rails5/app_with_scaffold:rails-5.0.0 [!?]$ rails s=> Booting Puma=> Rails 5.0.6 application startingindevelopment on http://localhost:3000=> Run `rails server -h`formorestartup optionsNo configurationfilefoundincontrast_security.yml, contrast_security.yaml, config/contrast_security.yml, config/contrast_security.yaml,/etc/contrast_security.yml,/etc/contrast_security.yamlPuma startinginsingle mode...* Version 3.10.0 (ruby 2.4.1-p111), codename: Russell's Teapot* Min threads: 5, max threads: 5* Environment: development* Listening on tcp://0.0.0.0:3000Use Ctrl-C to stop

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Ruby Agent

Sub Category

Troubleshooting

Objective

In rare scenarios, bad instrumentation causes a web server process to crash or a specific page to error out. If you ever encounter a crash or error caused by Contrast, please report the error via the Submit a Requestlink at the bottom of this page.

If possible, follow the steps below to gather agent logs and process dumps; this additional information is vital to reproducing and fixing these types of bugs.

Process

Agent Logs Directory

The Ruby agent logs information based on the configuration provided by the YAML configuration. By default, agent logs are written to the current working directory under the name contrast_agent.log.

You can change which information is logged by changing the logging level in the Ruby agent configuration.

Types of Bugs

There are two primary types of agent bugs for which Contrast needs to gather logs and other information:

Process Crash

Unhandled Managed Exception

Process Crash Bugs

Verify that the web server process crashed

Check your scenario against the following indicators to confirm that the web server process crashed.

The web application is unresponsive after installing the Ruby agent.

The console log for the process has a Segmentation Fault. This will probably result in an error message containing [BUG] Segmentation fault at: followed by a memory address.

Once you confirm that the observed bug is a process crash, you're ready to gather information to file a support request.

Gather information on the process crash

Complete the following steps to gather information to send to Contrast.

Add the following configuration to the YAML file to enable verbose logging and logging of every interaction between the agent and the Contrast Service.

agent:

logger: level: DEBUG path: ./contrast_agent_debug.log service: logger: level: DEBUG path: ./contrast_service_debug.log

If your application is running in an SaaS/CaaS environment where the logs are not easily accessible, logging can be sent to STDOUT instead by setting path: /dev/stdout

Verify that the Contrast-Service is no longer running. If the Service is still running in your environment, terminate it manually.

In Windows, you can run tasklist /fi "imagename eq Contrast-Service.exe".

In Linux or Mac, you can run ps aux | grep Contrast-Service.

Clear your console.

Exercise the application to reproduce the crash.

Once you've reproduced the crash, gather the following items and include them in your bug report:

Agent logs: The contrast_agent_debug.logand contrast_service_debug.logfiles; both files should be in the application's root directory.

Application logs:The logs from your application run, which are most likely in the logsdirectory of the application.

Console logs:The results of the process crash in your terminal.

State logs:While the agent logs capture much of the operating system information, it's helpful to know what, if any, third-party Gems are being used. Even though it's not strictly a log, include your Gemfile.lockas part of the reproduction information collected.

You should restore your logging levels to their original settings at the end of this process.

Unhandled Managed Exception or Page Error Bugs

Verify an unhandled exception

The process described in the previous section can also help the Ruby agent team resolve issues such as application errors caused by the Ruby agent. Use the following indicators to determine if the Ruby agent is causing an application error.

You've observed the application working normally without the agent.

You've observed a page of the application "crashing" (returning a 500 error) under the agent.

There are no errors in the application logs in the logsdirectory.

Gather information on the exception

Follow the process outlined in the previous section to Gather information on the process crash, but omit the Console logsstep.

Additional Logging

In rare cases, additional information may be required to properly diagnose the root cause of the undesired behavior. To facilitate this, the Ruby agent provides a convenience wrapper around Ruby's built in heap dump utility. To take a heap dump of your application with the agent enabled, add the following to your YAML configuration and restart the application. A background thread is then spawned and begin to automatically take heap dumps.

agent: heap_dump: enable: true path: ./contrast_heap_dumps

The results are saved to the contrast_heap_dumpsdirectory in the root of your application. By default, the Ruby agent takes five heap dumps spanning 10 seconds with a 10 second pause between each. You can find additional configurations in the Ruby agent configuration articles.

Note:Taking these measurements will adversely impact performance; you should restore your configuration to the original settings at the end of this process.

Other Bugs

If you encountered a bug other than a process crash or unhandled exception - such as a false positive found by the agent - please contact us via the Submit a Requestlink at the bottom of this page. Contrast doesn't usually need console error logs, but debug-level logs and a detailed description of the problem are very helpful when it's time to fix these bugs.

Queue sizes

TeamServer uses message queues and background processing for many tasks. Monitoring the sizes of the message queues may yield useful information on activity and warn of an influx of data or the need for more resources.

Outputting info about this is controlled by editing $CONTRAST_HOME/conf/servo.properties, which is an encrypted properties file, so to edit it, youll need to use the Encrypted Editor

Steps:

Run the encrypted editor:

$CONTRAST_HOME/bin/edit-properties -e $CONTRAST_HOME/esapi/ -f $CONTRAST_HOME/conf/servo.properties

Enter the name of the property to edit:

servo.activemq.observers

Enter a new value for servo.activemq.observers:

FILE

Save the changes (to all TeamServer nodes if using a distributed setup)

After a restart, queue status should be logged once a minute to $CONTRAST_HOME/data/logs/servo_activemq_<date>.log

Example format:

2019-05-15 12:27:32,082,Active,{Jmx.destinationName=queue.library.update, hostname=Contrast-MBP, ip=192.168.0.30},1

To change the logging interval, you can set the propertyservo.refresh to a positive integer, representing how many minutes between each output.

Recommendation: Collect and graph this data, in order to observe trends in your environment. Consider alerting when queue size is above some factor of normal.

Metric Logging

TeamServer can output metrics about a number of operational details; including, but not limited to database connection pool statistics, response times for API calls & methods, and JVM memory usage.

Outputting this info is controlled by editing$CONTRAST_HOME/conf/metrics.properties which may not exist in your current TeamServer setup.

Steps:

Create $CONTRAST_HOME/conf/metrics.properties (on each TeamServer if using a distributed setup)

The contents of that file should be as follows:metrics.enabled=true

metrics.collect.garbage=truemetrics.collect.memory=truemetrics.collect.threads=truemetrics.slf4j.enabled=truemetrics.slf4j.pollingPeriod=1

After the TeamServer process has been restarted, there will be new log entries from Slf4jReporter in $CONTRAST_HOME/data/logs/contrast.log with key value pairs.

Some of these are GUAGE metrics, e.g. those detailing memory usage; some are HISTOGRAM metrics, e.g. those detailing database connection pool statistics; and some are TIMER metrics, detailing how long it took for a certain task to complete.(More details are available on the types of metrics available here.)

As with the queue statistics detailed above, our recommendation is to collect and graph this data in order to gather statistics about normal operation of your TeamServer(s).

In our SaaS environment, these are sent off to Graphite with the following additional configuration in metrics.properties:

metrics.graphite.enabled=truemetrics.graphite.host=localhostmetrics.graphite.port=2003metrics.graphite.server=UDPmetrics.graphite.protocol=UDPmetrics.hostname=app

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

.NET Agent

Sub Category

Troubleshooting

Objective

In rare scenarios, bad instrumentation causes a web server process to crash or a specific page to error out. If you ever encounter a crash or error caused by Contrast, please report the error via the Submit a Requestlink at the bottom of this page. If possible, follow the steps below to gather agent logs and process dumps; this additional information is vital to reproducing and fixing these types of bugs.

Process

Agent Logs Directory

The .NET agent logs information to the LOGSdirectory within C:\ProgramData\Contrast\dotnet\, the Windows 2008/2012/2016 ProgramDatadirectory. Depending on the setup of the Windows profile and folder view settings, the directories may be hidden. If so, paste the paths into the Windows Explorer location; you may need to replace the drive letter Cwith D.

You can change which information is logged by setting the logging level in yourcontrast_security.yaml file to TRACE:

agent: logger: level: TRACE dotnet: debug: log_method_sigs: true log_modified_il: true

Types of Bugs

There are two primary types of agent bugs for which Contrast needs to gather logs and other information:

Process Crash

Unhandled Managed Exception/Page Error/500

Process Crash Bugs

Verify that the web server process crashed

Check your scenario against the following indicators to confirm that the web server process crashed.

The web application is unresponsive after installing the .NET agent.

The Windows Event Log (Event Viewer > Windows Logs > Application) has Error entries for the .NET Runtime and Application Error.

The ".NET Runtime" error has details such as:

Application: w3wp.exeFramework Version: v4.0.30319Description: The process was terminated due to an internal error in the .NET Runtime at IP XXXXXXXXX with exit code YYYYYYY

The "Application Error" entry has details similar to:

Faulting application name: w3wp.exe, version: 8.5.9600.16384, time stamp: 0x5215df96

Faulting module name: clr.dll, version: 4.7.2114.0, time stamp: 0x59a63e48

Exception code: 0xc0000005

Fault offset: 0x00000000002ff61c

Faulting process id: 0x3724

Faulting application start time: 0x01d337d711f21e68

Faulting application path: c:\windows\system32\inetsrv\w3wp.exe

Faulting module path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll

Report Id: 4fc99650-a3ca-11e7-80e8-005056bd4248

Once you confirm that the observed bug is a process crash, you're ready to gather information to file a bug.

Gather information on the process crash

Complete the following steps to gather information to send to Contrast.

Set up the ProcDump utility to capture crash dump.

Download current version of ProcDump from the Microsoft documentation site to the Windows server with the agent.

From an administrator command prompt:

md c:\dumps

procdump.exe -ma -i c:\dumps

Install the latest .NET agent.

Stop the .NET agent service.

Enable additional logging.

Start > Notepad > Right click > Run as Administrator

File > Open >C:\ProgramData\Contrast\dotnet\contrast_security.yaml

Add the following configuration to the yaml file to enable verbose logging and logging of every method JIT-compiled by the CLR:

agent: logger: level: TRACE dotnet: debug: log_method_sigs: true log_modified_il: true

Start the .NET agent service.

Exercise the application to reproduce the crash.

Once you've reproduced the crash, gather the following items and include them in your bug report:

Agent Logs: All files in the agent log directory,C:\ProgramData\Contrast\dotnet\LOGS; right click on theLOGS folder > Send To > Compressed (zip) folder.

Windows Event Log:Event Viewer > Windows Logs > Application > Save All Events As > "MyEvents.evtx"

Crash Dumps: Create a zip file of each w3wp process dump file inC:\dumps(e.g.,w3wp.exe_171002_151601.dmp). Dump files can be quite large.

You can then uninstall ProcDump withC:>procdump.exe -u.

Unhandled Managed Exception or Page Error Bugs

Verify an unhandled exception

The above process also helps the .NET engineering team resolve issues such as application errors caused by the .NET agent. Use the following indicators to determine if the .NET agent is causing an application error.

You've observed the application working normally without the agent.

You've observed a page of the application "crashing" (returning a 500 error) under the agent.

There are no errors for ."NET Runtime" and "Application Error" in the Windows Event Log.

Theremay be warningsfor "ASP.NET" in the Windows Event Log. The warning should look similar to the following:

Source: ASP.NET 4.0.30319.0

Date: 10/9/2017 9:22:46 AM

Event ID: 1309

Task Category: Web Event

Level: Warning

Keywords: Classic

User: N/A

Computer: FOO.COMPUTER.COM

Description:

Event code: 3005

Event message: An unhandled exception has occurred.

Event time: 09/10/2017 9:22:46 AM

Event time (UTC): 09/10/2017 2:22:46 PM

Event ID: f706787c1f1247e6a87b777a90413c3d

Event sequence: 9

Event occurrence: 1

Event detail code: 0

Application information:

Application domain: /LM/W3SVC/1/ROOT/FOO-1-131520325424796488

Trust level: Full

Application Virtual Path: /Foo

Application Path: E:\MCMSFiles\inetpub\wwwroot\Foo\

Machine name: FOO

Process information:

Process ID: 176840

Process name: w3wp.exe

Account name: System

Exception information:

Exception type: ArgumentOutOfRangeException

Exception message: Index was out of range. Must be non-negative and less than the size of the collection.

Parameter name: index

at System.Collections.ArrayList.get_Item(Int32 index)

at System.Web.UI.WebControls.DataListItemCollection.get_Item(Int32 index)

at Fabrikam.SetTabCount(Int32 index, NullableInt32 summaryCount) in C:\Foo\Fabrikam.aspx.cs:line 1686

at Fabrikam.GetSummaryCounts() in C:\Foo\Fabrikam.aspx.cs:line 1468

at Fabrikam.OnPreRender(EventArgs e) in C:\Foo\Fabrikam.aspx.cs:line 549

at System.Web.UI.Control.PreRenderRecursiveInternal()

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

Request information:

Request URL: https://www.foo-staging.com/Foo/Fabrikam.aspx

Request path: /Foo/Fabrikam.aspx

User host address: 1.2.3.4

User: msteeber

Is authenticated: True

Authentication Type:

Thread account name: System

Thread information:

Thread ID: 19

Thread account name: System

Is impersonating: False

Stack trace: at System.Collections.ArrayList.get_Item(Int32 index)

at System.Web.UI.WebControls.DataListItemCollection.get_Item(Int32 index)

at Fabrikam.SetTabCount(Int32 index, NullableInt32 summaryCount) in C:\Foo\Fabrikam.aspx.cs:line 1686

at Fabrikam.GetSummaryCounts() in C:\Foo\Fabrikam.aspx.cs:line 1468

at Fabrikam.OnPreRender(EventArgs e) in C:\Foo\Fabrikam.aspx.cs:line 549

at System.Web.UI.Control.PreRenderRecursiveInternal()

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

As the process hasn't crashed, ProcDump won't capture process dumps. Instead, you must gather the process dump manually by completing the following steps.

Find the Process ID of the worker process that you need.

IIS Manager > Worker Processes: Find the "Application Pool Name" you need, and take note of the value in the "Process Id".

From an administrator command prompt, replaceNNNNNwith the process ID from the previous step.

C:\>procdump -ma NNNNN

Follow a similar process to gather agent logs, windows events and process dumps to include with your bug report.

Other Bugs

If you encountered a bug other than a process crash or unhandled exception - maybe the .NET Tray has an inaccurate state, or the agent found a false positive - please contact us via the Submit a Requestlink at the bottom of this page. Contrast doesn't usually need process dumps, but trace-level logs and a detailed description of the problem are very helpful when it's time to fix these bugs.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Python Agent

Sub Category

Installation

Issues

For version 2.3.0 of the agent and earlier, when starting the bundled service using the host/port configuration.

It is necessary for the port value to be given explicitly. A useful default is 30555. An alternative is to use the socket configuration instead.

There may be installation issues when using older versions of pip.

It may be useful to try to update to the latest version by running pip install

--upgrade pip.

There may be conflicts between the versions of app dependencies and agent dependencies.

In these cases, it is important to make sure that all app dependencies are installed before installing the agent.

On Linux, the dependency psutil requires linux/ethtool.h to be installed, so it may be necessary to install the ethtool package for your system if this fails.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast SDK

Sub Category

.NET SDK

Issue

An exception is being thrown while using the .NET SDK.

Causes & Resolutions

The following list details possible exceptions along with their causes and resolutions:

Exception

System.ArgumentException

Cause

This error may be thrown for a couple of reasons:

A required parameter for requests is missing (username, service key, api key)

The URL parameter has an invalid URI value

Resolution

Check to make sure all parameters have been supplied and are valid.

Exception

System.AggregateException

Cause

This error is often thrown when the request cannot resolve or reach the host name supplied.

Resolution

Check to ensure that the hostname of the Contrast UI can be resolved successfully on the client system.

Exception

contrast_rest_dotnet.ResourceNotFoundException

Cause

This error is thrown whenever HTTP status code 404 is returned by the host.

Resolution

Ensure the resource requested is valid.

Exception

contrast_rest_dotnet.ForbiddenException

Cause

This error can be thrown due to insufficient permissions or the credentials supplied are invalid.

Resolution

Ensure that the user's credentials are valid and also the user has the required access to perform the action requested.

Exception

contrast_rest_dotnet.ContrastApiException

Cause

This exception will be thrown if the Contrast UI returns an unrecognized response to the SDK (usually in the form of a 500 Internal Server Error).

Resolution

If this exception presents itself too frequently while using the same function,please create an issue in our GitHub repository for us to take a closer look.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent | .NET Agent | Node.js Agent | Ruby Agent

Sub Category

Question

Does Contrast change application data?

Answer

No.

Contrast is a passive technology. Contrast changes your bytecode at runtime to install our sensors, but doesn't change your data or how your code flows.

License Type

On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Issue

If you're seeing errors in your log file related to aqueue, it's likely that the storage of your ActiveMQ database is causing the problem. You might observe any of the following in yourcontrast.logfile:

"Failed to page in more queue" messages

queue://queue.activity.app

Queue.java

A link to http://activemq.apache.org/producer-flow-control.html

Any reference toActiveMQwithERRORlog level

"Failed to page in more queue messages"java.lang.NegativeArraySizeExceptionthrown byactivemq.broker

"Failed to fill batch" caused byNegativeArraySizeException.

"Problem retrieving message for browse" thrown byQueue.java:1117

When the disk first fills up, you might start to see error messages incontrast.logevery 30 seconds. Example error message:

Main:store:queue://queue/activity.app:store) ... percentUsage=102% ... Persistent store is Full, 100% of SOME#. Stopping producer

Cause

At this point, you should check to see why Contrast thinks that your disk is full. It's likely that ActiveMQ surpassed its allotted storage; but, in some cases, your disk could be full.

Resolution

Before clearing any files, stop the Contrast process, and then begin the process of clearing out unnecessary files. If you see these ActiveMQ errors, you can resolve this in just a few steps:

Confirm the Contrast isn't running.

Navigate to$CONTRAST_HOME/data/activemq

Delete everything in this folder (both the data and index folders).

Start Contrast.

License Type

On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Issue

Base_Directory Error When Starting Contrast

Cause

If you receive an error that BASE_DIRECTORY must be set when you try to start Contrast, it's an indication that you have aCATALINA_HOMEdirectory already set in your system environment. Some products that use Tomcat require you to set this directory in the environment; but, unfortunately, it steps on our startup scripts.

Resolution

The simplest way to resolve this error is to edit the$CONTRAST_HOME/server/bin/start-teamserver.shfile, and set theCATALINA_HOMEto Contrast's Tomcat directory by adding:

# Address hard-coded environment variable for CATALINA_HOME - Set this directory to your# $CONTRAST_HOME/server directoryset CATALINA_HOME=/opt/contrast/server

License Type

On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Issue

If you're seeing an error in your$CONTRAST_HOME/data/logs/contrast.logthat's similar to the example below:

SEVERE: Servlet.service() for servlet [spring] in context with path [/Contrast] threw exception [Request processing failed; nested exception is java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: unable to create new native thread] with root causejava.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:714)

Cause

You might be bumping up against the thread limit set by your operating system.

Resolution

To address this issue, increase the number of threads that the Contrast user can start in your/etc/security/limits.d/90-nproc.conffile.

Note:To modify a system configuration file, you must have root access or the ability to sudo. Consult your System Administrator for additional details.

You can add the line<contrast-username> soft nproc unlimitedto the end of the file and restart the contrast-server using the commandsudo service contrast-server restart.

License Type

On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Issue

Service Won't Start Due to Missing sudo/su

Cause

Thesudoand/orsucommands might not be available on some flavors of Linux. However, this is a simple issue to resolve.

For example, therunusercommand is the default on the default installation of Amazon Linux within AWS, but it isn't an option during the installation. While Contrast makes an effort to support as many environments as possible with our Enterprise On-Premises (EOP) product, it's impossible to predict every potential scenario and test it internally.

Resolution

You can resolve this issue by either installingsudoorsuon your target server. If that's not an option, you can edit the$CONTRAST_HOME/bin/contrast-server.initdscript directly and update it to reflect your environment.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI | Java Agent | .NET Agent | Node.js Agent | Ruby Agent | Contrast Plugins

Sub Category

Question

What information does Contrast capture?

Answer

Contrast collects a lot of application analytics like URLs, file paths,jarnames, and other relatively non-sensitive information. It also sends all the data related to security-relevant events.

If a method occurs that's an integral part of a vulnerability, all the aspects of that event - the object, return value, parameters and more - are sent back to the Contrast site. This process allows you to analyze the collected data and decide if any of it is sensitive.

Example:If a XSS vulnerability in the login code, the username and password will probably be sent back as part of the XSS trace. If Contrast is installed on aDevelopmentorQAenvironment, as recommended, it's not likely that any Production or sensitive data is involved.

You can also take advantage of our data masking features to limit exposure of sensitive data found in vulnerabilities or Protect events.

You can also reference our Privacy Policy on our main site under the Product Security section.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI | Java Agent | .NET Agent | Node.js Agent | Ruby Agent

Sub Category

Question

Why does Contrast report an increasing number of URLs in an application?

Answer

Contrast is anInteractive Application Security Testing (IAST)tool. Therefore, it's only aware of the sections of an application that it's seen.

To effectively monitor your application, the Contrast agent must be running on your server while the application is exercised. During this phase, Contrast reports the new URLs as it encounters them. The same process happens when new URLs are added to your application after the initial introduction of the agent. In most cases, the amount that the number of URLs grow over time scales down logarithmically, and the minor growth you'll see won't have a significant effect on your overall score.

License Type

SaaS & On-Premise

Agent Mode

Assess

Main Product Category

Contrast UI | Java Agent | .NET Agent | Node.js Agent | Ruby Agent

Sub Category

Why does my open-source library show up as unknown?

Contrast identifies libraries by their SHA-1 digest, and updates library definitions periodically. As a result, Contrast might not recognize new libraries when agents report them.

Note:If any library repackaging occurs forJavaclients, which WebSphere does by default, the digest is different. To prevent repackaging, you can add the following JVM system property:

-Dorg.eclipse.jst.j2ee.commonarchivecore.ignore.web.fragment=true

Why doesn't Contrast recognize my library as the most-recent version?

ForJavaclients, issues with library version recognition result from the way that Contrast data sources store information about a library's version. Some of the versions are formatted to include the year at the beginning, and so the sort method reads them as a more recent version. The problem should phase itself out as the industry is moving to a more normalized version convention.

How can I get my library file added to the known list of files?

If you're using a new, publicly available library file, your cache file may be out of date, and may be updated with the next release of Contrast. If you want to ensure that this is the case, please contact us, and provide the name and version of the library.

If you're using a publicly available library file released before the version of Contrast that you're running, please contact us, and provide the name and version of the library so that we can add it to the database.

If you're using a custom file, and would like it added to the known list, please contact us with information about the library file. To effectively add it to our database, Contrast needs a hash of the file as well as its name, version number, release date, and any known CVEs affecting it.

What if the library file I'm using is the most-recent version, but still several years old?

Contrast takes this into account when grading a library file, and reduces the impact of the file's age.

Why do my libraries have zero classes used?

Contrast doesn't know that a class has been used from aJavaor.NETlibrary file until it sees the class in your application. Further testing of your application should increase these numbers to give you a more accurate analysis of class usage.

License Type

On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI & Administration

Sub Category

Objective

Log Database Queries with Embedded MySQL

Enterprise-on-Premises (EOP) customers can use the embedded MySQL instance packaged with Contrast or integrate Contrast with a distributed MySQL instance. For customers leveraging the embedded MySQL instance, Contrast provides a configurable property file in$CONTRAST_HOME/data/conf/mysql.properties. This file is very similar to themy.cnffile that is leveraged natively within MySQL.

Process

Sample Properties File

Two logging flags are available: general logging and slow query logging.

General loggingis used to log every statement executed by MySQL. Customers who use this option should be very careful, as this log can grow to many 100s of gigabytes of storage over a very short window of time. Contrast recommends that you use general logging only for debugging purposes and that you disable it at all other times.

Slow query loggingis used to evaluate queries that take a particularly long period of time. MySQL monitors the SQL and measures the time it takes to execute. When enabled, this log grows from many megabytes to gigabytes depending on whether a system is very slow. Contrast recommends that you keep this option enabled, but that you pay close attention to the size of the log file over time.

An example from themysql.propertiesfile created with Contrast as part of an embedded installation/upgrade:

# The destination for general query log and slow query log outputlog_output=FILElog_queries_not_using_indexes=ONgeneral_log=OFFgeneral_log_file=${Contrast.Data.Log}/mysql.log# Log errors and startup messages to this filelog_error=${Contrast.Data.Log}/mysql_error.log# Whether the slow query log is enabled. "Slow" is determined by the value of the long_query_time variableslow_query_log=OFF#The name of the slow query log fileslow_query_log_file=${Contrast.Data.Log}/mysql-slow.log

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI | Java Agent | .NET Agent | Node.js Agent | Ruby Agent | Administration

Sub Category

Upgrade Agents

Enterprise-on-Premise (EOP) administrators can download a new agent from the Contrast Hub Site and copy the individual agent artifact(s) to a directory on the Contrast application. In the event an issue happens after copying files during configuration, review thecontrast.loglocated under$CONTRAST_HOME/logsfor more information about what's causing an issue within the Contrast application. A few troubleshooting techniques are outlined in the following sections.

Permissions Issue on the Agent

If this issue occurs, make sure to verify that the user who owns the$CONTRAST_HOMEdirectories and files happens to be the same owner of the agent files placed in the respective directories. Try to avoid copying files as the ROOT user or another named user. Make sure the ownership (contrast-service) and permissions (755) match other files and directories under$CONTRAST_HOME.

File Placed in the Wrong Directory

It may seem obvious to place the NodeJS agent in the Node directory and the same for other agents, but you should always double-check that the files are in the right directory. If the wrong files are placed in a particular directory, the server shouldn't have an issue. However, you won't be able to download the updated agent until the file is moved to the appropriate directory.

Size of Download Doesn't Match

Occasionally, downloads can become corrupt when transferring via download over HTTP/HTTPS as well as copying files from one machine to the next. If this happens, double-check the file size from the original source. If a file was placed on Contrast's download site but is corrupt, don't hesitate to log a ticket with our Support Team for remediation.

Agent Version Is Lower than the Last Contrast Agent

The service that handles agent updates is designed to look for the latest agent based on build number. If you place an older agent in this directory, the agent probably won't be accessible for download from the Contrast application.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Troubleshooting

Objective

In rare scenarios, bad instrumentation causes a web server process to crash or a specific page to error out. If you ever encounter a crash or error caused by Contrast, please report the error via the Submit a Requestlink at the bottom of this page.

If possible, follow the steps below to gather agent logs; this additional information is vital to reproducing and fixing these types of bugs.

Process

You can ask Contrast to log verbose information by doing one of the following.

System properties:

-Dagent.logger.path=/path/to/trace_contrast_agent.log -Dagent.logger.level=DEBUG

Contrast_security.yaml file:

agent: logger: path: /path/to/trace_contrast_agent.log level: DEBUG

EnvironmentalVariables:

CONTRAST__AGENT__LOGGER__PATH='/path/to/trace_contrast_agent.log'CONTRAST__AGENT__LOGGER__LEVEL='DEBUG'

If your application is running in an SaaS/CaaS environment where the logs are not easily accessible, logging can be sent to STDOUT instead by one of the following methods:

System properties:

-Dcontrast.agent.logger.stdout=true-Dagent.logger.level=DEBUG

Contrast_security.yaml file:

agent: logger: stdout: true level: DEBUG

Environment Variables:

CONTRAST__AGENT__LOGGER__LEVEL='DEBUG'CONTRAST__AGENT__LOGGER__STDOUT=true

These options will put Contrast into debug mode and log all output to the log file specified as the system property value. It will also prevent the log from rolling over into multiple files. The log file will start with some messages that look similar to this:

submit a ticket

These verbose logs will allow our support team to diagnose any issues you may encounter. To contact the Contrast Support Team, please to our online support portal.

License Type

On-Premise

Agent Mode

N/A

Main Product Category

Administration

Sub Category

Distributed Installation

Objective

Use an Amazon RDS database instance to host the Contrast database for a distributed EOP Installation.

Process

As documented here, the usual configuration for the Contrast database user is to grant all privileges using:

GRANT ALL PRIVILEGES ON *.* TO 'contrast'@'%';

With an RDS instance, even the admin user that is provided for managing your instance does not possess all privileges, so it will be impossible to configure the contrast user as suggested.

Instead, the maximum permission level you can set for the contrast user is as follows:

GRANT

SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, REFERENCES,

INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE,

REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE,

ALTER ROUTINE, CREATE USER, EVENT, TRIGGER ON *.* TO 'contrast'@'%' WITH GRANT

OPTION;

In addition to configuring the user as such, the following steps need also be taken to avoid errors:

Set the log_bin_trust_function_creators database parameter to 1 as described here.

Add the parameter noAccessToProcedureBodies=true to the jdbc.url setting in the database.properties file (details on how to edit this file are here ).

License Type

On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Installation

Issue

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)

Cause

In some fresh vanilla installations of Redhat or CentOS, the server could have an unresolvable hostname. During the installation process, Contrast detects the hostname for the server to set configuration parameters for how to connect to the database.

Example:A default CentOS 5.6 installation sets its hostname tocentos.localdomain, but a corresponding entry isn't added to the/etc/hostsfile.

If the hostname is unresolvable, the system defaults tolocalhost, which works for other areas of the system. However, the automated backup uses themysqldumptool that associates thelocalhosthostname with a file socket connection instead of a network socket connection.

Resolution

To resolve this error, set the database hostname to either a resolvable hostname or the IP address assigned to the server. This can be done using the encrypted properties editor tool that ships with the product.

$ cd /path/to/CONTRAST_HOME_DIR

$ bin/edit-properties -e data/esapi -f data/conf/database.properties

Once you're in the properties editor, type the name of the property that you would like to change. In this case, you want to update the value of thejdbc.hostto the new hostname or IP address assigned to the server. Once you update this value, hitQto quit out of the tool, selectYto save, then enter a comment for your change. You can verify the backup functions by running theCONTRAST_HOME_DIR/bin/db-backup.shscript manually (as thecontrast_serviceuser, or whatever user you designated during installation).

$ cd /path/to/<CONTRAST_HOME_DIR>

$ sudo -u contrast_service /bin/backup-db.sh

Reading Database Configuration

Ensuring Database Connectivity: Up

Backing up Schema: contrast to /data/backups/db/contrast.20150223.sql

License Type

On-Premise

Agent Mode

N/A

Main Product Category

Administration

Sub Category

Distributed Installation

Objective

Modify the size of the database connections pool used by the Contrast application and modify the corresponding Maximum Connections that the database server can support.

Background

The Contrast application uses a pool of JDBC connections to its database, the size of which defaults to 50. In addition, as of Version 3.6.9, an additional pool, also of 50 connections, is reserved for future use to improve reporting performance in later EOP versions. As a result of these defaults, 100 connections (from 3.6.9 onwards) or 50 connections (3.6.8 and earlier) will be made to the database on startup by each Contrast Application Server. In the case of a distributed installation with a single Contrast application server, the default MySQL database server Maximum Connections of 151 will be sufficient to accommodate the default pool sizes detailed above, however if multiple Contrast application server instances are deployed it will be necessary to modify either the pool size or the database server Maximum Connections to suit.

Process

To modify the Maximum Connections that the MySQL database server can support:

Locate the my.cnf file on the MySQL database server (often in the /etc or /etc/mysql folder on Linux distributions).

Edit my.cnf and set max_connections to a value that is larger than the combined connection pool sizes on each Contrast application server (for example, for 5 application servers each with a connection pool size of 50, then set max_connections to at least 251).

Restart the database server.

To modify the size of the JDBC connection pool on a Contrast application server:

Use the Encrypted Properties Editor to edit the database.properties file as follows:

To change only the size of the non-reporting-related JDBC connections pool, add a property named jdbc.maxPoolSize (note that these are case-sensitive).

To change the size of the reporting-related JDBC connections pool, you must add a property named reporting.jdbc.maxPoolSize but also for each existing property in the file named jdbc.<propertyname> you must create a corresponding reporting.jdbc.<propertyname> property (so you will end up with reporting.jdbc.type, reporting.jdbc.url etc.)

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Installation

Objective

Learn one way you can integrate the Java agent with your Docker infrastructure. In this example, we'll be instrumenting WebGoat, an executable .jar file deployed in a Linux Docker container.

Process

Create a folder for the application then change to that directory

mkdir ~/projects/docker/webgoat7.1/

cd ~/projects/docker/webgoat7.1/

Create a blank Dockerfile

touch Dockerfile

Edit the Dockerfile in a text editor and paste the following into it:NOTE: You'll need to edit the curl command on line 6 to point it to your own organization.

FROM anapsix/alpine-java:jdk8

ENV APP /

RUN apk update && apk add ca-certificates && update-ca-certificates && apk add openssl

RUN apk update; apk add curl

RUN wget https://github.com/WebGoat/WebGoat/releases/download/7.1/webgoat-container-7.1-exec.jar

RUN curl -X GET <Teamserver URL>/Contrast/api/ng/<OrganizationID> -H 'Authorization: <Authorization>' -H 'API-Key: <API KEY>' -o contrast.jar

WORKDIR $APP

EXPOSE 8080

CMD ["java","-javaagent:contrast.jar","-Dcontrast.agent.java.standalone_app_name=WebGoatDocker","-Dcontrast.server.name=DockerServer","-jar","webgoat-container-7.1-exec.jar"]

We are downloading the latest Java agent through a curl command and placing it in the same directory as our WebGoat springboot jarfile. For steps on creating a curl command, please refer to our documentation here.

The Java startup command contains the javaagent along with a few configuration properties to make sure the application shows up in the teamserver correctlyBuild the Dockerfile and create a new image

Build the Dockerfile and create a new image

docker build ~/projects/docker/webgoat7.1/

Verify the new image was created

docker images

You should see the following output, the image with 1526b2ff885c was just created with the above Docker build command.

http://localhost:8080/WebGoat

Create a container out of the image and run it

docker run -p 8080:8080 -t <Image ID>

In your browser, go to to make sure the container started up correctly and you can access the application

In the Contrast UI, you should see the application show up:

Along with the server:

Best Practices

Docker will cache layers to speed up subsequent builds of the same Dockerfile. This means the curl command will not be re-executed to download the latest version of the Contrast agent if nothing has changed in steps prior to the curl command.

In order to keep up-to-date with the latest version of the agent if using this approach, you should periodically re-build the entire image to ensure the latest versions are used. To do this, run docker build with the --no-cache and --pull switches:

docker build --no-cache --pull ~/projects/docker/webgoat7.1/

License Type

On-Premise (Pre 3.6.9)

Agent Mode

N/A

Main Product Category

Contrast UI

Sub Category

Installation

Note: As of 3.6.9 the Contrast UI installer configures this functionality by default so there is no need to set it up manually as described below.

Objective

By default, when installing the Contrast server on Linux, an init.d task is created which allows it to automatically startup as the OS starts up. However, some Linux distro's are moving to use systemd as the default means of starting/stopping services.

The following process shows how to create a systemd service for the Contrast server to allow it to be controlled with systemctl.

Process

We'll first need to create the following file at the given location:

/etc/systemd/system/contrast-server.service

The contents of this file will be:

[Unit]

Description=Contrast Server

[Service]

Type=forking

ExecStart=/opt/contrast/bin/contrast-server start

ExecStop=/opt/contrast/bin/contrast-server stop

User=contrast_serviceTimeoutSec=900

[Install]

WantedBy=multi-user.target

Please note: /opt/contrast is given here as an example of the Contrast server installation directory, it should be replaced with the actual install directory. Some Linux distributions don't allow for relative paths or variables to be used within systemd services.

Once in place, run the following command to load your new systemd service:

sudo systemctl daemon-reload

Finally, you can start and stop the Contrast server with the following commands:

sudo systemctl start contrast-server

sudo systemctl stop contrast-server

The Contrast server should also start when the OS starts.

License Type

SaaS & On-Premise

Agent Mode

Assess

Main Product Category

Contrast UI & Administration

Sub Category

Jira Integration

Issue

NPE (NULL Pointer Exceptions) are seen in Contrast log when creating Jira tickets from vulnerabilities logged in Contrast UI.

Unchecking all the boxes (for what details to send over to jira) will successfully create the issue.

Cause

This issue can be caused by changes made in Jira that have not yet been indexed.

Resolution

Check with Jira administration to see if there were any changes to the project or jira and reindexing has not yet been performed.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Connectivity

Issue

The following error is seen when the Java agent is attempting to make a secure connection to the Contrast UI.

Problem resolving features with com.contrastsecurity.agent.features.%javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-emptyCaused by: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty

Cause

There can be many causes for this error but generally it indicates an inability to access the truststore.

Resolution

This problem can occur when the JVM is unable to find or access the truststore being passed to it. The first step in diagnosing the issue is to check the following:

Is a custom truststore being supplied (-Djavax.net.ssl.truststore=)?

Is the file being pointed to accessible to the user running the JVM?

If this JVM property isn't in use, is the default Java truststore accessible to the user running the JVM and on disk?

You can read more on Oracle's site about truststore

You can also run a couple of tests with the Contrast Agent itself to help further identify the problem:

Using default truststore, run:

java -Djavax.net.debug=all - jar contrast.jar diagnostic

Using a custom truststore, run:

java -Djavax.net.debug=all - Djavax.net.ssl.trustStore=/opt/ssl/customTrustStore.jks -jar contrast.jar diagnostic

The output from these commands will confirm whether the agent is able to make a successful connection to the Contrast UI.

If the output of these commands indicates that the agent is unable to connect to the Contrast UI, please use the Submit a Request link below and the Contrast Support team will be able to assist further.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Application Management

Issue

Application shows up in the Contrast UI with (1) alongside it.

Cause

This is what's happening:

If an app is reported by the agent asbaa, but you change the display name in the UI tofoo- then an app is reported by the agent asfoo, it'll get calledfoo(1).

If an app is reported by the agent asfoo- then another app is reported by the agent asfoo, they both report tofoo.

It basically comes down to what the agent is reporting in - if they're the same, they get treated as the same app, if they differ they get treated separately, even if the display name's manually been changed to be the same in the UI.

Resolution

There's effectively 2 options:

You can merge the 2 apps togetherfooand foo(1). Both agents will then happily report to the single app entry. The downside of this is that the LOC count will include both "apps". There may also be some duplicate vulns from each module. These can be manually merged but requires some effort.

You can fully delete (archive then delete) the existing app entry, renamed,foo. Any new agents coming in with the name correctly set in their config will take it's place. This is a little more effort now, but the "cleanest" option for future.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Configuration

Issue

Upon start up the Java agent, an exception that resembles the following appears:

[ContrastLauncher] Tue Aug 07 15:14:40 UTC 2018 Couldn't load Contrast! Running without it.java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.contrastsecurity.agent.launcher.InstalledAgent.launchAgent(InstalledAgent.java:82) at com.contrastsecurity.agent.launcher.LauncherAgent.launch(LauncherAgent.java:174) at com.contrastsecurity.agent.launcher.LauncherAgent.premain(LauncherAgent.java:66) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386) at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)Caused by: com.contrastsecurity.agent.i: Unable to create references to working directories at com.contrastsecurity.agent.config.j.a(DefaultLogLocationConfigProvider.java:57) at com.contrastsecurity.agent.config.d.a(ChainedConfigProvider.java:27) at com.contrastsecurity.agent.config.f.b(Config.java:230) at com.contrastsecurity.agent.ContrastAgent.a(ContrastAgent.java:218) at com.contrastsecurity.agent.ContrastAgent.a(ContrastAgent.java:165) at com.contrastsecurity.agent.ContrastAgent.setup(ContrastAgent.java:128) ... 13 moreCaused by: java.io.IOException: Can't promise read/write on cache dir

Cause

The error Can't promise read/write on cache dir, indicates that the agents' cache directory may not have the required permissions.

Resolution

By default, this directory ($user.home/.contrast/cache) should have 755 permissions (i.e. chmod 755). As well, the user that starts up the application server should also have access to this directory. Please ensure that this is indeed the case.

Alternatively, you can add-Dcontrast.dir to point to a directory that the user has ensured access to.This value will override the default Contrast working directory and creates the directory if it does not exist.

For further details on the Java agent's configurable properties, please see: Java Agent System Properties

License Type

On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Installation

Contrast UI comes in two separate installers for each of the operating systems supported.

Contrast-<version>--NO-CACHE

Contains no OSS library information

Must have internet connectivity to load the latest library data

Installer is generally much smaller

Contrast-<version>

Contains all the latest library information up to its release

Does not require an internet connection to our services

Installer is generally much larger than the no-cache version

Installation can take much longer

More information about the installation and installers can be found here in our docs. For this issue we will be focusing on the CACHED version of the installer.

Question

The EOP cached installer is taking a long time to come up. How can I monitor library data ingestion?

Answer

The cached installer can take quite some time to complete. How long depends on your system and the resources available to it. Installation occurs in two parts:

Installer sets up system, services, database, copies binaries and prepares for first start.

The contrast-service is started for the first time and contrast database is upgraded and populated.

EOP version(s) prior to 3.6.9

The second part of the cached install can take anywhere from 30 minutes an 1 1/2 hours to complete. The main reason for this is the ingestion of library data into the Contrast database. The Contrast UI will not be available until this process completes.

There are a log messages you can watch for in the contrast.log file to see how it is coming along.

030919 13.35.58,981 {} {} {} INFO (AbstractImporter.java:211) Beginning CSV import of node type: 'mapping' from 'C:\Program Files\Contrast\data\libraries\v3_6_0\module_vulns.csv' into 'artifacts_vulnerabilities_tmp'030919 13.35.59,199 {} {} {} INFO (AbstractImporter.java:222) Import temporary table 'artifacts_vulnerabilities_tmp' completed, version: v3_6_0, language: node, time: 0.218s030919 13.35.59,199 {} {} {} INFO (AbstractImporter.java:246) Total New records to insert of node type: 'mapping' in 'artifacts_vulnerabilities_tmp': 4784030919 13.35.59,199 {} {} {} INFO (AbstractImporter.java:266) Insert records of node type: 'mapping' into 'artifacts_vulnerabilities_tmp'030919 13.35.59,215 {} {} {} INFO (AbstractImporter.java:274) Insert records table 'artifacts_vulnerabilities_tmp' complete, version: v3_6_0, language: node, time: 0.016s030919 13.35.59,465 {} {} {} INFO (AbstractImporter.java:327) Enabling library data reload preference: node, version: v3_6_0030919 13.35.59,465 {} {} {} INFO (AbstractImporter.java:332) Finished library data reload preference update: node, version: v3_6_0, time: 0.0s

tail -f${Contrast_Home}/data/logs/contrast.log |grep "AbstractImporter"

EOP version(s) 3.6.9+

Changes were made to load cache data in the background and reduce startup time.

With this you can expect the server to come up much quicker than before but it requires you to wait for the process to finish prior to seeing library data. There are a log messages you can watch for in the contrast.log file.

311019 20.30.05,585 {} {} {} INFO (ArtifactImporter.java:28) [ArtifactImporter] Importing from /opt/contrast/libraries/python_vulns.csv into artifacts_vulnerabilities_tmp311019 20.30.05,588 {} {} {} INFO (ArtifactImporter.java:124) Beginning CSV import from '/opt/contrast/libraries/python_vulns.csv' into 'artifacts_vulnerabilities_tmp'311019 20.30.05,604 {} {} {} INFO (ArtifactImporter.java:129) Import temporary table 'artifacts_vulnerabilities_tmp' completed, time: 0.016s311019 20.30.05,606 {} {} {} INFO (ArtifactImporter.java:149) Total New records to insert 'artifacts_vulnerabilities_tmp': 3672311019 20.30.05,606 {} {} {} INFO (ArtifactImporter.java:161) Insert records into 'artifacts_vulnerabilities_tmp'311019 20.30.05,694 {} {} {} INFO (ArtifactImporter.java:168) Insert records table 'artifacts_vulnerabilities_tmp' complete, time: 0.089s311019 20.30.05,701 {} {} {} INFO (ArtifactImporter.java:182) Enabling library data reload preference311019 20.30.05,702 {} {} {} INFO (ArtifactImporter.java:187) Finished library data reload preference update: time: 0.001s

tail -f${Contrast_Home}/data/logs/contrast.log |grep "ArtifactImporter"

You can also look under ${Contrast_Home}/data/libraries/%filename%.csv. If there are csv's in that path then the migration is not yet complete.

License Type

On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Installation

Issue

Error on rename ... (Errcode: 13)

Cause

If you see this error, a migration failed during Contrast startup. The failure was most likely caused by the current user having insufficient privileges to modify files in the directory in which Contrast was installed.

Resolution

To resolve the issue, uninstall Contrast and delete the remaining data files. You can then either elevate the user's permissions in the installation directory or choose a new directory where the user has full privileges. While it's uncommon, Administrator users can encounter this problem if there are extra file restrictions put in place by your organization.

License Type

On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Installation

Issue

gunzip: sfx_archive.tar.gz: not in gzip format

Cause

If you see this error, something went wrong during the transfer process of the installer. The easiest way to confirm this is to calculate the MD5 of the installer you're attempting to run, and compare it to the MD5 hash listed on our Hub site. Most likely this occurred if the file was transferred in ASCII mode instead of binary mode, particularly if SCP/SFTP was used.

Resolution

To resolve the issue, transfer the file again, and explicitly specifying binary mode.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Installation

Running APM Agents with Contrast Agents

Many customers run multiple agent-based technologies, such as Application Performance Management (APM) technologies, in parallel with the Contrast Java and .NET agents. Contrast has performed testing of various, known APM agents in parallel with the Contrast agents. If a particular APM vendor or alternative agent vendor isn't included in the list, use theSupportwidget in OpenDocs to send a message to Contrast's Customer Support team.

Compatible Java Agents

Agent

Status

AppDynamics

Many versions tested without issue.

New Relic

Tested daily by Contrast.

YourKit

Tested daily by Contrast.

CA Wily

Newer versions tested without issue.

JaCoCo

Tested regularly by Contrast, but must be specified as the first-javaagent.

DynaTrace

Newer versions tested without issue.

Note:There have been rare instances of compatibility issues with very old versions of DynaTrace, Wily and JavaMelody.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Installation

Issue

One of following exceptions are encountered when attempting to startup my application with the Contrast Java Agent.

java.lang.IllegalStateException: There was a problem while creating the temporary file at com.contrastsecurity.agent.ContrastLoaderAgent.getTempFileFromInputStream(ContrastLoaderAgent.java:207) at com.contrastsecurity.agent.ContrastLoaderAgent.getJarFileFromInputStream(ContrastLoaderAgent.java:178) at com.contrastsecurity.agent.ContrastLoaderAgent.getAgentCoreJar(ContrastLoaderAgent.java:88) at com.contrastsecurity.agent.injection.ClassInjector.inject(ClassInjector.java:37) at com.contrastsecurity.agent.ContrastLoaderAgent.premain(ContrastLoaderAgent.java:72) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386) at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)Caused by: java.io.IOException: Permission denied at java.io.UnixFileSystem.createFileExclusively(Native Method) at java.io.File.createTempFile(File.java:2024) at java.io.File.createTempFile(File.java:2070) at com.contrastsecurity.agent.ContrastLoaderAgent.getTempFileFromInputStream(ContrastLoaderAgent.java:197) ... 10 moreUnexpected error during Contrast Agent initialization. Continuing application startup without Contrast Agent....Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386) at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)Caused by: java.lang.NoClassDefFoundError: com/contrastsecurity/agent/core/ContrastAgent at com.contrastsecurity.agent.ContrastLoaderAgent.premain(ContrastLoaderAgent.java:79) ... 6 moreCaused by: java.lang.ClassNotFoundException: com.contrastsecurity.agent.core.ContrastAgent at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 7 more

or

java.lang.IllegalStateException: There was a problem while creating the temporary file at com.contrastsecurity.agent.ContrastLoaderAgent.getTempFileFromInputStream(ContrastLoaderAgent.java:207) at com.contrastsecurity.agent.ContrastLoaderAgent.getJarFileFromInputStream(ContrastLoaderAgent.java:178) at com.contrastsecurity.agent.ContrastLoaderAgent.getAgentCoreJar(ContrastLoaderAgent.java:88) at com.contrastsecurity.agent.injection.ClassInjector.inject(ClassInjector.java:37) at com.contrastsecurity.agent.ContrastLoaderAgent.premain(ContrastLoaderAgent.java:72) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386) at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)Caused by: java.io.IOException: No such file or directory at java.io.UnixFileSystem.createFileExclusively(Native Method) at java.io.File.createTempFile(File.java:2024) at java.io.File.createTempFile(File.java:2070) at com.contrastsecurity.agent.ContrastLoaderAgent.getTempFileFromInputStream(ContrastLoaderAgent.java:197) ... 10 moreUnexpected error during Contrast Agent initialization. Continuing application startup without Contrast Agent....Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386) at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)Caused by: java.lang.NoClassDefFoundError: com/contrastsecurity/agent/core/ContrastAgent at com.contrastsecurity.agent.ContrastLoaderAgent.premain(ContrastLoaderAgent.java:79) ... 6 moreCaused by: java.lang.ClassNotFoundException: com.contrastsecurity.agent.core.ContrastAgent at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 7 moreFATAL ERROR in native method: processing of -javaagent failed

Cause

The Contrast Java agent requires a small amount of space to use as a working directories.During startup:

The Java agent will extract a few jar files to the java tmp directory

The Java agent will create/write files to it's working directory

Resolution

To avoid these type of exceptions, pre-create these folders and set permissions so the application server has full access to them.These directories can be controlled with the following JVM properties.

-Dcontrast.dir=/path to folder/ Location of Contrast "workspace"

-Djava.io.tmpdir=/path to folder/ Location of java tmp directory

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Node Agent

Sub Category

Troubleshooting

Objective

In rare scenarios, bad instrumentation causes a web server process to crash or a specific page to error out. If you ever encounter a crash or error caused by Contrast, please report the error via the Submit a Requestlink at the bottom of this page.

If possible, follow the steps below to gather agent logs; this additional information is vital to reproducing and fixing these types of bugs.

Process

You can ask Contrast to log verbose information by doing one of the following.

General properties: (more information can be found here in our docs )

--agent.logger.path /path/to/trace_contrast_agent.log --agent.logger.level TRACE Example:

npm run contrast -- --agent.logger.path ./contrast_trace.log --agent.logger.level TRACE

Contrast_security.yaml file:(more information can be found here in our docs )

agent: logger: path: /path/to/trace_contrast_agent.log level: TRACE

EnvironmentalVariables:

AGENT__LOGGER__PATH='/path/to/trace_contrast_agent.log'AGENT__LOGGER__LEVEL='TRACE'

If your application is running in an SaaS/CaaS environment where the logs are not easily accessible, logging can be sent to STDOUT instead by one of the following methods:

General properties: (more information can be found here in our docs )

--agent.logger.stdout true--agent.logger.level TRACE

Contrast_security.yaml file:(more information can be found here in our docs )

agent: logger: stdout: true level: TRACE

Environment Variables:

AGENT__LOGGER__LEVEL='TRACE'AGENT__LOGGER__STDOUT=true

These options will put Contrast into debug mode and log all output to the log file specified as the system property value. It will also prevent the log from rolling over into multiple files. The log file will start with some messages that look similar to this:

submit a ticket

These verbose logs will allow our support team to diagnose any issues you may encounter. To contact the Contrast Support Team, please to our online support portal.

License Type

On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Authentication

Issue

If you added a user to a Microsoft Active Directory (AD) or LDAP group, but Contrast says that they can't be found during configuration, you might have added them to both the User and SuperAdmin groups in your AD or LDAP instance.

Cause

This is not allowed, the user must exist in only one of the groups.

Resolution

After you choose one group for the user and remove them from the other in your AD or LDAP instance, go back to the Contrast interface. If you chose to keep the user in the SuperAdmin group, Contrast automatically adds them as a SuperAdmin-level user in the Contrast interface but doesn't assign them to any Organization Role or Application Access Groups. If you chose to keep the user in the User group, you must go through the steps to add a user in Contrast.

To learn more about configuring these authentication methods, go to the Authentication page.

License Type

SaaS & EOP

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Connectivity

Issue

The following error is seen when starting the application with the Contrast Java agent.

ERROR - Problem resolving features with com.contrastsecurity.agent.features.%javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested targetCaused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Cause

There can be many causes for this error but generally it is a failure to recognize the root CA. One possible cause for this is that the CA is being replaced during the package inspection of a firewall or proxy server.

Resolution

To check that this is the case, one can run a simple curl command to the Contrast UI and validate the CA

curl -HAccept:application/json -HAuthorization:test -HAPI-Key:test https://app.contrastsecurity.com/Contrast/api/applications -vOrwith a proxycurl -HAccept:application/json -HAuthorization:test -HAPI-Key:test https://app.contrastsecurity.com/Contrast/api/applications -v --proxy http://proxyserver.company.com:8080

Check the Server Certificateresponse in the returned example:

Server certificate:*subject: CN=*.contrastsecurity.com*start date: Sep 10 00:00:00 2018 GMT*expire date: Oct 10 12:00:00 2019 GMT*subjectAltName: host "app.contrastsecurity.com" matched cert's "*.contrastsecurity.com"*issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon*SSL certificate verify ok.

The issuer should be Amazon. If this is replaced with a local companies domaincertificate it is likely that a system is inspecting SSL traffic between the agent and server. If this is the case, you will need to contact the admin of the proxy/firewall (example: BlueCoat) to see if a bypass can be added for this server.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Authorization

Issue

You might run into an issue connecting to your identity provider, if you're using a metadata URL with an HTTPS certificate from an unsupported authority, such as a self-signed certificate.

Resolution

To resolve the issue, configure SSO by uncheckingI have access to the metadata URLand pasting the metadata XML for the IDP into the text box.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Authentication

Backup Methods

If you've implemented two-step verification, but haven't successfully received a verification code through the method you chose, you can click theCan't Sign In?link in Step Two of the login process. Contrast will then email a temporary code to you, which is valid for five minutes. You can also use a backup code.

If email is already the chosen notification method, contact your Administrator to investigate potential issues with email settings.

Reset Your Device

If you're having issues with Google Authenticator, you manually reset your device by clicking theReset Devicelink in User Settings. This clears all data for the current device and requires resetting Google Authenticator on the same or a new device.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Installation

Issue

I see java.io.IOException: Can't promise read/write on cache dir, error when starting up

Cause

This means that you don't have read/write access to thecachedirectory.

Resolution

Thecachedirectory is located in thecontrastdirectory, and has permissions755by default. If the location of thecontrastdirectory has been modified with thecontrast.dirproperty, thecachedirectory has also moved consequently. You must update the permissions to thecachedirectory.

More information on configuring the contrast directory can be found in our Java docs.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Performance

Issue

After configuring your application to use the Contrast agent, your app's startup time is degraded.

Cause

The time for an application to start up, referred to asstartup performance, can be affected when Contrast is configured for assessment purposes. The startup is a crucial time for the agent, as the main operation involves analyzing libraries for reporting purposes. Many applications are affected by20% to 30%as part of the startup cost for security analysis. Some applications can be affected by2x to 3xdepending on the number of libraries loaded and, in Java versions prior to JDK 8, the configuration of thePermanent Generationspace.

Resolution

If the expected range of performance degradation is too much for your environment, or you're experiencing worse conditions, we have a few options which may help.

Skip Deep Inspection

The Contrast Assess product deeply inspects every piece of bytecode loaded in the JVM. This extra analysis ensures that every possible place where security-relevant functionality is occurring gets a sensor loaded. In many environments, this is not necessary. Accordingly, this deep inspection will eventually be off by default.

To enable skipping these deep inspections, add the following JVM properties:

-Dcontrast.inspect.allclasses=false-Dcontrast.process.codesources=false

Run It Again

The first time Contrast runs on a new environment, or with a new version of the agent or JVM, it will perform extra code analysis as it gets loaded. The results of this analysis is cached, so the next run will have a much faster startup (50-60%).

In PaaS or Docker environments where applications are run in essentially a "factory new" setting, it's likely that there is no "previous run" from which the cached analysis can be reused. This means that every run will have the maximum startup penalty. We will be providing functionality to accommodate these environments in the future.

Using -Xquickstart in the IBM JDK

Web applications leveraging the IBM JDK will experience slower-than-expected startup times. A startup option is available for the JDK to streamline the performance of startup. IBM released a JDK option -Xquickstart, which is disabled by default. Customers leveraging the IBM JDK can enable this option in their startup script and/or any location in which JVM parameters are managed (often in the WebSphere performance console). This particular JVM option causes the Just In Time (JIT) compiler to run with a subset of optimizations.

IBM -Xquickstart Overview "The effect is faster compilation times that improve startup time, but longer-running applications might run slower. When the AOT compiler is active (both shared classes and AOT compilation enabled), -Xquickstart causes all methods to be AOT compiled. The AOT compilation improves the startup time of subsequent runs, but might reduce performance for longer-running applications. -Xquickstart can degrade performance if it is used with long-running applications that contain hot methods. The implementation of -Xquickstart is subject to change in future releases. By default, -Xquickstart is disabled. Another way to specify a behavior identical to -Xquickstart is to use the -client option. These two options can be used interchangeably on the command line."

Note: Customers should test their application without Contrast enabled to ensure this option does not cause any functional instability with their application. This option has been tested by Contrast, but as stated by IBM, this option is subject to change in future releases.

If startup is still unmanageable after using these options, let's do a couple things that will help us isolate the problem:

Get Contrast startup profile information. The agent can be configured to dump startup profile information to stdout. To enable this, pass the following JVM option alongside your -javaagent flag: -Dcontrast.profile.startup=true

Record a full profile of the application startup with Contrast following these steps: How to generate a YourKit session

Please submit a ticket and attach the results of this profiling.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Performance

Issue

After running your application with the Contrast agent, it now runs out of memory.

Cause

The Contrast agent requires a little extra memory in order to operate.

Contrast Protect requires very little, if any, extra memory. Unless you're already very close to the heap limit, it's possible no additions will be necessary. At the maximum, you should add an extra 128MB. The official minimum required for Contrast is 64MB.

Contrast Assess requires more memory to record the analytics necessary to tell a complete vulnerability story. Contrast recommends using 2x the maximum heap usage during typical Assess usage. However the minimum required is 1.3x. (E.g., if you normally run with 128MB of memory, run with 256MB.)

Memory usage with Contrast Assess will be higher in agent versions prior to 3.5.5+.

Resolution

If after adding memory, you're still running out of space, we should generate a heap dump. You can do this a couple different ways:

If you can reliably cause OutOfMemoryErrors to occur, add the following JVM options alongside your -javaagent flag:

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/contrast_dumps/

An alternative way to generate heap dumps is through the jmap command (not available on IBM):

$ jmap -dump:file=/tmp/contrast_dumps/contrast.hprof,format=b <pid_of_app_server>

Of course, replace the /tmp/ path with a good place to write. You can then securely send us the heap dump via our secure transfer system. The Contrast Support team will provide a link.

If you have no way of generating heap dumps, we can try to infer where the problem is by using a debug log. You can enable debug logging by adding the following JVM option alongside your -javaagent flag:

-Dcontrast.level=DEBUG

To contact the Contrast Support Team, please submit a ticket to our online support portal.

License Type

SaaS & On-Premise

Agent Mode

Assess

Main Product Category

Java Agent

Sub Category

Performance

Question

After configuring your application to use the Contrast agent in Assess mode, what is the anticipated performance impact on the application?

Answer

When running in Assess mode, Contrast's analysis will make your application run a little slower. But, the time difference is usually minimal, and the results are definitely worth it.

It's probably more important to think about how Contrast affects the round-trip time. In typical applications, Contrast adds most time to a request that contains a lot of business logic. Round trip times for static resources typically don't get measurably worse. In requests where the total round-trip time is dominated by database or Web Service calls, Contrast's effect will be less noticeable.

Performance tuning

The following steps can be takento tune the agent's performance inAssess mode:

Ensure that the server meets the recommended system requirements and the server has enough free memory before the Java agent is installed.

Run Contrast in Sampling mode, and change sampling frequency to be less frequent.

Run Contrast during nightly integration tests.

Run Contrast in an alternate environment (QA system or DEV environment).

Run Contrast on a single node in a load balanced environment.

While the options above should provide the biggest boost to performance, you can try the following steps to tune performance further.

Check that the agent's logging level is set to "Warn" or "Error".

Turn offresponse scanningby setting:-Dcontrast.assess.enable_scan_response=false(please note that this will lead to certain findings, such as those around misconfigured headers for the application, to no longerbe reported).

Further analysis

If performance is still outside of the expected degradation after following the steps above, we can investigate further by recording a full profile of your app running with Contrast following these steps: How to generate a YourKit session

Please submit a ticket and attach the results of this profiling.

License Type

On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

LDAP

Common Issues

The most common reasons for LDAP or Microsoft Active Directory (AD) issues:

Account information for connecting to a directory service isn't correct.

Users assumed to be in a DN don't exist.

Lookup fields such as mail or userID aren't correctly populated.

The sub-tree of a DN isn't searchable.

Required fields such as First Name, Last Name and Email are missing.

Enterprise-on-Premises (EOP) customers configuring an LDAP service or AD may also run into setup and configuration issues, which you can resolve by logging configuration guidance.

AD and LDAP Configurations

Configuring a directory service can be challenging for Contrast administrators. As noted in the configuration guide, there are many pieces of information needed for basic connectivity, as well as dependencies for configuration. Many customers find this administrative task to be the most challenging part of Contrast setup.

Debug a directory service setup

Generally the default logging for LDAP should be sufficient to troubleshoot most issues. You can review the following log.

$CONTRAST_HOME/data/logs/ldap_ad.log

Should more verbose logging be required, review the article on logging to get up to speed on changing the log configuration and levels.

Turning on additional logging about directory services is a simple, one-line change to thelog4j2.xmlfile located in$CONTRAST_HOME/data/confdirectory. Change directories through a Unix command prompt or Windows Explorer window. You can edit the file in real-time, and shouldn't have to restart Contrast. Locate the section referencingLogger, edit the line below, replacing the level= to TRACE. Contrast picks up the change and begin writing log messages to theldap_ad.log.

<Logger name="contrast.teamserver.service.ldap" level="TRACE"></Logger>

Review log messages

Once the setting takes effect, Contrast begins sending directory service log messages to the$CONTRAST_HOME/data/logs/ldap_ad.logfile. Contrast recommends that you walk through the configuration of either LDAP or AD as a SuperAdmin after this setting is added.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Configuration

Note! These configuration values are considered legacy configuration options. As of version Java Agent Version 3.6.0 these configuration options began deprecating. New configuration values will only be supported via YAML-based configuration. All users are encouraged to migrate to YAML configuration properties.

If you'd like to override configuration options in your agent, you can run with a custom configuration. To start, let's copy the configuration that's shipped with the agent. The following JAR command will copy the configuration file out of an agent that's been downloaded from the Contrast site:

user:tomcat majordomo$ jar -xf contrast.jar contrast.config

Now that you have acontrast.configfile, which is just XML, you can edit it like any other file. However, to tell the agent to use this configuration file, we have to modify our-javaagentline to point to it, as shown here:

export JAVA_OPTS="$JAVAOPTS -javaagent:/tomcat6/contrast.jar=/tomcat6/contrast.config"

The following marked-upcontrast.configfile shows what can be controlled here:

<?xml version="1.0"?>

<contrast>

<id/> <!-- Used to 'id' this agent, if such a need exists. -->

<log level="error" console="false"/> <!-- The level is one of the standard log4j levels. -->

<global-key>demo</global-key> <!-- Your organization's API Key. Needed for talking to the REST API. -->

<user>

<id>contrast_admin</id> <-- Your username. -->

<key>demo</key> <!-- Your service key, which is needed for talking to the REST API. -->

</user>

<!--

Where Contrast results are reported. You can add the following attributes to the 'url' element in order

to ask Contrast to use a proxy:

- proxyHost (just the hostname or IP)

- proxyPort

- proxyUser

- proxyPassword

- proxyAuthenticationType (one of NTLM, Digest or Basic)

-->

<url>https://app.contrastsecurity.com/Contrast/s/</url>

<local-results mode="never"/> <!-- Results can be captured locally with "error" or "always". -->

<plugins>

<!-- The contents of this area shouldn't be altered. -->

</plugins>

<capture-stacktraces>ALL</capture-stacktraces> <!-- Set to SOME or NONE to gain performance boosts. -->

<!--

Setting 'enabled' to true in the sampling section tells Contrast to skip analysis of redundant of URIs

after some baseline samples have been collected.

-->

<sampling>

<enabled>false</enabled>

<baseline>5</baseline>

<request-frequency>10</request-frequency>

<response-frequency>50</response-frequency>

<window>180</window>

</sampling>

</contrast>

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

.NET Agent

Sub Category

Configuration

Note:These configuration values are considered legacy configuration options. As of version 19.5.4 these configuration options began deprecating. New configuration values will only be supported via YAML-based configuration. All users are encouraged to migrate to YAML configuration properties.

The Contrast configuration fileDotnetAgentService.exe.configcontains several settings that you can modify to change the behavior of the .NET agent for Windows. In all cases, configuration values in the agent configuration file will override any configuration values that have the same name specified in the Contrast UI (e.g., logging level, sampling and stack trace configuration).

General

Parameter

Description

Version

AutoUpdateBehavior

Determines if the agent automatically updates to a newer version when a newer version is available on Contrast. The default value isDaily.

Daily: The agent checks for a new version on service start and every 24 hours afterwards.

Startup: The agent only checks for and installs updates on service start

Disabled: The agent checks for, but doesn't install, any updates.

4.6+

OverrideExistingProfiler

Due to .NET Profiling API technology limitations, only one program can use it at a time. This API is commonly used by APM agents like NewRelic, AppDynamics or DynaTrace. By default, this is set to "false"; the Contrast agent will fail to start if it detects another program using the .NET Profiling API so that the other program can continue working. If set to "true", Contrast will attempt to force itself to start, which will break the other agent.

18.3.4+

RestartIISOnConfigChange

Contrast will automatically restart IIS in the background when any configuration settings that require IIS restart are changed. Changes that enable or disable Assess or Defend mode, add security controls, or change process whitelist or blacklist require restart. These changes can come from changing the applicationconfigfile or from the Contrast website. The default value is "true". If set to "false", you must restart IIS for changes to the given configuration settings to take effect.

3.2.7+

RouteDiscoveryEnabled

Turn on and off the route coverage collection feature. The default is "true". If set to "false", routes will not be collected for supported .NET frameworks.

18.8.23

Communication

Parameter

Description

Version

TeamServerUrl

Overrides the Contrast URL that's packaged with the agent. This can be useful for networks that proxy the information.

All

ProxyAuth, ProxyAddress

Controls the proxy used by the agent (if any) to connect to the Contrast interface. Proxy credentials (if applicable) are stored in a separateDotnetAgent.Protectedsection as described below.

All

EncryptProtectedSettings

Controls whetherProxyUserandProxyPasssettings are encrypted. See theProxy Credentialssection below.

4.2.0+

TlsVersion

Controls the version of TLS that the agent uses to communicate with the Contrast interface. ValidTlsVersionvalues includeTls,Tls11andTls12. Agent default behavior isSecurityProtocolType.Tls|SecurityProtocolType.Tls11|SecurityProtocolType.Tls12.

3.3.6+

Proxy credentials

To avoid storing sensitive proxy credentials in plain text, the agent stores them in theDotnetAgent.Protectedsection which is encrypted on startup. To change existing credentials, delete the contents of the section and manually add the keys. They will be re-encrypted on the next agent service startup. To turn off this encryption, use the setEncryptProtectedSettingssetting given in the previous section.

Parameter

Description

Version

ProxyUser

Username for the proxy

All

ProxyPass

Password for the proxy

All

Display Customization and Tagging

Parameter

Description

Version

ServerName

Customizes the display name used by the Contrast interface for the server running the .NET agent. If theServerNameconfiguration setting is not present, the .NET agent will use the computer name for the server's display name. You can view the computer name by viewing theSystemproperties in the WindowsControl Panel.

3.1.4+

Contrast.AppVersion

Controls the application version tag sent to Contrast.

3.3.6+

Contrast.AppGroup

Specifies the group to which this application will be added in the Contrast UI, if this application is not already a member of a group.

3.4.5+

ServerEnvironment

Controls the environment value sent to Contrast when servers are created. ValidServerEnvironmentvalues areDEVELOPMENT,QAorPRODUCTION(case insensitive). The default value isQA. This does not affect servers that already exist in Contrast.

3.4.2+

ServerTags

Controls free-form tags sent to Contrast for servers; you can use tags to search for servers in the Contrast interface. See the article on Application-Specific Settings for details on tagging applications, libraries and vulnerabilities.

4.8.20+

Note:Setting any of theContrast.*(e.g., Contrast.AppVersion) parameters in the agent's configuration file will cause the agent to use the same parameter value for all applications that donothave that parameter set in theirweb.configfile. See the article on Application-Specific Settings for more details.

Diagnostics

More detailed levels of logging degrade performance, but can generate useful information for debugging Contrast. The logging level is configured in Server Settings in the Contrast interface by default; however, you can also configure it at the agent level.

LogLevel

Controls

Error

Only log error conditions, such as unhandled exceptions

Warn

Errormessages and unexpected conditions that don't impact the agent

Info

ErrorandWarnmessages as well as general information about the agent's sensors (startup, shutdown, start and end of requests, etc.)

Debug

Infomessages and some high-level debugging information (e.g., number of vulnerabilities detected for a request)

Trace

Debugmessages as well as logging every trace event (e.g.,String.Concat); this logging level greatly degrades performance

Parameter

Description

Version

ShouldLogMethodSignatures

Controls logging of method signatures during CLR JIT compilation. The default value is "false". Set to "true" to enable method signature logging. This setting has a noticeable impact on startup time but can help troubleshoot issues during application startup.

All

Performance

Parameter

Description

SamplingBaseline, SamplingFrequency, SamplingWindow

Enable and configure sampling mode. Configured in Server Settings in the Contrast interface by default. See the article on Sampling for more information.

StackTraceConfig

Limits stack traces captured by the agent. Configured in Server Settings in the Contrast interface by default. The default isFull.

Full: Captures all stack traces with file and line number information. Deployments must include.PDBfiles for line number information.

Limited: Better performance; captures all stack traces but without file and line number information. Best used for builds without debugging symbols (.PDBfiles).

Minimal: Best performance; doesn't capture intermediate propagator stack traces, no file and line information. Best used for Release builds and Production environments.

ThreadAnalysis

Web (default) or Full. Web follows data flow through normal web operations. Full instruments all threading operations which adds overhead. It can be used for more thorough analysis if your application manually creates background threads.

DetectPotentialSecurityControls

Set to "true" or "false" (default). All code signatures will be checked if they are a potential security validator or sanitizer, when their code is JIT compiled. Detected signatures are reported to Contrast website and can be set as validators or sanitizers there. Set it to "false" to slightly improve start-up performance or bypass issues with this feature.

Analysis

Parameter

Description

ResponseUrlWhiteListRegex

Controls the .NET agent's collection and analysis of response headers and bodies. Responses aren't captured and are analyzed for request paths (HttpRequest.Path) that match this regex. This setting is required to work around a known Microsoft bug in the .NET framework (HttpModules) with filters can cause resources such asWebResource.axdto return0 bytes. (This can result in 500 status responses for embedded resources, such as images.) The default value isWebResource.axd.

ProcessBlacklist

Controls the .NET agent's monitoring of application pools. Set the value of this setting to a comma-separated list of application pool names that the agent shouldn't analyze. The agent should have no performance impact on applications that aren't analyzed due to this setting. This list accepts*as a wildcard.

ProcessWhitelist

Controls the .NET agent's monitoring of application pools. You should set the value of this setting to a comma-separated list of application pool names that the agent should analyze. The agent doesn't monitory any other applications. The agent should have no performance impact on applications that aren't analyzed due to this setting. This list accepts*as a wildcard.

See the Application Pool Filter article for more information on usingProcessBlacklistandProcessWhitelistagainst IIS application pools.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Configuration

Note! These configuration values are considered legacy configuration options. As of version Java Agent Version 3.6.0 these configuration options began deprecating. New configuration values will only be supported via YAML-based configuration. All users are encouraged to migrate to YAML configuration properties.

Parameters for certain rules are configurable by the end user. The following are methods of applying custom parameters to these rules.

Setting In "rules.xml"

An optional top level<properties>element has been added to therules.xml.

<?xml version="1.0" encoding="UTF-8"?>

<policies>

<policy>

<properties>

<web.session.timeout>30</web.session.timeout>

.

.

.

</properties>

<org-packages/>

<propagators>

.

.

.

</propagators>

.

.

.

</policy>

.

.

.

</policies>

Properties File

Also, users can use-Dcontrast.properties="/path/to/properties.file"to point to a standard Java properties file when launching their application container. This will override any settings in the<properties>element in therules.xmlfile.

For example, in thecatalina.sh:

export CONTRAST_AGENT_JAR= "..."

export JAVA_OPTS= "$JAVA_OPTS -javaagent:$CONTRAST_AGENT_JAR -Dcontrast.properties=" /path/to/properties.file "..."

And a properties file located at/path/to/properties.filewould look likeweb.session.timeout= 30.

Direct Definition

Finally, the user could specify a property directly when launching their application container, like-Dweb.session.timeout=30. This will override any settings inrules.xmland the properties file.

export CONTRAST_AGENT_JAR= "..."

export JAVA_OPTS= "$JAVA_OPTS -javaagent:$CONTRAST_AGENT_JAR -Dweb.session.timeout=" 30 "..."

Supported Properties

Currently, the following properties are supported by this feature:

Property Name

Default Value

Description

web.session.timeout

30

The security plugin will report a vulnerability if the<session-timeout>value configured in an application'sweb.xmlexceeds this value. This value is in minutes.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Performance

Objective

Remote monitoring and collection of performance data using JMX and VisualVM for a Java environment.

Process

1. Either on the problematic system or on a remote system install the VisualVM software for data collection.

VisualVM is an app for Windows/Mac/Linux: https://visualvm.github.io/download.html

2. For the Java environment being monitored. The followingJVM properties are needed (more information on these can be found here ). Make sure to add these BEFORE the javaagent:

-Dcom.sun.management.jmxremote.ssl=false-Dcom.sun.management.jmxremote.authenticate=false-Dcom.sun.management.jmxremote.port=9010-Dcom.sun.management.jmxremote.rmi.port=9011 <not needed in most cases>-Djava.rmi.server.hostname=<server IP address, really only needed if there are multiple Nic cards>-Dcom.sun.management.jmxremote.local.only=false

Note! Port 9010 will be used later when connecting to the hostname/port defined with VisualVM

The following example shows the properties along with the Java agent:

java -Dcom.sun.management.jmxremote.ssl=false \ -Dcom.sun.management.jmxremote.authenticate=false \ -Dcom.sun.management.jmxremote.port=9010 \ -Dcom.sun.management.jmxremote.rmi.port=9011 \ -Djava.rmi.server.hostname=192.168.1.27 \ -Dcom.sun.management.jmxremote.local.only=false \ -javaagent:./contrast.jar \ -Dcontrast.config.path=contrast_security.yaml \ -jar webgoat-container-7.1-exec.jar \ -httpPort 8082

3. Start up the Application and check to ensure the 9010 port is listening.

netstat -tupln

Note! If running remotely you may need to open up the systems firewall to allow the connection from VisualVM.

4. Launch VisualVM and add a Remote Host:

5. Then add the JMX Connection:

6. Once connected you will start seeing live threads within the JVM along with Contrast:

7. Click on the Sampler tab, then select the CPU button to start collecting data on CPU performance:

8. Recreate the performance problem while this is running.

9. Once recreated select and export the data

10. To collect Thread data select the Threads tab and select Thread Dump:

This will open a separate tab where all the text should be copied and pasted into a text document:

11. Lastly to collect details on the environment. Right click on the Remote Host connection and select Application Snapshot:

This will show up under the snapshots menu, right click and save this:

12. At this point you should have collected the CPU snapshot, thread dump and application snapshot. These should be zipped up and sent to support. If the resulting files are large, ask your support rep for a SendSafely link to post them to.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Java Agent

Sub Category

Performance

Objective

Generate a YourKit session for debugging purposes.

Process

Here are the steps to manually generate a YourKit session:

Download the Linux .zip archive from: https://www.yourkit.com/download/. (This contains the necessary binaries for all environments.)

Unzip the contents to any location (this is your Profiler Directory referenced below).

Add the -agentpath flag as shown on this page: https://www.yourkit.com/docs/java/help/agent.jsp. Remember to put this new flag before the -javaagent flag. We also want to add the following options for the profiling session:

sampling -- this will turn on CPU profiling as soon as the app starts.

onexit=snapshot -- this will tell the agent to dump a snapshot to disk ($USER/Snapshots) after the JVM exits cleanly

"alloceach=10" -- turn on object allocation recording at the start, recording each 10th allocation.

"usedmem=90" -- capture a snapshot when the memory usage reaches 90%

So, our final path will look something like:

-agentpath:<Profiler Directory>/bin/<architecture>/libyjpagent.so=sampling,alloceach=10,usedmem=90,onexit=snapshot

For agent versions 3.6.1 and newer, an internal feature for advanced instrumentation of strings will have to be disabled to allow running YourKit from the command line. Add the following System Property to the command line when launching with YourKit.

-Dcontrast.java.agent.enable_java_string_support=false

Make sure the Contrast log level is ERROR or WARN.

Delete the contrast.log file

Start the server

Browse the application enough to experience the unwanted behavior (slowness, crashing, etc)

Shutdown the server

Check the $USER/Snapshots directory, please zip up and send to support. Also include standard out logs that will confirm that YourKit was running properly.

For problems not related to memory, you can use the same steps except change #2:

Add the -agentpath flag as shown on this page: https://www.yourkit.com/docs/java/help/agent.jsp. Remember to put this new flag before the -javaagent flag. We also want to add the following options for the profiling session:

sampling -- this will turn on CPU profiling as soon as the app starts.

onexit=snapshot -- this will tell the agent to dump a snapshot to disk ($USER/Snapshots) after the JVM exits cleanly

It is worth noting that problems with memory can masquerade as problems with CPU or slow requests, as memory thrashing will cause all operations to slow down.

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Authentication

Question

What is the logout URL that Contrast uses for it's single sign on configuration?

Answer:

The SAML metadata for the Contrast UI can be found by hitting the following URL:

<contrast_url>/Contrast/saml/metadata

For SaaS customers, the URL will be:

https://app.contrastsecurity.com/Contrast/saml/metadata

When enabling SAML Single Logout, the URL will take the form:

<contrast_url>/Contrast//saml/SingleLogout

For SaaS customers, the URL will be:

https://app.contrastsecurity.com/Contrast//saml/SingleLogout

License Type

SaaS & On-Premise

Agent Mode

Assess & Protect

Main Product Category

Contrast UI

Sub Category

Installation

Issue

When installing the Contrast UI in a distributed manner, as per the steps here: Distributed Configuration, I've used Amazon Relational Database Service (RDS) as my backend database. The Contrast UI's Java process is starting but the migration scripts which run against the database always fail.

Cause

RDS uses a different set of default configurations than a local MySQL install.

Resolution

In the AWS console, go to RDS > Parameter groups and open the group used by your database. You can then click Edit Parameters and the Values box will become editable. The following parameters should be updated:

max_allowed_packetsshould be set to1073741824. This is 1GB and allows the database to handle larger packets and strings.

log_bin_trust_function_creatorsshould be set to1. This enables triggers on the database which are required to create the Contrast schema.

Once these changes have been made, please restart the RDS instance and then restart the Contrast UI Java process. The installation should continue from where it got to.