Overview
This page describes how to get started with the Unified Agent.
Prerequisites
Ensure you have one of the following Java versions on the computer on which you want to run the Unified Agent.
Java JDK 8
Java JRE 8
Java JDK 11
Additionally, depending on your project type, ensure that the relevant package managers are installed:
Project Type | Package Manager |
---|---|
C# |
|
Elixir, Erlang | MIX |
Go |
|
Haskell | Cabal |
Java |
|
JavaScript |
|
Objective-C, Swift | CocoaPods |
OCaml | Opam |
PHP | Composer |
Python |
|
R | Packrat |
Ruby | Bundler |
Rust | Cargo |
Scala | SBT |
Unified Agent Usage Overview
Step # | Step Name |
---|---|
1 | Download the latest version of the Unified Agent and verify its integrity. |
2 | |
3 | Do one of the following:
(See execution examples on this page) |
4 |
Downloading the Unified Agent
The Unified Agent latest version can be downloaded from Amazon S3 or GitHub.
Latest Unified Agent Version | File | Features | Release Date | MD5 | Comments |
---|---|---|---|---|---|
21.9.1.1 | 25-10-2021 | 706694E349EA14CB04C4621B70D99A93 | N/A |
Setting Up the Unified Agent
There are several methods for configuring the Unified Agent:
Environment Variables (Recommended)
All the parameters available in the configuration file can be passed to the Unified Agent using environment variables. For more information, refer here.Configuration File
The path to the configuration file can be passed to the Unified Agent in the command line using the -c argument. If no file is specified, the Unified Agent will look for a configuration file named wss-unified-agent.config in the current working directory. Refer here for more information.
It is recommended to create a blank configuration file and only add parameters that you would like to change in order to make use of the default configuration settings.Command-line Parameters
The Unified Agent supports a select number of command-line options and parameters. For more information refer here.
The configuration is applied in the following order of precedence:
Command-line parameters
Environment variables
Configuration file
Default values
For the full configuration parameters reference, refer to the Unified Agent Configuration Parameters page.
Setting the Minimum Required Configuration Parameters
Set the following configuration parameters, in any of the available methods, for the Unified Agent's execution:
Parameter Name | Environment Variable Name | Configuration File Parameter Name | Description |
---|---|---|---|
API Key | WS_APIKEY | apiKey | The identifier of the organization. This can be found on the Integrate page of the WhiteSource User Interface under the Organization section. Requires admin level access to see this page. |
WhiteSource URL | WS_WSS_URL | wss.url | The Server URL with For Example: |
User Key | WS_USERKEY | userKey | Required if enforce user level access has been enabled as shown here. See the following link for how to generate a user key. |
Product Name | WS_PRODUCTNAME | productName | The name of the product created after running a scan |
Project Name | WS_PROJECTNAME | projectName | The name of the project created after running a scan |
Scanning Best Practices
General Tips
Require a userKey by enabling enforce user level access in order to see which team members are scanning.
The userKey is also required for API calls and reporting parameters such as generateScanReport
Optimal detection is achieved when scanning after a successful build where dependency files used to create the application are available.
This is will allow the unified agent to detect libraries with all three of its detections methods shown below
Dependency Resolution
During the detection, manifest files (such as requirements.txt in python, for example) are used to pinpoint a specific version of the package used.
Binary and Source File Hash Matching
The WhiteSource Unified Agent also detects binaries and source files (such as .py files in Python or a .jar files in Java) and matches them against the WhiteSource Index.
Scanning Binary and Source Files Overview
WhiteSource matches binary and source files to the repository (GitHub, SourceForge, etc.) from which they most likely originated.
The WhiteSource Index includes ~340M files and ~45M open-source projects.
The hash matching method is required when there are no known packages that can be resolved by utilizing the dependency resolution process.
Binary matches occur only for the exact hash of each file
For each matched source file, the likely origin of that source is determined using a property algorithm
Source Files Matching Algorithm: SmartMatch
It is recommended to enable SmartMatch* for any existing organization
This feature is enabled by default for any new organization created.
Supported File Formats lists all currently supported file formats for hash matching.
This feature can be disabled by setting
fileSystemScan=false
as the default value is true
*SmartMatch is trademarked
Running the Unified Agent
To run the Unified Agent from the command line, execute the following commands in a shell script task as part of your build pipeline or in the directory where your codebase is located
cd <your codebase directory>
Linux/macOS:
export WS_APIKEY=my-apiKey export WS_USERKEY=my-userKey export WS_PRODUCTNAME=my-product export WS_PROJECTNAME=my-project java -jar wss-unified-agent.jar
Windows:
set WS_APIKEY=<your-api-key> set WS_USERKEY=<your-user-key> set WS_PRODUCTNAME=<your-product-name> set WS_PROJECTNAME=<your-project-name> java -jar wss-unified-agent.jar
Specify the -d
parameter to scan another directory besides the current working directory. Full or relative paths can be used, however paths with spaces needed to be enclosed with ""
Running the Unified Agent in a Docker Container
The Unified Agent can also be executed via Docker container which is available on https://hub.docker.com/r/whitesourceft/dockerua
The original Dockerfile template containing different package managers (e.g. maven, npm, etc.) can be found here. Within the file are installation instructions that enable you to create a customizable environment for scanning projects/files, plus a basic (editable) set of package managers.
NOTE: The dockerized unified agent is currently not capable of scanning docker images or containers.
Viewing and Understanding the Scan Steps and Summary
The Unified Agent command-line interface enables you to view the steps that ran as part of a scan and understand how long each step took.
Start/End Indication
A start/end indication is displayed for each scan step. For example:
------------------------------------------------------------------------ -------------------- Start: Pre-Step & Resolve Dependencies ------------ ------------------------------------------------------------------------ [INFO] [2019-03-07 13:58:02,775 +0200] - Trying to resolve MAVEN dependencies [INFO] [2019-03-07 13:58:02,776 +0200] - topFolder = C:\Users\Me\Desktop\UAtests\GenerateScanReport\generateScanReport\Data [INFO] [2019-03-07 13:58:07,105 +0200] - Start parsing pom files [INFO] [2019-03-07 13:58:07,112 +0200] - End parsing pom files , found : search-engine,search-engine-client,search-engine-server [INFO] [2019-03-07 13:58:07,191 +0200] - Trying to resolve HTML dependencies [INFO] [2019-03-07 13:58:09,113 +0200] - ------------------------------------------------------------------------ -------------------- End: Pre-Step & Resolve Dependencies -------------- ------------------------------------------------------------------------
Summary Table
A summary at the end of the scan with all the relevant information on each step is also displayed. It includes the following columns:
Step: The relevant step of the scan
Completion Status: Either 'COMPLETED' or 'FAILED'
Elapsed: The time that step took. Note that the sub-steps are not included in the total elapsed running time (e.g., Maven, HTML).
Comments: When available, more information on the step.
For example:
Step Completion Status Elapsed Comments ====================================================================================================================================================== Fetch Configuration COMPLETED 00:00:00.078 -------- Scan Files Matching 'Includes' Pattern COMPLETED 00:00:00.014 1 source/binary files Pre-Step & Resolve Dependencies COMPLETED 00:00:06.378 7 total dependencies (7 unique) MAVEN COMPLETED 00:00:04.416 5 total dependencies (5 unique) HTML COMPLETED 00:00:01.922 2 total dependencies (2 unique) Update Inventory COMPLETED 00:00:01.551 2 updated projects ====================================================================================================================================================== Elapsed running time: 00:00:08.021 ====================================================================================================================================================== Process finished with exit code SUCCESS (0)
Execution Examples
The following are several syntax examples for various use cases of the Unified Agent execution:
Executing the Unified Agent with environment variables:
Executing the Unified Agent with inline environment variables:
export WS_APIKEY=my-apiKey export WS_USERKEY=my-userKey WS_PRODUCTNAME=my-product WS_PROJECTNAME=my-project java -jar ./wss-unified-agent.jar
Executing the Unified Agent with the config file:
java -jar ./wss-unified-agent.jar -c /path/to/config/file -d /directory/to/scan
Executing the Unified Agent on multiple folders or files:
export WS_APIKEY=my-apiKey export WS_USERKEY=my-userKey export WS_PRODUCTNAME=my-product export WS_PROJECTNAME=my-project java -jar ./wss-unified-agent.jar -d /directory/to/scan,/directory/to/scan2,/file/to/scan
Executing the Unified Agent with a policy check to return an error code in order to break a CI/CD pipeline:
export WS_APIKEY=my-apiKey export WS_USERKEY=my-userKey export WS_PRODUCTNAME=my-product export WS_PROJECTNAME=my-project export WS_CHECKPOLICIES=true export WS_FORCECHECKALLDEPENDENCIES=true export WS_FORCEUPDATE=true export WS_FORCEUPDATE_FAILBUILDONPOLICYVIOLATION=true java -jar ./wss-unified-agent.jar
Executing the Unified Agent with a proxy:
export WS_APIKEY=my-apiKey export WS_USERKEY=my-userKey export WS_PRODUCTNAME=my-product export WS_PROJECTNAME=my-project export WS_PROXY_HOST=my-proxy-host-name export WS_PROXY_PORT=my-proxy-port-number export WS_PROXY_USER=my-proxy-username export WS_PROXY_PASS=my-proxy-password java -jar ./wss-unified-agent.jar
Additional examples for CI/CD pipelines and executing WhiteSource Prioritize can be found at https://github.com/whitesource-ft/ws-examples