Maximizing system performance for web protection solutions

Applies to: In this topic
  • Forcepoint Web Security, v8.5.x
  • Forcepoint URL Filtering, v8.5.x
  • Network Agent
  • HTTP request logging
  • Microsoft SQL Server (Log Database)
  • Log Database sizing considerations

Adjust web protection components to improve policy enforcement and logging response time, system throughput, and CPU performance.

Network Agent

As the number of users grows, or if Network Agent does not block Internet requests as expected, place Network Agent on a different machine from Filtering Service and Policy Server. You can also deploy additional Network Agent instances and divide network monitoring between them.

In a high-load environment, or an environment with a high-capacity Internet connection, you can increase throughput and implement load balancing by installing multiple Network Agent instances. Install each agent on a different machine, and configure each agent to monitor a different portion of the network.
  • Network Agent must have bidirectional visibility into the network segment it monitors.
  • If multiple Network Agents are installed, each agent must monitor a different network segment (IP address range).
  • If a Network Agent machine connects to a switch, the monitor NIC must plug into a port that mirrors, monitors, or spans the traffic of all other ports.

HTTP request logging

You can use Content Gateway, Network Agent, or a third-party integration product to track HTTP requests and pass the information to Filtering Service, which uses the data to manage and log requests.

Content Gateway, Network Agent and some integration products also track bandwidth activity (bytes sent and received), and the duration of each permitted Internet request. This data is also logged.

When Network Agent is deployed with Content Gateway or an integration product, and both components provide logging data, the amount of processor time required by Filtering Service increases.

If you are using Network Agent with Content Gateway or an integration product, you can avoid extra processing by specifying whether Network Agent or another component logs HTTP requests. Consult the Administrator Help for configuration instructions.

Microsoft SQL Server (Log Database)

Under high load, Microsoft SQL Server operations are resource intensive, and can be a performance bottleneck for reporting tools. For best results:
  • Do not install Log Server on the database engine machine.
  • Provide adequate disk space to accommodate the growth of the Log Database. You can monitor growth and sizing information on the Web > Settings > Reporting > Log Database page in the Forcepoint Security Manager.
  • Use a disk array controller with multiple drives to increase I/O bandwidth.
  • Increase the RAM on the Microsoft SQL Server machine to reduce time- consuming disk I/O operations.

SQL Server clustering is supported for failover or high availability.

Consult your Microsoft documentation for detailed information about optimizing Microsoft SQL Server performance.

Log Database sizing considerations

Log Database disk space requirements vary, based on:
  • Network size
  • Volume of Internet activity
  • How long data must be available for use in reporting
  • Logging settings

    It is important to host the database engine and Log Database on hardware that matches or exceeds the requirements for expected load and for historical data retention.

    Depending on the volume of Internet traffic in your network, and how much data your organization is required to store (based on organizational policy or compliance regulations, for example), the Log Database can become very large.

    To help determine an effective logging and reporting strategy for your organization, consider:

  • When is the network traffic busiest?

    Schedule resource intensive database and reporting jobs at lower-volume times to improve logging and reporting performance during peak periods.

    See the Administrator Help (accessible from the Web Security module of the Forcepoint Security Manager) for information about scheduling database jobs, investigative reports, and presentation reports.

  • How long should log data be kept to support historical reporting?

    Automatically delete partitions and trend data (stored in the catalog database) after they reach this age to reduce the amount of disk space required for the Log Database.

    See the Administrator Help for information about managing Log Database partitions.

  • How much detail is really needed in reports? To decrease Log Database size, consider:
    • logging visits instead of hits (see Logging visits (default) vs. logging hits)
    • disabling full URL logging (see Logging full URLs)
    • enabling consolidation (see Consolidation
    • only logging non-HTTP protocol traffic for selected protocols (see Protocol logging)
    • only logging HTTP and HTTPS traffic in selected categories (see Selective category logging)

All of these logging settings can be customized in the Web module of the Security Manager. Tune your logging settings to achieve the appropriate balance of size savings and report detail for your organization.

Logging visits (default) vs. logging hits

When you log visits, one log record is created for each web page requested by a user, rather than each separate file included in the web page request. This creates a smaller database and allows faster reporting.

When you log hits, a separate log record is generated for each HTTP request to display any element of a web page, including graphics and ads. This type of logging results in a larger and more detailed database than the logging visits option.

Logging full URLs

Enabling full URL logging creates a larger database than with logging hits, and also provides the most detailed reports. Log records include the domain name and the full path to specific pages requested. Use this option if you want reports of real-time scanning activity.

If the Log Database is growing too quickly, you can turn off full logging to decrease the size of each entry and slow growth.

Consolidation

Consolidation helps to reduce the size of the database by combining Internet requests that share the same value for all of the following elements, within a certain interval of time (1 minute, by default):
  • Domain name (for example: www.forcepoint.com)
  • Category
  • Keyword
  • Action (for example: Category Blocked)
  • User
For example, the user visits www.cnn.com and receives multiple pop-ups during the session. The visit is logged as a record.
  • If consolidation is turned off (the default), and the user returns to the site later, a second visit is logged.
  • If consolidation is turned on, additional visits to the site within a specified period are logged as a single record, with a hits (i.e., visits) count indicating the number of times the site was visited in that period.

Protocol logging

If your deployment includes Network Agent, you have the option to log non-HTTP protocol traffic (for example, instant messaging or streaming media traffic) in addition to HTTP and HTTPS traffic.

The more protocols you choose to log, the greater the impact on the size of the Log Database. You can specify whether or not to log a specific protocol in each protocol filter that you create.

Selective category logging

By default, requests for URLs in all categories are logged. If your organization does not want to report on Internet requests for some categories, you can disable logging for those categories to help reduce Log Database size.