How does the logging process work?

When users browse the Web, their activity as recorded as log data:

Steps

  1. Network Agent, Content Gateway, or a third-party integration forwards the Internet request to Filtering Service.
  2. Filtering Service determines the appropriate response to the request.
  3. By default, Filtering Service forwards a copy of the transaction for logging.
    The transaction goes through Multiplexer to reach Log Server and (if configured) the integrated SIEM tool.

Next steps

See the Security Information Event Management (SIEM) (or this v8.4 version) paper for more information.

  1. Log Server stores the data in temporary cache or BCP files on the local hard disk.

  2. When the load of incoming data is not too heavy, Log Server performs preprocessing on the cached files and forwards them to the Log Database. Log Server can:
    • Process multiple, similar log records into a single record in a process called log record consolidation
    • Combine the elements that make up a web page (like advertisements, graphics, and text) into a single record using visits processing

    Information about Log Server preprocessing options can be found in the Administering Databases paper.

  3. The Log Database temporarily stores each log record in a table in the catalog database.
  4. Database jobs move the data to various tables in the partition databases. For more information about how the jobs work, see Database jobs in Administrator Help.

    Data in the partition databases can be used in dashboard, investigative, and presentation reports (see Use Reports to Evaluate Internet Activity for v8.5 or this version for v8.4).