Log Export Script

The Log Export Script fetches logs from the Log Export API and saves them locally. Administrators must install and use Python 3.6 or above in a Linux environment to run the script.

Prerequisites

  1. Tenant must be registered on the platform and have access to the tenant portal.
  2. Tenant must have an API Key generated on the Platform UI.

For more details refer to this link
Note:
  1. The script needs to be started/restarted/stopped separately for each log collection. Do not attempt to perform these actions for multiple collections in a single command.
    This means that the script must be run separately for each log collection:
    1. Admin
    2. CASBAPI
    3. CASBInline
    4. DLP
    5. Health
    6. SWG
    7. ZTNA
  2. Logs will be saved in the logs folder with nested year, month, date sub-folders based on eventinsertedtime.
  3. The script can be started and stopped at any time. Its state is saved and execution can be resumed from where it left off (using the -d, --date option in the start command overrides the stored state and starts collecting from the date specified).
  4. The script version (currently 1.0.0) can be viewed using the -h,--help option.

Setup

  1. Unzip the contents of the downloaded zip file in a folder of choice.
  2. Ensure Python 3.6+ is installed.
  3. Create a virtual environment if required.
    apt-get install python3-venv
python3 -m venv venv
source venv/bin/activate
  4. Install the scripts additional requirements as specified in the requirements.txt file.
    pip install -r requirements.txt
  5. Script execution logs are saved in insapilogs_{ppcode}_{collection}.log. Check the log to ensure logs are being collected correctly.
  6. The config.ini file in the root folder specifies custom fields and filters. Use the INI format as shown below, where each section applies to a specific collection. More information on fields and filters can be found at Exporting logs using Export API.
    1. Fields: Define a section named [{ppcode}_{collection}_fields] and list the fields as a comma-separated string under the fields key. The field names listed must exactly match those defined in the collection. You can find the mapping between display names and actual field names here.
    2. Filters: Define a section named [{ppcode}_{collection}_filter_{field}] and specify filter operations (e.g., eq, ne) as the keys with their values as comma-separated strings.
Note:

If a config file is not supplied, the saved logs will include default fields (as specified here) for all collections and and default filters, i.e. action not equal to “Allowed” logs for SWG.

If the config file is supplied but does not have a fields section for a collection, the saved logs will include all fields. The filters defined in the config file will override the default ones mentioned above.

# Fields
[SSE_SWG_fields]
fields = eventTime,crc,cry

[SSE_Admin_fields]
fields = eventTime,uri,cty

# Filters
[SSE_Admin_filter_crc]
eq = US

[SSE_SWG_filter_act]
ne = Allowed

Usage

python log_export_example.py [-h] -a {start,stop,restart} [-pp {SSE}] -c {Admin,CASBAPI,CASBInline,DLP,Health,SWG,ZTNA} [options]
Options Description Requirement
-h, --help Show help message and exit Optional
-a, --action {start,stop,restart} Start, stop, restart the log export daemon Required
-pp, --ppcode {SSE} Point product code (default: SSE) Optional
-c, --collection{Admin,CASBAPI,CASBInline,DLP,Health,SWG,ZTNA} Exact collection name Required
-d, --date ISO-formatted start date (UTC) Required on first run
-k, --api-key Prompt for API key Required on first run
-p, --portal-host Portal hostname, e.g. portal.forcepointone.com Required on first run
-i, --insights-host Insights hostname, e.g. <tenant>.insights.forcepointone.com Required on first run
-df, --data-format {csv,json} Log data format (default: csv) Optional
-f, --config Config file name (placed in root folder) for custom fields and filters Optional
Examples
  1. Start collecting logs from a specific date.
    python3 insights_log_exporter.py -a start -d '2025-01-01 00:00:00' -k -pp SSE -c SWG -p portal.forcepointone.com -i tenant.insights.forcepointone.com -df csv -f config.ini
  2. Restart collection with previous credentials and state.
    python3 insights_log_exporter.py -a restart -pp SSE -c SWG
  3. Stop collecting logs.
    python3 insights_log_exporter.py -a stop -pp SSE -c SWG