Downloading log files

To download log data when it is available, run the script that you have set up. If you are using the provided sample script, the available parameters to use with the script are described below.

Some parameters have a short form (for example, -v) and a long form (for example, --verbose). For these parameters, both options are listed.

Parameter Description

-u <username>

--username

Mandatory. Defines the logon user name for connecting to the cloud service. This must be an administrator contact with Log Export permissions.

For example:

-u FTL_user@example.com

-p <password>

--password

Mandatory. This is the password for the specified user name. For example:

-p Ft2016Logs

-v

--verbose

Optional. Runs the script in verbose mode, which displays progress messages.

Verbose mode provides feedback on the script’s progress, for example:

  • Downloading filelist from <host name> as <user name>
  • No files available to download
  • Downloading <file> to <file name location>

-h <hostname>

--host

Optional. Defines the host name to connect to. This is specified in the script by default, so you would only need this option if you have edited the script to remove it, or if you have been given a different URL to connect to.

For example:

-h https://sync-web.mailcontrol.com

-d <file path>

--destination

Optional. Defines the destination directory for the downloaded log files. If not specified, the files are downloaded into your current working directory.

For example:

-d /cloudweb/logs

-m

--md5sum

 Optional. Checks the md5sum of each downloaded file. The MD5 hash is commonly used to verify the integrity of files (i.e. to verify that a file has not changed as a result of file transfer or disk error), and can therefore be used to check the files before they are deleted from the server.

-l

--list-only

 Optional. Displays a list of available log files without downloading them.
--proxy <proxy details>

Optional. Specifies an HTTP proxy to use if you are having difficulty connecting to the cloud service. The proxy must be in the form http://username:password@host:port

For example:

--proxy http:// jsmith:Abc123@proxy_server:80
--format= <format>

Optional. Creates a new data file containing the original downloaded data rewritten in the desired format. The new file’s name has the relevant data format as a suffix. Note that when this parameter is used, by default the original *.gz file from the source server is not saved to the destination directory.

Valid data formats are:

csv: Comma Separated Values

cef: ArcSight Common Event Format

wc3: WC3 Extended Log file Format (http://www.w3.org/TR/WD-logfile.html)

For example:

--format=csv
--keepgz

Optional. Use in conjunction with the format parameter to download and keep a copy of the *.gz data file in the destination directory. This overrides the default behavior of the format parameter.

For example:

--format=csv --keepgz
--delete Optional. Deletes the original data file from the source server following download. The default option is to delete the file from the server.
--nodelete Optional.Keeps the original data file on the source server after download. This parameter is provided for testing purposes while configuring the format parameter described above, enabling you to download a file in different formats. Note that files are still only retained for 14 days, and you will still receive a warning after 7 days if a downloaded data file remains on the server.
--max_batch_size Optional. Specifies the maximum number of files to download. When set, each time the script is run, the configured number of files are downloaded, starting with the newest files.
--man Optional. Displays the list of parameters with their descriptions.
--help Optional. Displays a brief description of the program’s purpose.

Due to the volume of data, we recommend importing the information into a database to analyze the downloaded log files. For more information about the downloaded data, see File format definition for full traffic logging.