Column Search

In the Column Search view, you can use GQL queries to filter and display selected columns, corresponding table name, schema name, classification, and other details. Following is an example:

1a
Filter: Filter search results based on selected option.
1b
GQL Query Box: Enter GQL Query to filter search results.

2
Trends: Displays the time when data was ingested.

3
Column Configuration: Allows to select columns to display in the search results.

4
Hamburger Menu: Additional options can be accessed from the hamburger menu for each scan results row.

  1. Edit classification: Edit the ML suggested classification.

  2. View table: Displays the full table details under Table Search. An example, is shown below:

  3. View schema: Displays the full schema structure under data discovery. An example, is shown below:

  4. Column audit log: With the Column Log Audit feature, you can now view the details for each column after a database scan has been completed, by navigating to the Column Search section in the Structured Data section.

    You can view what stages of the classification pipeline completed and if there was any error that happened.

    Structured data pipeline processing modules and their corresponding statuses:
    DD (Data discoverer) Shows if an entity (file or column) has successfully discovered in the data sources. Available status are:
    • DISCOVERY_COMPLETED: With this status user can see existence of the column in our system.
    • MODIFIED_OR_DELETED: Indicates either column was renamed or deleted in the data sources during at least 2 scans (fired because of dropped table)
    CLASSIFICATION_PIPELINE Shows if the content of the entity (file or column) was classified with content inspection of detectors, regular expressions, AI/ML and DLP. Available status are:
    • CLASSIFICATION_STARTED: Beginning of classification with detectors, regular expressions, AI/ML, and DLP
    • CLASSIFICATION_FINISHED: End of classification with detectors, regular expressions, AI/ML, and DLP
    • FILTERED_OUT: Filtering out columns, which cannot be classified (e.g. complex, binary column type, empty table or no column values exist for classification).
    • ERROR_IN_PROCESSING: Indicates whether a classification error happened.
    SDM (Scan Data Manager) Shows if information has been successfully saved about the entity (file or column) in the system. Available status are:
    • CLASSIFICATION_STARTED: Beginning of classification persistence.
    • CLASSIFICATION_FINISHED: End of classification persistence.
    • CLASSIFICATION_EDITED: Manual classification override was done by user on the console.
    • MODIFIED_OR_DELETED: Indicates either column was renamed or deleted in the data sources during at least 2 scans (fired because of dropped column only).