Lookup Tables (BETA)
This feature is not available yet. It will be available in version 1.27 in 2022.
Lookup Tables allow you to add important context to your detections and alerts for improved investigation workflows. Use Lookup Tables to enhance alerts with identity/asset information, vulnerability context, network maps, and more.
Lookup Tables are best for data that is relatively static, such as information about AWS accounts or corporate subnets.

Set up a Lookup Table

Example scenario: Let's say you want to add metadata to distinguish developer accounts from production accounts in your AWS CloudTrail logs. Here's an example where isProduction has been added:
To configure a Lookup Table, follow these steps in the Panther UI:
  1. 1.
    From the left sidebar, click Enrichment > Lookups.
  2. 2.
    In the upper right side of the page, click + to add a new Lookup Table.
3. Configure the Lookup Table Basic Information:
a. Enter a descriptive Lookup Name. b. Enter a Description (optional) and a Reference (optional). Description is meant for content about the table, while Reference can be used to hyperlink to an internal resource. c. Next to Enabled? toggle the setting to Yes. Note: This is required to import your data later in this process. d. Click Continue.
4. Configure the Associated Log Types: a. Select the Log Type from the dropdown. b. Type in the name of the Selectors, the foreign key fields from the log type you want enriched with your Lookup Table. (In the example screen shot below, we selected AWS.CloudTrail logs and typed in accountID and recipientAccountID to represent keys in the CloudTrail logs.) c. Click Add Log Type to add another if needed. d. Click Continue.
5. Configure the Table Schema.
Note: If you have not already created a new schema, please see our documentation on creating schemas. Once you have created a schema, you will be able to choose it from the dropdown on the Table Schema page while configuring a Lookup Table. a. Select a Schema Name from the dropdown. b. Select a Primary Key Name from the dropdown. This should be a unique column on the table, such as accountID. c. Click Continue.
Note: You can upload your Lookup Table data to infer the schema if you want -- here is an example:
6. Drag & drop a file or click Select File to choose the file of your Lookup Table data to import. The file must be in .csv or .jsonl format. The maximum file size supported is 5MB.
7. After you successfully import a file, click View in Data Explorer to query that table data or click Finish Setup to go back to a list of your custom Lookup Tables.

Write a detection using Lookup Table data

Next, you can write detections based on the additional context from your Lookup Table.
Continuing the example mentioned above where you configure a Lookup Table to distinguish between developer and production accounts in AWS CloudTrail logs, you might want receive an alert only if the following circumstances are both true:
  • A user logged in who did not have MFA enabled.
  • The AWS account is a production (not a developer) account.
See an example of a Python rule to detect this:
1
from panther_base_helpers import deep_get
2
def rule(event):
3
is_production = deep_get(event, 'p_enrichment', 'account_metadata',
4
'recipientAccountId', 'isProduction')
5
return not event.get('mfaEnabled') and is_production
Copied!
The Panther rules engine will take the looked up matches and append that data to the event using the key p_enrichment in the following JSON structure:
1
{
2
'p_enrichment': {
3
<name of lookup table>: {
4
<key in log that matched>: <matching row looked up>,
5
...
6
<key in log that matched>: <matching row looked up>,
7
} }
8
}
Copied!
Example:
1
{
2
'p_enrichment': {
3
'account_metadata': {
4
'recipientAccountId': {
5
'accountID': '90123456',
6
'isProduction': 'False',
7
'email': '[email protected]',
8
}
9
}
10
}
11
}
Copied!

Unit testing

For rules that use p_enrichment, click Enrich Test Data in the upper right side of the JSON code editor to populate it with your Lookup Table data. This allows you to test a Python function with an event that contains p_enrichment.

Additional Examples

Example for translating 1Password UUIDs into human readable names

Please see our guide about using Lookup Tables to translate 1Password's Universally Unique Identifier (UUID) values into human readable names: Using Lookup Tables: 1Password UUIDs.

Example using CIDR matching

Example scenario: Let's say you want to write detections that consider the traffic logs from company IP space (e.g. VPNs and hosted systems) differently from others logs originating from public IP space. You have a list of your company's allowed CIDR blocks listed in a .csv file (e.g. 4.5.0.0/16):

Set up a Lookup Table with the CIDR list

  1. 1.
    Follow the steps above under "Set up a Lookup Table" to add a new Lookup Table and configure its basic information.
  2. 2.
    On the Associated Log Types page, choose the Log Type and Selectors. For this example, we used AWS.VPCFlow logs and associated the source IP (srcAddr) and destination (dstAddr) keys.
3. Associate a schema for your Lookup Table: select an existing one from your list or create a new one.
Note: The primary key column which will hold the CIDR blocks needs to have a CIDR validation applied in the schema that indicates this lookup table will do CIDR block matching on IP addresses. See our log schema reference.
1
# Will allow valid ip6 CIDR ranges
2
# e.g. 2001:0db8:85a3:0000:0000:0000:0000:0000/64
3
- name: address
4
type: string
5
validate:
6
cidr: "ipv6"
7
8
# Will allow valid ipv4 IP addresses e.g. 100.100.100.100/00
9
- name: address
10
type: string
11
validate:
12
cidr: "ipv4"
Copied!
4. Drag & drop a file or click Select File to choose the file of your CIDR block list to import. The file must be in .csv or .jsonl format. The maximum file size supported is 5MB.
5. After you successfully import a file, click View in Data Explorer to query that table data or click Finish Setup to go back to a list of your custom Lookup Tables.
Successful upload
View uploaded data in Data Explorer

Write a detection

You might like to receive an alert if any VPC traffic comes from a source IP address that is not part of your company's allowed CIDR blocks. Here is an example of Python rule that will send an alert in this case:
1
def rule(event):
2
if event.get('flowDirection') == 'egress': # we care about inbound
3
return False
4
if event.get('action') == 'REJECT': # we don't care about these either
5
return False
6
if deep_get(event, 'p_enrichment','Company CIDR Blocks','srcAddr'): # these are ok
7
return False
8
return True # alert if NOT from an approved network range
Copied!
Last modified 9d ago