Panther log processing appends some fields to all log records. These fields provide standard names for attributes over all data sources allowing within and between data source correlation of data. For example, each data source has a name for the time an event occurred but each data source will likely not name the attribute the same nor is it guaranteed that the associated time has a time zone consistent with other data sources. The Panther attribute "p_event_time" described below is mapped to each data source's corresponding event time and normalized to UTC. In this way you can query over multiple data sources joining and ordering by "p_event_time" to properly align and correlate the data despite the disparate schemas of each data source.
All appended standard fields begin with "p_".
The fields below are appended to all log records.
The type of log.
Unique id (UUID) for the row.
The associated event time for the log type is copied here and normalized to UTC.
A common security question is often of the form “Was some-indicator ever observed in our logs?” Notice that the relationship of the indicator is not a concern initially, simply the presence or absence of activity is of interest. To allow this question to be answered over all data sources the "any" fields below are appended to rows of data as appropriate.
The "all_logs" Athena view is provided over all data sources to make queries easy for users to find activity for an indicator in a single query.
List of ip addresses (v4 or v6 in string form) related to row.
List of domain names related to row.
List of was account ids related to row.
List of was instance ids related to row.
List of arns related to row.
List of tags related to row as "key:value" pairs.
Panther manages an Athena view over all data sources using the Panther standard fields. This allows you to ask questions like "Was there ANY activity from some-bad-ip and if so where?". For example this will show how many records by log type that was associated with IP address 126.96.36.199:
SELECTp_log_type, count(1) AS row_countFROM panther_views.all_logsWHERE year=2020 AND month=1 AND day=31 AND contains(p_any_ip_addresses, '188.8.131.52')GROUP BY p_log_type
From this information you can then explore the particular logs where activity is indicated.
The Panther standard fields can be used in rules. For example, this rule triggers when any GuardDuty alert is on a resource tagged as 'critical':