diff --git a/Packs/SplunkPy/.pack-ignore b/Packs/SplunkPy/.pack-ignore index 430e7d6184c..a2771f9334f 100644 --- a/Packs/SplunkPy/.pack-ignore +++ b/Packs/SplunkPy/.pack-ignore @@ -18,6 +18,7 @@ splunk splunkpy hec splunk-search +drilldown [file:classifier-SplunkPy.json] ignore=BA101 diff --git a/Packs/SplunkPy/Integrations/SplunkPy/README.md b/Packs/SplunkPy/Integrations/SplunkPy/README.md index 85b1965325f..b42e834eae9 100644 --- a/Packs/SplunkPy/Integrations/SplunkPy/README.md +++ b/Packs/SplunkPy/Integrations/SplunkPy/README.md @@ -3,7 +3,7 @@ Use the SplunkPy integration to: - Push events from Cortex XSOAR to SplunkPy - Fetch SplunkPy ES notable events as Cortex XSOAR incidents. -This integration was integrated and tested with Splunk Enterprise v9.0.4 and Enterprise Security v7.1.1. +This integration was integrated and tested with Splunk Enterprise v9.0.4 and Enterprise Security v7.2.0. ## Use Cases --- @@ -46,11 +46,11 @@ This integration was integrated and tested with Splunk Enterprise v9.0.4 and Ent | HEC Token (HTTP Event Collector) | | False | | HEC Token (HTTP Event Collector) | | False | | HEC BASE URL (e.g: https://localhost:8088 or https://example.splunkcloud.com/). | | False | - | Enrichment Types | Enrichment types to enrich each fetched notable. If none are selected, the integration will fetch notables as usual \(without enrichment\). For more info about enrichment types see the integration additional info. | False | + | Enrichment Types | Enrichment types to enrich each fetched notable. If none are selected, the integration will fetch notables as usual \(without enrichment\). Multiple drilldown searches enrichment is supported from Enterprise Security v7.2.0. For more info about enrichment types see [Enriching Notable Events](#enriching-notable-events). | False | | Asset enrichment lookup tables | CSV of the Splunk lookup tables from which to take the Asset enrichment data. | False | | Identity enrichment lookup tables | CSV of the Splunk lookup tables from which to take the Identity enrichment data. | False | | Enrichment Timeout (Minutes) | When the selected timeout was reached, notable events that were not enriched will be saved without the enrichment. | False | - | Number of Events Per Enrichment Type | The limit of how many events to retrieve per each one of the enrichment types \(Drilldown, Asset, and Identity\). To retrieve all events, enter "0" \(not recommended\). | False | + | Number of Events Per Enrichment Type | The limit of how many events to retrieve per each one of the enrichment types \(Drilldown, Asset, and Identity\). In a case of multiple drilldown enrichments the limit will apply for each drilldown search query. To retrieve all events, enter "0" \(not recommended\). | False | | Advanced: Extensive logging (for debugging purposes). Do not use this option unless advised otherwise. | | False | | Advanced: Fetch backwards window for the events occurrence time (minutes) | The fetch time range will be at least the size specified here. This will support events that have a gap between their occurrence time and their index time in Splunk. To decide how long the backwards window should be, you need to determine the average time between them both in your Splunk environment. | False | | Advanced: Unique ID fields | A comma-separated list of fields, which together are a unique identifier for the events to fetch in order to avoid fetching duplicates incidents. | False | @@ -78,7 +78,8 @@ The integration allows for fetching Splunk notable events using a default query. This integration allows 3 types of enrichments for fetched notables: Drilldown, Asset, and Identity. #### Enrichment types -1. **Drilldown search enrichment**: fetches the drilldown search configured by the user in the rule name that triggered the notable event and performs this search. The results are stored in the context of the incident under the **Drilldown** field. +1. **Drilldown search enrichment**: Fetches the drilldown search configured by the user in the rule name that triggered the notable event and performs this search. The results are stored in the context of the incident under the **Drilldown** field as follows: [{result1}, {result2}, {result3}]. +Getting results from multiple drilldown searches is supported from Enterprise Security v7.2.0. In that case, the results are stored in the context of the incident under the **Drilldown** field as follows: [{'query_name':, 'query_search': , 'query_results': [{result1}, {result2}, {result3}], 'enrichment_status': }]. 2. **Asset search enrichment**: Runs the following query: *| inputlookup append=T asset_lookup_by_str where asset=$ASSETS_VALUE | inputlookup append=t asset_lookup_by_cidr where asset=$ASSETS_VALUE | rename _key as asset_id | stats values(*) as * by asset_id* where the **$ASSETS_VALUE** is replaced with the **src**, **dest**, **src_ip** and **dst_ip** from the fetched notable. The results are stored in the context of the incident under the **Asset** field. @@ -91,7 +92,7 @@ where the **$IDENTITY_VALUE** is replaced with the **user** and **src_user** fro 2. *Enrichment Types*: Select the enrichment types you want to enrich each fetched notable with. If none are selected, the integration will fetch notables as usual (without enrichment). 3. *Fetch events query*: The query for fetching events. The default query is for fetching notable events. You can edit this query to fetch other types of events. Note that to fetch notable events, make sure the query uses the \`notable\` macro. 4. *Enrichment Timeout (Minutes)*: The timeout for each enrichment (default is 5min). When the selected timeout was reached, notable events that were not enriched will be saved without the enrichment. -5. *Number of Events Per Enrichment Type*: The maximal amount of events to fetch per enrichment type (default to 20). +5. *Number of Events Per Enrichment Type*: The maximal amount of events to fetch per enrichment type (Drilldown, Asset, and Identity). In a case of multiple drilldown enrichments the limit will apply for each drilldown search query. (default to 20). #### Configure User Mapping between Splunk and Cortex XSOAR When fetching incidents from Splunk to Cortex XSOAR and when mirroring incidents between Splunk and Cortex XSOAR, the Splunk Owner Name (user) associated with an incident needs to be mapped to the relevant Cortex XSOAR Owner Name (user). @@ -135,7 +136,7 @@ Define the lookup table in Splunk. #### Troubleshooting enrichment status Each enriched incident contains the following fields in the incident context: -- **successful_drilldown_enrichment**: whether the drill down enrichment was successful. +- **successful_drilldown_enrichment**: whether the drilldown enrichment was successful. In a case of multiple drilldown enrichments, the status is successful if at least one drilldown search enrichment was successful. - **successful_asset_enrichment**: whether the asset enrichment was successful. - **successful_identity_enrichment**: whether the identity enrichment was successful. diff --git a/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy.py b/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy.py index 4ce9a2720c7..468e327ea63 100644 --- a/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy.py +++ b/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy.py @@ -74,6 +74,8 @@ TYPE = 'type' ID = 'id' CREATION_TIME = 'creation_time' +QUERY_NAME = 'query_name' +QUERY_SEARCH = 'query_search' INCIDENT_CREATED = 'incident_created' DRILLDOWN_REGEX = r'([^\s\$]+)=(\$[^\$]+\$)|(\$[^\$]+\$)' @@ -505,7 +507,7 @@ def fetch_incidents(service: client.Service, mapper: UserMappingObject, comment_ # =========== Enriching Fetch Mechanism =========== class Enrichment: - """ A class to represent an Enrichment. Each notable has 3 possible enrichments: Drilldown, Asset & Identity + """ A class to represent an Enrichment. Each notable has 3 possible enrichment types: Drilldown, Asset & Identity Attributes: type (str): The enrichment type. Possible values are: Drilldown, Asset & Identity. @@ -513,6 +515,8 @@ class Enrichment: data (list): The enrichment's data list (events retrieved from the job's search). creation_time (str): The enrichment's creation time in ISO format. status (str): The enrichment's status. + query_name (str): The enrichment's query name. + query_search (str): The enrichment's query search. """ FAILED = 'Enrichment failed' EXCEEDED_TIMEOUT = 'Enrichment exceed the given timeout' @@ -520,26 +524,32 @@ class Enrichment: SUCCESSFUL = 'Enrichment successfully handled' HANDLED = (EXCEEDED_TIMEOUT, FAILED, SUCCESSFUL) - def __init__(self, enrichment_type, status=None, enrichment_id=None, data=None, creation_time=None): + def __init__(self, enrichment_type, status=None, enrichment_id=None, data=None, creation_time=None, + query_name=None, query_search=None): self.type = enrichment_type self.id = enrichment_id self.data = data or [] self.creation_time = creation_time if creation_time else datetime.utcnow().isoformat() self.status = status or Enrichment.IN_PROGRESS + self.query_name = query_name + self.query_search = query_search @classmethod - def from_job(cls, enrichment_type, job: client.Job): + def from_job(cls, enrichment_type, job: client.Job, query_name=None, query_search=None): """ Creates an Enrichment object from Splunk Job object Args: enrichment_type (str): The enrichment type job (splunklib.client.Job): The corresponding Splunk Job + query_name: The enrichment query name + query_search: The enrichment query search Returns: The created enrichment (Enrichment) """ if job: - return cls(enrichment_type=enrichment_type, enrichment_id=job["sid"]) + return cls(enrichment_type=enrichment_type, enrichment_id=job["sid"], + query_name=query_name, query_search=query_search) else: return cls(enrichment_type=enrichment_type, status=Enrichment.FAILED) @@ -552,13 +562,16 @@ def from_json(cls, enrichment_dict): Returns: An instance of the Enrichment class constructed from JSON representation. + """ return cls( enrichment_type=enrichment_dict.get(TYPE), data=enrichment_dict.get(DATA), status=enrichment_dict.get(STATUS), enrichment_id=enrichment_dict.get(ID), - creation_time=enrichment_dict.get(CREATION_TIME) + creation_time=enrichment_dict.get(CREATION_TIME), + query_name=enrichment_dict.get(QUERY_NAME), + query_search=enrichment_dict.get(QUERY_SEARCH) ) @@ -656,22 +669,65 @@ def to_incident(self, mapper: UserMappingObject, comment_tag_to_splunk: str, com """ Gathers all data from all notable's enrichments and return an incident """ self.incident_created = True + total_drilldown_searches = self.drilldown_searches_counter() + for e in self.enrichments: - self.data[e.type] = e.data - self.data[ENRICHMENT_TYPE_TO_ENRICHMENT_STATUS[e.type]] = e.status == Enrichment.SUCCESSFUL + if e.type == DRILLDOWN_ENRICHMENT and total_drilldown_searches > 1: + # A notable can have more than one drilldown search enrichment, in that case we keep the searches results in + # a list of dictionaries - each dict contains the query detail and the search results of a drilldown search + + drilldown_enrichment_details = {"query_name": e.query_name, "query_search": e.query_search, + "query_results": e.data, "enrichment_status": e.status} + + if not self.data.get(e.type): # first drilldown enrichment result to add - initiate the list + self.data[e.type] = [drilldown_enrichment_details] + + else: # there are previous drilldown enrichments in the notable's data + self.data[e.type].append(drilldown_enrichment_details) + + if not self.data.get('successful_drilldown_enrichment'): + # Drilldown enrichment is successful if at least one drilldown search was successful + self.data['successful_drilldown_enrichment'] = e.status == Enrichment.SUCCESSFUL + + else: # asset enrichment, identity enrichment or a single drilldown enrichment + # (return a list to maintain Backwards compatibility) + self.data[e.type] = e.data + self.data[ENRICHMENT_TYPE_TO_ENRICHMENT_STATUS[e.type]] = e.status == Enrichment.SUCCESSFUL return self.create_incident(self.data, self.occurred, mapper=mapper, comment_tag_to_splunk=comment_tag_to_splunk, comment_tag_from_splunk=comment_tag_from_splunk) + def drilldown_searches_counter(self): + """ Counts the drilldown searches of a notable """ + drilldown_search_cnt = 0 + + for e in self.enrichments: + if e.type == DRILLDOWN_ENRICHMENT: + drilldown_search_cnt += 1 + + return drilldown_search_cnt + def submitted(self) -> bool: """ Returns an indicator on whether any of the notable's enrichments was submitted or not """ + notable_enrichment_types = {e.type for e in self.enrichments} return any(enrichment.status == Enrichment.IN_PROGRESS for enrichment in self.enrichments) and len( - self.enrichments) == len(ENABLED_ENRICHMENTS) + notable_enrichment_types) == len(ENABLED_ENRICHMENTS) + + # Explanation of the conditions: + # 1. First condition - if any of the notable's enrichments is 'in progress', it means that it was submitted to splunk. + # 2. Second condition - The ENABLED_ENRICHMENTS list contains the enrichment types that the user wants to enrich. + # According to the logic of the submit_notable() function, in a normal situation (where the code wasn't interrupted) + # the notable.enrichments list should include an enrichment object for each enrichment type that exist in the + # ENABLED_ENRICHMENTS list. That is because in the submit_notable() function we always add Enrichments objects to the + # notable.enrichments list regardless their statuses (failed\success). So if the function had finished it's run without + # any interruption we will have at least one enrichment object for each enrichment type (for drilldown enrichment we could + # have more than one enrichment object - in a case of multiple drilldown searches enrichment). def failed_to_submit(self): """ Returns an indicator on whether all notable's enrichments were failed to submit or not """ + notable_enrichment_types = {e.type for e in self.enrichments} return all(enrichment.status == Enrichment.FAILED for enrichment in self.enrichments) and len( - self.enrichments) == len(ENABLED_ENRICHMENTS) + notable_enrichment_types) == len(ENABLED_ENRICHMENTS) def handled(self): """ Returns an indicator on whether all notable's enrichments were handled or not """ @@ -890,19 +946,19 @@ def get_notable_field_and_value(raw_field, notable_data, raw=None): for field in raw: if field in raw_field: return field, raw[field] - demisto.error(f'Failed building drilldown search query. field {raw_field} was not found in the notable.') + demisto.error(f'Field {raw_field} was not found in the notable.') return "", "" -def build_drilldown_search(notable_data, search, raw_dict): - """ Replaces all needed fields in a drilldown search query - +def build_drilldown_search(notable_data, search, raw_dict, is_query_name=False): + """ Replaces all needed fields in a drilldown search query, or a search query name Args: notable_data (dict): The notable data search (str): The drilldown search query raw_dict (dict): The raw dict + is_query_name (bool): Whether the given query is a query name (default is false) - Returns (str): A searchable drilldown search query + Returns (str): A searchable drilldown search query or a parsed query name """ searchable_search: list = [] start = 0 @@ -913,6 +969,8 @@ def build_drilldown_search(notable_data, search, raw_dict): raw_field = (groups[1] or groups[2]).strip('$') field, replacement = get_notable_field_and_value(raw_field, notable_data, raw_dict) if not field and not replacement: + if not is_query_name: + demisto.error(f'Failed building drilldown search query. Field {raw_field} was not found in the notable.') return "" if prefix: replacement = get_fields_query_part(notable_data, prefix, [field], raw_dict) @@ -924,7 +982,7 @@ def build_drilldown_search(notable_data, search, raw_dict): return ''.join(searchable_search) -def get_drilldown_timeframe(notable_data, raw): +def get_drilldown_timeframe(notable_data, raw) -> tuple[str, str]: """ Sets the drilldown search timeframe data. Args: @@ -932,11 +990,9 @@ def get_drilldown_timeframe(notable_data, raw): raw (dict): The raw dict Returns: - task_status: True if the timeframe was retrieved successfully, False otherwise. earliest_offset: The earliest time to query from. latest_offset: The latest time to query to. """ - task_status = True earliest_offset = notable_data.get("drilldown_earliest", "") latest_offset = notable_data.get("drilldown_latest", "") info_min_time = raw.get(INFO_MIN_TIME, "") @@ -947,57 +1003,132 @@ def get_drilldown_timeframe(notable_data, raw): earliest_offset = info_min_time else: demisto.debug("Failed retrieving info min time") - task_status = False if not latest_offset or latest_offset == f"${INFO_MAX_TIME}$": if info_max_time: latest_offset = info_max_time else: demisto.debug("Failed retrieving info max time") - task_status = False - return task_status, earliest_offset, latest_offset + return earliest_offset, latest_offset -def drilldown_enrichment(service: client.Service, notable_data, num_enrichment_events): +def parse_drilldown_searches(drilldown_searches: list) -> list[dict]: + """ Goes over the drilldown searches list, parses each drilldown search and converts it to a python dictionary. + + Args: + drilldown_searches (list): The list of the drilldown searches. + + Returns: + list[str]: A list of the drilldown searches dictionaries. + """ + demisto.debug("There are multiple drilldown searches to enrich, parsing each drilldown search object") + searches = [] + + for drilldown_search in drilldown_searches: + try: + search = json.loads(drilldown_search) + searches.append(search) + except json.JSONDecodeError as e: + demisto.error(f"Caught an exception while parsing a drilldown search object." + f"Drilldown search is: {drilldown_search}, Original Error is: {str(e)}") + + return searches + + +def drilldown_enrichment(service: client.Service, notable_data, num_enrichment_events) -> list[tuple[str, str, client.Job]]: """ Performs a drilldown enrichment. + If the notable has multiple drilldown searches, enriches all the drilldown searches. Args: service (splunklib.client.Service): Splunk service object. notable_data (dict): The notable data num_enrichment_events (int): The maximal number of events to return per enrichment type. - Returns: The Splunk Job + Returns: A list that contains tuples of a query name, query search and the splunk job that runs the query. + [(query_name, query_search, splunk_job)] """ - job = None + jobs_and_queries = [] demisto.debug(f"notable data is: {notable_data}") - if search := notable_data.get("drilldown_search") or notable_data.get("drilldown_searches", ""): + if drilldown_search := ((notable_data.get("drilldown_search")) or argToList(notable_data.get("drilldown_searches", []))): + # Multiple drilldown searches is a feature added to Enterprise Security v7.2.0. + # If a user set more than one drilldown search, we get a list of drilldown search objects (under + # the 'drilldown_searches' key) and submit a splunk enrichment for each one of them. + # To maintain backwards compatibility we keep using the 'drilldown_search' key as well. raw_dict = rawToDict(notable_data.get("_raw", "")) - if searchable_query := build_drilldown_search( - notable_data, search, raw_dict - ): - status, earliest_offset, latest_offset = get_drilldown_timeframe(notable_data, raw_dict) - if status: - kwargs = {"max_count": num_enrichment_events, "exec_mode": "normal"} - if latest_offset: - kwargs['latest_time'] = latest_offset - if earliest_offset: - kwargs['earliest_time'] = earliest_offset - query = build_search_query({"query": searchable_query}) - demisto.debug(f"Drilldown query for notable {notable_data[EVENT_ID]}: {query}") - try: - job = service.jobs.create(query, **kwargs) - except Exception as e: - demisto.error(f"Caught an exception in drilldown_enrichment function: {str(e)}") - else: - demisto.debug(f'Failed getting the drilldown timeframe for notable {notable_data[EVENT_ID]}') + + if isinstance(drilldown_search, list): + # There are multiple drilldown searches to enrich + searches = parse_drilldown_searches(drilldown_search) + else: - demisto.debug( - f"Couldn't build search query for notable {notable_data[EVENT_ID]} with the following drilldown search {search}" - ) + # Got a single drilldown search (BC) + searches = [drilldown_search] + + total_searches = len(searches) + demisto.debug(f'Notable {notable_data[EVENT_ID]} has {total_searches} drilldown searches to enrich') + + for i in range(total_searches): + # Iterates over the drilldown searches of the given notable to enrich each one of them + search = searches[i] + demisto.debug(f'Enriches drilldown search number {i+1} out of {total_searches} for notable {notable_data[EVENT_ID]}') + + if isinstance(search, dict): + query_name = search.get("name", "") + query_search = search.get("search", "") + earliest_offset = search.get("earliest", "") # The earliest time to query from. + latest_offset = search.get("latest", "") # The latest time to query to. + + else: + # Got a single drilldown search under the 'drilldown_search' key (BC) + query_search = search + query_name = notable_data.get("drilldown_name", "") + earliest_offset, latest_offset = get_drilldown_timeframe(notable_data, raw_dict) + + try: + parsed_query_name = build_drilldown_search(notable_data, query_name, raw_dict, True) + if not parsed_query_name: # if parsing failed - keep original unparsed name + demisto.debug( + f'Failed parsing drilldown search query name, using the original ' + f'un-parsed query name instead: {query_name}.') + parsed_query_name = query_name + except Exception as e: + demisto.error( + f"Caught an exception while parsing the query name, using the original query name instead: {str(e)}") + parsed_query_name = query_name + + if searchable_query := build_drilldown_search( + notable_data, query_search, raw_dict + ): + demisto.debug(f"Search Query was build successfully for notable {notable_data[EVENT_ID]}") + + if earliest_offset and latest_offset: + kwargs = {"max_count": num_enrichment_events, "exec_mode": "normal"} + if latest_offset: + kwargs['latest_time'] = latest_offset + if earliest_offset: + kwargs['earliest_time'] = earliest_offset + query = build_search_query({"query": searchable_query}) + demisto.debug(f"Drilldown query for notable {notable_data[EVENT_ID]} is: {query}") + try: + job = service.jobs.create(query, **kwargs) + jobs_and_queries.append((parsed_query_name, query, job)) + + except Exception as e: + demisto.error(f"Caught an exception in drilldown_enrichment function: {str(e)}") + else: + demisto.debug(f'Failed getting the drilldown timeframe for notable {notable_data[EVENT_ID]}') + jobs_and_queries.append((None, None, None)) + else: + demisto.debug( + f"Couldn't build search query for notable {notable_data[EVENT_ID]} " + f"with the following drilldown search {query_search}" + ) + jobs_and_queries.append((None, None, None)) else: demisto.debug(f"drill-down was not configured for notable {notable_data[EVENT_ID]}") + jobs_and_queries.append((None, None, None)) - return job + return jobs_and_queries def identity_enrichment(service: client.Service, notable_data, num_enrichment_events) -> client.Job: @@ -1027,7 +1158,7 @@ def identity_enrichment(service: client.Service, notable_data, num_enrichment_ev kwargs = {"max_count": num_enrichment_events, "exec_mode": "normal"} job = service.jobs.create(query, **kwargs) except Exception as e: - demisto.error(f"Caught an exception in drilldown_enrichment function: {str(e)}") + demisto.error(f"Caught an exception in identity_enrichment function: {str(e)}") else: demisto.debug(f'No users were found in notable. {error_msg}') @@ -1115,28 +1246,31 @@ def handle_submitted_notable(service: client.Service, notable: Notable, enrichme task_status = False if not notable.is_enrichment_process_exceeding_timeout(enrichment_timeout): - demisto.debug(f"Trying to handle open enrichment {notable.id}") + demisto.debug(f"Trying to handle open enrichment for notable {notable.id}") for enrichment in notable.enrichments: if enrichment.status == Enrichment.IN_PROGRESS: try: job = client.Job(service=service, sid=enrichment.id) if job.is_done(): - demisto.debug(f'Handling {enrichment.type=} for notable {notable.id}') + demisto.debug(f'Handling {enrichment.id=} of {enrichment.type=} for notable {notable.id}') for item in results.JSONResultsReader(job.results(output_mode=OUTPUT_MODE_JSON)): if handle_message(item): continue enrichment.data.append(item) enrichment.status = Enrichment.SUCCESSFUL - demisto.debug(f'{notable.id} {enrichment.type} status is successful. {len(enrichment.data)=}') + demisto.debug(f'{enrichment.id=} of {enrichment.type=} for notable {notable.id} status is successful ' + f'{len(enrichment.data)=}') else: - demisto.debug(f'Enrichment {enrichment.type} for notable {notable.id} is still not done') + demisto.debug(f'{enrichment.id=} of {enrichment.type=} for notable {notable.id} is still not done') except Exception as e: demisto.error( - f"Caught an exception while retrieving {enrichment.type}\ - enrichment results for notable {notable.id}: {str(e)}" + f"Caught an exception while retrieving {enrichment.id=} of {enrichment.type=}\ + results for notable {notable.id}: {str(e)}" ) + enrichment.status = Enrichment.FAILED + demisto.error(f'{enrichment.id=} of {enrichment.type=} for notable {notable.id} was failed.') if notable.handled(): task_status = True @@ -1147,7 +1281,7 @@ def handle_submitted_notable(service: client.Service, notable: Notable, enrichme else: task_status = True demisto.debug( - f"Open enrichment {notable.id} has exceeded the enrichment timeout of {enrichment_timeout}.\ + f"Open enrichment for notable {notable.id} has exceeded the enrichment timeout of {enrichment_timeout}.\ Submitting the notable without the enrichment." ) @@ -1211,8 +1345,11 @@ def submit_notable(service: client.Service, notable: Notable, num_enrichment_eve submitted_drilldown, submitted_asset, submitted_identity = notable.get_submitted_enrichments() if DRILLDOWN_ENRICHMENT in ENABLED_ENRICHMENTS and not submitted_drilldown: - job = drilldown_enrichment(service, notable.data, num_enrichment_events) - notable.enrichments.append(Enrichment.from_job(DRILLDOWN_ENRICHMENT, job)) + jobs_and_queries = drilldown_enrichment(service, notable.data, num_enrichment_events) + for job_and_query in jobs_and_queries: + notable.enrichments.append( + Enrichment.from_job(DRILLDOWN_ENRICHMENT, job=job_and_query[2], + query_name=job_and_query[0], query_search=job_and_query[1])) if ASSET_ENRICHMENT in ENABLED_ENRICHMENTS and not submitted_asset: job = asset_enrichment(service, notable.data, num_enrichment_events) notable.enrichments.append(Enrichment.from_job(ASSET_ENRICHMENT, job)) diff --git a/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy.yml b/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy.yml index da8f1ba114d..b9143f3029f 100644 --- a/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy.yml +++ b/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy.yml @@ -195,7 +195,7 @@ configuration: type: 16 section: Collect advanced: true - additionalinfo: Enrichment types to enrich each fetched notable. If none are selected, the integration will fetch notables as usual (without enrichment). For more info about enrichment types see the integration additional info. + additionalinfo: Enrichment types to enrich each fetched notable. If none are selected, the integration will fetch notables as usual (without enrichment). Multiple drilldown searches enrichment is supported from Enterprise Security v7.2.0. For more info about enrichment types see the integration additional info. options: - Drilldown - Asset @@ -223,7 +223,7 @@ configuration: additionalinfo: When the selected timeout was reached, notable events that were not enriched will be saved without the enrichment. defaultvalue: '5' required: false -- additionalinfo: The limit of how many events to retrieve per each one of the enrichment types (Drilldown, Asset, and Identity). To retrieve all events, enter "0" (not recommended). +- additionalinfo: The limit of how many events to retrieve per each one of the enrichment types (Drilldown, Asset, and Identity). In a case of multiple drilldown enrichments the limit will apply for each drilldown search query. To retrieve all events, enter "0" (not recommended). display: 'Number of Events Per Enrichment Type' name: num_enrichment_events type: 0 diff --git a/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy_description.md b/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy_description.md index 5577e3dcf99..40602510948 100644 --- a/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy_description.md +++ b/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy_description.md @@ -44,7 +44,7 @@ You do not need to specify the classifier as all Splunk incidents are ingested a Outgoing mirroring is recommended for Cortex XSOAR version 6.2 and above. If you enable mirroring, you need to add the timezone of the Splunk server (in minutes). For example, if using GMT and the timezone is GMT +3 hours, set the timezone to +180. For UTC, set the timezone to 0. Set this only if the Splunk server is different than the Cortex XSOAR server. This is relevant only for fetching notable events. 13. Select *Close Mirrored XSOAR Incident* and *Close Mirrored Splunk Notable Event*, so when closing in one environment, it closes in the other. 14. In the Enrichment Types field, select *Asset*, *Drilldown* and *Identity*. -This enrichment provides additional information about assets, drilldown, and identities that are related to the notable events you ingest. For more information, see [Enriching Notable Events](#enriching-notable-events). +This enrichment provides additional information about assets, drilldown, and identities that are related to the notable events you ingest. Multiple drilldown searches enrichment is supported from Enterprise Security v7.2.0. For more information, see [Enriching Notable Events](#enriching-notable-events). 15. Fetch backwards window - this backward window is for cases where there is a gap between the event occurrence time and the event index time on the server. In Splunk, there is often a delay between the time an incident is created (the event's "occurrence time") and the time it is actually searchable in Splunk and visible in the index (the event's "index time"). This delay can be caused by an inefficient Splunk architecture, causing higher event indexing latency. However, it can also be "by design", e.g., if some endpoints / machines that generate Splunk events are usually offline. @@ -65,7 +65,8 @@ Use this parameter with careful consideration. This integration allows 3 types of enrichments for fetched notables: Drilldown, Asset, and Identity. #### Enrichment types -1. **Drilldown search enrichment**: Fetches the drilldown search configured by the user in the rule name that triggered the notable event and performs this search. The results are stored in the context of the incident under the **Drilldown** field. +1. **Drilldown search enrichment**: Fetches the drilldown search configured by the user in the rule name that triggered the notable event and performs this search. The results are stored in the context of the incident under the **Drilldown** field as follow: [{result1}, {result2}, {result3}]. +Getting results from multiple drilldown searches is supported from Enterprise Security v7.2.0. In that case, the results are stored in the context of the incident under the **Drilldown** field as follow: [{'query_name':, 'query_search': , 'query_results': [{result1}, {result2}, {result3}], 'enrichment_status': }]. 2. **Asset search enrichment**: Runs the following query: *| inputlookup append=T asset_lookup_by_str where asset=$ASSETS_VALUE | inputlookup append=t asset_lookup_by_cidr where asset=$ASSETS_VALUE | rename _key as asset_id | stats values(*) as * by asset_id* where the **$ASSETS_VALUE** is replaced with the **src**, **dest**, **src_ip** and **dst_ip** from the fetched notable. The results are stored in the context of the incident under the **Asset** field. @@ -78,11 +79,11 @@ where the **$IDENTITY_VALUE** is replaced with the **user** and **src_user** fro 2. *Enrichment Types*: Select the enrichment types you want to enrich each fetched notable with. If none are selected, the integration will fetch notables as usual (without enrichment). 3. *Fetch notable events ES query*: The query for the notable events enrichment (defined by default). If you decide to edit this, make sure to provide a query that uses the \`notable\` macro. See the default query as an example. 4. *Enrichment Timeout (Minutes)*: The timeout for each enrichment (default is 5min). When the selected timeout was reached, notable events that were not enriched will be saved without the enrichment. -5. *Number of Events Per Enrichment Type*: The maximal amount of events to fetch per enrichment type (default to 20). +5. *Number of Events Per Enrichment Type*: The maximal amount of events to fetch per enrichment type (Drilldown, Asset, and Identity). In a case of multiple drilldown enrichments the limit will apply for each drilldown search query. (default to 20). #### Troubleshooting enrichment status Each enriched incident contains the following fields in the incident context: -- **successful_drilldown_enrichment**: whether the drilldown enrichment was successful. +- **successful_drilldown_enrichment**: whether the drilldown enrichment was successful. In a case of multiple drilldown enrichments, the status is successful if at least one drilldown search enrichment was successful. - **successful_asset_enrichment**: whether the asset enrichment was successful. - **successful_identity_enrichment**: whether the identity enrichment was successful. diff --git a/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy_test.py b/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy_test.py index 45e038ebe97..b5d418ca01a 100644 --- a/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy_test.py +++ b/Packs/SplunkPy/Integrations/SplunkPy/SplunkPy_test.py @@ -994,7 +994,7 @@ def test_reset_enriching_fetch_mechanism(mocker): def test_is_enrichment_exceeding_timeout(mocker, drilldown_creation_time, asset_creation_time, enrichment_timeout, output): """ - Scenario: When one of the notable's enrichments is exceeding the timeout, we want to create an incident we all + Scenario: When one of the notable's enrichments is exceeding the timeout, we want to create an incident with all the data gathered so far. Given: @@ -1042,14 +1042,14 @@ def test_store_incidents_for_mapping(integration_context, incidents, output): assert integration_context.get(splunk.INCIDENTS, []) == output -@pytest.mark.parametrize('notable_data, raw, status, earliest, latest', [ - ({}, {}, False, "", ""), +@pytest.mark.parametrize('notable_data, raw, earliest, latest', [ + ({}, {}, "", ""), ({"drilldown_earliest": f"${splunk.INFO_MIN_TIME}$", "drilldown_latest": f"${splunk.INFO_MAX_TIME}$"}, - {splunk.INFO_MIN_TIME: '1', splunk.INFO_MAX_TIME: '2'}, True, '1', '2'), - ({"drilldown_earliest": '1', "drilldown_latest": '2', }, {}, True, '1', '2') + {splunk.INFO_MIN_TIME: '1', splunk.INFO_MAX_TIME: '2'}, '1', '2'), + ({"drilldown_earliest": '1', "drilldown_latest": '2', }, {}, '1', '2') ]) -def test_get_drilldown_timeframe(notable_data, raw, status, earliest, latest, mocker): +def test_get_drilldown_timeframe(notable_data, raw, earliest, latest, mocker): """ Scenario: Trying to get the drilldown's timeframe from the notable's data @@ -1065,8 +1065,7 @@ def test_get_drilldown_timeframe(notable_data, raw, status, earliest, latest, mo - Return the expected result """ mocker.patch.object(demisto, 'info') - task_status, earliest_offset, latest_offset = splunk.get_drilldown_timeframe(notable_data, raw) - assert task_status == status + earliest_offset, latest_offset = splunk.get_drilldown_timeframe(notable_data, raw) assert earliest_offset == earliest assert latest_offset == latest @@ -1105,6 +1104,18 @@ def test_get_notable_field_and_value(raw_field, notable_data, expected_field, ex ({'a': '1', '_raw': 'c=3'}, 'search a=$a|s$ c=$c$ suffix', {'c': '3'}, 'search a="1" c="3" suffix'), ({'a': ['1', '2'], 'b': '3'}, 'search a=$a|s$ b=$b|s$ suffix', {}, 'search (a="1" OR a="2") b="3" suffix'), ({'a': '1', '_raw': 'b=3', 'event_id': '123'}, 'search a=$a|s$ c=$c$ suffix', {'b': '3'}, ''), + ({"signature": "Backdoor.test"}, "View related '$signature$' events for $dest$", {"dest": "ACME-test-005"}, + "View related 'Backdoor.test' events for ACME-test-005"), + ({}, 'View all wineventlogs involving user="$user$"', {'user': "test"}, + 'View all wineventlogs involving user="test"'), + ({}, 'Test query name', {}, 'Test query name') +], ids=[ + "search query fields in notables data and raw data", + "search query fields in notable data more than one value", + "search query fields don't exist in notable data and raw data", + "query name fields in notables data and raw data", + "query name fields in raw data", + "query name without fields to replace" ]) def test_build_drilldown_search(notable_data, search, raw, expected_search, mocker): """ @@ -1115,6 +1126,10 @@ def test_build_drilldown_search(notable_data, search, raw, expected_search, mock - A raw search query with fields both in the notable's data and in the notable's raw data - A raw search query with fields in the notable's data that has more than one value - A raw search query with fields that does not exist in the notable's data or in the notable's raw data + - A raw query name with fields both in the notable's data and in the notable's raw data + - A raw query name with fields in the notable's raw data + - A raw query name without any fields to replace. + When: - build_drilldown_search is called @@ -1153,6 +1168,451 @@ def test_get_fields_query_part(notable_data, prefix, fields, query_part): assert splunk.get_fields_query_part(notable_data, prefix, fields) == query_part +@pytest.mark.parametrize('enrichments, expected_result', [ + ([splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1'), + splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='2'), + splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='3')], 3), + ([splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1'), + splunk.Enrichment(splunk.ASSET_ENRICHMENT, enrichment_id='2'), + splunk.Enrichment(splunk.IDENTITY_ENRICHMENT, enrichment_id='3')], 1), + ([splunk.Enrichment(splunk.ASSET_ENRICHMENT, enrichment_id='1'), + splunk.Enrichment(splunk.ASSET_ENRICHMENT, enrichment_id='2'), + splunk.Enrichment(splunk.IDENTITY_ENRICHMENT, enrichment_id='3')], 0) +], ids=[ + "A Notable with 3 drilldown enrichments", + "A Notable with 1 drilldown enrichment, 1 asset enrichment and 1 identity enrichment", + "A Notable with 2 asset enrichments and 1 identity enrichment" +]) +def test_drilldown_searches_counter(enrichments, expected_result): + """ + Tests the drilldown searches enrichment counter. + + Given: + - A Notable with 3 drilldown enrichments. + - A Notable with 1 drilldown enrichment, 1 asset enrichment and 1 identity enrichment. + - A Notable with 2 asset enrichments and 1 identity enrichment. + + When: + - drilldown_searches_counter function is called + + Then: + - Return the expected result - number of drilldown enrichments. + """ + notable = splunk.Notable({}, notable_id='id', enrichments=enrichments) + assert notable.drilldown_searches_counter() == expected_result + + +@pytest.mark.parametrize('enrichments, expected_data', [ + ([splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.SUCCESSFUL, + query_name='query_name1', query_search='query_search1', data=[{'result1': 'a'}, {'result2': 'b'}]), + splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='2', status=splunk.Enrichment.SUCCESSFUL, + query_name='query_name2', query_search='query_search2', data=[{'result1': 'c'}, {'result2': 'd'}]), + splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='3', status=splunk.Enrichment.SUCCESSFUL, + query_name='query_name3', query_search='query_search3', data=[{'result1': 'e'}, {'result2': 'f'}])], + [{'query_name': 'query_name1', 'query_search': 'query_search1', 'query_results': [{'result1': 'a'}, {'result2': 'b'}], + 'enrichment_status': splunk.Enrichment.SUCCESSFUL}, + {'query_name': 'query_name2', 'query_search': 'query_search2', 'query_results': [{'result1': 'c'}, {'result2': 'd'}], + 'enrichment_status': splunk.Enrichment.SUCCESSFUL}, + {'query_name': 'query_name3', 'query_search': 'query_search3', 'query_results': [{'result1': 'e'}, {'result2': 'f'}], + 'enrichment_status': splunk.Enrichment.SUCCESSFUL}] + ), + ([splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.SUCCESSFUL, + query_name='query_name1', query_search='query_search1', data=[{'result1': 'a'}, {'result2': 'b'}]), + splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='2', status=splunk.Enrichment.SUCCESSFUL, + query_name='query_name2', query_search='query_search2', data=[{'result1': 'c'}, {'result2': 'd'}])], + [{'query_name': 'query_name1', 'query_search': 'query_search1', 'query_results': [{'result1': 'a'}, {'result2': 'b'}], + 'enrichment_status': splunk.Enrichment.SUCCESSFUL}, + {'query_name': 'query_name2', 'query_search': 'query_search2', 'query_results': [{'result1': 'c'}, {'result2': 'd'}], + 'enrichment_status': splunk.Enrichment.SUCCESSFUL}] + ), + ([splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.SUCCESSFUL, + query_name='query_name1', query_search='query_search1', data=[{'result1': 'a'}, {'result2': 'b'}])], + [{'result1': 'a'}, {'result2': 'b'}] + ), + ([], None) +], ids=[ + "A Notable with 3 drilldown enrichments, 1 asset enrichment and 1 identity enrichment", + "A Notable with 2 drilldown enrichment, 1 asset enrichment and 1 identity enrichment", + "A Notable with 1 drilldown enrichment, 1 asset enrichment and 1 identity enrichment", + "A Notable without drilldown enrichments, 1 asset enrichments and 1 identity enrichment" +]) +def test_to_incident_notable_enrichments_data(enrichments, expected_data): + """ + Tests the logic of the Notable.to_incident() function, regarding the results data of multiple drilldown enrichments. + + Given: + 1. A Notable with 3 drilldown enrichments, 1 asset enrichment and 1 identity enrichment. + 2. A Notable with 2 drilldown enrichment, 1 asset enrichment and 1 identity enrichment. + 3. A Notable with 1 drilldown enrichment, 1 asset enrichment and 1 identity enrichment. + 4. A Notable without drilldown enrichments, 1 asset enrichments and 1 identity enrichment. + + When: + - Notable.to_incident() function is called + + Then: + - Verify that the data of the notable includes the expected enrichements result as follow: + 1. A dictionary with the results of the 3 drilldown searches by query names. + 2. A dictionary with the results of the 2 drilldown searches by query names. + 3. A list of the drilldown searches results (backwards competability). + 4. No 'Drilldown' key in the notables data. + + """ + notable = splunk.Notable({}, notable_id='id', enrichments=enrichments) + enrichments_to_add = [ + splunk.Enrichment(splunk.ASSET_ENRICHMENT, enrichment_id='111', status=splunk.Enrichment.SUCCESSFUL, + data=[{'result1': 'a'}, {'result2': 'b'}]), + splunk.Enrichment(splunk.IDENTITY_ENRICHMENT, enrichment_id='222', status=splunk.Enrichment.FAILED, + data=[{'result1': 'a'}, {'result2': 'b'}]) + ] + notable.enrichments.extend(enrichments_to_add) + + service = Service('DONE') + mapper = splunk.UserMappingObject(service, False) + notable.to_incident(mapper, 'comment_tag_to_splunk', 'comment_tag_from_splunk') + + assert notable.data.get(splunk.ASSET_ENRICHMENT) == [{'result1': 'a'}, {'result2': 'b'}] + assert notable.data.get(splunk.IDENTITY_ENRICHMENT) == [{'result1': 'a'}, {'result2': 'b'}] + assert notable.data.get(splunk.DRILLDOWN_ENRICHMENT) == expected_data + + +@pytest.mark.parametrize('enrichments, enrichment_type, expected_stauts_result', [ + ([splunk.Enrichment(splunk.ASSET_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.SUCCESSFUL, + data=[{'result1': 'a'}, {'result2': 'b'}])], splunk.ASSET_ENRICHMENT, True + ), + ([splunk.Enrichment(splunk.ASSET_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.FAILED, + data=[{'result1': 'a'}, {'result2': 'b'}])], splunk.ASSET_ENRICHMENT, False + ), + ([splunk.Enrichment(splunk.IDENTITY_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.SUCCESSFUL, + data=[{'result1': 'a'}, {'result2': 'b'}])], splunk.IDENTITY_ENRICHMENT, True + ), + ([splunk.Enrichment(splunk.IDENTITY_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.FAILED, + data=[{'result1': 'a'}, {'result2': 'b'}])], splunk.IDENTITY_ENRICHMENT, False + ), + ([splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.SUCCESSFUL, + query_name='query_name1', query_search='query_search1', data=[{'result1': 'a'}, {'result2': 'b'}])], + splunk.DRILLDOWN_ENRICHMENT, True + ), + ([splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.FAILED, + query_name='query_name1', query_search='query_search1', data=[{'result1': 'a'}, {'result2': 'b'}])], + splunk.DRILLDOWN_ENRICHMENT, False + ), + ([splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.SUCCESSFUL, + data=[{'result1': 'a'}, {'result2': 'b'}]), + splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.FAILED, + data=[{'result1': 'a'}, {'result2': 'b'}])], splunk.DRILLDOWN_ENRICHMENT, True + ), + ([splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.FAILED, + data=[{'result1': 'a'}, {'result2': 'b'}]), + splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.SUCCESSFUL, + data=[{'result1': 'a'}, {'result2': 'b'}])], splunk.DRILLDOWN_ENRICHMENT, True + ), + ([splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.FAILED, + data=[{'result1': 'a'}, {'result2': 'b'}]), + splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.FAILED, + data=[{'result1': 'a'}, {'result2': 'b'}])], splunk.DRILLDOWN_ENRICHMENT, False + ), + ([splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.SUCCESSFUL, + data=[{'result1': 'a'}, {'result2': 'b'}]), + splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.SUCCESSFUL, + data=[{'result1': 'a'}, {'result2': 'b'}])], splunk.DRILLDOWN_ENRICHMENT, True + ), + ([splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.FAILED, + data=[{'result1': 'a'}, {'result2': 'b'}]), + splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.SUCCESSFUL, + data=[{'result1': 'a'}, {'result2': 'b'}]), + splunk.Enrichment(splunk.DRILLDOWN_ENRICHMENT, enrichment_id='1', status=splunk.Enrichment.FAILED, + data=[{'result1': 'a'}, {'result2': 'b'}])], splunk.DRILLDOWN_ENRICHMENT, True + ) +], ids=[ + "A Notable with 1 successful Asset enrichment", + "A Notable with 1 failed Asset enrichment", + "A Notable with 1 successful Identity enrichment", + "A Notable with 1 failed Identity enrichment", + "A Notable with 1 successful Drilldown enrichment", + "A Notable with 1 failed Drilldown enrichment", + "A Notable with 1 successful Drilldown enrichment and 1 failed drilldown enrichment (the first is successful)", + "A Notable with 1 successful Drilldown enrichment and 1 failed drilldown enrichment (the second is successful)", + "A Notable with 2 Drilldown enrichments [failed, failed]", + "A Notable with 2 Drilldown enrichments [successful, successful]", + "A Notable with 3 Drilldown enrichments [failed, successful, failed]" +]) +def test_to_incident_notable_enrichments_status(enrichments, enrichment_type, expected_stauts_result): + """ + Tests the logic of the Notable.to_incident() function, regarding the statuses of enrichments. + + Given: + 1. A Notable with 1 successful Asset enrichment. + 2. A Notable with 1 failed Asset enrichment. + 3. A Notable with 1 successful Identity enrichment. + 4. A Notable with 1 failed Identity enrichment. + 5. A Notable with 1 successful Drilldown enrichment. + 6. A Notable with 1 failed Drilldown enrichment. + 7. A Notable with 1 successful Drilldown enrichment and 1 failed drilldown enrichment (the first is successful). + 8. A Notable with 1 successful Drilldown enrichment and 1 failed drilldown enrichment (the second is successful). + 9. A Notable with 2 Drilldown enrichments [failed, failed]. + 10. A Notable with 2 Drilldown enrichments [successful, successful]. + 11. A Notable with 3 Drilldown enrichments [failed, successful, failed]. + + + When: + - Notable.to_incident() function is called + + Then: + - Verify that the status of the notable enrichments is as follow: + 1. Asset Enrichment status is: successful_asset_enrichment = True. + 2. Asset Enrichment status is: successful_asset_enrichment = False. + 3. Identity Enrichment status is: successful_identity_enrichment = True. + 4. Identity Enrichment status is: successful_identity_enrichment = False. + + # In Drilldown enrichment - if at least one drilldown enrichment is successful the status is Success. + 5. Drilldown Enrichment status is: successful_drilldown_enrichment = True. + 6. Drilldown Enrichment status is: successful_drilldown_enrichment = False. + 7. Drilldown Enrichment status is: successful_drilldown_enrichment = True. + 8. Drilldown Enrichment status is: successful_drilldown_enrichment = True. + 9. Drilldown Enrichment status is: successful_drilldown_enrichment = False. + 10. Drilldown Enrichment status is: successful_drilldown_enrichment = True. + 11. Drilldown Enrichment status is: successful_drilldown_enrichment = True. + + """ + notable = splunk.Notable({}, notable_id='id', enrichments=enrichments) + service = Service('DONE') + mapper = splunk.UserMappingObject(service, False) + notable.to_incident(mapper, 'comment_tag_to_splunk', 'comment_tag_from_splunk') + + assert notable.data[splunk.ENRICHMENT_TYPE_TO_ENRICHMENT_STATUS[enrichment_type]] == expected_stauts_result + + +def test_parse_drilldown_searches(): + """ + Given: + - A list of valid Json strings with splunk drilldown searches data. + + When: + - Running the splunk.parse_drilldown_searches function + + Then: + - Verify that the search data was parsed into a python dictionary as expected. + """ + searches = ["{\"name\":\"View related '$signature$' events for $dest$\",\"search\":\"| from datamodel:\\\"Malware\\\"." + "\\\"Malware_Attacks\\\" | search dest=$dest|s$ signature=$signature|s$\",\"earliest\":17145" + "63300,\"latest\":1715168700}", + "{\"name\":\"View related '$category$' events for $signature$\",\"search\":\"| from datamodel:\\\"Malw" + "are\\\".\\\"Malware_Attacks\\\" \\n| fields category, dest, signature | search dest=$dest|s$ signature=" + "$signature|s$\",\"earliest\":1714563300,\"latest\":1715168700}" + ] + parsed_searches = splunk.parse_drilldown_searches(searches) + for search in parsed_searches: + assert isinstance(search, dict) + assert parsed_searches == [ + {'name': "View related '$signature$' events for $dest$", + 'search': '| from datamodel:"Malware"."Malware_Attacks" | search dest=$dest|s$ signature=$signature|s$', + 'earliest': 1714563300, + 'latest': 1715168700 + }, + {'name': "View related '$category$' events for $signature$", + 'search': '| from datamodel:"Malware"."Malware_Attacks" \n| fields category, dest, signature | search dest=$dest|s$ ' + 'signature=$signature|s$', + 'earliest': 1714563300, + 'latest': 1715168700 + } + ] + + +@pytest.mark.parametrize('notable_data, expected_call_count', [ + ({'event_id': 'test_id', 'drilldown_search': 'test_search', 'drilldown_searches': ['test_search1', 'test_search2']}, 0), + ({'event_id': 'test_id', 'drilldown_search': '', 'drilldown_searches': ['test_search1', 'test_search2']}, 1), + ({'event_id': 'test_id', 'drilldown_searches': ['test_search1', 'test_search2']}, 1) +], ids=[ + "A notable data with both 'drilldown_search' and 'drilldown_searches' keys with values", + "A notable data with both 'drilldown_search' and 'drilldown_searches' keys but 'drilldown_search' has no value", + "A notable data with 'drilldown_searches' key only" +]) +def test_drilldown_enrichment_main_condition(mocker, notable_data, expected_call_count): + """ + Tests the logic of the first (main) condition in the drilldown_enrichment() function. + We want to make sure that in a case that the notable data include both 'drilldown_search' and 'drilldown_searches' + keys (happens when there is only one drilldown search to enrich) the 'drilldown_search' value will be taken to maintain + backwards cometability. In any other case the value of the 'drilldown_searches' key will be used. + + Given: + 1. A notable data that includes both 'drilldown_search' and 'drilldown_searches' keys with values. + 2. A notable data that includes both 'drilldown_search' and 'drilldown_searches' keys but 'drilldown_search' has no value. + 3. A notable data that includes 'drilldown_searches' key only. + + When: + - Running the splunk.drilldown_enrichment function + + Then: + - Verify that: + 1. The value of the 'drilldown_search' key is taken (to maintain backwards competability), and therefore we don't call the + parse_drilldown_searches function. + 2. The value of the 'drilldown_searches' key is taken, and therefore we call the parse_drilldown_searches function. + 3. The value of the 'drilldown_searches' key is taken, and therefore we call the parse_drilldown_searches function. + + """ + mock_parse_drilldown_searches = mocker.patch('SplunkPy.parse_drilldown_searches', return_value=[]) + service = Service('DONE') + splunk.drilldown_enrichment(service, notable_data, 5) + assert mock_parse_drilldown_searches.call_count == expected_call_count + + +@pytest.mark.parametrize('notable_data, expected_call_count', [ + ({'event_id': 'test_id', 'drilldown_search': 'test_search', 'drilldown_searches': [{}], '_raw': "{'test':1}"}, 1), + ({'event_id': 'test_id', + 'drilldown_searches': + ["{\"name\":\"View related '$signature$' events for $dest$\",\"search\":\"| from datamodel:\\\"Malware\\\".\\\"Malwa" + "re_Attacks\\\" | search dest=$dest|s$ signature=$signature|s$\",\"earliest\":1714563300,\"latest\":1715168700}", + "{\"name\":\"View related '$category$' events for $signature$\",\"search\":\"| from datamodel:\\\"Malware\\\".\\\"M" + "alware_Attacks\\\" \\n| fields category, dest, signature | search dest=$dest|s$ signature=$signature|s$\",\"ear" + "liest\":1714563300,\"latest\":1715168700}" + ] + }, + 0) +], ids=[ + "A notable data with one drilldown search", + "A notable data with multiple drilldown searches" +]) +def test_drilldown_enrichment_get_timeframe(mocker, notable_data, expected_call_count): + """ + Tests that in a case of one drildown search we extract the search timeframe from the notable data by calling the + get_drilldown_timeframe() function, and in a case of multiple drilldown searches, we get the timeframe from the drilldown + search data dictionary without calling the get_drilldown_timeframe() function. + + Given: + 1. A notable data with one drilldown search. + 2. A notable data with multiple drilldown searches. + + + When: + - Running the splunk.get_drilldown_timeframe function. + + Then: + - Verify that: + 1. The timeframe is determined according to fields in the notable data and raw data by using the + get_drilldown_timeframe function. + 2. The timeframe is determined according to fields of each drilldown search data dict. + + """ + mock_get_drilldown_timeframe = mocker.patch('SplunkPy.get_drilldown_timeframe', return_value=("", "")) + mocker.patch('SplunkPy.build_drilldown_search', return_value='') + service = Service('DONE') + splunk.drilldown_enrichment(service, notable_data, 5) + assert mock_get_drilldown_timeframe.call_count == expected_call_count + + +@pytest.mark.parametrize('notable_data, expected_result', [ + ({'event_id': 'test_id', 'drilldown_name': 'View all login attempts by system $src$', + 'drilldown_search': "| from datamodel:\"Authentication\".\"Authentication\" | search src=$src|s$", + 'drilldown_searches': "{\"name\":\"View all login attempts by system $src$\",\"search\":\"| from datamodel:\\\"Authent" + "ication\\\".\\\"Authentication\\\" | search src=$src|s$\",\"earliest\":1715040000,\"latest\":1715126400}", + '_raw': "src=\'test_src\'", "drilldown_latest": "1715126400.000000000", "drilldown_earliest": "1715040000.000000000"}, + [ + ("View all login attempts by system 'test_src'", + '| from datamodel:"Authentication"."Authentication" | search src="\'test_src\'"')]), + + ({'event_id': 'test_id2', 'drilldown_searches': + ["{\"name\":\"View all login attempts by system $src$\",\"search\":\"| from datamodel:\\\"Authentication\\\".\\\"Authe" + "ntication\\\" | search src=$src|s$\",\"earliest\":1715040000,\"latest\":1715126400}", + "{\"name\":\"View all test involving user=\\\"$user$\\\"\",\"search\":\"index=\\\"test\\\"\\n| where " + "user = $user|s$\",\"earliest\":1716955500,\"latest\":1716959400}"], + '_raw': "src=\'test_src\', user='test_user'"}, + [("View all login attempts by system 'test_src'", + '| from datamodel:"Authentication"."Authentication" | search src="\'test_src\'"'), + ('View all test involving user="\'test_user\'"', + 'search index="test"\n| where user = \'test_user\'')]), +], ids=[ + "A notable data with one drilldown search enrichment", + "A notable data with multiple (two) drilldown searches to enrich" +]) +def test_drilldown_enrichment(notable_data, expected_result): + """ + Tests the logic of the drilldown_enrichment function. + + Given: + 1. A notable data with one drilldown search enrichment. + 2. A notable data with multiple (two) drilldown searches to enrich. + + + When: + - Running the splunk.drilldown_enrichment function. + + Then: + - Verify that the returned jobs and queries are as expected. + + """ + from splunklib import client + service = Service('DONE') + jobs_and_queries = splunk.drilldown_enrichment(service, notable_data, 5) + for i in range(len(jobs_and_queries)): + job_and_queries = jobs_and_queries[i] + assert job_and_queries[0] == expected_result[i][0] + assert job_and_queries[1] == expected_result[i][1] + assert isinstance(job_and_queries[2], client.Job) + + +@pytest.mark.parametrize('notable_data, debug_log_message', [ + ({'event_id': 'test_id'}, 'drill-down was not configured for notable test_id'), + + ({'event_id': 'test_id', 'drilldown_name': 'View all login attempts by system $src$', + 'drilldown_search': "| from datamodel:\"Authentication\".\"Authentication\" | search src=$src|s$", + '_raw': "src=\'test_src\'", "drilldown_latest": "", "drilldown_earliest": ""}, + 'Failed getting the drilldown timeframe for notable test_id'), + + ({'event_id': 'test_id', 'drilldown_name': 'View all login attempts by system $src$', + 'drilldown_search': "| from datamodel:\"Authentication\".\"Authentication\" | search src=$src|s$", '_raw': "", + "drilldown_latest": "00101", "drilldown_earliest": "00001"}, + "Couldn't build search query for notable test_id with the following drilldown search "), + + ({'event_id': 'test_id', + 'drilldown_searches': [ + "{\"name\":\"View all login attempts by system $src$\",\"search\":\"| from datamodel:\\\"Authentica" + "tion\\\".\\\"Authentication\\\" | search src=$src|s$\",\"earliest\":,\"latest\":}", + "{\"name\":\"View all test involving user=\\\"$user$\\\"\",\"search\":\"index=\\\"test\\\"\\n| where user =" + "$user|s$\",\"earliest\":,\"latest\":}"], + '_raw': "src=\'test_src\', user='test_user'"}, + 'Failed getting the drilldown timeframe for notable test_id'), + + ({'event_id': 'test_id', + 'drilldown_searches': + ["{\"name\":\"View all login attempts by system $src$\",\"search\":\"| from datamodel:\\\"Authentic" + "ation\\\".\\\"Authentication\\\" | search src=$src|s$\",\"earliest\":,\"latest\":}", + "{\"name\":\"View all test involving user=\\\"$user$\\\"\",\"search\":\"index=\\\"test\\\"\\n| where user =" + "$user|s$\",\"earliest\":,\"latest\":}"], '_raw': ""}, + "Couldn't build search query for notable test_id with the following drilldown search"), +], ids=[ + "A notable data without drilldown enrichment data", + "A notable data with a single drilldown enrichment without search timeframe data", + "A notable data with a single drilldown enrichment with an invalid search query", + "A notable data with multiple drilldown enrichments without search timeframe data", + "A notable data with multiple drilldown enrichments with invalid search queries" +]) +def test_drilldown_enrichment_no_enrichement_cases(mocker, notable_data, debug_log_message): + """ + Tests the logic of the drilldown_enrichment function when for some reason the enrichments raw data is invalid. + + Given: + 1. A notable data without drilldown enrichment data. + 2. A notable data with a single drilldown enrichment without search timeframe data. + 3. A notable data with a single drilldown enrichment with an invalid search query. + 4. A notable data with multiple drilldown enrichments without search timeframe data. + 5. A notable data with multiple drilldown enrichments with invalid search queries. + + When: + - Running the splunk.drilldown_enrichment function. + + Then: + - Verify that the returned value is a tuple of None values as expected. + + """ + debug_log = mocker.patch.object(demisto, 'debug') + mocker.patch.object(demisto, 'error') + service = Service('DONE') + jobs_and_queries = splunk.drilldown_enrichment(service, notable_data, 5) + for i in range(len(jobs_and_queries)): + assert jobs_and_queries[i] == (None, None, None) + assert debug_log_message in debug_log.call_args.args[0] + + """ ========== Mirroring Mechanism Tests ========== """ diff --git a/Packs/SplunkPy/ReleaseNotes/3_1_28.json b/Packs/SplunkPy/ReleaseNotes/3_1_28.json new file mode 100644 index 00000000000..b9cfca802a5 --- /dev/null +++ b/Packs/SplunkPy/ReleaseNotes/3_1_28.json @@ -0,0 +1 @@ +{"breakingChanges":true,"breakingChangesNotes":"You can now get results of multiple drilldown searches enrichment associated with a Splunk notable. The results of multiple drilldown searches are stored in the context of the incident under the 'Drilldown' field as follows: [{'query_name':, 'query_search': , 'query_results': [{result1}, {result2}, {result3}], 'enrichment_status': }]."} diff --git a/Packs/SplunkPy/ReleaseNotes/3_1_28.md b/Packs/SplunkPy/ReleaseNotes/3_1_28.md new file mode 100644 index 00000000000..7fa6458368d --- /dev/null +++ b/Packs/SplunkPy/ReleaseNotes/3_1_28.md @@ -0,0 +1,13 @@ + +#### Integrations + +##### SplunkPy + +Added support for getting results of multiple drilldown searches enrichment associated with a Splunk notable (supported from Enterprise Security v7.2.0). + +#### Scripts + +##### SplunkShowDrilldown + +- Added support for displaying results of multiple drilldown searches enrichment associated with a Splunk notable (supported from Enterprise Security v7.2.0). +- Updated the Docker image to: *demisto/python3:3.10.14.95956*. diff --git a/Packs/SplunkPy/Scripts/SplunkShowDrilldown/SplunkShowDrilldown.py b/Packs/SplunkPy/Scripts/SplunkShowDrilldown/SplunkShowDrilldown.py index 703ff460cce..0a824f5e7a9 100644 --- a/Packs/SplunkPy/Scripts/SplunkShowDrilldown/SplunkShowDrilldown.py +++ b/Packs/SplunkPy/Scripts/SplunkShowDrilldown/SplunkShowDrilldown.py @@ -25,10 +25,29 @@ def main(): return CommandResults(readable_output='Drilldown was not configured for notable.') if isinstance(drilldown_results, list): - events_arr = [] - for event in drilldown_results: - events_arr.append(event) - markdown = tableToMarkdown("", events_arr, headers=events_arr[0].keys()) + if 'query_name' in drilldown_results[0]: + # Get drilldown results of multiple drilldown searches + markdown = "#### Drilldown Searches Results\n" + + for drilldown in drilldown_results: + markdown += f"**Query Name:** {drilldown.get('query_name','')}\n\n **Query"\ + f"Search:**\n{drilldown.get('query_search','')}\n\n **Results:**\n" + + if drilldown.get('enrichment_status') == 'Enrichment failed': + markdown += "\nDrilldown enrichment failed." + + elif results := drilldown.get("query_results", []): + markdown += tableToMarkdown("", results, headers=results[0].keys()) + + else: + markdown += "\nNo results found for drilldown search." + + markdown += "\n\n" + + else: + # Drilldown results of a single drilldown search + markdown = tableToMarkdown("", drilldown_results, headers=drilldown_results[0].keys()) + else: markdown = tableToMarkdown("", drilldown_results) diff --git a/Packs/SplunkPy/Scripts/SplunkShowDrilldown/SplunkShowDrilldown.yml b/Packs/SplunkPy/Scripts/SplunkShowDrilldown/SplunkShowDrilldown.yml index 9970c7039a6..6a5a196c3f1 100644 --- a/Packs/SplunkPy/Scripts/SplunkShowDrilldown/SplunkShowDrilldown.yml +++ b/Packs/SplunkPy/Scripts/SplunkShowDrilldown/SplunkShowDrilldown.yml @@ -13,7 +13,7 @@ type: python contentitemexportablefields: contentitemfields: fromServerVersion: '' -dockerimage: demisto/python3:3.10.13.87159 +dockerimage: demisto/python3:3.10.14.95956 runas: DBotWeakRole tests: - SplunkShowEnrichment diff --git a/Packs/SplunkPy/Scripts/SplunkShowDrilldown/SplunkShowDrilldown_test.py b/Packs/SplunkPy/Scripts/SplunkShowDrilldown/SplunkShowDrilldown_test.py index 556b1b07176..4e6cad28245 100644 --- a/Packs/SplunkPy/Scripts/SplunkShowDrilldown/SplunkShowDrilldown_test.py +++ b/Packs/SplunkPy/Scripts/SplunkShowDrilldown/SplunkShowDrilldown_test.py @@ -1,3 +1,4 @@ +import json import SplunkShowDrilldown from pytest import raises @@ -11,10 +12,10 @@ def test_incident_with_empty_custom_fields(mocker): Then: Verifies that the output returned is correct """ - incident = {'CustomFields': {}} - mocker.patch('demistomock.incident', return_value=incident) + incident = {"CustomFields": {}} + mocker.patch("demistomock.incident", return_value=incident) res = SplunkShowDrilldown.main() - assert res.readable_output == 'Drilldown was not configured for notable.' + assert res.readable_output == "Drilldown was not configured for notable." def test_incident_not_notabledrilldown(mocker): @@ -26,10 +27,10 @@ def test_incident_not_notabledrilldown(mocker): Then: Verifies that the output returned is correct """ - incident = {'CustomFields': {"notabledrilldown": {}}} - mocker.patch('demistomock.incident', return_value=incident) + incident = {"CustomFields": {"notabledrilldown": {}}} + mocker.patch("demistomock.incident", return_value=incident) res = SplunkShowDrilldown.main() - assert res.readable_output == 'Drilldown was not configured for notable.' + assert res.readable_output == "Drilldown was not configured for notable." def test_incident_not_successful(mocker): @@ -41,10 +42,10 @@ def test_incident_not_successful(mocker): Then: Verifies that the output returned is correct """ - incident = {'labels': [{'type': 'successful_drilldown_enrichment', 'value': 'false'}]} - mocker.patch('demistomock.incident', return_value=incident) + incident = {"labels": [{"type": "successful_drilldown_enrichment", "value": "false"}]} + mocker.patch("demistomock.incident", return_value=incident) res = SplunkShowDrilldown.main() - assert res.readable_output == 'Drilldown enrichment failed.' + assert res.readable_output == "Drilldown enrichment failed." def test_json_loads_fails(mocker): @@ -56,7 +57,294 @@ def test_json_loads_fails(mocker): Then: Verifies that the output returned is correct """ - incident = {'labels': [{'type': 'Drilldown', 'value': {'not json'}}]} - mocker.patch('demistomock.incident', return_value=incident) + incident = {"labels": [{"type": "Drilldown", "value": {"not json"}}]} + mocker.patch("demistomock.incident", return_value=incident) with raises(ValueError): SplunkShowDrilldown.main() + + +def test_incident_single_drilldown_search_results(mocker): + """ + Given: + incident with results of a single drilldown search + When: + Calling to SplunkShowDrilldown + Then: + Verifies that the output returned is correct + """ + incident = { + "labels": [ + {"type": "successful_drilldown_enrichment", "value": "true"}, + { + "type": "Drilldown", + "value": """[ + {"_bkt": "main~Test1", + "_cd": "524:1111111", + "_indextime": "1715859867", + "_raw": "2024-05-16 11:26:32,Virus found,IP Address: 1.1.1.1,Computer name: Test1", + "_serial": "0", + "_si": [ + "ip-1-1-1-1", + "main" + ], + "_sourcetype": "test1", + "_time": "2024-05-16T11:26:32.000+00:00", + "category": "Other", + "dest": "Test_dest1", + "signature": "test_signature1" + }, + {"_bkt": "main~Test2", + "_cd": "524:2222222", + "_indextime": "1715859867", + "_raw": "2024-05-16 11:26:32,Virus found,IP Address: 2.2.2.2,Computer name: Test2", + "_serial": "0", + "_si": [ + "ip-2-2-2-2", + "main" + ], + "_sourcetype": "test2", + "_time": "2024-05-16T11:26:32.000+00:00", + "category": "Other", + "dest": "Test_dest2", + "signature": "test_signature2" + } + ]""", + }, + ] + } + mocker.patch("demistomock.incident", return_value=incident) + res = SplunkShowDrilldown.main() + contents: str = res.get("Contents") + # Verify that all results are in the markdown table + assert ("main~Test1" and "test_signature1" and "main~Test2" and "test_signature2") in contents + + +def test_incident_multiple_drilldown_search_results(mocker): + """ + Given: + incident with results of multiple drilldown searches + When: + Calling to SplunkShowDrilldown + Then: + Verifies that the output returned is correct + """ + drilldown = [ + {"query_name": "query_name1", + "query_search": "query_search1", + "enrichment_status": "Enrichment successfully handled", + "query_results": [ + { + "_bkt": "main~Test1", + "_cd": "524:1111111", + "_indextime": "1715859867", + "_raw": "2024-05-16 11:26:32,Virus found,IP Address: 1.1.1.1,Computer name: Test1", + "_serial": "0", + "_si": ["ip-1-1-1-1", "main"], + "_sourcetype": "test1", + "_time": "2024-05-16T11:26:32.000+00:00", + "category": "Other", + "dest": "Test_dest1", + "signature": "test_signature1", + }, + { + "_bkt": "main~Test2", + "_cd": "524:2222222", + "_indextime": "1715859867", + "_raw": "2024-05-16 11:26:32,Virus found,IP Address: 2.2.2.2,Computer name: Test2", + "_serial": "0", + "_si": ["ip-2-2-2-2", "main"], + "_sourcetype": "test2", + "_time": "2024-05-16T11:26:32.000+00:00", + "category": "Other", + "dest": "Test_dest2", + "signature": "test_signature2", + }, + ], + + }, + {"query_name": "query_name2", + "query_search": "query_search2", + "enrichment_status": "Enrichment successfully handled", + "query_results": [ + { + "_bkt": "main~Test3", + "_cd": "524:1111111", + "_indextime": "1715859867", + "_raw": "2024-05-16 11:26:32,Virus found,IP Address: 1.1.1.1,Computer name: Test1", + "_serial": "0", + "_si": ["ip-1-1-1-1", "main"], + "_sourcetype": "test1", + "_time": "2024-05-16T11:26:32.000+00:00", + "category": "Other", + "dest": "Test_dest1", + "signature": "test_signature3", + }, + { + "_bkt": "main~Test4", + "_cd": "524:2222222", + "_indextime": "1715859867", + "_raw": "2024-05-16 11:26:32,Virus found,IP Address: 2.2.2.2,Computer name: Test2", + "_serial": "0", + "_si": ["ip-2-2-2-2", "main"], + "_sourcetype": "test2", + "_time": "2024-05-16T11:26:32.000+00:00", + "category": "Other", + "dest": "Test_dest2", + "signature": "test_signature4", + }, + ], + } + ] + str_drilldown = json.dumps(drilldown) + incident = { + "labels": [ + {"type": "successful_drilldown_enrichment", "value": "true"}, + {"type": "Drilldown", + "value": str_drilldown + } + ] + } + mocker.patch("demistomock.incident", return_value=incident) + res = SplunkShowDrilldown.main() + contents: str = res.get("Contents") + # Verify that all results are in the markdown table + assert ("main~Test1" and "test_signature1" and "main~Test2" and "test_signature2") in contents + assert ("query_name1" and "query_search1" and "query_name2" and "query_search2") in contents + assert ("main~Test3" and "test_signature3" and "main~Test4" and "test_signature4") in contents + assert ("Drilldown Searches Results") in contents + + +def test_incident_multiple_drilldown_search_no_results(mocker): + """ + Given: + incident with results of multiple drilldown searches, one of the drilldown searches returned no results + When: + Calling to SplunkShowDrilldown + Then: + Verifies that the output returned is correct + """ + drilldown = [ + {"query_name": "query_name1", + "query_search": "query_search1", + "enrichment_status": "Enrichment successfully handled", + "query_results": [ + { + "_bkt": "main~Test1", + "_cd": "524:1111111", + "_indextime": "1715859867", + "_raw": "2024-05-16 11:26:32,Virus found,IP Address: 1.1.1.1,Computer name: Test1", + "_serial": "0", + "_si": ["ip-1-1-1-1", "main"], + "_sourcetype": "test1", + "_time": "2024-05-16T11:26:32.000+00:00", + "category": "Other", + "dest": "Test_dest1", + "signature": "test_signature1", + }, + { + "_bkt": "main~Test2", + "_cd": "524:2222222", + "_indextime": "1715859867", + "_raw": "2024-05-16 11:26:32,Virus found,IP Address: 2.2.2.2,Computer name: Test2", + "_serial": "0", + "_si": ["ip-2-2-2-2", "main"], + "_sourcetype": "test2", + "_time": "2024-05-16T11:26:32.000+00:00", + "category": "Other", + "dest": "Test_dest2", + "signature": "test_signature2", + }, + ], + + }, + {"query_name": "query_name2", + "query_search": "query_search2", + "enrichment_status": "Enrichment successfully handled", + "query_results": [], + } + ] + str_drilldown = json.dumps(drilldown) + incident = { + "labels": [ + {"type": "successful_drilldown_enrichment", "value": "true"}, + {"type": "Drilldown", + "value": str_drilldown + } + ] + } + mocker.patch("demistomock.incident", return_value=incident) + res = SplunkShowDrilldown.main() + contents: str = res.get("Contents") + # Verify that all results are in the markdown table + assert ("main~Test1" and "test_signature1" and "main~Test2" and "test_signature2") in contents + assert ("Drilldown Searches Results") in contents + assert ("query_name1" and "query_search1" and "query_name2" and "query_search2") in contents + assert ("No results found for drilldown search") in contents + + +def test_incident_multiple_drilldown_search_enrichment_failed(mocker): + """ + Given: + incident with results of multiple drilldown searches, one of the drilldown searches enrichment was failed + When: + Calling to SplunkShowDrilldown + Then: + Verifies that the output returned is correct + """ + drilldown = [ + {"query_name": "query_name1", + "query_search": "query_search1", + "enrichment_status": "Enrichment successfully handled", + "query_results": [ + { + "_bkt": "main~Test1", + "_cd": "524:1111111", + "_indextime": "1715859867", + "_raw": "2024-05-16 11:26:32,Virus found,IP Address: 1.1.1.1,Computer name: Test1", + "_serial": "0", + "_si": ["ip-1-1-1-1", "main"], + "_sourcetype": "test1", + "_time": "2024-05-16T11:26:32.000+00:00", + "category": "Other", + "dest": "Test_dest1", + "signature": "test_signature1", + }, + { + "_bkt": "main~Test2", + "_cd": "524:2222222", + "_indextime": "1715859867", + "_raw": "2024-05-16 11:26:32,Virus found,IP Address: 2.2.2.2,Computer name: Test2", + "_serial": "0", + "_si": ["ip-2-2-2-2", "main"], + "_sourcetype": "test2", + "_time": "2024-05-16T11:26:32.000+00:00", + "category": "Other", + "dest": "Test_dest2", + "signature": "test_signature2", + }, + ], + + }, + {"query_name": "query_name2", + "query_search": "query_search2", + "enrichment_status": "Enrichment failed", + "query_results": [], + } + ] + str_drilldown = json.dumps(drilldown) + incident = { + "labels": [ + {"type": "successful_drilldown_enrichment", "value": "true"}, + {"type": "Drilldown", + "value": str_drilldown + } + ] + } + mocker.patch("demistomock.incident", return_value=incident) + res = SplunkShowDrilldown.main() + contents: str = res.get("Contents") + # Verify that all results are in the markdown table + assert ("main~Test1" and "test_signature1" and "main~Test2" and "test_signature2") in contents + assert ("Drilldown Searches Results") in contents + assert ("query_name1" and "query_search1" and "query_name2" and "query_search2") in contents + assert ("Drilldown enrichment failed.") in contents diff --git a/Packs/SplunkPy/pack_metadata.json b/Packs/SplunkPy/pack_metadata.json index 7c640c2c731..c61e7ca5b46 100644 --- a/Packs/SplunkPy/pack_metadata.json +++ b/Packs/SplunkPy/pack_metadata.json @@ -2,7 +2,7 @@ "name": "Splunk", "description": "Run queries on Splunk servers.", "support": "xsoar", - "currentVersion": "3.1.27", + "currentVersion": "3.1.28", "author": "Cortex XSOAR", "url": "https://www.paloaltonetworks.com/cortex", "email": "",