Skip to content

Commit

Permalink
v2.2.0 release (#19)
Browse files Browse the repository at this point in the history
  • Loading branch information
bradchiappetta authored May 10, 2023
1 parent 7836b58 commit 23cedc8
Show file tree
Hide file tree
Showing 242 changed files with 5,245 additions and 3,440 deletions.
37 changes: 37 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
repos:
- repo: local
hooks:
- id: yamllint
name: Style check (yamllint)
entry: yamllint
language: system
exclude: ^bin/SA_GreyNoise/
types:
- yaml
- id: black
name: Style check (black)
entry: black
language: system
exclude: ^bin/SA_GreyNoise/
types:
- python
- id: isort
name: Style check (isort)
entry: isort
language: system
exclude: ^bin/SA_GreyNoise/
types:
- python
- id: flake8
name: Static analysis (flake8)
entry: flake8
language: system
exclude: ^bin/SA_GreyNoise/
types:
- python
- repo: https://github.com/adrienverge/yamllint.git
rev: v1.17.0
hooks:
- id: yamllint
args: [-c=.yamllint]
9 changes: 9 additions & 0 deletions .yamllint
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---

extends: default

rules:
line-length:
max: 88

ignore-from-file: .gitignore
38 changes: 35 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This is an app powered by the Splunk Add-on Builder.
GreyNoise Splunk app provides multiple dashboards to effectively analyse and visualize the contextual and statistical data provided by GreyNoise. It also includes custom commands and alert actions which can be used along with Splunk searches to leverage GreyNoise APIs for custom use cases. It periodically scans the Splunk deployment through saved search to indicate the noise and RIOT IPs in the complete Splunk deployment. Along with this, the workflow action provided can be used to obtain live context information of any CIM compliant field containing an IP address.

- Author: GreyNoise Intelligence Inc
- Version: 2.1.5
- Version: 2.2.0
- Creates Index: False
- Has index-time operation: True
- Implements summarization: False
Expand All @@ -17,8 +17,17 @@ GreyNoise Splunk app provides multiple dashboards to effectively analyse and vis
- OS: Platform independent
- Vendor Products: GreyNoise API

# RELEASE NOTES (Version 2.2.0) #
- Added new FEED component to create lookuptable based on GreyNoise indicators
- Added new command `gnipsimilar` and new `Similar IP Lookup` dashboard
- Added new command `gniptimeline` and new `IP Timeline Lookup` dashboard
- Updated `gnenrich` command to use batch lookups
- Updated `gnquery` command with new parameters `page_size` and `exclude_raw`
- Updated GreyNoise SDK to v2.0.1

# RELEASE NOTES (Version 2.1.5) #
- Fix bug with `gnenrich`, `gnriot`, and `gnfilter` where proxy wasn't being used for API key validation
- Fix credentials.py to deal with null API keys on fresh install

# RELEASE NOTES (Version 2.1.4) #
- Add support for configuring proxy information in conf file
Expand Down Expand Up @@ -110,6 +119,10 @@ Follow the below-listed steps to install an app from the bundle:

*Note: This app contains Adaptive Response Actions, which can be used along with Splunk Enterprise Security. To use these alert actions on the Splunk instance without Splunk Enterprise Security, kindly install `Splunk Common Information Model (CIM)`.*

# UPGRADES #
After applying an update to the app, ensure that the GreyNoise API key is re-entered on the Configuration page.


# CONFIGURATION #
The app can be configured in the following way:

Expand All @@ -135,6 +148,16 @@ This feature helps user to scan the Splunk Deployment and identify the noise and
- Enable Scan Deployment: Checkbox to enable or disable scanning of the deployment.
- Force Scan Deployment: This is useful when user wants to override current running scan immediately and start a new one.

# FEED #
This feature allows users to ingest GreyNoise indicators into a lookup table to be usage within the Splunk environment:

- From the Splunk UI navigate to `Apps > GreyNoise App for Splunk > Configuration`.
- Click on Feed Configuration tab.
- Enter the following details to set up the Feed:
- Enable Feed Import: turns the feature on to enable the daily ingest of GreyNoise indicators via feed
- Force Feed Run Now: starts a manual run of the feed import, rather than waiting for the daily scheduled run
- Feed Selection: select the appropriate option to choose which type of feed to ingest into the system

# Caching #
This feature helps user to enable/disable caching for all the custom commands and saved searches. It can be configured in the
following way:
Expand Down Expand Up @@ -165,8 +188,8 @@ The following commands are included as a part of the app:
- Search format: `| gnquick ip="<ip_address1>,<ip_address2>,<ip_address3>" [OR] SPL_QUERY | gnquick ip_field="<ip_field>"`
- Purpose: Retrieve the noise and RIOT status of all the IP addresses as separate events [OR] Retrieve the noise and RIOT status for all the given IPs returned by the SPL_QUERY for specified ip_field.
- gnquery
- Search format: `| gnquery query="<GNQL_query>" result_size="<result_size>"`
- Purpose: Retrieve the results of the given GNQL query from GreyNoise. result_size denotes the number of results to be retrieved which is capped at 50,000. result_size is an optional parameter with default value of 50,000.
- Search format: `| gnquery query="<GNQL_query>" result_size="<result_size>" page_size="<page_size>"`
- Purpose: Retrieve the results of the given GNQL query from GreyNoise. result_size denotes the number of results to be retrieved which is capped at 50,000. result_size is an optional parameter with default value of 50,000. page_size is an option parameter with a default value of 1000.
- gnstats
- Search format: `| gnstats query="<GNQL_query>" count="<stats_count>"`
- Purpose: Fetch the aggregate statistics for the top organizations, actors, tags, ASNs, countries, classifications, and operating systems of all the results for a given GNQL query. count denotes the number of stats to be retrieved. count is an optional parameter.
Expand All @@ -185,6 +208,12 @@ The following commands are included as a part of the app:
- gnriot
- Search format: `| gnriot ip="<ip_address>" [OR] SPL_QUERY | gnriot ip_field="<ip_field>"`
- Purpose: Retrieves RIOT information for a given IP address from the GreyNoise [OR] Retrieves RIOT information for all the given IPs returned by the SPL_QUERY for specified ip_field.
- gniptimeline
- Search format: `| gniptimeline ip_address="<ip_address>" days=<days> limit=<limit>`
- Purpose: Retrieves Timeline information for a given IP address from the GreyNoise Timeline API.
- gnipsimilar
- Search format: `| gnipsimilar ip_address="<ip_address>" min_score=<min_score> limit=<limit>`
- Purpose: Retrieves Similarity information for a given IP address from the GreyNoise Similarity API.

*Note : While executing the transforming commands from Splunk search UI, ensure that the event count passed to the command is less than 50,000, as per standard limits of Splunk. If the event count is higher than this number, user can create a Saved Search and pass higher number of Splunk statistical data to the command.*

Expand Down Expand Up @@ -219,6 +248,9 @@ This app contains the following saved searches, which are used for populating da
- greynoise_overview_once: Used to populate `gn_overview_lookup` lookup, and is triggered after configuring the API key.
- greynoise_overview: Used to populate `gn_overview_lookup` lookup, and is triggered at an interval of 6 hours.
- greynoise_cache_maintenance: Used to remove those responses whose TTL is expired from the Cache for all the custom commands, and is triggered at an interval of 60 minutes.
- greynoise_feed_once: Used to populate the `greynoise_indicators` lookup with feed results and is triggered on-demand when enabling a Feed.
- greynoise_feed: Used to populate the `greynoise_indicators` lookup with feed results
- greynoise_feed_purge: Used to purge stale indicators (last_seen value over 7 days ago) from the `greynoise_indicators` lookup

*Note : greynoise_scan_deployment_once and greynoise_scan_deployment savedsearches are used for scanning the data indexed in Splunk. So, in case when these saved searches are skipped, the data indexed during that interval will not be scanned for noise and RIOT status.*

Expand Down
5 changes: 5 additions & 0 deletions README/app_greynoise_settings.conf.spec
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@ force_enable_ss =
job_id_overview =
job_id_scan_deployment =

[feed_configuration]
enable_feed_import =
force_enable_ss =
feed_selection =

[caching]
enable_caching =
ttl =
Expand Down
2 changes: 1 addition & 1 deletion app.manifest
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"id": {
"group": null,
"name": "SA-GreyNoise",
"version": "2.1.5"
"version": "2.2.0"
},
"author": [
{
Expand Down
58 changes: 54 additions & 4 deletions appserver/static/js/build/globalConfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,56 @@
"title": "Scan Deployment",
"name": "scan_deployment"
},
{
"entity": [
{
"field":"enable_feed_import",
"type":"checkbox",
"label":"Enable Feed Import",
"defaultValue": 0,
"required":false,
"help":"Select this checkbox to enable the ingest of a GreyNoise Feed. IMPORTANT: Ensure your subscription includes an appropriate license."
},
{
"field": "force_enable_ss",
"label": "Force Feed Run Now",
"type": "checkbox",
"help": "Select this checkbox to immediately begin a feed ingest based on the selected options.",
"required": false
},
{
"required": true,
"type": "singleSelect",
"label": "Feed Selection",
"field": "feed_selection",
"defaultValue": "BENIGN",
"options": {
"autoCompleteFields": [
{
"label": "Benign Indicators - Last 24 Hours",
"value": "BENIGN"
},
{
"label": "Malicious Indicators - Last 24 Hours",
"value": "MALICIOUS"
},
{
"label": "Malicious and Benign Indicators - Last 24 Hours",
"value": "MALICIOUS_BENIGN"
},
{
"label": "All Indicators - Last 24 Hours",
"value": "ALL"
}
],
"disableSearch": true
},
"help": "Select which GreyNoise feed type to ingest."
}
],
"title": "Feed Configuration",
"name": "feed_configuration"
},
{
"entity": [
{
Expand Down Expand Up @@ -218,7 +268,7 @@
"errorMsg": "Time to live should be a valid integer only",
"type": "regex"
}
],
],
"help":"Enter a positive integer in the range 1-72 representing the caches' Time to live (in hours)."
},
{
Expand All @@ -228,8 +278,8 @@
"required":false,
"help":"Check this option to the purge the cache. After clicking the save button, the cache will be cleared."
}
],
"title": "Caching",
],
"title": "Caching",
"name": "caching"
},
{
Expand Down Expand Up @@ -274,7 +324,7 @@
},
"meta": {
"displayName": "GreyNoise App for Splunk",
"version": "2.1.5",
"version": "2.2.0",
"restRoot": "app_greynoise",
"name": "SA-GreyNoise",
"apiVersion": "3.0.0"
Expand Down
138 changes: 138 additions & 0 deletions appserver/static/parameter_validator_modified.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
require(["splunkjs/mvc/utils"], function (SplunkUtil) {
var app_name = SplunkUtil.getCurrentApp();
require.config({
paths: {
'jquery_greynoise': '../app/'+app_name+'/jquery_greynoise',
'datetime-range-picker': '../app/'+app_name+'/daterangepicker.min',
'moment_greynoise': '../app/'+app_name+'/moment'
}
});
require([
'jquery_greynoise',
'splunkjs/mvc',
'splunkjs/mvc/searchmanager',
'moment_greynoise',
'datetime-range-picker',
'splunkjs/mvc/simplexml/ready!'
], function ($, mvc, SearchManager, moment) {

// Get token models
var tokens = mvc.Components.getInstance("default");
var tokensSub = mvc.Components.getInstance("submitted");

var last_7_days = tokens.get("form.last_7_days");

var start_date = moment.utc()
var end_date = moment.utc()

if(last_7_days=="true")
{
start_date = moment.utc().subtract(6, 'days')
end_date = moment.utc()
// Setting the token to null so that on reload it will not auto run the process_inputs
tokens.set("form.last_7_days", null);
tokensSub.set("last_7_days", null)
}

// setting token for the selected default date in text box
tokens.set("form.daterange", start_date.format('YYYY-MM-DD') + ' to ' + end_date.format('YYYY-MM-DD'))

var min_date = ""
var max_date = ""
function setdatetimerange(start, end) {
// setting token for the selected date in text box
tokens.set("form.daterange",start.format('YYYY-MM-DD') + ' to ' + end.format('YYYY-MM-DD'))
min_date = start.format('YYYY-MM-DD')
max_date = end.format('YYYY-MM-DD')
}

$('#daterange').daterangepicker({
startDate: start,
endDate: end,
opens: 'left',
locale: {
format: 'YYYY-MM-DD'
},
ranges: {
'Today': [moment.utc(), moment.utc()],
'Yesterday': [moment.utc().subtract(1, 'days'), moment.utc().subtract(1, 'days')],
'Last 7 Days': [moment.utc().subtract(6, 'days'), moment.utc()],
'Last 30 Days': [moment.utc().subtract(29, 'days'), moment.utc()],
'This Month': [moment.utc().startOf('month'), moment.utc().endOf('month')],
'Last Month': [moment.utc().subtract(1, 'month').startOf('month'), moment.utc().subtract(1, 'month').endOf('month')]
}
}, setdatetimerange);

// Set tokens for start time and end time
setdatetimerange(start_date, end_date);

// Setting token for initializing the default value on load of dashboard
tokens.set("tkn_start_time", start_date);
tokens.set("tkn_end_time", end_date);
$('#daterange').data('daterangepicker').setStartDate(start_date);
$('#daterange').data('daterangepicker').setEndDate(end_date);
$('#daterange :input').attr('readonly', 'true');
$('#daterange :input').css('cursor', 'text');

function process_inputs(){
// Unset tokens which may be set previously
var tokensToReset = ["message", "token_search_trigger"]
for (var i = 0; i < tokensToReset.length; i++) {
tokens.unset(tokensToReset[i]);
tokensSub.unset(tokensToReset[i]);
}

// Validate the parameters
var re = new RegExp("(^\\\"?[\\\*\\\s]*\\\"?$)");
var is_invalid = 1;
var requiredFields = ["tkn_ip_address", "tkn_organization", "tkn_actor", "tkn_tag", "tkn_asn"]
for (i = 0; i < requiredFields.length; i++) {
if (!re.test(tokens.get(requiredFields[i]).trim())) {
is_invalid = 0;
break;
}
}

if (is_invalid == 1) {
tokens.set("message", "Please enter value (without asterisk) in at least one of the following parameters: IP Address, Organization, Actor, Tags or ASN");
tokensSub.set("message", "Please enter value (without asterisk) in at least one of the following parameters: IP Address, Organization, Actor, Tags or ASN");
return;
}

// Escaping the '"'
var tokensToEscapeSpecialCharacter = ["tkn_ip_address", "tkn_organization", "tkn_actor", "tkn_tag", "tkn_os", "tkn_category", "tkn_country", "tkn_asn", "tkn_min_score", "tkn_limit", "tkn_days"]
for (i = 0; i < tokensToEscapeSpecialCharacter.length; i++) {
let tokenVal = tokens.get(tokensToEscapeSpecialCharacter[i]);
if (tokenVal) {
tokenValTrim = tokenVal.trim();
tokenValTrim = tokenValTrim.replace(new RegExp('"','g'),'\\\\\\\"');
tokens.set(tokensToEscapeSpecialCharacter[i] + "_escaped",tokenValTrim);
tokensSub.set(tokensToEscapeSpecialCharacter[i] + "_escaped",tokenValTrim);
}
}

// setting token for API use
tokens.set("tkn_start_time", min_date);
tokens.set("tkn_end_time", max_date);

// Setting the token to run the search again
tokens.set("token_search_trigger", " ");
tokensSub.set("token_search_trigger", " ");
}

// Call the function on click of the Submit button
$("#submit .btn.btn-primary").on("click", process_inputs);

// Token will have value if it is coming to Live Investigation using drilldown.
var redirect = tokens.get("form.redirect");

if(redirect=="redirect")
{
// Setting the token to null so that on reload it will not auto run the process_inputs
tokens.set("form.redirect", null);
tokensSub.set("redirect", null)
$('.applyBtn.btn.btn-sm.btn-primary').click();
$("#submit .btn.btn-primary").click();
}
});
});
8 changes: 8 additions & 0 deletions bin/SA_GreyNoise/bin/greynoise
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/Users/bradchiappetta/repos/SA-GreyNoise/venv/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from greynoise.cli import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())
2 changes: 1 addition & 1 deletion bin/SA_GreyNoise/bin/normalizer
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/usr/local/opt/[email protected]/bin/python3.9
#!/Users/bradchiappetta/repos/SA-GreyNoise/venv/bin/python
# -*- coding: utf-8 -*-
import re
import sys
Expand Down
Loading

0 comments on commit 23cedc8

Please sign in to comment.