Sensor Release Notes
New Features
- Event output can now be exported to S3 buckets in JSON format for ingest into other SIEM tools.
- Added support for Netflow log ingest.
- Improved handling of DNS responses in Device History to properly identify LAN devices.
- Large data spikes now generate a Real Time Event when passing a configurable threshold instead of trying to process excessive amounts of data.
Bug Fixes
- Fixed an issue that caused T1090.004 Domain Fronting Events to be generated when the server name was encrypted.
- Fixed a bug that caused incorrect identification of internal/external IPs when there were multiple internal ranges explicitly defined.
- User lists when sharing dashboard and assigning events should no longer show system accounts.
New Features
- Added detection for ARP Cache Poisoning Adversary in the Middle attacks (T1557.002).
- Added detection for unauthorized network traffic flows to/from IPs based on user specified IP ranges.
- Added parsing for QUIC protocol data.
- Event Filters can now be restricted by Event Type.
- Events can now be marked as unread after viewing them.
Bug Fixes
- Fixed a bug that caused File Masquerading Events (T1036) to trigger on moved files.
New Features
- Users can now assign Events to other users in addition to themselves. Assigned Events can not be aged off until they are unassigned or closed manually.
- High Volume IP Detection now also alerts for high volume Responding IPs in addition to Originators.
- Rules and Filters now support CIDR notation for IP addresses.
- Event data can now be exported to ServiceNow.
Bug Fixes
- Improved batching on Byte Estimator retraining to help ensure new models are created weekly.
New Features
- Biography pages are now unified into a single Device Biography page which encompasses IPs, MAC Addresses, and Hostnames.
- Users can now assign Events to themselves. Event tables can be filtered to only display assigned Events and they will not automatically age off.
- Connection Timelines can now be filtered similarly to other Metrics views.
- IPs can now be whitelisted to prevent generating Ping Sweep (T1595) Events.
- File paths can now be whitelisted by regex to prevent File Masquerading (T1036) Events.
- HK (Hong Kong) has been added to country select fields.
Bug Fixes
- Fixed a bug that caused External Self Signed Certificate (T1587) Events to always display the originating IP regardless of which side was external.
- Fixed a bug that caused Latest Activity Date to not update correctly on High Volume IP Events.
- Fixed a bug that caused database ingest to be significantly slower with large numbers of repeated x509 certificates.
- New Local MAC Events no longer include Evidence of other types.
New Features
- Hostname Biography: Clicking a hostname will now display a summary of traffic that has been associated with that hostname.
- Users can now share dashboards with other users on their deployment. The other user will receive a newly created copy of the dashboard, modifying one does not affect the other.
- High Volume IP Detection: Events will now be generated when a single IP is responsible for a large percentage of traffic on a sensor.
- Adversary in the Middle (T1557.001) now combines counts for the same IP both originating and responding to determine if it meets the activity threshold.
- Dashboard cards now indicate when they are under the effect of any global filter. The global filters page also has a clear all button.
- The Notes text area is now expandable and appears at the top of the Notes tab.
Bug Fixes
- View Notes permission has been renamed to Access Notes to more correctly indicate the permission allows viewing and editing.
- Fixed a bug that allowed users to edit other users' dashboards through API calls.
New Features
- Events now display an indicator when a user has made a Note. A red dot will appear on the magnifying glass if there is an unread Note. The dot will be gray if the Note has been read.
- The Event table can now be filtered by whether or not an Event has Notes attached.
- CIDR Notation can now be used in filters for IP fields and Traffic Query.
- Users can now duplicate their saved dashboards.
- The SHA256 field is now populated on File objects.
- T1133 Events now indicate whether the SSH connection is inbound or outbound.
- DNS Answers can now be used as a match field for Indicator Uploads.
- Data can now be exported to Splunk. Users can choose whether to export Zeek data, Events, or both.
- Users can now fully log out of their Keycloak session.
Bug Fixes
- Fixed a bug that could cause Events to appear multiple times in an Event Reports card when using a not equals (!=) filter.
- The Responder IP Bytes field is now correctly populated on Connection objects.
- Fixed a bug that caused an error when trying to view Geolocation on the IP Biography page.
- Fixed a bug that caused Proofpoint enrichments to be duplicated on some Events.
- Fixed a bug that caused the CYBERSPAN logo to not appear on popped out windows.
- Fixed a bug that caused the Graceful Shutdown button to not gracefully shutdown.
New Features
- IP Addresses can now be clicked to link out to the following external sites: AbuseIPDB, Censys, Domain Dossier, Shodan, and VirusTotal
- Autonomous System (AS) Number and Organization are now associated with IP addresses throughout the system. They are displayed on the Foreign IP Reports card and can be used in Filters.
- DNS Answers can now be used in Rules and Filters.
- Redesigned the Network Service Discovery (T1046) detection analytic to prevent Events from growing too large.
- Redesigned the Denial of Service (T1498) detection analytic to operate as a standard analytic instead of a pretrained cluster.
- Narrowed the set of DNS query types that are checked by the Domain Generation Algorithm Detection model to exclude validation string and machine intended domain names (such as TXT and MX records).
- Updated the Event Viewer to keep the Notes view visible when switching between tabs.
- Added port 22 for SSH to the standard port list.
Bug Fixes
- Fixed a bug that caused all dashboard cards to refresh whenever any one card was changed or moved.
- Fixed a bug that caused model retraining to fail if there were no DHCP items in a data batch.
New Features
- IP Addresses can now be clicked to link out to the following external sites: AbuseIPDB, Censys, Domain Dossier, Shodan, and VirusTotal
- Autonomous System (AS) Number and Organization are now associated with IP addresses throughout the system. They are displayed on the Foreign IP Reports card and can be used in Filters.
- DNS Answers can now be used in Rules and Filters.
- Redesigned the Network Service Discovery (T1046) detection analytic to prevent Events from growing too large.
- Redesigned the Denial of Service (T1498) detection analytic to operate as a standard analytic instead of a pretrained cluster.
- Narrowed the set of DNS query types that are checked by the Domain Generation Algorithm Detection model to exclude validation string and machine intended domain names (such as TXT and MX records).
- Updated the Event Viewer to keep the Notes view visible when switching between tabs.
- Added port 22 for SSH to the standard port list.
Bug Fixes
- Fixed a bug that caused all dashboard cards to refresh whenever any one card was changed or moved.
- Fixed a bug that caused model retraining to fail if there were no DHCP items in a data batch.
New Features
- Cumulative Byte counts are now displayed on the Foreign IP Reports card.
- Download Manager: Long running tasks (such as downloads of large files) no longer require sitting on the same page while they process. These now run in the background on the server and can be picked up on the Long Tasks page when completed.
- Event Card Configurator: When creating a new Event Reports card on the dashboard, the user will now be walked through a setup process for any filters or configurations they want to apply to that card.
- Domain Generation Algorithm (DGA) Detection Model: This model identifies suspicious domains based on their name. Any connections with domains deemed suspicious are created as Evidence, so a suspicious domain alone will not generate an Event. We will monitor this Evidence and see if it makes sense to promote it to a full Event based on model performance.
- Added Nomic Networks threat data as an enrichment on Events. Nomic identifies IPs conducting scanning or reconnaissance activity across the internet.
- Brute Force (T1110) Events will now be tagged as Anomalous instead of Threat if they are Internal to Internal.
- Suricata Event Representative Connections now display fields from relevant child objects (i.e. HTTP, DNS, DHCP, etc.) based on the rule that triggered.
- Added child object fields to all other Events that didn’t previously get them.
- Rules and Filters can now be created using hostname as a field.
- Added support for MITRE ATT&CK version 18.
Bug Fixes
- Fixed a bug that caused Rule and Filter descriptions to not display properly when using User Agent String fields.
New Features
- Event Tagging: Events are now labeled with Tags. These Tags are used to indicate several characteristics of an Event, including whether the traffic is a significant threat or just an anomaly, whether there is a foreign IP involved, and whether the traffic is all internal to the local network.
- The representative Connection displayed when clicking into an Event can now contain fields from the specific child object relevant to the Event (i.e. HTTP, DNS, DHCP, etc.). These fields have been added for a subset of detections for this version: T1021, T1133, T1557, T1568, T1587, Expired Certificate, and Ransomware. This will continue to be added to more detections in coming releases until all relevant detections have these additional fields.
- Prebuilt Dashboards: There are now prebuilt dashboards to give users a better starting point. Users can select from options tailored toward use cases of the tool.
- Event Card configurations are now split into tabs. This includes a new Quick Filters tab to let users quickly filter the view by tags and other commonly used criteria.
- New Detection T1036.008: Masquerading File Type. This will detect cases where a file in an SMB file share was renamed from a non-executable extension to executable (exe, bat, js, etc.) to indicate that someone may be trying to evade detection of a suspicious executable.
- The Foreign IP Reports card can now be filtered in the card configuration.
- Broadened thresholds for Time Clustering such that there will be fewer outliers and thus fewer Odd Traffic Events.
- Sensors will now add to standard port lists based on activity seen on the network.
Bug Fixes
- Fixed a bug that caused hard disk health checks to fail on sensors.
New Features
- Event Viewer Redesign: The Event Viewer has been reworked to convey more useful information up front.
- Reasons displayed in the Event Reports table are now more descriptive of the activity instead of only using the ATT&CK TTP name. The TTP code is still included after the text.
- More details have been added to the Description displayed after clicking on the Event. It now better describes why an Event is notable and how it may relate to malicious behavior.
- Evidence has been moved to the Explanations tab.
- All items on the Evidence tab are now better labeled as Root Cause or Supporting.
- Added detection for T1021.001 and T1021.005 Remote Services. Alerts on RDP or VNC connections to internal resources.
- Updated detection for T1021.003 to consider timing windows and uncommon named pipes.
- T1190 SQL Injection Events now display the suspicious URI in the Description text.
- Added ability to filter Events based on User Agent String.
- User Feedback Models are now automatically updated based on Likes/Dislikes and retrained weekly.
- Suricata rules are now automatically updated weekly from ETPro.
- Updated Suricata Events to be sensitive to time as well as Community ID when linking the detection to Zeek data.
- Migrated to GreyNoise v3 API endpoints.
Bug Fixes
- Fixed a bug that caused rules with long descriptions to throw an error in auditing and appear to not be created.
- Fixed a bug that caused an error when clicking to IP Biographies from the Geolocation tab in an Event on management.
- Fixed a bug that caused the Ransomware analytic to fail when it ran at the same time as the Real Time Ransomware analytic.
- Fixed an error that caused T1557 Adversary in the Middle to sometimes fail when data was missing a MAC address.
- Fixed a bug that caused SSL/TLS certificates with negative values for valid dates to throw errors.
New Features
Bug Fixes
- Foreign IP Reports Card: This card displays all foreign IPs that have been seen based on their IP geolocation at the time they were seen. Users can also download a complete list of the IPs even if it exceeds the 10,000 display limit.
- Updated T1557 Analytic to identify potentially spoofed DHCP servers.
- Users can now upload filename patterns as Indicators.
- Indicator Events now display information about the uploaded Indicator in their Description and Reason fields.
- Added a page to view currently active Indicators.
- Events will now delete Evidence older than 60 days when receiving new evidence. This keeps long running Events from growing excessively large.
- Firewall rules can now be batch exported into a single Ansible playbook from the Event Viewer card.
- Added an Expand All button to tables across the tool where applicable.
- The current user’s roles and permissions are displayed on the dashboard user settings page.
- Users can now set an email address other than their username for alerts to be sent to.
Bug Fixes
- Fixed a bug that caused T1595 Events to associate unrelated conns with their Evidence.
- Fixed a bug that caused the T1568 analytic to fail if multiple conns referenced the same DNS object.
- Fixed a bug that would cause a user to be stuck at an unauthorized screen when attempting to log in without the proper permissions, even after their permissions changed.
New Features
Bug Fixes
- T1595.001 Echo Ping Detection has been rewritten from scratch to reduce false positives. It now specifically looks for at least 30 ICMP Pings within a 15 minutes period (both values are configurable).
- Expired Certificate Events will now roll up by fingerprint and no longer generate Events for internal to internal traffic. They also display the certificate expiration date in the description.
- New Local MAC Events will now display the MAC address on the description of each piece of Evidence.
- Refined T1568.001 Fast Flux DNS to be more selective by considering sequences of DNS answers and geolocation.
- User created Rules now also run as part of Real Time Analytics and will return results every 15 minutes in addition to full Events every 6 hours.
- Added configuration for Filtered MACs for T1557 Adversary in the Middle enabling MAC addresses to be filtered from the analytic dynamic. Added a filter to that configuration for 33:33:00:01:00:03.
- IP Geolocation is now displayed in a tab on the Event view and Geolocation for all Connections is displayed.
- Power Users can now view Login history for the deployment.
- The Help menu now includes a link to email questions to help@m3.intelligenesisllc.com.
Bug Fixes
- Fixed a bug with T1110 Brute Force which would ignore successful logins if they were linked to the same connection that had failures.
- Fixed a bug that could cause Rules with a Geolocation country to throw an error and not process.
- T1190 no longer displays an incorrect reason and description. It was displaying reason and description for T1090.
- Fixed a bug that caused priority to be lost when editing an existing Rule.
- Added a missing comma in the T1046 Firewall Rule Mitigation.
New Features
Bug Fixes
- Feedback Adjusted Event Prioritization: Events are now given a score based on the like/dislike feedback provided by users. There is a new Curated Event Reports card which displays only Events with a preference score higher than a customizable threshold. This preference score can also be added as a column in the existing Event Reports card.
- Events are now enriched with data from the CISA AIS threat feed.
- IP Conversation Metrics: The IP Conversation screen now has a Metrics tab that functions similarly to the Traffic Counts card, but only includes data from the selected IP Conversation.
- Network Metrics Card: Users can add this card to their dashboard to view metrics such as connections, duration, and bytes in/out on a timeline.
- Events are now generated for expired SSL/TLS certificates utilized in connections.
- Users can now download an Ansible playbook to undo suggested Firewall Rules.
- Rules and Filters now have a name field. A Rule’s name will be displayed on the Event Reports card as the title of an Event it generates.
- The FF:FF:FF:FF:FF:FF broadcast MAC is excluded from generating T1557 (Adversary in the Middle) Events.
Bug Fixes
- Fixed an issue which caused Split PCAP download to not appear as an option when clicking a context menu for a conn.
- Fixed an issue which caused IP Conversations to show all Events and Evidence linked to one of the IPs instead of only items linked to the pair.
New Features
- IP Conversations: Users can now enter a pair of IPs and view the traffic between them as well as any Events or Evidence associated with the pair. This feature has been combined with IP Biographies on the new IP Lookup page that provides access to both features.
- PCAP Downloads from Events and IP Conversations have the option to only contain data relevant to the Event or IP pair.
- Events are now filtered based on data from GreyNoise.
- TTP Events will not be generated if there is a Trust Level 1 enrichment or Trust Level 2 with a classification of Benign.
- Other TTP Events classified as Benign or Unknown will have their priority level reduced.
- Odd Traffic Events will not be generated if there is a Trust Level 1 or Trust Level 2 enrichment.
- Users can now Like or Dislike events using the Thumbs Up/Down buttons on the Event. This currently has no effect, but the data gathered will be used in future versions to refine the Events displayed based on the user’s preferences.
- Added a Submit and Close button to Event Notes to save the Note and Close the Event with one click.
- Real Time Events now have a detailed viewer similar to traditional Events.
- Firewall Rule Mitigations can now be downloaded as Ansible formatted YAML files.
- Renamed FTP_DATA to FILE to more accurately describe that the data displayed there encompasses all files transmitted over the connection, not just files FTPed.
Bug Fixes
- Fixed a bug that caused GreyNoise enrichment to only pull Noise data when RIOT and Noise were both present.
- Fixed a bug on the Rules creation page which caused filter fields to not recognize that a value was entered if the field type was switched.
New Features
- Grey Noise threat feed data is now added as an enrichment to Events.
- Data Exfiltration analytics (T1029, T1030, T1041) now only consider successful connections. T1029 and T1041 also now utilize the Producer-Consumer ratio of bytes sent and received to aid in exfiltration detection.
- T1110 (Brute Force) detection now includes the auth_attempts field in its analysis. It also now resets the count of failed attempts when a successful attempt occurs.
- T1205 (Port Knocking) now searches all traffic (instead of just anomalies/evidence) for a configurable number of failed connections (default 3) followed by a successful connection all within 1 minute and all on ports greater than 1024.
- T1046 (Network Service Discovery) now excludes connections initiating from port 7680 on internal IPs with a connection history of “R”.
- Rules and Filters can now use the not equals (!=) operator.
- All New Local MAC Events on a sensor are now rolled up into a single Event.
- Ransomware Events now roll up by the pattern that was matched, reducing the number of unique Events from files of the same type.
- Improved the Network Graph visualization with highlighting of selected and hovered nodes, more information about each node, and a pivot feature to move from node to node with a back button.
- The MaxMind IP Geolocation database is now updated automatically twice per week.
- Maps, Timelines, Pie charts, and Bar charts all now use HighCharts and have improved accessibility features.
Bug Fixes
- Fixed a bug in T1110 (Brute Force) detection that was counting authentication attempts as failed when no value was specified in the data.
- Fixed a bug which caused some pre-trained data to age off incorrectly.
- Fixed a bug which caused retrained user agent and byte estimator models to overwrite previous versions of the model instead of preserving them.
- Fixed a bug which caused long running API requests generated by the UI to keep running even if the user closed that view. This could stall future API requests if the user had too many canceled ones still running.
New Features
- Traffic Query and all Cards can now choose whether all of their filters are ANDed or ORed. This setting applies to all filters on the query or card collectively, but can not be mixed or applied to individual field pairs.
- Columns in all tables can now be rearranged and the ordering and sort selections will be saved in the user’s cookies.
- Events now have a Click to Filter button and Conns have a Click to Rule button. Each uses the information from the respective item to populate a Filter/Rule that the user can edit and confirm.
- T1090 Events are no longer produced by IPs or Hosts that are whitelisted in AlienVault.
- Events generated by IPs and Hosts that are whitelisted in AlienVault are now reduced to Low Priority. An enrichment is added to the Event to indicate this.
- Multicast IPs no longer generate New Local MAC Events.
- Brute Force over HTTP (T1110) now only generates Events for traffic on ports 80, 8080, 443, 8443, 14443, and 8008.
- Greater Than (>) and Less Than (<) operators can now be used with Priority and Confidence fields in Event filters.
- Cards and views which limit the amount of data shown (e.g. 10,000 conns in Traffic Viewer) can now be set to order by time ascending or descending ensuring the oldest or most recent data is returned depending on the user’s choice.
- A glossary of fields is now available in the Help Menu.
Bug Fixes
- Fixed a bug that caused filters on the Reason field to not function properly.
- Fixed a bug that caused the Suricata Events count on the System Info card to always be 0 event when Suricata Events existed.
- Using the external IP field in filters no longer throws an error on the Event Counts card.
- Fixed a bug that could cause the T1110 analytic to crash when trying to update an Event from a previous run.
- Descriptions longer than 1500 characters are now trimmed when displayed.
- Fixed a bug that caused new lines to not display in long Descriptions.
- Added accessibility text to buttons for Screen Reader support.
New Features
- Event mitigations now include Snort formatted suggested firewall rules that users can choose to implement to block potentially malicious traffic.
- Added collection filters for APIPA (169.254.*.*) addresses
- Added backwards compatibility support for times when the sensor version is older than management
- Duplicate Real Time Events are now rolled up into one instance, and have a date created and date updated field to track occurrences
Bug Fixes
- Fixed a bug in Traffic Query that caused the Submit button to stop working if a user first use the Click to pivot and the edited the date or display fields
- Fixed a bug that could cause the Timeline configuration to crash when attempting to display Evidence data
- Fixed a bug that caused sensors with very low data flow to crash and get stuck if they hadn't passed the burn in threshold after several months
- Added the Reset to Default button to cards that are popped out
New Features
- T1029, T1030, T1041 Analytics have been reworked to reduce false positives
- Created several new parameters that can be tuned in configuration to adjust behavior on the fly
- Increased thresholds for alerting on several existing parameters
- Removed UDP connections from consideration
- APIPA addresses are no longer considered in detecting T1557
- Analytics now run in parallel instead of serial, improving runtime of the pipeline
- New Local MAC now also runs as a real time analytic
- Added geolocation fields to connections in views throughout the tool
- Dashboard cards can now be popped out into their own window
- Updated Suricata rules from ProofPoint
- Added full Suricata rule enrichment to events which contain Suricata evidence
Bug Fixes
- Added more back end pings to keep login sessions from dying when a user is on a page other than the dashboard
- Deleting an element on the Timeline Card configuration no longer resets all other elements
- Fixed a mismatch between FTP, SMB, and SSL names that caused query filters to not match those data types
- Fixed a bug that could cause conns to lose the association with their source file if a crash happened at a specific time during ingest
- Fixed a bug that caused RDP data to not display correctly on drilldowns
- Removed Pivot to Traffic Query button for users that don't have permission to use Traffic Query
- Fixed a bug that prevented PDF reports from displaying correctly
New Features
- Local Model Retraining: Byte Estimator and User Agent String will now retrain weekly on data specific to the local sensor to provide more accurate anomaly detection tailored to the DIB.
- Traffic Search on Sensors: Users can now perform traffic searches on data on their sensor.
- Timeline improvements: The timeline card can now be configured to display any combination of data types over any time range.
- Drill down views now have tabs for more related objects. Conns have tabs for Evidence and Events related to that conn and Evidence has tabs for Events linked to them.
- The T1029 Analytic no longer generates Events for UDP Traffic
- Events with Suricata enrichment now display the rule text of the rule that hit.
- Events by External IP: The Event Table dashboard card can now be filtered by External IP.
- Dashboard cards can now be resized by the user to allow further customization.
- Release notes are now viewable in the tool. You're reading them right now! On the first login after each release the user will receive a popup displaying the notes. They can review them any time after that by clicking the version number in the Help menu.
- Users can now select multiple Events at once to close, add alerts, mark as read, or pin.
- Added a button to reset the entire dashboard to default settings.
- The default Event table view now includes the Confidence column.
Bug Fixes
- Fixed cases where T1587 and T1021 analytics were using a hard coded list to determine whether an IP was local or not.
- Events were incorrectly being generated with a single piece of evidence if it came from a Device Clustering anomaly. Fixed to require at least two pieces of evidence.
- T1029 Events are limited to 5000 conns in their linked evidence, this limit could cause the representative conn shown to the user to not be present in the evidence associated with the event. This has been fixed so that the representative conn is always added to the evidence.
- T1090.004 Events could be generated due to considering a hostname with a port appended different than the plain hostname. This comparison now ignores ports appended to the hostname.
- Fixed duplication and overcounting that could occur when setting queries for the Event by Identifying Conn dashboard card if an Event had multiple representative conns.
- Fixed colors for Confidence values in Dark Mode to make them more legible.