Events and Alerts

Imagine yourself as a SOC intern, looking at the monitor of your mentor, a SOC L1 or L2 analyst, and seeing hundreds of alerts appear, moving between statuses and eventually disappearing from some dashboard or console. You notice lots of alerts named “Email Marked as Phishing”, a few “Unusual Gmail Login Location”, and one scary “Unapproved Mimikatz Usage” inside a red column. What are these alerts, how are they formed, and what should we do with them? Let’s find out!

Alert dashboard where different alerts are moved in real time from New to In Progress, True Positive, and other columns

From Events To Alerts

First, an event must occur, like user login, process launch, or file download. Then, the system, like your OS, a firewall, or a cloud provider must log the event. After that, all system logs must be shipped to a security solution like SIEM or EDR. The SOC team can receive millions of logs per day from thousands of different systems, where most of the events are expected, but some require attention. Alert, a notification generated by a security solution when a specific event or sequence of events occurs, is what saves SOC analysts from manual log review by highlighting only suspicious, anomalous events. With alerts, analysts triage just dozens of alerts per day instead of millions of raw logs.

Alert Management Platforms

SolutionExamplesDescription
SIEM
System
Splunk ES, 
Elastic
SIEM have solid alert management capabilities and are a perfect choice for most SOC teams
EDR or
NDR
MS Defender,
CrowdStrike
While EDR and NDR provide their own alert dashboards, it is preferred to use SIEM or SOAR
SOAR
System
Splunk SOAR,
Cortex SOAR
Bigger SOC teams can use SOAR to aggregate and centralise alerts from multiple solutions
ITSM
System
Jira,
TheHive
Some teams may have a custom ticket management (ITSM) setup using a dedicated solution
(The GIF above is taken from Trello, a simple tool that can be adapted to ITSM needs)

L1 Role in Alert Triage

SOC L1 analysts are the first line of defence, and they are the ones who work with alerts the most. Depending on various factors, L1 analysts may receive zero to a hundred alerts a day, every one of which can reveal a cyberattack. Still, everyone in the SOC team is somehow involved in the alert triage:

  • SOC L1 analysts:  Review the alerts, distinguish bad from good, and notify L2 analysts in case of a real threat
  • SOC L2 analysts:  Receive the alerts escalated by L1 analysts and perform deeper analysis and remediation
  • SOC engineers:  Ensure the alerts contain enough information required for efficient alert triage
  • SOC manager:  Track speed and quality of alert triage to ensure that real attacks won’t be missed

Alert Properties

Now that you know how alerts are generated, it’s time to review their content. While the details differ for every SIEM or security solution, the alerts generally have a few main properties you must understand before taking them. Don’t worry if you find some confusing, as you will hear more about some in the upcoming tasks.

Alert example from a SOC dashboard highlighting differnt main property fields

Alert Properties

PropertyDescriptionExamples
1Alert TimeShows alert creation time. Alert usually triggers
a few minutes after the actual event
- Alert Time: March 21, 15:35
- Event Time: March 21, 15:32
2Alert NameProvides a summary of what happened,
based on the detection rule’s name
- Unusual Login Location
- Email Marked as Phishing
- Windows RDP Bruteforce
- Potential Data Exfiltraton
3Alert SeverityDefines the urgency of the alert,
initially set by detection engineers,
but can be altered by analysts if needed
- (🟢) Low / Informational
- (🟡) Medium / Moderate
- (🟠) High / Severe
- (🔴) Critical / Urgent
4Alert StatusInforms if somebody is working on the alert
or if the triage is done
- (🆕) New / Unassigned
- (🔄) In Progress / Pending
- (✅) Closed / Resolved
- And often other custom statuses
5Alert VerdictAlso called alert classification,
explains if the alert is a real threat or noise
- (🔴) True Positive / Real Threat
- (🟢) False Positive / No Threat
- And often other custom verdicts
6Alert AssigneeShows the analyst that was assigned
or assigned themselves to review the alert
- Assignee can sometimes be called alert owner

- Assignee takes responsibility for their alerts
7Alert DescriptionExplains what the alert is about,
usually in three sections on the right
- The logic of the alert generating rule
- Why this activity can indicate an attack
- Optionally, how to triage this alert
8Alert FieldsProvides SOC analysts’ comments
and values on which the alert was triggered
- Affected Hostname
- Entered Commandline
- And many more, depending on the alert

Alert Prioritization

Okay, you can now read and understand the alert. What’s next? Recall the second task, where you see hundreds of alerts but have to choose which to pick up first. The process of deciding which to take is called Alert Prioritisation, and it is crucial to ensure timely detection of a threat, especially with many alerts in the queue.

Three doors with different alert labels on them demonstrating alert prioritisation decision

Picking the Right Alert

Every SOC team decides on its own prioritisation rules and usually automates them by setting the appropriate alert sorting logic in SIEM or EDR. Below, you may see the generic, simplest, and most commonly used approach:

  1. Filter the alerts
    Make sure you don’t take the alert that other analysts have already reviewed, or that is already being investigated by one of your teammates. You should only take new, yet unseen and unresolved alerts.
  2. Sort by severity
    Start with critical alerts, then high, medium, and finally low. This is because detection engineers design rules so that critical alerts are much more likely to be real, major threats and cause much more impact than medium or low ones.
  3. Sort by time
    Start with the oldest alerts and end with the newest ones. The idea is that if both alerts are about two breaches, the hacker from the older breach is likely already dumping your data, while the “newcomer” has just started the discovery.

Alert Triage

Finally, you are ready to review the chosen alert! The process is quite operationally heavy, but you will soon see why every step is important. Also, note that the alert review by SOC analysts can also be called alert triage, alert handling, alert processing, alert investigation, or alert analysis. During this module, we will stick to the Alert Triage option.

Diagram showcasing required steps to triage alerts

Initial Actions

The initial steps are designed to ensure that you take ownership of the assigned alert and avoid interfering with alerts being handled by other analysts, and confirm that you are fully prepared to proceed with the detailed investigation. You achieve it by first assigning the alert to yourself, moving it to In Progress, and then familiarising yourself with the alert details like its name, description, and key indicators.

Investigation

This is the most complex step, requiring you to apply your technical knowledge and experience to understand the activity and properly analyse its legitimacy in SIEM or EDR logs. To support L1 analysts with this step, some teams develop Workbooks (also known as playbooks or runbooks) - instructions on how to investigate the specific category of alerts. If workbooks are not available, below are some key recommendations:

  1. Understand who is under threat, like the affected user, hostname, cloud, network, or website
  2. Note the action described in the alert, like whether it was a suspicious login, malware, or phishing
  3. Review surrounding events, looking for suspicious actions shortly after or before the alert
  4. Use threat intelligence platforms or other available resources to verify your thoughts

Final Actions

Your decisions here determine whether you found or missed the potential cyberattack. Some actions like Escalation or Commenting will be explained in the following rooms, so don’t worry if they sound complex right now. First, decide if the alert you investigated is malicious (True Positive) or not (False Positive). Then, prepare your detailed comment explaining your analysis steps and verdict reasoning, return to the dashboard and move it to the Closed status.

SOC Dashboard Notes

If you didn’t receive a flag after your triage, it means that the values you set are wrong.
You can reset the SOC dashboard by clicking Restart on the top right in the TryHackMe SIEM.