# A practical guide to security logging and detection engineering [2026 edition]: building a state where you 'can notice' with Sigma, MITRE ATT&CK, and SIEM

> A practical guide to log design and detection engineering that solves 'you can't protect what you can't detect.' With real code: 'what to record and what not to,' per OWASP; type-safe structured logs and secret masking; vendor-neutral detection rules with Sigma; mapping to MITRE ATT&CK; and Detection as Code.

- Published: 2026-06-28
- Author: 友田 陽大
- Tags: セキュリティ, ログ設計, 検知エンジニアリング, Sigma, セキュリティエンジニア
- URL: https://tomodahinata.com/en/blog/security-logging-detection-engineering-sigma-mitre-attack-siem-guide
- Category: Security engineering & career
- Pillar guide: https://tomodahinata.com/en/blog/security-engineer-how-to-become-roadmap-skills-certification-guide

## Key points

- 'You can't protect what you can't detect.' Attacks can't always be prevented, but 'not being able to notice' is fatal. In OWASP Top 10 too, 'Security Logging and Alerting Failures (A09)' is a regular weakness.
- The core of log design is 'what to record and what not to.' Record auth success/failure, authorization denials, and important operations. Meanwhile, never write passwords, tokens, card numbers, or personal information to logs (logs become a new leak source).
- Structured logs (JSON) + a correlation ID + automatic secret masking are the foundation. Emit in a form machines can search and correlate, not human-readable text. With a type-safe logger, make it 'structurally impossible to write dangerous values.'
- Write detection rules in Sigma. Sigma is a vendor-neutral YAML format convertible to Splunk, Sentinel, Elastic, etc. Tie each rule to a MITRE ATT&CK technique ID (e.g., T1110 Brute Force) to visualize gaps.
- Operate detection 'as code' (Detection as Code). Version-manage rules in Git, test them, and deploy via CI. To avoid alert fatigue, continuous tuning of false positives determines quality.

---

The security world has a cruel fact: **"attacks can't always be prevented."** Zero-day vulnerabilities, insider misconduct, sophisticated phishing — no matter how you harden defenses, you can't zero the possibility of compromise. That's exactly why, as important as defense, is — **"being able to notice."**

The most common tragedy in investigating a compromise is "we can't tell when, from where, or what happened, because there are no logs." In fact, in [OWASP Top 10](https://owasp.org/Top10/2025/) too, **"Security Logging and Alerting Failures (A09)"** repeatedly ranks in. This article explains log design and detection engineering for **building a "state where you can detect,"** in real code faithful to official information ([OWASP Logging Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html), [Sigma](https://github.com/SigmaHQ/sigma), [MITRE ATT&CK](https://attack.mitre.org/)).

> **The positioning of this article:** it's the core skill bearing NIST CSF 2.0's **"Detect."** For the role overall, see [how to become a security engineer](/blog/security-engineer-how-to-become-roadmap-skills-certification-guide); for the defense before detection, [secure-coding practices](/blog/secure-coding-practices-nist-ssdf-owasp-asvs-engineer-guide); and for the response after detection, [incident response](/blog/incident-response-nist-800-61r3-csirt-runbook-playbook-production-guide).

---

## 0. What to record and what not to

Log design fails with "just emit everything." That's because **important anomalies get buried in noise, and you scatter confidential information.** Following the [OWASP Logging Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html), separate what should be recorded from what must never be.

### Security events to record

| Category | Concrete examples |
|---|---|
| **Authentication** | Login success/failure, logout, MFA success/failure, password change |
| **Authorization** | Access denial (insufficient permission), role change, privileged operation |
| **Important operations** | Data create/update/delete, config change, export |
| **Input-validation failures** | Rejection of invalid input (a sign of attack) |
| **Anomalies** | Exceptions, rate-limit triggering, unexpected state transitions |

### What must never be recorded

**The log itself becomes a new leak source** — don't forget this.

- **Passwords, tokens, API keys, session IDs** (plaintext, of course, but also pre-hash)
- **Credit card numbers, CVV** (a PCI DSS violation)
- **Personal information (PII)** — minimize names, addresses, phone numbers, etc., and mask if needed
- **Highly confidential business data**

"A secret emitted temporarily for debugging" remains in the production log and leaks — a typical accident. The correct answer is to have a structure on the logger side where you **"can't carelessly write a secret."**

---

## 1. A type-safe structured logger — make it structurally impossible to write secrets

A text log for humans to read (`"user alice logged in from 1.2.3.4"`) isn't suited for machines to search and correlate. The modern standard is **structured logs (JSON).** You can search by field, follow a sequence with a correlation ID (request ID), and ingest it directly into a SIEM.

Furthermore, a logger that **automatically masks confidential information** structurally prevents "carelessly leaking" accidents.

```ts
// security-logger.ts — 構造化セキュリティログを出し、秘密情報を自動マスキングする。
import { z } from "zod";

// ① 記録してよいイベント種別を列挙（型で縛る＝勝手な自由文字列を防ぐ）。
const SecurityEvent = z.enum([
  "auth.login.success",
  "auth.login.failure",
  "authz.access.denied",
  "data.export",
  "ratelimit.triggered",
]);
type SecurityEvent = z.infer<typeof SecurityEvent>;

// ② ログに含めてはいけないキー。値を見ずにキー名で機械的に伏せる。
const REDACT_KEYS = new Set([
  "password", "token", "secret", "authorization",
  "apikey", "api_key", "sessionid", "session_id", "cookie", "creditcard",
]);

function redact(meta: Record<string, unknown>): Record<string, unknown> {
  const safe: Record<string, unknown> = {};
  for (const [key, value] of Object.entries(meta)) {
    safe[key] = REDACT_KEYS.has(key.toLowerCase()) ? "[REDACTED]" : value;
  }
  return safe;
}

interface LogContext {
  readonly requestId: string;  // 相関ID：一連のリクエストを横断で追える
  readonly userId?: string;    // 主体（無ければ匿名）
  readonly ip?: string;
}

/** セキュリティイベントを構造化JSONで出力する。秘密は自動で伏せられる。 */
export function logSecurityEvent(
  event: SecurityEvent,
  ctx: LogContext,
  meta: Record<string, unknown> = {},
): void {
  const entry = {
    ts: new Date().toISOString(),
    level: "security",
    event,
    ...ctx,
    ...redact(meta), // ③ 秘密はここで構造的に伏せられる
  };
  // 1行1JSON（JSON Lines）。SIEM/ログ基盤がそのまま取り込める。
  console.log(JSON.stringify(entry));
}
```

```ts
// 使用例：ログイン失敗を記録する。password を渡しても自動で伏せられる。
logSecurityEvent("auth.login.failure", { requestId, ip }, {
  email: "alice@example.com",
  password: "hunter2",       // → "[REDACTED]" として出力される（漏れない）
  reason: "invalid_credentials",
});
// 出力: {"ts":"2026-06-28T...","level":"security","event":"auth.login.failure",
//        "requestId":"...","ip":"...","email":"alice@example.com",
//        "password":"[REDACTED]","reason":"invalid_credentials"}
```

There are three key points to the design. **(1) Enumerate event types with types** to prevent unrestrained strings, **(2) a correlation ID** to enable cross-tracking, and **(3) key-name-based masking** to structurally hide secrets. With this, you get a log foundation that's "searchable, correlatable, and doesn't leak."

---

## 2. Detection engineering — define "anomalies" with Sigma

Just accumulating logs, no one looks. You need detection rules that say **"if this condition occurs, raise an alert."** What's becoming the industry standard here is **[Sigma](https://github.com/SigmaHQ/sigma).**

Sigma is a **vendor-neutral, YAML-format detection rule.** Write it once and you can **convert** it to the queries of each SIEM — Splunk, Microsoft Sentinel, Elastic, QRadar, etc. The biggest benefit is being freed from the hell of "relearning a query language per SIEM."

Below is a Sigma rule that detects **consecutive login failures in a short time (a sign of a brute-force attack).** By **tying it to a MITRE ATT&CK technique ID (T1110: Brute Force)**, "which attack technique of ATT&CK we can cover" becomes visible.

```yaml
# rules/brute_force_login.yml — 総当たりログインの検知（Sigma形式）
title: 短時間における連続ログイン失敗（総当たりの疑い）
id: 8f2e1a3c-0b4d-4e5f-9a1b-2c3d4e5f6a7b
status: stable
description: 同一IPからの連続したログイン失敗を検知し、総当たり攻撃を早期に捉える
references:
  - https://attack.mitre.org/techniques/T1110/
author: friend yota
date: 2026/06/28
logsource:
  product: application
  service: auth
detection:
  selection:
    event: "auth.login.failure"   # 構造化ログの event フィールドに一致
  timeframe: 5m                    # 5分の時間枠で…
  condition: selection | count() by ip > 10   # 同一IPから10回超の失敗
fields:
  - ip
  - email
falsepositives:
  - 正規ユーザーのパスワード失念（ロックアウト方針と併せて評価する）
level: high
tags:
  - attack.credential_access
  - attack.t1110          # MITRE ATT&CK: Brute Force
```

The rule structure is simple — **`logsource` (which log) + `detection` (what to match) + `condition` (the alert condition) + metadata (`tags` for the ATT&CK technique ID).** The culture of stating `falsepositives` (the possibility of a false positive) works for the alert-fatigue countermeasure described later.

---

## 3. Measure "coverage" with MITRE ATT&CK

Just writing individual rules, you can't tell "what we can detect and what we're missing." What you use there is **[MITRE ATT&CK](https://attack.mitre.org/).** ATT&CK is a globally common "map of attacks" that systematizes real attackers' tactics and techniques.

Mapping each Sigma rule to an ATT&CK technique with `attack.tXXXX` lets you color in "the techniques you can cover" on the **[ATT&CK Navigator](https://mitre-attack.github.io/attack-navigator/).** With this, the detection discussion changes from "somehow anxious" to —

- **"Detection of Initial Access is thin"**
- **"Privilege Escalation is covered"**

— a **concrete coverage discussion.** Surface "the threats to protect" with threat modeling ([STRIDE](/blog/threat-modeling-stride-data-flow-diagram-secure-design-practical-guide)) and measure "detection coverage" with ATT&CK — the offensive map and the defensive map mesh.

---

## 4. Detection as Code — operate detection "as code"

Operate detection rules with the same discipline as app code (**Detection as Code**).

- **Version-manage in Git.** You can trace who added the rule, when, and why.
- **Test.** Verify the rule fires on known attack logs (samples) and doesn't fire on normal logs.
- **Deploy via CI.** Once merged, automatically reflect the rule into the SIEM.
- **Keep tuning.** Continuously cut false positives.

That last "tuning" determines the quality of detection. **Alert fatigue** — a state where too many false positives lead even real alerts to be ignored — is the biggest enemy of a detection regime. So state `falsepositives` on each rule, observe the firing rate, and operate to **keep only "rules that don't cry wolf."** Make your metric "the quality of trustworthy alerts," not "the number of rules."

After a detection fires, who moves and how — that's the realm of [incident response](/blog/incident-response-nist-800-61r3-csirt-runbook-playbook-production-guide). Detection and response have meaning only when designed as a set.

---

## 5. Log integrity — if it can be tampered with, it's not evidence

Finally, a point that tends to be overlooked. **If the log itself can be tampered with, it's not non-repudiation (the STRIDE Repudiation countermeasure).** An attacker deleting logs to destroy evidence after intrusion is a typical tactic.

- Make it **append-only.** Don't let the app's execution permission rewrite or delete logs.
- **Aggregate.** Decouple logs from the originating server and immediately forward them to a separate aggregation foundation (SIEM / log-dedicated storage). Don't rely only on logs on the compromised server.
- **Define a retention period.** Per legal, contractual, and investigative needs, retain for a certain period in a tamper-proof form.

Between "taking logs" and "logs being evidence" lies this difference of **integrity.**

---

## 6. Frequently asked questions (FAQ)

**Q. Is all this needed even for a small app?**
A. Starting "light" per scale is enough. First, structured logs + secret masking (§1). Even without a SIEM, just emitting auth success/failure, authorization denials, and important operations in JSON makes your investigative power vastly different when it matters.

**Q. Can I do it with an existing logger (pino/winston, etc.)?**
A. You can. The logger here is a minimal implementation to show the principle. pino, etc., have a `redact` feature as standard, and using it lets you quickly achieve production-quality structured logs + masking.

**Q. Isn't a SIEM expensive? Can I start free?**
A. A managed cloud SIEM is usage-based and can start small. For OSS, there's the Elastic Stack, OpenSearch, Wazuh, etc. First "structure logs and aggregate them somewhere"; SIEM selection can come later.

**Q. Where do I get Sigma rules?**
A. The [SigmaHQ official repository](https://github.com/SigmaHQ/sigma) has thousands of public rules. The quick path is to first take in ones that fit your environment and convert them to each SIEM's query with `sigma-cli`.

**Q. How much PII may I include in logs?**
A. The principle is minimization. If essential for investigation, keep it to an identifier (user ID), and mask or reference names, addresses, etc. From the standpoint of personal-information protection laws and GDPR, "not holding" PII in logs is the safest.

---

## 7. Conclusion

As important as "protecting," being able to "notice" is important. Detection is **the last, and decisive, line of defense** against attacks that slip past defenses.

- **Decide what to record and what not to.** Record auth, authorization, and important operations; never write secrets and PII.
- **Structured logs + a correlation ID + automatic masking.** Build a foundation that's searchable, correlatable, and doesn't leak, with a type-safe logger.
- **Write detection in Sigma.** Vendor-neutral YAML, convertible to each SIEM. Tie it to a technique ID with MITRE ATT&CK.
- **Measure coverage with ATT&CK.** Change "somehow anxious" into "which tactic is thin."
- **Operate with Detection as Code.** Git, test, CI, continuous tuning. Avoid alert fatigue and make quality the metric.
- **Protect log integrity.** Append-only, aggregation to a separate foundation, appropriate retention.

"You can't protect what you can't detect" — place this one sentence at the start of your design. If you want to review your product's log design or detection regime once, or want accompaniment on SIEM introduction or building out detection rules, feel free to consult me.

---

### References (official primary sources)

- Log design: [OWASP Logging Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html) / [OWASP Top 10:2025 A09](https://owasp.org/Top10/2025/)
- Detection: [Sigma (SigmaHQ)](https://github.com/SigmaHQ/sigma) / [MITRE ATT&CK](https://attack.mitre.org/) / [ATT&CK Navigator](https://mitre-attack.github.io/attack-navigator/)
- Related: [incident response](/blog/incident-response-nist-800-61r3-csirt-runbook-playbook-production-guide) / [threat modeling](/blog/threat-modeling-stride-data-flow-diagram-secure-design-practical-guide)