🔄 Topic

Hermes became a real secretary: it can read my calendars and handle personal email — so before it got those powers, it got a sender policy, an approval flow, and a remote emergency stop.

🎯 Goal

Give the agent genuinely personal capabilities (calendars, email) while keeping every risky action behind policy, and keep one guaranteed way to stop everything from anywhere.

🛠 What I Did

I wired personal-data integrations and their controls in the same session.

Main areas covered:

  • gave Hermes access to macOS Calendar via icalbuddy, tested with real queries like finding a specific person’s next birthday
  • consolidated the scattered calendars onto Apple Calendar so the agent queries one source of truth instead of guessing
  • handled locale details properly — Italian date formats, accented names — because a secretary that mangles “Niccolò” fails at its one job
  • rolled out a personal email sender policy: the agent may only send to defined recipients, and anything else requires explicit approval
  • added the approval flow for out-of-policy sends rather than silently blocking or silently allowing
  • built a remote emergency control — a way to stop the secretary’s autonomous behavior from my phone, immediately, without SSH
  • cleaned up superseded local models as part of the same hygiene pass

🔗 Key Cybersecurity Connections

Calendars and email are the most personal data an agent can touch: who I know, where I am, what I say. Granting that access is a privacy decision, and the controls mirror real-world ones — the sender policy is an egress allowlist, the approval flow is exception handling with a human decision, and the emergency control is the incident-response requirement that every autonomous system needs a stop that works when things are weird.

🔍 Investigation Questions

  • Exactly which calendars and mailboxes can the agent read?
  • Can it send to anyone outside the policy list without an approval?
  • Does the emergency stop work when the normal channels are down?
  • Are the agent’s calendar queries logged somewhere reviewable?
  • What is the blast radius if the secretary misbehaves for an hour before I notice?

🚨 Detection Opportunities

Checks for a personal-assistant agent:

  • send attempt to a recipient outside the sender policy
  • approval flow bypassed or auto-approved
  • emergency stop invoked — always worth reviewing what prompted it
  • calendar queries spiking outside normal usage
  • policy file modified without a corresponding decision record

Example:

project=hermes-secretary
signal=send_attempt_outside_sender_policy
risk_area=unauthorized_personal_communication
triage=hold_message_review_policy_and_approval_log

🧭 MITRE ATT&CK Techniques

Possible mappings for the risks being controlled:

  • T1114 — Email Collection
  • T1534 — Internal Spearphishing

The sender policy and approval flow exist so the secretary cannot drift into these.

🗺 Visual Investigation Diagram

Personal capability requested
    ↓
Policy written first (who, what, when)
    ↓
In policy → act
Out of policy → approval flow
    ↓
Everything → logged
    ↓
Emergency control → stop all, from anywhere

⚠ Challenges

The subtle work was data quality: an assistant is only trustworthy if its source is. Consolidating calendars mattered as much as any control, because an agent confidently answering from an incomplete calendar is wrong with authority.

📚 What I Learned

I learned to build the stop button before the feature. Writing the emergency control while the capability was still new took an hour; retrofitting it after an incident would have been a very different story.

➡ Next Steps

  • Test the emergency control routinely, not just once
  • Review the sender policy list on a schedule
  • Extend logging to calendar query patterns
  • Keep new personal capabilities behind the same policy-first pattern

🧠 Reflection

Handing an agent my calendar felt more sensitive than handing it a filesystem. Data about people demands a different level of care than data about code, and the controls should show that.

🧩 Lessons Learned

What worked

Policy and kill switch shipped in the same session as the capability.

What broke

Early calendar answers were wrong because the calendars themselves were fragmented.

Why it broke

Years of events scattered across accounts with no single source of truth.

Fix / takeaway

Fix the data before trusting the agent that reads it — and never ship a personal capability without its stop button.

📈 Skill Progression Context

This supports my cybersecurity progression because egress policies, exception approval flows, and emergency-stop design are the same controls that govern privileged automation in any organization.

😄 TL;DR

The secretary got my calendar and my inbox — after it got a policy, an approval flow, and a kill switch.