🎯 Goal

Today’s goal was to better understand the growing ecosystem of local AI models and how they compare with cloud-hosted AI services.

Instead of simply using AI tools as black boxes, I wanted to understand:

• how local models actually run
• the difference between local inference vs API inference
• how developers integrate AI into workflows
• the implications for security, privacy, and operational control

This knowledge is becoming increasingly relevant for cybersecurity professionals, especially when dealing with automation, investigation tooling, and AI-assisted workflows.

🛠 What I Did

Investigated Local LLM Infrastructure

I explored the idea of running AI models locally instead of relying on cloud APIs.

Key components involved in local AI systems include:

• model runtimes (like Ollama or LM Studio)
• quantized models (GGUF format)
• local hardware acceleration (CPU / GPU / Apple Silicon)

This architecture allows AI models to run directly on a user’s machine, removing dependence on external APIs.

Compared Local AI vs Cloud AI

I examined the trade-offs between both approaches.

Cloud AI advantages:

• state-of-the-art models
• massive compute resources
• frequent updates

Local AI advantages:

• privacy and data control
• predictable operating cost
• independence from external providers

For cybersecurity professionals, data locality and control can be a major factor.

Explored Token-Based Billing Models

One interesting discovery was how token-based billing works in cloud AI systems.

Each API call typically includes:

• the user prompt
• the entire previous conversation context
• the model response

This means that long conversations increase cost exponentially.

Understanding this mechanism is important when building AI agents or automated systems that may run continuously.

🔗 Key Cybersecurity Connections

Understanding AI infrastructure has several security implications.

For example:

• local models reduce the risk of data exfiltration through third-party APIs
• API-based AI introduces supply chain risk if a provider changes policies or access
• AI agents interacting with systems must be sandboxed and tested before production use

The same engineering mindset used in cybersecurity — test, isolate, verify — applies directly to AI systems.

⚠ Challenges

One challenge was separating real technical information from hype.

The AI ecosystem currently contains:

• many exaggerated claims
• unrealistic “get rich with AI apps” narratives
• unclear explanations of how the systems actually work

Filtering signal from noise requires careful analysis and verification.

📚 What I Learned

Key lessons from today:

• local AI models are becoming increasingly viable
• cloud models still lead in capability but the gap is shrinking
• token-based pricing can create unexpected costs
• understanding infrastructure is more important than simply using tools

➡ Next Steps

Next steps include:

• exploring local model runtimes in more depth
• experimenting with AI tools inside controlled environments
• understanding how AI agents interact with operating systems

🧠 Reflection

One recurring pattern in cybersecurity is that tools alone never solve problems.

Just as the best forensic software will not help an untrained analyst, the most advanced AI model does not replace understanding.

Real competence comes from:

• understanding systems
• testing behavior
• learning the underlying mechanisms

🧩 Lessons Learned

What worked
Breaking down the AI ecosystem into infrastructure components made the topic far clearer.

What broke
Many explanations online simplify the technology too much.

Why it broke
AI is currently surrounded by hype cycles.

Fix / takeaway
Always examine how systems actually operate under the hood.

🔎 Investigation Questions

• How are AI agents safely integrated into operational environments?
• What are the security risks of AI systems with system access?
• How does token context expansion affect large automated workflows?

🛡 Detection Opportunities

Future SOC environments may need detections for:

• abnormal AI API usage
• automated credential harvesting via AI agents
• large volumes of automated outbound queries

🎯 MITRE ATT&CK Techniques

Potential future relevance:

T1071 — Application Layer Protocol
T1046 — Network Service Discovery
T1195 — Supply Chain Compromise

🧭 Investigation Flow

User Input
↓
AI Model Processing
↓
External API / Local Runtime
↓
System Interaction
↓
Security Monitoring

📈 Skill Progression Context

This exploration expands my understanding beyond traditional SOC skills and into AI-assisted security tooling, an area that is likely to become increasingly important for modern analysts.