🔄 Topic

The local models got treated like a fleet instead of a collection: benchmarked, reconfigured, migrated, retired, and put on a token budget.

🎯 Goal

Keep only the models that earn their disk space and memory, route each lane to the best current option, and stop letting context windows silently eat the machine.

🛠 What I Did

I ran a lifecycle pass over the whole local model stack.

Main areas covered:

  • benchmarked and configured the Hermes local models against real tasks
  • migrated the local coding lane to Gemma after the numbers favored it, and tuned its context window deliberately
  • audited the resulting configuration instead of assuming the migration landed correctly
  • retired superseded models — the old Gemma 3 27B and my earlier custom coder build — after confirming nothing routed to them
  • added token-aware context controls so agents trim what they send instead of stuffing the window
  • set up an LM Studio Qwen workflow for Aider as an alternative lane
  • wired a local model into Xcode as an ACP agent, joining Codex and Claude as in-IDE options

🔗 Key Cybersecurity Connections

This is lifecycle management, the unglamorous discipline security depends on: inventory what runs, measure it, migrate on evidence, decommission cleanly. A retired model still installed is like a retired service still listening — probably harmless, definitely unaccounted for.

Token budgeting has a security face too: context windows carry data. Controlling what gets stuffed into a prompt is controlling what a model gets to see, which is an exposure decision as much as a performance one.

🔍 Investigation Questions

  • Which lane routes to which model, and does the config match the intent?
  • Did anything still reference the retired models?
  • What does the context window actually contain on a typical task?
  • Did the migration change output quality, or only speed?
  • Is there a benchmark record to justify each routing choice?

🚨 Detection Opportunities

Checks for a local model fleet:

  • lane routing to a model that no longer exists
  • context payloads growing beyond the token budget
  • benchmark results missing for a model in active routing
  • retired model files still present after decommission
  • config drift between the documented setup and the live one

Example:

project=local-model-fleet
signal=lane_routed_to_retired_model
risk_area=configuration_drift
triage=compare_routing_config_benchmarks_and_installed_models

🧭 MITRE ATT&CK Techniques

No direct mapping claimed. This is configuration and lifecycle hygiene for local AI infrastructure.

🗺 Visual Investigation Diagram

Benchmark the fleet
    ↓
Route lanes on evidence
    ↓
Migrate + audit the config
    ↓
Retire what lost, verify nothing points at it
    ↓
Budget tokens per task
    ↓
Leaner, measured, documented stack

⚠ Challenges

Deleting models felt wasteful — gigabytes I downloaded, a custom build with my name on it. But the audit habit from earlier this month applies: unused capability is not an asset, it is unaccounted surface.

📚 What I Learned

I learned that “which model” is a decision that expires. Last month’s benchmark crowned a different model than this month’s. Routing has to follow current evidence, not loyalty.

➡ Next Steps

  • Re-benchmark after each significant model release
  • Watch the token controls for tasks that legitimately need more context
  • Keep the Xcode ACP lane on the same benchmark discipline
  • Record every retirement with what replaced it and why

🧠 Reflection

Fleet thinking changed my relationship with the models: less fandom, more operations. The stack got smaller and better at the same time, which is usually the sign the process is right.

🧩 Lessons Learned

What worked

Benchmark → migrate → audit → retire, as one deliberate sequence.

What broke

My attachment to models that no longer earned their place.

Why it broke

Sunk cost: downloaded gigabytes and custom builds feel like property.

Fix / takeaway

Route on current evidence, retire cleanly, and treat context tokens as a budgeted resource.

📈 Skill Progression Context

This supports my cybersecurity progression because lifecycle management, configuration audits, and decommissioning discipline transfer directly to managing any production infrastructure securely.

😄 TL;DR

Benchmarked the fleet, promoted Gemma, retired the losers, and put every prompt on a token diet.