🔄 Topic

Turning a local LLM in LM Studio into an agent that can actually do things, using function calling and MCP servers.

🎯 Goal

Understand how a chat-only local model becomes a tool-using agent, and what new risks appear the moment it gets real capabilities.

🛠 What I Did

I explored how to give a local model running in LM Studio real capabilities instead of just text output. A plain chat model can only talk. To act, it needs function calling: the model emits a structured tool request, and a runtime executes it. I set up and tested MCP servers as the bridge, including a desktop-control tool, and compared what the local setup could do against a cloud agent.

Main areas covered:

  • LM Studio local model hosting
  • function calling requirements
  • MCP servers as tool bridges
  • desktop-control tooling
  • comparing local agent vs cloud agent capabilities
  • planning remote control from the iPhone

🔗 Key Cybersecurity Connections

The moment a model can execute tools, it stops being a toy and becomes an attack surface. Every MCP server is a capability grant: filesystem access, shell access, network access. This is exactly the kind of privilege-boundary thinking that defenders apply to any service account.

🔍 Investigation Questions

  • Which tools does the agent actually have access to?
  • What is the blast radius if the model is tricked into a bad tool call?
  • Are tool executions logged anywhere reviewable?
  • Does the setup expose anything to the network?
  • Which model families support function calling reliably?

🚨 Detection Opportunities

Potential monitoring ideas:

  • unexpected shell commands from the agent runtime
  • tool calls outside normal working folders
  • new MCP server registrations
  • network connections from the model host
  • spikes in tool-call frequency

Example:

project=local-agent-stack
change_type=new_tool_capability_granted
risk_area=agent_tool_execution
triage=review_tool_scope_and_logging

🧭 MITRE ATT&CK Techniques

Possible mappings depending on confirmed behavior:

  • T1059 — Command and Scripting Interpreter
  • T1071 — Application Layer Protocol
  • T1078 — Valid Accounts

🗺 Visual Investigation Diagram

Local model
    ↓ Function calling
    ↓ MCP servers
    ↓ Real capabilities
    ↓ New attack surface

⚠ Challenges

The challenge was that not every local model supports function calling well. Some models emit broken tool requests, and debugging whether the model or the runtime failed takes patience.

📚 What I Learned

I learned that “agent” is not a model feature. It is an architecture: model plus tool bridge plus permissions. The security posture lives in the bridge, not in the model.

➡ Next Steps

  • Add more MCP servers deliberately, not all at once
  • Keep a written inventory of granted capabilities
  • Test remote access to the agent from the iPhone
  • Evaluate stronger local models for tool use

🧠 Reflection

This was useful because it forced me to think about capability grants the way a defender thinks about service accounts: every new power needs a reason and a boundary.

🧩 Lessons Learned

What worked

Treating each MCP server as an explicit permission decision.

What broke

Assuming any local model could do function calling.

Why it broke

Tool use depends on model training and runtime support, not just on the chat interface.

Fix / takeaway

Verify tool-calling support first, then grant capabilities one at a time.

📈 Skill Progression Context

This supports my cybersecurity progression because AI agents with tool access are becoming real infrastructure, and defenders will need to audit them like any other privileged automation.

😄 TL;DR

A chatbot with hands needs a leash.