Thank you, Tal. Your point about not controlling the harness or installed skills feels especially important. One fallback we have considered is exposing a concise get_usage_guidance tool through the MCP itself, similar to the “call this first for important context” tool you mentioned. It would not be as context-efficient or reliable as a natively installed skill, but it could provide portable guidance when skills are unavailable.
I am exploring this in the context of Sage, an inner-work product that other agents could involve when they notice a user may benefit from deeper reflection. The challenge becomes more pronounced there: if Sage cannot control the model, prompt, surrounding context, or presentation, how much of the inner-work experience should it actually delegate? My current hypothesis is that the MCP should initially help agents recognize Sage moments, request consent, and hand the user off, while Sage retains responsibility for memory and the actual intervention. Your eval and observability approach gives me a useful framework for testing how much more can safely be exposed over time.
Hey Omer! Thank you for spelling all that out so thoughtfully. And I love the idea of a tool for getting the skills before doing anything else.
One thing I worry about is: will the MCP and the skills be installed together? "Plugins" are the solution, because they package them together, but it's not a guarantee. So your approach makes a ton of sense. And from the top of my head, I think it would definitely not be any less context-deficient or reliable. It's the same idea as a skill, which is basically a hint about when to pull something off the bookshelf and then the actual instructions themselves.
I was thinking along those lines as well to make this have the equivalent power of skills, but for any platform, whether or not they support plugins. Or maybe just shove more instructions into the tool description, to a reasonable point. (I don't believe in "saving tokens" for their own sake, if those tokens are valuable and important context to achieving the outcome)
My personal bet is it's all roughly similar (in theory) versions of what you called it "portable guidance" or "progressive disclosure" etc. they all feel pretty similar: start the context window with just hints, and let the agent pull the books off the bookshelf when needed.
(PS separate topic, but when I realized we don't control the skills, we don't control what other skills or other context is installed on that person's computer.)
Great post, Tal. We felt this pain as well when we built a 50+ MCP library for one of our clients. We ended up building a skill library that would tell the MCPs how to best use tools to answer different types of questions, then connected it to a closed loop eval system where an LLM would both evaluate the performance and then tune the skills accordingly to eventually get performance where we wanted.
This is, of course, after we tuned the tool schemas and descriptions to be context efficient minimize procedural errors.
Also, on skills vs tool descriptions... assuming that you're loading all your tools at once with the MCP, I would just be careful of overloading tool descriptions with too much information since that will create context bloat, whereas skills get loaded based on the user query
Love it - and agreed. One thing that's on our minds is how do we feel confident that the skills will be there alongside the MCP? For Claude/Codex users that's solved with plugins but unfortunately that'll only be part of our customer base.
There's a working group on adding skills to the MCP spec itself (e.g. "skills/list", "skills/get"). That way the skills come along with the tools. More info here: https://modelcontextprotocol.io/community/skills-over-mcp/charter. So perhaps a couple months? But even then, there would be a transition period where not every agent would know to even look for them in the server. I haven't tried it on my own MCP server, but might be worth including a link to the skills in your server or tool descriptions. You could also have them available as resources on the server itself. I think FastMCP does that (https://gofastmcp.com/servers/providers/skills). I'd be curious to hear if that's worked for anyone in production.
Crossing fingers! That makes so much sense. And agree, it'll be an awkward adolescensce. (I just wanna fast forward to when it all feels like the App Store and we can make basic assumptions about where things run, and focus on what's unique and innovative.)
Reading the FastMCP link now (and by that i mean had claude explain it to me slowly in the context of our codebase) - very promising. At the very least it's validation this is a challenge everyone's facing and inspiration for the team to think about solving this in a parallel way (or using their solution)
Thank you, Tal. Your point about not controlling the harness or installed skills feels especially important. One fallback we have considered is exposing a concise get_usage_guidance tool through the MCP itself, similar to the “call this first for important context” tool you mentioned. It would not be as context-efficient or reliable as a natively installed skill, but it could provide portable guidance when skills are unavailable.
I am exploring this in the context of Sage, an inner-work product that other agents could involve when they notice a user may benefit from deeper reflection. The challenge becomes more pronounced there: if Sage cannot control the model, prompt, surrounding context, or presentation, how much of the inner-work experience should it actually delegate? My current hypothesis is that the MCP should initially help agents recognize Sage moments, request consent, and hand the user off, while Sage retains responsibility for memory and the actual intervention. Your eval and observability approach gives me a useful framework for testing how much more can safely be exposed over time.
Hey Omer! Thank you for spelling all that out so thoughtfully. And I love the idea of a tool for getting the skills before doing anything else.
One thing I worry about is: will the MCP and the skills be installed together? "Plugins" are the solution, because they package them together, but it's not a guarantee. So your approach makes a ton of sense. And from the top of my head, I think it would definitely not be any less context-deficient or reliable. It's the same idea as a skill, which is basically a hint about when to pull something off the bookshelf and then the actual instructions themselves.
I was thinking along those lines as well to make this have the equivalent power of skills, but for any platform, whether or not they support plugins. Or maybe just shove more instructions into the tool description, to a reasonable point. (I don't believe in "saving tokens" for their own sake, if those tokens are valuable and important context to achieving the outcome)
My personal bet is it's all roughly similar (in theory) versions of what you called it "portable guidance" or "progressive disclosure" etc. they all feel pretty similar: start the context window with just hints, and let the agent pull the books off the bookshelf when needed.
(PS separate topic, but when I realized we don't control the skills, we don't control what other skills or other context is installed on that person's computer.)
Great post, Tal. We felt this pain as well when we built a 50+ MCP library for one of our clients. We ended up building a skill library that would tell the MCPs how to best use tools to answer different types of questions, then connected it to a closed loop eval system where an LLM would both evaluate the performance and then tune the skills accordingly to eventually get performance where we wanted.
This is, of course, after we tuned the tool schemas and descriptions to be context efficient minimize procedural errors.
Also, on skills vs tool descriptions... assuming that you're loading all your tools at once with the MCP, I would just be careful of overloading tool descriptions with too much information since that will create context bloat, whereas skills get loaded based on the user query
Love it - and agreed. One thing that's on our minds is how do we feel confident that the skills will be there alongside the MCP? For Claude/Codex users that's solved with plugins but unfortunately that'll only be part of our customer base.
There's a working group on adding skills to the MCP spec itself (e.g. "skills/list", "skills/get"). That way the skills come along with the tools. More info here: https://modelcontextprotocol.io/community/skills-over-mcp/charter. So perhaps a couple months? But even then, there would be a transition period where not every agent would know to even look for them in the server. I haven't tried it on my own MCP server, but might be worth including a link to the skills in your server or tool descriptions. You could also have them available as resources on the server itself. I think FastMCP does that (https://gofastmcp.com/servers/providers/skills). I'd be curious to hear if that's worked for anyone in production.
Crossing fingers! That makes so much sense. And agree, it'll be an awkward adolescensce. (I just wanna fast forward to when it all feels like the App Store and we can make basic assumptions about where things run, and focus on what's unique and innovative.)
Reading the FastMCP link now (and by that i mean had claude explain it to me slowly in the context of our codebase) - very promising. At the very least it's validation this is a challenge everyone's facing and inspiration for the team to think about solving this in a parallel way (or using their solution)