View on Threads

With the explosion of model:effort combinations, I’m wondering if we’re getting the interface wrong. Do people really use fine-grained control of effort levels so much that OpenAI needs to sell a special keyboard?

I don’t want to specify the model’s effort. I want to specify my effort.

If I am an expert in a domain, I want a very specific interaction with my agent. Even if I don’t have the exact outcome ironed out, I have a sense of what could work and what won’t. I’ll sometimes specify how I want things done, depending on how much it matters. In other words, what I want is a very eager and capable intern - one that takes good-faith interpretations of my prompts and is careful to offer thoughtful suggestions or ask only non-obvious questions.

A really great example of an expert using AI is Terence Tao’s ChatGPT conversation about the Jacobian Conjecture. Now, I’m not a mathematician, and I have no idea what the “Jacobian Conjecture” is or why proving it’s false is important. But in looking at Terence’s conversation, it’s clear that he is an expert. He always directs the conversation and ChatGPT’s suggestions are largely ignored.

However, there’s so much extraneous output from ChatGPT, like “What remains genuinely special” (Terence already knows) or “I actually like this description better” (Terence does not care). Terence had the patience to deal with it, but in an ideal world this conversation would’ve been much smoother.

Conversely, I want to indicate when I am totally fish out of water and I want to lean on the agent as an expert. Perhaps I want my agent to do a complex task, like an accountant doing my taxes. Or maybe I want to learn something new, and I need a teacher to give a lecture.

And like any good expert, I expect an agent to tell me if it doesn’t have the capabilities I need.