Socratic questions are not asked blindly. They are meant to help the student to realize the incompleteness or falsety of what they think they know. That us, to realise they do not actually know something and thus put them into a position of not knowing, which is the only way to gain understanding.
One man said, that enlightement (as in Zen) cannot be faked. Whoever had it, will be able to tell if someone else had it or not. There is no way to merely sound like it. Socratic method is like enlightement.
This is really cool. I think we will see more of these, inching towards 'refined' thought and expression percolating from AI models into human minds.
I noticed that the quality of email text has gone up around my work these days. everyone is sounding like a master of drafting emails. Ofcourse Ai has a role in it. But real cool thing about this is, this is pulling up people's thinking to have a structure, minimalism and sequencing of expression. Same as how textile industry has enabled people to have an style and expression in clothing.
If you read them, the skills are completely different (with the Jane Street skill being more complex). Socratic questioning [0] is a well-known technique. Indeed, a Github search returns 92 repositories for "socratic tutor skill" [1], suggesting that the well-known technique has been added to LLM tools independently by many people.
Yep - just looked at my projects in Claude, and my first "socratic tutor" was registered on 2nd of December 2024. I forget where I got the inspiration though (probably X)
Same with dialog-tuning in the first place. That you could manipulate the context of unicorn-text generators like GPT-2 in order to actually have something like a conversation with them, or that a model could generate "private thoughts" before answering, were ideas which weren't as self-evident as they seem now. But still ideas a lot of people had at the same time.
I dunno if you really need this. I asked AI to generate me a Lean tutorial with exercises and it did a really great job, one-shot with no skills. My 4th attempt to learn Lean and I'm actually making progress this time. (Mostly because you can say "I'm a bit stuck here, can you give me a hint" - it's really good at Lean even without access to the Lean compiler.)
Maybe it improves things a bit though. I might try it next time.
One man said, that enlightement (as in Zen) cannot be faked. Whoever had it, will be able to tell if someone else had it or not. There is no way to merely sound like it. Socratic method is like enlightement.
I noticed that the quality of email text has gone up around my work these days. everyone is sounding like a master of drafting emails. Ofcourse Ai has a role in it. But real cool thing about this is, this is pulling up people's thinking to have a structure, minimalism and sequencing of expression. Same as how textile industry has enabled people to have an style and expression in clothing.
[0]: https://en.wikipedia.org/wiki/Socratic_questioning [1]: https://github.com/search?q=socratic+tutor+skill&type=reposi...
My client just got done firing someone who thought filling the codebase up with files like this would be a good idea.
It would seem the AI revolution is happening at wildly different rates depending on where you are standing.
Maybe it improves things a bit though. I might try it next time.