> I've been finding that my reaction to inaccuracies in LLM output about my system has been largely that there's no way a human would know the right answer there, either

I've had some success with pointing an LLM at a piece of code and asking it to identify the parts that are not obvious, and for those parts to describe what it thinks it does and to suggest a comment. It does a pretty good job of identifying unusual stuff and the explanations are ok, but just as often they're bugs. As a bonus, once you fix the comments it suggested, then it's more readable for humans and LLMs alike.

It also works to ask it to review your README.md for suggestions for improvements. It's also done wonders for my error path handling, since it auto-suggests useful error messages including all the relevant details. Good error messages are important but sometimes they're a distraction while you're coding.

Coding LLMs have seen every common programming pattern under the sun, so if it's not recognising it it definitely needs a comment.