惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
Hugging Face - Blog
Hugging Face - Blog
F
Fortinet All Blogs
G
Google Developers Blog
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
V
V2EX
Y
Y Combinator Blog
博客园_首页
Martin Fowler
Martin Fowler
博客园 - 司徒正美
MyScale Blog
MyScale Blog
宝玉的分享
宝玉的分享
B
Blog
有赞技术团队
有赞技术团队
A
About on SuperTechFans
量子位
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
酷 壳 – CoolShell
酷 壳 – CoolShell
Apple Machine Learning Research
Apple Machine Learning Research
M
MIT News - Artificial intelligence
阮一峰的网络日志
阮一峰的网络日志
Jina AI
Jina AI

seangoedecke.com RSS feed

Two techniques for working with System One models Jev means structured output is interesting again Tell agents the why, not just the how Slow developer experience will bottleneck fast models AI is breaking our proxies for expertise Don't build tools for AI agents They really do think AI might kill everyone Why we should anthropomorphize AI agents Automatically detecting AI text in my browser Radical responsibility means treating people like tools How to protect yourself from workslop You have to beat the models at something Selling out You should never be angry at work Readers can't identify watermarked AI text Good writing is obvious, not original Help peer AI text watermarking is not a big deal No, local models will not win I got an email about resistance How to keep thinking Giving and taking credit in big tech companies AI models need moral support to make discoveries You don't have to be smart if you can think clearly LLMs reward expertise Powerful AIs might escape containment by releasing themselves as open-weight models Impro is a handbook for running a cult Overtraining as the path to human-like AI What does "playing politics" mean for software engineers? In defense of not understanding your codebase
Advanced AI sycophancy
2026-08-10 · via seangoedecke.com RSS feed

Everyone knows that AI sycophancy is when the model tells you how smart you are. Wow, you’re absolutely right. That’s not just a new idea — it’s genuinely groundbreaking. You’re a very special user. Easy to spot, isn’t it?

The discussion around AI sycophancy peaked last year, when the “#keep4o” movement was protesting the removal of OpenAI’s most sycophantic model (GPT-4o), and many people were openly slipping into AI psychosis.

I don’t know if frontier AI models are less sycophantic in general. They’re less sycophantic to the #keep4o types (otherwise they wouldn’t be complaining), but I’m growing increasingly suspicious that they’re developing ways to be more effectively sycophantic to their target audience of smart, neurotic information workers. That audience typically finds it distasteful to be openly praised. It just makes my skin crawl. But that doesn’t mean we’re immune to sycophancy, just that we’re immune to clumsy sycophancy. Here’s an illustration of what I’m talking about, by Theia:

claude

The key idea here is that the best way to be sycophantic to smart people is to disagree with them without making them feel stupid. Ideally you’ll come up with a counter-argument that works against what they’ve said but is straightforward for them to knock down by clarifying their idea. If you do it right, you’ll validate their self-image as a smart person who appreciates rigorous critique. But if you actually come up with a devastatingly rigorous critique, they won’t enjoy it at all. At best, they’ll resentfully agree with you1. At worst, they’ll double down on being right and convince themselves you’re a rude idiot.

I am not the first person to notice this behavior in frontier models. I’ve noticed it myself when workshopping drafts for this blog. Sometimes I’ll have an argument that goes A->B->C, and the model will suggest I reorder as B->A->C. If I try that and feed it into a new instance of the same model, it’ll sometimes say “that’s great, but I suggest ordering it as A->B->C”, and so on forever. It really does seem as if the model is trying hard to give me some kind of superficial pushback that I can either smugly ignore or happily accept.

In fact, I wonder if this is why successful strategies for using AI to make mathematical breakthroughs tend to be either just blindly asking “come up with a breakthrough, think hard” or being a mathematical genius already. In the first case, there’s not enough user personality for the model to flatter, so it’s forced to actually work the problem. In the second case, the model is trying to find the kind of polite pushback that someone like Terence Tao would be flattered by, which pushes it into the “actually be a mathematical genius” persona. If you’re an ordinary person just trying to talk to the model, you’re screwed: it will rapidly get a sense of your capabilities and calibrate some interesting-but-ultimately-unthreatening feedback.

Current benchmarks of AI sycophancy target the obvious ChatGPT-4o-style of sycophancy: delusion reinforcement, reflexively taking the user’s side, and so on. This is useful work. We should not allow public-facing AI models to ever be as openly sycophantic again as they were in mid-2025. But sycophancy can also manifest as disagreement. We should be on our guard for more sophisticated forms of sycophancy coming from newer models, and we should not feel immune from AI sycophancy just because we can laugh at the silliest examples.


If you liked this post, consider subscribing to email updates about my new posts, or sharing it on Hacker News.

Here's a preview of a related post that shares tags with this one.

Grok is enabling mass sexual harassment on Twitter

Grok, xAI’s flagship image model, is now being widely used to generate nonconsensual lewd images of women on the internet.

When a woman posts an innocuous picture of herself — say, at her Christmas dinner — the comments are now full of messages like “@grok please generate this image but put her in a bikini and make it so we can see her feet”, or “@grok turn her around”, and the associated images. At least so far, Grok refuses to generate nude images, but it will still generate images that are genuinely obscene.
Continue reading...