跳到正文
@emollick· @emollick · X·· 2026-08-24AI 评分30
AI 导读

发表仅引用旧模型(如GPT-4)的AI影响研究并非不可,但需谨慎讨论并向非技术读者明确说明。正面能力发现通常不会因模型迭代而失效,负面结论则需更严谨:可明确限定“GPT-4无法完成X”、跨代对比(至少含一个推理模型)、论证固有局限、聚焦调节变量或人类反应。负面能力论断的持久性普遍弱于正面论断。

正文

It is not inherently bad to publish research on the impact of AI that only refers to older models, but it requires a very careful discussion and has to be very clear to non-technical readers.

If you show AI can do something, its fine. Generally, once AI has gained an ability, it does not regress in future generations. So papers that show that AI has crossed a threshold (such as "good as a human") or that seek to establish some sort of minimum impact or effect still hold up even if they used GPT-4.

If the finding is that AI is bad or biased at something, you need to be much more careful. You cannot claim that because GPT-4 fails at something, that AI is bad at that task, because current or future models may do it. So what can you claim? These are just some examples:
1) You can just be explicit: "GPT-4 could not do X." This is a rare way to frame things because it is not really an interesting publishable paper in many cases. However, if you provide the information needed to reproduce your work, it can become a useful benchmark to measure progress.
2) You can measure trends: compare GPT-4 to GPT-5 to GPT-5.6 Sol or whatever (it is really important to include at least one reasoning model). You can then make better claims about abilities relating to this task.
3) You can make a strong, grounded argument that AI has a natural flaw or limitation that means it cannot do X, and then demonstrate evidence that this might be the case.
4) You can focus on a moderator or mediator: this sort of prompting or approach or social context or connection impacts the ability of GPT-4 to do X, and needs to be a concern for the future.
5) You can focus on humans: how do people react to AI? what are the failures and successes? what dangers or advantages or changes does it bring to us?

Again, these aren't exhaustive, but generally negative capability claims have been much less durable than positive ones.

来源:@emollick · x.com