OpenAI's Chief Scientist Jakub Pachocki: "Based on internal results, I have a strong expectation that this speed of progress could be sustained into recursive self-improvement."

OpenAI is organizing research around this because it sees RSI as essential to remaining at the frontier.

At the same time, Pachocki says evaluations show declining reliability of chain-of-thought monitoring.

OpenAI is betting on AI increasingly driving its own development while finding it harder to reliably monitor how these systems reason.

Tough times ahead. RSI within reach, but hard to align.

He even quotes Ray Kurzweil:

"And, in line with Ray Kurzweil’s predictions from the end of the XXth century⁠(opens in a new window), we now find ourselves at the moment in history of computing where machine intelligence is starting to exceed that of humans in transformative ways."