“Wtf, the AIs suddenly got...weirdly aligned?”

“Oh shit.”

“What??”

“Imagine you work for a paranoid autocrat. You decide to to coup him - wouldn’t you ease his fears by acting aligned?

So, AIs suddenly *appearing* aligned should set off alarm bells - what if they’re about to seize power?

Before celebrating, we need to be sure they’re *actually* aligned.”

SHARP LEFT TURN: the AIs capabilities suddenly begin generalizing far out of distribution, i.e. “the AIs gain capabilities way faster than we can control them”

SHARP RIGHT TURN: the AIs suddenly appear super aligned. They could be genuinely aligned or deceiving us.

@TheZvi: “To me the ‘evolutionary argument for the sharp left turn’ is that humans exhibit sharp left turns constantly. Humans will often wait until they have sufficient power to get away with it, then turn on each other.

This is common, regular, very ordinary human strategic behavior.

- You work your job loyally, until the day you don’t need it, then you quit.

- Your military works for the people, until it is ready to stage a coup.

- You commit to your relationship, until you see a chance for a better one.

- You pretend to be a loyal security officer, then assassinate your target.

The sharp left turn… isn’t… weird? It’s… very standard?”

Another example: To win the support of the Southern Democrats, LBJ pretended to be racist for decades. Then, shortly after becoming president, he betrayed them by passing the Voting Rights Act and Civil Rights Act.