The AI Futures Project^[1] distinguishes several different plans for navigating the AI transition, which I'll lump into two broader categories:

Plan A (my combination of their Plan A/S/Other) involves working for a global pause on AI development, hopefully lasting decades. During this pause, we either research preparatory technologies like alignment or human intelligence amplification, or take actions to "fortify" the pause so that it can last indefinitely.

Plan C (my combination of their Plan B/C+/C/sort of D)^[2] is for a world where we fail at getting a global pause, and race dynamics continue to dominate. If the leading AI companies are alignment-pilled, they coordinate a mutual pause among themselves, lasting the length of their joint lead over the leading non-alignment-pilled AI company. Then they "burn the lead" researching alignment, and release hopefully-aligned superintelligence at the last possible moment before the racing non-alignment-pilled AI companies beat them. The exact calculus changes depending on who is in the lead when - is there one alignment-pilled AI company with a six-month lead in front of everyone else? Are there two alignment-pilled AI companies with a joint two-month lead in front of their nearest competitor? Etc. The point is, if it's impossible to stop the race, that six-month or two-month lead is the amount of time they have to prepare, and they can spend it more or less wisely.^[3] The point of burning their lead at the last moment is that they can do alignment research with very advanced, right-on-the-brink-of-superintelligence AIs. This is a dangerous plan, and there would be many subtle points they would have to get right, but in a world where nobody can get a global pause in time, it's hard to think of a better one.

Plan A benefits from raising mass awareness, educating politicians, and preparing the verification technology that could enforce a pause treaty. Plan C benefits from alignment-pilling companies, having the alignment-pilled companies be as far in the lead as possible, making sure the alignment-pilled companies will burn their lead when the time comes, and giving them as good alignment teams as possible (so that burning the lead produces good results).

From the beginning of AI safety as a community to about 2023, the conventional wisdom was that our chances of Plan A were slim, and we should be preparing for Plan C. From about 2023 onward, as the general population woke up to the implications of ChatGPT, we rightly concluded that the chance of getting Plan A was better than expected, and we should invest more resources into trying to make it work. During both periods, the natural strategy was to work as hard as we can to get Plan A, where chances of survival are highest, but to be prepared to execute Plan C competently as a backup if that didn't pan out.

Now I think the pendulum has swung too far. New members of the community are unaware that Plan C was ever considered a possibility, or think it's some kind of accelerationist plot to distract us from Plan A. Older members heap scorn on it and act like everybody knows it's been debunked. Most concerningly, people propose burning our chances of executing Plan C in exchange for infinitesimal increased chances of getting Plan A, or even for no increased chance of Plan A at all.

I'll lump these ideas together under the heading of "SURA" - the idea that the only ethical course is for people/companies/countries to Stop Unilaterally + Raise Awareness, with no plan for what happens if race dynamics continue anyway. I interpret the planks of SURA's platform as:

• Individual alignment-pilled AI companies are morally obligated to stop unilaterally, regardless of whether they expect race dynamics to continue in their absence.
• Individual alignment-pilled employees at these companies should quit, regardless of whether they're in capabilities or alignment roles, and regardless of whether they think their company is morally better than its competitors.
• Individual alignment-pilled countries (for example, a hypothetical future alignment-pilled US) should ban or heavily restrict AI progress unilaterally, regardless of whether they expect other countries (for example, China) to continue racing in their absence.

All of these planks are net negative in a world where we end up needing Plan C:

• If we demand that leading alignment-pilled companies quit the race unilaterally, then we ensure that non-alignment-pilled companies end up in the lead.
• If we demand that alignment-pilled company employees quit their jobs, this makes companies less alignment-pilled and less prepared to research alignment if they need to. It also is likely to decrease the lead that alignment-pilled companies have over the nearest non-alignment pilled company.
• If we demand that the US cease AI research unilaterally (ie without a mutual pause treaty with China), we ensure either that China gets superintelligence first, ^[4]or that the US has to restart with zero lead when it realizes its mistakes.

SURA's continuing popularity despite these downsides comes from a set of questionable assumptions:

1. Alignment is doomed (without some extraordinary extra advantage like decades of time, or superhuman geniuses). Therefore, Plan C is also doomed - or, at best, it raises our chances of survival from something like 0.00001% to something like 0.00002%. Therefore, there is no reason to optimize for Plan C.
2. Racing to build superhuman AI is deontologically evil, and it's prohibited for good people to do it regardless of whether or not it may have good consequences.
3. Racing burns timeline, so anyone unilaterally quitting the race would give us more timeline, which we could use for good things.
4. If lots of people quit, that will raise awareness, which could be helpful for getting a global pause.