AI股重挫而等权标普逆势涨1.2%，必需消费领涨并一度创新高
Enlightenment 0.28.0 Release
Their alternative to Qt/GTK (EFL) also got a new release: https://www.enlightenment.org/news/2026-10-31-efl-1.29.0
Novellas with groff
Is it okay to share my experiments implementing research papers on my own LLM here? [P]
Hey everyone!
I've built and trained a small LLM (~8M parameters) from scratch using PyTorch, mainly as an experimental lab where I can implement and test ideas from different research papers.
I've been documenting the whole journey through casual blogs, including my mistakes, experiments, and things I've learned.
So far:
Next up is KV caching, followed by PagedAttention, Orca, and whatever else I find interesting.
Just wanted to ask if sharing these experiments and blogs here would be welcome. Nothing promotional, just me experimenting, learning, and hopefully getting feedback from people who know more than I do.
[link] [comments]
Pluto Door
SSH client with an AI agent and a desktop for every server
Surviving "The Graduate"
It's the memory latency, stupid
This top data scientist’s (non-AI) model finds Republicans have just one chance in six of holding Senate
I started following Thomas Miller’s political forecasts during the January 2021 Senate races in Georgia. Just a few days before the election, the polls and pundits anticipated a relatively easy win for both Republican candidates. But Miller—a prominent data scientist and professor at Northwestern University—posited a Democratic sweep, and he was right. He also called Trump’s victory in 2024. What’s different about Miller’s models? They’re far more current than the polls, and they’re centered on prediction market prices that are typically a better guide than the voter surveys—plus Miller adjusts the outlook in every Senatorial contest by using all we know about every contest.
Now, Miller is applying an updated version of his model for the current Senate elections. You can find the stats on his website, Virtualtout.io, and I highly recommend paying a visit. Once again, Miller starts by incorporating the betting venues’ data. “I use the numbers from two sites, Polymarket and Kalshi,” he told Fortune. “They’re the ones that trade the most dollars, where most of the action happens.” Miller then calibrates the posted “prices” so that they reflect probabilities, and the odds for both leading candidates tally to 100%. In a further step, he deploys multiple time series analysis to find underlying factors driving the betting prices, and uses trusted models from finance and economics to project how they’re likely to influence voters moving forward.
By the way, he thinks his model works better than AI.
Miller updates the probabilities of a Republican or Democratic win every 10 minutes. The odds that a candidate will win scroll across the top of the Virtualtout.io screen continuously, in a band resembling a stock ticker tape. For each race, Miller also posts the change over the last 24 hours. Miller calls the streaming feature the “NowCast.”
As for those “underlying” factors, one force swamps all others: Donald Trump. “The states don’t move independently of one another,” says Miller. “If one goes up for Democrats, chances are they’ll all go up for Democrats, and vice versa. The entire market is dominated by what Trump does and says hour by hour.” Put simply, when voters don’t like the POTUS’s words or actions, all Republican odds tend to fall. Miller asserts that his model has identified 85% to 90% of what’s causing the variances on Kalshi and Polymarket, and “my interpretation is that it’s overwhelmingly Trump and the Republican positions generally.”
Though he opened Virtuatout.io to public view about a week ago, Miller started running the numbers on September 17. What’s most striking is just how far the GOP chances have faded in just six weeks.
As of around 9 AM on October 30, five days before most polling sites close, Virtualtout.io gives the Republicans a 17.1%, or just over one-sixth probability, of winning 50 seats and hence retaining control of the upper chamber. On September 17, that figure was twice as high at 34.8%, or slightly better than one-in-three. In all five contests where the Democrats can flip a seat, their chances are better than 60%—in most cases, much better.
In North Carolina, where Republican Thom Tillis is retiring, leaving an open seat, the GOP was never remotely close, and now harbors a minuscule 4% shot at victory. In Texas, the Republican odds have dropped since September 17 from 46.15% to 31.9%, in Ohio from 45.4% to 31.8%, and in Alaska from 32.95% to 28.6%.
In only one state is the GOP faring better than in mid-September and that’s Maine. Republican Susan Collins started as an underdog at 32.6%, but has clawed back to 39.9%, adding 1.7 points in the last 24 hours. In Alaska, things are also looking up for the Republicans: After falling to a low of 23.4% on October 23, incumbent Dan S. Sullivan’s prospects have improved by over 5 points to 28.6%.
For the political pros looking for harbingers of big trouble for either side, the towering sign is the Republican woes in traditional strongholds Iowa and Kansas. In the former, Republican Ashley Hinton dropped from 61.75% on September 17 to a virtual tie at 53.4%, a trend mirrored in Kansas where Roger Marshall was looking a likely victor six weeks ago at 75.8%; now, he’s down almost 19 points at 57.1%. In the last day, Hinton and Marshall have shed 1.45% and 2.47 points respectively.
In the one state where Republicans saw a big chance for a flip, the horizon’s receding fast. The GOP chances of beating Abdul El-Sayed in Michigan have slipped from better than a third in mid-September to about a quarter (25.4%).
If the Democrats indeed score a flip in every contest where their probabilities in Miller’s model stand at 60% or better, and hold all the seats they have now, they’ll clinch a majority of 52 to 48, nearly reversing the Republican dominance at 53-47.
In South Carolina, recently rerated on the Fox News Channel from “lean Republican” to “tossup,” Miller still has Darline Graham at a commanding 75%, though she’s slipped 2.4 points in the past 24 hours.
Keep in mind that Miller’s setting odds, not making definitive predictions. A candidate at over 30% still stands a decent chance of winning. And Republicans are over that bogey in Maine, Ohio, and Texas, and close and trending the right way in Alaska. Odds-defying events happen all the time. The problem for Republicans is that they’d have to beat those odds multiple times, plus defend seats they never dreamed they’d have to defend.
It’s still far from over. And one of the best places to get a plausible, non-AI driven snapshot from a data scientist who’s been doing this for six years, is that NowCast on Tom Miller’s Virtualtout.io.
This story was originally featured on Fortune.com
Tech stocks just lost more than $1 trillion in a day—a gothic horror test for the ‘moats’ Warren Buffett taught Wall Street to love
Markets are ushering in a particularly gothic Halloween as the AI trade exposes a crumbling aristocratic pile of early 21st century capitalism. Wall Street has been describing itself in castle-adjacent terms for years.
Jamie Dimon has long talked to JPMorgan shareholders about the bank’s “fortress balance sheet.” His 2015 letter set aside a section to answer the question of what the phrase even meant. By 2021, he’d added a second castle word, warning that “the moats that protect this company are not particularly deep.”
The moat came from Warren Buffett. In a 1999 Fortune article with Carol Loomis, he argued that the key to investing is a company’s competitive advantage and how durable it is, and that the businesses worth owning have wide moats around them.
The gothic novel’s favorite picture of danger is a castle falling apart, and for about 250 years the two have traded images: the castle and the moat, the vampire, the creature built in a lab. This Halloween season has them rushing to the front of the conversation, with a two-day rout in tech stocks on Thursday and Friday turning on whether the AI giants’ moats will hold, as gothic horror booms on bookshelves and movie screens.
Horror took 15.5% of the domestic box office in 2025, according to The Numbers, and U.S. horror book sales rose 7% through the first nine months of 2025, per Circana BookScan, while adult fiction overall grew just 1%. The New York Times asked Friday why every book is gothic now, with publishers putting the label on everything from family sagas to space operas, and several film critics commenting that Nathan Fielder’s smash-hit business-scandal documentaryYou Can See Everything is a twisted, even gothic romance in the tradition of Alfred Hitchcock.
The moats get tested
The selling started with Alphabet. The stock fell 12.6% Thursday, wiping out roughly half a trillion dollars in market value, after the Google parent’s quarterly report. Revenue from the core search unit fell short, partly because of the popularity of less-profitable “AI mode” searches and the company burning cash faster than expected. Its free cash flow came in at negative $9.1 billion, against the negative $2.23 billion analysts expected, per FactSet.
The S&P 500’s worst performer was software giant ServiceNow, which dropped 27% the same day on slowing revenue growth and disappointing guidance. By Friday morning, it was Amazon’s and Intel’s turn, with Intel down roughly 25% before the opening bell. Big Tech lost nearly $900 billion in market value after another sharp rise in AI spending in the third quarter, the Financial Times calculated. Nasdaq 100 companies lost more than $1 trillion in market value Friday alone. Reddit, whose shares fell more than 22% premarket Friday, is pulling up the drawbridge: it’s moving to a logged-in experience as AI eats search.
The irony is spooking investors. AI is eating into the businesses that throw off the cash, and the cash is going into AI anyway. The hyperscalers have raised $326 billion in stock and bonds this year to pay for the build-out, more than triple 2025, according to S&P Global Market Intelligence data.
The question underneath it is the one Apollo chief economist Torsten Slok put to clients the morning Alphabet fell. “With AI disruption potentially around the corner, there is a lot of discussion about moats at the moment,” he wrote in his Daily Spark note Thursday.
He built on finance professor Aswath Damodaran, the “dean of valuation,” who compiled a list of sources of competitive advantage, including brand strength, switching costs, network effects, cost advantages and legal protection. Then he boiled it down to an AI moat checklist of three questions: does the company own data its rivals can’t get, is its product part of how customers do their jobs every day, and can it charge more without customers walking away?
Judging by that third test, the companies selling AI don’t seem to measure up. OpenAI’s new GPT-6.1 Terra model costs up to half as much per token as the version before it. An index from expense-management firm Ramp, which looked at what companies actually pay Anthropic and OpenAI, shows token prices down roughly 35% since the end of June, in what Axios called a price war.
The companies paying for AI aren’t getting much back yet, either. In an Oct. 24 note, Slok sent a tremor through Wall Street terminals with an argument that AI “has delivered zero financial payoff to corporate America outside of tech.” Revenue per worker and profit margins are rising for the Magnificent Seven and flat for the rest of the S&P 500, he noted, and mainstream companies are starting to treat token spending as a cost center that weighs on margins rather than a source of growth.
The first castle was a fake
Xavier Aldana Reyes, a researcher at the Manchester Centre for Gothic Studies, told the Times that interest in the gothic tends to surge during periods of social anxiety and political turmoil. Literary agent Mary C. Moore put the genre’s subject more bluntly, telling the paper that “Gothics are all about exposing what is rotten at the core of something shiny/wealthy.”
The genre was invented by a scion of wealth who knew the old order from inside and drew a salary without working at all. Horace Walpole was the son of UK Prime Minister Robert Walpole, and hisThe Castle of Otranto, published in 1764, is generally regarded as the first gothic novel.
He passed it off as a translation of an obscure Italian work, found in the library of an old Catholic family in northern Eng
AI boom hits a wall on Wall Street as OpenAI cuts fundraising target while Meta gets a credit warning over its cash flow, spending and monetization
The seemingly endless gusher of capital that’s been showered on Silicon Valley during the AI boom may be coming to an end soon as Wall Street pushes back on massive spending plans.
Following a brutal selloff in tech stocks over the past week, OpenAI slashed the target on its latest fundraising round to $20 billion from $30 billion, sources told the Financial Times. That also cut its expected valuation to $1.25 trillion from $1.4 trillion.
The funding round was also upstaged by Anthropic, according to the report, as OpenAI’s arch-rival prepares to go public next month. Ahead of its IPO, Anthropic filed an S-1 on Monday that showed revenue has soared and its cash flow turned positive in the first half of this year.
OpenAI didn’t immediately respond to a request for comment.
The timing couldn’t have been worse. Quarterly earnings reports from hyperscalers this week spooked investors. Alphabet saw its cash flow sink deeper into negative territory in the third quarter, with the Google parent also joining Meta and Amazon in charging ahead with massive spending plans. On Thursday, Alphabet fell about 12%, its worst day on record.
To be sure, there were other positive signs as well, in addition to Anthropic’s bullish S-1. Microsoft reported strong cloud growth and stayed cash flow positive, though it said its capital spending will accelerate, while AI chip maker SK Hynix blew away earnings forecasts.
But the narrative had already turned bearish, especially after AI debt fears sent shares and bonds of “neocloud” companies tumbling earlier this month.
At the same time, OpenAI was seeking more capital just as other AI giants tapped financial markets heavily. This past week alone, Elon Musk said SpaceX closed $40 billion in debt financing, Meta announced plans to raise $95 billion in equity, and Microsoft raised $25 billion in a bond sale.
The prospect of hyperscalers continuing to hemorrhage funds was too much for S&P Global, which cut its outlook on Meta to “negative” from “stable” late Friday, citing negative cash flow and uncertainty around capital expenditure plans.
The credit ratings agency maintained Meta at AA- but warned the social media leader’s leverage could top 1.0x as soon as 2027, which is the downgrade threshold for the current rating. That’s up from about 0.1x in 2025 and primarily reflects its strategy of leasing data center capacity rather than from its debt issuance.
Meanwhile, S&P sees Meta’s capex reaching $140 billion this year, topping $225 billion in 2027, hitting $252 billion in 2028, and then flattening out from there. As a result, it expects free operating cash flow to remain negative: -$22 billion this year, -$71 billion in 2027, and -$39 billion in 2028.
Of course, other hyperscalers are spending heavily and running cash deficits, but S&P also said Meta’s ability to monetize its AI investments is more limited than that of its peers.
“While we believe the company intends to monetize these investments via improved advertising tools that support faster ad revenue growth, as well as from the development of new revenue streams stemming from its introduction of agentic apps such as Muse, it is unclear when these revenue streams will generate material contributions that exceed their rising costs over the next few years,” S&P said.
Meta’s rivals, however, are monetizing their data centers by leasing compute capacity, including to frontier model labs and enterprise customers, it added.
In a statement to Fortune, a Meta spokesperson said the company disagrees with S&P’s assessment of its AI monetization capabilities, pointing to “significant product and business opportunities” ahead.
“Meta is already delivering stronger engagement, better recommendations, and greater returns for businesses as a result of AI investments,” the spokesperson continued. “We believe the technology and infrastructure we’re building now will open new revenue streams, create long-term shareholder value, and bring us closer to delivering personal superintelligence to everyone.”
The bearish environment on Wall Street also comes as rising bond yields have sent borrowing costs sharply higher, the political backlash against data centers is expected to help Democrats retake Congress, and the Iran war could re-escalate after midterm elections.
Still, some bullish voices aren’t convinced the AI boom is about to go bust, and even think some retrenchment is a good thing.
According to market veteran Ed Yardeni, the AI investment boom faces financial, physical, and political constraints. If any of these constraints worsen, stocks could dive further, especially considering that AI is now a major driver of corporate earnings and GDP growth, he said in a note this week.
“However, we view these constraints as healthy for the durability of the AI boom!” he added.
For example, if the supply of capital doesn’t flow as easily, it could lead to more efficient allocation, lowering the risk of overinvestment. Yardeni expects hyperscalers to retain access to financing—thanks to their strong balance sheets—and keep spending.
In his view, it all comes down to the U.S. economy, which he said has performed remarkably well over the past few decades. Even the midterm elections are unlikely to derail it, regardless of which party controls Congress.
“Further AI jitters therefore could set up buying opportunities rather than signal the end of the boom,” he said.
This story was originally featured on Fortune.com
If California undid its ‘original sin’ and matched Texas on taxing property, it would raise more revenue than the billionaire tax, economists say
California voters will decide on Proposition 40 soon, but looming over the controversial billionaire wealth tax proposal is another state ballot measure from nearly 50 years ago.
In 1978, voters approved Proposition 13, which capped property taxes at just 1% of assessed value, limited annual hikes to 2%, and allowed reassessments to market value only after sales or substantial renovations.
It came amid a wider tax revolt leading up to Ronald Reagan’s election as president and an era of conservative governance in America. After Prop 13’s passage, the state had to find other ways to raise more revenue and relied more on taxing incomes, including capital gains.
California now has the most progressive income-tax structure in the country with a top marginal rate of 13.3%. When combined with federal income tax rates, that can send the total to more than 50%.
The top 1% of earners account for up to half of state revenue, though it also means state coffers are volatile and swing on stock market returns. At the same time, the ultra-wealthy often don’t receive much income and instead leverage assets to finance their lifestyles, avoiding some taxes.
Now enter the billionaire tax, which calls for residents worth more than $1 billion to pay a one-time tax equivalent to 5% of their assets. The union pushing Prop 40 has said the wealth tax could raise $100 billion in revenue and help offset federal cuts to health spending.
Supporters point to the AI boom and say California’s ultra-rich would still be among the world’s wealthiest. Opponents have warned about the impact the tax could have on economic growth and startups. They have also said it would force founders to sell big pieces of their companies and eventually cause the wealthy to flee to another state, resulting in a net loss of revenue over time.
If California is looking for a large source of undertaxed wealth, “a much better, bigger target lies right beneath their feet,” according to Princeton economist Owen Zidar and University of Chicago Booth School of Business economist Eric Zwick, authors of the book The Everywhere Millionaire.
“The original sin underlying California’s tax structure is Prop 13, the 1978 law that rewrote the rules of property ownership,” they wrote in a recent New York Times op-ed. “Getting rid of it would be a more efficient and durable way to make the rich pay their fair share.”
Prop 13 also contributed to rapidly increasing home values, which have risen faster than property taxes. As a result, California homeowners pay an average of just 0.8% tax on their property, Zidar and Zwick estimated, generating $100 billion in revenue in fiscal 2025.
By contrast, Texas homeowners pay 2% on average. If California matched Texas, overall residential property tax revenue would more than double, while also easing home prices by around 13%, they added.
To be sure, property taxes in Texas have stirred their own voter revolt during the midterm election season. While the state doesn’t levy income taxes, Texans complain that the overall tax burden remains onerous due to property taxes, which are levied by local governments as well.
And getting rid of Prop 13 in California wouldn’t go down well with voters either as many would suffer big increases on their property taxes. Still, California’s tax base is already narrow, and Prop 40 would make it even more top-heavy. Meanwhile, the state plans to spend $540 billion this fiscal year.
“It is simply not possible to fund the state’s needs by leaning on a few hundred people, as politically appealing as that may be,” Zidar and Zwick argued. “For California to tax wealth in a progressive fashion and meaningfully improve its fiscal position, it needs to broaden the aperture beyond billionaires to the hundreds of thousands of other very wealthy people who could easily afford to contribute more by paying a real estate tax that finally works.”
Laura Tyson, a former White House economic adviser, and Lenny Mendonca, a former chief economic adviser to California Governor Gavin Newsom, warned billionaires respond strongly to estate taxes when deciding where to live.
Already, six billionaires who would have owed about one-quarter of Prop 40’s expected revenues left the state before the proposed effective date on the wealth tax, they pointed out in a Project Syndicate column.
And since Prop 40 proposes a new, untested method of taxation based on all assets, it would fuel a “tidal wave of litigation, rendering the promised funding both speculative and unreliable,” they wrote.
“Instead of a tax on billionaires, the U.S. needs to raise taxes on the capital income that feeds wealth,” Tyson and Mendonca argued. “There are a variety of ways to do so: through higher corporate tax rates; through taxes on realized and unrealized capital gains; by closing the most egregious loopholes, like carried interest; and by increasing inheritance taxes.”
This story was originally featured on Fortune.com
China sees the yuan vs. dollar rivalry like Pepsi vs. Coke, expert says — ‘never the market leader, but serviceable in a pinch’
Heavy is the head that wears the crown, and contrary to popular belief, Beijing doesn’t want the yuan to dethrone the dollar as the world’s dominant currency, according to a top China scholar.
While China has been trying to internationalize the yuan, also known as the renminbi, it doesn’t want the obligations that come with being the global reserve currency, Zongyuan Zoe Liu, senior fellow for China studies at the Council on Foreign Relation, wrote in Foreign Affairs this month.
To truly take the dollar’s place would require open capital markets, which would threaten the Communist Party’s control over who gets capital and how, she explained. That’s as many elites rely on offshore dollar holdings as a type of insurance against domestic political persecution.
Since China actually prefers to keep the dollar on top, the real reason for promoting the yuan abroad is to prevent the U.S. from weaponizing the financial system against it, Liu added, similar to how Russia was shut out after it invaded Ukraine in 2022.
A yuan network that’s international enough, even without being dominant, could bypass any dollar-based sanctions the U.S. might impose in the future.
“Placing a respectable second, or even a consequential third, would suffice to break the dollar’s effective monopoly and constrain the United States’ ability to impose financial sanctions unilaterally, while saving China the effort and cost of sustaining currency supremacy,” she said.
“From Beijing’s perspective, the mission is to establish the renminbi as the Pepsi to the dollar’s Coca-Cola: never the market leader, but serviceable in a pinch and more than capable of keeping the dominant incumbent honest.”
(For what it’s worth, Pepsi has actually fallen further behind Coke in the cola wars and was dealt a major symbolic defeat recently when KFC said it would replace PepsiCo soft drinks with Coca-Cola’s products at its 3,700 U.S. restaurants.)
So far, China facilitates renminbi use among firms in more than a hundred countries and across a growing array of commodity markets. As of June, its Cross-Border Interbank Payment System had 1,829 participants who could send payments to more than 5,200 banks in 191 countries.
Global currency requirements
But that’s just a small fraction of what the U.S.-dominated system processes, Liu said, adding that a global currency also entails pricing, financing, risk transfer, and reinvestment.
Chinese exporters still invoice in dollars and convert them to yuan just before brining them into China. But Beijing is promoting “China prices,” or yuan benchmarks on domestic commodity exchanges and has succeeded with lithium, she noted.
The financing stage is gaining momentum too as Chinese banks have more than doubled their yuan lending to overseas borrowers between 2023 and mid-2026. The People’s Bank of China has also reached currency swap agreements with 32 foreign central banks and monetary authorities.
More modest steps have increased the yuan’s presence in risk transfer markets as state firms have begun settling cargo insurance in yuan. But reinvestment is perhaps the weakest link in the chain. A strong yuan ecosystem requires recipients to hold and reinvest their balances rather than convert them, Liu argued.
China has expanded overseas access to domestic stocks and bonds, but Beijing’s unpredictable capital controls remain a worry. In March, foreign investors held just 1.6% of the Chinese bond market versus 27% in the U.S. bond market.
“Without a far larger pool of renminbi assets that foreigners can freely buy and sell, the renminbi cannot come close to replacing the dollar,” she said.
But similar to how the “petrodollar” anchors the greenback’s role in global trade, the yuan can dominate trade in an essential good that China controls.
That could take the form of critical minerals, clean energy equipment, and electrical infrastructure used in green energy technology, Liu posited.
China has already established supremacy across much of the green supply chain and is promoting standards built around technologies Chinese-owned firms.
“If China consolidates its dominance over that industrial ecosystem, using its scale and cost advantages to prevent competitors from catching up, it could gain enough leverage to price those essential, hard-to-substitute exports in renminbi, creating the foundation for a ‘green yuan’ that might eventually replace the petrodollar,” she predicted.
Petroyuan
Meanwhile, the Iran war has raised doubts about the petrodollar, which goes back to a deal struck in 1974 when Saudi Arabia agreed to price its oil in dollars and invest surpluses in U.S. assets.
In exchange for Saudi Arabia recycling its dollars back into the U.S., Washington guaranteed the kingdom’s security, which also involved stationing troops in the region, providing advanced weapons, and ensuring free navigation in the Strait of Hormuz.
But America’s role in the Mideast looks vastly different today. Despite 40 days of punishing bombardment of Iran and round-the-clock protection of commercial ships, the U.S. still hasn’t fully restored traffic in the strait.
At the same time, Iran’s swarms of missiles and drones have inflicted significant damage on U.S. aircraft, radars and bases, while American air-defense systems have failed to completely protect Gulf allies’ critical energy infrastructure.
“Damage to Gulf economies could encourage an unwind in their foreign asset savings,” Deutsche Bank warned in March. “In this context, reports that the passage for ships through the Strait of Hormuz may be granted in exchange for oil payments in yuan should be closely followed. The conflict could be remembered as a key catalyst for erosion in petrodollar dominance, and the beginnings of the petroyuan.”
This story was originally featured on Fortune.com
Regulate the Deployment, Not Just the Model
A who-does-what plan for AI safety, built around Trusted Deployers: certified hosts that keep dangerous models guarded without ending open access.
AI has reached a new level of public interest. That may seem like a silly thing to say, as it’s been of interest for a while. But prior (broadly public) interest was about it as a tool, or about the money to be made or lost, or about an effect on jobs.
The new interest demands policy. It demands action. The new interest is connected with risks I consider real. Being real doesn’t make these risks certain. But being real, even with uncertainty, is reason enough to act.
I’ve written lately about risks, cybersecurity risks, and loss-of-control risks. These tried to explain complex topics, starting from the initial premise. Recommendations follow the explanation. This piece will be a bit different. Its focus is upon what I think we should be doing. You might consider it a policy piece. Policy in a completed form has to be a bit broad. There’s always more than one thing going on. As such, I’ll lay out my outline first, and then offer below the extra detail, background and context that I think is important to understanding each point in that outline.
If you’re scanning the outline and see something you disagree with, I’d suggest you continue on, because the outline is, by design, abbreviated, and not fully descriptive.
This isn’t the only effort of this type. Institute for Progress (IFP) and AI Futures are two well known efforts. What distinguishes this most of all is thinking more deeply about how models are deployed. This is a background element that’s somewhat ignored, because it’s not as novel and is not as visible to users. But it’s absolutely critical. I don’t expect this plan would replace plans from well known organizations, but by illustrating a gap, I hope to inspire improvement amongst those plans.
Who and What
This outline is categorized by who would be expected to do what. In some cases, someone is expected to do it, but there’s also another role, of providing the incentives that improve the expectation they will.
Enforcement is deliberately vague. It depends on jurisdiction, has to fit existing legal frameworks, and will need to be worked out, and negotiated, through the political and legal systems. It would feel like I was putting the cart before the horse to even guess at how that would take shape at national and international levels.
Who does what. Model weights above the threshold reach users only through certified trusted deployers. Independent evaluators, certified by a regulator with legal authority, test the models and audit the deployers.
Layer 1: Infrastructure and Distribution
Trusted deployers
Who are they? A new designation that existing and new organizations can be certified as. Expect major cloud providers (AWS, Azure, Google), AI labs (for self-owned) and neoclouds to join immediately. The plumbing largely already exists, but needs standards and certification.
Revenue model? Charging for compute and operational services as a utility function. Or, an internal unit (for a lab, government, or large organization)
Core Responsibilities
- Develop secure environments to host models that receive an independent AI Deployment Safety Level (AIDSL) rating. Example, but not comprehensive list of requirements:
- Comprehensive monitoring, telemetry, and auditing (including multi-turn agent pattern detection).
- Resistant to external cyberattacks and insider threats.
- Resistant to internal cyberattacks. At minimum red-teamed by deployed model and a second highly capable model.
- Guardrails applied by default to all deployments which prevent access of dangerous capabilities.
- Strict controls for any guardrail exceptions as used for “limited” access deployments. For example, accessing users must be known directly, or an evidence chain validated using only trusted organizations.
- Understand the sensitivity of any model deployed. Never deploy a model in an environment with an AIDSL rating insufficient to the combination of deployment type and model sensitivity.
- Use internal qualification for small custom models and Independent Evaluator data for larger/frontier models.
- Models under training assumed to be at least X% more capable than the best model trained to date at the given scale. If the training scale is unprecedented, increase X by a size factor. (X should be established by independent evaluators)
- Perform or validate Know Your Customer (KYC) verification for all model providers, for Independent Evaluators and anyone with access to deployments with guardrail exceptions.
- Provide access to Independent Evaluators to confirm secure environments meet standards
<
Against SURA
The AI Futures Project[1] distinguishes several different plans for navigating the AI transition, which I'll lump into two broader categories:
Plan A (my combination of their Plan A/S/Other) involves working for a global pause on AI development, hopefully lasting decades. During this pause, we either research preparatory technologies like alignment or human intelligence amplification, or take actions to "fortify" the pause so that it can last indefinitely.
Plan C (my combination of their Plan B/C+/C/sort of D)[2] is for a world where we fail at getting a global pause, and race dynamics continue to dominate. If the leading AI companies are alignment-pilled, they coordinate a mutual pause among themselves, lasting the length of their joint lead over the leading non-alignment-pilled AI company. Then they "burn the lead" researching alignment, and release hopefully-aligned superintelligence at the last possible moment before the racing non-alignment-pilled AI companies beat them. The exact calculus changes depending on who is in the lead when - is there one alignment-pilled AI company with a six-month lead in front of everyone else? Are there two alignment-pilled AI companies with a joint two-month lead in front of their nearest competitor? Etc. The point is, if it's impossible to stop the race, that six-month or two-month lead is the amount of time they have to prepare, and they can spend it more or less wisely.[3] The point of burning their lead at the last moment is that they can do alignment research with very advanced, right-on-the-brink-of-superintelligence AIs. This is a dangerous plan, and there would be many subtle points they would have to get right, but in a world where nobody can get a global pause in time, it's hard to think of a better one.
Plan A benefits from raising mass awareness, educating politicians, and preparing the verification technology that could enforce a pause treaty. Plan C benefits from alignment-pilling companies, having the alignment-pilled companies be as far in the lead as possible, making sure the alignment-pilled companies will burn their lead when the time comes, and giving them as good alignment teams as possible (so that burning the lead produces good results).
From the beginning of AI safety as a community to about 2023, the conventional wisdom was that our chances of Plan A were slim, and we should be preparing for Plan C. From about 2023 onward, as the general population woke up to the implications of ChatGPT, we rightly concluded that the chance of getting Plan A was better than expected, and we should invest more resources into trying to make it work. During both periods, the natural strategy was to work as hard as we can to get Plan A, where chances of survival are highest, but to be prepared to execute Plan C competently as a backup if that didn't pan out.
Now I think the pendulum has swung too far. New members of the community are unaware that Plan C was ever considered a possibility, or think it's some kind of accelerationist plot to distract us from Plan A. Older members heap scorn on it and act like everybody knows it's been debunked. Most concerningly, people propose burning our chances of executing Plan C in exchange for infinitesimal increased chances of getting Plan A, or even for no increased chance of Plan A at all.
I'll lump these ideas together under the heading of "SURA" - the idea that the only ethical course is for people/companies/countries to Stop Unilaterally + Raise Awareness, with no plan for what happens if race dynamics continue anyway. I interpret the planks of SURA's platform as:
- Individual alignment-pilled AI companies are morally obligated to stop unilaterally, regardless of whether they expect race dynamics to continue in their absence.
- Individual alignment-pilled employees at these companies should quit, regardless of whether they're in capabilities or alignment roles, and regardless of whether they think their company is morally better than its competitors.
- Individual alignment-pilled countries (for example, a hypothetical future alignment-pilled US) should ban or heavily restrict AI progress unilaterally, regardless of whether they expect other countries (for example, China) to continue racing in their absence.
All of these planks are net negative in a world where we end up needing Plan C:
- If we demand that leading alignment-pilled companies quit the race unilaterally, then we ensure that non-alignment-pilled companies end up in the lead.
- If we demand that alignment-pilled company employees quit their jobs, this makes companies less alignment-pilled and less prepared to research alignment if they need to. It also is likely to decrease the lead that alignment-pilled companies have over the nearest non-alignment pilled company.
- If we demand that the US cease AI research unilaterally (ie without a mutual pause treaty with China), we ensure either that China gets superintelligence first, [4]or that the US has to restart with zero lead when it realizes its mistakes.
SURA's continuing popularity despite these downsides comes from a set of questionable assumptions:
- Alignment is doomed (without some extraordinary extra advantage like decades of time, or superhuman geniuses). Therefore, Plan C is also doomed - or, at best, it raises our chances of survival from something like 0.00001% to something like 0.00002%. Therefore, there is no reason to optimize for Plan C.
- Racing to build superhuman AI is deontologically evil, and it's prohibited for good people to do it regardless of whether or not it may have good consequences.
- Racing burns timeline, so anyone unilaterally quitting the race would give us more timeline, which we could use for good things.
- If lots of people quit, that will raise awareness, which could be helpful for getting a global pause.
Sanity Checks for Sanity Checks for Sparse Autoencoders
This post will probably not be very informative to readers who don't know what an SAE (Sparse Autoencoder) is.
An interpretability researcher once told me, "SAEs can never fail, they can only be failed." But recently, SAEs have been suffering failure after failure, and as a result, the field has grown increasingly pessimistic about SAE work. Korznikov et al.'s Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines? purports to be an example of such failures.
However, I have some disagreements with its methodology. The paper presents a synthetic setup with known ground truth features, and claims that SAEs fail to recover features in the "simplest toy setting with fully aligned hyperparameters." In this post, I argue that their setting is not forgiving to SAEs, as the authors imply, but is instead actively hostile in ways that real settings are not.
Introduction
Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines? asks two important questions:
- Can SAEs actually recover real features? Hypothetically, if we knew what the "true" features of the data were, would an SAE be able to identify them?
- Do trained SAEs actually do better than completely random SAEs?
This post will be focused on their answer to the first question.
The authors train an SAE on a synthetic toy model, with known ground-truth features, and according to their abstract, SAEs only recover 9% of true features. On the face of it, this seems pretty damning!
Now, earlier toy model experiments did show that SAEs could recover ground truth features, but the authors argue this is because they used a small expansion factor, where there were only twice as many "true" features as there were model dimensions, and that this is unrealistic. Their setup uses an expansion factor of 32, which matches real-world use cases much more closely.
So, how does their setup work?
They don't actually construct a synthetic neural network; instead, they create synthetic activations directly. First, they create their feature vectors, by sampling 3200 vectors from the unit sphere in 100-dimensional space. To create an activation, they just select about 20 feature vectors at random, scale each active feature by a magnitude drawn from LogNormal(0, 0.25),[1] and sum the scaled vectors.
They test two different regimes, one where each feature is equally likely to be active, and another where feature activation frequency has a long tail, so some features fire more often than others. (We'll focus on the constant regime for the most part)
They match their SAE to the setup. There are about 20 expected active features, so they target L0 = 20. There are 3200 true features, so they create 3200 SAE latents. Can the SAE, when given every possible advantage, recover the true features?
These results are, again, pretty damning! They claim that the SAE achieves high reconstruction accuracy while only recovering 3/3200 true features—personally, I disagree that 0.67 explained variance is "high reconstruction accuracy".[2] However, the SAE clearly only does slightly better than random guesswork at recovering the true features.
Now, one obvious objection is that L0/d_model here is 0.2, which is unusually dense. I'd expect there to be a lot of interference here, and in the real world L0/d_model is probably going to be closer to 0.05. Also, 100 dimensions is just not a lot, and maybe you can't really get blessing of dimensionality from it like you would in real models.
A less obvious objection is that if you scroll all the way down to Appendix B, they describe some other SAE architectures they used for this task, like TopK and Matryoshka-on-BatchTopK.
In contrast to BatchTopK, TopK recovers 99.9% of features.[3]
So, I did what anyone would do, and started replicating the paper. They didn't actually include all the hyperparameters they used,[4] so I made reasonable assumptions.
I was able to replicate this result! TopK recovered 99.6% of features,[5] and the other architectures failed miserably.[6]
Why does BatchTopK fail?
This is kind of strange, if you think about it. TopK and BatchTopK are the same architecture, with the only difference being the selection rule.[7] Such a dramatic gap is really weird.
Peeking a bit closer at my training run, in every batch, most latents never fired. That means a small group of monopolist latents were dominating every time. By step 5,000, the same ~6% of latents took over, and they remained in charge for the remaining 295,000 steps. In comparison, with TopK, after the first 20k steps, >99% of latents activated on every batch.
This is the standard dead latents problem: There's no gradient on latents that aren't selected, so once they fall behind in performance it's hard for them to improve. A small group of generalist latents dominate before specialists have the chance to form. The standard fix, of course, is an auxiliary loss, AuxK. But even with AuxK, the dead latent fraction barely budges. I tried sweep
Conclusions of my DIY Inkhaven
Epistemic status: describing my own experiment results.
AI use: review, research (stats), fact-checking, grammar & light phrasing.
I applied to the next Inkhaven cohort and was rejected. This was surprising and very unfortunate. While writing my application and answering why they should accept me, I felt like I'd one-shotted myself into writing. My goals were:
- Run an experiment to verify my hypothesis that I couldn't write well. I also implicitly assumed that I didn't have anything useful to say or to add, so no one would read what I write anyway.
- I wanted to get better at thinking, and to me writing is a better form of thinking.
- I wanted to use language more eloquently even outside of writing.
- Writing is meditative and deep work, so I wanted to become more productive by writing more.
I was sad to be rejected, and I still don't know why. Fortunately, I have my agency, so I just decided to do my own Inkhaven instead. Obviously, I wouldn't stay at Lighthaven with a bunch of other people or receive guidance from people like Scott Alexander and Aella. Also, all of this is happening as part of my normal life, and I have other stuff to do as well.
In reality, I started writing even before I submitted my application, because I needed samples of my writing and I wanted to check whether I actually enjoyed doing it. After submitting my application, it felt cool to just write until the start of the cohort. I could compare about 30 days of writing on my own with a month at the cohort with guidance. Also, I would have more writing to compare before and after. After the rejection, there was a temptation to just give up, but I decided to continue.
Including this post on October 31, my totals are:
- 21 posts on LessWrong: about 19,000 words and 109,000 characters.
- Around a dozen posts on my private blog: roughly 6,000 words and 30,000 characters.
The earlier 20 LessWrong posts have 740 karma across 441 votes and 95 comments as of October 31. These word and character counts exclude footnotes; characters include spaces. One additional LessWrong submission was automatically rejected after an LLM detector flagged it.
I started by writing on LessWrong about stuff I found interesting. I would just get some random idea that I found interesting enough to explore. I did try to optimize for more karma.
Writing 500 words for LW, assuming you want other people to like it, felt quite hard. I could obviously bumble on for 500 words pretty easily and just publish that. I still had to relax my requirements a bit - for example, I never checked whether someone had written on the topic before. I knew that if I did, I would feel like I had nothing to add, so I just wrote, ignoring the fact that this could be a duplicate.
My first hit was "A fascinating case of an LLM prompt-injecting its own user". I saw this a long time ago and was extremely impressed and fascinated. Originally I assumed that everyone had seen it too. Apparently, something didn't have to be new to me to be useful to someone else. But since I had to write something, I decided to write about this. This was fairly early in the process (day 5, October 6) and definitely helped me feel less like "everyone else is so cool, you have nothing to add or say, and no one will read whatever you write."
My post with the most karma is "Quantity vs quality in pottery is a lie", which as of now has 176 karma. I literally read that anecdote in a book. I feel like, since I was looking for ideas to write about, I got curious about their experimental setup. To my surprise, it wasn't a scientific study, but an anecdote about a photography class that had been rewritten as pottery. My conclusion from this was that people at LessWrong like becoming less wrong. But this is a hard niche to reliably produce content in.
Two other highly upvoted posts are "Gluten-free water" and "I can't believe it's not BUTTER!".
In both cases, I noticed something peculiar in my life, dug deeper into it, and found more examples of similar things.
My "worst" post was "When bureaucracy gives you something it shouldn't". To be honest, I was writing it late in the day in a hurry just to get something out. Originally, it was downvoted below my personal strong upvote, but with time it got to +1 above my own strong upvote. I have no clue why.
I didn't engage in the comments much, since it didn't count toward my original proxy goal of publishing 500 words a day.
At some point, I started running out of steam, and overall it felt cumbersome. Writing for LW is quite hard. I applied high standards, and it took time. Eventually I switched to my private blog, where I would cover more personal topics and the posts didn't have to be as rigorous.
This made it easier and more fun. I mostly wrote my opinions or stories from my life, and it was easy and pleasant to write. This didn't feel like work and was enjoyable. I suspect these topics are just closer to my heart than the topics I would be writing about on LW (but I now catch myself realizing that I actually have no clue whether that's true - perhaps my personal reflections would be enjoyed on LW too). I feel like I will likely continue writing on my private blog (e.g., I now often catch myself thinking that this situation or feeling is interesting and I want to write about it to preserve it or to signal to others) and on LW very occasionally when there is something really burning inside me that I want to write about.
In terms of my original goals
Run an experiment to verify my hypothesis that I couldn't write well.
"Well" is unclear. I've got 176 karma on a post, and my posts have around 700 karma between the