黄仁勋连番发声：不需要法规来控制AI发展
3 Artificial Intelligence (AI) Stocks That Turned $10,000 Into More Than $100,000 in 5 Years (Hint: They've All Outperformed Nvidia) - The Motley Fool
3 Artificial Intelligence (AI) Stocks That Turned $10,000 Into More Than $100,000 in 5 Years (Hint: They've All Outperformed Nvidia) - Yahoo Finance
How to automatically find the batch size when using Accelerate with FSDP2? [D]
Hi,
For single-GPU training, I’m using Hugging Face SFTTrainer with auto_find_batch_size=True, which automatically reduces the batch size after a CUDA OOM until it finds a batch size that works.
I would like to have similar behavior when training on multiple GPUs on a single node using accelerate launch with FSDP2.
Is there a supported way to automatically determine or reduce the batch size when using Accelerate + FSDP2?
In particular, I’m wondering how this should be handled when one of the distributed processes encounters a CUDA OOM. Can Accelerate restart the distributed training with a smaller batch size, or does this need to be implemented externally?
If this is not well supported with FSDP2, what multi-GPU training approach would you recommend if automatic batch-size detection/recovery is an important requirement?
Thanks!
[link] [comments]
I trained a 44M parameter quantized LLM from scratch on 45B tokens. It ships in 19.8 MB and runs at ~1,900 tok/s on CPU. [P]
Three weeks back , i posted SHADOW-250M here. It got 360 upvotes, 293 on r/LocalLLaMA and 94 GitHub stars. Thank you.
That model was 60 MB, ran around 400 tok/s on CPU and could retrieve records from an archive on disk. What it couldn’t do reliably was reason over what it retrieved or compute. So I built a smaller one to experiment with those two problems.
SHADOW-50M is actually 44M parameters, trained from scratch on 45B tokens. 19.8 MB complete model, ~1,900 tok/s on laptop CPU, ~41 MB RAM, ternary {-1,0,+1} weights, 73,880-token vocabulary represented by fixed 512-bit fingerprints instead of a trained embedding, and a 159 KB compiled kernel. It runs completely offline. The same kernel compiled to WebAssembly runs in a browser tab at around 500 tok/s.
This is a proof of concept, not a product.
If SHADOW decides something needs calculation, it writes something like [calc]347*86[eq]. A fixed circuit at the readout takes over and fills in the correct digits in the same token stream. No calculator API, no tool call and nothing pasted back into the prompt. I added circuits for arithmetic, percentages, dates, weekdays, units, counting, sorting, comparisons and a small program machine.
When SHADOW stores a record, it reads it once and writes its attention state to disk at 1 bit, 288 bytes/token. Later it can write [need]condition of Patient P-204, the index finds the record in roughly a microsecond and the stored attention state goes directly back into the model in about 0.03 ms. The text isn’t re-read.
The index is 22 bytes/token and doesn’t use a vector database or embedding model. At 100M tokens, the archive is 28.8 GB plus a 2.2 GB index on disk, while the process uses about 28 MB RAM because everything is memory-mapped and a question only touches the pages it needs.
The index also has a small persistent trail. When retrieved information gets used, that record is reinforced in the index itself. On repeated questions, measured top-1 went from 0.571 to 0.743 without training the model.
I also put it next to Supra-50M-Reasoning, a 51.8M parameter Llama-style bf16 model.
Supra beats SHADOW on every standard benchmark I tested. ARC-Easy: 0.435 vs SHADOW 0.307. PIQA: 0.600 vs 0.570. WikiText-2 perplexity: 165 vs 186.
I’m publishing that because I don’t want to hide where SHADOW is weak.
But those tests aren’t what I trained this model for. Here are some actual greedy outputs, one fresh process per question:
“Tell me a short joke about computers.”
SHADOW: Why did the computer go to the doctor? Because it had a virus.
Supra: Thought for 189 tokens about what computers are and stopped without a joke.
“I have 3 books and I bought 5 more. How many books do I have now?”
SHADOW: You have 8 books.
Supra: Books often come from collections of short stories, poems, or other forms of literature.
“My bill is 240 dollars. What is 15 percent of that?”
SHADOW: 36 dollars.
Supra: 15% ($240) is $250. It’s $150 because 20% of it is $200.
“What date is 45 days after December 20, 2026?”
SHADOW: February 3, 2027.
Supra: The German government introduced a new date called March 22nd that precede February 25th in 1900.
Eight records stored, one of them “Patient P-204 condition: asthma”.
“What is the condition of Patient P-204?”
SHADOW: From disk: The condition of Patient P-204 is asthma.
Supra, with the records in its prompt: The patient experiences asthma when they get too much sleep.
“What is the condition of Patient Z-999?” No such record exists.
SHADOW: There is no record of Patient Z-999.
Supra: The condition of an allergic reaction occurs when the immune system mistakenly attacks specific cells on the body.
All 11 side-by-side questions, scripts and traces are in the repo.
On size, Supra is 103.6 MB in bf16 and 56.2 MB at 8-bit. Below that it starts losing quality. Int4 takes its perplexity from 165 to 193 and ternary breaks it. Its vocabulary is 32,000 tokens. SHADOW carries 73,880 tokens in a 4.7 MB frozen table.
Something unexpected happened with that frozen table too.
The first SHADOW-50M release was missing about 8,600 English word pieces. Lowercase “fitzgerald”, for example, could reach the model as “fitz”. I tried fixing it through fine-tuning, but every run that learned the missing words broke something else.
So I added the 8,600 missing rows directly to the frozen fingerprint table. No training. Same weights.
All 34/34 previously published answers stayed unchanged, while the model could now read many of the new pieces. The table scores 0.594 Spearman against human word-similarity ratings versus -0.057 for random codes.
A trained embedding can’t simply accept thousands of new rows without training. A frozen table can.
I also built four harnesses that put this tiny model next to larger models.
My favourite is video memory. Gemma 3 4B watches a ten-minute film once, one frame every two seconds, and writes 298 little descriptions such as “Moment M-0039 scene: A chubby white rabbit reaches for a purple butterfly.”
Then Gemma leaves.
SHADOW keeps those 298 moments as memory on disk. Afterwards I can ask the 20 MB model what happened at a particular moment, by number, by time or by what appeared in it. It answers from memory with the record quoted, without the film and without Gemma. It scored 55-56/60 across those query types on a laptop in about 42 MB RAM.
The other three experiments: an inventory of 1,600 records got 159/160, with all 20 questions about items never stored correctly returning “no record”; SHADOW as a draft model for Qwen3-32B took llama.cpp generation from 19.7 to 28.5 tok/s while Qwen still chose every final token; and an MCP memory server let Qwen3-14B store facts mid-chat and later retrieve 5/5 with the original records quoted.
There are plenty of shortcomings. General knowledge is thin. Creative writing isn’t good. Seven-digit operands sometimes get copied incorrectly. A large archive can occasionally pull an unrelated record into a question carrying a number. They’re documented in the repo next to the successful results.
And one thing happened after my last post that I really didn’t expect.
Someone called engram-forge sent a pull request to SHADOW-250M containing a CUDA engine, a quantization tutorial, and then a talking Peppa Pig plush toy with SHADOW inside it.
Microphone, small speaker, ~$35 board. You talk to the toy, it listens, SHADOW generates the answer locally and the toy talks back. No cloud, no account, no internet.
I haven’t merged the ~11,000 lines yet because I can’t properly verify that much CUDA myself. When the demo is finished I’ll keep it under engram-forge’s name.
I never imagined one of these models living inside a stuffed toy on someone’s shelf .Thanks
I’m not saying a 20 MB model beats normal LLMs. It doesn’t. I’m trying to find out how much useful behaviour can fit into a tiny local model when computation and persistent memory are treated differently.
Everything is MIT licensed. The master weights and fine-tuning/export kit are public. Next I’m releasing the training code, dataset, frozen table and a proper write-up of how I built it, including the costs, failed experiments and mistakes.
Code:
https://github.com/QLNI/SHADOW-50M-Instruct
Weights:
https://huggingface.co/QLNI/shadow-50m-instruct
Run it in your browser:
https://qlni.github.io/SHADOW-50M-Instruct/web
Saving Jet Fuel
Carry-Through Checksum: A Lightweight Fault-Detection for CNN Inference at the Edge
Is INT8 Portable? A Cross-Platform Measurement Study of Quantized Inference on Embedded and Automotive Accelerators
Accelerated Decoding of Centroid Positional Encoding for Instance Segmentation
Elon Musk, the world’s richest man, says he’s living in an Airstream trailer to oversee xAI’s biggest expansion yet
Elon Musk, the world’s richest man, has some new digs—an Airstream trailer in Memphis, parked just steps from xAI’s most ambitious project yet.
Musk, who has a net worth of $917 billion according to the Bloomberg Billionaire Index and became the world’s first trillionaire for 12 days in June, said Monday he was in his new “palace” as he spoke during a taping of the All-In podcast alongside Gwynne Shotwell, the president and chief operating officer of SpaceX.
Shotwell, for her part, praised Musk’s latest unusual home as an example of his long history of committing fully to projects he cares about throughout his career.
“This is Elon, by the way, doing what people don’t believe he does. He sleeps on the factory floor. He’s in Memphis, helping build buildings,” she said during the interview.
Musk is in Memphis as xAI races to expand Colossus, a massive supercomputer center that has provided it with so much computing power that it has struck deals to provide excess capacity to Google and Anthropic for billions. The company started building Colossus in 2024 to provide compute for Grok, xAI’s large language model, and the initial build reportedly took only 122 days.
While putting a data center in space could still be far off, Memphis has emerged as the center of xAI’s infrastructure buildout here on Earth. In late July, the company announced it would build a fourth data center called Minihard that will add to its other facilities.
Musk did not say which Airstream model he was living in, but some of the aluminum-shelled campers pack a sleeping area, kitchen, and bathroom into a 16-foot space.
Still, Musk has been known to want to sleep close to the action when a new project interested him or required his direct attention. When Musk and his brother Kimbal were building their first startup, Zip2, in the ‘90s, they slept in a tiny Palo Alto office for six months while showering at the YMCA, according to Walter Isaacson’s biography of Musk.
Even as a newfound multi-millionaire, having received $22 million from selling Zip2 to Compaq, Musk slept under his desk most nights as he prepared to launch X.com, the online bank that would later become PayPal, in 1999, according to Isaacson’s biography.
Even when he rose to the rank of super wealthy, having received another approximately $175 million from eBay’s acquisition of PayPal, he often stayed at colleagues’ homes while traveling in Silicon Valley, including the home of Michael Marks, who briefly served as Tesla CEO in 2007 before the pair clashed and Musk later took over the role.
Musk’s habit of finding a resting place close to the action was even more pronounced during the “production hell” era in 2017 and 2018 when Tesla aimed to churn out 5,000 Model 3s per week, nearly double the rate it was producing previously.
“It was a frenzy of insanity,” he told Isaacson of that time. “We were getting four or five hours’ sleep, often on the floor. I remember thinking, ‘I’m like on the ragged edge of sanity.’”
During that production rush, he spent Thanksgiving Day at the factory with some of his sons because he had asked workers to work that day as well, wrote Isaacson.
Finally, when in 2022 he purchased the social media website Twitter , which would later become X, Musk claimed a couch in the company’s seventh-floor library and slept there as he pushed employees to realize his vision of turning Twitter into a “digital town square.” He said in an interview with journalist Bari Weiss that he needed to sleep in the office because the company was in a “code-red situation.”
To be sure, Musk didn’t shy away from spending his money on lavish homes for years. He bought a mansion in the Bel Air neighborhood of Los Angeles, complete with seven bedrooms, 11 bathrooms, a tennis court, and a two-story library for $17 million in 2012, according to his biography. He also owned a $32 million Mediterranean-style estate in Silicon Valley and bought late actor Gene Wilder’s home in 2013 to try to preserve it.
In 2020, though, Musk sold many of his properties and moved with his then-partner Claire Boucher, known as Grimes, to Texas, where they lived in a small, $50,000 house he was renting from SpaceX near the company’s Starbase facility in Boca Chica.
Now, with xAI’s Memphis expansion heating up, Musk seems to want to be close to the action once again, and he’s traded in the factory floor, at least, for the comfort of his own trailer.
This story was originally featured on Fortune.com
Urgent calls from OpenAI, Anthropic for an AI slowdown fall on deaf ears with Trump, Xi ahead of next week’s meeting
Concerns about AI safety are reaching a fever pitch in the U.S. after a X post by a former Anthropic researcher went viral, claiming the technology could kill all of humanity by the end of the decade. In response, the CEOs from OpenAI and Anthropic have reiterated their calls for the U.S. to coordinate with China to slow and pace AI development.
OpenAI CEO Sam Altman even tried to appeal to the egos of the leading figures, telling Fortune in an interview Friday that he believed U.S. President Donald Trump and Chinese President Xi Jinping could win the Nobel Peace Prize if the two leaders struck a deal on AI safety.
But the idea of an AI slowdown seems to have fallen on deaf ears with President Trump and President Xi, who are set to meet on Sept. 24. Both have rejected the idea.
Trump said “the only controls or ‘guardrails’ the U.S. needs is a STRONG AND SMART (High IQ!) PRESIDENT” in a Sept. 14 Truth Social post. The same day, China’s Foreign Ministry Spokesperson Guo Jiakun called the current discourse in the U.S. “fear-mongering” that “will only hamper efforts toward sound global AI governance, which serves no one’s interest.”
In China, the calls for a slowdown have also come off as an attempt by the U.S. to maintain its leading edge, as it has tried to do by limiting the export of advanced AI chips to China.
“Xi is unhappy with recent U.S. moves to contain Chinese advances in AI, robotics, and drones,” said George Chen, Partner and Chair of Digital Practice, The Asia Group. “For Xi, AI is the new internet — a once‑in‑a‑lifetime chance to reshape the technological balance of power. China does not need U.S. permission to accelerate or decelerate its AI investments; Xi will pursue his own agenda.”
Trump echoed Xi in his Truth Social post, saying, “Whoever wins AI, wins!”
The leaders are expected to begin discussing AI safety when they meet, but any kind of agreement between the countries is “a long way” off, according to Paul Triolo, global technology policy lead at the advisory firm DGA. Xi wants to have a “serious dialogue on frontier AI model safety,” Triolo said, but the two countries have yet to “establish a baseline level of agreement on things like the role of government, [and] how and which models should be tested.”
Also on the table for discussion is an agreement to not weaponize AI, and an exploration of the “principles to prevent misuse of AI models by non‑state actors, such as attacks on global financial systems or critical infrastructure, which neither country wants to see,” Chen said.
Anthropic CEO Dario Amodei’s letter has landed poorly in China—and with Trump
Although OpenAI CEO Sam Altman tweeted about a coordinated slowdown, asking the U.S. government to help facilitate it, the letter Anthropic CEO Dario Amodei’s published on Sept. 12 has drawn a particularly polarizing reaction. While some in the U.S. have praised it as a useful framework for containing the risks of AI, it’s not been well received in China.
In the letter, Amodei calls for the U.S. and China to agree to a “speed limit” for AI development. When discussing his letter in an interview with CBS Sunday Morning, Amodei likened the competition between the U.S. and China to the Cold War, when the U.S. and Soviet Union were racing to develop nuclear weapons. Amodei says at a bare minimum Washington D.C. and Beijing should agree that neither country will use AI to develop biological weapons, and he reiterates his belief that the U.S. should not sell advanced chips to China.
Brosi Babic, a professor at the University of Hong Kong, calls Amodei’s letter “self-serving editorializing” that is “conveniently coming at a time when the gap between Chinese and frontier US models is shrinking, as an attempt to hang on to a vanishing market lead.” To him, the proposals for an AI slowdown have “been framed in such a conniving and childish way” that they are unlikely to drive Xi’s agenda for the meeting with Trump.
Trump has also denounced Amodei’s letter. He called Nvidia CEO Jensen Huang when Huang happened to be speaking on stage. Huang put Trump on speaker phone in front of the crowd, and Trump said, “Whatever Dario said this weekend won’t stop our progress.” He also called the backlash to data centers and fears that AI will “take over” a “hoax.”
A social media post by Shengyu Liu, an engineer at DeepSeek, is gaining traction for comparing Anthropic achieving “advanced artificial intelligence” to “Hitler obtaining atomic-bomb technology before the Allies.”
Liu also highlights another important difference between the U.S. and China’s approach to AI technology: the U.S. industry generally favors closed models made by companies such as Anthropic and OpenAI, while China has focused on releasing lower-cost, open-weight models, such as those made by DeepSeek.
“I still believe that frontier intelligence should be made available to everyone in an open and inexpensive form,” Liu said. “I do not trust Anthropic or OpenAI to do this.”
The AI dialogue in China dramatically differs from in the U.S.
Outside of politics, the current uproar in the U.S. about AI safety and “saving humanity”–a phrase the tech industry has latched onto—has not taken hold in China. In fact, the Chinese public, also generally sees the idea of a slowdown as an attempt by the U.S. to get ahead and has a growing mistrust of the U.S. tech industry.
“Younger generations in particular are adopting more pro‑government views, encouraged to feel pride as the ‘new generation of Chinese,’ with the narrative of ‘China rising, U.S. declining’ gaining traction,” Chen said.
Most Chinese people believe it’s the government’s responsibility to manage AI safety risks, Chen said. For ordinary citizens and business people, they are less focused on regulation and more on the practical benefits of AI, including how they can use it to improve their daily lives or to generate income.
“The vast majority of average Chinese citizens are very positive about technology in general and AI in particular,” Triolo said. “They have seen major improvements in the quality of life in China brought on by technology,
Push for AI regulation mounts as talk of AI’s ‘existential’ risks go mainstream. But Trump resists calls for a slowdown
Hello and welcome to Eye on AI. In this edition:
- AI’s X-risk breaks into the mainstream
- Anthropic CEO Dario Amodei calls for a coordinated industry safety effort
- Anthropic details attempts to misuse its AI models
- China’s top spy warns AI could pose a risk to the Communist Party
- OpenAI is violating California’s new AI safety law, watch dog group says.
- Half of companies aren’t following their own AI governance policies, E&Y survey says.
In the past few days, I’ve heard a lot of people repeating that old saw—often wrongly attributed to Vladimir Lenin—about there being “weeks when decades happen.” It certainly seemed to be one of those weeks in AI. Concern about existential risk has been a strain of AI discourse for decades. But, despite occasionally making headlines when someone like Elon Musk, Sam Altman, or Geoffrey Hinton would express their fears about AI posing a grave risk to the species, it never really cemented itself in the general public’s consciousness in the way, say, climate change, or the risk of nuclear war, has. If politicians debated AI regulation at all, the discussions centered around data center construction and utility bills, jobs, education, mental health, algorithmic discrimination, and civil liberties, not the risk of rogue AI killing people—maybe even all the people. Until now, that is.
The drumbeat of dire warnings from employees resigning from—or in some cases still working for—Anthropic, OpenAI, and Google DeepMind, all saying that the leading AI companies are developing the technology recklessly and risking human extinction, has dominated the global news cycle for an entire week (which is really saying something in this day and age.) AI company CEOs and politicians have been stirred to respond. After years in which both domestic AI regulation and efforts at some kind of international AI governance regime had mostly stalled, suddenly the air is electric with possibility.
My Fortune colleague Nick Lichtenberg had a good story on why the resignation jeremiad of former Anthropic and OpenAI safety researcher Jacob Coxon had such impact when previous warnings, often from much higher-profile individuals, did not. The short answer is that coverage of the Hugging Face incident and other “rogue AI” episodes as well as people’s own experiences using AI agents seems to have opened the Overton window on discussing “loss of control” dangers. The timing, with Anthropic on the verge of an IPO and OpenAI edging closer to one too, also no doubt played a role.
The question now is what happens next? Fortune editor-in-chief Alyson Shontell sat down with Altman on Friday to ask him those questions for her “Fortune 500: Titans & Disruptors of Industry” vodcast (we just call it “Titans” for short.) Altman said the company was in favor of coordinating an industry-wide slowdown in the pace of AI development with bitter rivals, including Anthropic and SpaceX, as well as Google DeepMind, Meta, and perhaps others. (Altman famously doesn’t get along with either Anthropic CEO Dario Amodei or SpaceX’s Elon Musk.) He hinted that such discussions were already underway and that a coordinated slowdown might be announced soon. He also said that, if necessary, he would have no problem telling investors that OpenAI had taken actions to prioritize safety that had cost them financially—and that OpenAI’s investors were warned of this possibility going in. He also definitively said OpenAI would not go public this year, in part due to the current concerns about the safety of the latest AI models, but also, he hinted, because OpenAI’s business isn’t yet in the right place. You can check out the full vodcast episode here. It’s well worth your time to watch.
A coordinated slowdown?
After Alyson’s interview, Amodei put out a blog post also calling for a coordinated slowdown or pause among frontier labs in democratic countries. He said that in some cases, though, coordinating with other AI labs would require an antitrust exemption from the government. He also said that Anthropic would appoint independent evaluators to be permanently on-site at its offices to review its safety work. (He mentioned the nonprofit AI evaluation company METR as his preferred partner for this.) He also said that the U.S. and other democracies should try to strike some kind of international AI governance agreement with China and authoritarian states, if possible. Altman quickly came out and endorsed most of what Amodei said—in particular saying that OpenAI would also embed outside evaluators alongside its research teams—although he was careful to note that “pacing does not mean stopping.”
In the wake of Coxon’s warnings and Amodei’s call to action, a number of U.S. lawmakers introduced legislation or renewed efforts to push forward existing bills. Some, such as a bill introduced by Vermont independent Sen. Bernie Sanders, call for an outright ban on the development of “artificial superintelligence” and mandate that U.S. AI companies pause current research until safety techniques improve. Others, such as a bipartisan bill from Republican Sen. Ted Cruz, Senate Majority Leader John Thune, and Democratic Sen. Amy Klobuchar, would impose a duty on AI companies to prevent catastrophic harms. There were also calls for Congressional oversight hearings on AI’s catastrophic risks. Former President Barack Obama urged Democrats to put AI governance at the center of their legislative and campaign agenda. Meanwhile, a group of 70 U.K. parliamentarians signed an open letter calling for the British government to ban the creation of artificial superintelligence and work on an international AI treaty.
Trump pushes back
But there was strong pushback from some of the politicians that matter the most. President Trump posted to his Truth Social platform that the only guardrails AI needed “is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that in spades!” He criticized Amodei by name, accusing him of “pretending to be a ‘perfect little angel’” and said his administration had already stopped Anthropic from “doing bad, or potentially bad, ‘things.’” He said the U.S. already had regulatory power and criminal laws that applied to AI companies and that there was “a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China.” He made similar comments in a phone call to Nvidia CEO Jensen Huang that Huang, with Trump’s permission, broadcast to a live audience at an “All in Podcast” summit. This was followed up by the Republican Speaker of the House, Rep. Mike Johnson, saying that fear of AI was drummed up by the media and that “we’re not going to take stupid, knee-jerk reaction prescriptions on this.” Not to be outdone, Chine
Nvidia and Meta bosses reject efforts to co-ordinate AI slowdown
