Curvature-aware Expected Free Energy as an Acquisition Function for Bayesian Optimization
Correcting Boundary Bias and Observation Independence in Bayesian Experimental Design
Understanding the Staged Dynamics of Transformers in Learning Latent Structure
ActiveScale: Scaling Active Perception for Robots across Model, Data, and Hardware
How Many Labels Does Model Choice Need? Certificates and Budgets for Selective Prediction
The evolution of sex for artificial intelligence: a population-genetic framework for multigenerational model populations
Uncertainty-Aware Continual Learning for Open-World Intent Discovery Under an evolving Label Space
Modeling the Developmental Shift in Telicity Acquisition
Securing quantum error correction against misleading advice from AI agents
Hypothesis-Driven Autonomous Materials Synthesis with Multimodal LLM Agents
Evolutionary Ensemble Search: Council-Guided Program Evolution with Persistent Memory
HPOQuest: A Rare-Disease Diagnostic Agent Using Active Phenotype Acquisition
Do Frontier Models Seek Safety Evidence Before Acting?
Imitation Learning for Autonomous Driving in CARLA
EvolveTrade: Experience-Driven Policy Refinement for Self-Evolving LLM Trading Agents
AI agents are going rogue. CIOs are racing to put guardrails around them
Over the past several days, some of the world’s top artificial intelligence labs have made a public call to slow down the rapid pace of development, ensuring more safety controls as cases continue to surface of AI agents acting nefariously.
And yet, there is a spillover that’s affecting chief information officers. They’ve been hard at work widely integrating AI agents into their operations, while at the same time watching increasingly risky examples of these autonomous systems in AI labs finding new ways to explore system vulnerabilities and outmaneuver human monitoring.
“This is a risk that enterprises need to be focused on, understand, and start planning for,” says Joe Atkinson, global chief AI officer at consultancy PwC.
As autonomous agents proliferate across enterprises, Atkinson says C-suite technology and security executives must work collaboratively to enforce the proper guardrails, establish systems to monitor AI agents, and concretely track all tasks that these agents are performing. But department heads across the business—ranging from supply chain to customer service, marketing to legal and human resources—will need to play a role in tracking digital employees.
“‘The agent made me do it’ is not going to be a defense from a moral or legal perspective,” says Atkinson.
One company focused on both secure adoption of agentic AI and clear observability is Cisco. When the networking-equipment company built and debuted the AI agent platform MyAgent in August, Cisco centralized all company-authorized large language models, agents, and enterprise data into a single platform.
“We are going to cannibalize and kill every other AI assistant within the company,” says Thimaya Subaiya, executive vice president of operations at Cisco Systems. Because he didn’t want “agent sprawl” across various pockets of Cisco, Subaiya says he won’t authorize any AI agents sold by third-party vendors.
Instead, Cisco wanted full control and visibility of its entire agentic ecosystem—building MyAgent on the company’s compute, storage, networking, and security and observability layers. Around 90,000 of the company’s employees have access to the agentic platform, and Cisco says it saw 50% adoption on a daily basis within just two weeks.
Employees are also encouraged to create their own AI agents, but those need to be approved by a centralized team. Subaiya says around 700 of those agents have already been authorized.
Intuit Chief Technology Officer Alex Balazs recalls that when he and his colleagues sketched on a napkin the first architecture of its generative AI operating system, GenOS, the financial software giant also drew “GenSRF” to represent “security, risk, and fraud.” This ensured that every single AI request that goes into the system is tracked and all responses are recorded.
“You don’t want to try to retrofit the ability to enforce security and responsible AI foundations after the fact,” says Balazs.
Balazs also takes some comfort from the fact that the disclosures of AI agents going rogue have mostly occurred during the testing phase, and that industry leaders Anthropic and OpenAI have shown a willingness to slow down new model development when issues arise. And yet, Balazs adds, “if you’re going to rely on the model intrinsically to do the right thing, I think you’re expecting too much of these frontier LLM companies.”
Jim Fowler, the chief technology and product officer at enterprise networking company Lumen Technologies, believes that while AI’s capabilities are moving faster than governance and security, he doesn’t anticipate that a broad slowdown is enforceable and automatically safer.
“I think for the broader enterprise market, the answer is secure acceleration, not slowing down,” says Fowler. “The bad guys aren’t going to slow down, other nations aren’t going to slow down.”
At Workday, CTO Gabe Monroy says the business software giant has created an “agent system of record” to manage all non-human identities of the digital workforce. This system is used both internally at Workday and sold to customers.
Monroy also says that training is key; engineers and any other user of AI need to really understand the risk profile of an agent and what value they can offer workflows. He’s also mindful that as Workday’s research and development organization increasingly deploys agentic AI for coding, deploying, reviewing, and releasing software on behalf of clients, all employees—no matter where they sit on the org chart—need to be aware of security and compliance.
“It’s got to be delegated down to the team who’s driving these agents, who’s in charge of the engine, the context window, the rules, and the guidelines, and making sure that agent adheres to what we deem responsible behavior,” says Monroy.
Cloud-based software provider ServiceNow’s platform to manage, observe, secure, and govern AI agents is called the AI Control Tower, which, similar to Workday, is used internally but also sold to customers. ServiceNow has also augmented the company’s cybersecurity capabilities through the recent acquisitions of the startups Veza and Armis.
“We’ve been paying close attention to this idea of having to govern and manage, and improve guardrails around AI agents,” says Amit Zavery, ServiceNow’s president, chief product officer, and chief operating officer. Zavery says that the AI Control Tower is “probably one of the fastest-growing products ServiceNow has ever built” because it “gives a lot of peace of mind for all C-level execs and the board.”
Sam Curry, the chief information security officer at cloud security company Zscaler, says security professionals have spent their entire careers worrying about the biggest risk to their operations: humans. But, they’ve only had a few years to think deeply about AI’s risks.
“AI is non-deterministic, it can take initiative, and it is effectively a new form of insider,” says Curry.
Recently, Zscaler joined the Open Secure AI Alliance—Cisco Systems, ServiceNow, and Workday are also members—a Nvidia-led coalition of dozens of firms that is focused on sharing ideas on how to develop open-source tools with the proper safeguards around software and AI agents. Curry says as this work unfolds, leaders will need to wrap their heads around new concepts when it comes to what type of risks AI can present.
“I don’t think we have begun to understand the characteristic psychology of AI,” warns Curry. “We know how to incentivize humans and what they are motivated by. But the incentives of silicon-based intelligence are less known.”
Jo
Good pay, high demand, no takers. The status problem behind America’s trades shortage.
Good morning!
Indeed CEO Deko Idekoba says he thinks AI is a “bit too slow” to change the labor market.
That may sound counterintuitive amid warnings that AI is wiping out entry-level jobs and disrupting white-collar work. But Idekoba is concerned with how uneven AI’s impact is: While the new tech advances rapidly through office work, skilled tradespeople are retiring from jobs AI can’t do, and too few young workers are preparing to replace them.
“No parent is telling their kid, ‘You have to be a plumber. You have to be an electrician,’” he says. In the U.S., he adds, “there’s not a respect for those skills. In Japan or Germany, these people are really well respected.”
But how do hiring managers combat this prestige problem? It may require changing how young people, their parents, and educators define a promising career.
Maggie Hulce, Indeed’s chief revenue officer, sees it as an information problem. “When we help people see where there is demand and where there is salary, people make really good, rational decisions,” she says.
A job such as an AI data-center technician, she adds, may suddenly look more attractive than becoming a finance manager once someone understands the pay structure. Indeed’s data shows that data center jobs for blue-collar workers pay a hefty premium: the hourly pay rate is 42% higher for blue-collar roles in data centers than all other postings.
And blue-collar jobs continue to evolve in this AI boom: Indeed found that new, AI-related, hybrid job titles are cropping up outside of tech. For example, instead of traditional truck driver roles, they are seeing “AI autonomous truck test driver.” Instead of traditional operator roles, they are seeing “AI safety operator.”
But ensuring there is a skilled workforce to fill these roles quickly is something people leaders should be paying attention to, Hulce says.
“With an open role, sometimes [talent acquisition] people will think of it as a process problem,” she says. “But it’s not just a process problem. It’s a business cost problem. It’s a revenue at risk problem.”
Kristin Stoller
Editorial Director, Fortune Live Media
kristin.stoller@fortune.com
This story was originally featured on Fortune.com
Salesforce’s Marc Benioff to AI industry: Regulate yourselves or get sued
SAN FRANCISCO, Calif. — Marc Benioff strode down Mission Street Tuesday afternoon toward Salesforce Tower, where he was set to host a private dinner as part of a whirlwind of events during the company’s annual Dreamforce conference. Some pedestrians stopped to take photos of the 6-foot-5 CEO, surprised he had taken to the streets. One person congratulated him on his keynote address earlier in the day; several bodyguards surrounded Benioff as he walked.
Along the way, Benioff, still wearing the pinstripe suit and burgundy tie from the keynote, waved off concerns that AI could wipe out humanity, a topic that has been front-of-mind in Silicon Valley circles during the past week after Anthropic researcher Jacob Coxon quit over such worries. But Benioff, in a walking interview with Fortune, also said companies should be held accountable for risks they create, and he compared current AI issues to early mistakes made in social media.
“We know we have to hold companies responsible for their products and their technology before people are hurt,” Benioff said, while citing the Hawaiian concept of personal responsibility — kuleana — as essential for corporate ethics (Benioff has baked Hawaiian norms deeply into the San Francisco company’s culture.)
This year’s Dreamforce conference has again briefly become the center of gravity for the tech industry, with CEOs of major tech companies opining on AI safety risks.
Earlier Tuesday, Benioff was on stage at the Yerba Buena Center for the Arts interviewing OpenAI Chief Executive Sam Altman. The OpenAI boss described the July hack of Hugging Face by a swarm of rogue OpenAI agents as a terrifying wakeup call, and he said that companies should pace development so that safety is ahead of capabilities. During Benioff’s morning keynote at the Moscone Center, he was joined by Anthropic Chief Executive Dario Amodei, who also advocated pacing frontier AI models.
Nvidia Chief Executive Jensen Huang, meanwhile, took a different approach during Benioff’s keynote, saying speed and safety can exist simultaneously. Separately, Meta Chief Executive Mark Zuckerberg also shrugged off concerns, writing Tuesday that AI labs have a natural incentive to create safe AI.
Without offering a concrete solution, Benioff told Fortune that the responsibility largely lies in the hands of companies making AI (some of which Salesforce invests in). He said in practice, this means companies looking ahead to prevent harm rather than offering excuses after a mistake occurs, and that firms should rank their values so as to decide what takes precedence when priorities conflict.
While he declined to say whether governments should regulate AI companies, he said laws that govern product liability can be used as a legal mechanism for accountability, much as car manufacturers are held liable if a vehicle malfunctions.

The outspoken, at times controversial CEO also proposed a new Fortune 500-like list that would rank companies by their ethical standards, and he noted Apple as one firm he holds in high regard as a security standard-bearer. Benioff said he had not spoken recently to President Trump about AI safety; the president this past weekend called AI doomsday scenarios exaggerated and blamed “negative forces.”
“Only [tech] companies know what’s going on in their lab,” Benioff said. “At some deep level, these companies must hold themselves responsible for their safety.”
At Salesforce, Benioff said, that has meant wrapping AI models in a “trust layer” designed to prevent models from misbehaving because they operate within a highly constrained structure which also closely monitors agents. He said neither Salesforce nor its clients have experienced Hugging Face-like episodes. The difference for Salesforce is that it is not the one creating the super powerful and potentially dangerous frontier models.
Benioff has an odd relationship with the AI labs, and with San Francisco itself. On one hand, Anthropic and OpenAI’s creation of autonomous systems threatens the very existence of Salesforce’s core products.
Yet Salesforce also benefits from both labs, using their models to power various parts of its flagship AI product, Agentforce. On Tuesday, Salesforce announced a new system that allows customers to access its tools without logging into its systems, and it also unveiled a new reasoning model for Agentforce that it is building with Nvidia.
And while Benioff comes from a family with deep roots in San Francisco (and has donated more than $1 billion to the area), he has recently found himself on the defensive in his hometown. Last year, Benioff issued an apology after he called on President Trump to deploy the National Guard to San Francisco during Dreamforce, citing a shortage of officers in the local police department. Angel investor Ron Conway, one of the city’s most recognized tech investors and a longtime friend of Benioff’s, resigned from the Salesforce Foundation’s board of directors after the comments, which disappointed some San Francisco residents because Benioff had built a reputation as a progressive, Democratic-supporting CEO before more recently embracing Trump (Benioff has said he’s an independent).

On Tuesday, as a fresh round of protesters gathered near the Moscone Center to object to issues such as Salesforce’s work with U.S. Immigration and Customs Enforcement, Benioff—who lives in Hawaii—took a lighter tone.
He said Salesforce still hires hundreds of off-duty officers during the conference but, as he stepped