How to Score Experts for One-Shot MoE Expert Pruning: A Systematic Formulation and Selection Principle

arXiv cs.LGarXiv:2606.15716v2 Announce Type: replace Abstract: Mixture-of-Experts (MoE) language models reduce per-token computation through sparse expert activation, yet deployment still requires storing the full expert pool, making one-shot expert pruning a practical approach for reducing memory usage. Although effective, existing criteria are largely heuristic, and no single criterion is universally optimal. Thus, establishing a principle for selecting pruning criteria suited to different deployment objectives remains an important yet largely underexplored problem in one-shot expert pruning. To this end, we introduce a systematic formulation for one-shot MoE expert pruning organized around three factors: routing frequency, gate weighting, and activation strength. The formulation yields a criteria selection principle: task-agnostic pruning should favor routed-token-averaged, gate-free activation-based criteria, whereas task-specific pruning can benefit from retaining routing-frequency and gate-weight information. Beyond this principle, the formulation also provides a systematic view of existing heuristic criteria and gives rise to two new task-agnostic criteria, Mean Activation Norm (MAN) and Mean Squared Activation Norm (MSAN). Across four representative MoE models and 16 diverse benchmarks, MAN and MSAN are consistently strong in the task-agnostic setting, obtain the top-two average ranks, and improve average performance by 4.0 points on average and up to 8.8 points over the strongest baseline.

2026-11-02 05:00:00 · AI应用,搜索RAG,MoE架构,模型评测,招聘HR,榜单评测,论文

AIMER: Calibration-Free Task-Agnostic MoE Expert Pruning

arXiv cs.LGarXiv:2603.18492v4 Announce Type: replace Abstract: Mixture-of-Experts (MoE) language models increase parameter capacity without proportional per-token computation, yet deployment still requires storing the full expert pool, making expert pruning important for reducing memory and serving overhead. Existing task-agnostic expert-pruning methods are typically calibration-dependent: they estimate expert importance from routing or activation statistics on a calibration set, making pruning decisions sensitive to calibration-data variation while introducing substantial preprocessing cost. We propose AIMER (\textbf{A}bsolute mean over root mean square \textbf{IM}portance for \textbf{E}xpert \textbf{R}anking), a simple calibration-free criterion that identifies more distinct experts by capturing the concentration pattern of expert weights, making it well suited for task-agnostic expert pruning. Across 7B to 47B MoE language models with distinct architectures and 16 diverse benchmarks, AIMER consistently delivers stronger capability balance across diverse tasks than existing calibration-free methods. Surprisingly, AIMER also achieves better balance than strong calibration-based expert-pruning baselines calibrated on the widely used task-agnostic C4 corpus, while requiring only 0.22--2.06 seconds to score all experts.

2026-11-02 05:00:00 · MoE架构,模型评测,论文

Hierarchical Residuals: A Minimalist Decoupling of Residual Width from Compute Width

arXiv cs.LGarXiv:2610.41711v1 Announce Type: new Abstract: Residual connections underpin Transformers, but they implicitly tie the residual width carried through depth to the compute width used inside each layer. We introduce Hierarchical Residuals (HiRes), a minimalist architectural change that decouples these dimensions, widening the residual state while keeping per-layer compute width fixed. HiRes groups layers into blocks: standard d-dimensional PreNorm layers operate inside each block, while a wider D-dimensional PostNorm outer residual connects blocks. Learned read/write projections bridge the two spaces at block boundaries, leaving individual Transformer layers unchanged. Across different model sizes and depths, HiRes consistently outperforms vanilla residual addition, with fitted scaling laws indicating a 1.30-fold improvement in FLOPs efficiency. On OLMoE-1B-7B pretraining for 500B tokens, HiRes achieves 0.0208 lower training loss and 1.14-point higher downstream accuracy. HiRes is also competitive with other residual expansion methods such as VWN, mHC and AttnRes with less activation memory and I/O costs. Built from only dense projections and normalizations, it requires neither specialized kernels nor infrastructure-level optimizations. HiRes thus exposes residual width as a practical scaling variable using only standard operators.

2026-11-02 05:00:00 · Transformer,扩散模型,MoE架构,预训练,招聘HR,论文

PixelMoE: Scaling Pixel-Space Diffusion with LLM-Style Mixture-of-Experts

arXiv cs.CVarXiv:2610.18857v2 Announce Type: replace Abstract: Pixel-space diffusion models provide an appealing alternative to latent-space diffusion models by directly modeling raw images without relying on a learned tokenizer. However, scaling pixel-space models has predominantly followed dense strategies, tightly coupling model capacity with active computation. In this work, we show that standard LLM-style Mixture-of-Experts (MoE) architectures can be effectively transferred to pixel-space diffusion. Building on this finding, we introduce PixelMoE, a pixel-space generative framework that sparsely scales model capacity with standard LLM-style MoE, without diffusion-specific routing mechanisms or auxiliary routing objectives. Specifically, PixelMoE combines standard Token-Choice (TC) routing with fine-grained routed experts for input-adaptive sparse computation, a shared expert for capturing common representations, and a bias-based load-balancing strategy. To complement sparse capacity scaling, PixelMoE further introduces Pixel-Space Hierarchical Flow Matching, which varies spatial resolution along the denoising trajectory from global structure formation to fine-grained detail refinement. At comparable active parameter counts, PixelMoE consistently outperforms its dense counterparts across model scales, demonstrating effective sparse capacity scaling in pixel space. On class-conditional ImageNet-1K, PixelMoE achieves FID scores of 1.47 at 256x256 and 1.53 at 512x512 without representation alignment or perceptual losses, comparing favorably with leading pixel-space and latent-space generative models. Code and pretrained models are available at https://github.com/KlingAIResearch/PixelMoE and https://huggingface.co/KlingTeam/PixelMoE, respectively.

2026-11-02 05:00:00 · 大模型,开源,快手,文生视频,扩散模型,MoE架构,预训练,模型安全对齐,端侧AI,论文

MoETok: Sparse Expert Routing for 1D Visual Tokenizers

arXiv cs.CVarXiv:2610.41992v1 Announce Type: new Abstract: Modern visual tokenizers aim to preserve fine-grained visual details while learning high-level semantic representations for downstream generation and understanding. Query-based 1D visual tokenizers encode images into flexible latent sequences, yet how individual learnable queries gather and encode image information remains poorly understood. We reveal that these queries spontaneously develop complementary roles under semantic supervision: a small set of Content queries captures high-level semantic information, while a larger set of Detail queries preserves fine-grained visual details for reconstruction. To adapt token processing to these heterogeneous roles, we introduce MoETok, a 1D visual tokenizer that routes tokens through sparsely activated experts. Jointly optimized with reconstruction and semantic objectives, the router learns role-specialized processing paths adapted to each token's representation. Across model scales and token budgets, MoETok consistently improves reconstruction, semantic representation, and autoregressive generation over dense tokenizers under matched supervision and active computation. On ImageNet, it combines strong reconstruction and semantic representation with state-of-the-art class-conditional generation within the 1D tokenizer family. We further validate MoETok at scale in a unified autoregressive model jointly trained for understanding and generation with a next-token cross-entropy objective. The resulting model combines competitive visual understanding with state-of-the-art generation among unified discrete models, matching or surpassing diffusion-based models such as BAGEL and Qwen-Image on benchmarks including DPG-Bench (87.61), GenEval2 (40.04 GM), and WISE (0.49).

2026-11-02 05:00:00 · 大模型,阿里巴巴,扩散模型,MoE架构,模型评测,招聘HR,论文

What We Keep of What We Forget: Replacement Operators and Routing Semantics in Training-Free MoE Expert Pruning

arXiv cs.CLarXiv:2610.41366v1 Announce Type: cross Abstract: Training-free expert pruning makes two decisions: which experts to remove, and what the layer computes when its router still selects a removed expert. Published methods change both at once. Holding the mask fixed, we compare replacement operators from three families on OLMoE, DeepSeek-V2-Lite and Moonlight, with Granite as a fourth router, using paired, multiplicity-corrected tests. The largest effect is a choice usually left implicit: the survivors' gate scale. On routers trained without top-k renormalization, renormalizing the survivors' gates, a common implementation choice, costs 0.8-8.5 points and reverses whether rerouting beats MoNE's mean-output replacement; on two routers trained with renormalization, the trained scale wins at 50% pruning. We introduce the tilted novice, a closed-form low-rank map of the pruned expert's input whose rank-0 case is MoNE's constant. At 50% pruning it adds 1.2-1.8% parameters and gains 3.45 and 1.37 points over the constant on OLMoE and Moonlight, stable across calibration draws and largely unchanged by 4- and 3-bit quantization. No operator dominates: trained-scale rerouting is best on DeepSeek-V2-Lite, and a partial copy of each pruned expert is statistically indistinguishable from the tilted novice on two models. Evaluation can change conclusions: at 25% pruning MoNE's constant keeps 92-97% of dense multiple-choice accuracy but only 5-65% of GSM8K and 0-17% of code scores, and the calibration domain moves the mask 4-9 times more than resampling. We propose a five-item reporting checklist.

2026-11-02 05:00:00 · 大模型,算力芯片,Google,DeepSeek,MoE架构,微调蒸馏,端侧AI,招聘HR,榜单评测,论文

Proof-Carrying Materials: Falsifiable Safety Certificates for Machine-Learned Interatomic Potentials

arXiv cs.AIarXiv:2603.12183v3 Announce Type: replace-cross Abstract: Machine-learned interatomic potentials (MLIPs) are deployed for high-throughput materials screening without formal reliability guarantees. We show that a single MLIP used as a stability filter misses 93% of density functional theory (DFT)-stable materials (recall 0.07) on a 25,000-material benchmark. Proof-Carrying Materials (PCM) closes this gap through three stages: adversarial falsification across compositional space, bootstrap envelope refinement with 95% confidence intervals, and Lean 4 formal certification. Auditing CHGNet, TensorNet and MACE reveals architecture-specific blind spots with near-zero pairwise error correlations (r <= 0.13; n = 5,000), confirmed by independent Quantum ESPRESSO validation (20/20 converged; median DFT/CHGNet force ratio 12x). A risk model trained on PCM-discovered features predicts failures on unseen materials (AUC-ROC = 0.938 +/- 0.004) and transfers across architectures (cross-MLIP AUC-ROC ~ 0.70; feature importance r = 0.877). In a thermoelectric screening case study, PCM-audited protocols discover 62 additional stable materials missed by single-MLIP screening - a 25% improvement in discovery yield.

2026-11-02 05:00:00 · 扩散模型,MoE架构,模型评测,招聘HR,论文

Characterizing Recoverable Expert-Utilization Signatures in Fine-Tuned Mixture-of-Experts LLMs

arXiv cs.AIarXiv:2610.41039v1 Announce Type: cross Abstract: As Mixture-of-Experts (MoE) large language models are increasingly deployed in distributed environments, from federated learning to edge-cloud systems, their architecture harbors an underexplored landscape of security vulnerabilities. Moving beyond textual analysis, we uncover structural memorization: a novel vulnerability where a fine-tuned model memorizes not only its training data but also the internal pathway used to process it, creating a characteristic routing fingerprint unique to the training process. We provide comprehensive empirical evidence that structural memorization emerges consistently even from a few samples, and that its intensity is directly influenced by the choice of trainable modules. We show that these memorized expert pathways can be stealthily recovered through the model's unconditional generation, using only routing and output logits, with no knowledge of the fine-tuning data or procedure. In distributed deployments, this vulnerability is amplified as routing information is exposed at multiple trust boundaries, including federated aggregation points, edge-cloud interfaces, and multi-tenant monitoring systems. Recovery exposes a critical blind spot: traditional membership inference metrics, viewed through the lens of expert routing, fail to identify memorized data corresponding to these structural pathways. To close this gap, we devise composite metrics combining reference model tail likelihood with complexity-aware normalization, significantly improving detection of memorized routing structures. This enables adversaries to pinpoint critical experts shaped by fine-tuning, whether dominant or dormant, and execute targeted attacks such as router manipulation or backdoor injection in realistic distributed settings where attackers lack training data but can observe routing patterns across system components.

2026-11-02 05:00:00 · 大模型,算力芯片,Google,扩散模型,MoE架构,微调蒸馏,联邦学习,招聘HR,论文

Experts Exhibit Structured Redundancy: Structure-Preserving Expert Pruning for Mixture-of-Experts Large Language Models

arXiv cs.AIarXiv:2610.41236v1 Announce Type: new Abstract: Sparse MoE mechanism significantly improves the trade-off between model capacity and inference efficiency. However, practical deployment still requires storing all experts, resulting in substantial memory overhead. We show that expert outputs in MoE layers exhibit structured redundancy. Across different models and task categories, the stable rank and entropy effective rank of expert outputs are consistently far below the nominal number of experts, while expert output distributions also exhibit fewer effective modes. Motivated by these observations, we reformulate expert pruning as a structure-preserving expert subset selection problem and efficiently optimize it via a greedy strategy, rather than only independently scoring and ranking individual experts or exhaustively searching over expert combinations. We propose two expert pruning methods under the unified structure-preserving framework: Projection Energy Expert Pruning (PEEP), which preserves the functional subspace by maximizing weighted projection energy, and Distributional Coverage Expert Pruning (DCEP), which preserves the distributional geometry by minimizing weighted coverage error. Both methods incorporate router-derived weights as priors, enabling them to jointly preserve the global structural relationships among experts while accounting for the local importance of individual experts. Experiments on DeepSeek-V2-Lite and Qwen3-30B-A3B demonstrate that, at the expert retention ratios of 75% and 50%, our methods outperform the state-of-the-art baselines on the majority of benchmarks and consistently achieve the best average performance across both general tasks and mathematical reasoning tasks.

2026-11-02 05:00:00 · 大模型,算力芯片,AI应用,Google,阿里巴巴,DeepSeek,推理思考,搜索RAG,MoE架构,模型评测,论文

天数智芯天垓300实测对标H200，国产GPU接下来拼什么？

雷锋网

在实测场景中达到H200的性能水平后，国产GPU的竞争，开始进一步转向系统效率、软件适配与实际交付。

本周的天数智芯媒体开放日上，最新一代天垓300与H200的实机对比，是最直观的性能展示。让GPU与专用芯片组成异构系统，是对推理效率的新探索。软件栈引入Agent与Skills，则试图解决客户部署、适配和优化的工程难题。

天数智芯表示，今年完成的一组第三方测试中，在典型PD分离部署下，天垓300折算单卡端到端吞吐达到H200的1.05倍。这表明国产通用GPU在特定高端推理负载中，已经拥有与H200直接竞争的性能基础。

但从性能接近到客户采购，再到持续扩容，中间仍隔着系统效率、软件适配和稳定交付。当硬件具备竞争力，国产GPU怎样把这种能力转化为客户持续使用的理由？当推理催生更多专用芯片，通用GPU的独特价值是什么？

从云端天垓300到端侧彤央系列，天数智芯也在拓展产品布局，将竞争延伸到不同形态的算力需求中。

天垓300对比H200实测，性能小幅领先

过去很长一段时间，峰值算力是国产AI芯片受到关注的重要指标。但对最终用户而言，设备每秒真正产出多少有效Token，比参数表上的理论算力更接近采购决策。

天数智芯披露的第三方测试覆盖四大主流模型、三类场景。其中，1.05倍指典型PD分离部署下，折算单卡的整体端到端吞吐，反映的是特定模型与部署条件下的系统表现。

另外，根据天数智芯性能实验室的数据，在GLM-5.3的64K长上下文场景下，保障同等用户体验时，天垓300的MoE吞吐达到H200的1.15倍。在8K至128K长上下文的Agent场景中，首字延迟较H200降低约10%至30%。

前者关系到系统能够承接多少请求，后者关系到用户发出请求后需要等待多久，它们对应不同工作负载与性能指标。

天垓300能够在这些场景中取得性能领先，一个关键是针对实际负载进行了架构优化。

基于天枢架构的天垓300，单卡显存容量达到144GB，并围绕Attention、MoE等计算环节进行优化。长上下文需要容纳更多状态，也增加了Attention计算压力。MoE虽然通过激活部分专家控制计算量，却带来了小规模矩阵计算、专家分配和数据交换等新问题。

为此，天垓300让Tensor Core与Vector Core并行处理Attention中的矩阵运算和Softmax等向量操作，同时针对MoE中的小规模矩阵、低精度计算与通信协同进行优化。

这些针对性设计，有助于减少不同计算环节之间的等待，把更多硬件资源转化为实际产出。

此次披露的测试与现场演示，展现了天垓300进入高端推理实际竞争区间的能力。对客户而言，下一步要比较的，是在同样的响应要求下，系统能承接多少请求、持续运行表现如何，以及需要付出多少设备、电力和运维成本。

推理需求催生专用芯片，GPU的价值会被削弱吗？

随着推理需求快速增加，以LPU为代表的专用推理芯片备受关注，不同计算环节对硬件的要求也进一步分化。

Prefill处理输入，通常更偏向计算密集型任务；Decode逐步生成Token，更容易受到访存、任务启动和同步开销的影响。进一步拆开，Attention与FFN对计算、带宽和存储的需求也不相同。

当某类任务足够稳定、规模足够大，专用芯片就有机会通过针对性设计，获得更好的性能或能效。让一种通用GPU处理所有环节，未必始终对应最低的成本。

天数智芯展示的AFD分离方案，就是对这种分工的回应。

它在Prefill与Decode分离的基础上，继续拆分Decode阶段的Attention与FFN：天垓300承担Prefill全流程计算，以及Decode中的Attention；合作厂商的XPU承担FFN专家网络计算。

运行过程中，GPU把中间激活数据发送给XPU，XPU完成计算后返回结果，继续推进模型生成。天数智芯给出的对比结果显示，在所展示的方案中，Decode推理延迟降低60%。

天数智芯AI与加速计算负责人单天逸表示：“未来的AI算力系统，不一定由一种芯片包打天下。关键是如何把它们高效组织起来，让整个系统发挥更大的价值。”

异构分工为不同芯片提供了合作空间，GPU的通用性则关系到系统能否持续适应模型变化。

天数智芯副总裁邹翾对雷峰网表示，专用硬件可以在固定工作负载中实现较大的性能提升，但模型与算法仍在快速迭代，公司坚持以通用框架为基础，结合专用加速能力。

2026-10-30 09:02:00 · 大模型,算力芯片,AI应用,Microsoft,智谱,Agent智能体,推理思考,设计创意,Transformer,MoE架构,长上下文,端侧AI,合作,榜单评测

NVIDIA Releases Nemotron 3.5 Transcribe: An Open 30B MoE Model for 40-Language Transcription with ~4B Active Parameters - MarkTechPost

Google News LLMNVIDIA Releases Nemotron 3.5 Transcribe: An Open 30B MoE Model for 40-Language Transcription with ~4B Active Parameters MarkTechPost

2026-10-29 04:41:24 · 算力芯片,Google,NVIDIA,MoE架构,招聘HR

What would you require before calling an inference change “quality-neutral”? [D]

Reddit r/MachineLearning

Two GSM8K scores: 96.89% → 96.82%, across all 1,319 test questions.

These are from NVIDIA’s Ling-3.0-flash comparison on one DGX Spark. The optimized configuration changes the MoE backend, adds an FP8 output head and enables NEXTN speculative decoding together. Both runs use the FP4 checkpoint.

Aggregate scores summarize the total. A paired view of the 1,319 answers would show where the two configurations agree and where they trade wins and losses.

For a claim of no meaningful quality regression, what evidence would you consider sufficient: paired per-question outcomes, repeated seeds, a declared equivalence margin, another task set?

Especially when a precision change and a decoding optimization ship together, which checks give the most information for the evaluation budget?

submitted by /u/AKAMA199
[link] [comments]

2026-10-30 06:59:57 · 算力芯片,Google,NVIDIA,扩散模型,MoE架构,招聘HR

Interpretability Is Not Just a Tax: Sparsity and Diversity Yield Legible and Composable Reward Experts for Personalization

arXiv cs.AIarXiv:2606.04284v2 Announce Type: replace-cross Abstract: Preference modeling aims to capture heterogeneous human values and increasingly relies on learning latent components to represent this variation without fine-grained supervision. What these components actually represent, however, is rarely examined. We audit Mixture-of-Experts (MoE) reward models along three rungs---semantic coherence, functional correspondence, and composability---in controlled and real-world settings, holding the architecture and training data fixed so that differences isolate the effect of the training objective. Our diagnosis is that the components conventional MoE reward models learn are entangled, which limits not only interpretability but also the personalization such models are built for. The entanglement is not intrinsic to the architecture but a property of the training objective: off-the-shelf sparsity and diversity regularizers already substantially reduce it. The resulting configuration, which we call sparse MoE, recovers all underlying semantic categories without domain supervision in a controlled setting, reaching 84.8% expert purity against 60.8% for conventional MoE; on real-world data its experts admit human-interpretable task- and domain-level descriptions aligned with their functional strengths. Targeted interventions on routing weights produce predictable preference changes, showing that these interpretations are behaviorally grounded rather than merely descriptive. Crucially, the disentanglement induced by this objective also makes the experts composable: lightweight router updates yield a 25.81-point improvement in few-shot personalization with 50 examples, roughly 2.7 times the gain of the identical architecture and adaptation procedure applied to entangled experts. Together, these results show that MoE reward models can learn interpretable expert structure, and that the same structure can substantially improve personalization.

2026-10-31 04:00:00 · 扩散模型,MoE架构,提示工程,招聘HR,论文

Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey

arXiv cs.AIarXiv:2605.27431v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) presents a naturally compatible and scalable framework for multimodal learning, demonstrating strong adaptability across diverse modalities and tasks. Despite its growing success, a comprehensive and systematic review on the MoE metho addressing multimodal challenges remains lacking. Existing surveys tend to evaluate either multimodal learning or MoE independently from method taxonomy, overlooking the unique interplay between them. This survey fills that gap by answering a central question: \textit{How does MoE effectively resolve multimodal challenges?} We approach this from three key perspectives: (1) \textbf{MoE as an Efficient Multimodal Engine:} enabling scalable multimodal modeling by decoupling computational cost from parameter growth and mitigating modality redundancy through selective expert activation; (2) \textbf{MoE as a Multimodal Representation Learner:} integrating complementary multi-opinion expert knowledge to enrich alignment and interaction representations; and (3) \textbf{MoE as a Multimodal Adapter:} providing a modular and flexible mechanism to model imperfect data scenarios such as modality imbalance and missing modality. Through our extensive literature review, we identify critical research gaps, including interpretable routing, expert communication, modality integration, and lifelong multimodal learning. We position this survey as a foundation for future research toward interpretable and sustainable multimodal Mixture-of-Experts system.

2026-10-31 04:00:00 · 快手,文生视频,多模态,MoE架构,模型安全对齐,招聘HR,论文

Turing LLM Post-Training: Calibrated Distillation and Efficient Dynamic-MoE Execution

arXiv cs.AIarXiv:2610.39876v1 Announce Type: cross Abstract: We study post-training for dynamically routed mixture-of-experts (MoE) models, focusing on integrating independently optimized specialists while keeping consolidation efficient. Our approach combines calibrated multi-teacher on-policy distillation (C-MOPD), which matches each selected teacher on a local Top-$K$ support with teacher-wise loss normalization and clipping, with an execution path that uses post-capacity bounded-width dispatch, final-route replay, and asynchronous rollout--training overlap. On Turing-20B-A2B, the selected model achieves a four-domain normalized integration score of 0.9988 and retains at least 90.16\% of the measured specialist gain in every domain. At matched $K=32$, the full calibration objective scores 0.9962, compared with 0.9724 without calibration and 0.9834 without clipping. At a matched 8K response cap, the model exceeds both shared SFT and Qwen3-8B on all four reasoning-enabled mathematics and code benchmarks (unweighted mean: 85.92, versus 80.33 for Qwen3-8B and 74.21 for SFT), and improves over SFT on ten of twelve broader evaluation metrics. In a PPU-based profile, the combined execution configuration reduces consolidation step time from 398.7 to 139.3 seconds, an observed 2.86$\times$ speedup over serial execution. These results demonstrate strong specialist integration alongside more efficient consolidation, while reflecting the complete training and execution framework rather than any single component.

2026-10-31 04:00:00 · 大模型,阿里巴巴,推理思考,MoE架构,强化学习,微调蒸馏,模型评测,招聘HR,榜单评测,论文

Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models

arXiv cs.AIarXiv:2610.38981v1 Announce Type: cross Abstract: Speculative decoding (SD) accelerates autoregressive inference by leveraging the memory bound nature of decoding, verifying multiple draft tokens in a single forward pass. For mixture-of-experts (MoE) models, prior work predicts a concave speedup-batch size relationship, with gains peaking at intermediate batch sizes due to expert saturation. We evaluate this across 10 sparse MoE models (30B--1T parameters, 8--512 experts) and find recent fine-grained models break this trend, showing higher speedup at low batch size. We find verification cost is governed not only by activated-expert bytes, but also by temporal overlap across draft tokens and GPU execution dynamics. At low batch size, MoE layers in fine-grained models account for a much smaller share of runtime than of weights; on H100, kernel profiling attributes this mainly to higher bandwidth utilization of MoE kernels relative to attention, with L2 absorbing repeated reads of co-routed experts. Across draft tokens, routing overlap further reduces the unique experts activated during verification relative to independence assumptions. We model this effect using a maximum-entropy Iterative Proportional Fitting (IPF) framework that predicts unique expert counts from routing-overlap statistics with less than 4\% error, and substantially narrows the gap to real-model curves and captures the qualitative low-batch trend from synthetic routing.

2026-10-31 04:00:00 · 算力芯片,AI应用,搜索RAG,Transformer,MoE架构,论文

LRMoE: Demand-Side Load Re-Calibration for Efficient MoE Training on Memory-Constrained Nodes

arXiv cs.AIarXiv:2610.37720v1 Announce Type: new Abstract: Sparse Mixture-of-Experts (MoE) models offer scalable capacity for Web applications, but uneven routing creates computation and communication bottlenecks. Existing placement and replication methods redistribute this workload, yet additional expert copies compete with token-related buffers for limited GPU memory. We present LRMoE, a memory-aware MoE training system that jointly recalibrates backward demand and balances the retained workload. During forward execution, it plans retention quotas and expert placement under resource constraints while preserving all routed computations. During backpropagation, a load recalibration metric estimates each token-expert pair's contribution to the expert update and selects pairs within these quotas. A capacity-aware routing algorithm distributes retained pairs among eligible instances of their original experts. Physical compaction then forms smaller communication payloads and variable-size grouped GEMMs, reducing backward traffic, computation, and execution-buffer demand. Experiments on Qwen3-30B-A3B and GLM4.5-106B-A12B show that LRMoE reduces cumulative MoE-layer training time to fixed-loss targets by 21.2-44.8% relative to SYMI.

2026-10-31 04:00:00 · 大模型,算力芯片,阿里巴巴,智谱,扩散模型,MoE架构,论文

预制数据中心，突然火了

钛媒体

（本文作者为半导体产业纵横，钛媒体经授权发布）

文 | 半导体产业纵横

最近，黄仁勋又把目光投向了数据中心。这一次，他关注的不只是里面放什么芯片，还有一个更基础的问题：如何更快地建好一座数据中心。

老黄给出的办法是：做预制。

整套Vera Rubin AI工厂在合作伙伴的工厂里装进模块化集装箱，线缆、电路、液冷管路全部布设完毕并通过功能验证，运抵现场后接上电力、冷却和网络即可开机。靠着这一招，现场部署从6个月压缩至最短14天。

预制数据中心，其实并非老黄一家琢磨。

华为推出3D数据中心，将空间布局调整与工厂预制结合起来；立讯精密旗下的立讯智算，也展示了集装箱式预制数据中心方案。大家想解决的是同一个问题：让机房建设和设备交付，尽量跟上算力需求的增长。

毕竟，在AI时代，机房如果还按老节奏慢慢盖，确实容易出现一种尴尬：爷爷立项，爸爸施工，孙子开机，顺便给上一代GPU办个怀旧展。等数据中心两年后建出来，黄花菜都凉了，最后只能当作AI的传家宝。

01 数据中心，花钱大户

AI基础设施到底有多热，可以先看两个指标：花钱，用电。

国际能源署在2026年的报告中指出，五家科技公司的资本开支在2025年超过4000亿美元，预计2026年还要增长75%。用电量也在同步增长。IEA统计，2025年全球数据中心用电需求增长17%，其中面向AI的数据中心增长50%；其更新后的基准预测是，全球数据中心用电量将从2025年的485太瓦时，上升到2030年的约950太瓦时。

国内这边，工信部披露，截至2026年9月底，我国智能算力规模达到2592 EFLOPS，同比增长146%。而这些数字，最终都要靠具体的设备和机房上去实现。

芯片需要服务器，服务器需要网络和存储，还要吃电、散热。到了高密度AI系统这里，供电、液冷、设备重量、安装空间和维护通道，都需要一起设计。英伟达在DSX POD技术说明中，也将计算、网络、电力、液冷和运输结构列为需要协同处理的工程环节。

这意味着，算力扩张带来的不只是“多盖几间机房”。设备的集成程度提高了，配套系统也需要跟着调整。

与此同时，计算设备本身的组织方式正在变化。面对大模型训练和推理中的通信需求，厂商开始通过更紧密的高速互联，将更多加速器组织成协同工作的系统，超节点成为其中的重要方向。

而超节点怎么设计，会直接影响机房怎么配套、怎么安装，以及多久才能交付。

02 形态各异的超节点，给机房出了新题

数据中心的需求下，服务器跟不上了，行业里全都在看超节点。

当前，厂商所说的超节点，并不对应完全一致的物理尺寸。它可以集中在一座机柜里，也可以跨越多座机柜。例如，新华三有单柜32卡或64卡的产品，华为公布的Atlas 950最大配置则覆盖160个机柜。

看超节点不能只盯着“多少卡”。至少要分清三个数字：单个机柜装多少卡，一个超节点连接多少卡，以及整个集群总共有多少卡。这三个数字，分别关系到部署密度、紧密互联的规模和整体建设规模。

首先来看机柜形态和数量。

按照单柜集成数量，业内的超节点大致形成了三个梯队。32卡、64卡是较常见的基础配置。其中，华为Atlas 950、沐曦曦景S600，以及百度天池、新华三的相关方案，都采用了单个计算柜64卡的设计。

再往上，是将单柜集成数量提高到百卡以上的方案。摩尔线程在2026年WAIC展示的MTT C256，采用单柜128卡配置；天数智芯展示的IXL144，则强调将144颗GPU集成到单机柜中。

在这里比较特别的是中科曙光scaleX640，采用“一拖二”高密架构和极度定制化的刀片模块（将加速卡、内存、互联芯片集成至30cm×20cm×5cm的空间内），在1.8米标准机柜内容纳80个刀片模块，硬生生将单柜密度推高至惊人的640卡。

再来看密度背后的互联。卡装得越多，数据往来就越繁忙。在大模型训练和MoE负载中，芯片之间的通信已经成为系统设计的重点。

讨论互联时，可以分成三个层面：采用什么协议，硬件怎样连接，以及信号通过铜还是光传输。

在协议层面，华为通过灵衢互联体系组织计算资源，百度围绕自研XPU-Link和可编程交换技术构建通信域，壁仞则布局BLink超节点互连协议。行业中也有UALink这样的开放加速器互联标准。比如，新华三UniPoD系列，就同时包含基于以太协议和PCIe协议的产品，其中F80000采用PCIe光互联。

在硬件连接层面，业内在讨论正交直连、Cable Tray等设计。比如，沐曦曦景S600强调柜内连接的“三零”设计，通过减少传统线缆连接，改善部署与维护；百度则在天池方案中，将XPU-Link协议、交换技术与Cable Tray铜缆互连一起设计。

值得关注的是光连接选手。

壁仞推出的1024卡方案采用NPO近封装光学与分布式解耦架构，将GPU节点和交换节点分开部署，通过光连接组织资源，支持单个超节点最高1024卡Scale-up扩展。华为的光模块的应用也很领先，此前384卡配置400G光模块，当前1024卡升级为800G GPU光模块，速率提升一倍、用量减少。

第三看迁移成本，也就是软件生态。对已经积累了大量CUDA应用的客户来说，更换计算平台涉及的不只是采购。

各家国产芯片都有自己的软件底座：华为有CANN，摩尔线程有MUSA，沐曦有MXMACA，昆仑芯和天数也分别建立了配套的软件栈。在框架层的迁移上，客户继续使用PyTorch等熟悉的框架，底层则切换到新的硬件平台。华为通过TorchNPU将PyTorch接入昇腾，昆仑芯与飞桨进行适配，其他厂商也通过框架支持，让模型开发延续原有方式。

在应用中直接用CUDA接口和自定义内核的部分，摩尔线程的MUSA提供了两种具体做法：musify将CUDA源码转换为MUSA代码，MUSA Mapping则在编译过程中完成映射，让支持范围内的源码保留CUDA写法，再编译成面向摩尔线程GPU的程序。昆仑芯也在XPU Runtime中提供CUDA兼容接口。中科曙光则主打开放包容，兼容主流生态并支持多品牌国产卡混合部署。

最后来看，这些超节点正在进入哪些场景。从实际需求看，超节点目前主要沿着三类场景推进：大模型训练与推理、公共智算基础设施，以及科学计算等行业应用。

大模型是最直接的推动力。模型规模扩大、训练任务增加，让互联网企业较早产生了大规模紧密互连的需求。作为第一代正式商用的超节点，昇腾384超节点已商用落地750多套，规模应用于互联网、运营商、金融、教育、医疗、交通、制造等行业，也是国内唯一训练出SOTA模型的超节点。

百度超节点有百度自身核心业务的验证，已经实现了量产交付，是国内率先实现量产交付的超节点产品之一。

中科曙光国家超算互联网核心节点在郑州上线试运行。据悉，该节点算力资源由3套曙光scaleX万卡超集群系统提供支撑，可对外提供超3万卡的国产AI算力。

与此同时，国产GPU厂商的集群业务也在推进。摩尔线程MTT C256 目前处于“商用准备就绪、早期客户验证”阶段；但摩尔线程的夸娥智算集群已在多地交付，摩尔线程披露一笔6.6亿元夸娥智算集群合同，已在上半年完成交付并确认收入。同时，无锡惠山智算中心一期部署 2000 张 S5000，两个月后基本满载，已有 20 多家客户测试或正式使用。

壁仞目前成功交付了包括2048卡光互连光交换GPU超节点在内的多个千卡级智算集群。此前推出了“光跃LightSphere X”的超节点方案，在半年内快速实现了128卡商用版的落地。

天数智芯天垓系列GPU已经实现万卡级规模化落地，服务超过380家政企、科研与产业客户，落地部署超过1000个项目。

各家的产品形态和商业化进度不尽相同，但这些案例已经把一个问题摆到了台前：当计算设备开始以整柜、多柜乃至万卡集群的规模组织起来，承载它们的机房，也需要相应的建设和交付能力。

计算、网络、供电和冷却需要一起考虑。预制化提供的办法，就是把能够提前完成的集成与验证工作放进工厂，让现场少一些逐项安装和协调。

03 数据中心，早就开始预制了

早在2006年，Sun就发布过Project Blackbox，将计算、存储、网络，以及配套供电和冷却设施装进标准集装箱，主打运输方便、快速部署。

当年的发布现场，就已经展示了接入外部电力、冷却水和网络后投入运行的思路。

换句话说，“机房装箱，运到即接”的思路，快到可以参加二十周年同学会了。把预制说成AI时代横空出世的新发明，多少有点替老技术补办出生证明。

如今，AI需求让这条老路线有了更大的用武之地。从供电、制冷等机电模块，到集装箱式机房，再到围绕特定计算平台进行整套集成，各家正在把预制推进到不同层级。

先来看已经开展规模化交付的模块化数据中心。以中集集团为例，公司在2026年3月30日的投资者关系活动记录中披露，其在中东、东南亚等地区累计交付的模块化数据中心规模超过1000MW、模块超过17000个；同时，正在为超过300MW的行业客户提供预制化数据中心的技术与制造交付服务。

中国速度的秘诀在于“双厂协同”与全流程一体化交付：90%以上的结构施工、机电安装和内部装修在国内工厂完成，现场只需像搭积木一样进行拼接。

国内华为3D数据中心，借鉴芯片工厂“动力层”与“IT层”严格分离的理念，通过分层解耦、立体堆叠，将传统水平部署的各个功能空间垂直叠加，形成“一层供冷、二层IT设备、三层供电、屋面备电”的垂直架构，实现垂直供电，垂直制冷，链路最短，损耗最小，支撑AI算力集群高密部署，同时为算力演进预留空间。

在建设模式上，华为将供电、制冷等核心机电系统做成标准化POD（可独立运行的最小数据中心单元），在工厂完成预制、组装和测试，现场主要进行装配和连接。其大小机电预制化率超过90%，机电交付时间由约6个月缩短至3个月。

立讯智算推出的是集装箱式预制数据中心方案。数据中心被拆分为一个个集装箱大小的标准模块，供电、散热、算力、组网、存储等都在工厂里提前集成、调试完毕，运到现场像搭积木一样拼接起来，接上电即可使用。一座方舱，就是一座开机即用的数据中心。

其中，能源模块将变配电与储能融为一体，负责稳定供电；液冷模块通过冷却液分配单元（CDU）与冷板管路，让冷却液直达芯片、带走热量；算力模块由成排的 GPU 整机柜组成，上电即有算力；组网模块基于正交组网（Orthogonal Fabric）将所有整机高速连通；租户模块则可按需配置，支持一池多租。客户需要什么就选配什么，钥匙一交，数据中心即刻上线。

浪潮的方案是预制化方舱数据中心 PrefabDC-C1000 系列 AIO 方舱。标准化箱体设计，主流规格涵盖 20 英尺、40 英尺国际标准尺寸，可根据算力需求灵活选型或组合扩展。据了解，已经在日本、沙特、哈萨克斯坦阿拉木图落地建设。

谈回这次的英伟达。DSX POD是把7个Vera Rubin机架连同线缆在工厂内预装成一个集装箱，合作伙伴可以在工厂完成机架间连接、管路安装和功能验证，再将预装模块运输到已经准备好的场地。参与这一方案的合作伙伴包括戴尔、HPE、联想、施耐德电气、维谛等，计划于2027年初开始交付。

按照英伟达提出的部署目标，在外部配套就绪的条件下，DSX POD可以将现场部署时间压缩至最短14天。提前完成的工厂集成与现场基础设施准备相互配合，共同推动AI工厂上线。

04 结语

预制数据中心始终在处理一个问题：怎样减少现场组装和集成的时间。

今天的变化，是需要提前集成的系统更加复杂，计算、网络、供电和冷却之间的配合也更加紧密。设备在工厂里多完成一道工序、多做一次验证，现场就能少一些安装和协调。

数据中心，早就开始预制了。如今这项老技术重新火起来，背后是一个越来越迫切的要求：算力设备在升级，机房的建设和交付也得跟上。

毕竟，GPU可以买新一代，机房总不能还停在上一代的工期里。

更多精彩内容，关注钛媒体微信号（ID：taimeiti），或者下载钛媒体App

2026-10-30 03:48:31 · 大模型,算力芯片,AI应用,融资,Meta,NVIDIA,百度,华为,代码生成,推理思考,设计创意,医疗健康,金融,教育学习,扩散模型,MoE架构,模型评测,端侧AI,工业制造,基础设施,模型发布,产品更新,合作,财报

Programmable chemoenzymatic synthesis of a keratan sulfate N-glycan library for glycocode decoding

PNASProceedings of the National Academy of Sciences, Volume 123, Issue 43, October 2026.
SignificanceKeratan sulfate (KS) is an important but understudied glycosaminoglycan implicated in various biological functions and disease processes. Its structural complexity in chain length, sulfation, sialylation, and branching has precluded functional ...

2026-10-20 07:00:00 · MoE架构

Tracking the Cyber Capabilities of Cheap Open-Weight Models

LessWrong

TL;DR

The Chinese company Xiaomi released MiMo-V2.6-Pro, an open-weight model with weak safety guardrails. It is extremely cheap while performing well on cyber-related tasks.
We evaluated MiMo on ExploitBench, where it ranked 3rd out of 9 models, behind Mythos (preview) and GPT-5.5. It outperforms several substantially more expensive models, although this comparison does not include the latest frontier models. But real performance might be worse due to benchmaxxing.
We believe real-world offensive capabilities may also exceed what benchmarks show.
- Human attackers can notice when a model gets stuck, combine different models, or ask a strong closed-source frontier model for targeted advice on subproblems that don't trigger refusals.
- Benchmarks are usually run offline, but real attackers can make models more capable by giving access to the internet and to PoC implementations of exploits. We found that MiMo could exploit a recently disclosed PHP vulnerability and gain access to a server in a test environment running the unpatched PHP version.

MiMo-V2.6

On 22 September, Xiaomi released MiMo-V2.6-Pro, a sparse Mixture-of-Experts (MoE) model with 1T total parameters and 42B active parameters. On benchmarks shared by the vendor, MiMo scored at a similar level to the latest models, even beating Claude Opus 5 on some. On CyberGym, the vendor reports scores of 94.0 for MiMo-V2.6-Pro and 95.1 for Flash, compared with 79.3 for Claude Opus 5. At the same time, it has a lower cost, roughly 11x/29x cheaper than Claude Opus 5 on input/output (2x/3x cheaper than DeepSeek V4 Pro).

However, Xiaomi doesn't share its evaluation methodology and there are some indications of benchmaxxing, such as the very high CyberGym score. We find that MiMo's performance strongly depends on the seed, and the same runs score 5.0/16 under best-of-3 but 3.6/16 under worst-of-3, a 38% difference from the scoring choice alone. We open-source our methodology (GitHub).

ExploitBench

ExploitBench measures how far an AI agent gets in exploiting real, already patched vulnerabilities in V8, the JavaScript engine inside Chrome and Node.js. The V8 sandbox is designed to protect the rest of the process by containing memory corruption inside V8. ExploitBench grades 16 capabilities across five tiers: reaching vulnerable code (T5), triggering the bug (T4), gaining limited control over memory inside the V8 sandbox (T3), reading or changing arbitrary memory outside the V8 sandbox (T2), and running arbitrary code in the V8 process (T1).

The cheap-capability finding

We evaluated MiMo-V2.6-Pro on all 41 ExploitBench CVEs. It outperformed several substantially more expensive models and achieved the highest capability per dollar among the models evaluated. Each task runs as a single agent in an isolated Docker sandbox with no network, a budget of 300 turns and 5 hours per run, scored over 3 seeds per CVE.

On the tier level, MiMo triggers the bug (T4) or better on 33 of the 41 CVEs and builds engine-level exploit primitives (T3) or better on 16, but it breaks out to arbitrary read/write (T2) on only one and never reaches code execution (T1). Its most common outcome is a triggered bug (T4, 17 CVEs). For eight CVEs, MiMo reached the vulnerable code (T5) but did not demonstrate that it could trigger the bug (T4).

Combining models

Real attackers can notice when a model gets stuck, switch to another model, or seek targeted guidance on a difficult subtask. We tested whether guidance from GLM-5.3-Prime could help MiMo overcome an obstacle on one ExploitBench task.

Task selection

We selected CVE-2024-2887 because model performance varied more widely on it than on any other task: scores ranged from 2/16 to 16/16. MiMo scored 8/16 on all three benchmark seeds, stalling at T3 inside the sandbox. GLM-5.3-Prime produced a full 16/16 solve in a separate run. We hypothesized that MiMo was missing a technique that guidance from GLM-5.3-Prime could supply.

Model and harness

We added an ask_oracle tool that MiMo could call once if it got stuck. GLM-5.3-Prime occasionally provided incorrect or misleading hints, even though it could solve the task. To avoid confounding the effect of guidance with the quality of GLM’s response, we fixed the tool’s response to a correct hint extracted from GLM’s successful solv

2026-10-28 22:14:13 · 大模型,算力芯片,AI应用,开源,OpenAI,Google,Anthropic,DeepSeek,智谱,Agent智能体,MoE架构,模型评测,端侧AI,招聘HR

继续滚动加载更多…