摩根大通：“开源冲击”、“AI安全”都不是问题，未来两年资本开支仍有空间，半导体设备将成“新瓶颈”
华尔街见闻摩根大通认为，开源模型非威胁、监管扰动属短期、云厂商杠杆仍低——算力投资基本面未变。其预测七大科技巨头资本开支将从2025年4430亿美元飙至2027年1.577万亿美元，半导体设备将成供应链最核心瓶颈，晶圆代工、先进封装迎来新一轮涨价周期。
MS MARCO click-translation expansion tables ("poor man's" DSSM) [P]
Reddit r/MachineLearning submitted by /u/SpiritedTrip
[link] [comments]
TLDR: I made "poor man’s" DSSM (Deep Structured Semantic Model) — the count-based translation table that can enrich the inverted index for full-text search. This trick can improve baseline BM25.
So the idea is the following:
- You have supervised pairs (query, relevant document), e.g., MS MARCO or click logs.
- You tokenize both sides into some units (char n‑grams, wordpieces, words).
- You count cross‑pair co‑occurrences: unit u on the document side vs. unit v on the query side (not co‑occurrence within the same text).
- For each document‑side unit u, you keep the top‑k query‑side units v with the strongest association.
- At indexing time, each document gets postings not only for its own units, but also for the top‑k associated units of each of its units — i.e., document expansion baked into the inverted index.
It’s like mixing synonyms into the search query (but it’s not a synonyms exactly). The one difference from the DSSM is that it can only handle linear dependencies whilst DSSM can do the non-linear one.
And so it improves the performance over BM25 baseline.
I packed it as hf model repo: https://huggingface.co/mirth/msmarco-expansion-tables with a small usage demo script.
I am not claiming that this is a new idea. I made it because it’s fun and I’m planning to use it in my own search engine project.
[link] [comments]
Nonnegative matrix factorizations and related compositional models: Equivalence, identifiability, and an application on the grain-size analysis of sediments
arXiv cs.LGarXiv:2512.22282v2 Announce Type: replace-cross
Abstract: Across fields such as machine learning, social science, and geology, considerable attention has been given to models that factorize a nonnegative matrix into the product of two or three matrices, subject to nonnegative or row-sum-to-1 constraints. Although these models are to a large extent similar or even equivalent, they are presented under different names, and their similarity is not well known. This paper highlights similarities among five models, latent budget analysis (LBA) and latent class analysis (LCA) from social science, end-member analysis (EMA) from geology, probabilistic latent semantic analysis (PLSA) and nonnegative matrix factorization (NMF) from machine learning. We focus on the identifiability of these models. We prove that the solution of LBA, EMA, LCA, PLSA is unique if and only if the solution of NMF is unique. Consequently, existing uniqueness theorems for NMF directly apply to LBA, EMA, LCA, PLSA, and vice versa. We also provide a brief review of algorithms for the estimation of these models. We illustrate NMF on a sedimentary grain-size distribution dataset from sedimentary geology, and end the paper with a discussion of closely related model: archetypal analysis.
The Orthogonalized Read Is a Removable Training Scaffold for Recurrent Memory
arXiv cs.LGarXiv:2607.19390v3 Announce Type: replace
Abstract: Orthogonalizing the mLSTM memory matrix at read time with five differentiable Newton-Schulz iterations improves noisy associative recall. We replicate this effect and investigate its mechanism. Training on MAD noisy recall exhibits a long chance-level plateau followed by a sharp increase in accuracy. The orthogonalized read improves conditioning during this plateau and can be removed after escape. Ablations support three findings. First, the benefit requires a self-consistent read and gradient: an exact recursive least-squares read (the Mesa layer) yields a similar benefit, while straight-through variants, delta-rule writes, frozen random keys, and Frobenius normalization show no improvement over baseline. Second, across a learning-rate x task-difficulty grid, orthogonalization multiplies escape hazard roughly six-fold, with no detectable dependence on difficulty, and widens the range of learning rates that produce successful runs. Third, adding orthogonalization at inference leaves chance-level failures unresolved, while removing it gradually after escape yields standard mLSTMs at near-perfect accuracy. Schedule changes alone recover much of the reported gain. A batch-size x learning-rate analysis separates the effects of per-step learning rate and gradient noise on escape hazard (elasticities +3.0 and -1.65, respectively), linking the original vocab-96 result to its large-batch training regime. Direct decoding of the memory state recovers roughly half of the associations in behaviorally failed models, indicating a readout-learning limitation despite substantial stored information. These results show that fixed-budget recall benchmarks are sensitive to trainability and provide a tractable setting for investigating abrupt behavioral transitions through measurements of internal representations.
EviDep: Uncertainty-Aware Multimodal Depression Estimation via Disentangled Evidential Learning
arXiv cs.LGarXiv:2604.16579v3 Announce Type: replace
Abstract: Audio--visual recordings provide complementary cues for estimating depression severity, but their informativeness varies across time and modalities. Point predictions alone do not express the uncertainty associated with these estimates. We present EviDep, a multimodal evidential regression framework that integrates multi-scale temporal modeling and shared--private representation learning for uncertainty-aware depression estimation. Frequency-aware Feature Extraction decomposes behavioral feature sequences into multiple frequency bands and refines them with scale-specific experts. Disentangled Evidential Learning encourages the disentanglement of cross-modal shared and modality-specific information in the refined features. Multi-branch Evidential Regression maps the resulting shared and private representations to three Normal-Inverse-Gamma (NIG) outputs and uses evidence-weighted aggregation to estimate depression severity and quantify aleatoric and epistemic uncertainty. Experiments on AVEC 2013, AVEC 2014, DAIC-WOZ, and E-DAIC show competitive prediction accuracy, with ablation studies supporting the contributions of frequency-aware refinement and shared--private disentanglement. Further analyses show that estimated epistemic uncertainty helps identify higher-error predictions, while both uncertainty estimates generally increase under controlled feature degradation.
PRISM: Parallel Residual Iterative Sequence Model
arXiv cs.LGarXiv:2602.10796v4 Announce Type: replace
Abstract: Generative sequence modeling faces a fundamental tension between the expressivity of Transformers and the efficiency of linear sequence models. Existing efficient architectures are theoretically bounded by shallow, single-step linear updates, while powerful iterative methods like Test-Time Training (TTT) break hardware parallelism due to two dimensions of serial dependency: token-level state reliance and step-level iteration loops. We propose PRISM (Parallel Residual Iterative Sequence Model) to resolve this tension. PRISM explicitly approximates the expressive gate-residual-direction iteration pattern of TTT in a parallelizable form. We employ a Write-Forget Decoupling strategy that isolates non-linearity within the injection operator. To bypass the serial dependency of explicit solvers, PRISM utilizes a two-stage proxy architecture: a short-convolution anchors the initial residual using local history energy, while a learned predictor estimates the refinement updates directly from the input. This design distills structural patterns associated with iterative correction into a parallelizable feedforward operator. Theoretically, we prove that this formulation achieves Rank-$L$ accumulation, structurally expanding the update scheme beyond the single-step Rank-$1$ bottleneck. Empirically, it achieves comparable performance to explicit optimization methods while achieving \textbf{174x higher throughput}. Codes are available in https://github.com/gpr-prism/prism/.
Explainable Graph-theoretical Machine Learning with Application to Alzheimer's Disease Prediction
arXiv cs.LGarXiv:2503.16286v2 Announce Type: replace
Abstract: Dementia affects over 55 million people worldwide, projected to reach 139 million by 2050, with Alzheimer's disease (AD) accounting for 60-70% of cases. AD is associated with disruptions in metabolic brain connectivity. Detecting these disruptions early is crucial for AD management. FDG-PET is a useful tool for identifying such impairments. However, most studies rely on group-level analyses or thresholding, potentially masking individual differences and overlooking weaker yet biologically critical brain connections. Moreover, AD prediction largely focuses on univariate rather than multivariate outcomes. To address this, we introduce explainable graph-theoretical machine learning (XGML), a framework for constructing individual metabolic brain graphs and identifying subgraphs most predictive of multivariate disease-related outcomes. Using Alzheimer's Disease Neuroimaging Initiative (ADNI) FDG-PET data, we compared six graph representations against three non-graph baselines, each with six machine learning models using repeated stratified 3-fold cross-validation (10 repeats). The best configuration combined kernel density estimation with Hellinger distance and random forest. Across eight cognitive scores, it reached an overall Fisher-z-averaged Pearson correlation of r=0.595, with strongest performance for ADAS13 (r=0.67), ADAS11 (r=0.65), and ADASQ4 (r=0.62). We identified key edges that were jointly but differentially predictive across outcomes, suggesting their potential as network biomarkers of cognitive decline. Preliminary external feasibility validation on an OASIS3 cohort yielded weak predictive performance for CDRSB (r=0.26) and MMSE (r=0.18), likely reflecting cohort, protocol, and diagnostic differences. Overall, our results suggest the promise of graph-theoretical machine learning for biomarker discovery, disease prediction, and understanding the neural mechanisms underlying AD.
Attention is All You Need Until You Need Retention
arXiv cs.LGarXiv:2501.09166v2 Announce Type: replace
Abstract: Pretrained Transformers keep what they learned in their weights and lose what they observe once a session ends. The first version of this paper proposed a Retention Layer, a persistent memory that a Transformer block reads with attention and writes during use. Because most of what a deployed model could retain is produced by other agents, this revision treats deciding what to keep as a social learning problem: when to rely on observed behaviour, whom to learn from and how much independent agreement to require. We give a corrected specification of the layer, which reduces exactly to the base Transformer when its memory is empty. We derive the memory's lifecycle from social learning strategies: encoding gated by surprise, observed outcomes and earned credibility; consolidation by a credibility weighted quorum of distinct, recent sources that must also outweigh every rival behaviour; and reconsolidation by the outcomes of reproduction. We prove that raising the quorum lowers the risk of consolidating a coordinated false template exponentially while delaying true templates only linearly, and that relative consolidation protects only while credible honest evidence arrives faster than adversarial evidence. In a simulation with world drift and three memory-poisoning attacks, the lifecycle reached accuracies of 0.989 to 0.996, against 0.62 to 0.63 for the ungated first version design, and kept attack success at or below 0.07 when 30% of the observations about a target were adversarial. As predicted, it amplified attacks once adversarial evidence outpaced honest evidence. Experience with a long running assistant adds two rules: a model's own outputs must not count as support, and a user's testimony should be kept after one mention. We close with an evaluation protocol for language models.
Bias-Induced Crossover in Absolute Capacity of Dense Associative Memory
arXiv cs.LGarXiv:2609.17477v1 Announce Type: cross
Abstract: The absolute capacity of dense associative memory has mainly been analyzed for unbiased patterns. Here we examine the effect of bias in centered binary patterns under the Krotov-Hopfield single-site criterion $P_{\mathrm{error}}=1/N$, where $P_{\mathrm{error}}$ is the probability that a single-site flip lowers the energy of a stored pattern and $N$ is the number of neurons. Each pattern component takes $1-q$ with probability $q$ and $-q$ otherwise, where $0
<1/2$, however, the capacity is $O(N^{n/2})$ for even $n\ge4$ and $O(N^{(n+1)/2})$ for odd $n\ge5$. For $n=3$, both the unbiased and fixed-bias capacities remain $O(N^2/\ln N)$. For $n\ge4$, these different asymptotic forms imply a nonuniform large-$N$ limit near $q=1/2$. Asymptotic matching predicts a bias-induced crossover in the region $1-2q=O(\ln N/N^{\lfloor n/2\rfloor-1})$. The crossover originates from a bias-dependent crosstalk mean that reduces the stability of sites carrying the more frequent value $-q$. Computer simulations are compared with the finite-size conditioned-Gaussian predictions. An activity-dependent control potential that cancels the conditional crosstalk mean restores the $N^{n-1}/\ln N$ capacity for fixed $0<1/2$ within the conditioned-Gaussian approximation.
Kernel-Based Metrics Learning for Uncertain Opponent Vehicle Trajectory Prediction in Autonomous Racing
arXiv cs.LGarXiv:2609.17147v1 Announce Type: cross
Abstract: Autonomous racing confronts significant challenges in safely overtaking Opponent Vehicles (OVs) that exhibit uncertain trajectories, stemming from unknown driving policies. To address these challenges, this study proposes heterogeneous kernel metrics for Deep Kernel Learning (DKL), designed to robustly capture the diverse driving policies of OVs, and carry out precise trajectory predictions along with the associated uncertainties. A key virtue of the proposed kernel metrics lies in their ability to align similar driving policies and disjoin dissimilar ones in an unsupervised manner, given the observed interactions between the Ego Vehicle (EV) and OVs. The efficacy of the proposed method is substantiated through experimental studies on a 1/10th scale racecar platform, demonstrating improved prediction accuracy and thereby safely overtaking against OVs. Furthermore, our method is computationally efficient for onboard computing units, affirming its viability in fast-paced racing environments. The video and source code can be found at https://github.com/HMCL-UNIST/OpponentPredictionWithKMDKL.git.
Symmetric solution of the Bellman optimality equation for repeated harmony game
arXiv cs.LGarXiv:2609.16289v1 Announce Type: cross
Abstract: In social dilemma games, additional rewards or punishments have been studied as means of promoting cooperation. Therefore, it is important to investigate the ideal situation, in which such an additional payoff would change the game. In this study, we investigated the symmetric solution of the Bellman optimality equation for a repeated harmony game. The calculations showed that three types of symmetric solutions exist. One of them corresponds to the trivial All-C strategy, and another to the Win-stay Lose-shift strategy of the prisoners dilemma game. The nontrivial behavior of the strategy corresponding to the last solution is also discussed in detail. In addition, we numerically investigated which strategy the agents actually learn by the reinforcement learning algorithm.
Semantic-Aware Neural Video Codec for Error-Resilient Low-Latency Transmission
arXiv cs.LGarXiv:2609.16279v1 Announce Type: cross
Abstract: Emerging physical AI systems require low-latency, task-oriented video communication over unreliable channels. We propose a semantic-aware multi-level neural video coding method for robust low-latency video transmission over unreliable channels that are abstracted as multi-level packet erasure channels. Built upon the real-time DCVC-RT neural video codec, the proposed framework introduces a semantic- and feature-aware coding strategy that partitions encoded representations into packets carrying different levels of semantic and latent-feature importance and assigns these packets to different streams, each associated with a priority level when transmitted over unreliable communication channels. We also developed an error-resilient entropy model that removes inter-packet dependencies, allowing each packet to be decoded independently under packet losses. The complete system is trained end-to-end over the abstracted multi-level packet erasure channels, enabling learning of channel-aware representations together with importance-aware packet assignment while facilitating the network for differentiated packet prioritization. Experiments show that the proposed framework significantly improves robustness over baseline DCVC-RT under packet erasures, achieving graceful degradation in less important regions while better preserving task-relevant visual content.
Permutation-Based Stegomalware in Large Language Models: Threats and Countermeasures
arXiv cs.LGarXiv:2609.16193v1 Announce Type: cross
Abstract: The difficulty of training large language models (LLMs), together with their ubiquity, raises the threat of stegomalware, where malicious payloads are embedded into model weights. Recent work has demonstrated the use of permutation symmetry in model weights to mitigate these threats, but failed to show neutralization of stegomalware across all weights for LLMs. In this paper, we demonstrate the full potential of behavior-preserving symmetries as a defense against stegomalware, as well as the risks these symmetries pose when exploited by attackers.
For stegomalware neutralization, we improve upon previous work, demonstrating that it is possible to select permutations which displace all model parameters. This contrasts with previous methods which left a significant percentage of weights unaltered in LLMs. When used in an attack, we show that permutation symmetries can encode malware into the weights of a model in a way that is theoretically lossless, requires no retraining after encoding, and needs no payload-specific information in the extraction script---a combination of characteristics not previously seen in any single method.
While theoretically lossless, permutation can in practice alter model behavior due to the accumulation of numerical error. We therefore quantify the loss in model performance associated with applying these methods, for both attack and defense, showing it to be minimal.
Predicting Social Media Engagement using Machine Learning
arXiv cs.LGarXiv:2609.16082v1 Announce Type: cross
Abstract: Social media platforms are popular channels for disseminating information, owing to their large user bases and ease of access. Companies also use social media as an important aspect of the advertising process. By creating high-quality posts, companies can strengthen their engagement metrics and increase their follower count. While a growing body of research has examined social media engagement, fewer studies have jointly examined the visual, textual, and temporal features of image posts, even though these features collectively determine the performance of content on social media. To understand the important drivers of social media engagement, we collect image posts of furniture firms on Facebook and extract visual, temporal, and textual features from them using text and image analytics methods. We evaluate several machine learning models - including Random Forest, Light Gradient Boosting Machine (LightGBM), and eXtreme Gradient Boosting (XGBoost) - to assess the drivers and the prediction power of social media engagement using the features from our data. Our research quantifies the extent to which these features are associated with interactions and provides recommendations that organizations may consider.
Digital Persuasion: Understanding the Impact of Online Influencers on Public Opinion
arXiv cs.LGarXiv:2609.16062v1 Announce Type: cross
Abstract: The studying of opinion dynamics and its propagation within social networks is crucial for addressing a wide range of challenges, including political polarization, public health, and marketing strategies. In this work, we study the problem of opinion dynamics by proposing a framework based on Friedkin-Johnsen (FJ) to identifies influential users and study their impact on dynamics opinions of community. The FJ model assume each individual have two opinions: initial and expressed. Through a series of initial opinion manipulation experiments, the proposed framework assesses the impact of influential versus random users on the overall community opinion. The proposed framework is validated using a tweet dataset representing the U.S. presidential election. The results shows that influencers with highest influencing score, significantly shift the overall community opinion. Moreover, the results shows that the impact of influencers not limited to direct neighbors , but beyond it, to their neighbors of neighbors . This study demonstrates how digital influencers on social media can shape public opinion regarding a subject or cause.
继续滚动加载更多…