title: Categorified Jones-Wenzl Projectors and Generalizations

abstract: This preprint comprises the first four out of five chapters of the Master's thesis I wrote 2022 under the supervision of Catharina Stroppel and Paul Wedrich at the University of Bonn titled "Categorified Jones-Wenzl projectors and Generalizations". Chapter 1 can serve as introductory literature for researchers or graduate students familiar with Schur-Weyl duality between the symmetric group and the general linear Lie algebra or their quantum counter parts, and who are seeking to learn a type B/D analogue of the famous type A story. Chapter 1 serves as an introduction to the type B combinatorics and blob-diagrammatics. Chapters 2,3, and 4 contain proofs of three main theorem each - the first one is an explicit/computational proof of the easiest case of quantum coideal Schur-Weyl duality involving non-trivial quantizations of weight spaces, the second one shows the generalization of Jones-Wenzl projectors to the type B Temperley-Lieb algebra and the third one uses generalized Reidemeister moves and a functional analytic argument to prove a version of the fact that powers of the type D full-twist converge towards the type D Jones-Wenzl projector.


title: Automated Evaluation of Classroom Instructional Support with LLMs and BoWs: Connecting Global Predictions to Specific Feedback

abstract: With the aim to provide teachers with more specific, frequent, and actionable feedback about their teaching, we explore how Large Language Models (LLMs) can be used to estimate ``Instructional Support'' domain scores of the CLassroom Assessment Scoring System (CLASS), a widely used observation protocol. We design a machine learning architecture that uses either zero-shot prompting of Meta's Llama2, and/or a classic Bag of Words (BoW) model, to classify individual utterances of teachers' speech (transcribed automatically using OpenAI's Whisper) for the presence of Instructional Support. Then, these utterance-level judgments are aggregated over a 15-min observation session to estimate a global CLASS score. Experiments on two CLASS-coded datasets of toddler and pre-kindergarten classrooms indicate that (1) automatic CLASS Instructional Support estimation accuracy using the proposed method (Pearson $R$ up to $0.48$) approaches human inter-rater reliability (up to $R=0.55$); (2) LLMs generally yield slightly greater accuracy than BoW for this task, though the best models often combined features extracted from both LLM and BoW; and (3) for classifying individual utterances, there is still room for improvement of automated methods compared to human-level judgments. Finally, (4) we illustrate how the model's outputs can be visualized at the utterance level to provide teachers with explainable feedback on which utterances were most positively or negatively correlated with specific CLASS dimensions.


title: Optimal rates for ranking a permuted isotonic matrix in polynomial time

abstract: We consider a ranking problem where we have noisy observations from a matrix with isotonic columns whose rows have been permuted by some permutation $\pi$ *. This encompasses many models, including crowd-labeling and ranking in tournaments by pair-wise comparisons. In this work, we provide an optimal and polynomial-time procedure for recovering $\pi$ * , settling an open problem in [7]. As a byproduct, our procedure is used to improve the state-of-the art for ranking problems in the stochastically transitive model (SST). Our approach is based on iterative pairwise comparisons by suitable data-driven weighted means of the columns. These weights are built using a combination of spectral methods with new dimension-reduction techniques. In order to deal with the important case of missing data, we establish a new concentration inequality for sparse and centered rectangular Wishart-type matrices.


title: Existential Second-Order Logic Over Graphs: Parameterized Complexity

abstract: By Fagin's Theorem, NP contains precisely those problems that can be described by formulas starting with an existential second-order quantifier, followed by only first-order quantifiers (ESO formulas). Subsequent research refined this result, culminating in powerful theorems that characterize for each possible sequence of first-order quantifiers how difficult the described problem can be. We transfer this line of inquiry to the parameterized setting, where the size of the set quantified by the second-order quantifier is the parameter. Many natural parameterized problems can be described in this way using simple sequences of first-order quantifiers: For the clique or vertex cover problems, two universal first-order quantifiers suffice ("for all pairs of vertices ... must hold"); for the dominating set problem, a universal followed by an existential quantifier suffice ("for all vertices, there is a vertex such that ..."); and so on. We present a complete characterization that states for each possible sequence of first-order quantifiers how high the parameterized complexity of the described problems can be. The uncovered dividing line between quantifier sequences that lead to tractable versus intractable problems is distinct from that known from the classical setting, and it depends on whether the parameter is a lower bound on, an upper bound on, or equal to the size of the quantified set.


title: Paucity of rational points on fibrations with multiple fibres

abstract: Given a family of varieties over the projective line, we study the density of fibres that are everywhere locally soluble in the case that components of higher multiplicity are allowed. We use log geometry to formulate a new sparsity criterion for the existence of everywhere locally soluble fibres and formulate new conjectures that generalise previous work of Loughran-Smeets. These conjectures involve geometric invariants of the associated multiplicity orbifolds on the base of the fibration in the spirit of Campana. We give evidence for the conjectures using Chebotarev's theorem and sieve methods.


title: Deterministic Treasure Hunt and Rendezvous in Arbitrary Connected Graphs

abstract: Treasure hunt and rendezvous are fundamental tasks performed by mobile agents in graphs. In treasure hunt, an agent has to find an inert target (called treasure) situated at an unknown node of the graph. In rendezvous, two agents, initially located at distinct nodes of the graph, traverse its edges in synchronous rounds and have to meet at some node. We assume that the graph is connected (otherwise none of these tasks is feasible) and consider deterministic treasure hunt and rendezvous algorithms. The time of a treasure hunt algorithm is the worst-case number of edge traversals performed by the agent until the treasure is found. The time of a rendezvous algorithm is the worst-case number of rounds since the wakeup of the earlier agent until the meeting. To the best of our knowledge, all known treasure hunt and rendezvous algorithms rely on the assumption that degrees of all nodes are finite, even when the graph itself may be infinite. In the present paper we remove this assumption for the first time, and consider both above tasks in arbitrary connected graphs whose nodes can have either finite or countably infinite degrees. Our main result is the first universal treasure hunt algorithm working for arbitrary connected graphs. We prove that the time of this algorithm has optimal order of magnitude among all possible treasure hunt algorithms working for arbitrary connected graphs. As a consequence of this result we obtain the first universal rendezvous algorithm working for arbitrary connected graphs. The time of this algorithm is polynomial in a lower bound holding in many graphs, in particular in the tree all of whose degrees are infinite.


title: The $*$-exponential as a covering map

abstract: We employ tools from complex analysis to construct the $*$-logarithm of a quaternionic slice regular function. Our approach enables us to achieve three main objectives: we compute the monodromy associated with the $*$-exponential; we establish sufficient conditions for the $*$-product of two $*$-exponentials to also be a $*$-exponential; we calculate the slice derivative of the $*$-exponential of a regular function.


title: Target-Aware Contextual Political Bias Detection in News

abstract: Media bias detection requires comprehensive integration of information derived from multiple news sources. Sentence-level political bias detection in news is no exception, and has proven to be a challenging task that requires an understanding of bias in consideration of the context. Inspired by the fact that humans exhibit varying degrees of writing styles, resulting in a diverse range of statements with different local and global contexts, previous work in media bias detection has proposed augmentation techniques to exploit this fact. Despite their success, we observe that these techniques introduce noise by over-generalizing bias context boundaries, which hinders performance. To alleviate this issue, we propose techniques to more carefully search for context using a bias-sensitive, target-aware approach for data augmentation. Comprehensive experiments on the well-known BASIL dataset show that when combined with pre-trained models such as BERT, our augmentation techniques lead to state-of-the-art results. Our approach outperforms previous methods significantly, obtaining an F1-score of 58.15 over state-of-the-art bias detection task.


title: Stability and Generalization for Minibatch SGD and Local SGD

abstract: The increasing scale of data propels the popularity of leveraging parallelism to speed up the optimization. Minibatch stochastic gradient descent (minibatch SGD) and local SGD are two popular methods for parallel optimization. The existing theoretical studies show a linear speedup of these methods with respect to the number of machines, which, however, is measured by optimization errors. As a comparison, the stability and generalization of these methods are much less studied. In this paper, we study the stability and generalization analysis of minibatch and local SGD to understand their learnability by introducing a novel expectation-variance decomposition. We incorporate training errors into the stability analysis, which shows how small training errors help generalization for overparameterized models. We show both minibatch and local SGD achieve a linear speedup to attain the optimal risk bounds.


title: Neural Processing of Tri-Plane Hybrid Neural Fields

abstract: Driven by the appealing properties of neural fields for storing and communicating 3D data, the problem of directly processing them to address tasks such as classification and part segmentation has emerged and has been investigated in recent works. Early approaches employ neural fields parameterized by shared networks trained on the whole dataset, achieving good task performance but sacrificing reconstruction quality. To improve the latter, later methods focus on individual neural fields parameterized as large Multi-Layer Perceptrons (MLPs), which are, however, challenging to process due to the high dimensionality of the weight space, intrinsic weight space symmetries, and sensitivity to random initialization. Hence, results turn out significantly inferior to those achieved by processing explicit representations, e.g., point clouds or meshes. In the meantime, hybrid representations, in particular based on tri-planes, have emerged as a more effective and efficient alternative to realize neural fields, but their direct processing has not been investigated yet. In this paper, we show that the tri-plane discrete data structure encodes rich information, which can be effectively processed by standard deep-learning machinery. We define an extensive benchmark covering a diverse set of fields such as occupancy, signed/unsigned distance, and, for the first time, radiance fields. While processing a field with the same reconstruction quality, we achieve task performance far superior to frameworks that process large MLPs and, for the first time, almost on par with architectures handling explicit representations.


title: Obstructions for Gabor frames of the second order B-spline

abstract: For a window $g\in L^2(\mathbb{R})$, the subset of all lattice parameters $(a, b)\in \mathbb{R}^2_+$ such that $\mathcal{G}(g,a,b)=\{e^{2\pi ib m\cdot}g(\cdot-a k) : k, m\in\mathbb{Z}\}$ forms a frame for $L^2(\mathbb{R})$ is known as the frame set of $g$. In time-frequency analysis, determining the Gabor frame set for a given window is a challenging open problem. In particular, the frame set for B-splines has many obstructions. Lemvig and Nielsen in \cite{counter} conjectured that if \begin{align} a_0=\dfrac{1}{2m+1},~ b_0=\dfrac{2k+1}{2},~k,m\in \mathbb{N},~k>m,~a_0b_0<1,\nonumber \end{align} then the Gabor system $\mathcal{G}(Q_2, a, b)$ of the second order B-spline $Q_2$ is not a frame along the hyperbolas \begin{align} ab=\dfrac{2k+1}{2(2m+1)},\text{ for }b\in \left[b_0-a_0\dfrac{k-m}{2}, b_0+a_0\dfrac{k-m}{2}\right],\nonumber \end{align} for every $a_0$, $b_0$. Nielsen in \cite {Nielsenthesis} also conjectured that $\mathcal{G}(Q_2, a,b)$ is not a frame for $$a=\dfrac{1}{2m},~b=\dfrac{2k+1}{2},~k,m\in \mathbb{N},~k>m,~ab<1\text{ with }\gcd(4m,2k+1)=1.$$ In this paper, we prove that both conjectures are true.


title: [Re] CLRNet: Cross Layer Refinement Network for Lane Detection

abstract: The following work is a reproducibility report for CLRNet: Cross Layer Refinement Network for Lane Detection. The basic code was made available by the author. The paper proposes a novel Cross Layer Refinement Network to utilize both high and low level features for lane detection. The authors assert that the proposed technique sets the new state-of-the-art on three lane-detection benchmarks


title: Preliminary Performance Evaluation of a Satellite-to-HAP Communication Link

abstract: The emergence of Fifth-Generation (5G) communication networks has brought forth unprecedented connectivity with ultra-low latency, high data rates, and pervasive coverage. However, meeting the increasing demands of applications for seamless and high-quality communication, especially in rural areas, requires exploring innovative solutions that expand 5G beyond traditional terrestrial networks. Within the context of Non-Terrestrial Networks (NTNs), two promising technologies with vast potential are High Altitude Platforms (HAPs) and satellites. The combination of these two platforms is able to provide wide coverage and reliable communication in remote and inaccessible areas, and/or where terrestrial infrastructure is unavailable. This study evaluates the performance of the communication link between a Geostationary Equatorial Orbit (GEO) satellite and a HAP using the Internet of Drones Simulator (IoD-Sim), implemented in ns-3 and incorporating the 3GPP TR 38.811 channel model. The code base of IoD-Sim is extended to simulate HAPs, accounting for the Earths curvature in various geographic coordinate systems, and considering realistic mobility patterns. A simulation campaign is conducted to evaluate the GEO-to-HAP communication link in terms of Signal-to-Noise Ratio (SNR) in two different scenarios, considering the mobility of the HAP, and as a function of the frequency and the distance.


title: The Map Equation Goes Neural: Mapping Network Flows with Graph Neural Networks

abstract: Community detection is an essential tool for unsupervised data exploration and revealing the organisational structure of networked systems. With a long history in network science, community detection typically relies on objective functions, optimised with custom-tailored search algorithms, but often without leveraging recent advances in deep learning. Recently, first works have started incorporating such objectives into loss functions for neural graph clustering and pooling. We consider the map equation, a popular information-theoretic objective function for unsupervised community detection, and express it in differentiable tensor form for optimisation through gradient descent. Our formulation turns the map equation compatible with any neural network architecture, enables end-to-end learning, incorporates node features, and chooses the optimal number of clusters automatically, all without requiring explicit regularisation. Applied to unsupervised graph clustering tasks, we achieve competitive performance against state-of-the-art neural graph clustering baselines in synthetic and real-world datasets.


title: Parallel-in-Time Probabilistic Numerical ODE Solvers

abstract: Probabilistic numerical solvers for ordinary differential equations (ODEs) treat the numerical simulation of dynamical systems as problems of Bayesian state estimation. Aside from producing posterior distributions over ODE solutions and thereby quantifying the numerical approximation error of the method itself, one less-often noted advantage of this formalism is the algorithmic flexibility gained by formulating numerical simulation in the framework of Bayesian filtering and smoothing. In this paper, we leverage this flexibility and build on the time-parallel formulation of iterated extended Kalman smoothers to formulate a parallel-in-time probabilistic numerical ODE solver. Instead of simulating the dynamical system sequentially in time, as done by current probabilistic solvers, the proposed method processes all time steps in parallel and thereby reduces the span cost from linear to logarithmic in the number of time steps. We demonstrate the effectiveness of our approach on a variety of ODEs and compare it to a range of both classic and probabilistic numerical ODE solvers.


title: NewsRecLib: A PyTorch-Lightning Library for Neural News Recommendation

abstract: NewsRecLib is an open-source library based on Pytorch-Lightning and Hydra developed for training and evaluating neural news recommendation models. The foremost goals of NewsRecLib are to promote reproducible research and rigorous experimental evaluation by (i) providing a unified and highly configurable framework for exhaustive experimental studies and (ii) enabling a thorough analysis of the performance contribution of different model architecture components and training regimes. NewsRecLib is highly modular, allows specifying experiments in a single configuration file, and includes extensive logging facilities. Moreover, NewsRecLib provides out-of-the-box implementations of several prominent neural models, training methods, standard evaluation benchmarks, and evaluation metrics for news recommendation.


title: Optimizing Symbol Visibility through Displacement

abstract: In information visualization, the position of symbols often encodes associated data values. When visualizing data elements with both a numerical and a categorical dimension, positioning in the categorical axis admits some flexibility. This flexibility can be exploited to reduce symbol overlap, and thereby increase legibility. In this paper we initialize the algorithmic study of optimizing symbol legibility via a limited displacement of the symbols. Specifically, we consider unit square symbols that need to be placed at specified $y$-coordinates. We optimize the drawing order of the symbols as well as their $x$-displacement, constrained within a rectangular container, to maximize the minimum visible perimeter over all squares. If the container has width and height at most $2$, there is a point that stabs all squares. In this case, we prove that a staircase layout is arbitrarily close to optimality and can be computed in $O(n\log n)$ time. If the width is at most $2$, there is a vertical line that stabs all squares, and in this case, we give a 2-approximation algorithm that runs in $O(n\log n)$ time. As a minimum visible perimeter of $2$ is always trivially achievable, we measure this approximation with respect to the visible perimeter exceeding $2$. We show that, despite its simplicity, the algorithm gives asymptotically optimal results for certain instances.


title: Cryptocurrency Portfolio Optimization by Neural Networks

abstract: Many cryptocurrency brokers nowadays offer a variety of derivative assets that allow traders to perform hedging or speculation. This paper proposes an effective algorithm based on neural networks to take advantage of these investment products. The proposed algorithm constructs a portfolio that contains a pair of negatively correlated assets. A deep neural network, which outputs the allocation weight of each asset at a time interval, is trained to maximize the Sharpe ratio. A novel loss term is proposed to regulate the network's bias towards a specific asset, thus enforcing the network to learn an allocation strategy that is close to a minimum variance strategy. Extensive experiments were conducted using data collected from Binance spanning 19 months to evaluate the effectiveness of our approach. The backtest results show that the proposed algorithm can produce neural networks that are able to make profits in different market situations.


title: On $b$-Matching and Fully-Dynamic Maximum $k$-Edge Coloring

abstract: Given a graph $G$ that is modified by a sequence of edge insertions and deletions, we study the Maximum $k$-Edge Coloring problem Having access to $k$ colors, how can we color as many edges of $G$ as possible such that no two adjacent edges share the same color? While this problem is different from simply maintaining a $b$-matching with $b=k$, the two problems are closely related: a maximum $k$-matching always contains a $\frac{k+1}k$-approximate maximum $k$-edge coloring. However, maximum $b$-matching can be solved efficiently in the static setting, whereas the Maximum $k$-Edge Coloring problem is NP-hard and even APX-hard for $k \ge 2$. We present new results on both problems: For $b$-matching, we show a new integrality gap result and for the case where $b$ is a constant, we adapt Wajc's matching sparsification scheme~[STOC20]. Using these as basis, we give three new algorithms for the dynamic Maximum $k$-Edge Coloring problem: Our MatchO algorithm builds on the dynamic $(2+\epsilon)$-approximation algorithm of Bhattacharya, Gupta, and Mohan~[ESA17] for $b$-matching and achieves a $(2+\epsilon)\frac{k+1} k$-approximation in $O(poly(\log n, \epsilon^{-1}))$ update time against an oblivious adversary. Our MatchA algorithm builds on the dynamic $8$-approximation algorithm by Bhattacharya, Henzinger, and Italiano~[SODA15] for fractional $b$-matching and achieves a $(8+\epsilon)\frac{3k+3}{3k-1}$-approximation in $O(poly(\log n, \epsilon^{-1}))$ update time against an adaptive adversary. Moreover, our reductions use the dynamic $b$-matching algorithm as a black box, so any future improvement in the approximation ratio for dynamic $b$-matching will automatically translate into a better approximation ratio for our algorithms. Finally, we present a greedy algorithm that runs in $O(\Delta+k)$ update time, while guaranteeing a $2.16$~approximation factor.


title: Caf\'e conversations: A tribute to Norbert A'Campo

abstract: This is an intrusion in the life and the mathematics of Norbert A'Campo, intended to be a tribute to him and an acknowledgement of his impact on those who know him and his work. The final version of this paper appears in the book ``Essays in Geometry, Dedicated to Norbert A'Campo'', IRMA Lectures in Mathematics and Theoretical Physics, EMS Press, Berlin, 2023.


title: Computing Functions by Teams of Deterministic Finite Automata

abstract: We consider the task of computing functions $f: \mathbb{N}^k\to \mathbb{N}$, where $ \mathbb{N}$ is the set of natural numbers, by finite teams of agents modelled as deterministic finite automata. The computation is carried out in a distributed way, using the {\em discrete half-line}, which is the infinite graph with one node of degree 1 (called the root) and infinitely many nodes of degree 2. The node at distance $j$ from the root represents the integer $j$. We say that a team $\mathcal{A}^f$ of automata computes a function $f$, if in the beginning of the computation all automata from $\mathcal{A}^f$ are located at the arguments $x_1,\dots,x_k$ of the function $f$, in groups $\mathcal{A}^f _j$ at $x_j$, and at the end, all automata of the team gather at $f(x_1,\dots,x_k)$ and transit to a special state $STOP$. At each step of the computation, an automaton $a$ can ``see'' states of all automata colocated at the same node: the set of these states forms an input of $a$. Our main result shows that, for every primitive recursive function, there exists a finite team of automata that computes this function. We prove this by showing that basic primitive recursive functions can be computed by teams of automata, and that functions resulting from the operations of composition and of primitive recursion can be computed by teams of automata, provided that the ingredient functions of these operations can be computed by teams of automata. We also observe that cooperation between automata is necessary: even some very simple functions $f: \mathbb{N}\to \mathbb{N}$ cannot be computed by a single automaton.


title: Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectives

abstract: This paper provides a systematic analysis of the opportunities, challenges, and potential solutions of harnessing Large Language Models (LLMs) such as GPT-4 to dig out vulnerabilities within smart contracts based on our ongoing research. For the task of smart contract vulnerability detection, achieving practical usability hinges on identifying as many true vulnerabilities as possible while minimizing the number of false positives. Nonetheless, our empirical study reveals contradictory yet interesting findings: generating more answers with higher randomness largely boosts the likelihood of producing a correct answer but inevitably leads to a higher number of false positives. To mitigate this tension, we propose an adversarial framework dubbed GPTLens that breaks the conventional one-stage detection into two synergistic stages $-$ generation and discrimination, for progressive detection and refinement, wherein the LLM plays dual roles, i.e., auditor and critic, respectively. The goal of auditor is to yield a broad spectrum of vulnerabilities with the hope of encompassing the correct answer, whereas the goal of critic that evaluates the validity of identified vulnerabilities is to minimize the number of false positives. Experimental results and illustrative examples demonstrate that auditor and critic work together harmoniously to yield pronounced improvements over the conventional one-stage detection. GPTLens is intuitive, strategic, and entirely LLM-driven without relying on specialist expertise in smart contracts, showcasing its methodical generality and potential to detect a broad spectrum of vulnerabilities. Our code is available at: https://github.com/git-disl/GPTLens.


title: Post-hoc and Anytime Valid Permutation and Group Invariance Testing

abstract: We study post-hoc ($e$-value-based) and post-hoc anytime valid inference for testing exchangeability and general group invariance. Our methods satisfy a generalized Type I error control that permits a data-dependent selection of both the number of observations $n$ and the significance level $\alpha$. We derive a simple analytical expression for all exact post-hoc valid $p$-values for group invariance, which allows for a flexible plug-in of the test statistic. For post-hoc anytime validity, we derive sequential $p$-processes by multiplying post-hoc $p$-values. In sequential testing, it is key to specify how the number of observations may depend on the data. We propose two approaches, and show how they nest existing efforts. To construct good post-hoc $p$-values, we develop the theory of likelihood ratios for group invariance, and generalize existing optimality results. These likelihood ratios turn out to exist in different flavors depending on which space we specify our alternative. We illustrate our methods by testing against a Gaussian location shift, which yields an improved optimality result for the $t$-test when testing sphericity, connections to the softmax function when testing exchangeability, and an improved method for testing sign-symmetry.


title: Modularity in Deep Learning: A Survey

abstract: Modularity is a general principle present in many fields. It offers attractive advantages, including, among others, ease of conceptualization, interpretability, scalability, module combinability, and module reusability. The deep learning community has long sought to take inspiration from the modularity principle, either implicitly or explicitly. This interest has been increasing over recent years. We review the notion of modularity in deep learning around three axes: data, task, and model, which characterize the life cycle of deep learning. Data modularity refers to the observation or creation of data groups for various purposes. Task modularity refers to the decomposition of tasks into sub-tasks. Model modularity means that the architecture of a neural network system can be decomposed into identifiable modules. We describe different instantiations of the modularity principle, and we contextualize their advantages in different deep learning sub-fields. Finally, we conclude the paper with a discussion of the definition of modularity and directions for future research.


title: EIP-4844 Economics and Rollup Strategies

abstract: We study the economics of the Ethereum improvement proposal 4844 and its effect on rollups' data posting strategies. Rollups' cost consists of two parts: data posting and delay. In the new proposal, the data posting cost corresponds to a blob posting cost and is fixed in each block, no matter how much of the blob is utilized by the rollup. The tradeoff is clear: the rollup prefers to post a full blob, but if its transaction arrival rate is low, filling up a blob space causes too large delay cost. The first result of the paper shows that if a rollup transaction arrival rate is too low, it prefers to use the regular blockspace market for data posting, as it offers a more flexible cost structure. Second, we show that shared blob posting is not always beneficial for participating rollups and change in the aggregate blob posting cost in the equilibrium depends on the types of participating rollups. In the end, we discuss blob cost-sharing rules from an axiomatic angle.


title: Neural Fiber Activation in Unipolar vs Bipolar Deep Brain Stimulation

abstract: Deep Brain Stimulation (DBS) is an established and powerful treatment method in various neurological disorders. It involves chronically delivering electrical pulses to a certain stimulation target in the brain in order to alleviate the symptoms of a disease. Traditionally, the effect of DBS on neural tissue has been modeled based on the geometrical intersection of the static Volume of Tissue Activated (VTA) and the stimulation target. Recent studies suggest that the Dentato-Rubro-Thalamic Tract (DRTT) may serve as a potential common underlying stimulation target for tremor control in Essential Tremor (ET). However, clinical observations highlight that the therapeutic effect of DBS, especially in ET, is strongly influenced by the dynamic DBS parameters such as pulse width and frequency, as well as stimulation polarity. This study introduces a computational model to elucidate the effect of the stimulation signal shape on the DRTT under neural input. The simulation results suggest that achieving a specific pulse amplitude threshold is necessary before eliciting the therapeutic effect through adjustments in pulse widths and frequencies becomes feasible. Longer pulse widths proved more likely to induce firing, thus requiring a lower stimulation amplitude. Additionally, the modulation effect of bipolar configurations on neural traffic was found to vary significantly depending on the chosen stimulation polarity and the direction of neural traffic. Further, bipolar configurations demonstrated the ability to selectively influence firing patterns in different fiber tracts.


title: RRR-Net: Reusing, Reducing, and Recycling a Deep Backbone Network

abstract: It has become mainstream in computer vision and other machine learning domains to reuse backbone networks pre-trained on large datasets as preprocessors. Typically, the last layer is replaced by a shallow learning machine of sorts; the newly-added classification head and (optionally) deeper layers are fine-tuned on a new task. Due to its strong performance and simplicity, a common pre-trained backbone network is ResNet152.However, ResNet152 is relatively large and induces inference latency. In many cases, a compact and efficient backbone with similar performance would be preferable over a larger, slower one. This paper investigates techniques to reuse a pre-trained backbone with the objective of creating a smaller and faster model. Starting from a large ResNet152 backbone pre-trained on ImageNet, we first reduce it from 51 blocks to 5 blocks, reducing its number of parameters and FLOPs by more than 6 times, without significant performance degradation. Then, we split the model after 3 blocks into several branches, while preserving the same number of parameters and FLOPs, to create an ensemble of sub-networks to improve performance. Our experiments on a large benchmark of $40$ image classification datasets from various domains suggest that our techniques match the performance (if not better) of ``classical backbone fine-tuning'' while achieving a smaller model size and faster inference speed.


title: Invariants of genus 4 curves

abstract: The present paper gives an explicit classification of the isomorphism classes of non-hyperelliptic genus 4 curves over an algebraically closed field of characteristic 0. A non-hyperelliptic genus 4 curve lies on a quadric in $\mathbb{P^3}$ of rank 3 or 4. In the case of rank 3, we give a set of 60 invariants which classify the isomorphism classes, and in the case of rank 4, we find 65 invariants. These invariants are defined by transvectants and can be efficiently computed on a given example.


title: Iterative Semi-Supervised Learning for Abdominal Organs and Tumor Segmentation

abstract: Deep-learning (DL) based methods are playing an important role in the task of abdominal organs and tumors segmentation in CT scans. However, the large requirements of annotated datasets heavily limit its development. The FLARE23 challenge provides a large-scale dataset with both partially and fully annotated data, which also focuses on both segmentation accuracy and computational efficiency. In this study, we propose to use the strategy of Semi-Supervised Learning (SSL) and iterative pseudo labeling to address FLARE23. Initially, a deep model (nn-UNet) trained on datasets with complete organ annotations (about 220 scans) generates pseudo labels for the whole dataset. These pseudo labels are then employed to train a more powerful segmentation model. Employing the FLARE23 dataset, our approach achieves an average DSC score of 89.63% for organs and 46.07% for tumors on online validation leaderboard. For organ segmentation, We obtain 0.9007\% DSC and 0.9493\% NSD. For tumor segmentation, we obtain 0.3785% DSC and 0.2842% NSD. Our code is available at https://github.com/USTguy/Flare23.


title: Design, Modelling and Control of an Amphibious Quad-Rotor for Pipeline Inspection

abstract: Regular inspections are crucial to maintaining waste-water pipelines in good condition. The challenge is that inside a pipeline the space is narrow and may have a complex structure. The conventional methods that use pipe robots with heavy cables are expensive, time-consuming, and difficult to operate. In this work, we develop an amphibious system that combines a quad-copter with a surface vehicle, creating a hybrid unmanned aerial floating vehicle (HUAFV). Nonlinear dynamics of the HUAFV are modeled based on the dynamic models of both operating modes. The model is validated through experiments and simulations. A PI controller designed and tuned on the developed model is implemented onto a prototype platform. Our experiments demonstrate the effectiveness of the new HUAFV's modeling and design.


title: Centaurus A: Exploring the Nature of the Hard X-ray/Soft Gamma-ray Emission with INTEGRAL

abstract: The question of the origin of the hard X-ray/soft gamma-ray emission in Centaurus A (Cen A) persists despite decades of observations. Results from X-ray instruments suggest a jet origin since the implied electron temperature (kT_e) would cause pair production runaway in the corona. In contrast, instruments sensitive to soft gamma-rays report electron temperatures indicating a corona origin may be possible. In this context, we analyzed archival INTEGRAL/IBIS-ISGRI and SPI data and observations from a 2022 Cen A monitoring program. Our analysis did not find any spectral variability. Thus we combined all observations for long-term average spectra, which were fit with a NuSTAR observation to study the 3.5 keV - 2.2 MeV spectrum. Spectral fits using a CompTT model found kT_e ~ 550 keV, near pair-production runaway. The spectrum was also well described by a log-parabola to model synchrotron self-Compton emission from the jet. Additionally, a spectral fit with the 12-year catalog Fermi/LAT spectrum using a log-parabola can explain the data up to ~ 3 GeV. Above ~ 3 GeV, a power-law excess is present, which has been previously reported in LAT/H.E.S.S. analysis. However, including a corona spectral component can also describe the data well. In this scenario, the hard X-rays/soft gamma-rays are due the corona and the MeV to GeV emission is due to the jet.


title: DINE: Dimensional Interpretability of Node Embeddings

abstract: Graphs are ubiquitous due to their flexibility in representing social and technological systems as networks of interacting elements. Graph representation learning methods, such as node embeddings, are powerful approaches to map nodes into a latent vector space, allowing their use for various graph tasks. Despite their success, only few studies have focused on explaining node embeddings locally. Moreover, global explanations of node embeddings remain unexplored, limiting interpretability and debugging potentials. We address this gap by developing human-understandable explanations for dimensions in node embeddings. Towards that, we first develop new metrics that measure the global interpretability of embedding vectors based on the marginal contribution of the embedding dimensions to predicting graph structure. We say that an embedding dimension is more interpretable if it can faithfully map to an understandable sub-structure in the input graph - like community structure. Having observed that standard node embeddings have low interpretability, we then introduce DINE (Dimension-based Interpretable Node Embedding), a novel approach that can retrofit existing node embeddings by making them more interpretable without sacrificing their task performance. We conduct extensive experiments on synthetic and real-world graphs and show that we can simultaneously learn highly interpretable node embeddings with effective performance in link prediction.


title: Trust-Aware Motion Planning for Human-Robot Collaboration under Distribution Temporal Logic Specifications

abstract: Recent work has considered trust-aware decision making for human-robot collaboration (HRC) with a focus on model learning. In this paper, we are interested in enabling the HRC system to complete complex tasks specified using temporal logic that involve human trust. Since human trust in robots is not observable, we adopt the widely used partially observable Markov decision process (POMDP) framework for modelling the interactions between humans and robots. To specify the desired behaviour, we propose to use syntactically co-safe linear distribution temporal logic (scLDTL), a logic that is defined over predicates of states as well as belief states of partially observable systems. The incorporation of belief predicates in scLDTL enhances its expressiveness while simultaneously introducing added complexity. This also presents a new challenge as the belief predicates must be evaluated over the continuous (infinite) belief space. To address this challenge, we present an algorithm for solving the optimal policy synthesis problem. First, we enhance the belief MDP (derived by reformulating the POMDP) with a probabilistic labelling function. Then a product belief MDP is constructed between the probabilistically labelled belief MDP and the automaton translation of the scLDTL formula. Finally, we show that the optimal policy can be obtained by leveraging existing point-based value iteration algorithms with essential modifications. Human subject experiments with 21 participants on a driving simulator demonstrate the effectiveness of the proposed approach.


title: Segment Any Building

abstract: The task of identifying and segmenting buildings within remote sensing imagery has perennially stood at the forefront of scholarly investigations. This manuscript accentuates the potency of harnessing diversified datasets in tandem with cutting-edge representation learning paradigms for building segmentation in such images. Through the strategic amalgamation of disparate datasets, we have not only expanded the informational horizon accessible for model training but also manifested unparalleled performance metrics across multiple datasets. Our avant-garde joint training regimen underscores the merit of our approach, bearing significant implications in pivotal domains such as urban infrastructural development, disaster mitigation strategies, and ecological surveillance. Our methodology, predicated upon the fusion of datasets and gleaning insights from pre-trained models, carves a new benchmark in the annals of building segmentation endeavors. The outcomes of this research both fortify the foundations for ensuing scholarly pursuits and presage a horizon replete with innovative applications in the discipline of building segmentation.


title: Towards guarantees for parameter isolation in continual learning

abstract: Deep learning has proved to be a successful paradigm for solving many challenges in machine learning. However, deep neural networks fail when trained sequentially on multiple tasks, a shortcoming known as catastrophic forgetting in the continual learning literature. Despite a recent flourish of learning algorithms successfully addressing this problem, we find that provable guarantees against catastrophic forgetting are lacking. In this work, we study the relationship between learning and forgetting by looking at the geometry of neural networks' loss landscape. We offer a unifying perspective on a family of continual learning algorithms, namely methods based on parameter isolation, and we establish guarantees on catastrophic forgetting for some of them.


title: Gotcha! This Model Uses My Code! Evaluating Membership Leakage Risks in Code Models

abstract: Given large-scale source code datasets available in open-source projects and advanced large language models, recent code models have been proposed to address a series of critical software engineering tasks, such as program repair and code completion. The training data of the code models come from various sources, not only the publicly available source code, e.g., open-source projects on GitHub but also the private data such as the confidential source code from companies, which may contain sensitive information (for example, SSH keys and personal information). As a result, the use of these code models may raise new privacy concerns. In this paper, we focus on a critical yet not well-explored question on using code models: what is the risk of membership information leakage in code models? Membership information leakage refers to the risk that an attacker can infer whether a given data point is included in (i.e., a member of) the training data. To answer this question, we propose Gotcha, a novel membership inference attack method specifically for code models. We investigate the membership leakage risk of code models. Our results reveal a worrying fact that the risk of membership leakage is high: although the previous attack methods are close to random guessing, Gotcha can predict the data membership with a high true positive rate of 0.95 and a low false positive rate of 0.10. We also show that the attacker's knowledge of the victim model (e.g., the model architecture and the pre-training data) impacts the success rate of attacks. Further analysis demonstrates that changing the decoding strategy can mitigate the risk of membership leakage. This study calls for more attention to understanding the privacy of code models and developing more effective countermeasures against such attacks.


title: Nil-Hecke rings and the Schubert calculus

abstract: The purpose of the present notes is to give a self-contained exposition on the use of the techniques of Nil-Hecke algebras in the localization approach to the equivariant Schubert calculus for cohomology of flag varieties. We also demonstrate how these techniques can be applied to non-crystallographic root systems as well as to study (connective) $K$-theory of flag varieties.


title: Nucleon electric polarizabilities and nucleon-pion scattering at physical pion mass

abstract: We present a lattice QCD calculation of the nucleon electric polarizabilities at the physical pion mass. Our findings reveal the substantial contributions of the $N\pi$ states to these polarizabilities. Without considering these contributions, the lattice results fall significantly below the experimental values, consistent with previous lattice studies. This observation has motivated us to compute both the parity-negative $N\pi$ scattering length and $N\gamma^*\to N\pi$ matrix elements using lattice QCD. Our results confirm that the inclusion of dynamic $N\pi$ contributions allows for reliable determination of the polarizabilities from lattice QCD. This methodology lays the groundwork for future lattice QCD investigations into various other polarizabilities.


title: Fitting an ellipsoid to random points: predictions using the replica method

abstract: We consider the problem of fitting a centered ellipsoid to $n$ standard Gaussian random vectors in $\mathbb{R}^d$, as $n, d \to \infty$ with $n/d^2 \to \alpha > 0$. It has been conjectured that this problem is, with high probability, satisfiable (SAT; that is, there exists an ellipsoid passing through all $n$ points) for $\alpha < 1/4$, and unsatisfiable (UNSAT) for $\alpha > 1/4$. In this work we give a precise analytical argument, based on the non-rigorous replica method of statistical physics, that indeed predicts a SAT/UNSAT transition at $\alpha = 1/4$, as well as the shape of a typical fitting ellipsoid in the SAT phase (i.e., the lengths of its principal axes). Besides the replica method, our main tool is the dilute limit of extensive-rank "HCIZ integrals" of random matrix theory. We further study different explicit algorithmic constructions of the matrix characterizing the ellipsoid. In particular, we show that a procedure based on minimizing its nuclear norm yields a solution in the whole SAT phase. Finally, we characterize the SAT/UNSAT transition for ellipsoid fitting of a large class of rotationally-invariant random vectors. Our work suggests mathematically rigorous ways to analyze fitting ellipsoids to random vectors, which is the topic of a companion work.

categories: cond-mat.dis-nn cond-mat.stat-mech cs.DS math.PR math.ST stat.TH


title: Determination of $|V_{cb}|$ using $\overline{B}^0\to D^{*+}\ell^-\bar\nu_\ell$ decays with Belle II

abstract: We determine the CKM matrix-element magnitude $|V_{cb}|$ using $\overline{B}^0\to D^{*+}\ell^-\bar\nu_\ell$ decays reconstructed in $189 \, \mathrm{fb}^{-1}$ of collision data collected by the Belle II experiment, located at the SuperKEKB $e^+e^-$ collider. Partial decay rates are reported as functions of the recoil parameter $w$ and three decay angles separately for electron and muon final states. We obtain $|V_{cb}|$ using the Boyd-Grinstein-Lebed and Caprini-Lellouch-Neubert parametrizations, and find $|V_{cb}|_\mathrm{BGL}=(40.57\pm 0.31 \pm 0.95\pm 0.58)\times 10^{-3}$ and $|V_{cb}|_\mathrm{CLN}=(40.13 \pm 0.27 \pm 0.93\pm 0.58 )\times 10^{-3}$ with the uncertainties denoting statistical components, systematic components, and components from the lattice QCD input, respectively. The branching fraction is measured to be ${\cal B}(\overline{B}^0\to D^{*+}\ell^-\bar\nu_\ell)=(4.922 \pm 0.023 \pm 0.220)\%$. The ratio of branching fractions for electron and muon final states is found to be $0.998 \pm 0.009 \pm 0.020$. In addition, we determine the forward-backward angular asymmetry and the $D^{*+}$ longitudinal polarization fractions. All results are compatible with lepton-flavor universality in the Standard Model.


title: Estimation of the FR4 Microwave Dielectric Properties at Cryogenic Temperature for Quantum-Chip-Interface PCBs Design

abstract: Ad-hoc interface PCBs are today the standard connection between cryogenic cabling and quantum chips. Besides low-loss and low-temperature-dependent-dielectric-permittivity materials, FR4 provides a low-cost solution for fabrication of cryogenic PCBs. Here, we report on an effective way to evaluate the dielectric performance of a FR4 laminate used as substrate for cryogenic microwave PCBs. We designed a coplanar waveguide {\lambda}/2 open-circuit series resonator and we fabricated the PCB using a low-cost manufacturing process. Such a geometry allows to exploit the resonance peak of the resonator to measure the variation of the complex dielectric permittivity as a function of the temperature. Resonance peak frequency and magnitude were used as sensing parameters for the real part of dielectric permittivity and dielectric loss tangent, respectively. We estimated a 9 % reduction of the real part of the dielectric permittivity and a 70 % reduction of the dielectric loss tangent in the temperature range from 300 to 4 K. The proposed approach can be immediately extended to the detection of cryogenic temperature-dependent dielectric performance of any kind on substrate.


title: Stability conditions for mean-field limiting vorticities of the Ginzburg-Landau equations in 2D

abstract: We analyse the limit of stable solutions to the Ginzburg-Landau (GL) equations when $\varepsilon$, the inverse of the GL parameter, goes to zero and in a regime where the applied magnetic field is of order $|\log \varepsilon |$ whereas the total energy is of order $|\log \varepsilon|^2$. In order to do that we pass to the limit in the second inner variation of the GL energy. The main difficulty is to understand the convergence of quadratic terms involving derivatives of functions converging only weakly in $H^1$. We use an assumption of convergence of energies, the limiting criticality conditions obtained by Sandier-Serfaty by passing to the limit in the first inner variation and properties of limiting vorticities to find the limit of all the desired quadratic terms. At last we investigate the limiting stability condition we have obtained. In the case with magnetic field we study an example of an admissible limiting vorticity supported on a line in a square $\Omega=(-L,L)^2$ and show that if $L$ is small enough this vorticiy satisfies the limiting stability condition whereas when $L$ is large enough it stops verifying that condition. In the case without magnetic field we use a result of Iwaniec-Onninen to prove that every measure in $H^{-1}(\Omega)$ satisfying the first order limiting criticality condition also verifies the second order limiting stability condition.


title: Gradient COBRA: A kernel-based consensual aggregation for regression

abstract: In this article, we introduce a kernel-based consensual aggregation method for regression problems. We aim to exibly combine individual regression estimators $r_1, \ldots, r_M$ using a weighted average where the weights are dened based on predicted features given by all the basic estimators and some kernel function. This work extends the context of Biau et al. (2016) to a more general kernel-based framework. We show that this more general conguration also inherits the consistency of the basic consistent estimators, and the same convergence rate as in the classical method is achieved. Moreover, an optimization method based on gradient descent algorithm is proposed to eciently and rapidly estimate the key parameter of the strategy. Various numerical experiments carried out on several simulated and real datasets are also provided to illustrate the eciency and accuracy of the proposed method. Moreover, a domain adaptation-like property of the aggregation strategy is also illustrated on a physics data provided by Commissariat {\`a} l'{\'E}nergie Atomique (CEA).


title: Light Schr\"odinger Bridge

abstract: Despite the recent advances in the field of computational Schr\"odinger Bridges (SB), most existing SB solvers are still heavy-weighted and require complex optimization of several neural networks. It turns out that there is no principal solver which plays the role of simple-yet-effective baseline for SB just like, e.g., $k$-means method in clustering, logistic regression in classification or Sinkhorn algorithm in discrete optimal transport. We address this issue and propose a novel fast and simple SB solver. Our development is a smart combination of two ideas which recently appeared in the field: (a) parameterization of the Schr\"odinger potentials with sum-exp quadratic functions and (b) viewing the log-Schr\"odinger potentials as the energy functions. We show that combined together these ideas yield a lightweight, simulation-free and theoretically justified SB solver with a simple straightforward optimization objective. As a result, it allows solving SB in moderate dimensions in a matter of minutes on CPU without a painful hyperparameter selection. Our light solver resembles the Gaussian mixture model which is widely used for density estimation. Inspired by this similarity, we also prove an important theoretical result showing that our light solver is a universal approximator of SBs. Furthemore, we conduct the analysis of the generalization error of our light solver. The code for our solver can be found at https://github.com/ngushchin/LightSB


title: Homogenization of supremal functionals in the vectorial case (via $L^p$-approximation)

abstract: We propose a homogenized supremal functional rigorously derived via $L^p$-approximation by functionals of the type $\underset{x\in\Omega}{\mbox{ess-sup}}\hspace{0.03cm} f\left(\frac{x}{\varepsilon}, Du\right)$, when $\Omega$ is a bounded open set of $\mathbb R^n$ and $u\in W^{1,\infty}(\Omega;\mathbb R^d)$. The homogenized functional is also deduced directly in the case where the sublevel sets of $f(x,\cdot)$ satisfy suitable convexity properties, as a corollary of homogenization results dealing with pointwise gradient constrained integral functionals.


title: Cross-adversarial local distribution regularization for semi-supervised medical image segmentation

abstract: Medical semi-supervised segmentation is a technique where a model is trained to segment objects of interest in medical images with limited annotated data. Existing semi-supervised segmentation methods are usually based on the smoothness assumption. This assumption implies that the model output distributions of two similar data samples are encouraged to be invariant. In other words, the smoothness assumption states that similar samples (e.g., adding small perturbations to an image) should have similar outputs. In this paper, we introduce a novel cross-adversarial local distribution (Cross-ALD) regularization to further enhance the smoothness assumption for semi-supervised medical image segmentation task. We conducted comprehensive experiments that the Cross-ALD archives state-of-the-art performance against many recent methods on the public LA and ACDC datasets.


title: Escaping of Fast Radio Bursts

abstract: We reconsider the escape of high brightness coherent emission of Fast Radio Bursts (FRBs) from magnetars' magnetospheres, and conclude that there are numerous ways for the powerful FRB pulse to avoid nonlinear absorption. Sufficiently strong surface fields, $\geq 10\%$ of the quantum field, limit the waves' non-linearity to moderate values. For weaker fields, the electric field experienced by a particle is limited by a combined ponderomotive and parallel-adiabatic forward acceleration of charges by the incoming FRB pulse along the magnetic field lines newly opened during FRB/Coronal Mass Ejection (CME). As a result, particles surf the weaker front part of the pulse, experiencing low radiative losses, and are cleared from the magnetosphere for the bulk of the pulse to propagate. We also find: (i) for propagation across magnetic field, the O-mode suffers much smaller dissipation than the X-mode; (ii) quasi-parallel propagation suffers minimal dissipation; (iii) initial mildly relativistic radial plasma flow further reduces losses; (iv) for oblique propagation of a pulse with limited transverse size, the leading part of the pulse would ponderomotively sweep the plasma aside.


title: Geodesic tracking of gravitational collapse and migration of boson stars

abstract: Boson stars are potential sources of gravitational radiation. To improve our comprehension of these exotic objects, we present the study of geodesics within the space-times of stable, collapsing, and migrating boson stars. We focus on timelike geodesics that are initially circular or reciprocating. We verify that orbits initially bound within a stable boson stars persist in their bound states. For a collapsing boson star, we show that orbits initially bound and reciprocating finally either become unbound or plunge into the newly formed black hole, depending on their initial maximal radii. For initially circular geodesics, orbits with small radii plunge into the black hole; orbits with intermediate radii become unbound; and large radii orbits linger around the black hole with nonvanishing eccentricities. For the migrating case, a black hole does not form. In this case, the reciprocating orbits span a wider radial range. For initially circular geodesics, orbits with small radii become unbound, and orbits with large radii remain bound with nonvanishing eccentricities. This geodesic study provides a novel approach to investigate the gravitational collapse and migration of boson stars.


title: Tuning ultrasmall theranostic nanoparticles for MRI contrast and radiation dose amplification

abstract: Background: The introduction of magnetic resonance (MR)-guided radiation treatment planning has opened a new space for theranostic nanoparticles to reduce acute toxicity while improving local control. In this work, second-generation AGuIX nanoparticles (AGuIX-Bi) are synthesized and validated. AGuIX-Bi are shown to maintain MR positive contrast while further amplifying the radiation dose by the replacement of some Gd$^{3+}$ cations with higher Z Bi$^{3+}$. These next-generation nanoparticles are based on the AGuIX platform, which is currently being evaluated in multiple Phase II clinical trials in combination with radiotherapy. Methods: In this clinically scalable methodology, AGuIX is used as an initial chelation platform to exchange Gd$^{3+}$ for Bi$^{3+}$. AGuIX-Bi nanoparticles are synthesized with three ratios of Gd/Bi, each maintaining MR contrast while further amplifying radiation dose relative to Bi$^{3+}$. Safety, efficacy, and theranostic potential of the nanoparticles were evaluated in vitro and in vivo in a human non-small cell lung cancer model. Results: We demonstrated that increasing Bi$^{3+}$ in the nanoparticles is associated with more DNA damage and improves in vivo efficacy with a statistically significant delay in tumor growth and 33% complete regression for the largest Bi/Gd ratio tested. The addition of Bi$^{3+}$ by our synthetic method leads to nanoparticles that present slightly altered pharmacokinetics and lengthening of the period of high tumor accumulation with no observed evidence of toxicity. Conclusions: We confirmed the safety and enhanced efficacy of AGuIX-Bi with radiation therapy at the selected ratio of 30Gd/70Bi. These results provide crucial evidence towards patient translation.


title: Evolutionary Neural Architecture Search for Transformer in Knowledge Tracing

abstract: Knowledge tracing (KT) aims to trace students' knowledge states by predicting whether students answer correctly on exercises. Despite the excellent performance of existing Transformer-based KT approaches, they are criticized for the manually selected input features for fusion and the defect of single global context modelling to directly capture students' forgetting behavior in KT, when the related records are distant from the current record in terms of time. To address the issues, this paper first considers adding convolution operations to the Transformer to enhance its local context modelling ability used for students' forgetting behavior, then proposes an evolutionary neural architecture search approach to automate the input feature selection and automatically determine where to apply which operation for achieving the balancing of the local/global context modelling. In the search space, the original global path containing the attention module in Transformer is replaced with the sum of a global path and a local path that could contain different convolutions, and the selection of input features is also considered. To search the best architecture, we employ an effective evolutionary algorithm to explore the search space and also suggest a search space reduction strategy to accelerate the convergence of the algorithm. Experimental results on the two largest and most challenging education datasets demonstrate the effectiveness of the architecture found by the proposed approach.


title: Graph Isomorphic Networks for Assessing Reliability of the Medium-Voltage Grid

abstract: Ensuring electricity grid reliability becomes increasingly challenging with the shift towards renewable energy and declining conventional capacities. Distribution System Operators (DSOs) aim to achieve grid reliability by verifying the n-1 principle, ensuring continuous operation in case of component failure. Electricity networks' complex graph-based data holds crucial information for n-1 assessment: graph structure and data about stations/cables. Unlike traditional machine learning methods, Graph Neural Networks (GNNs) directly handle graph-structured data. This paper proposes using Graph Isomorphic Networks (GINs) for n-1 assessments in medium voltage grids. The GIN framework is designed to generalise to unseen grids and utilise graph structure and data about stations/cables. The proposed GIN approach demonstrates faster and more reliable grid assessments than a traditional mathematical optimisation approach, reducing prediction times by approximately a factor of 1000. The findings offer a promising approach to address computational challenges and enhance the reliability and efficiency of energy grid assessments.


title: Multivariate Singular Spectrum Analysis by Robust Diagonalwise Low-Rank Approximation

abstract: Multivariate Singular Spectrum Analysis (MSSA) is a powerful and widely used nonparametric method for multivariate time series, which allows the analysis of complex temporal data from diverse fields such as finance, healthcare, ecology, and engineering. However, MSSA lacks robustness against outliers because it relies on the singular value decomposition, which is very sensitive to the presence of anomalous values. MSSA can then give biased results and lead to erroneous conclusions. In this paper a new MSSA method is proposed, named RObust Diagonalwise Estimation of SSA (RODESSA), which is robust against the presence of cellwise and casewise outliers. In particular, the decomposition step of MSSA is replaced by a new robust low-rank approximation of the trajectory matrix that takes its special structure into account. A fast algorithm is constructed, and it is proved that each iteration step decreases the objective function. In order to visualize different types of outliers, a new graphical display is introduced, called an enhanced time series plot. An extensive Monte Carlo simulation study is performed to compare RODESSA with competing approaches in the literature. A real data example about temperature analysis in passenger railway vehicles demonstrates the practical utility of the proposed approach.


title: Status of the SST Camera for the Cherenkov Telescope Array

abstract: The Cherenkov Telescope Array will be the next generation ground-based gamma ray observatory in the energy range from a few tens of GeV to hundreds of TeV. It will be built on two sites, one for each hemisphere, to cover the entire sky. The observatory will consist of telescopes of three different sizes: large, medium and small, with primary reflectors of 23, 11.5 and 4.3 m in diameter, respectively. The Small-Sized Telescopes (SSTs) will focus on the highest energies; at least 37 (and up to 70) will be deployed at the southern site in Paranal, Chile, covering several square kilometers. They will have a Schwarzschild-Couder dual-mirror design, with a primary reflector of about 4 meters in diameter. This configuration leads to a compact camera, with a diameter of about 50 cm and a weight of less than 100 kg. Its focal plane consists of 2048 Silicon Photomultiplier pixels, each one read independently by a state-of-the-art full waveform readout. The camera design is now in the final stage and the first components are being tested. In this contribution we discuss the design choices, and present test results from latest developments.


title: Applications of Improvements to the Pythagorean Won-Loss Expectation in Optimizing Rosters

abstract: Bill James' Pythagorean formula has for decades done an excellent job estimating a baseball team's winning percentage from very little data: if the average runs scored and allowed are denoted respectively by ${\rm RS}$ and ${\rm RA}$, there is some $\gamma$ such that the winning percentage is approximately ${\rm RS}^\gamma / ({\rm RS}^\gamma + {\rm RA}^\gamma)$. One important consequence is to determine the value of different players to the team, as it allows us to estimate how many more wins we would have given a fixed increase in run production. We summarize earlier work on the subject, and extend the earlier theoretical model of Miller (who estimated the run distributions as arising from independent Weibull distributions with the same shape parameter; this has been observed to describe the observed run data well). We now model runs scored and allowed as being drawn from independent Weibull distributions where the shape parameter is not necessarily the same, and then use the Method of Moments to solve a system of four equations in four unknowns. Doing so yields a predicted winning percentage that is consistently better than earlier models over the last 30 MLB seasons (1994 to 2023). This comes at a small cost as we no longer have a closed form expression but must evaluate a two-dimensional integral of two Weibull distributions and numerically estimate the solutions to the system of equations; as these are trivial to do with simple computational programs it is well worth adopting this framework and avoiding the issues of implementing the Method of Least Squares or the Method of Maximum Likelihood.


title: Skyrmion stripes in twisted double bilayer graphene

abstract: Two dimensional moir\'e systems have recently emerged as a platform in which the interplay between topology and strong correlations of electrons play out in non-trivial ways. Among these systems, twisted double bilayer graphene (TDBG) is of particular interest as its topological properties may be tuned via both twist angle and applied perpendicular electric field. In this system, energy gaps are observed at half filling of particular bands, which can be associated with correlated spin polarized states. In this work, we investigate the fate of these states as the system is doped away from this filling. We demonstrate that, for a broad range of fractional fillings, the resulting ground state is partially valley polarized, and supports multiple broken symmetries, including a textured spin order indicative of skyrmions, with a novel $\textit{stripe}$ ordering that spontaneously breaks $C_3$ symmetry. Experimental signatures of this state are discussed.


title: Hypergraph anti-Ramsey theorems

abstract: The anti-Ramsey number $\mathrm{ar}(n,F)$ of an $r$-graph $F$ is the minimum number of colors needed to color the complete $n$-vertex $r$-graph to ensure the existence of a rainbow copy of $F$. We prove a general upper bound for $\mathrm{ar}(n,F)$ when $F$ is the expansion of a hypergraph with smaller uniformality, which refines the general bound $\mathrm{ar}(n,F) = \mathrm{ex}(n,F_{-}) + o(n^r)$ by Erd{\H o}s--Simonovits--S{\' o}s. Here $F_{-}$ is the family of $r$-graphs obtained from $F$ by removing one edge. We also determine the exact value of $\mathrm{ar}(n,F)$ for large $n$ when $F$ is the expansion of a complete graph, extending a result of Erd{\H o}s--Simonovits--S{\' o}s from graphs to hypergraphs.


title: Fluid-structure model for disks vibrating at ultra-high frequency in a compressible viscous fluid

abstract: Radial mechanical modes of miniature disk-shaped resonators are promising candidates for probing the ultra-high-frequency rheological properties of liquids. However, the lack of an analytical fluid-structure model hinders the inference of liquid properties from the experimental measurement of such radial vibrations. Here we develop analytical models for the case of a disk vibrating in a compressible viscous liquid. Closed-form expressions for the mechanical quality factor and resonant frequency shift upon immersion are obtained, which we compare to the results of numerical modeling for a few significant cases. At frequencies above 1 GHz, our model points out the significance of compressibility effects.


title: Quantifying the Plausibility of Context Reliance in Neural Machine Translation

abstract: Establishing whether language models can use contextual information in a human-plausible way is important to ensure their trustworthiness in real-world settings. However, the questions of when and which parts of the context affect model generations are typically tackled separately, with current plausibility evaluations being practically limited to a handful of artificial benchmarks. To address this, we introduce Plausibility Evaluation of Context Reliance (PECoRe), an end-to-end interpretability framework designed to quantify context usage in language models' generations. Our approach leverages model internals to (i) contrastively identify context-sensitive target tokens in generated texts and (ii) link them to contextual cues justifying their prediction. We use \pecore to quantify the plausibility of context-aware machine translation models, comparing model rationales with human annotations across several discourse-level phenomena. Finally, we apply our method to unannotated model translations to identify context-mediated predictions and highlight instances of (im)plausible context usage throughout generation.


title: If there is no underfitting, there is no Cold Posterior Effect

abstract: The cold posterior effect (CPE) (Wenzel et al., 2020) in Bayesian deep learning shows that, for posteriors with a temperature $T<1$, the resulting posterior predictive could have better performances than the Bayesian posterior ($T=1$). As the Bayesian posterior is known to be optimal under perfect model specification, many recent works have studied the presence of CPE as a model misspecification problem, arising from the prior and/or from the likelihood function. In this work, we provide a more nuanced understanding of the CPE as we show that misspecification leads to CPE only when the resulting Bayesian posterior underfits. In fact, we theoretically show that if there is no underfitting, there is no CPE.


title: Graph-Theoretic B\'ezier Curve Optimization over Safe Corridors for Safe and Smooth Motion Planning

abstract: As a parametric motion representation, B\'ezier curves have significant applications in polynomial trajectory optimization for safe and smooth motion planning of various robotic systems, including flying drones, autonomous vehicles, and robotic manipulators. An essential component of B\'ezier curve optimization is the optimization objective, as it significantly influences the resulting robot motion. Standard physical optimization objectives, such as minimizing total velocity, acceleration, jerk, and snap, are known to yield quadratic optimization of B\'ezier curve control points. In this paper, we present a unifying graph-theoretic perspective for defining and understanding B\'ezier curve optimization objectives using a consensus distance of B\'ezier control points derived based on their interaction graph Laplacian. In addition to demonstrating how standard physical optimization objectives define a consensus distance between B\'ezier control points, we also introduce geometric and statistical optimization objectives as alternative consensus distances, constructed using finite differencing and differential variance. To compare these optimization objectives, we apply B\'ezier curve optimization over convex polygonal safe corridors that are automatically constructed around a maximal-clearance minimal-length reference path. We provide an explicit analytical formulation for quadratic optimization of B\'ezier curves using B\'ezier matrix operations. We conclude that the norm and variance of the finite differences of B\'ezier control points lead to simpler and more intuitive interaction graphs and optimization objectives compared to B\'ezier derivative norms, despite having similar robot motion profiles.


title: Unveiling Symmetries Patterns: A Study of Circular and Linear Harmonic Oscillator Chains

abstract: The purpose of this article is the study of the symmetries in a circular and linear harmonic oscillator chains system, and consequently use them as a means to find the eigenvalues of these configurations. Furthermore, a hidden $\mathbb{Z}_2$ group structure arises in both problems, showing how a degenerate spectrum in the circular case is attributable to the specific geometry producing a $\mathbb{Z}_N$ symmetry.


title: Eigenvalues and resonances of dissipative acoustic operator for strictly convex obstacles

abstract: We examine the wave equation in the exterior of a strictly convex bounded domain $K$ with dissipative boundary condition $\partial_{\nu} u - \gamma(x) \partial_t u = 0$ on the boundary $\Gamma$ and $0 < \gamma(x) <1, \:\forall x \in \Gamma.$ The solutions are described by a contraction semigroup $V(t) = e^{tG}, \: t \geq 0.$ The poles of the meromorphic incoming resolvent $(G - \lambda)^{-1}: \: {\mathcal H}_{comp} \rightarrow {\mathcal D}_{loc}$ are eigenvalues of G if ${\rm Re}\: \lambda < 0$ and incoming resonances if ${\rm Re}\: \lambda > 0$. We obtain sharper results for the location of the eigenvalues of $G$ and incoming resonances in $\Lambda = \{\lambda \in {\mathbb C}:\: |{\rm Re}\: \lambda| \leq C_2(1 + |{\rm Im}\: \lambda|)^{-2},\: |{\rm Im}\: \lambda| \geq A_2 > 1\}$ and we prove a Weyl formula for their asymptotic. For $K = \{x \in {\mathbb R}^3:\:|x| \leq 1\}$ and $\gamma$ constant we show that $G$ has no eigenvalues so the Weyl formula concerns only the incoming resonances.


title: The primitive equations with rough transport noise: Global well-posedness and regularity

abstract: In this paper we establish global well-posedness and instantaneous regularization results for the primitive equations with transport noise of H\"{o}lder regularity $ \gamma>\frac{1}{2}$. It is known that if $\gamma<1$, then the noise is too rough for a strong formulation of primitive equations in an $L^2$-based setting. To handle rough noise, we crucially use $L^q$-techniques with $q> 2$. Interestingly, we identify a family of critical anisotropic Besov spaces for primitive equations, which is new even in the deterministic case. The behavior of these spaces reflects the intrinsic anisotropy of the primitive equations and plays an essential role in establishing global well-posedness and regularization. Our results cover Kraichnan's type noise with correlation greater than one, and as a by-product, a 2D noise reproducing the Kolmogorov spectrum of turbulence. Moreover, the instantaneous regularization is new also in the widely studied case of $H^1$-data and $\gamma>1 $.


title: Reducing Hyperexponential Functions over Monomial Extensions

abstract: We extend the shell and kernel reductions for hyperexponential functions over the field of rational functions to a monomial extension. Both of the reductions are incorporated into one algorithm. As an application, we present an additive decomposition in rationally hyperexponential towers. The decomposition yields an alternative algorithm for computing elementary integrals over such towers. The alternative can find some elementary integrals that are unevaluated by the integrators in the latest versions of Maple and Mathematica.


title: Federated K-means Clustering

abstract: Federated learning is a technique that enables the use of distributed datasets for machine learning purposes without requiring data to be pooled, thereby better preserving privacy and ownership of the data. While supervised FL research has grown substantially over the last years, unsupervised FL methods remain scarce. This work introduces an algorithm which implements K-means clustering in a federated manner, addressing the challenges of varying number of clusters between centers, as well as convergence on less separable datasets.


title: Optimal transport for mesh adaptivity and shock capturing of compressible flows

abstract: We present an optimal transport approach for mesh adaptivity and shock capturing of compressible flows. Shock capturing is based on a viscosity regularization of the governing equations by introducing an artificial viscosity field as solution of the Helmholtz equation. Mesh adaptation is based on the optimal transport theory by formulating a mesh mapping as solution of Monge-Ampere equation. The marriage of optimal transport and viscosity regularization for compressible flows leads to a coupled system of the compressible Euler/Navier-Stokes equations, the Helmholtz equation, and the Monge-Ampere equation. We propose an iterative procedure to solve the coupled system in a sequential fashion using homotopy continuation to minimize the amount of artificial viscosity while enforcing positivity-preserving and smoothness constraints on the numerical solution. We explore various mesh monitor functions for computing r-adaptive meshes in order to reduce the amount of artificial dissipation and improve the accuracy of the numerical solution. The hybridizable discontinuous Galerkin method is used for the spatial discretization of the governing equations to obtain high-order accurate solutions. Extensive numerical results are presented to demonstrate the optimal transport approach on transonic, supersonic, hypersonic flows in two dimensions. The approach is found to yield accurate, sharp yet smooth solutions within a few mesh adaptation iterations.


title: Temperature inhomogeneities in Mrk71 can not be discarded

abstract: In a very recent work, [1] claim that the scenario of temperature inhomogeneities proposed by [2] ($t2$ > 0) is not able to explain the O$^{2+}$/H$^{+}$ abundance discrepancy observed between the calculations based on the optical [OIII] collisional excited lines (CELs) and the OII recombination lines (RLs) in the star forming galaxy Mrk71. In this work, we show that conclusions of [1] depend on several assumptions on the absolute flux calibration, reddening correction and the adopted electron density. In fact, using the data of [1] in a different way and even considering their 1{\sigma} uncertainties, it is possible to reach the opposite conclusion, consistent with $t2$ = $0.097 ^{+0.008}_{-0.009}$. Therefore, the existence of temperature inhomogeneities causing the O$^{2+}$/H$^{+}$ abundance discrepancy in Mrk71 can not be ruled out.


title: Likelihood Based Inference for ARMA Models

abstract: Autoregressive moving average (ARMA) models are frequently used to analyze time series data. Despite the popularity of these models, likelihood-based inference for ARMA models has subtleties that have been previously identified but continue to cause difficulties in widely used data analysis strategies. We provide a summary of parameter estimation via maximum likelihood and discuss common pitfalls that may lead to sub-optimal parameter estimates. We propose a random initialization algorithm for parameter estimation that frequently yields higher likelihoods than traditional maximum likelihood estimation procedures. We then investigate the parameter uncertainty of maximum likelihood estimates, and propose the use of profile confidence intervals as a superior alternative to intervals derived from the Fisher's information matrix. Through a series of simulation studies, we demonstrate the efficacy of our proposed algorithm and the improved nominal coverage of profile confidence intervals compared to the normal approximation based on Fisher's Information.


title: A robust algorithm for computational floating body dynamics

abstract: We present a non-iterative algorithm, FloatStepper, for coupling the motion of a rigid body and an incompressible fluid in computational fluid dynamics (CFD) simulations. The purpose of the algorithm is to remove the so-called added mass instability problem, which may arise when a light floating body interacts with a heavy fluid. The idea underlying the presented coupling method is to precede every computational time step by a series of prescribed probe body motions in which the fluid response is determined, thus revealing the decomposition of the net force and torque into two components: 1) An added mass contribution proportional to the instantaneous body acceleration, and 2) all other forces and torques. The algorithm is implemented and released as an open source extension module to the widely used CFD toolbox, OpenFOAM, as an alternative to the existing body motion solvers. The accuracy of the algorithm is investigated with several single-phase and two-phase flow benchmark cases. The benchmarks demonstrate excellent stability properties, allowing simulations even with massless bodies. They also highlight aspects of the implementation, such as the mesh motion method, where more work is needed to further enhance the flexibility and predictive capabilities of the code.


title: A worldwide overview on the information security posture of online public services

abstract: The availability of public services through online platforms has improved the coverage and efficiency of essential services provided to citizens worldwide. These services also promote transparency and foster citizen participation in government processes. However, the increased online presence also exposes sensitive data exchanged between citizens and service providers to a wider range of security threats. Therefore, ensuring the security and trustworthiness of online services is crucial to Electronic Government (EGOV) initiatives' success. Hence, this work assesses the security posture of online platforms hosted in 3068 governmental domain names, across all UN Member States, in three dimensions: support for secure communication protocols; the trustworthiness of their digital certificate chains; and services' exposure to known vulnerabilities. The results indicate that despite its rapid development, the public sector still falls short in adopting international standards and best security practices in services and infrastructure management. This reality poses significant risks to citizens and services across all regions and income levels.


title: SWoTTeD: An Extension of Tensor Decomposition to Temporal Phenotyping

abstract: Tensor decomposition has recently been gaining attention in the machine learning community for the analysis of individual traces, such as Electronic Health Records (EHR). However, this task becomes significantly more difficult when the data follows complex temporal patterns. This paper introduces the notion of a temporal phenotype as an arrangement of features over time and it proposes SWoTTeD (Sliding Window for Temporal Tensor Decomposition), a novel method to discover hidden temporal patterns. SWoTTeD integrates several constraints and regularizations to enhance the interpretability of the extracted phenotypes. We validate our proposal using both synthetic and real-world datasets, and we present an original usecase using data from the Greater Paris University Hospital. The results show that SWoTTeD achieves at least as accurate reconstruction as recent state-of-the-art tensor decomposition models, and extracts temporal phenotypes that are meaningful for clinicians.


title: Unified Uncertainty Calibration

abstract: To build robust, fair, and safe AI systems, we would like our classifiers to say ``I don't know'' when facing test examples that are difficult or fall outside of the training classes.The ubiquitous strategy to predict under uncertainty is the simplistic \emph{reject-or-classify} rule: abstain from prediction if epistemic uncertainty is high, classify otherwise.Unfortunately, this recipe does not allow different sources of uncertainty to communicate with each other, produces miscalibrated predictions, and it does not allow to correct for misspecifications in our uncertainty estimates. To address these three issues, we introduce \emph{unified uncertainty calibration (U2C)}, a holistic framework to combine aleatoric and epistemic uncertainties. U2C enables a clean learning-theoretical analysis of uncertainty estimation, and outperforms reject-or-classify across a variety of ImageNet benchmarks. Our code is available at: https://github.com/facebookresearch/UnifiedUncertaintyCalibration


title: Escape from an Orbiting Pursuer with a Nonzero Capture Radius

abstract: This paper explores a multi-agent containment problem, where a fast evader, modeled having constant speed and using constant heading, attempts to escape a circular containment region that is orbited by a slower pursuer with a nonzero capture radius. The pursuer is constrained to move along the edge of the containment region and seeks to capture the evader. This paper presents an in-depth analysis of this pursuer-evader containment scenario. First, multiple types of capture conditions for a single-pursuer case are analyzed defining the worst-case initial position for the pursuer. Second, a parametric study is performed to demonstrate the effects of speed ratio, capture radius, and initial location of the evader. Finally, a reachability analysis is performed to investigate the viable escape headings and reachable regions by the evader. This work provides a foundation for the analysis of escape under more general evader inputs as well as a multiple-pursuer version of the scenario.


title: The multi-channel Dyson equation: coupling many-body Green's functions

abstract: We present the multi-channel Dyson equation that combines two or more many-body Green's functions to describe the electronic structure of materials. In this work we use it to model photoemission spectra by coupling the one-body Green's function with the three-body Green's function. We demonstrate that, unlike methods using only the one-body Green's function, our approach puts the description of quasiparticles and satellites on an equal footing. We propose a multi-channel self-energy that is static and only contains the bare Coulomb interaction, making frequency convolutions and self-consistency unnecessary. Despite its simplicity, we demonstrate with a diagrammatic analysis that the physics it describes is extremely rich. Finally, we present a framework based on an effective Hamiltonian that can be solved for any many-body system using standard numerical tools. We illustrate our approach by applying it to the Hubbard dimer and show that it is exact both at 1/4 and 1/2 filling.


title: Local disclosure of quantum memory in non-Markovian dynamics

abstract: Non-Markovian processes may arise in physics due to memory effects of environmental degrees of freedom. For quantum non-Markovianity, it is an ongoing debate to clarify whether such memory effects have a verifiable quantum origin, or whether they might equally be modeled by a classical memory. In this contribution, we propose a criterion to test locally for a truly quantum memory. The approach is agnostic with respect to the environment, as it solely depends on the local dynamics of the system of interest. Experimental realizations are particularly easy, as only single-time measurements on the system itself have to be performed. We study memory in a variety of physically motivated examples, both for a time-discrete case, and for time-continuous dynamics. For the latter, we are able to provide an interesting class of non-Markovian master equations with classical memory that allows for a physically measurable quantum trajectory representation.


title: appjsonify: An Academic Paper PDF-to-JSON Conversion Toolkit

abstract: We present appjsonify, a Python-based PDF-to-JSON conversion toolkit for academic papers. It parses a PDF file using several visual-based document layout analysis models and rule-based text processing approaches. appjsonify is a flexible tool that allows users to easily configure the processing pipeline to handle a specific format of a paper they wish to process. We are publicly releasing appjsonify as an easy-to-install toolkit available via PyPI and GitHub.


title: Learn to Follow: Decentralized Lifelong Multi-agent Pathfinding via Planning and Learning

abstract: Multi-agent Pathfinding (MAPF) problem generally asks to find a set of conflict-free paths for a set of agents confined to a graph and is typically solved in a centralized fashion. Conversely, in this work, we investigate the decentralized MAPF setting, when the central controller that posses all the information on the agents' locations and goals is absent and the agents have to sequientially decide the actions on their own without having access to a full state of the environment. We focus on the practically important lifelong variant of MAPF, which involves continuously assigning new goals to the agents upon arrival to the previous ones. To address this complex problem, we propose a method that integrates two complementary approaches: planning with heuristic search and reinforcement learning through policy optimization. Planning is utilized to construct and re-plan individual paths. We enhance our planning algorithm with a dedicated technique tailored to avoid congestion and increase the throughput of the system. We employ reinforcement learning to discover the collision avoidance policies that effectively guide the agents along the paths. The policy is implemented as a neural network and is effectively trained without any reward-shaping or external guidance. We evaluate our method on a wide range of setups comparing it to the state-of-the-art solvers. The results show that our method consistently outperforms the learnable competitors, showing higher throughput and better ability to generalize to the maps that were unseen at the training stage. Moreover our solver outperforms a rule-based one in terms of throughput and is an order of magnitude faster than a state-of-the-art search-based solver.


title: Label Supervised LLaMA Finetuning

abstract: The recent success of Large Language Models (LLMs) has gained significant attention in both academia and industry. Substantial efforts have been made to enhance the zero- and few-shot generalization capabilities of open-source LLMs through finetuning. Currently, the prevailing approach is instruction-tuning, which trains LLMs to complete real-world tasks by generating responses guided by natural language instructions. It is worth noticing that such an approach may underperform in sequence and token classification tasks. Unlike text generation tasks, classification tasks have a limited label space, where precise label prediction is more appreciated than generating diverse and human-like responses. Prior research has unveiled that instruction-tuned LLMs cannot outperform BERT, prompting us to explore the potential of leveraging latent representations from LLMs for supervised label prediction. In this paper, we introduce a label-supervised adaptation for LLMs, which aims to finetuning the model with discriminant labels. We evaluate this approach with Label Supervised LLaMA (LS-LLaMA), based on LLaMA-2-7B, a relatively small-scale LLM, and can be finetuned on a single GeForce RTX4090 GPU. We extract latent representations from the final LLaMA layer and project them into the label space to compute the cross-entropy loss. The model is finetuned by Low-Rank Adaptation (LoRA) to minimize this loss. Remarkably, without intricate prompt engineering or external knowledge, LS-LLaMA substantially outperforms LLMs ten times its size in scale and demonstrates consistent improvements compared to robust baselines like BERT-Large and RoBERTa-Large in text classification. Moreover, by removing the causal mask from decoders, LS-unLLaMA achieves the state-of-the-art performance in named entity recognition (NER). Our work will shed light on a novel approach to adapting LLMs for various downstream tasks.


title: Self-distilled Masked Attention guided masked image modeling with noise Regularized Teacher (SMART) for medical image analysis

abstract: Hierarchical shifted window transformers (Swin) are a computationally efficient and more accurate alternative to plain vision transformers. Masked image modeling (MIM)-based pretraining is highly effective in increasing models' transferability to a variety of downstream tasks. However, more accurate and efficient attention guided MIM approaches are difficult to implement with Swin due to it's lack of an explicit global attention. We thus architecturally enhanced Swin with semantic class attention for self-supervised attention guided co-distillation with MIM. We also introduced a noise injected momentum teacher, implemented with patch dropout of teacher's inputs for improved training regularization and accuracy. Our approach, called \underline{s}elf-distilled \underline{m}asked \underline{a}ttention MIM with noise \underline{r}egularized \underline{t}eacher (SMART) was pretrained with \textbf{10,412} unlabeled 3D computed tomography (CT)s of multiple disease sites and sourced from institutional and public datasets. We evaluated SMART for multiple downstream tasks involving analysis of 3D CTs of lung cancer (LC) patients for: (i) [Task I] predicting immunotherapy response in advanced stage LC (n = 200 internal dataset), (ii) [Task II] predicting LC recurrence in early stage LC before surgery (n = 156 public dataset), (iii) [Task III] LC segmentation (n = 200 internal, 21 public dataset), and (iv) [Task IV] unsupervised clustering of organs in the chest and abdomen (n = 1,743 public dataset) \underline{without} finetuning. SMART predicted immunotherapy response with an AUC of 0.916, LC recurrence with an AUC of 0.793, segmented LC with Dice accuracy of 0.81, and clustered organs with an inter-class cluster distance of 5.94, indicating capability of attention guided MIM for Swin in medical image analysis.


title: Towards Robust Cardiac Segmentation using Graph Convolutional Networks

abstract: Fully automatic cardiac segmentation can be a fast and reproducible method to extract clinical measurements from an echocardiography examination. The U-Net architecture is the current state-of-the-art deep learning architecture for medical segmentation and can segment cardiac structures in real-time with average errors comparable to inter-observer variability. However, this architecture still generates large outliers that are often anatomically incorrect. This work uses the concept of graph convolutional neural networks that predict the contour points of the structures of interest instead of labeling each pixel. We propose a graph architecture that uses two convolutional rings based on cardiac anatomy and show that this eliminates anatomical incorrect multi-structure segmentations on the publicly available CAMUS dataset. Additionally, this work contributes with an ablation study on the graph convolutional architecture and an evaluation of clinical measurements on the clinical HUNT4 dataset. Finally, we propose to use the inter-model agreement of the U-Net and the graph network as a predictor of both the input and segmentation quality. We show this predictor can detect out-of-distribution and unsuitable input images in real-time. Source code is available online: https://github.com/gillesvntnu/GCN_multistructure


title: From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communication

abstract: It has been observed that representations learned by distinct neural networks conceal structural similarities when the models are trained under similar inductive biases. From a geometric perspective, identifying the classes of transformations and the related invariances that connect these representations is fundamental to unlocking applications, such as merging, stitching, and reusing different neural modules. However, estimating task-specific transformations a priori can be challenging and expensive due to several factors (e.g., weights initialization, training hyperparameters, or data modality). To this end, we introduce a versatile method to directly incorporate a set of invariances into the representations, constructing a product space of invariant components on top of the latent representations without requiring prior knowledge about the optimal invariance to infuse. We validate our solution on classification and reconstruction tasks, observing consistent latent similarity and downstream performance improvements in a zero-shot stitching setting. The experimental analysis comprises three modalities (vision, text, and graphs), twelve pretrained foundational models, nine benchmarks, and several architectures trained from scratch.


title: Enabling predictable parallelism in single-GPU systems with persistent CUDA threads

abstract: Graphics Processing Unit, or GPUs, have been successfully adopted both for graphic computation in 3D applications, and for general purpose application (GP-GPUs), thank to their tremendous performance-per-watt. Recently, there is a big interest in adopting them also within automotive and avionic industrial settings, imposing for the first time real-time constraints on the design of such devices. Unfortunately, it is extremely hard to extract timing guarantees from modern GPU designs, and current approaches rely on a model where the GPU is treated as a unique monolithic execution device. Unlike state-of-the-art of research, we try to open the box of modern GPU architectures, providing a clean way to exploit intra-GPU predictable execution.