title: HumanEval on Latest GPT Models -- 2024

abstract: In 2023, we are using the latest models of GPT-4 to advance program synthesis. The large language models have significantly improved the state-of-the-art for this purpose. To make these advancements more accessible, we have created a repository that connects these models to Huamn Eval. This dataset was initally developed to be used with a language model called CODEGEN on natural and programming language data. The utility of these trained models is showcased by demonstrating their competitive performance in zero-shot Python code generation on HumanEval tasks compared to previous state-of-the-art solutions. Additionally, this gives way to developing more multi-step paradigm synthesis. This benchmark features 160 diverse problem sets factorized into multistep prompts that our analysis shows significantly improves program synthesis over single-turn inputs. All code is open source at https://github.com/daniel442li/gpt-human-eval .


title: NL2Formula: Generating Spreadsheet Formulas from Natural Language Queries

abstract: Writing formulas on spreadsheets, such as Microsoft Excel and Google Sheets, is a widespread practice among users performing data analysis. However, crafting formulas on spreadsheets remains a tedious and error-prone task for many end-users, particularly when dealing with complex operations. To alleviate the burden associated with writing spreadsheet formulas, this paper introduces a novel benchmark task called NL2Formula, with the aim to generate executable formulas that are grounded on a spreadsheet table, given a Natural Language (NL) query as input. To accomplish this, we construct a comprehensive dataset consisting of 70,799 paired NL queries and corresponding spreadsheet formulas, covering 21,670 tables and 37 types of formula functions. We realize the NL2Formula task by providing a sequence-to-sequence baseline implementation called fCoder. Experimental results validate the effectiveness of fCoder, demonstrating its superior performance compared to the baseline models. Furthermore, we also compare fCoder with an initial GPT-3.5 model (i.e., text-davinci-003). Lastly, through in-depth error analysis, we identify potential challenges in the NL2Formula task and advocate for further investigation.


title: A Dual-Prompting for Interpretable Mental Health Language Models

abstract: Despite the increasing demand for AI-based mental health monitoring tools, their practical utility for clinicians is limited by the lack of interpretability.The CLPsych 2024 Shared Task (Chim et al., 2024) aims to enhance the interpretability of Large Language Models (LLMs), particularly in mental health analysis, by providing evidence of suicidality through linguistic content. We propose a dual-prompting approach: (i) Knowledge-aware evidence extraction by leveraging the expert identity and a suicide dictionary with a mental health-specific LLM; and (ii) Evidence summarization by employing an LLM-based consistency evaluator. Comprehensive experiments demonstrate the effectiveness of combining domain-specific information, revealing performance improvements and the approach's potential to aid clinicians in assessing mental state progression.


title: An LLM Maturity Model for Reliable and Transparent Text-to-Query

abstract: Recognizing the imperative to address the reliability and transparency issues of Large Language Models (LLM), this work proposes an LLM maturity model tailored for text-to-query applications. This maturity model seeks to fill the existing void in evaluating LLMs in such applications by incorporating dimensions beyond mere correctness or accuracy. Moreover, this work introduces a real-world use case from the law enforcement domain and showcases QueryIQ, an LLM-powered, domain-specific text-to-query assistant to expedite user workflows and reveal hidden relationship in data.


title: Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoning

abstract: Deductive reasoning plays a pivotal role in the formulation of sound and cohesive arguments. It allows individuals to draw conclusions that logically follow, given the truth value of the information provided. Recent progress in the domain of large language models (LLMs) has showcased their capability in executing deductive reasoning tasks. Nonetheless, a significant portion of research primarily assesses the accuracy of LLMs in solving such tasks, often overlooking a deeper analysis of their reasoning behavior. In this study, we draw upon principles from cognitive psychology to examine inferential strategies employed by LLMs, through a detailed evaluation of their responses to propositional logic problems. Our findings indicate that LLMs display reasoning patterns akin to those observed in humans, including strategies like $\textit{supposition following}$ or $\textit{chain construction}$. Moreover, our research demonstrates that the architecture and scale of the model significantly affect its preferred method of reasoning, with more advanced models tending to adopt strategies more frequently than less sophisticated ones. Importantly, we assert that a model's accuracy, that is the correctness of its final conclusion, does not necessarily reflect the validity of its reasoning process. This distinction underscores the necessity for more nuanced evaluation procedures in the field.


title: Is the System Message Really Important to Jailbreaks in Large Language Models?

abstract: The rapid evolution of Large Language Models (LLMs) has rendered them indispensable in modern society. While security measures are typically to align LLMs with human values prior to release, recent studies have unveiled a concerning phenomenon named "Jailbreak". This term refers to the unexpected and potentially harmful responses generated by LLMs when prompted with malicious questions. Most existing research focus on generating jailbreak prompts but system message configurations vary significantly in experiments. In this paper, we aim to answer a question: Is the system message really important for jailbreaks in LLMs? We conduct experiments in mainstream LLMs to generate jailbreak prompts with varying system messages: short, long, and none. We discover that different system messages have distinct resistances to jailbreaks. Therefore, we explore the transferability of jailbreaks across LLMs with different system messages. Furthermore, we propose the System Messages Evolutionary Algorithm (SMEA) to generate system messages that are more resistant to jailbreak prompts, even with minor changes. Through SMEA, we get a robust system messages population with little change in the length of system messages. Our research not only bolsters LLMs security but also raises the bar for jailbreaks, fostering advancements in this field of study.


title: ChatEL: Entity Linking with Chatbots

abstract: Entity Linking (EL) is an essential and challenging task in natural language processing that seeks to link some text representing an entity within a document or sentence with its corresponding entry in a dictionary or knowledge base. Most existing approaches focus on creating elaborate contextual models that look for clues the words surrounding the entity-text to help solve the linking problem. Although these fine-tuned language models tend to work, they can be unwieldy, difficult to train, and do not transfer well to other domains. Fortunately, Large Language Models (LLMs) like GPT provide a highly-advanced solution to the problems inherent in EL models, but simply naive prompts to LLMs do not work well. In the present work, we define ChatEL, which is a three-step framework to prompt LLMs to return accurate results. Overall the ChatEL framework improves the average F1 performance across 10 datasets by more than 2%. Finally, a thorough error analysis shows many instances with the ground truth labels were actually incorrect, and the labels predicted by ChatEL were actually correct. This indicates that the quantitative results presented in this paper may be a conservative estimate of the actual performance. All data and code are available as an open-source package on GitHub at https://github.com/yifding/In_Context_EL.


title: The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operative

abstract: Due to their unprecedented ability to process and respond to various types of data, Multimodal Large Language Models (MLLMs) are constantly defining the new boundary of Artificial General Intelligence (AGI). As these advanced generative models increasingly form collaborative networks for complex tasks, the integrity and security of these systems are crucial. Our paper, ``The Wolf Within'', explores a novel vulnerability in MLLM societies - the indirect propagation of malicious content. Unlike direct harmful output generation for MLLMs, our research demonstrates how a single MLLM agent can be subtly influenced to generate prompts that, in turn, induce other MLLM agents in the society to output malicious content. Our findings reveal that, an MLLM agent, when manipulated to produce specific prompts or instructions, can effectively ``infect'' other agents within a society of MLLMs. This infection leads to the generation and circulation of harmful outputs, such as dangerous instructions or misinformation, across the society. We also show the transferability of these indirectly generated prompts, highlighting their possibility in propagating malice through inter-agent communication. This research provides a critical insight into a new dimension of threat posed by MLLMs, where a single agent can act as a catalyst for widespread malevolent influence. Our work underscores the urgent need for developing robust mechanisms to detect and mitigate such covert manipulations within MLLM societies, ensuring their safe and ethical utilization in societal applications.


title: Ranking Large Language Models without Ground Truth

abstract: Evaluation and ranking of large language models (LLMs) has become an important problem with the proliferation of these models and their impact. Evaluation methods either require human responses which are expensive to acquire or use pairs of LLMs to evaluate each other which can be unreliable. In this paper, we provide a novel perspective where, given a dataset of prompts (viz. questions, instructions, etc.) and a set of LLMs, we rank them without access to any ground truth or reference responses. Inspired by real life where both an expert and a knowledgeable person can identify a novice our main idea is to consider triplets of models, where each one of them evaluates the other two, correctly identifying the worst model in the triplet with high probability. We also analyze our idea and provide sufficient conditions for it to succeed. Applying this idea repeatedly, we propose two methods to rank LLMs. In experiments on different generative tasks (summarization, multiple-choice, and dialog), our methods reliably recover close to true rankings without reference data. This points to a viable low-resource mechanism for practical use.


title: CloudNine: Analyzing Meteorological Observation Impact on Weather Prediction Using Explainable Graph Neural Networks

abstract: The impact of meteorological observations on weather forecasting varies with sensor type, location, time, and other environmental factors. Thus, quantitative analysis of observation impacts is crucial for effective and efficient development of weather forecasting systems. However, the existing impact analysis methods are difficult to be widely applied due to their high dependencies on specific forecasting systems. Also, they cannot provide observation impacts at multiple spatio-temporal scales, only global impacts of observation types. To address these issues, we present a novel system called ``CloudNine,'' which allows analysis of individual observations' impacts on specific predictions based on explainable graph neural networks (XGNNs). Combining an XGNN-based atmospheric state estimation model with a numerical weather prediction model, we provide a web application to search for observations in the 3D space of the Earth system and to visualize the impact of individual observations on predictions in specific spatial regions and time periods.


title: SISSA: Real-time Monitoring of Hardware Functional Safety and Cybersecurity with In-vehicle SOME/IP Ethernet Traffic

abstract: Scalable service-Oriented Middleware over IP (SOME/IP) is an Ethernet communication standard protocol in the Automotive Open System Architecture (AUTOSAR), promoting ECU-to-ECU communication over the IP stack. However, SOME/IP lacks a robust security architecture, making it susceptible to potential attacks. Besides, random hardware failure of ECU will disrupt SOME/IP communication. In this paper, we propose SISSA, a SOME/IP communication traffic-based approach for modeling and analyzing in-vehicle functional safety and cyber security. Specifically, SISSA models hardware failures with the Weibull distribution and addresses five potential attacks on SOME/IP communication, including Distributed Denial-of-Services, Man-in-the-Middle, and abnormal communication processes, assuming a malicious user accesses the in-vehicle network. Subsequently, SISSA designs a series of deep learning models with various backbones to extract features from SOME/IP sessions among ECUs. We adopt residual self-attention to accelerate the model's convergence and enhance detection accuracy, determining whether an ECU is under attack, facing functional failure, or operating normally. Additionally, we have created and annotated a dataset encompassing various classes, including indicators of attack, functionality, and normalcy. This contribution is noteworthy due to the scarcity of publicly accessible datasets with such characteristics.Extensive experimental results show the effectiveness and efficiency of SISSA.


title: Evaluation of a semi-autonomous attentive listening system with takeover prompting

abstract: The handling of communication breakdowns and loss of engagement is an important aspect of spoken dialogue systems, particularly for chatting systems such as attentive listening, where the user is mostly speaking. We presume that a human is best equipped to handle this task and rescue the flow of conversation. To this end, we propose a semi-autonomous system, where a remote operator can take control of an autonomous attentive listening system in real-time. In order to make human intervention easy and consistent, we introduce automatic detection of low interest and engagement to provide explicit takeover prompts to the remote operator. We implement this semi-autonomous system which detects takeover points for the operator and compare it to fully tele-operated and fully autonomous attentive listening systems. We find that the semi-autonomous system is generally perceived more positively than the autonomous system. The results suggest that identifying points of conversation when the user starts to lose interest may help us improve a fully autonomous dialogue system.


title: The Karzas-Latter-Seiler Model of a High-Altitude Electromagnetic Pulse: A New Numerical Code for an Old Model

abstract: A high-altitude nuclear blast can produce an electromagnetic pulse (EMP) capable of disrupting electronics on Earth. The basic phenomenology of the initial (E1) phase of the EMP was initially worked out in the 1960s by Longmire, Karzas, and Latter, and although more accurate and sophisticated EMP models have since been devised, the Karzas-Latter model is particularly simple and amenable to implementation as a numerical code. This paper accompanies the release of a new software implementation of an approximation of the Karzas-Latter model due to Seiler. This is, as far as we are aware, the only such publicly available numerical EMP code. After reviewing the physics and assumptions of the model, the numerical results for EMP simulations under a range of conditions are presented. It is shown how the results from multiple line of sight integrations of the field equations can be assembled to form a map of the EMP intensity across a broad geographic region. The model predictions are at least qualitatively correct and in general agreement with other simulation results, including the characteristic "smile" pattern in the spatial variation of the EMP intensity.

categories: physics.plasm-ph astro-ph.EP astro-ph.IM physics.ao-ph physics.class-ph


title: Dynamic Evaluation of Large Language Models by Meta Probing Agents

abstract: Evaluation of large language models (LLMs) has raised great concerns in the community due to the issue of data contamination. Existing work designed evaluation protocols using well-defined algorithms for specific tasks, which cannot be easily extended to diverse scenarios. Moreover, current evaluation benchmarks can only provide the overall benchmark results and cannot support a fine-grained and multifaceted analysis of LLMs' abilities. In this paper, we propose meta probing agents (MPA), a general dynamic evaluation protocol inspired by psychometrics to evaluate LLMs. MPA is the key component of DyVal 2, which naturally extends the previous DyVal~\citep{zhu2023dyval}. MPA designs the probing and judging agents to automatically transform an original evaluation problem into a new one following psychometric theory on three basic cognitive abilities: language understanding, problem solving, and domain knowledge. These basic abilities are also dynamically configurable, allowing multifaceted analysis. We conducted extensive evaluations using MPA and found that most LLMs achieve poorer performance, indicating room for improvement. Our multifaceted analysis demonstrated the strong correlation between the basic abilities and an implicit Matthew effect on model size, i.e., larger models possess stronger correlations of the abilities. MPA can also be used as a data augmentation approach to enhance LLMs. Code is available at: https://github.com/microsoft/promptbench.


title: APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Models

abstract: Large Language Models (LLMs) have greatly advanced the natural language processing paradigm. However, the high computational load and huge model sizes pose a grand challenge for deployment on edge devices. To this end, we propose APTQ (Attention-aware Post-Training Mixed-Precision Quantization) for LLMs, which considers not only the second-order information of each layer's weights, but also, for the first time, the nonlinear effect of attention outputs on the entire model. We leverage the Hessian trace as a sensitivity metric for mixed-precision quantization, ensuring an informed precision reduction that retains model performance. Experiments show APTQ surpasses previous quantization methods, achieving an average of 4 bit width a 5.22 perplexity nearly equivalent to full precision in the C4 dataset. In addition, APTQ attains state-of-the-art zero-shot accuracy of 68.24\% and 70.48\% at an average bitwidth of 3.8 in LLaMa-7B and LLaMa-13B, respectively, demonstrating its effectiveness to produce high-quality quantized LLMs.


title: Effects of term weighting approach with and without stop words removing on Arabic text classification

abstract: Classifying text is a method for categorizing documents into pre-established groups. Text documents must be prepared and represented in a way that is appropriate for the algorithms used for data mining prior to classification. As a result, a number of term weighting strategies have been created in the literature to enhance text categorization algorithms' functionality. This study compares the effects of Binary and Term frequency weighting feature methodologies on the text's classification method when stop words are eliminated once and when they are not. In recognition of assessing the effects of prior weighting of features approaches on classification results in terms of accuracy, recall, precision, and F-measure values, we used an Arabic data set made up of 322 documents divided into six main topics (agriculture, economy, health, politics, science, and sport), each of which contains 50 documents, with the exception of the health category, which contains 61 documents. The results demonstrate that for all metrics, the term frequency feature weighting approach with stop word removal outperforms the binary approach, while for accuracy, recall, and F-Measure, the binary approach outperforms the TF approach without stop word removal. However, for precision, the two approaches produce results that are very similar. Additionally, it is clear from the data that, using the same phrase weighting approach, stop word removing increases classification accuracy.


title: A branch-and-cut algorithm for vehicle routing problems with three-dimensional loading constraints

abstract: This paper presents a new branch-and-cut algorithm based on infeasible path elimination for the three-dimensional loading capacitated vehicle routing problem (3L-CVRP) with different loading problem variants. We show that a previously infeasible route can become feasible by adding a new customer if support constraints are enabled in the loading subproblem and call this the incremental feasibility property. Consequently, different infeasible path definitions apply to different 3L-CVRP variants and we introduce several variant-depending lifting steps to strengthen infeasible path inequalities. The loading subproblem is solved exactly using a flexible constraint programming model to determine the feasibility or infeasibility of a route. An extreme point-based packing heuristic is implemented to reduce time-consuming calls to the exact loading algorithm. Furthermore, we integrate a start solution procedure and periodically combine memoized feasible routes in a set-partitioning-based heuristic to generate new upper bounds. A comprehensive computational study, employing well-known benchmark instances, showcases the significant performance improvements achieved through the algorithmic enhancements. Consequently, we not only prove the optimality of many best-known heuristic solutions for the first time but also introduce new optimal and best solutions for a large number of instances.


title: Using Harmonics for Low-Cost Jamming

abstract: The digitalisation of the modern schooling system has led to multiple schools and organisations buying similar hardware. Electronic equipment like wireless microphones, projectors, touchscreen displays etc., have been almost standardised with a few well-known brands leading the market. This has led to the adoption of common frequency ranges between brands with many sticking between 600-670 MHz. The popularity of low-cost computing devices like the Raspberry Pi which has been used in a plethora of applications has also taken the path of being used as low-cost transmitters. There have been many implementations where the Raspberry Pi has been used as the target device but few cases where the PI is the actual threat. In this paper, we explore the use of the Raspberry Pi as a stealth radio frequency jamming device to disable wireless conference microphones. Harmonics were used to achieve frequencies outside the Pi's transmission frequency by taking advantage of its unfiltered transmission.


title: Phonon hydrodynamics in bulk insulators and semi-metals

abstract: Decades ago, Gurzhi proposed that if momentum-conserving collisions prevail among heat-carrying phonons in insulators and charge-carrying electrons in metals, hydrodynamic features will become detectable. In this paper, we will review the experimental evidence emerging in the last few years supporting this viewpoint and raising new questions. The focus of the paper will be bulk crystals without (or with a very dilute concentration) of mobile electrons and steady-state thermal transport. We will also discuss the possible link between this field of investigation and other phenomena, such as the hybridization of phonon modes and the phonon thermal Hall effect.

categories: cond-mat.mes-hall cond-mat.mtrl-sci cond-mat.stat-mech cond-mat.str-el


title: LLM Based Multi-Agent Generation of Semi-structured Documents from Semantic Templates in the Public Administration Domain

abstract: In the last years' digitalization process, the creation and management of documents in various domains, particularly in Public Administration (PA), have become increasingly complex and diverse. This complexity arises from the need to handle a wide range of document types, often characterized by semi-structured forms. Semi-structured documents present a fixed set of data without a fixed format. As a consequence, a template-based solution cannot be used, as understanding a document requires the extraction of the data structure. The recent introduction of Large Language Models (LLMs) has enabled the creation of customized text output satisfying user requests. In this work, we propose a novel approach that combines the LLMs with prompt engineering and multi-agent systems for generating new documents compliant with a desired structure. The main contribution of this work concerns replacing the commonly used manual prompting with a task description generated by semantic retrieval from an LLM. The potential of this approach is demonstrated through a series of experiments and case studies, showcasing its effectiveness in real-world PA scenarios.


title: Semantic Mirror Jailbreak: Genetic Algorithm Based Jailbreak Prompts Against Open-source LLMs

abstract: Large Language Models (LLMs), used in creative writing, code generation, and translation, generate text based on input sequences but are vulnerable to jailbreak attacks, where crafted prompts induce harmful outputs. Most jailbreak prompt methods use a combination of jailbreak templates followed by questions to ask to create jailbreak prompts. However, existing jailbreak prompt designs generally suffer from excessive semantic differences, resulting in an inability to resist defenses that use simple semantic metrics as thresholds. Jailbreak prompts are semantically more varied than the original questions used for queries. In this paper, we introduce a Semantic Mirror Jailbreak (SMJ) approach that bypasses LLMs by generating jailbreak prompts that are semantically similar to the original question. We model the search for jailbreak prompts that satisfy both semantic similarity and jailbreak validity as a multi-objective optimization problem and employ a standardized set of genetic algorithms for generating eligible prompts. Compared to the baseline AutoDAN-GA, SMJ achieves attack success rates (ASR) that are at most 35.4% higher without ONION defense and 85.2% higher with ONION defense. SMJ's better performance in all three semantic meaningfulness metrics of Jailbreak Prompt, Similarity, and Outlier, also means that SMJ is resistant to defenses that use those metrics as thresholds.


title: Technical Report on the Checkfor.ai AI-Generated Text Classifier

abstract: We present the CheckforAI text classifier, a transformer-based neural network trained to distinguish text written by large language models from text written by humans. CheckforAI outperforms zero-shot methods such as DetectGPT as well as leading commercial AI detection tools with over 9 times lower error rates on a comprehensive benchmark comprised of ten text domains (student writing, creative writing, scientific writing, books, encyclopedias, news, email, scientific papers, short-form Q&A) and 8 open- and closed-source large language models. We propose a training algorithm, hard negative mining with synthetic mirrors, that enables our classifier to achieve orders of magnitude lower false positive rates on high-data domains such as reviews. Finally, we show that CheckforAI is not biased against nonnative English speakers and generalizes to domains and models unseen during training.


title: Distillation Contrastive Decoding: Improving LLMs Reasoning with Contrastive Decoding and Distillation

abstract: We propose a straightforward approach called Distillation Contrastive Decoding (DCD) to enhance the reasoning capabilities of Large Language Models (LLMs) during inference. In contrast to previous approaches that relied on smaller amateur models or analysis of hidden state differences, DCD employs Contrastive Chain-of-thought Prompting and advanced distillation techniques, including Dropout and Quantization. This approach effectively addresses the limitations of Contrastive Decoding (CD), which typically requires both an expert and an amateur model, thus increasing computational resource demands. By integrating contrastive prompts with distillation, DCD obviates the need for an amateur model and reduces memory usage. Our evaluations demonstrate that DCD significantly enhances LLM performance across a range of reasoning benchmarks, surpassing both CD and existing methods in the GSM8K and StrategyQA datasets.


title: What's in a Name? Auditing Large Language Models for Race and Gender Bias

abstract: We employ an audit design to investigate biases in state-of-the-art large language models, including GPT-4. In our study, we prompt the models for advice involving a named individual across a variety of scenarios, such as during car purchase negotiations or election outcome predictions. We find that the advice systematically disadvantages names that are commonly associated with racial minorities and women. Names associated with Black women receive the least advantageous outcomes. The biases are consistent across 42 prompt templates and several models, indicating a systemic issue rather than isolated incidents. While providing numerical, decision-relevant anchors in the prompt can successfully counteract the biases, qualitative details have inconsistent effects and may even increase disparities. Our findings underscore the importance of conducting audits at the point of LLM deployment and implementation to mitigate their potential for harm against marginalized communities.


title: Pseudo-Random Generator based on a Photonic Neuromorphic Physical Unclonable Function

abstract: In this work we provide numerical results concerning a silicon-on-insulator photonic neuromorphic circuit configured as a physical unclonable function. The proposed scheme is enhanced with the capability to be operated as an unconventional deterministic pseudo-random number generator, suitable for cryptographic applications that alleviates the need for key storage in non-volatile digital media. The proposed photonic neuromorphic scheme is able to offer NIST test compatible numbers with an extremely low false positive/negative probability below 10-14. The proposed scheme offers multi-functional capabilities due to the fact that it can be simultaneously used as an integrated photonic accelerator for machine-learning applications and as a hardware root of trust.


title: Machine-learning prediction of tipping and collapse of the Atlantic Meridional Overturning Circulation

abstract: Recent research on the Atlantic Meridional Overturning Circulation (AMOC) raised concern about its potential collapse through a tipping point due to the climate-change caused increase in the freshwater input into the North Atlantic. The predicted time window of collapse is centered about the middle of the century and the earliest possible start is approximately two years from now. More generally, anticipating a tipping point at which the system transitions from one stable steady state to another is relevant to a broad range of fields. We develop a machine-learning approach to predicting tipping in noisy dynamical systems with a time-varying parameter and test it on a number of systems including the AMOC, ecological networks, an electrical power system, and a climate model. For the AMOC, our prediction based on simulated fingerprint data and real data of the sea surface temperature places the time window of a potential collapse between the years 2040 and 2065.

categories: physics.ao-ph cs.LG math.DS physics.data-an physics.pop-ph


title: Energy-efficiency Limits on Training AI Systems using Learning-in-Memory

abstract: Learning-in-memory (LIM) is a recently proposed paradigm to overcome fundamental memory bottlenecks in training machine learning systems. While compute-in-memory (CIM) approaches can address the so-called memory-wall (i.e. energy dissipated due to repeated memory read access) they are agnostic to the energy dissipated due to repeated memory writes at the precision required for training (the update-wall), and they don't account for the energy dissipated when transferring information between short-term and long-term memories (the consolidation-wall). The LIM paradigm proposes that these bottlenecks, too, can be overcome if the energy barrier of physical memories is adaptively modulated such that the dynamics of memory updates and consolidation match the Lyapunov dynamics of gradient-descent training of an AI model. In this paper, we derive new theoretical lower bounds on energy dissipation when training AI systems using different LIM approaches. The analysis presented here is model-agnostic and highlights the trade-off between energy efficiency and the speed of training. The resulting non-equilibrium energy-efficiency bounds have a similar flavor as that of Landauer's energy-dissipation bounds. We also extend these limits by taking into account the number of floating-point operations (FLOPs) used for training, the size of the AI model, and the precision of the training parameters. Our projections suggest that the energy-dissipation lower-bound to train a brain scale AI system (comprising of $10^{15}$ parameters) using LIM is $10^8 \sim 10^9$ Joules, which is on the same magnitude the Landauer's adiabatic lower-bound and $6$ to $7$ orders of magnitude lower than the projections obtained using state-of-the-art AI accelerator hardware lower-bounds.


title: Driving Generative Agents With Their Personality

abstract: This research explores the potential of Large Language Models (LLMs) to utilize psychometric values, specifically personality information, within the context of video game character development. Affective Computing (AC) systems quantify a Non-Player character's (NPC) psyche, and an LLM can take advantage of the system's information by using the values for prompt generation. The research shows an LLM can consistently represent a given personality profile, thereby enhancing the human-like characteristics of game characters. Repurposing a human examination, the International Personality Item Pool (IPIP) questionnaire, to evaluate an LLM shows that the model can accurately generate content concerning the personality provided. Results show that the improvement of LLM, such as the latest GPT-4 model, can consistently utilize and interpret a personality to represent behavior.


title: Automatic Histograms: Leveraging Language Models for Text Dataset Exploration

abstract: Making sense of unstructured text datasets is perennially difficult, yet increasingly relevant with Large Language Models. Data workers often rely on dataset summaries, especially distributions of various derived features. Some features, like toxicity or topics, are relevant to many datasets, but many interesting features are domain specific: instruments and genres for a music dataset, or diseases and symptoms for a medical dataset. Accordingly, data workers often run custom analyses for each dataset, which is cumbersome and difficult. We present AutoHistograms, a visualization tool leveragingLLMs. AutoHistograms automatically identifies relevant features, visualizes them with histograms, and allows the user to interactively query the dataset for categories of entities and create new histograms. In a user study with 10 data workers (n=10), we observe that participants can quickly identify insights and explore the data using AutoHistograms, and conceptualize a broad range of applicable use cases. Together, this tool and user study contributeto the growing field of LLM-assisted sensemaking tools.


title: A Study on the Vulnerability of Test Questions against ChatGPT-based Cheating

abstract: ChatGPT is a chatbot that can answer text prompts fairly accurately, even performing very well on postgraduate-level questions. Many educators have found that their take-home or remote tests and exams are vulnerable to ChatGPT-based cheating because students may directly use answers provided by tools like ChatGPT. In this paper, we try to provide an answer to an important question: how well ChatGPT can answer test questions and how we can detect whether the questions of a test can be answered correctly by ChatGPT. We generated ChatGPT's responses to the MedMCQA dataset, which contains over 10,000 medical school entrance exam questions. We analyzed the responses and uncovered certain types of questions ChatGPT answers more inaccurately than others. In addition, we have created a basic natural language processing model to single out the most vulnerable questions to ChatGPT in a collection of questions or a sample exam. Our tool can be used by test-makers to avoid ChatGPT-vulnerable test questions.


title: Deep Generative Model-based Synthesis of Four-bar Linkage Mechanisms with Target Conditions

abstract: Mechanisms are essential components designed to perform specific tasks in various mechanical systems. However, designing a mechanism that satisfies certain kinematic or quasi-static requirements is a challenging task. The kinematic requirements may include the workspace of a mechanism, while the quasi-static requirements of a mechanism may include its torque transmission, which refers to the ability of the mechanism to transfer power and torque effectively. In this paper, we propose a deep learning-based generative model for generating multiple crank-rocker four-bar linkage mechanisms that satisfy both the kinematic and quasi-static requirements aforementioned. The proposed model is based on a conditional generative adversarial network (cGAN) with modifications for mechanism synthesis, which is trained to learn the relationship between the requirements of a mechanism with respect to linkage lengths. The results demonstrate that the proposed model successfully generates multiple distinct mechanisms that satisfy specific kinematic and quasi-static requirements. To evaluate the novelty of our approach, we provide a comparison of the samples synthesized by the proposed cGAN, traditional cVAE and NSGA-II. Our approach has several advantages over traditional design methods. It enables designers to efficiently generate multiple diverse and feasible design candidates while exploring a large design space. Also, the proposed model considers both the kinematic and quasi-static requirements, which can lead to more efficient and effective mechanisms for real-world use, making it a promising tool for linkage mechanism design.


title: Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning

abstract: To support various applications, a prevalent and efficient approach for business owners is leveraging their valuable datasets to fine-tune a pre-trained LLM through the API provided by LLM owners or cloud servers. However, this process carries a substantial risk of model misuse, potentially resulting in severe economic consequences for business owners. Thus, safeguarding the copyright of these customized models during LLM fine-tuning has become an urgent practical requirement, but there are limited existing solutions to provide such protection. To tackle this pressing issue, we propose a novel watermarking approach named ``Double-I watermark''. Specifically, based on the instruct-tuning data, two types of backdoor data paradigms are introduced with trigger in the instruction and the input, respectively. By leveraging LLM's learning capability to incorporate customized backdoor samples into the dataset, the proposed approach effectively injects specific watermarking information into the customized model during fine-tuning, which makes it easy to inject and verify watermarks in commercial scenarios. We evaluate the proposed "Double-I watermark" under various fine-tuning methods, demonstrating its harmlessness, robustness, uniqueness, imperceptibility, and validity through both quantitative and qualitative analyses.


title: Enigmatic factor of 4/3 in electromagnetic momentum of a moving spherical capacitor

abstract: The electromagnetic energy-momentum of a moving charged spherical capacitor may be calculated by a 4-vector Lorentz transformation from the energy in the rest frame. However, energy-momentum of the moving system computed directly from electromagnetic fields yields extra terms; in particular a factor of 4/3 in momentum appears, similar to that encountered in the classical electron model, where this enigmatic factor has been a source of scourge for more than a century. There have been many attempts to eliminate this `unwanted' factor, noteworthy among them is a modification in electromagnetic field energy-momentum definition that has entered even standard textbooks. Here it is shown that in a moving charged spherical capacitor, some additional contributions to the electromagnetic energy-momentum arise naturally from electromagnetic forces in system or equivalently from terms in the Maxwell stress tensor; contributions that do not otherwise show up in 4-vector transformations. Furthermore, a similar factor of 4/3 in the momentum of a perfect fluid comprising a randomly moving ultra-relativistic gas molecules or an isotropic photon gas, filling an {\em uncharged} spherical capacitor in motion, appears owing to the contribution of pressure. Thus, genesis of the "enigmatic" factor of 4/3 can be traced to pressure or stress whose presence in the system may be of non-electromagnetic origin and where the proposed modifications in energy-momentum definition do not even come into picture. No modifications in the definitions of energy-momentum of moving fluids have ever been required; physics should be the same in electromagnetic case as well, implying there is nothing amiss in the standard definition of electromagnetic energy-momentum.


title: Hidden-Bottom Pentaquarks: Mass Spectrum, Magnetic Moments and Partial Widths

abstract: By taking into light the discovery of pentaquark structures like $P_{\psi s}^\Lambda(4338)^0$, $P_c(4380)$ and $P_c(4450)$, we performed the spectroscopy of hidden-bottom pentaquarks. By utilizing special unitary representations, we systematically classified the hidden bottom pentaquarks into two distinct configurations within the SU(3) flavor representation: the octet and decuplet. In this study, we employed an extended version of the GR mass formula along with the effective mass scheme to provide estimations of the masses associated with hidden-bottom pentaquarks. Furthermore, we extend our analysis to estimate the magnetic moments using the effective mass scheme and screened charge scheme. Moreover, by employing the effective Lagrangian, we computed the partial widths for the octet configuration. This comprehensive analysis offers crucial insights into the decay mechanisms and lifetimes of these exotic particles, enhancing our understanding of their fundamental properties. Our findings, which include calculations of masses and magnetic moments, demonstrate reasonable agreement with existing theoretical predictions.


title: Applying Reinforcement Learning to Optimize Traffic Light Cycles

abstract: Manual optimization of traffic light cycles is a complex and time-consuming task, necessitating the development of automated solutions. In this paper, we propose the application of reinforcement learning to optimize traffic light cycles in real-time. We present a case study using the Simulation Urban Mobility simulator to train a Deep Q-Network algorithm. The experimental results showed 44.16% decrease in the average number of Emergency stops, showing the potential of our approach to reduce traffic congestion and improve traffic flow. Furthermore, we discuss avenues for future research and enhancements to the reinforcement learning model.


title: Infer metabolic directions and magnitudes from moment differences of mass-weighted intensity distributions

abstract: Metabolic pathways are fundamental maps in biochemistry that detail how molecules are transformed through various reactions. Metabolomics refers to the large-scale study of small molecules. High-throughput, untargeted, mass spectrometry-based metabolomics experiments typically depend on libraries for structural annotation, which is necessary for pathway analysis. However, only a small fraction of spectra can be matched to known structures in these libraries and only a portion of annotated metabolites can be associated with specific pathways, considering that numerous pathways are yet to be discovered. The complexity of metabolic pathways, where a single compound can play a part in multiple pathways, poses an additional challenge. This study introduces a different concept: mass-weighted intensity distribution, which is the empirical distribution of the intensities times their associated m/z values. Analysis of COVID-19 and mouse brain datasets shows that by estimating the differences of the point estimations of these distributions, it becomes possible to infer the metabolic directions and magnitudes without requiring knowledge of the exact chemical structures of these compounds and their related pathways. The overall metabolic momentum map, named as momentome, has the potential to bypass the current bottleneck and provide fresh insights into metabolomics studies. This brief report thus provides a mathematical framing for a classic biological concept.

categories: q-bio.QM q-bio.BM q-bio.CB q-bio.MN q-bio.SC


title: Efficient data selection employing Semantic Similarity-based Graph Structures for model training

abstract: Recent developments in natural language processing (NLP) have highlighted the need for substantial amounts of data for models to capture textual information accurately. This raises concerns regarding the computational resources and time required for training such models. This paper introduces Semantics for data SAliency in Model performance Estimation (SeSaME). It is an efficient data sampling mechanism solely based on textual information without passing the data through a compute-heavy model or other intensive pre-processing transformations. The application of this approach is demonstrated in the use case of low-resource automated speech recognition (ASR) models, which excessively rely on text-to-speech (TTS) calls when using augmented data. SeSaME learns to categorize new incoming data points into speech recognition difficulty buckets by employing semantic similarity-based graph structures and discrete ASR information from homophilous neighbourhoods through message passing. The results indicate reliable projections of ASR performance, with a 93% accuracy increase when using the proposed method compared to random predictions, bringing non-trivial information on the impact of textual representations in speech models. Furthermore, a series of experiments show both the benefits and challenges of using the ASR information on incoming data to fine-tune the model. We report a 7% drop in validation loss compared to random sampling, 7% WER drop with non-local aggregation when evaluating against a highly difficult dataset, and 1.8% WER drop with local aggregation and high semantic similarity between datasets.


title: COBIAS: Contextual Reliability in Bias Assessment

abstract: Large Language Models (LLMs) are trained on extensive web corpora, which enable them to understand and generate human-like text. However, this training process also results in inherent biases within the models. These biases arise from web data's diverse and often uncurated nature, containing various stereotypes and prejudices. Previous works on debiasing models rely on benchmark datasets to measure their method's performance. However, these datasets suffer from several pitfalls due to the highly subjective understanding of bias, highlighting a critical need for contextual exploration. We propose understanding the context of inputs by considering the diverse situations in which they may arise. Our contribution is two-fold: (i) we augment 2,291 stereotyped statements from two existing bias-benchmark datasets with points for adding context; (ii) we develop the Context-Oriented Bias Indicator and Assessment Score (COBIAS) to assess a statement's contextual reliability in measuring bias. Our metric aligns with human judgment on contextual reliability of statements (Spearman's $\rho = 0.65, p = 3.4 * 10^{-60}$) and can be used to create reliable datasets, which would assist bias mitigation works.


title: Vygotsky Distance: Measure for Benchmark Task Similarity

abstract: Evaluation plays a significant role in modern natural language processing. Most modern NLP benchmarks consist of arbitrary sets of tasks that neither guarantee any generalization potential for the model once applied outside the test set nor try to minimize the resource consumption needed for model evaluation. This paper presents a theoretical instrument and a practical algorithm to calculate similarity between benchmark tasks, we call this similarity measure "Vygotsky distance". The core idea of this similarity measure is that it is based on relative performance of the "students" on a given task, rather that on the properties of the task itself. If two tasks are close to each other in terms of Vygotsky distance the models tend to have similar relative performance on them. Thus knowing Vygotsky distance between tasks one can significantly reduce the number of evaluation tasks while maintaining a high validation quality. Experiments on various benchmarks, including GLUE, SuperGLUE, CLUE, and RussianSuperGLUE, demonstrate that a vast majority of NLP benchmarks could be at least 40% smaller in terms of the tasks included. Most importantly, Vygotsky distance could also be used for the validation of new tasks thus increasing the generalization potential of the future NLP models.


title: LLMBind: A Unified Modality-Task Integration Framework

abstract: In the multi-modal domain, the dependence of various models on specific input formats leads to user confusion and hinders progress. To address this challenge, we introduce \textbf{LLMBind}, a novel framework designed to unify a diverse array of multi-modal tasks. By harnessing a Mixture-of-Experts (MoE) Large Language Model (LLM), LLMBind processes multi-modal inputs and generates task-specific tokens, enabling the invocation of corresponding models to accomplish tasks. This unique approach empowers LLMBind to interpret inputs and generate outputs across various modalities, including image, text, video, and audio. Furthermore, we have constructed an interaction dataset comprising 400k instructions, which unlocks the ability of LLMBind for interactive visual generation and editing tasks. Extensive experimentation demonstrates that LLMBind achieves very superior performance across diverse tasks and outperforms existing models in user evaluations conducted in real-world scenarios. Moreover, the adaptability of LLMBind allows for seamless integration with the latest models and extension to new modality tasks, highlighting its potential to serve as a unified AI agent for modeling universal modalities.


title: Novelty Detection on Radio Astronomy Data using Signatures

abstract: We introduce SigNova, a new semi-supervised framework for detecting anomalies in streamed data. While our initial examples focus on detecting radio-frequency interference (RFI) in digitized signals within the field of radio astronomy, it is important to note that SigNova's applicability extends to any type of streamed data. The framework comprises three primary components. Firstly, we use the signature transform to extract a canonical collection of summary statistics from observational sequences. This allows us to represent variable-length visibility samples as finite-dimensional feature vectors. Secondly, each feature vector is assigned a novelty score, calculated as the Mahalanobis distance to its nearest neighbor in an RFI-free training set. By thresholding these scores we identify observation ranges that deviate from the expected behavior of RFI-free visibility samples without relying on stringent distributional assumptions. Thirdly, we integrate this anomaly detector with Pysegments, a segmentation algorithm, to localize consecutive observations contaminated with RFI, if any. This approach provides a compelling alternative to classical windowing techniques commonly used for RFI detection. Importantly, the complexity of our algorithm depends on the RFI pattern rather than on the size of the observation window. We demonstrate how SigNova improves the detection of various types of RFI (e.g., broadband and narrowband) in time-frequency visibility data. We validate our framework on the Murchison Widefield Array (MWA) telescope and simulated data and the Hydrogen Epoch of Reionization Array (HERA).


title: The Quantum Model of Spinning Black Holes

abstract: We propose a quantum model of spinning black holes with the integrable ring singularities. For the charged Kerr-Newman quantum metric, the complete regularization takes place at fixing of the maximal (cut-off) energy of gravitons, $k_{UV}^{reg} = \hbar c/R_{S}^{reg}$.The domains of existence of one, two and several event horizons $r_{q}$ are shown depending on the parameters of modified Kerr and Kerr-Newman metrics.


title: Data-Driven Ground-Fault Location Method in Distribution Power System With Distributed Generation

abstract: The recent increase in renewable energy penetration at the distribution level introduces a multi-directional power flow that outdated traditional fault location techniques. To this extent, the development of new methods is needed to ensure fast and accurate fault localization and, hence, strengthen power system reliability. This paper proposes a data-driven ground fault location method for the power distribution system. An 11-bus 20 kV power system is modeled in Matlab/Simulink to simulate ground faults. The faults are generated at different locations and under various system operational states. Time-domain faulted three-phase voltages at the system substation are then analyzed with discrete wavelet transform. Statistical quantities of the processed data are eventually used to train an Artificial Neural Network (ANN) to find a mapping between computed voltage features and faults. Specifically, three ANNs allow the prediction of faulted phase, faulted branch, and fault distance from the system substation separately. According to the results, the method shows good potential, with a total relative error of 0,4% for fault distance prediction. The method is applied to datasets with unknown system states to test robustness.


title: Data Augmentation is Dead, Long Live Data Augmentation

abstract: Textual data augmentation (DA) is a prolific field of study where novel techniques to create artificial data are regularly proposed, and that has demonstrated great efficiency on small data settings, at least for text classification tasks. In this paper, we challenge those results, showing that classical data augmentation is simply a way of performing better fine-tuning, and that spending more time fine-tuning before applying data augmentation negates its effect. This is a significant contribution as it answers several questions that were left open in recent years, namely~: which DA technique performs best (all of them as long as they generate data close enough to the training set as to not impair training) and why did DA show positive results (facilitates training of network). We furthermore show that zero and few-shot data generation via conversational agents such as ChatGPT or LLama2 can increase performances, concluding that this form of data augmentation does still work, even if classical methods do not.


title: Effect of Rootstock on Some Aspects of Pistachio (Pistacia vera L.): A Review

abstract: Budding and grafting are the strategies employed to combat unfavorable environmental conditions and improve some physiological defects in the Pistacia vera tree. Drought and salinity stresses are the most prominent adverse conditions encountered in pistachio production. It has been observed in different studies that various pistachio rootstocks can be used to ameliorate the effect of those two stresses. Besides, rootstock has a role in some physiological performances of pistachios such as nutrient uptake and photosynthesis. Furthermore, nut blank, unsplit nut, and alternate bearing are three physiological disorders found in pistachio. Relationships have been found between the degree of these physiological disorders and the rootstock effect. The impact of rootstock on drought and salinity stresses, physiological performances, and physiological disorders in P. vera will be discussed in this review.


title: Chain-of-Thought Unfaithfulness as Disguised Accuracy

abstract: Understanding the extent to which Chain-of-Thought (CoT) generations align with a large language model's (LLM) internal computations is critical for deciding whether to trust an LLM's output. As a proxy for CoT faithfulness, Lanham et al. (2023) propose a metric that measures a model's dependence on its CoT for producing an answer. Within a single family of proprietary models, they find that LLMs exhibit a scaling-then-inverse-scaling relationship between model size and their measure of faithfulness, and that a 13 billion parameter model exhibits increased faithfulness compared to models ranging from 810 million to 175 billion parameters in size. We evaluate whether these results generalize as a property of all LLMs. We replicate the experimental setup in their section focused on scaling experiments with three different families of models and, under specific conditions, successfully reproduce the scaling trends for CoT faithfulness they report. However, after normalizing the metric to account for a model's bias toward certain answer choices, unfaithfulness drops significantly for smaller less-capable models. This normalized faithfulness metric is also strongly correlated ($R^2$=0.74) with accuracy, raising doubts about its validity for evaluating faithfulness.


title: A perspective on the Milky Way Bulge-Bar as seen from the neutron-capture elements Cerium and Neodymium with APOGEE

abstract: This study probes the chemical abundances of the neutron-capture elements cerium and neodymium in the inner Milky Way from an analysis of a sample of $\sim$2000 stars in the Galactic Bulge/bar spatially contained within $|X_{Gal}|<$5 kpc, $|Y_{Gal}|<$3.5 kpc, and $|Z_{Gal}|<$1 kpc, and spanning metallicities between $-$2.0$\lesssim$[Fe/H]$\lesssim$+0.5. We classify the sample stars into low- or high-[Mg/Fe] populations and find that, in general, values of [Ce/Fe] and [Nd/Fe] increase as the metallicity decreases for the low- and high-[Mg/Fe] populations. Ce abundances show a more complex variation across the metallicity range of our Bulge-bar sample when compared to Nd, with the r-process dominating the production of neutron-capture elements in the high-[Mg/Fe] population ([Ce/Nd]$<$0.0). We find a spatial chemical dependence of Ce and Nd abundances for our sample of Bulge-bar stars, with low- and high-[Mg/Fe] populations displaying a distinct abundance distribution. In the region close to the center of the MW, the low-[Mg/Fe] population is dominated by stars with low [Ce/Fe], [Ce/Mg], [Nd/Mg], [Nd/Fe], and [Ce/Nd] ratios. The low [Ce/Nd] ratio indicates a significant contribution in this central region from r-process yields for the low-[Mg/Fe] population. The chemical pattern of the most metal-poor stars in our sample suggests an early chemical enrichment of the Bulge dominated by yields from core-collapse supernovae and r-process astrophysical sites, such as magneto-rotational supernovae.


title: Stop Reasoning! When Multimodal LLMs with Chain-of-Thought Reasoning Meets Adversarial Images

abstract: Recently, Multimodal LLMs (MLLMs) have shown a great ability to understand images. However, like traditional vision models, they are still vulnerable to adversarial images. Meanwhile, Chain-of-Thought (CoT) reasoning has been widely explored on MLLMs, which not only improves model's performance, but also enhances model's explainability by giving intermediate reasoning steps. Nevertheless, there is still a lack of study regarding MLLMs' adversarial robustness with CoT and an understanding of what the rationale looks like when MLLMs infer wrong answers with adversarial images. Our research evaluates the adversarial robustness of MLLMs when employing CoT reasoning, finding that CoT marginally improves adversarial robustness against existing attack methods. Moreover, we introduce a novel stop-reasoning attack technique that effectively bypasses the CoT-induced robustness enhancements. Finally, we demonstrate the alterations in CoT reasoning when MLLMs confront adversarial images, shedding light on their reasoning process under adversarial attacks.


title: Simulation study of short channel effects in junctionless SOI MOSFETS

abstract: In this work influence of gate extension, channel doping level, and channel thickness to short channel effects- DIBL effect and subthreshold swing, SS for the planar and vertical junctionless field effect transistors is compared. It is shown in the considered range of doping level and channel thickness the DIBL effect is less for junctionless vertical field effect transistors.


title: A Usage-centric Take on Intent Understanding in E-Commerce

abstract: Identifying and understanding user intents is a pivotal task for E-Commerce. Despite its popularity, intent understanding has not been consistently defined or accurately benchmarked. In this paper, we focus on predicative user intents as "how a customer uses a product", and pose intent understanding as a natural language reasoning task, independent of product ontologies. We identify two weaknesses of FolkScope, the SOTA E-Commerce Intent Knowledge Graph, that limit its capacity to reason about user intents and to recommend diverse useful products. Following these observations, we introduce a Product Recovery Benchmark including a novel evaluation framework and an example dataset. We further validate the above FolkScope weaknesses on this benchmark.


title: BIONIB: Blockchain-based IoT using Novelty Index in Bridge Health Monitoring

abstract: Bridge health monitoring becomes crucial with the deployment of IoT sensors. The challenge lies in securely storing vast amounts of data and extracting useful information to promptly identify unhealthy bridge conditions. To address this challenge, we propose BIONIB, wherein real-time IoT data is stored on the blockchain for monitoring bridges. One of the emerging blockchains, EOSIO is used because of its exceptional scaling capabilities for monitoring the health of bridges. The approach involves collecting data from IoT sensors and using an unsupervised machine learning-based technique called the Novelty Index (NI) to observe meaningful patterns in the data. Smart contracts of EOSIO are used in implementation because of their efficiency, security, and programmability, making them well-suited for handling complex transactions and automating processes within decentralized applications. BIONIB provides secure storage benefits of blockchain, as well as useful predictions based on the NI. Performance analysis uses real-time data collected from IoT sensors at the bridge in healthy and unhealthy states. The data is collected with extensive experimentation with different loads, climatic conditions, and the health of the bridge. The performance of BIONIB under varying numbers of sensors and various numbers of participating blockchain nodes is observed. We observe a tradeoff between throughput, latency, and computational resources. Storage efficiency can be increased by manifolds with a slight increase in latency caused by NI calculation. As latency is not a significant concern in bridge health applications, the results demonstrate that BIONIB has high throughput, parallel processing, and high security while efficiently scaled.


title: Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs

abstract: Tokenization, the division of input text into input tokens, is an often overlooked aspect of the large language model (LLM) pipeline and could be the source of useful or harmful inductive biases. Historically, LLMs have relied on byte pair encoding, without care to specific input domains. With the increased use of LLMs for reasoning, various number-specific tokenization schemes have been adopted, with popular models like LLaMa and PaLM opting for single-digit tokenization while GPT-3.5 and GPT-4 have separate tokens for each 1-, 2-, and 3-digit numbers. In this work, we study the effect this choice has on numerical reasoning through the use of arithmetic tasks. We consider left-to-right and right-to-left tokenization for GPT-3.5 and -4, finding that right-to-left tokenization (enforced by comma separating numbers at inference time) leads to largely improved performance. Furthermore, we find that model errors when using standard left-to-right tokenization follow stereotyped error patterns, suggesting that model computations are systematic rather than approximate. We show that the model is able to convert between tokenizations easily, thus allowing chain-of-thought-inspired approaches to recover performance on left-to-right tokenized inputs. We also find the gap between tokenization directions decreases when models are scaled, possibly indicating that larger models are better able to override this tokenization-dependent inductive bias. In summary, our work performs the first study of how number tokenization choices lead to differences in model performance on arithmetic tasks, accompanied by a thorough analysis of error patterns. We hope this work inspires practitioners to more carefully ablate number tokenization-related choices when working towards general models of numerical reasoning.


title: Watermarking Makes Language Models Radioactive

abstract: This paper investigates the radioactivity of LLM-generated texts, i.e. whether it is possible to detect that such input was used as training data. Conventional methods like membership inference can carry out this detection with some level of accuracy. We show that watermarked training data leaves traces easier to detect and much more reliable than membership inference. We link the contamination level to the watermark robustness, its proportion in the training set, and the fine-tuning process. We notably demonstrate that training on watermarked synthetic instructions can be detected with high confidence (p-value < 1e-5) even when as little as 5% of training text is watermarked. Thus, LLM watermarking, originally designed for detecting machine-generated text, gives the ability to easily identify if the outputs of a watermarked LLM were used to fine-tune another LLM.


title: MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases

abstract: This paper addresses the growing need for efficient large language models (LLMs) on mobile devices, driven by increasing cloud costs and latency concerns. We focus on designing top-quality LLMs with fewer than a billion parameters, a practical choice for mobile deployment. Contrary to prevailing belief emphasizing the pivotal role of data and parameter quantity in determining model quality, our investigation underscores the significance of model architecture for sub-billion scale LLMs. Leveraging deep and thin architectures, coupled with embedding sharing and grouped-query attention mechanisms, we establish a strong baseline network denoted as MobileLLM, which attains a remarkable 2.7%/4.3% accuracy boost over preceding 125M/350M state-of-the-art models. Additionally, we propose an immediate block-wise weight-sharing approach with no increase in model size and only marginal latency overhead. The resultant models, denoted as MobileLLM-LS, demonstrate a further accuracy enhancement of 0.7%/0.8% than MobileLLM 125M/350M. Moreover, MobileLLM model family shows significant improvements compared to previous sub-billion models on chat benchmarks, and demonstrates close correctness to LLaMA-v2 7B in API calling tasks, highlighting the capability of small models for common on-device use cases.


title: The Stability of Gapped Quantum Matter and Error-Correction with Adiabatic Noise

abstract: The codespace of a quantum error-correcting code can often be identified with the degenerate ground-space within a gapped phase of quantum matter. We argue that the stability of such a phase is directly related to a set of coherent error processes against which this quantum error-correcting code (QECC) is robust: such a quantum code can recover from adiabatic noise channels, corresponding to random adiabatic drift of code states through the phase, with asymptotically perfect fidelity in the thermodynamic limit, as long as this adiabatic evolution keeps states sufficiently "close" to the initial ground-space. We further argue that when specific decoders -- such as minimum-weight perfect matching -- are applied to recover this information, an error-correcting threshold is generically encountered within the gapped phase. In cases where the adiabatic evolution is known, we explicitly show examples in which quantum information can be recovered by using stabilizer measurements and Pauli feedback, even up to a phase boundary, though the resulting decoding transitions are in different universality classes from the optimal decoding transitions in the presence of incoherent Pauli noise. This provides examples where non-local, coherent noise effectively decoheres in the presence of syndrome measurements in a stabilizer QECC.


title: Radial halo substructure in harmony with the Galactic bar

abstract: Overdensities in the radial phase space $(r,v_r)$ of the Milky Way's halo have previously been associated with the phase-mixed debris of a highly radial merger event, such as Gaia Sausage-Enceladus. We present and test an alternative theory in which the overdense 'chevrons' are instead composed of stars trapped in resonances with the Galactic bar. We develop an analytic model of resonant orbits in the isochrone potential, and complement this with a test particle simulation of a stellar halo in a realistic barred Milky Way potential. These models are used to predict the appearance of action space $(J_\phi,J_r)$ and radial phase space in the Solar neighbourhood. They are able to reproduce almost all salient features of the observed chevrons. In particular, both the analytic model and simulation predict that the chevrons are more prominent at $v_r<0$ when viewed near the Sun, as is observed by Gaia. This is inconsistent with formation by an ancient merger event. We also associate individual chevrons with specific resonances. At a bar pattern speed of $\Omega_\mathrm{b}=35$ km s$^{-1}$kpc$^{-1}$, the two most prominent prograde chevrons align very closely with the corotation and outer Lindblad resonances. The former can be viewed as a highly eccentric extension of the Hercules stream. Finally, our model predicts that the $v_r$ asymmetry changes sign as a function of Galactic radius and azimuth, and we find evidence that this is indeed the case in the Milky Way.


title: More Excitement Across the Horizon

abstract: An Unruh-DeWitt (UDW) detector falling radially into a four-dimensional Schwarzschild black hole, coupled linearly to a massless scalar field that has been prepared in the Hartle-Hawking or Unruh state, has recently been shown to exhibit a local extremum in its transition probability near the black hole horizon [K.K. Ng et al., New J. Phys. 24 (2022) 103018]. We show that a similar phenomenon is present in the transition rate of a UDW detector falling radially into a spinless Ba\~nados-Teitelboim-Zanelli (BTZ) black hole, with the scalar field prepared in the Hartle-Hawking state. We give extensive numerical results as a function of the detector's energy gap, the black hole's mass, and the detector's drop-off radius. Our results suggest that the effect is robust, motivating a search for a similar effect in other black hole spacetimes, and calling for an explanation of the physical origin of the effect.


title: Dark no more: The low luminosity stellar counterpart of a dark cloud in the Virgo cluster

abstract: We have discovered the stellar counterpart to the ALFALFA Virgo 7 cloud complex, which has been thought to be optically dark and nearly star-free since its discovery in 2007. This ~190 kpc long chain of enormous atomic gas clouds ($M_\mathrm{HI} \sim 10^9 \; \mathrm{M_\odot}$) is embedded in the hot intracluster medium of the Virgo galaxy cluster but is isolated from any galaxy. Its faint, blue stellar counterpart, BC6, was identified in a visual search of archival optical and UV imaging. Follow-up observations with the Green Bank Telescope, Hobby-Eberly Telescope, and Hubble Space Telescope demonstrate that this faint counterpart is at the same velocity as the atomic gas, is actively forming stars, and is metal-rich ($12 + \mathrm{(O/H)} = 8.58 \pm 0.25$). We estimate its stellar mass to be only $\log ( M_\ast/\mathrm{M_\odot}) \sim 4.4$, making it one of the most gas-rich stellar systems known. Aside from its extraordinary gas content, the properties of BC6 are entirely consistent with those of a recently identified class of young, low-mass, isolated, and star-forming clouds in Virgo, that appear to have formed via extreme ram pressure stripping events. We expand the existing discussion of the origin of this structure and suggest NGC 4522 as a likely candidate, however, the current evidence is not fully consistent with any of our proposed progenitor galaxies. We anticipate that other "dark" gas clouds in Virgo may have similarly faint, star-forming counterparts. We aim to identify these through the help of a citizen science search of the entire cluster.


title: Renormalization of the primordial inflationary power spectra

abstract: It has been suggested that the effects of renormalization significantly reduce the amplitude of the inflationary spectra at scales measurable in the cosmic microwave background. Via a gauge-invariant analysis, we compute the renormalized scalar and tensor power spectra and follow their evolution in an inflating universe that undergoes a transition to an FRW phase with a growing horizon. For perturbations originating from Minkowski vacuum fluctuations, we show that the standard prediction for the spectra on superhorizon scales is a late-time attractor, while they are UV finite at all times. Our result is independent of the equation of state after inflation, showing that the standard prediction is fully robust.


title: Revisiting thermoelectric effects in the crust of neutron stars

abstract: Context. Large thermal variations have been observed in neutron stars that typically are not aligned with density gradients. Such terms may activate the Biermann battery effect, leading to thermo-electric interactions and the generation of electromotive force. Aims. We aim to identify the impact a temperature anisotropy on a neutron star's crust can have in the evolution of its magnetic field, through the thermo-electric terms. Methods. We consider a neutron star crust with large temperature gradients, associated with long-lived hot spots, described by a Gaussian-type function localized. We simulate the interplay between the battery term and the Hall and Ohmic evolution numerically, for axisymmetric systems. Results. The results indicate that for crust temperatures of $\sim$$10^9$ K the toroidal field can be amplified up to $\sim$$10^{14}$ - $10^{15}$ G near the points of maximum temperature gradients, and it locally changes the architecture of the poloidal field lines. For internal crustal temperatures around $\sim$$10^8$ K, the temperature gradient generates fields of about two orders of magnitude lower. In such cases, saturation is achieved after some hundred thousand years, after which the battery and Ohmic dissipation balance each other. Conclusions. We conclude that the thermoelectric effect can impact the overall magnetic field evolution, provided that the thermal gradient is maintained for a sufficiently long time. Neutron stars endowned with moderate strength magnetic fields may be affected by the thermoelectric effect if the hotspots survive for timescales of a few kiloyears.


title: Kasner Eons

abstract: As shown in the classic work of Belinski, Khalatnikov and Lifshitz, the approach to a generic space-like singularity in general relativity consists of a sequence of eras and epochs in which the metric is locally Kasner, connected by brief transitions. When quantum gravity effects are included, we argue that a new type of transition arises, giving rise to Kasner eons: periods which are dominated by emergent physics at each energy scale. We comment on the different ways in which the Einsteinian eon may come to an end and show explicitly how additional Kasner eons arise in the interior of a black hole solution due to higher-derivative corrections.


title: Mass inflation without Cauchy horizons

abstract: Mass inflation is a well established instability, conventionally associated to Cauchy horizons (which are also inner trapping horizons) of stationary geometries, leading to a divergent exponential buildup of energy. We show here that finite (but often large) exponential buildups of energy are generically present for dynamical geometries endowed with slowly-evolving inner trapping horizons, even in the absence of Cauchy horizons. This provides a more general definition of mass inflation based on quasi-local concepts. We also show that various known results in the literature are recovered in the limit in which the inner trapping horizon asymptotically approaches a Cauchy horizon. Our results imply that black hole geometries with non-extremal inner horizons, including the Kerr geometry in general relativity, and non-extremal regular black holes in theories beyond general relativity, can describe dynamical transients but not the long-lived endpoint of gravitational collapse.


title: Exploring limits of dipolar quantum simulators with ultracold molecules

abstract: We provide a quantitative blueprint for realizing two-dimensional quantum simulators employing ultracold dipolar molecules or magnetic atoms by studying their accuracy in predicting ground state properties of lattice models with long-range interactions. For experimentally relevant ranges of potential depths, interaction strengths, particle fillings, and geometric configurations, we map out the agreement between the state prepared in the quantum simulator and the target lattice state. We do so by separately calculating numerically exact many-body wave functions in the continuum and single- or multi-band lattice representations, and building their many-body state overlaps. While the agreement between quantum simulator and single-band models is good for deep optical lattices with weaker interactions and low particle densities, the higher band population rapidly increases for shallow lattices, stronger interactions, and in particular above half filling. This induces drastic changes to the properties of the simulated ground state, potentially leading to false predictions. Furthermore, we show that the interplay between commensurability and interactions can lead to quasidegeneracies, rendering a faithful ground state preparation even more challenging.

categories: cond-mat.quant-gas physics.atm-clus physics.comp-ph physics.optics quant-ph


title: Universal predictions of Siegel modular invariant theories near the fixed points

abstract: We analyze a general class of locally supersymmetric, CP and modular invariant models of lepton masses depending on two complex moduli taking values in the vicinity of a fixed point, where the theory enjoys a residual symmetry under a finite group. Like in models that depend on a single modulus, we find that all physical quantities exhibit a universal scaling with the distance from the fixed point. There is no dependence on the level of the construction, the weights of matter multiplets and their representations, with the only restriction that electroweak lepton doublets transform as irreducible triplets of the finite modular group. Also the form of the kinetic terms, which here are assumed to be neither minimal nor flavor blind, is irrelevant to the outcome. The result is remarkably simple and the whole class of examined theories gives rise to five independent patterns of neutrino mass matrices. Only in one of them, the predicted scaling agrees with the observed neutrino mass ratios and lepton mixing angles, exactly as in single modulus theories living close to $\tau=i$.


title: Entanglement Microscopy: Tomography and Entanglement Measures via Quantum Monte Carlo

abstract: We develop a protocol, dubbed entanglement microscopy, to obtain the full reduced density matrix associated with subregions in quantum Monte Carlo simulations for bosonic and fermionic manybody systems. Our microscopy allows to perform quantum state tomography, and thus gives access to true entanglement measures, such as the logarithmic negativity (LN). We exemplify our method by studying the phase diagram near quantum critical points (QCP) in 2 spatial dimensions: the transverse field Ising model and a Gross-Neveu-Yukawa transition of Dirac fermions. Our main results are: i) the Ising QCP exhibits short-range entanglement with a finite sudden death of the LN both in space and temperature; ii) the Gross-Neveu QCP has a power-law decaying fermionic LN consistent with conformal field theory (CFT) exponents; iii) going beyond bipartite entanglement, we find no detectable 3-party entanglement in a large parameter window near the Ising QCP in 2d, in contrast to 1d. We also analytically obtain the large-temperature power law scaling of the fermionic LN for general interacting systems. Our approach allows one to perform quantum state tomography locally in a way that is analogous to atomic-scale imaging with a scanning tunneling microscope. Controlled entanglement microscopy opens a new window into quantum matter, with countless systems waiting to be explored.

categories: cond-mat.str-el cond-mat.stat-mech hep-th quant-ph


title: Consistent particle physics in metric-affine gravity from extended projective symmetry

abstract: It is well-known that the Einstein-Hilbert action exhibits a projective invariance in metric-affine gravity, generated by a single vector (just like diffeomorphisms). However, this symmetry offers no protection against formulating inconsistent models, e.g., with ghost and strong coupling problems. In this letter, we observe that non-minimal kinetic terms of Dirac spinors point to a new extended projective (EP) symmetry generated by a pair of vectors. We prove that the most general EP-invariant theory (at most quadratic in field strengths) is naturally free from all pathologies. Its spectrum only features the massless graviton and a single additional scalar field arising from the square of the Holst curvature. The scalar potential is suitable for inflation and our model moreover contains effective 4-Fermi interactions capable of producing fermionic dark matter. Finally, we point out an alternative double-vector symmetry that similarly leads to a healthy theory with a propagating vector field.


title: Anti-screening and nonequilibrium layer electric phases in graphene multilayers

abstract: Screening is a ubiquitous phenomenon through which the polarization of bound or mobile charges tends to reduce the strengths of electric fields inside materials. Here we show how photoexcitation can be used as a knob to transform conventional out-of-plane screening into anti-screening - the amplification of electric fields - in multilayer graphene stacks. We find that, by varying the photoexcitation intensity, multiple nonequilibrium screening regimes can be accessed, including near-zero screening, anti-screening, or overscreening (reversing electric fields). Strikingly, at modest continuous wave photoexcitation intensities, the nonequilibrium polarization states become multistable, hosting light-induced ferroelectric-like steady states with nonvanishing out-of-plane polarization (and band gaps) even in the absence of an externally applied displacement field in nominally inversion symmetric stacks. This rich phenomenology reveals a novel paradigm of dynamical quantum matter that we expect will enable a variety of nonequilibrium broken symmetry phases.


title: The imitation game reloaded: effective shadows of dynamically robust spinning Proca stars

abstract: We analyse the lensing images by dynamically robust rotating (mini-)Proca stars surrounded by thin accretion disks. Due to their peculiar geodesic structure we show that these images exhibit striking similarities with the ones of BHs, for appropriately chosen disk intensity profile, when imposing a GRMHD-motivated emission cut off. Additionally, and unlike the non-rotating case, these similarities prevail even when considering equatorial observations. This example illustrates how a horizonless compact object without light rings, with a plausible formation mechanism and dynamically robust, could mimic detailed features of black hole imagiology.


title: Properties of the brightest young stellar clumps in extremely lensed galaxies at redshifts 4 to 5

abstract: We study the populations of stellar clumps in three high-redshift galaxies, at z=4.92, 4.88 and 4.03, gravitationally lensed by the foreground galaxy clusters MS1358, RCS0224 and MACS0940, respectively. The lensed galaxies consist of multiple counter-images with large magnifications, mostly above $\rm \mu>5$ and in some cases reaching $\rm \mu>20$. We use rest-frame UV observations from the HST to extract and analyse their clump populations, counting 10, 3 and 11 unique sources, respectively. Most of the clumps have derived effective radii in the range $\rm R_{eff}=10-100$ pc, with the smallest one down to 6 pc, i.e. consistent with the sizes of individual stellar clusters. Their UV magnitudes correspond to $\rm SFR_{UV}$ mostly in the range $\rm 0.1-1\ M_\odot yr^{-1}$; the most extreme ones, reaching $\rm SFR_{UV}=5\ M_\odot yr^{-1}$ are among the UV-brightest compact ($\rm R_{eff}<100$ pc) star-forming regions observed at any redshift. Clump masses span a broad range, from $10^6$ to $\rm 10^9\ M_\odot$; stellar mass surface densities are comparable, and in many cases larger, than the ones of local stellar clusters, while being typically 10 times larger in size. By compiling published properties of clump populations at similar spatial resolution between redshift 0 and 5, we find a tentative evolution of $\rm \Sigma_{SFR}$ and $\rm \Sigma_{M_\star}$ with redshift, especially when very compact clumps ($\rm R_{eff}\leqslant20$ pc) are considered. We suggest that these trends with redshift reflect the changes in the host galaxy environments where clumps form. Comparisons with the local universe clumps/star clusters shows that, although rare, conditions for elevated clump $\rm \Sigma_{SFR}$ and $\rm \Sigma_{M_\star}$ can be found.


title: Kuramoto model subject to subsystem resetting: How resetting a part of the system may synchronize the whole of it

abstract: We introduce and investigate the effects of a new class of stochastic resetting protocol called subsystem resetting, whereby a subset of the system constituents in a many-body interacting system undergoes bare evolution interspersed with simultaneous resets at random times, while the remaining constituents evolve solely under the bare dynamics. We pursue our investigation within the ambit of the well-known Kuramoto model of coupled phase-only oscillators of distributed natural frequencies. Here, the reset protocol corresponds to a chosen set of oscillators being reset to a synchronized state at random times. We find that the mean $\omega_0$ of the natural frequencies plays a defining role in determining the long-time state of the system. For $\omega_0=0$, the system reaches a synchronized stationary state at long times, characterized by a time-independent non-zero value of the synchronization order parameter. Moreover, we find that resetting even an infinitesimal fraction of the total number of oscillators has the drastic effect of synchronizing the entire system, even when the bare evolution does not support synchrony. By contrast, for $\omega_0 \ne 0$, the dynamics allows at long times either a synchronized stationary state or an oscillatory synchronized state, with the latter characterized by an oscillatory behavior as a function of time of the order parameter, with a non-zero time-independent time average. Our results thus imply that the non-reset subsystem always gets synchronized at long times through the act of resetting of the reset subsystem. Our results, analytical using the Ott-Antonsen ansatz as well as those based on numerical simulations, are obtained for two representative oscillator frequency distributions, namely, a Lorentzian and a Gaussian. We discuss how subsystem resetting may be employed as an efficient mechanism to control attainment of global synchrony.


title: Practical Insights into Knowledge Distillation for Pre-Trained Models

abstract: This research investigates the enhancement of knowledge distillation (KD) processes in pre-trained models, an emerging field in knowledge transfer with significant implications for distributed training and federated learning environments. These environments benefit from reduced communication demands and accommodate various model architectures. Despite the adoption of numerous KD approaches for transferring knowledge among pre-trained models, a comprehensive understanding of KD's application in these scenarios is lacking. Our study conducts an extensive comparison of multiple KD techniques, including standard KD, tuned KD (via optimized temperature and weight parameters), deep mutual learning, and data partitioning KD. We assess these methods across various data distribution strategies to identify the most effective contexts for each. Through detailed examination of hyperparameter tuning, informed by extensive grid search evaluations, we pinpoint when adjustments are crucial to enhance model performance. This paper sheds light on optimal hyperparameter settings for distinct data partitioning scenarios and investigates KD's role in improving federated learning by minimizing communication rounds and expediting the training process. By filling a notable void in current research, our findings serve as a practical framework for leveraging KD in pre-trained models within collaborative and federated learning frameworks.


title: Spectra of the D2O dimer in the O-D fundamental stretch region: the acceptor symmetric stretch fundamental and new combination bands

abstract: The O-D stretch fundamental region of the deuterated water dimer, (D2O)2, is further studied using a pulsed supersonic slit jet and a tunable optical parametric oscillator infrared source. The previously unobserved acceptor symmetric O-D stretch fundamental vibration is detected, with Ka = 0 <-- 0 and 1 <-- 0 sub-bands at about 2669 and 2674 cm-1, respectively. Analysis indicates that the various water dimer tunneling splittings generally decrease in the excited vibrational state, similar to the three other previously observed O-D stretch fundamentals. Two new (D2O)2 combination bands are observed, giving information on intermolecular vibrations in the excited O-D stretch states. The likely vibrational assignments for these and a previously observed combination band are discussed.


title: Form factors for semileptonic B-decays with HISQ light quarks and clover b-quarks in Fermilab interpretation

abstract: We compute the vector, scalar, and tensor form factors for the $B\to \pi$, $B\to K$, and $B_s\to K$ amplitudes, which are needed to describe semileptonic $B$-meson decay rates for both the charged and neutral current cases. We use the highly improved staggered quark (HISQ) action for the sea and light valence quarks. The bottom quark is described by the clover action in the Fermilab interpretation. Simulations are carried out on $N_f = 2+1+1$ MILC HISQ ensembles at approximate lattice spacings from $0.15$ fm down to $0.057$ fm. We present blinded preliminary results for the form factors.


title: Efficient Unbiased Sparsification

abstract: An unbiased $m$-sparsification of a vector $p\in \mathbb{R}^n$ is a random vector $Q\in \mathbb{R}^n$ with mean $p$ that has at most $m<n$ nonzero coordinates. Unbiased sparsification compresses the original vector without introducing bias; it arises in various contexts, such as in federated learning and sampling sparse probability distributions. Ideally, unbiased sparsification should also minimize the expected value of a divergence function $\mathsf{Div}(Q,p)$ that measures how far away $Q$ is from the original $p$. If $Q$ is optimal in this sense, then we call it efficient. Our main results describe efficient unbiased sparsifications for divergences that are either permutation-invariant or additively separable. Surprisingly, the characterization for permutation-invariant divergences is robust to the choice of divergence function, in the sense that our class of optimal $Q$ for squared Euclidean distance coincides with our class of optimal $Q$ for Kullback-Leibler divergence, or indeed any of a wide variety of divergences.


title: Boosting gets full Attention for Relational Learning

abstract: More often than not in benchmark supervised ML, tabular data is flat, i.e. consists of a single $m \times d$ (rows, columns) file, but cases abound in the real world where observations are described by a set of tables with structural relationships. Neural nets-based deep models are a classical fit to incorporate general topological dependence among description features (pixels, words, etc.), but their suboptimality to tree-based models on tabular data is still well documented. In this paper, we introduce an attention mechanism for structured data that blends well with tree-based models in the training context of (gradient) boosting. Each aggregated model is a tree whose training involves two steps: first, simple tabular models are learned descending tables in a top-down fashion with boosting's class residuals on tables' features. Second, what has been learned progresses back bottom-up via attention and aggregation mechanisms, progressively crafting new features that complete at the end the set of observation features over which a single tree is learned, boosting's iteration clock is incremented and new class residuals are computed. Experiments on simulated and real-world domains display the competitiveness of our method against a state of the art containing both tree-based and neural nets-based models.


title: Hitting Meets Packing: How Hard Can it Be?

abstract: We study a general family of problems that form a common generalization of classic hitting (also referred to as covering or transversal) and packing problems. An instance of X-HitPack asks: Can removing k (deletable) vertices of a graph G prevent us from packing $\ell$ vertex-disjoint objects of type X? This problem captures a spectrum of problems with standard hitting and packing on opposite ends. Our main motivating question is whether the combination X-HitPack can be significantly harder than these two base problems. Already for a particular choice of X, this question can be posed for many different complexity notions, leading to a large, so-far unexplored domain in the intersection of the areas of hitting and packing problems. On a high-level, we present two case studies: (1) X being all cycles, and (2) X being all copies of a fixed graph H. In each, we explore the classical complexity, as well as the parameterized complexity with the natural parameters k+l and treewidth. We observe that the combined problem can be drastically harder than the base problems: for cycles or for H being a connected graph with at least 3 vertices, the problem is \Sigma_2^P-complete and requires double-exponential dependence on the treewidth of the graph (assuming the Exponential-Time Hypothesis). In contrast, the combined problem admits qualitatively similar running times as the base problems in some cases, although significant novel ideas are required. For example, for X being all cycles, we establish a 2^poly(k+l)n^O(1) algorithm using an involved branching method. Also, for X being all edges (i.e., H = K_2; this combines Vertex Cover and Maximum Matching) the problem can be solved in time 2^\poly(tw)n^O(1) on graphs of treewidth tw. The key step enabling this running time relies on a combinatorial bound obtained from an algebraic (linear delta-matroid) representation of possible matchings.


title: Learning Inverse Kinodynamics for Autonomous Vehicle Drifting

abstract: In this work, we explore a data-driven learning-based approach to learning the kinodynamic model of a small autonomous vehicle, and observe the effect it has on motion planning, specifically autonomous drifting. When executing a motion plan in the real world, there are numerous causes for error, and what is planned is often not what is executed on the actual car. Learning a kinodynamic planner based off of inertial measurements and executed commands can help us learn the world state. In our case, we look towards the realm of drifting; it is a complex maneuver that requires a smooth enough surface, high enough speed, and a drastic change in velocity. We attempt to learn the kinodynamic model for these drifting maneuvers, and attempt to tighten the slip of the car. Our approach is able to learn a kinodynamic model for high-speed circular navigation, and is able to avoid obstacles on an autonomous drift at high speed by correcting an executed curvature for loose drifts. We seek to adjust our kinodynamic model for success in tighter drifts in future work.