Advancing the state of the art

Language encoders encode words and phrases in ways that capture their local semantic relatedness, but are known to be globally inconsistent. Global inconsistency can seemingly be corrected for, in part, by leveraging signals from knowledge bases, but previous results are partial and limited to monolingual English encoders. We extract a large-scale multilingual, multi-word analogy dataset from...

Realizing the potential of quantum computing will require achieving sufficiently low logical error rates. Many applications call for error rates below 10^-15, but state-of-the-art quantum platforms typically have physical error rates near 10^-3. Quantum error correction (QEC) promises to bridge this divide by distributing quantum logical information across many physical qubits so that errors...

Leveraging machine-learning (ML) techniques for compiler optimizations has been widely studied and explored in academia. However, the adoption of ML in general-purpose, industry strength compilers has yet to happen. We propose MLGO, a framework for integrating ML techniques systematically in an industrial compiler -- LLVM. As a case study, we present the details and results of replacing the...

Common datasets have the form of elements with keys (e.g., transactions and products) and the goal is to perform analytics on the aggregated form of key and frequency pairs. A weighted sample of keys by (a function of) frequency is a highly versatile summary that provides a sparse set of representative keys and supports approximate evaluations of query statistics. We propose private weighted...

Modern deep learning models have achieved great success in predictive accuracy for many data modalities. However, their application to many real-world tasks is restricted by poor uncertainty estimates, such as overconfidence on out-of-distribution (OOD) data and ungraceful failing under distributional shift. Previous benchmarks have found that ensembles of neural networks (NNs) are typically...

We propose a multitask training method for attention-basedend-to-end speech recognition models. We regularize the de-coder in a listen, attend, and spell model by multitask trainingon both audio-text and text-only data. Trained on the 100-hoursubset of LibriSpeech, the proposed method leads to an 11%relative performance improvement over the baseline and is com-parable to language model shallow...

Advances in generative modeling and adversarial learning have given rise to renewed interest in smooth games. However, the absence of symmetry in the matrix of second derivatives poses challenges that are not present in the classical minimization framework. While a rich theory of average-case analysis has been developed for minimization problems, little is known in the context of smooth games....