Systems for ML

Workshop on Systems for ML and Open Source Software at NeurIPS 2018

December 7, 2018

Deequ - Data Quality Validation for Machine Learning Pipelines: Sebastian Schelter (Apache Software Foundation); Tammo Rukat (Amazon Research); Dustin Lange (Amazon, USA); Philipp Schmidt (Amazon Research)

BlueConnect: Novel Hierarchical All-Reduce on Multi-tired Network for Deep Learning: Minsik Cho (IBM Research); Ulrich Finkler (IBM Research); David Kung (IBM Research)

Lipizzaner: A System That Scales Robust Generative Adversarial Network Training: Tom Schmiedlechner (MIT CSAIL); Ignavier Ng Zhi Yong (MIT CSAIL); Abdullah Al-Dujaili (MIT CSAIL); Erik Hemberg (CSAIL); Una-May O’Reilly (MIT)

Ballet: A lightweight framework for open-source, collaborative feature engineering: Micah J. Smith (MIT); Kelvin Lu (MIT); Kalyan Veeramachaneni (MIT)

Elastic CoCoA: Scaling In to Improve Convergence: Michael W Kaufmann (IBM Research, Karlsruhe Institute of Technology); Thomas Parnell (IBM Research); Kornilios Kourtis (IBM Research)

Population Based Training as a Service: Ang Li (Google DeepMind, Mountain View); Ola Spyra (DeepMind); Sagi Perel (Google); Valentin Dalibard (Google DeepMind); Max Jaderberg (Google); Chenjie Gu (Deepmind); David Budden (DeepMind); Tim Harley (); Pramod Gupta (DeepMind)

Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes: Xianyan Jia (Tencent Inc.); Shutao Song (Tencent Inc.); Shaohuai Shi (Hong Kong Baptist University); Wei He (Tencent Inc.); Yangzihao Wang (Tencent Inc.); Haidong Rong (Tencent Inc.); Feihu Zhou (Tencent Inc.); Liqiang Xie (Tencent Inc.); Zhenyu Guo (Tencent Inc.); Yuanzhou Yang (Tencent Inc.); Liwei Yu (Tencent Inc.); Tiegang Chen (Tencent Inc.); Guangxiao Hu (Tencent Inc.); Xiaowen Chu (Hong Kong Baptist University)

Dynamic Scheduling of MPI-based Distributed Deep Learning Training Jobs: Tim P Capes (SAIC Toronto); Vishal Raheja (SAIC Toronto); Mete Kemertas (SAIC Toronto); Iqbal Mohomed (Samsung Research America)

Infer2Train: leveraging inference for better training of deep networks: Elad Hoffer (Technion); Berry Weinstein (IDC); Itay Hubara (Technion); Sergei Gofman (Habana Labs); Daniel Soudry (Technion)

Learning kernels that adapt to GPU: Siyuan Ma (The Ohio State University); Mikhail Belkin (Ohio State University)

Accelerating Recurrent Neural Networks through Compiler Techniques and Quantization: Li-Wen Chang (Microsoft); Yang Chen (Microsoft); Wenlei Bao (Microsoft); Amit Agarwal (Microsoft); Eldar Akchurin (Microsoft); Ke Deng (Microsoft); Emad Barsoum (Microsoft)

Large-Batch Training for LSTM and Beyond: Yang You (UC Berkeley); Chris Ying (Google Brain); Cho-Jui Hsieh (UCLA, Google); James Demmel (UC Berkeley); Kurt Keutzer (EECS, UC Berkeley); Jonathan Hseu (Google Brain)

Explore-Exploit: A Framework for Interactive and Online Learning: Honglei Liu (Facebook Conversational AI); Anuj Kumar (Facebook Conversational AI); Wenhai Yang (Facebook Conversational AI); Benoit Dumoulin (Facebook Conversational AI)

Dynamic Automatic Differentiation of GPU Broadcast Kernels: Jarrett Revels (MIT)

Pythia - A platform for vision and language research: Amanpreet Singh (Facebook); Vivek Natrajan (Facebook); Yu Jiang (Facebook AI Research); Xinlei Chen (Facebook AI Research); Meet Shah (Facebook AI Research); Marcus Rohrbach (); Dhruv Batra (Facebook); Devi Parikh (Facebook)

Rethinking floating point for deep learning: Jeff Johnson (Facebook AI Research)

Explain to Fix: A Framework to Interpret and Correct DNN Object Detector Predictions: Denis A Gudovskiy (Panasonic); Alec Hodgkinson (Panasonic); Takuya Yamaguchi (Panasonic); Yasunori Ishii (Panasonic); Sotaro Tsukizawa (Panasonic)

Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD: Jianyu Wang (Carnegie Mellon University); Gauri Joshi (Carnegie Mellon University)

Massively Parallel Hyperparameter Tuning: Liam Li (Carnegie Mellon University); Kevin Jamieson (U Washington); Afshin Rostamizadeh (); Ekaterina Gonina (); Moritz Hardt (University of California, Berkeley); Benjamin Recht (UC Berkeley); Ameet Talwalkar (CMU)

Reuse in Pipeline-Aware Hyperparameter Tuning: Liam Li (Carnegie Mellon University); Evan Sparks (Determined AI); Kevin Jamieson (U Washington); Ameet Talwalkar (CMU)

Making Classical Machine Learning Pipelines Differentiable: A Neural Translation Approach: Gyeong-In Yu (Seoul National University); Saeed Amizadeh (Microsoft); Byung-Gon Chun (Seoul National University); Markus Weimer (Microsoft); Matteo Interlandi (Microsoft)

Parallel training of linear models without compromising convergence: Nikolas Ioannou (IBM Research); Celestine Duenner (IBM Research); Kornilios Kourtis (IBM Research); Thomas Parnell (IBM Research)

Machine Learning at Microsoft with ML.NET: Matteo Interlandi (Microsoft); Sergiy Matusevych (Microsoft); Mikhail Bilenko (Yandex); Saeed Amizadeh (Microsoft); Shauheen Zahirazami (Microsoft); Markus Weimer (Microsoft)

Fashionable Modelling with Flux: Michael J Innes (Julia Computing); Elliot Saba (Julia Computing); Keno Fischer (Julia Computing); Dhairya Gandhi (Julia Computing); Marco Concetto Rudilosso (University College London); Neethu Mariya Joy (Birla Institute of Technology and Science); Tejan Karmali (National Institute of Technology, India); Avik Pal (Indian Institute of Technology); Viral B. Shah (Julia Computing)

Don’t Unroll Adjoint: Differentiating SSA-Form Programs: Mike J Innes (Julia Computing)

Deep Learning Inference on Commodity Network Interface Cards: Giuseppe Siracusano (NEC Laboratories Europe); Davide Sanvito (Politecnico di Milano); Salvator Galea (University of Cambridge); Roberto Bifulco (NEC Laboratories Europe)

Stochastic Gradient Push for Distributed Deep Learning: Mahmoud Assran (McGill University/Facebook FAIR); Nicolas Loizou (University of Edinburgh); Nicolas Ballas (Facebook FAIR); Mike Rabbat (Facebook FAIR)

TrIMS: Transparent and Isolated Model Sharing forLow Latency Deep Learning Inference in Function asa Service Environments: Abdul Dakkak (UIUC); Cheng Li (UIUC); Simon Garcia de Gonzalo (UIUC); Jinjun Xiong (IBM Thomas J. Watson Research Center); Wen-Mei Hwu (University of Illinois at Urbana-Champaign)

Benchmarking and Optimization of Gradient Boosting Decision Tree Algorithms: Andreea Anghel (IBM Research); Nikolaos Papandreou (IBM Research Zurich); Thomas Parnell (IBM Research); Alessandro De Palma (IBM Research); Charalampos Pozidis (IBM Research Zurich)

ensmallen: a flexible C++ library for efficient function optimization: Shikhar Bhardwaj (Delhi Technological University); Ryan Curtin (RelationalAI); Marcus Edel (Free University of Berlin); Yannis Mentekidis (None); Conrad Sanderson (Data61/CSIRO)

Coded Elastic Computing: Yaoqing Yang (Carnegie Mellon University); Matteo Interlandi (Microsoft); Pulkit Grover (Carnegie Mellon University); Soummya Kar (); Saeed Amizadeh (Microsoft); Markus Weimer (Microsoft)

Dynamic Scheduling For Dynamic Control Flow in Deep Learning Systems: Jinliang Wei (Carnegie Mellon University); Garth A Gibson (Carnegie Mellon University); Vijay Vasudevan (Google Brain); Eric Xing (Petuum Inc. and CMU)

Deep Neural Inspection using DeepBase: Eugene Wu (Columbia University); Carl Vondrick (); Yiru Chen (Columbia University); Thibault Sellam (Columbia University); Yiliang Shi (Columbia University); Boyuan Chen (Columbia University)

Accelerating Deep Learning Workloads through Efficient Multi-Model Execution: Deepak Narayanan (Stanford); Keshav Santhanam (Stanford University); Amar Phanishayee (Microsoft Research); Matei Zaharia (Stanford and Databricks)

Training with Low-precision Embedding Tables: Jiyan Yang (Facebook Inc.); Hector Yuen (Facebook Inc.); Jian Zhang (Stanford)

Analysis of the Time-To-Accuracy Metric and Entries in the DAWNBench Deep Learning Benchmark: Cody Coleman (Stanford); Daniel Kang (Stanford University); Deepak Narayanan (Stanford); Luigi Nardi (Stanford); Tian Zhao (Stanford University); Jian Zhang (Stanford); Peter D Bailis (Stanford University); Kunle Olukotun (Stanford University); Christopher Re (Stanford University); Matei Zaharia (Stanford and Databricks)

ToyBox: Better Atari Environments for Testing Reinforcement Learning Agents: Emma Tosch (University of Massachusetts); John Foley (Smith College); Kaleigh Clary (University of Massachusetts); David Jensen (University of Massachusetts Amherst)

Image Classification at Supercomputer Scale: Chris Ying (Google Brain); Sameer Kumar (Google, Inc.)

Scalable CNN Training on Large-Scale HPC Systems: Nikoli Dryden (University of Illinois at Urbana-Champaign); Naoya Maruyama (Lawrence Livermore National Laboratory); Tom Benson (Lawrence Livermore National Laboratory); Tim Moon (Lawrence Livermore National Laboratory); Marc Snir (University of Illinois at Urbana-Champaign); Brian Van Essen (Lawrence Livermore National Laboratory)

AE: A domain-agnostic platform for adaptive experimentation: Konstantin Kashin (Facebook); Eytan Bakshy (Facebook)

VDMS: An Efficient Big-Visual-Data Access for Machine Learning Workloads: Luis Remis (Intel Labs); Vishakha Gupta-Cledat (Intel Labs); Christina R Strong (Intel Labs); Ragaad Altarawneh (Intel Labs)

Effortless Machine Learning on TPUs with Julia: Keno M Fischer (Julia Computing Inc); Elliot Saba (Julia Computing Inc)

Building Highly-Available Geo-Distributed Datastores for Continuous Learning: Nitin Agrawal (Samsung Research); Ashish Vulimiri (Samsung Research)

Dali: Lazy Compilation of Dynamic Computation Graphs: Jonathan Raiman (OpenAI)

A Case for Serverless Machine Learning: Joao Carreira (UC Berkeley); Pedro Fonseca (Purdue University); Alexey Tumanov (UC Berkeley); Andrew M Zhang (UC Berkeley ); Randy Katz (UC Berkeley)

A Case for Dynamic GPU Inference Multitenancy and Scheduling: Xiangxi Mo (UC Berkeley); Paras Jain (UC Berkeley); Harikaran Subbaraj (University of California, Berkeley); Ajay Jain (Massachusetts Institute of Technology); Alexey Tumanov (UC Berkeley); Joseph Gonzalez (UC Berkeley); Ion Stoica (UC Berkeley)

Weight Re-Initialization through Cyclical Batch Scheduling: Norman Mu (University of California, Berkeley); Zhewei Yao (University of California, Berkeley); Amir Gholami (UC Berkeley); Michael Mahoney (“University of California, Berkeley”); Kurt Keutzer (UC Berkeley)

Automatic Batching as a Compiler Pass in PyTorch: James Bradbury (Google Brain); Chunli Fu (Columbia University)

Lagrange Coded Computing: Optimal Design for Resiliency, Security, and Privacy: Qian Yu (University of Southern California); Netanel Raviv (Caltech); Songze Li (University of Southern California); Seyed Mohammadreza Mousavi Kalan (University of Southern California); Mahdi Soltanolkotabi (USC); Salman Avestimehr (University of Southern California)

Model Assertions for Debugging Machine Learning: Daniel Kang (Stanford University); Deepti Raghavan (Stanford University); Peter D Bailis (Stanford University); Matei Zaharia (Stanford and Databricks)

Just-in-Time Dynamic-Batching: Sheng Zha (Amazon Web Services); Ziheng Jiang (University of Washington); Haibin Lin (Amazon Web Services); Zhi Zhang (Amazon)