Parallel and Distributed Deep Learning Paper Database

The paper database below was collected for the purpose of the paper "Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis". It contains works that utilize parallel and distributed computing resources for training Deep Neural Networks. This includes hardware architectures, data representation, parallelization strategies, distributed algorithms, system implementations, frameworks, and programming models.


Format

The papers are organized in the YAML format, with one entry per paper, sorted by publication year. Each entry contains the paper title, year of publication, category (corresponding to sections in the paper), keywords, datasets used in experiments, frameworks, and hardware architectures.


The following listing shows an example of such a paper entry:


'Large Scale Distributed Deep Networks':
    year: 2012
    categories: 
        - Distributed
        - Systems
    keywords:
        - Asynchronous SGD
        - DistBelief
        - Downpour SGD
        - Sandblaster LBFGS
        - Parameter server
        - Model-parallelism
        - Data-parallelism
        - Layer pipelining
        - Hybrid parallelism
        
    hardware:
        - CPU Cluster:
            nodes: 5100
            commlayer: Sockets
            
    experiments:
        datasets:
            - Speech Recognition (internal)
            - ImageNet
        networks:
            - 4-layer MLP
            - LCN


Download


Version Date Changes
papers.yml - (Coming Soon) February 26, 2018 First Release

References

arXiv
[1] T. Ben-Nun, T. Hoefler:
 Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis CoRR. Vol abs/1802.09941, Feb. 2018,