KISS Seminars

Seminars for industrial education and training

Our industrial KISS seminars are based on the outcome of the performed research and start in summer 2021.

In 2021, we offered seminars on Advanced Training Methods for Deep Neural Networks, Simplification and Compression of Neural Networks and Optimization and Deployment of Deep Neural Networks for Heterogenous Systems.

Stay tuned for our KISS seminars coming up in 2022!

Recent Seminars

Advanced Training Methods for Deep Neural Networks

The third KISS Seminar discusses the training of data and memory intensive neural networks. The memory requirements of these highly complex networks especially during training can quickly exceed the capacity of a single computer or graphics card. The seminar addresses this problem by explaining the basics of resource consumption and presenting possible solutions.

In detail, the seminar is divided into the following chapters:

  • Basics of resource consumption during training
  • Data Parallel Training
  • Model Parallel Training
  • Pipeline Parallel Training
  • Distributed training
  • Mixed-precision training / use of tensor cores
  • Input pipelines & data handling
  • Data dependencies & Receiptive field & Scaling
  • Training with synthetic data

Simplification and Compression of Neural Networks

The last KISS seminar addressed the challenges of using large neural models. It presented different techniques for reducing the complexity and the size of a trained model.

The seminar discussed an end-to-end pipeline for model compression. It explained various state-of-the-art compression algorithms, with a focus on quantization and pruning algorithms. Practical use cases using common compression frameworks exemplified typical application scenarios.

By these means, the seminar introduced different compression tools and demonstrated their usability with several practical examples and hands-on exercises.

In more detail, the following topics were addressed:

  • Deep Compression: End-to-End compression
  • Comparison of different compression methods
  • Model Profiling
  • Model adaptations for compression
  • Compression with Pytorch
  • Introduction to Intellabs Distiller
  • Introduction to Microsoft Neural Network Intelligence
  • Recommendations for best practice
  • Comparisons with Compiler Optimizations
  • Useful examples
  • Hands-on exercises

The seminar was offered as an online seminar using Microsoft Teams as the online meeting platform. Furthermore the practical exercises were performed using virtual machines on Google Cloud Platform (GCP).

Optimization and Deployment of Deep Neural Networks for Heterogenous Systems

This seminar was the first in line of three seminar offerings from the KISS project. The focus of this seminar was on the transformation of a pre-trained AI model into an executable module optimized for the target platform.

Furthermore, the seminar also teached the basics of neural network compression using tools such as the Intellabs Distiller or the Microsoft Neural Network Interface (NNI). It was also shown how the interaction between distillers and DL compilers can reduce the inference time.

We addressed the following topics:

  • Basics of the DL compilers TensorRT and TVM
  • Basics about exchange formats, e.g. ONNX
  • Application of TensorRT and TVM in detail
    • Practical examples
    • Inference with reduced computational accuracy
    • Best practice
  • Performance profiling to identify bottlenecks
  • Basics of compression tools, e.g. compression in Pytorch / TFLite, Intellabs Distiller, Microsoft Neural Network Interface (NNI)
  • Interaction between compression tools and DL compilers
  • Integration and deployment of neural networks in C/C++
  • Target platforms, including CPU (x86/ARM), Nvidia desktop GPUs, Nvidia Jetson boards and Google Coral Edge TPU


The seminar was offered as an online seminar using Microsoft Teams as the online meeting plattform. Furthermore the practical exercises were performed using virtual machines on Google Cloud Plattform (GCP).

If you are a company, and have a specific need concerning the training content, the seminar can also be tailored to your specific needs for an individual training. To know more about this possibility, please contact us.

Also, if you have any suggestions, requests or questions about our seminars, feel free to contact us here!

 

Partners

The Friedrich-Alexander-Universität Erlangen-Nürnberg is a project partner.

Back to the KISS Project Overview

Credits Header: Fraunhofer IIS/fotomek – fotolia.de