Understanding incremental decoding in fairseq
Recently, I’ve been learning about sequence-to-sequence translation systems and going through Pytorch’s fairseq code. I’ve been focusing on the convolutional seq-to-seq method by Gehring et al. […]
Distributed data parallel training using Pytorch on AWS
In this post, I’ll describe how to use distributed data parallel techniques on multiple AWS GPU servers to speed up Machine Learning (ML) training. Along […]
Pipelining data processing and host-to-device data transfer
The purpose of this post is to show how to use multi-threading to parallelize data processing with data transfer from pageable to page-locked memory. I […]
Neural Network Loss Visualization
Plotting its shape helps in understanding the properties and behaviour of a function. Unfortunately since we live in a 3D world, we can’t visualize functions […]
Wrapping a Python Application into a Web Service using mod_wsgi and gunicorn
So you developed a cool AI algorithm and want to show it off through a web service? You know a lot about AI algorithms and […]
Understanding Matrix Multiplication on a Weight-Stationary Systolic Architecture
If you follow the hardware for deep learning space, you may have heard of the term “systolic array”. A 2D systolic array forms the heart […]
Understanding Roofline Charts
The purpose of this post is to discuss my current understanding of roofline charts. Let me lay some background first. Before I got into machine […]
Data Augmentation in SSD (Single Shot Detector)
Over the past few days, I have been investigating how SSD (Single Shot Detector), an object detector introduced in the following paper in Dec 2016 […]
Efficiently Computing the Fisher Vector Product in TRPO
The purpose of this post is to provide math proofs and clarify some implementation details in the recently introduced reinforcement learning method called “Trust Region […]
Initializing Weights for the Convolutional and Fully Connected Layers
You may have noticed that weights for convolutional and fully connected layers in a deep neural network (DNN) are initialized in a specific way. For […]