Deeptanshu Malik

Asynchronous Programming 101

Wed, 03 Jul 2019 00:00:00 +0000

In this blog post I’ll give a top-down introduction to asynchronous programming in C#. This post may be useful for anyone new to C# and for students who have learnt about threads and concurrency. I believe that a top-down approach will be ideal as it’ll first show what async programming is so that you can be interested in the how and why of it.

The code used in this blog is available on my GitHub here.

San Francisco, on some lazy Saturday afternoon in July, 2018.

A quick review of the fundamentals

Latency & Throughput

Latency is a measure of units of time taken to do work. Throughput is measure of the units of work done per unit of time.

Concurrency

Concurrency means multiple computations are happening at the same time^[1]. It is different than parallelism as explained in this stackoverflow answer:

Concurrency: A condition that exists when at least two threads are making progress. A more generalized form of parallelism that can include time-slicing as a form of virtual parallelism^[2].
Parallelism: A condition that arises when at least two threads are executing simultaneously^[2].

Tasks

A Task is an abstraction that represents an operation^[3]. This operation may or may not be backed by more than one thread and it may or may not be executed concurrently. We can use Tasks instead of working with threads directly so that we don’t need to manually, explicitly implement locking, joining and synchronizing operations for threads and we don’t need to manually create or destroy threads directly either. By default in C# Tasks use threads from a Thread Pool. A Thread Pool is a design pattern that maintains multiple threads such that they can be reused/recycled. We can use this design pattern is to reduce the latency of creating, starting and deleting threads repeatedly.

Synchronous versus Asynchronous Operations

A synchronous operation will do its work before returning to the caller whereas an asynchronous operation may do some (including all) part of its work after returning to the caller^[3].

An analogy to wrap up these fundamental concepts

Let’s consider a laundromat. If you put clothes in two machines at literally the same time, for example one washing machine with one hand and another washing machine with the other hand at the same time, then you’re loading the clothes in parallel. But if you load clothes simultaneously in two washing machine such that at a given instance in time you are only loading clothes in one washing machine at a time, then you are loading the clothes concurrently.

The time taken by the washing machine to wash one load of clothes will be the latency of a washing machine. To reduce time spent waiting for n loads of clothes to be washed, instead of running one machine n times, n machines can be run at the same time: the work done by a machine takes the same amount of time in both the cases but the difference is that the in the latter case amount of work done at in the same amount of time (n loads washed at the same time) is more, i.e., the throughput is more.

To do all these things maybe you asked the person at the front desk for quarters: you couldn’t do anything at the time, you were at the front of a queue and you gave the person two dollar bills and waited for them to return quarters. This was a synchronous operation. When a washing machine is running you still retain control- you can go ahead and load and start running another washing machine while the first one is still running - this makes the running of a washing machine an asynchronous operation.

Async Programming Demo

Let’s look at a demo to understand how async functions are defined and called and how async programming impacts program latency by improving throughput.

Asynchronous functions in the C# are defined by adding the async keyword to the function definition. They can be called like any other functions. Async Functions in C# must return a Task or some type of Task. In the demo program below we will see how these tasks can be initiated, how they can be run synchronously and asynchronously and how the code flow can be stopped to wait for the result of a task by using the await keyword.

Dummy Synchronous and Asynchronous Functions

Let’s implement a dummy async function:

private async Task<int> longRunningMultiplyBy10(int number)
{
    Console.WriteLine("# Initiated long running op on for the input number=" + number + " from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId + " #");

    //100 delays of 100 ms = 10 seconds
    for (int i = 0; i < 100 ; i++)
    {
        await Task.Delay(100);
    }

    Console.WriteLine("# Completed long running op on for the input number=" + number + " from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId + " #");
    return number * 10;
}

In this function we use the native asynchronous Delay function by calling it asynchronously using the await keyword to wait for a total of 10 seconds and then multiply the input number by 10 and return it. We will now see the difference in performance when it is called synchronously and asynchronously. To do so we will implement functions to take to input numbers, perform the dummy long running operation on each of them and return their sum.

The function to call longRunningMultiplyBy10 synchronously is:

private int syncMultiplyBy10AndAdd(int num1, int num2)
{
    Console.WriteLine("Calling long running Op from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    int a = longRunningMultiplyBy10(num1).Result;

    Console.WriteLine("Calling long running Op from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    int b = longRunningMultiplyBy10(num2).Result;

    Console.WriteLine("Executing return from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    return a + b;
}

Let’s write a function that uses asynchronous programming:

private async Task<int> asyncMultiplyBy10AndAddAwaitImmediately(int num1, int num2)
{
    Console.WriteLine("Calling long running Op from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    int a = await longRunningMultiplyBy10(num1);

    Console.WriteLine("Calling long running Op from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    int b = await longRunningMultiplyBy10(num2);

    Console.WriteLine("Executing return from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    return a + b;
}

We can observe here that we don’t need to await on the long running operation until we need to access the Task’s result— let’s write a function to reflect this and observe its performance as well.

private async Task<int> asyncMultiplyBy10AndAddAwaitAtEnd(int num1, int num2)
{
    Console.WriteLine("Calling long running Op from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    Task<int> a = longRunningMultiplyBy10(num1);

    Console.WriteLine("Calling long running Op from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    Task<int> b = longRunningMultiplyBy10(num2);

    Console.WriteLine("Executing return from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    return await a + await b;
}

Helper functions to call dummy functions and record their performance

We will make a call to demo to call our demo functions and measure their performance as follows:

private void demo()
{
    Console.WriteLine("##### Testing synchronous code #####");
    demoSyncFunction(syncMultiplyBy10AndAdd);
    Console.WriteLine("####################################" + Environment.NewLine);

    Console.WriteLine("##### Testing await immediately #####");
    demoAsyncFunction(asyncMultiplyBy10AndAddAwaitImmediately);
    Console.WriteLine("####################################" + Environment.NewLine);
    Console.WriteLine();

    Console.WriteLine("##### Testing await at end #####");
    demoAsyncFunction(asyncMultiplyBy10AndAddAwaitAtEnd);
    Console.WriteLine("####################################" + Environment.NewLine);
    Console.WriteLine();
}

private void demoSyncFunction(Func<int, int, int> asyncDemoFunction)
{
    int num1 = 15;
    int num2 = 33;
    int result;
    Stopwatch stopWatch = new Stopwatch();

    stopWatch.Start();
    result = asyncDemoFunction(num1, num2);
    stopWatch.Stop();
    displayRuntime(stopWatch.Elapsed);

}

private void demoAsyncFunction(Func<int, int, Task<int>> asyncDemoFunction)
{
    int num1 = 15;
    int num2 = 33;
    Stopwatch stopWatch = new Stopwatch();

    stopWatch.Start();

    int result = asyncDemoFunction(num1, num2).Result;

    stopWatch.Stop();

    displayRuntime(stopWatch.Elapsed);
}

private void displayRuntime(TimeSpan ts)
{
    string elapsedTime = String.Format("{0:00}min:{1:00}s.{2:00}ms",
        ts.Minutes, ts.Seconds,
        ts.Milliseconds / 10);

    Console.WriteLine("Runtime: " + elapsedTime);
}

The Results

The output is as follows:

For the function syncMultiplyBy10AndAdd

private int syncMultiplyBy10AndAdd(int num1, int num2)
{
    Console.WriteLine("Calling long running Op from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    int a = longRunningMultiplyBy10(num1).Result;

    Console.WriteLine("Calling long running Op from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    int b = longRunningMultiplyBy10(num2).Result;

    Console.WriteLine("Executing return from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    return a + b;
}

The output is:

##### Testing synchronous code #####
Calling long running Op from Thread 1
# Initiated long running op on for the input number=15 from Thread 1 #
# Completed long running op on for the input number=15 from Thread 8 #
Calling long running Op from Thread 1
# Initiated long running op on for the input number=33 from Thread 1 #
# Completed long running op on for the input number=33 from Thread 8 #
Executing return from Thread 1
Runtime: 00min:20s.08ms
####################################

We observe that Thread 1 began executing syncMultiplyBy10AndAdd. When it called longRunningMultiplyBy10 the execution of syncMultiplyBy10AndAdd was blocked. Thread 1 then began the execution of the long running operation which ran asynchronously. As the function ran asynchronously Thread 1 was released from the task of executing longRunningMultiplyBy10 . However, because syncMultiplyBy10AndAdd is synchronous, when longRunningMultiplyBy10 completed Thread 1 remained responsible for executing syncMultiplyBy10AndAdd throughout. We note that the function took 20s.08ms to execute.

Now when the function asyncMultiplyBy10AndAddAwaitImmediately was executed:

private async Task<int> asyncMultiplyBy10AndAddAwaitImmediately(int num1, int num2)
{
    Console.WriteLine("Calling long running Op from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    int a = await longRunningMultiplyBy10(num1);

    Console.WriteLine("Calling long running Op from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    int b = await longRunningMultiplyBy10(num2);

    Console.WriteLine("Executing return from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    return a + b;
}

The output was

##### Testing await immediately #####
Calling long running Op from Thread 1
# Initiated long running op on for the input number=15 from Thread 1 #
# Completed long running op on for the input number=15 from Thread 7 #
Calling long running Op from Thread 7
# Initiated long running op on for the input number=33 from Thread 7 #
# Completed long running op on for the input number=33 from Thread 8 #
Executing return from Thread 8
Runtime: 00min:20s.04ms
####################################

We observe that Thread 1 began executing asyncMultiplyBy10AndAddAwaitImmediately. The difference this time is that because the call to longRunningMultiplyBy10 was asynchronous Thread 1 was released from the task of executing asyncMultiplyBy10AndAddAwaitImmediately and another thread from the thread pool, Thread 7, resumed the execution. We note that the function took 20s.04ms. This is definitely faster than the synchronous-calls implementation and we can intuitively see how at scale in the real world the improvement in latency will be significant.

We can also observe an inefficiency: we awaited the completion of the long running operation earlier than when we needed to use its result. So although we used the thread pool to assign the task of executing asyncMultiplyBy10AndAddAwaitImmediately more efficiently to a thread by using async programming, we didn’t leverage the full power of async programming to allow long running operations (in this case longRunningMultiplyBy10) to run concurrently with the execution of the caller function (in this case asyncMultiplyBy10AndAddAwaitImmediately).

When we address this inefficiency and don’t await the results of the tasks until we need to use them and then test our function

private async Task<int> asyncMultiplyBy10AndAddAwaitAtEnd(int num1, int num2)
{
    Console.WriteLine("Calling long running Op from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    Task<int> a = longRunningMultiplyBy10(num1);

    Console.WriteLine("Calling long running Op from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    Task<int> b = longRunningMultiplyBy10(num2);

    Console.WriteLine("Executing return from Thread " + System.Threading.Thread.CurrentThread.ManagedThreadId);
    return await a + await b;
}

we get the following result:

##### Testing await at end #####
Calling long running Op from Thread 1
# Initiated long running op on for the input number=15 from Thread 1 #
Calling long running Op from Thread 1
# Initiated long running op on for the input number=33 from Thread 1 #
Executing return from Thread 1
# Completed long running op on for the input number=33 from Thread 8 #
# Completed long running op on for the input number=15 from Thread 6 #
Runtime: 00min:10s.04ms
####################################

The execution of the long running operation started as soon as it was called. The executions of the caller function and the long running operations continued concurrently. And the runtime of the effectively asynchronously coded function was only 00min:10s.04ms: that’s a solid 50% latency improvement over the synchronous implementation that called 2 long running operations. Since the asynchronously coded function operation calls the 2 long running ops in a way that allows them to run concurrently, the 50% improvement is as expected.

What is Async Programming and How it Works

As we have now observed, async programming allows us to effectively utilize threads to run tasks concurrently. When an async function is called the control is returned to the caller function immediately (as soon as the synchronous part of the asynchronous function is completed) and the execution of the caller and the called functions continues concurrently on separate threads until the caller function must await the completion of the called function. This helps improve the overall program latency (the total time the program takes to run) by increasing the throughput (the work done per unit of time) by effectively utilizing the thread pool to assign instances of tasks to different threads that run concurrently.

References

[1] https://web.mit.edu/6.005/www/fa14/classes/17-concurrency/
[2] Defining Multithreading Terms Oracle Multithreaded Programming Guide
[3] Chapter 14 Concurrency & Asynchrony of C# 5.0 in a Nutshell: The Definitive Reference Fifth Edition by Joseph Albahari, Ben Albahari (Author)

A shoutout to Hilton Lange: his explanation of asynchronous programming and demo async program inspired me to write this blog.

Pictures I clicked in the past 12 months that I am in awe of today

Sat, 11 May 2019 00:00:00 +0000

I was recently going through pictures that I had clicked in the past 12 months. As I did so I came across pictures I am in awe of today. This post is a collection of some of those pictures.

About these pictures

(Or click here to jump directly to the pictures)

I’ve shot them all on my iPhone X. I’ve cropped some and rotated some a bit to make sure that they are not tilted. But I have not edited them beyond this. I have not applied any filters or changed the contrast etc. My goal when I take a picture is to preserve that moment to see the picture again later exactly as I saw it. Of course, I’ve made the implicit assumption here that how my eyes perceive something and my phone camera captures it are adequately similar. Also, I feel that if I make even the slightest edit to my picture like such as changing its contrast or brightening it - that picture is sufficiently so far removed from reality that it might as well be a cartoon sketch.

There’s some degree of serendipity involved which led to each picture. In almost all cases I’ve taken the picture on a walk that I went on on a whim or when I was going somewhere, whether commuting to work or intentionally to a tourist spot or to do some recreational activity, and I just looked outside outside the cab window or kept my eyes open and remained In the present on my walk and saw a sight I wanted to capture.

So, in conclusion, this post is a collection of pictures from the past 12 months that looking back I’m proud I clicked. It’s not a collection of pictures of moments of pride or happiness from the past 12 months. This post is also a collection of pictures that I had taken during my travels in the past 12 months. However I don’t intend for it to be a travel diary by any means. I don’t want to write a travel blog. At most, perhaps someday in the distant future, I will write a series of city-by-city reviews of the American cities that I’ve visited.

The Pictures

Skylines of great American cites / Pictures of American prosperity / Capitalism creates a beautiful world / (I couldn’t choose a title)

San Francisco.

I was walking in Sf downtown my first weekend there after I had arrived in the city. After having spent a semester in West Lafayette, IN, it was therapeutic to be in a densely packed urban environment.

I took this picture from another ferry. There was no plan to go on a ferry. We were at Pier 39, saw a tourist ferry ride being advertised, decided to go on it. The color of the water, the sky, the city skyline and the bridge all make this picture.

LA skyline.

The greatest view in the world. Manhattan skyline.

WTC.

Seattle panorama.

Seattle on a rare not-gray-and-damp day.

Evening sunlight and SF skyline. And one prominent red lamppost but what the hell.

Sky Pictures

5:45 am sky in Seattle.

Evening Midwestern sky. The last time I played tennis at Purdue outdoors was under this sky.

California blue.

Menlo Park, CA, Caltrain station in the evening.

Cold January day in Indiana.

Miscellaneous

Santa Monica Beach. The pictured turned out to be this good purely by accident.

Snow @ Purdue.

Schwartz Tennis Center at Purdue University. Walk down this corridor to a tennis court.

I was in a train, saw this bridge approaching, took a picture of it.

Fall color at Purdue.

Sunnyvale in the evening.

Blue California sky. Blue California water. 4 identical ships. Taken on a ferry ride from Redwood City to SF.

Mount Rainier as seen from the floating bridge that connects Seattle and Bellevue.

Worst snow storm in 20 years in Seattle but the lights at this port are on and work continues. Amazing.

A random building in downtown Seattle.

Chicago financial district the day before Christmas.

Nothing special about this picture itself but I felt compelled to include it

Warren Tavern in Boston. George Washington drank here.

A Most Life Affirming Article by Riva Tez

Wed, 17 Apr 2019 00:00:00 +0000

I read this 3 days ago and it has made a not-insignificant improvement to my quality of life. Excellent article by @rivatez https://t.co/XmQ3bQWsEM
— Deeptanshu Malik (@deeptanshuMalik) April 17, 2019

From the article:

“To be in awe. That’s what the days are for.”

“To think that some of the best days of our lives haven’t happened yet, no matter how bad the current day has been, how bad the situation, how poor you currently are or how anguished.”

“Another day is another opportunity for kindness. It’s another day for passion. It’s another day to carve out your reality."

A new day is a new opportunity to carve out your reality

Read the whole article here: http://idealistmag.com/sleep/6am/

And you can find the author of this article, Riva-Melissa Tez, on Twitter here: @rivatez

Sky Colors in US Tech Hubs

Sun, 03 Mar 2019 00:00:00 +0000

This article is also available as a step-by-step tutorial on my Github here

Motivation for this article

I’ve recently moved to Seattle, WA. Before moving here I had read that the city is known for its gray skies. It’s also famous for having overcast clouds more than 200 days per year. In the few weeks that I’ve been here I’m reminded of this fact almost daily. I became curious: how often does Seattle have gray skies compared to other cities? Because I’m in the tech industry, I was curious about those other cities in the country that are also tech-hubs. I had a nagging suspicion that that the difference in the daily sky colors of Seattle and other tech hubs won’t be too different as the other tech-hubs are also coastal cities.

I had spent the last summer in the Bay Area for an internship and I had noticed that the skies in San Francisco were gray more often than they were blue. I had not expected that at all: I had imagined that San Francisco has sunny days all year round. The reality was that mist set in most days and as a consequence the sky was usually a light shade of gray. This recollection led me to think that perhaps the gray-sky weather in Seattle is not that different from other cities where I could have possibly been if I were not in Seattle, i.e., other tech hubs.

So I decided to make a simple data visualization: I made vertical bar charts where from left to right the most frequently occurring color of the sky everyday of the year is shown. I made these bar charts for the major tech-hubs I identified in the weather database I found to visually see just how different the color of the sky is in these cities throughout the year.

Because this project is based on open data provided by an Israeli (data from an open database on Kaggle provided by David Beniaguev. ) and Tel-Aviv is definitely a major tech-hub also, a bonus plot: In the data visualizations above you’ll see the color of the sky in the tech hubs of western hemisphere everyday of the year throughout the year. The plots are bar plots. Each bar in the bar chart represents one day, there are 365 bars corresponding to the days of the year, and each bar is a color that is the most frequently occurring color of the sky on that day in that city over the past few years.

Methodology

I found a database of hourly weather data for cities on Kaggle.

Raw data file from Kaggle

I extracted weather data for one city at a time from the raw data file. I mapped each date to day of the year. I calculated day-of-year from the year-month-day value. For example, 2015-01–01 is mapped to 1 as 1st Jan is the second day of the year and 2016-01–02 is mapped to 2 as 2nd Jan is the 2nd day of the year. I discarded data for what qualifies, roughly, as night time. As we are concerned with the differences in the colors of the sky and the color of the sky is always same at night we can disregard weather data for night time.

For each day-of-year for most hours I had multiple weather descriptions. Because I wanted to see what the sky looks like most often I chose the mode of the weather description value for each hour, i.e., I chose the most frequently reoccurring or most common weather description for each hour of each day. Similarly, because I wanted to visualize one color for one day I filtered the data further and discarded the hour values to keep only the day-of-year values and the most frequently reoccurring or most common weather description for each day of the year.

I now had a column of day-of-year and a corresponding column of weather descriptions.

At the very beginning of my data processing I had listed all the distinct weather descriptions that were there in the raw data for all cities. I then stored key-value pairs of the weather descriptions and their rgb color values. I calculated the rgb values using my Mac’s Digital Color Meter. I searched on Google Images for each distinct weather description (except for the different types of thunderstorms, I repeated the rgb values for those) and then used the Digital Color Meter to record the rgb value of the color of the sky from a relevant image search result. The process was extremely tedious, inefficient and subjective but perfectly adequate. I merged this information with the column of day-of-year and a corresponding column of weather descriptions such that the weather descriptions now had their color values associated with them and therefore, each day-of-year had a color value associated with it also.

Finally, I made a vertical bar chart such that each bar represents one day, the bars in the chart represent day 1 through day 366 of the year and the color of each bar corresponds to, what is according to my subjective opinion, the color of the sky on that day as per the weather description of that corresponding day of the year.

I used Pandas for my data analysis and Plot.ly to make the bar charts. You can see all the code, formatted as a step-by-step tutorial, for this project in this Jupyter NotebooK.

This project is based on data from an Open Database on Kaggle provided by David Beniaguev. Thank you David!

Concurrent Programming Reading list

Thu, 17 Jan 2019 00:00:00 +0000

This blog post is a collection of articles about threads, locks and concurrency.

Some context

I’ve had a lot of time to kill in the last few weeks so I recently completed a series of 3 classes, a specialization, on Coursera — Parallel, Concurrent, and Distributed Programming in Java Specialization.

The first two classes, Parallel Programming in Java and Concurrent Programming in Java were good revisions of material I had studied in one of my computer engineering classes at Purdue exactly an year ago. In the Distributed Programming in Java class I used Hadoop and Spark for the first time and revised more material related to networking and about different approaches to combine distribution with multithreading that I had learned in my classes at Purdue. This class was mostly about high-level abstractions and frameworks that are useful in distributed systems.

In the Concurrent Programming class I found various articles listed under the ‘Optional Reading’ section that were extremely resourceful. In college I had studied concurrency in my computer architecture class. I had referred to the textbook and my professor’s slides. I don’t have that course textbook or my professor’s slides anymore. So I’ve made this list of articles to be my go-to resource for fundamentals of concurrency.

The list

Threads

Wikipedia article on Threads

Structured Locks

Unstructured Locks

Tutorial on Lock Objects in Java

Liveness

Wikipedia article on Deadlock and Livelock
Java examples of deadlock due to synchronized methods and starvation and livelock

Critical sections

Object-Based Isolation

Wikipedia article on Monitors

Atomic Variables in Java

Tutorial on Atomic Integers in Java
Article in Java theory and practice series on Going atomic.
- “By exposing new low-level coordination primitives internally, and providing a set of public atomic variable classes, it now becomes practical, for the first time, to develop wait-free, lock-free algorithms in the Java language.”
Wikipedia article on Atomic Wrapper Classes in Java

Miscellanious

Dining philosophers problem: an example problem often used in concurrent algorithm design to illustrate synchronization issues and techniques for resolving them