I ❤️ .NET

In this article, let's learn about how to save bandwidth with compression while sending and receiving data using HTTP Client in .NET.

Note: If you have not done so already, I recommend you read the article on Free up resources with cancellation while accessing APIs using HTTPClient in dotnet.

Table of Contents

1. Introduction
2. Advantages of (De) Compression
3. Decompress data when reading
4. Compress data when sending
5. Summary

Introduction

We send data over the internet in a text format called JSON, and we can make this data smaller to save time and internet space, which is similar to making a big package smaller for shipping. To achieve this, we use compression and decompression. Our server is capable of doing this, employing methods like gzip or deflate to shrink the data. When the user's device requests data, it can specify that it wants the data in a smaller format like gzip by using a special header called Accept-Encoding. The user's device understands how to open this compressed data, so it un-squishes it for use.

Conversely, when the user's device needs to send data back to the server, it must compress the data first using tools such as GZipStream or DeflateStream. It communicates to the server that it has squished the data using gzip through a special header called Content-Encoding.

Note that the API, or the web server where the API is deployed, depending on which component in your system you want to make responsible for compression and decompression, needs to support this. We can enable automatic decompression on the built-in SocketsHttpHandler

Advantages of (De) Compression

Before we look into examples, it's important to know that compression helps us with following advantages

• Reduced Bandwidth which can save data cost over wire.
• Improved performance because of small size.
• Better SEO which results in faster page loads because of small size data.

Decompress data when reading

To decompress data when reading, we first need to inform the server to compress and send the data. To do this we need to pass Accept-Encoding Header with gzip value to the API request. If the server API supports compression, it will compress and return the data in requested format. So far so good the below code example will help us to understand this.

When you deserialize as shown above you will get an deserialization error. To successfully deserialize the compressed data from server API, we need to set AutomaticDecompression property of SocketsHttpHandler to DecompressionMethods.GZip | DecompressionMethods.Deflate. This will automatically decompress the data when reading from the API.

Compress data when sending

This is a two step process. We need to compress the payload before sending it to the API and then we need to inform the API that the data is compressed. We have already learnt the best practice to use Stream while sending the data to API. So we will use the same approach here and modify to support compression.

We need to compress our payload, which is currently being streamed to memoryContentStream. Instead of passing memoryContentStream as is when sending a request, we want to pass a compressed content stream. In .NET, you can achieve this using streams like GZipStream or DeflateStream for different compression algorithms.

In the above code,

1. Create a new stream called compressedMemoryContentStream to hold the compressed content.
2. Wrap the following code in a using statement to ensure proper stream disposal.
3. Use the gzipStream class to compress the incoming memoryContentStream, passing compressedMemoryContentStream as the destination and CompressionMode.Compress to specify compression.
4. Copy the data from memoryContentStream (uncompressed data) to gzipStream and flush it.
5. Set the position of compressedMemoryContentStream back to 0, as this stream will be used to send data in the request, and we want to start from the beginning.
6. Instead of passing memoryContentStream, create a StreamContent object and pass compressedMemoryContentStream as its content. This will handle compressing the response body using streams.
7. Next step is to let the API know that the data is compressed. To do this we need to pass Content-Encoding Header with gzip value to the API request.

Summary

In this article, we learned about how to (de) compress data while receiving from and sending to APIs using HTTP Client in .NET. We learned about Compression and its advantages. Compression helps to save bandwidth and improve performance. We learnt how to compress the payload and inform the API that the data is compressed. We also learnt how to decompress the data when reading from the API. Now its time to improve performance of your API's using the above technique. We also learnt how to inform API using Accept-Encoding and Content-Encoding headers. I hope you found this useful. Thank you for reading.