I ❤️ .NET

Table of Contents

1. What we gonna do?
2. Why we gonna do?
3. How we gonna do?
4. Summary

What we gonna do?

Your API gateway translates the same user profile data a hundred times per second. Each translation hits a database, burns CPU cycles, and adds latency. There's a smarter approach. In this article, let's explore Caching in .NET - the performance optimization technique that stores computed results in fast, temporary storage so repeated operations become instant lookups instead of expensive recalculations.

Caching is the practice of storing the results of expensive operations in a dedicated, high-speed storage layer for quick retrieval on subsequent requests. Think of it as keeping your most-used tools within arm's reach instead of walking to the garage every time you need them. This simple concept powers everything from your browser to massive distributed systems.

We'll explore what makes caching effective, the trade-offs involved, and the fundamental principles that guide when and how to cache data. We'll also cover memoization, a specialized form of caching for pure functions, and understand why caching isn't just an optimization - it's a fundamental building block of modern computing.

Why we gonna do?

The Performance Problem

Every data access comes with a cost. When your application needs product information, it might query a database, call a remote API, perform complex calculations, or decrypt sensitive data. Each operation burns CPU cycles, consumes memory, and adds milliseconds (or seconds) to response times.

Consider an API aggregation scenario: your BFF (Backend for Frontend) combines data from three microservices to build a user dashboard. Without caching, every dashboard request triggers three separate API calls:

// Without caching - multiple API calls on every request
public async Task<UserDashboard> GetUserDashboardAsync(string userId)
{
    // Three separate HTTP calls
    var profileTask = _httpClient.GetFromJsonAsync<UserProfile>(
        $"https://profile-api/users/{userId}");
    var metricsTask = _httpClient.GetFromJsonAsync<UserMetrics>(
        $"https://metrics-api/users/{userId}/summary");
    var activityTask = _httpClient.GetFromJsonAsync<Activity[]>(
        $"https://activity-api/users/{userId}/recent");
    
    await Task.WhenAll(profileTask, metricsTask, activityTask);
    
    return new UserDashboard 
    {
        Profile = await profileTask,
        Metrics = await metricsTask,
        RecentActivity = await activityTask
    };
    // Total latency: 300-800ms (network + processing)
}

The problem multiplies with concurrent users. If 500 users refresh their dashboard within the same minute, your system makes 1,500 external API calls for data that rarely changes. That's wasted network bandwidth, unnecessary load on downstream services, and slow dashboard loading times.

The Speed Advantage

Different storage systems have dramatically different performance characteristics. Understanding these differences explains why caching works:

.NET Cache Performance Benchmarks:
┌─────────────────────────────┬───────────────┬─────────────────┐
│ Operation                   │ Median Time   │ Ops/sec         │
├─────────────────────────────┼───────────────┼─────────────────┤
│ IMemoryCache.TryGetValue    │ 15 ns         │ ~66M            │
│ ConcurrentDictionary lookup │ 45 ns         │ ~22M            │
│ Redis GET (localhost)       │ 0.3-0.8 ms    │ ~3,000          │
│ SQL Server SELECT by PK     │ 2-15 ms       │ ~200            │
│ EF Core query (simple)      │ 8-35 ms       │ ~60             │
│ HTTP API call (same DC)     │ 20-150 ms     │ ~15             │
│ HTTP API call (external)    │ 200-2000 ms   │ ~2              │
└─────────────────────────────┴───────────────┴─────────────────┘

Real-World Impact:
• IMemoryCache: 4,000x faster than local SQL Server
• Redis: 25-50x faster than SQL Server  
• Both: Enable sub-millisecond response times at scale

Here's the thing: even if you don't use caching, you're still dealing with copies of data. When you query a database, the engine serializes data, sends it over the network, and your application deserializes it into objects. That's copying. The difference? Caching gives you control over where and how long you keep those copies.

Why Caching Matters Everywhere

Caching isn't just for databases. It's a fundamental pattern in computer science that appears at every layer:

• CPU Caching: L1, L2, L3 caches store frequently accessed memory
• Browser Caching: Saves images, scripts, and stylesheets locally
• CDN Caching: Distributes content geographically for faster delivery
• DNS Caching: Remembers IP address lookups
• Application Caching: Stores computed results and database queries

The web would be unbearably slow without caching. Loading a typical webpage involves hundreds of requests - images, fonts, scripts, stylesheets. Without browser and CDN caching, each page load would be 20 to 100 times slower. That's not an exaggeration; that's the measured difference between cached and non-cached web browsing.

The benefits of caching extend beyond speed:

• Reduced Database Load: Fewer queries mean better database performance
• Lower Infrastructure Costs: Less CPU, memory, and network usage
• Improved Scalability: Handle more users with same resources
• Better User Experience: Faster responses keep users engaged
• Resilience: Serve cached data even if backend is temporarily unavailable

How we gonna do?

The Fundamental Trade-Off

Caching isn't free. It trades memory for speed and accepts eventual consistency instead of real-time accuracy. Understanding this trade-off is crucial for effective caching.

// The caching decision tree
Caching Benefits:
✓ Faster response times
✓ Reduced computational cost  
✓ Lower database/API load
✓ Better scalability

Caching Costs:
✗ Memory consumption
✗ Data staleness (cached data may be outdated)
✗ Cache invalidation complexity
✗ Additional infrastructure (for distributed caching)

When to Cache

Caching works best when data is read frequently and changed infrequently. The return on investment comes from serving the same data multiple times from cache.

// Real API traffic patterns analysis
User Profile API (Authentication Service):
• 2.4M requests/hour during peak
• 85K unique user IDs accessed
• Average: 28 requests per user profile
• Hit Rate with 15min TTL: 96.4%
• Savings: 2.3M database queries/hour

Configuration API (App Settings):
• 450K requests/hour
• 12 unique config keys accessed
• Average: 37,500 requests per config
• Hit Rate with 5min TTL: 99.97%
• Savings: Nearly all database queries eliminated

Real-Time Inventory API (High Churn):
• 680K requests/hour
• 520K unique SKUs accessed  
• Average: 1.3 requests per SKU
• Hit Rate with 10sec TTL: 23%
• Result: Poor ROI - cache not recommended

The pattern is clear: cache effectiveness depends on request skew, not total volume. Configuration and authentication endpoints show extreme skew (few unique items, many requests) making them ideal for caching. Real-time inventory has low skew (many unique items, few repeats) making caching less valuable despite high traffic.

What to Cache

Not all data deserves caching. Focus on data that's expensive to compute or fetch and accessed frequently.

// .NET API caching candidates
✓ JWT token claims (validated on every authorized request)
✓ User permissions/roles (read-heavy, change infrequently)
✓ API rate limit counters (high-frequency reads/writes)
✓ Geolocation lookups (IP to country/city mapping)
✓ Exchange rate conversions (updated hourly)
✓ Computed dashboard aggregations (expensive LINQ queries)
✓ Third-party API responses (weather, maps, stock quotes)
✓ Compiled Razor views (OutputCache in .NET 7+)

// Poor caching candidates  
✗ Real-time WebSocket messages (ephemeral data)
✗ User shopping cart state (frequently modified)
✗ Live sports scores (constant updates)
✗ Password reset tokens (one-time use, security sensitive)
✗ Audit log entries (write-once, read-rarely)

You can cache at different granularities. Cache entire objects, specific properties, or computed results. You can even cache different instances of the same type differently:

// Adaptive caching strategy for APIs
public class ApiCachingPolicy
{
    public TimeSpan GetCacheDuration(string endpoint, HttpContext context)
    {
        // Authentication endpoints: short TTL for security
        if (endpoint.StartsWith("/api/auth"))
            return TimeSpan.FromMinutes(5);
        
        // User-specific data: medium TTL
        if (context.User.Identity?.IsAuthenticated == true)
            return TimeSpan.FromMinutes(15);
        
        // Public reference data: long TTL
        if (endpoint.StartsWith("/api/reference"))
            return TimeSpan.FromHours(4);
        
        // Real-time data: very short TTL
        if (endpoint.Contains("live") || endpoint.Contains("realtime"))
            return TimeSpan.FromSeconds(10);
        
        // Default: moderate caching
        return TimeSpan.FromMinutes(30);
    }
}

How to Cache Effectively

Effective caching requires careful consideration of cache duration, invalidation strategy, and cache key design.

.NET IMemoryCache with Eviction Callbacks

public class UserProfileCache
{
    private readonly IMemoryCache _cache;
    private readonly ILogger<UserProfileCache> _logger;
    
    public async Task<UserProfile> GetOrCreateAsync(string userId)
    {
        return await _cache.GetOrCreateAsync($"profile:{userId}", async entry =>
        {
            // Configure cache entry options
            entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(15);
            entry.SlidingExpiration = TimeSpan.FromMinutes(5);
            
            // Set priority for eviction under memory pressure
            entry.Priority = CacheItemPriority.Normal;
            
            // Register eviction callback for logging/metrics
            entry.RegisterPostEvictionCallback((key, value, reason, state) =>
            {
                _logger.LogInformation(
                    "Cache entry {Key} evicted. Reason: {Reason}",
                    key, reason);
                    
                // Track cache eviction metrics
                if (reason == EvictionReason.Capacity)
                {
                    // Memory pressure detected - consider alerting
                }
            });
            
            // Fetch from database on cache miss
            return await FetchUserProfileFromDatabaseAsync(userId);
        });
    }
}

Absolute expiration sets a hard deadline - the cache entry will be removed after that time regardless of access. Sliding expiration resets the timer on each access, keeping frequently used items in cache longer. Priority influences eviction order under memory pressure (Low, Normal, High, NeverRemove).

.NET ResponseCaching Middleware

// Program.cs configuration
builder.Services.AddResponseCaching(options =>
{
    options.MaximumBodySize = 64 * 1024 * 1024; // 64 MB
    options.UseCaseSensitivePaths = false;
});

var app = builder.Build();
app.UseResponseCaching();

// Controller usage
[ApiController]
[Route("api/[controller]")]
public class CatalogController : ControllerBase
{
    [HttpGet("categories")]
    [ResponseCache(Duration = 300, Location = ResponseCacheLocation.Any, 
                   VaryByHeader = "Accept-Language")]
    public async Task<IActionResult> GetCategoriesAsync()
    {
        var categories = await _catalogService.GetCategoriesAsync();
        return Ok(categories);
        // Cached for 5 minutes, browser + proxy caching enabled
    }
    
    [HttpGet("search")]
    [ResponseCache(Duration = 60, VaryByQueryKeys = new[] { "q", "page" })]
    public async Task<IActionResult> SearchAsync([FromQuery] string q, [FromQuery] int page = 1)
    {
        var results = await _searchService.SearchAsync(q, page);
        return Ok(results);
        // Cached by query parameters: different cache for "q=shoes" vs "q=boots"
    }
}

OutputCache Policies (.NET 10)

// Program.cs - OutputCache configuration
builder.Services.AddOutputCache(options =>
{
    // Base policy for all endpoints
    options.AddBasePolicy(builder => builder.Expire(TimeSpan.FromSeconds(30)));
    
    // Named policy for API endpoints
    options.AddPolicy("ApiEndpoints", builder => 
        builder.Expire(TimeSpan.FromMinutes(5))
               .SetVaryByQuery("page", "pageSize")
               .Tag("api-cache"));
    
    // Policy with custom cache key
    options.AddPolicy("UserSpecific", builder =>
        builder.Expire(TimeSpan.FromMinutes(10))
               .SetVaryByHeader("Authorization")
               .Cache());
});

var app = builder.Build();
app.UseOutputCache();

// Minimal API usage
app.MapGet("/api/weather/{city}", async (string city, WeatherService service) =>
{
    return await service.GetWeatherAsync(city);
})
.CacheOutput(builder => builder.Expire(TimeSpan.FromMinutes(15)).Tag("weather"));

// Programmatic cache eviction by tag
app.MapPost("/api/admin/clear-cache", async (IOutputCacheStore cache) =>
{
    await cache.EvictByTagAsync("api-cache", default);
    return Results.Ok("Cache cleared");
});

Distributed Caching with Redis

// Program.cs - Redis distributed cache
builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = builder.Configuration.GetConnectionString("Redis");
    options.InstanceName = "MyApp:";
    
    // Connection pool configuration
    options.ConfigurationOptions = new ConfigurationOptions
    {
        EndPoints = { "localhost:6379" },
        AbortOnConnectFail = false,
        ConnectTimeout = 5000,
        SyncTimeout = 5000,
        AsyncTimeout = 5000
    };
});

// Service usage
public class DistributedCacheService
{
    private readonly IDistributedCache _cache;
    
    public async Task<T?> GetOrSetAsync<T>(string key, 
        Func<Task<T>> factory, TimeSpan expiration)
    {
        // Try to get from Redis
        var cachedData = await _cache.GetStringAsync(key);
        if (cachedData != null)
        {
            return JsonSerializer.Deserialize<T>(cachedData);
        }
        
        // Cache miss - fetch and store
        var data = await factory();
        var serialized = JsonSerializer.Serialize(data);
        
        var options = new DistributedCacheEntryOptions
        {
            AbsoluteExpirationRelativeToNow = expiration
        };
        
        await _cache.SetStringAsync(key, serialized, options);
        return data;
    }
}

// Azure Cache for Redis with geo-replication
builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = 
        "mycache.redis.cache.windows.net:6380,password=***,ssl=True,abortConnect=False";
    options.InstanceName = "production:";
});

Cache Invalidation

There are only two hard problems in computer science: cache invalidation and naming things. When data changes, you need to update or remove the cached version to prevent serving stale data.

// Cache invalidation strategies
public class ProductService
{
    // Strategy 1: Time-based expiration (passive)
    // Cache entry expires automatically after set duration
    // Pro: Simple, no manual intervention
    // Con: May serve stale data until expiration
    
    // Strategy 2: Explicit invalidation (active)
    public async Task UpdateProductAsync(Product product)
    {
        await _repository.UpdateAsync(product);
        
        // Remove from cache after update
        string cacheKey = $"product:{product.Id}";
        _cache.Remove(cacheKey);
        // Next request will fetch fresh data
    }
    
    // Strategy 3: Cache-aside with update
    public async Task UpdateProductAsync(Product product)
    {
        await _repository.UpdateAsync(product);
        
        // Immediately update cache with new data
        string cacheKey = $"product:{product.Id}";
        _cache.Set(cacheKey, product, _cacheOptions);
        // No cache miss on next request
    }
}

Understanding Memoization

Memoization is a specialized caching technique for pure functions. A pure function always returns the same output for the same input with no side effects - perfect for caching.

// Pure function - LINQ expression compilation
public class ExpressionCompiler
{
    // Expensive: compiling expression trees
    public Func<User, bool> CompilePredicate(string filterExpression)
    {
        var parameter = Expression.Parameter(typeof(User), "user");
        var body = ParseAndBuildExpression(filterExpression, parameter);
        var lambda = Expression.Lambda<Func<User, bool>>(body, parameter);
        return lambda.Compile(); // Compilation is expensive
        // Same input = same compiled output (pure function)
    }
}

// Memoized version with ConcurrentDictionary
public class MemoizedExpressionCompiler
{
    private readonly ConcurrentDictionary<string, Func<User, bool>> 
        _compiledCache = new();
    
    public Func<User, bool> CompilePredicate(string filterExpression)
    {
        return _compiledCache.GetOrAdd(filterExpression, expr =>
        {
            var parameter = Expression.Parameter(typeof(User), "user");
            var body = ParseAndBuildExpression(expr, parameter);
            var lambda = Expression.Lambda<Func<User, bool>>(body, parameter);
            return lambda.Compile();
        });
    }
}

// Benchmark results (BenchmarkDotNet):
// Without memoization: 245 μs per compilation
// With memoization (hit): 18 ns (13,600x faster)
// Memory cost: ~2KB per cached expression

Referential transparency is the property that makes memoization safe. It means calling a function with certain inputs is equivalent to using its return value directly. You can substitute one for the other without changing program behavior.

// Referentially transparent (memoizable)
public decimal CalculateTotal(decimal price, decimal taxRate)
    => price * (1 + taxRate);
    // Pure calculation, same inputs = same output

// NOT referentially transparent (not memoizable)  
public decimal CalculateTotalWithTimestamp(decimal price, decimal taxRate)
    => price * (1 + taxRate) * DateTime.Now.Ticks;
    // Depends on current time - output changes even with same inputs

// NOT referentially transparent (not memoizable)
public async Task<Product> GetProductWithInventory(int productId)
{
    var product = await _db.Products.FindAsync(productId);
    product.CurrentStock = await GetRealTimeInventory(productId);
    return product;
    // Has side effect (database access)
    // Output changes based on external state
}

Cache Layers and Architecture

Real-world applications often use multiple cache layers, each optimized for different scenarios:

Multi-Layer Caching Architecture:

┌─────────────────────────────────────────────┐
│          Client Request                     │
└─────────────────┬───────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────┐
│     L1: In-Memory Cache (IMemoryCache)      │
│     • Fastest (nanoseconds)                 │
│     • Process-local                         │
│     • Lost on restart                       │
└─────────────────┬───────────────────────────┘
                  │ Miss
                  ▼
┌─────────────────────────────────────────────┐
│   L2: Distributed Cache (Redis, etc.)       │
│     • Fast (milliseconds)                   │
│     • Shared across instances               │
│     • Survives restarts                     │
└─────────────────┬───────────────────────────┘
                  │ Miss
                  ▼
┌─────────────────────────────────────────────┐
│          L3: Database / API                 │
│     • Slowest (hundreds of milliseconds)    │
│     • Source of truth                       │
│     • Authoritative data                    │
└─────────────────────────────────────────────┘

Monitoring and Performance Metrics

Production caching requires observability. Track these key metrics to measure effectiveness and detect issues:

public class CacheMetrics
{
    private long _hits;
    private long _misses;
    private long _evictions;
    
    public void RecordHit() => Interlocked.Increment(ref _hits);
    public void RecordMiss() => Interlocked.Increment(ref _misses);
    public void RecordEviction() => Interlocked.Increment(ref _evictions);
    
    public CacheStatistics GetStatistics()
    {
        var totalRequests = _hits + _misses;
        var hitRate = totalRequests > 0 
            ? (_hits / (double)totalRequests) * 100 
            : 0;
            
        return new CacheStatistics
        {
            Hits = _hits,
            Misses = _misses,
            Evictions = _evictions,
            HitRate = hitRate,
            TotalRequests = totalRequests
        };
    }
}

// Key metrics to monitor:
// • Hit Rate: Target >80% for effective caching
// • Miss Rate: High misses indicate poor cache key design or TTL
// • Eviction Rate: High evictions suggest memory pressure
// • Average Latency: Cache hits should be <1ms
// • Memory Usage: Track cache size growth
// • Staleness: Time between updates and cache expiration

In production systems, hit rates vary from 60% (poor) to 99.9% (excellent) depending on access patterns. A 95%+ hit rate typically indicates well-designed caching. Below 70% suggests the cached data has too much uniqueness or TTLs are too short.

Common Pitfalls to Avoid

Caching can backfire if done incorrectly. Here are critical mistakes to avoid:

// ❌ WRONG: Caching mutable objects
public class OrderService
{
    public Order GetOrder(int orderId)
    {
        if (_cache.TryGetValue(orderId, out Order order))
            return order; // Dangerous! Caller can modify cached object
    }
}

// ✓ CORRECT: Return defensive copy or use immutable types
public Order GetOrder(int orderId)
{
    if (_cache.TryGetValue(orderId, out Order order))
        return order.Clone(); // Return copy, protect cache
}

// ❌ WRONG: Unbounded cache growth
public void CacheUserSession(string sessionId, UserSession session)
{
    _cache.Set(sessionId, session); // No expiration!
    // Memory leak - cache grows forever
}

// ✓ CORRECT: Always set expiration
public void CacheUserSession(string sessionId, UserSession session)
{
    var options = new MemoryCacheEntryOptions
    {
        AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(1)
    };
    _cache.Set(sessionId, session, options);
}

// ❌ WRONG: Cache stampede - thundering herd problem
// 1000 concurrent requests miss cache
// All 1000 query database simultaneously
// Database overwhelmed, performance degrades

// ✓ CORRECT: Use locking or semaphore
private readonly SemaphoreSlim _cacheLock = new(1, 1);

public async Task<Product> GetProductAsync(int id)
{
    if (_cache.TryGetValue(id, out Product product))
        return product;
        
    await _cacheLock.WaitAsync();
    try
    {
        // Double-check after acquiring lock
        if (_cache.TryGetValue(id, out product))
            return product;
            
        // Only one thread fetches from database
        product = await _repository.GetByIdAsync(id);
        _cache.Set(id, product, _cacheOptions);
        return product;
    }
    finally
    {
        _cacheLock.Release();
    }
}

Monitoring and Metrics

You can't improve what you don't measure. Track these metrics to understand cache effectiveness:

// Key cache metrics
Cache Hit Rate = (Cache Hits / Total Requests) x 100%
// Target: >80% for read-heavy workloads

Cache Miss Rate = (Cache Misses / Total Requests) x 100%  
// Lower is better

Eviction Rate = (Items Evicted / Items Added) x 100%
// High rate may indicate cache size too small

Average Response Time (Cache Hit) vs (Cache Miss)
// Should show significant difference

// Example: Logging cache metrics
public class CachedProductService
{
    private long _cacheHits;
    private long _cacheMisses;
    
    public async Task<Product> GetProductAsync(int id)
    {
        if (_cache.TryGetValue(id, out Product product))
        {
            Interlocked.Increment(ref _cacheHits);
            return product;
        }
        
        Interlocked.Increment(ref _cacheMisses);
        product = await _repository.GetByIdAsync(id);
        _cache.Set(id, product, _options);
        return product;
    }
    
    public double GetHitRate() 
        => (double)_cacheHits / (_cacheHits + _cacheMisses) * 100;
}

Summary

Key Takeaways

• Caching trades memory for speed by storing frequently accessed data in fast storage layers. This can improve response times by 10x to 10,000x depending on the data source.
• Cache when data is read frequently and changed infrequently. The benefit comes from serving the same data multiple times from cache instead of repeatedly fetching it.
• Not all data should be cached. Focus on expensive operations, stable data, and high-traffic scenarios. Avoid caching rapidly changing or rarely accessed data.
• Cache invalidation is crucial. Use time-based expiration for simplicity or explicit invalidation for accuracy. Consider the staleness tolerance of your application.
• Memoization is caching for pure functions. Functions that are referentially transparent (same input always produces same output with no side effects) are perfect candidates.
• Multi-layer caching maximizes performance. Combine in-memory caching for speed with distributed caching for scale and persistence.
• Monitor cache effectiveness. Track hit rates, miss rates, and response times to validate your caching strategy and identify opportunities for improvement.
• Avoid common pitfalls like caching mutable objects, unbounded cache growth, and cache stampede. Always set expiration policies and protect against thundering herd problems.

What's Next?

Caching is fundamental to building high-performance .NET applications. Start with simple in-memory caching using IMemoryCache for frequently accessed data. Monitor the results and expand to distributed caching with Redis or SQL Server Cache when you need to scale across multiple instances. Remember: caching is a tool, not a silver bullet. Use it strategically where it provides the most value.