33. Compression & Performance

Optimizing API response sizes with compression, performance profiling, and database query optimization

About this chapter

Optimize API performance by enabling compression to reduce response sizes, profiling slow endpoints, and optimizing database queries.

  • Compression fundamentals: Gzip and Brotli trade-offs and benefits
  • ASP.NET Core compression: Response compression middleware configuration
  • Performance profiling: Identifying slow endpoints and bottlenecks
  • Database query optimization: N+1 problems, eager loading, and indexing
  • Measuring performance: Benchmarking and monitoring response times
  • Optimization priorities: Where to focus effort for maximum impact

Learning outcomes:

  • Enable and configure response compression in ASP.NET Core
  • Understand Gzip and Brotli compression options
  • Profile API performance to find bottlenecks
  • Optimize database queries and reduce N+1 problems
  • Use eager loading to prevent multiple round trips
  • Monitor and benchmark API performance

33.1 Why Compression Matters

API responses can be large. A list of 1,000 commands as JSON might be 500 KB. Over a slow network, that takes time.

Compression trades CPU for bandwidth:

Without compression:
500 KB response → 2 seconds to download (on slow 3G)

With Gzip compression:
500 KB → 50 KB (90% smaller) → 200ms to download

Who benefits?

  • Mobile clients (bandwidth-constrained)
  • Slow networks (satellite, rural, old infrastructure)
  • Large APIs returning lots of data
  • Paginated endpoints users request repeatedly

Who doesn’t need it?

  • Tiny responses (< 1 KB)—compression overhead makes it bigger
  • Already-compressed formats (images, videos, PDFs)
  • Real-time streaming (adds latency)

33.2 Gzip vs Brotli Compression

Gzip (1992)

  • Universal browser support
  • Fast compression/decompression
  • Good compression ratio (~70%)
  • Lower CPU cost
  • Industry standard

Brotli (2013, from Google)

  • Better compression ratio (~80%)
  • Slower compression (but decompression is fast)
  • Modern browser support (not IE 11)
  • Higher CPU cost

In practice:

  • Use Gzip by default—it’s fast and universally supported
  • Use Brotli if you have CPU headroom and modern clients
  • Use both—let clients choose via Accept-Encoding header

33.3 Enable Response Compression

ASP.NET Core includes built-in response compression middleware:

// Program.cs
using Microsoft.AspNetCore.ResponseCompression;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllers();

// Add response compression
builder.Services.AddResponseCompression(options =>
{
    // Which MIME types to compress
    options.MimeTypes = ResponseCompressionDefaults.MimeTypes.Concat(
        new[] { "application/json", "text/json", "text/plain" });

    // Which compression providers to use (order matters)
    options.Providers.Add<BrotliCompressionProvider>();
    options.Providers.Add<GzipCompressionProvider>();

    // Request threshold: only compress responses larger than this
    options.MinResponseDataLength = 1024;  // 1 KB minimum
});

// Configure Gzip provider
builder.Services.Configure<GzipCompressionProviderOptions>(options =>
{
    options.Level = System.IO.Compression.CompressionLevel.Fastest;
});

// Configure Brotli provider
builder.Services.Configure<BrotliCompressionProviderOptions>(options =>
{
    options.Level = System.IO.Compression.CompressionLevel.Optimal;
});

var app = builder.Build();

// Use response compression EARLY in pipeline
app.UseResponseCompression();

app.UseHttpsRedirection();
app.UseAuthorization();

app.MapControllers();

app.Run();

Result: Browser sends:

Accept-Encoding: gzip, deflate, br

API responds:

Content-Encoding: gzip
Content-Length: 45000  (was 500000)

Browser automatically decompresses. Transparent to your code.

33.4 Compression Strategies by Content Type

JSON APIs (compress):

// Default: JSON is compressed
// Gzip reduces 500KB → 50KB

HTML/Text (compress):

// Include text/html, text/plain in MimeTypes
options.MimeTypes = ResponseCompressionDefaults.MimeTypes.Concat(
    new[] { "text/html", "text/plain", "text/css" });

Images/Videos (don’t compress):

// Images are already compressed
// JPEG, PNG, MP4 waste CPU trying to compress further
// Exclude from compression

Already-compressed (don’t compress):

// .gz, .br, .zip files are already compressed
// Compressing them again wastes CPU with minimal benefit

33.5 Selective Compression with Attributes

Sometimes you want to disable compression for specific endpoints:

[ApiController]
[Route("api/[controller]")]
public class CommandsController : ControllerBase
{
    [HttpGet]
    public async Task<ActionResult> GetCommands()
    {
        // Compressed (small data, benefits from compression)
        var commands = await _repository.GetCommandsAsync();
        return Ok(commands);
    }

    [HttpGet("export")]
    [ResponseCache(NoStore = true)]  // Don't cache exports
    public async Task<FileResult> ExportCommands()
    {
        // This is already compressed (ZIP file)
        var zipData = await GenerateZipAsync();
        return File(zipData, "application/zip", "commands.zip");
    }
}

33.6 Performance Profiling

You can’t optimize what you don’t measure. Profile your API:

1. Measure response size:

[HttpGet]
public async Task<ActionResult> GetCommands()
{
    var commands = await _repository.GetCommandsAsync();
    
    var json = JsonSerializer.Serialize(commands);
    _logger.LogInformation("GetCommands response size: {Size} bytes", json.Length);
    
    return Ok(commands);
}

Output:

GetCommands response size: 489250 bytes
// After compression: ~49000 bytes (90% reduction)

2. Measure response time:

[HttpGet]
public async Task<ActionResult> GetCommands()
{
    var stopwatch = Stopwatch.StartNew();
    
    var commands = await _repository.GetCommandsAsync();
    
    stopwatch.Stop();
    _logger.LogInformation("GetCommands took {Elapsed}ms", stopwatch.ElapsedMilliseconds);
    
    return Ok(commands);
}

3. Use browser DevTools:

  • Open DevTools → Network tab
  • Look at “Size” vs “Transferred Size”
  • If they’re different, compression is working
  • Example: Size 500 KB, Transferred 50 KB → 90% reduction

4. Use benchmarking tools:

# Test with ApacheBench
ab -n 100 -c 10 https://api.example.com/api/commands

# Test with wrk
wrk -t4 -c100 -d30s https://api.example.com/api/commands

33.7 Database Query Optimization

Compression helps, but optimizing queries helps more.

Problem: N+1 Query Pattern

// ❌ WRONG: N+1 queries
var platforms = await _context.Platforms.ToListAsync();  // 1 query

foreach (var platform in platforms)
{
    var commands = await _context.Commands
        .Where(c => c.PlatformId == platform.Id)
        .ToListAsync();  // N more queries (1 per platform)
    
    platform.Commands = commands;
}
// Total: 1 + N queries (if 100 platforms → 101 queries!)

Solution: Use Include (eager loading)

// ✓ RIGHT: 1 query with eager loading
var platforms = await _context.Platforms
    .Include(p => p.Commands)  // Loads all commands in single query
    .ToListAsync();

// Single query with JOIN, much faster

Problem: Loading unnecessary columns

// ❌ WRONG: Load entire entity
var commands = await _context.Commands.ToListAsync();
// Returns: Id, PlatformId, HowTo, CommandLine, Description, LongDescription, ...

// Response: 500 KB

Solution: Use projections

// ✓ RIGHT: Load only needed columns
var commands = await _context.Commands
    .Select(c => new CommandDto
    {
        Id = c.Id,
        HowTo = c.HowTo,
        CommandLine = c.CommandLine
        // Skip Description, LongDescription, etc.
    })
    .ToListAsync();

// Response: 50 KB (after compression)

Problem: Fetching large datasets

// ❌ WRONG: Load all 100,000 commands
var allCommands = await _context.Commands.ToListAsync();
// 500 MB response, OutOfMemory exception

Solution: Use pagination

// ✓ RIGHT: Load one page at a time
var pageSize = 50;
var commands = await _context.Commands
    .Skip((pageIndex - 1) * pageSize)
    .Take(pageSize)
    .ToListAsync();

// Response: 2.5 MB per page

Optimization checklist:

  • ✓ Use Include() instead of lazy loading
  • ✓ Use projections to select only needed columns
  • ✓ Use pagination for large datasets
  • ✓ Add database indexes on frequently filtered columns
  • ✓ Use AsNoTracking() for read-only queries
  • ✓ Avoid ToList() before filters—let database filter first

33.8 Caching Strategies (Review)

From Part 9, you learned caching reduces database load:

// Cache frequently accessed data
public async Task<List<Platform>> GetPlatformsAsync()
{
    var cacheKey = "all_platforms";
    
    // Try cache first
    var cached = await _cache.GetAsync(cacheKey);
    if (cached != null)
        return JsonConvert.DeserializeObject<List<Platform>>(cached);
    
    // Cache miss—fetch from database
    var platforms = await _context.Platforms.ToListAsync();
    
    // Store in cache for 1 hour
    await _cache.SetAsync(cacheKey, 
        JsonConvert.SerializeObject(platforms),
        TimeSpan.FromHours(1));
    
    return platforms;
}

Caching + Compression = Fast APIs

  • Cache eliminates database queries
  • Compression reduces network transfer
  • Combined: 100x faster than unoptimized

33.9 HTTP Caching Headers

Let browsers and CDNs cache your responses:

[HttpGet]
[ResponseCache(Duration = 3600)]  // Cache for 1 hour
public async Task<ActionResult> GetPlatforms()
{
    var platforms = await _repository.GetPlatformsAsync();
    return Ok(platforms);
}

Equivalent to:

Cache-Control: public, max-age=3600

Browser caches the response. Next request within 1 hour doesn’t hit your API.

Cache with ETags (for updates):

[HttpGet("{id}")]
public async Task<ActionResult> GetPlatform(int id)
{
    var platform = await _repository.GetPlatformByIdAsync(id);
    
    // Generate ETag (hash of content)
    var json = JsonSerializer.Serialize(platform);
    var etag = $"\"{Guid.NewGuid().ToString()}\"";
    
    Response.Headers.Add("ETag", etag);
    Response.Headers.Add("Cache-Control", "public, max-age=3600");
    
    return Ok(platform);
}

Browser sends:

If-None-Match: "abc123"

API responds with 304 Not Modified if unchanged (no response body).

33.10 Measuring Compression Impact

Before optimization:

GET /api/commands
Content-Type: application/json
Content-Length: 489250
Transfer-Encoding: chunked

Response time: 2.5 seconds

After optimization (compression + pagination + caching):

GET /api/commands?pageIndex=1&pageSize=50
Content-Type: application/json
Content-Encoding: gzip
Content-Length: 4821  (was 489250)
Cache-Control: public, max-age=3600

Response time: 45ms (first request), 0ms (cached requests)

Improvements:

  • Response size: 489 KB → 5 KB (98% reduction)
  • Response time: 2.5s → 45ms (55x faster)
  • Cached requests: 0ms (infinite speedup)

33.11 ASP.NET Core Performance Best Practices

1. Use minimal APIs when appropriate (simpler routing = faster)

// Slightly faster than controllers
app.MapGet("/api/commands", async (ICommandRepository repo) =>
{
    var commands = await repo.GetCommandsAsync();
    return Results.Ok(commands);
});

2. Use async/await consistently

// ✓ RIGHT
public async Task<ActionResult> GetCommands()
{
    var commands = await _repository.GetCommandsAsync();
    return Ok(commands);
}

// ❌ WRONG (blocks thread)
public ActionResult GetCommands()
{
    var commands = _repository.GetCommandsAsync().Result;  // Blocks!
    return Ok(commands);
}

3. Disable unnecessary middleware

// Only add what you need
var app = builder.Build();

// Skip unused middleware:
// - app.UseSession() (if not using sessions)
// - app.UseStaticFiles() (if no static files)
// - app.UseDeveloperExceptionPage() (not in production)

app.UseRouting();
app.UseAuthorization();
app.MapControllers();

4. Use connection pooling

// EF Core already pools connections
var optionsBuilder = new DbContextOptionsBuilder<AppDbContext>()
    .UseNpgsql(connectionString, options =>
    {
        options.MaxPoolSize = 128);  // Default is 25
    });

5. Use Read-Only Queries

// No change tracking overhead
var commands = await _context.Commands
    .AsNoTracking()  // Tells EF Core not to track changes
    .ToListAsync();

// Faster for read-only endpoints

33.12 Load Testing Your API

Once optimized, ensure it performs under load:

# Using ApacheBench: 1000 requests, 100 concurrent
ab -n 1000 -c 100 https://api.example.com/api/commands

# Using wrk: 4 threads, 100 connections, 30 seconds
wrk -t4 -c100 -d30s https://api.example.com/api/commands

# Output tells you:
# - Requests per second
# - Response times (min/mean/max)
# - Failed requests

If performance degrades under load:

  1. Check database connections (connection pooling size)
  2. Check memory usage (memory leak?)
  3. Check CPU usage (slow queries?)
  4. Add more app instances behind load balancer
  5. Consider caching hot endpoints

33.13 Profiling Tools

Entity Framework Profiler:

// Log all database queries
builder.Services.AddLogging(config =>
{
    config.AddConsole();
    config.AddDebug();
});

// EF Core logs slow queries
optionsBuilder.LogTo(Console.WriteLine);

Application Insights (production monitoring):

builder.Services.AddApplicationInsights();

// Tracks:
// - Response times
// - Failed requests
// - Database query times
// - Exceptions
// - Dependencies (external APIs)

Flamegraph (CPU profiling):

# Capture CPU profile
dotnet-trace collect --process-id 12345

# Analyze where time is spent
perfview /gui

33.14 What’s Next

You now have:

  • ✓ Response compression (Gzip + Brotli)
  • ✓ Query optimization (Include, projections, pagination)
  • ✓ Caching strategies (output cache, HTTP cache headers)
  • ✓ Database optimization (connection pooling, AsNoTracking)
  • ✓ Load testing and monitoring
  • ✓ Performance profiling tools

Next: Rate Limiting—protect your API from abuse and resource exhaustion.