47. File Operations

File handling in APIs: multipart uploads, validation, streaming, and secure downloads

About this chapter

Handle file uploads and downloads safely by validating file types, streaming large files to avoid memory issues, and securing access to file resources.

File upload endpoints: Accepting multipart file uploads from clients
File validation: Checking file types, sizes, and preventing malicious uploads
Storage strategies: Where and how to store uploaded files securely
Streaming downloads: Returning large files efficiently without loading entirely in memory
Content-type headers: Proper MIME types and content disposition
Security considerations: Directory traversal prevention and access control

Learning outcomes:

Create file upload endpoints with IFormFile
Validate file types and sizes
Store uploads securely on disk or cloud storage
Stream file downloads efficiently
Return files with correct content-type headers
Prevent security vulnerabilities in file operations

47.1 Why File Operations Matter

Common scenarios:

Users upload CSV files to import data
Users upload images for profiles
API generates reports and returns PDF files
Admin uploads configuration files
Users download generated exports

Challenges:

Size: Files can be huge (1GB+ downloads)
Security: Validate file types, prevent directory traversal
Performance: Stream files to avoid loading entire file in memory
Headers: Correct content-type and disposition
Resume: Support pausing and resuming large uploads

47.2 File Upload Endpoints

Basic file upload:

// Controllers/FileController.cs
[ApiController]
[Route("api/[controller]")]
public class FilesController : ControllerBase
{
    private readonly ILogger<FilesController> _logger;
    private const string UploadDirectory = "/app/uploads";
    private const long MaxFileSize = 50 * 1024 * 1024;  // 50 MB

    public FilesController(ILogger<FilesController> logger)
    {
        _logger = logger;
    }

    [HttpPost("upload")]
    public async Task<IActionResult> UploadFile(IFormFile file)
    {
        // Validate file exists
        if (file == null || file.Length == 0)
        {
            return BadRequest("File is required");
        }

        // Validate file size
        if (file.Length > MaxFileSize)
        {
            return BadRequest($"File size exceeds {MaxFileSize / 1024 / 1024} MB limit");
        }

        // Validate file extension
        var allowedExtensions = new[] { ".csv", ".xlsx", ".json" };
        var extension = Path.GetExtension(file.FileName).ToLowerInvariant();
        
        if (!allowedExtensions.Contains(extension))
        {
            return BadRequest($"File type not allowed. Allowed: {string.Join(", ", allowedExtensions)}");
        }

        // Validate MIME type
        var allowedMimeTypes = new[] { "text/csv", "application/json", "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" };
        
        if (!allowedMimeTypes.Contains(file.ContentType))
        {
            return BadRequest("Invalid file content type");
        }

        // Generate unique filename
        var filename = $"{Guid.NewGuid()}{extension}";
        var filepath = Path.Combine(UploadDirectory, filename);

        try
        {
            // Create directory if doesn't exist
            Directory.CreateDirectory(UploadDirectory);

            // Save file
            using (var stream = System.IO.File.Create(filepath))
            {
                await file.CopyToAsync(stream);
            }

            _logger.LogInformation("File uploaded: {Filename}", filename);

            return Ok(new
            {
                filename = filename,
                originalName = file.FileName,
                size = file.Length,
                uploadedAt = DateTime.UtcNow
            });
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "File upload failed");
            return StatusCode(500, "File upload failed");
        }
    }
}

47.3 Multipart Form Data

Upload with additional metadata:

[HttpPost("upload-with-metadata")]
public async Task<IActionResult> UploadWithMetadata(
    IFormFile file,
    [FromForm] string description,
    [FromForm] string category)
{
    if (file == null || file.Length == 0)
        return BadRequest("File is required");

    var metadata = new
    {
        filename = file.FileName,
        description,
        category,
        uploadedAt = DateTime.UtcNow
    };

    _logger.LogInformation("File upload with metadata: {@Metadata}", metadata);

    // Process file with metadata
    var filename = $"{Guid.NewGuid()}{Path.GetExtension(file.FileName)}";
    var filepath = Path.Combine(UploadDirectory, filename);

    using (var stream = System.IO.File.Create(filepath))
    {
        await file.CopyToAsync(stream);
    }

    return Ok(new { success = true, filename, metadata });
}

Client (HTML form):

<form enctype="multipart/form-data">
    <input type="file" name="file" required />
    <input type="text" name="description" placeholder="Description" />
    <select name="category">
        <option value="reports">Reports</option>
        <option value="imports">Imports</option>
    </select>
    <button type="submit">Upload</button>
</form>

<script>
document.querySelector('form').addEventListener('submit', async (e) => {
    e.preventDefault();
    
    const formData = new FormData(this);
    
    const response = await fetch('/api/files/upload-with-metadata', {
        method: 'POST',
        body: formData
    });
    
    const result = await response.json();
    console.log('Upload result:', result);
});
</script>

Client (JavaScript):

const fileInput = document.querySelector('input[type="file"]');
const file = fileInput.files[0];

const formData = new FormData();
formData.append('file', file);
formData.append('description', 'My CSV import');
formData.append('category', 'imports');

const response = await fetch('/api/files/upload-with-metadata', {
    method: 'POST',
    body: formData
});

47.4 File Validation

Comprehensive validation:

// Services/FileValidationService.cs
public interface IFileValidationService
{
    ValidationResult ValidateFile(IFormFile file);
}

public class ValidationResult
{
    public bool IsValid { get; set; }
    public List<string> Errors { get; set; } = new();
}

public class FileValidationService : IFileValidationService
{
    private readonly ILogger<FileValidationService> _logger;
    private const long MaxFileSize = 50 * 1024 * 1024;

    private static readonly Dictionary<string, string[]> AllowedTypes = new()
    {
        { "csv", new[] { "text/csv", "text/plain" } },
        { "json", new[] { "application/json" } },
        { "xlsx", new[] { "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" } },
        { "pdf", new[] { "application/pdf" } },
        { "jpg", new[] { "image/jpeg" } },
        { "png", new[] { "image/png" } }
    };

    public FileValidationService(ILogger<FileValidationService> logger)
    {
        _logger = logger;
    }

    public ValidationResult ValidateFile(IFormFile file)
    {
        var result = new ValidationResult { IsValid = true };

        // Check file exists
        if (file == null || file.Length == 0)
        {
            result.IsValid = false;
            result.Errors.Add("File is required");
            return result;
        }

        // Check filename
        if (string.IsNullOrWhiteSpace(file.FileName))
        {
            result.IsValid = false;
            result.Errors.Add("Invalid filename");
            return result;
        }

        // Check for path traversal
        var filename = Path.GetFileName(file.FileName);
        if (filename != file.FileName)
        {
            result.IsValid = false;
            result.Errors.Add("Path traversal detected");
            return result;
        }

        // Check file size
        if (file.Length > MaxFileSize)
        {
            result.IsValid = false;
            result.Errors.Add($"File exceeds {MaxFileSize / 1024 / 1024}MB limit");
        }

        // Check extension
        var extension = Path.GetExtension(file.FileName).TrimStart('.').ToLowerInvariant();
        if (!AllowedTypes.ContainsKey(extension))
        {
            result.IsValid = false;
            result.Errors.Add($"Extension .{extension} not allowed");
        }

        // Check MIME type
        if (!AllowedTypes[extension].Contains(file.ContentType))
        {
            result.IsValid = false;
            result.Errors.Add($"Invalid MIME type. Expected: {string.Join(", ", AllowedTypes[extension])}");
        }

        // Verify file signature (magic bytes)
        if (!VerifyFileSignature(file, extension))
        {
            result.IsValid = false;
            result.Errors.Add("File signature doesn't match extension");
        }

        if (result.Errors.Count > 0)
        {
            _logger.LogWarning("File validation failed: {@Errors}", result.Errors);
            result.IsValid = false;
        }

        return result;
    }

    private bool VerifyFileSignature(IFormFile file, string extension)
    {
        // Read first few bytes
        var headerBytes = new byte[8];
        using (var stream = file.OpenReadStream())
        {
            stream.Read(headerBytes, 0, 8);
        }

        // Verify magic bytes
        return extension switch
        {
            "pdf" => headerBytes[0] == 0x25 && headerBytes[1] == 0x50,  // %P
            "xlsx" => headerBytes[0] == 0x50 && headerBytes[1] == 0x4B,  // PK
            "png" => headerBytes[0] == 0x89 && headerBytes[1] == 0x50,  // .P
            "jpg" => headerBytes[0] == 0xFF && headerBytes[1] == 0xD8,  // FF D8
            _ => true  // Skip for text files
        };
    }
}

// Register in Program.cs
builder.Services.AddScoped<IFileValidationService, FileValidationService>();

// Use in controller
[HttpPost("upload")]
public async Task<IActionResult> UploadFile(IFormFile file)
{
    var validationResult = _fileValidationService.ValidateFile(file);
    
    if (!validationResult.IsValid)
    {
        return BadRequest(new { errors = validationResult.Errors });
    }

    // Save file...
}

47.5 Streaming Large Files

Avoid loading entire file in memory:

[HttpPost("upload-large")]
public async Task<IActionResult> UploadLargeFile(IFormFile file)
{
    if (file == null || file.Length == 0)
        return BadRequest("File required");

    var filename = $"{Guid.NewGuid()}{Path.GetExtension(file.FileName)}";
    var filepath = Path.Combine(UploadDirectory, filename);

    try
    {
        // Stream directly to disk (doesn't load entire file in memory)
        using (var sourceStream = file.OpenReadStream())
        using (var destinationStream = System.IO.File.Create(filepath))
        {
            await sourceStream.CopyToAsync(destinationStream, 81920);  // 80KB chunks
        }

        _logger.LogInformation("Large file uploaded: {Filename}, Size: {Size}", 
            filename, file.Length);

        return Ok(new { filename, size = file.Length });
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, "Large file upload failed");
        
        // Clean up partial file
        if (System.IO.File.Exists(filepath))
            System.IO.File.Delete(filepath);
        
        return StatusCode(500, "Upload failed");
    }
}

47.6 File Download

Download file with proper headers:

[HttpGet("download/{filename}")]
public IActionResult DownloadFile(string filename)
{
    // Validate filename (prevent directory traversal)
    var safeName = Path.GetFileName(filename);
    if (safeName != filename || string.IsNullOrWhiteSpace(safeName))
    {
        return BadRequest("Invalid filename");
    }

    var filepath = Path.Combine(UploadDirectory, safeName);

    // Check file exists
    if (!System.IO.File.Exists(filepath))
    {
        return NotFound("File not found");
    }

    try
    {
        // Read file into memory
        var fileBytes = System.IO.File.ReadAllBytes(filepath);

        // Return with proper headers
        return File(fileBytes, "application/octet-stream", filename);
        
        // Or specify content-type:
        // return File(fileBytes, "application/pdf", filename);
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, "File download failed");
        return StatusCode(500, "Download failed");
    }
}

Stream file download (for large files):

[HttpGet("download-stream/{filename}")]
public IActionResult DownloadFileStream(string filename)
{
    var safeName = Path.GetFileName(filename);
    if (safeName != filename)
        return BadRequest("Invalid filename");

    var filepath = Path.Combine(UploadDirectory, safeName);

    if (!System.IO.File.Exists(filepath))
        return NotFound("File not found");

    // Determine content type
    var contentType = Path.GetExtension(filename).ToLowerInvariant() switch
    {
        ".pdf" => "application/pdf",
        ".csv" => "text/csv",
        ".json" => "application/json",
        ".xlsx" => "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
        _ => "application/octet-stream"
    };

    // Stream file (efficient for large files)
    var stream = System.IO.File.OpenRead(filepath);
    
    return File(stream, contentType, filename, enableRangeProcessing: true);
}

47.7 Resumable Uploads

Support pause and resume:

[HttpPost("upload-resumable")]
public async Task<IActionResult> UploadResumable(
    IFormFile file,
    [FromForm] int chunkIndex,
    [FromForm] int totalChunks,
    [FromForm] string uploadId)
{
    var chunkDirectory = Path.Combine(UploadDirectory, "chunks", uploadId);
    Directory.CreateDirectory(chunkDirectory);

    var chunkPath = Path.Combine(chunkDirectory, $"chunk_{chunkIndex}");

    try
    {
        // Save chunk
        using (var sourceStream = file.OpenReadStream())
        using (var fileStream = System.IO.File.Create(chunkPath))
        {
            await sourceStream.CopyToAsync(fileStream);
        }

        _logger.LogInformation("Chunk {ChunkIndex}/{TotalChunks} uploaded for upload {UploadId}", 
            chunkIndex + 1, totalChunks, uploadId);

        // Check if all chunks received
        var chunkFiles = Directory.GetFiles(chunkDirectory, "chunk_*");
        
        if (chunkFiles.Length == totalChunks)
        {
            // Merge chunks
            var finalPath = Path.Combine(UploadDirectory, $"{uploadId}.bin");
            
            using (var finalStream = System.IO.File.Create(finalPath))
            {
                for (int i = 0; i < totalChunks; i++)
                {
                    var chunkFile = Path.Combine(chunkDirectory, $"chunk_{i}");
                    using (var chunkStream = System.IO.File.OpenRead(chunkFile))
                    {
                        await chunkStream.CopyToAsync(finalStream);
                    }
                }
            }

            // Clean up chunk directory
            Directory.Delete(chunkDirectory, true);

            return Ok(new { uploadId, status = "complete", finalPath });
        }

        return Ok(new { uploadId, status = "chunk_received", chunkIndex });
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, "Resumable upload failed");
        return StatusCode(500, "Upload failed");
    }
}

Client-side resumable upload:

async function uploadResumable(file, chunkSize = 1024 * 1024) {  // 1MB chunks
    const uploadId = generateUUID();
    const totalChunks = Math.ceil(file.size / chunkSize);

    for (let chunkIndex = 0; chunkIndex < totalChunks; chunkIndex++) {
        const start = chunkIndex * chunkSize;
        const end = Math.min(start + chunkSize, file.size);
        const chunk = file.slice(start, end);

        const formData = new FormData();
        formData.append('file', chunk);
        formData.append('chunkIndex', chunkIndex);
        formData.append('totalChunks', totalChunks);
        formData.append('uploadId', uploadId);

        try {
            const response = await fetch('/api/files/upload-resumable', {
                method: 'POST',
                body: formData
            });

            const result = await response.json();
            console.log(`Chunk ${chunkIndex + 1}/${totalChunks} uploaded`);

            // Update progress
            const progress = ((chunkIndex + 1) / totalChunks) * 100;
            updateProgressBar(progress);

            if (result.status === 'complete') {
                console.log('Upload complete!');
                return result;
            }
        } catch (error) {
            console.error(`Chunk ${chunkIndex} failed:`, error);
            // Could retry with exponential backoff
        }
    }
}

47.8 Security Best Practices

1. Prevent directory traversal:

// ❌ WRONG
var filepath = Path.Combine(UploadDirectory, userProvidedFilename);

// ✓ CORRECT
var safeName = Path.GetFileName(userProvidedFilename);
var filepath = Path.Combine(UploadDirectory, safeName);

2. Store outside web root:

// ❌ WRONG - In web-accessible directory
private const string UploadDirectory = "/wwwroot/uploads";

// ✓ CORRECT - Outside web root
private const string UploadDirectory = "/app/uploads";

3. Validate MIME type:

if (!AllowedMimeTypes.Contains(file.ContentType))
{
    return BadRequest("Invalid file type");
}

4. Verify file signature:

// Check magic bytes match extension
if (!VerifyFileSignature(file, extension))
{
    return BadRequest("File signature doesn't match");
}

5. Scan for malware (optional):

dotnet add package ClamAV.Net

var clam = new ClamClient("localhost", 3310);
var result = await clam.ScanFileAsync(filepath);

if (result == ClamScanResult.Virus)
{
    System.IO.File.Delete(filepath);
    return BadRequest("File contains malware");
}

47.9 Testing File Operations

Test file upload:

// Tests/Controllers/FilesControllerTests.cs
public class FilesControllerTests
{
    [Fact]
    public async Task UploadFile_WithValidFile_ReturnsOk()
    {
        // Arrange
        var content = "test,data\n1,2\n3,4";
        var fileName = "test.csv";
        
        var file = new FormFile(
            new MemoryStream(Encoding.UTF8.GetBytes(content)),
            0,
            content.Length,
            "file",
            fileName)
        {
            Headers = new HeaderDictionary(),
            ContentType = "text/csv"
        };

        // Act
        var result = await _controller.UploadFile(file);

        // Assert
        var okResult = Assert.IsType<OkObjectResult>(result);
        Assert.NotNull(okResult.Value);
    }

    [Fact]
    public async Task UploadFile_WithOversizedFile_ReturnsBadRequest()
    {
        // Arrange
        var largeContent = new string('x', (50 * 1024 * 1024) + 1);  // 50MB + 1 byte
        
        var file = new FormFile(
            new MemoryStream(Encoding.UTF8.GetBytes(largeContent)),
            0,
            largeContent.Length,
            "file",
            "large.csv")
        {
            Headers = new HeaderDictionary(),
            ContentType = "text/csv"
        };

        // Act
        var result = await _controller.UploadFile(file);

        // Assert
        var badRequest = Assert.IsType<BadRequestObjectResult>(result);
        Assert.NotNull(badRequest.Value);
    }
}

47.10 What’s Next

You now have:

✓ Basic file upload endpoints
✓ Multipart form data with metadata
✓ Comprehensive file validation
✓ File signature verification
✓ Streaming for large files
✓ File download with proper headers
✓ Range request support
✓ Resumable uploads with chunks
✓ Security best practices (directory traversal, MIME types, signatures)
✓ Malware scanning
✓ Testing file operations

Next: GraphQL Introduction—Querying and mutating data with GraphQL as alternative to REST.