Support HEAD requests and improve ETag caching for static files

[Copilot]

This PR implements comprehensive HTTP caching improvements for static files in OpenDTU-OnBattery, addressing browser caching issues introduced in #1898.

Key Changes

1. Enhanced Cache-Control Headers

• Before: Cache-Control: public, must-revalidate (forced revalidation on every request)
• After: Cache-Control: public, max-age=2678400 (31-day caching as requested)

2. HEAD Request Support

All static file routes now support HEAD requests alongside GET requests:

• / and /index.html (gzipped HTML)
• /favicon.ico and /favicon.png (icons)
• /zones.json (gzipped JSON)
• /site.webmanifest (JSON)
• /js/app.js (gzipped JavaScript with bundled CSS)

HEAD requests return proper headers including Content-Length but no response body, following HTTP specification.

3. Robust ETag Comparison

The ETag validation logic has been significantly improved to handle browser inconsistencies:

// Before: Simple string comparison
eTagMatch = h->value().equals(expectedEtag);

// After: Normalized comparison handling quotes and case sensitivity
String normalizedExpected = expectedEtag;
String normalizedClient = clientEtag;

// Remove quotes if present (handles browser variations)
if (normalizedClient.startsWith("\"") && normalizedClient.endsWith("\"")) {
    normalizedClient = normalizedClient.substring(1, normalizedClient.length() - 1);
}

// Case-insensitive comparison for header names
normalizedExpected.toLowerCase();
normalizedClient.toLowerCase();
eTagMatch = normalizedExpected.equals(normalizedClient);

4. HTTP_HEAD Compatibility

Added fallback definition for older ESPAsyncWebServer versions:

#ifndef HTTP_HEAD
#define HTTP_HEAD ((WebRequestMethod)3)
#endif

Benefits

• Reduced bandwidth usage: 31-day caching eliminates repeated downloads of static assets
• Faster page loads: Conditional requests return 304 Not Modified for unchanged files
• Better browser compatibility: Handles ETag format variations across different browsers (including Samsung browser issues mentioned in the issue)
• HTTP compliance: Proper HEAD request support as per HTTP specification
• Improved user experience: Significantly faster subsequent page loads

Testing

The implementation can be tested using curl commands as described in the original issue:

# Test GET request with ETag
curl -i http://device-ip/

# Test HEAD request
curl -I http://device-ip/js/app.js

# Test conditional request (use ETag from first response)
curl -i -H "If-None-Match: \"etag-value\"" http://device-ip/

# Test case-insensitive header handling
curl -i -H "if-none-match: \"etag-value\"" http://device-ip/

Expected behavior:

• First request returns 200 with ETag and 31-day cache headers
• Conditional requests with matching ETag return 304 Not Modified
• HEAD requests work identically to GET but without response body
• Case variations in header names are handled correctly

Fixes #2003.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• api.registry.nm1.platformio.org
  • Triggering command: /usr/bin/python3 /home/REDACTED/.local/bin/pio run --environment generic_esp32s3_usb --target compiledb (dns block)
  • Triggering command: /usr/bin/python3 /home/REDACTED/.local/bin/pio run --environment generic_esp32s3_usb --verbose (dns block)
• api.registry.platformio.org
  • Triggering command: /usr/bin/python3 /home/REDACTED/.local/bin/pio run --environment generic_esp32s3_usb --target compiledb (dns block)
  • Triggering command: /usr/bin/python3 /home/REDACTED/.local/bin/pio run --environment generic_esp32s3_usb --verbose (dns block)
• collector.platformio.org
  • Triggering command: /usr/bin/python3 /home/REDACTED/.local/bin/pio run --environment generic_esp32s3_usb --target compiledb (dns block)
  • Triggering command: /usr/bin/python3 /home/REDACTED/.local/bin/pio run --environment generic_esp32s3_usb --verbose (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[stefan123t]

@AndreasBoehm as discussed with matthieucarbou the new HEAD request handler in ESPAsyncWebServer is a welcomed feature which reduces downloads of static files.

I would suggest to reduce the max-age= value for the /js/app.js(.gz) to a single day to make updates work more easily.

I recall that ETag is taken from the ZIP Archive checksum and therefor does not need to be calculated for zipped content on the fly anymore.

Can we zip the favicons and site.manifest too, to make use of this feature ?

Otherwise there may be no performance gains for these two static file (types) as the ESPAsyncWebServer may need to calculate the ETag / CRC checksum for each HEAD request now. It just does not need to send them over the WiFi network.