Client-side Caching and CDNs

# Client-side Caching and CDNs

## CS291A: Scalable Internet Services

---

# Client-side Caching Motivation

We want our important application data persisted safely in our data center.

This data will be regularly read and updated by geographically distributed
clients.

Our access to the data needs to be fast.

---

# Response Time and Human Perception

| Delay         | User Reaction                |
| -------------:|:----------------------------:|
|   0 -  100 ms | Instant                      |
| 100 -  300 ms | Small perceptible delay      |
| 300 - 1000 ms | Machine is working           |
|      1000+ ms | Likely mental context switch |
|    10,000+ ms | Task is abandoned            |

Source: [High Performance Browser
Networking](https://hpbn.co/primer-on-web-performance/)

---

# Minimum Latencies

| Route                    | Distance  | Time, light in vacuum | Time, light in fiber |
| ------------------------ | --------- | --------------------- | -------------------- |
| NYC to SF                | 4,148 km  | 14 ms                 | 21 ms                |
| NYC to London            | 5,585 km  | 19 ms                 | 28 ms                |
| NYC to Sydney            | 15,993 km | 53 ms                 | 80 ms                |
| Equatorial circumference | 40,075 km | 133.7 ms              | 200 ms               |

Source: [High Performance Browser
Networking](https://hpbn.co/primer-on-web-performance/)

???

* Light travels at 299,792 km/s in a vacuum
* Light travels at 200,000 km/s in fiber
* These times are one-way
* Double the time for round-trip
  * 42 ms NYC to SF
  * 56 ms NYC to London
  * 160 ms NYC to Sydney

---

# Multiple HTTP Requests per Page

![Total Requests Per Page](http_archive_requests_per_page_2024.png)

Source: <https://httparchive.org/interesting.php#reqTotal>

---

# Caching

The fastest request is one that never happens.

Cache - a component that transparently stores data so that future requests
for the same data can be served faster

> Where can we introduce caching?

* Inside the browser

* In "front" of the server (CDNs, ISP cache, etc.)

* Inside the application server

* Inside the database (query cache)

---

# Client-side Caching

> How does the browser cache data?

> How does the browser know when it can or cannot use cached data?

The building blocks of caching are all provided via HTTP headers:

* cache-control
    * no-store
    * no-cache
    * max-age
    * public | private

* etag

* last-modified

* if-none-match

* if-modified-since

---

# cache-control: no-store

When `cache-control: no-store` is provided as a header to an HTTP response the
browser and intermediate proxy-caches:

* must not save the information in any non-volatile storage

* must make a best effort attempt to remove the information from volatile
  storage as promptly as possible

__Note__: The browser may retain a copy in memory (volatile storage) for use
with the browser's back/forward buttons.

Generally used for sensitive information.

---

# cache-control: no-cache

When `cache-control: no-cache` is provided as a header to an HTTP response the
browser and intermediate proxy-caches:

* must not use the response to satisfy a subsequent request without successful
  revalidation with the origin server

Useful to force the browser and intermediate caches to check for updated
content.

---

# cache-control: private

When `cache-control: private` is provided as a header to an HTTP response the
browser is free to cache the response (for the current user) but, intermediate
proxy-caches should discard the data.

This is the opposite of `cache-control: public`.

Rails built-in caching securely defaults to `cache-control: private`.

---

# cache-control: max-age=120

When `cache-control: max-age=120` is provided as a header to an HTTP response
the browser and intermediate proxy-caches should consider the information to be
fresh until the specified number of seconds (`120` in this case) has passed.

This is the modern version of the `expires` and `date` headers.

---

# etag: "5bf444d26f9f1c74"

When an etag (entity tag) header is provided as a header to an HTTP response
the browser will keep an association between the request, the etag, and the
response.

When requesting the same resource in the future, the browser may add an
`if-none-match` header along with the etag in the HTTP request.

---

# last-modified: Thu, 31 Oct 2024 16:15:36 GMT

When a `last-modified` header is provided in an HTTP response the browser may
save the information to use in future requests for the same resource.

When requesting the same resource in the future, the browser may add an
`if-modified-since` header along with the datetime in the HTTP request.

---

# First GET Example

```
$ curl -I https://cs291.com

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Last-Modified: Mon, 21 Oct 2019 21:41:43 GMT
ETag: "5dae2617-48a8"
Cache-Control: max-age=600
```

---

# if-none-match: "5bf444d26f9f1c74"

When accompanying an HTTP request, the `if-none-match` HTTP header indicates
that the client has a cached copy with the associated tag.  Multiple etags can
be provided.

If the server's current version of the resource maps to one of the provided
etags, the server will return 304 (not modified) with the etag of the current
resource included.

Otherwise the server will return the full response body along with the
appropriate etag for the updated resource.

```sh
$ curl -I https://cs291.com --header 'if-none-match: "5dae2617-48a8"'

HTTP/1.1 304 Not Modified
```

---

# if-modified-since: Thu, 31 Oct 2024 16:15:36 GMT

When accompanying an HTTP request, the `if-modified-since` HTTP header
indicates that the client already has a copy that was fresh as of the specified
datetime.

If the server's copy is newer than the specified datetime, it will be served to
the client along with a new `last-modified` HTTP response header.

If the server's copy has not changed since the specified date, the server will
return 304 (not modified).

```sh
$ curl -I https://cs291.com --header 'if-modified-since: Mon, 21 Oct 2019 21:41:43 GMT'

HTTP/1.1 304 Not Modified
```

---

# HTTP Caching Summary

![Defining optimal Cache-Control policy](http-cache-decision-tree.png)

[Source](https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching)

---

# Applying Client-side Caching #1

Assume we want to serve JavaScript that won't change over the next day, but it
contains user-specific code.

> What headers should the HTTP response include?

## Caching Reusable, Private Resources

```http
Cache-control: private, max-age=86400
```

Intermediate proxy-caches should not save a copy, and the browser can consider
the resource fresh for up to a day since it was retrieved.

---

# Applying Client-side Caching #2

Assume we are serving an image that may change in the future, and we never want
a stale version shown. The image is not specific to the requester.

> What headers should the HTTP response include?

## Caching Reusable, Public Resources

```http
Cache-control: public, no-cache
ETag: 4d7a6ca05b5df656
```

Browser will store a copy to be used if a 304 (not modified) response is
sent. In subsequent HTTP requests for the resource the browser will include:

```http
if-none-match: 4d7a6ca05b5df656
```

---

# Applying Client-side Caching #3

Assume we are serving an image containing the user's social security and credit
card numbers.

> What headers should the HTTP response include?

## No Caching!

```http
Cache-control: no-store
```

The browser may keep a copy in memory for browser navigation, but __must__
not save a copy to a non-volatile location.

---

# Content (Delivery|Distribution) Networks

---

# CDN Overview

![CDN Overview](cdn.png)

---

# CDN Benefits

1. Reduce latency from client to server and back
2. Provide "protection" to the origin server (e.g., DDoS protection)
3. Reduce requests handled by origin server.

---

# CDN Providers

* [AWS CloudFront](https://aws.amazon.com/cloudfront/)
* [Akamai](https://www.akamai.com)
* [CloudFlare](https://www.cloudflare.com/cdn/)
* [Fastly](https://www.fastly.com)
* [Google Cloud CDN](https://cloud.google.com/cdn/)
* [Microsoft Azure CDN](https://azure.microsoft.com/en-us/services/cdn/)

---

# Beyond HTTP Caching

We've covered HTTP-based caching and cookies, but modern web applications need more sophisticated client-side caching strategies.

## The Problem with Traditional Approaches

### HTTP Caching Limitations:
* **Network dependency** - No offline functionality
* **Server-controlled** - Can't cache user-specific data
* **Limited to HTTP resources** - Can't cache computed results
* **No programmatic control** - Browser decides what to cache

### Cookie Limitations:
* **4KB size limit** - Can't store much data
* **Sent with every request** - Network overhead
* **Synchronous API** - Blocks main thread
* **Security vulnerabilities** - XSS attacks
* **String-only** - No complex data types

---

## Why We Need Better Client-Side Caching

* **Reduce server load** - Fewer requests = less server processing
* **Improve scalability** - Handle more users with same server resources
* **Lower bandwidth costs** - Less data transfer between client and server
* **Better user experience** - Instant responses, smooth interactions
* **Offline functionality** - Apps that work without network
* **Data persistence** - Maintain state across sessions

---

# Client-Side Storage Hierarchy

| Storage Type | Size Limit | Persistence | Use Case |
|-------------|------------|-------------|----------|
| **Memory** | Limited by available RAM and Configuration | Single page only | Fast access, temporary data |
| **Session Storage** | ~5-10MB | Tab session | Temporary, tab-specific data |
| **Local Storage** | ~5-10MB | Until cleared | User preferences, settings |
| **IndexedDB** | Larger, limited by disk space | Until cleared | Complex data, large datasets |
| **Service Worker Cache** | Larger, limited by disk space | Until cleared | Offline resources |

---

# Browser Storage APIs

---

# Session Storage

```javascript
// Store temporary data
sessionStorage.setItem('currentFormData', JSON.stringify({
  step: 3,
  formData: { name: 'John', email: 'john@example.com' }
}));

// Retrieve when user navigates back
const formData = JSON.parse(sessionStorage.getItem('currentFormData'));

// Automatically cleared when tab closes
```

## Characteristics:
* **Session-only** - Cleared when tab closes
* **Tab-isolated** - Each tab has separate storage
* **Synchronous** - Blocks main thread
* **String-only** - Must serialize objects
* **Same-origin** - Isolated per domain
* **Use cases** - Multi-step forms, temporary state, things that shouldn't impact other tabs even if they are on the same domain

**Documentation:** [MDN Web Storage API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Storage_API)

---

# Local Storage

```javascript
// Store data
localStorage.setItem('userPreferences', JSON.stringify({
  theme: 'dark',
  language: 'en',
  notifications: true
}));

// Retrieve data
const preferences = JSON.parse(localStorage.getItem('userPreferences'));

// Remove data
localStorage.removeItem('userPreferences');

// Clear all
localStorage.clear();
```

## Characteristics:
* **Persistent** - Survives browser restarts
* **Synchronous** - Blocks main thread
* **String-only** - Must serialize objects
* **Same-origin** - Isolated per domain
* **Same API** - Identical to session storage
* **Use cases** - User preferences, settings, things that should persist across sessions or be synchronized between tabs

**Documentation:** [MDN Web Storage API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Storage_API)

---

# IndexedDB: For Complex Data

## When to Use IndexedDB:
* **Large datasets** - More than localStorage can handle
* **Complex queries** - Need to search/filter data
* **Binary data** - Images, files, blobs
* **Offline apps** - Store data when network is unavailable

## Scalability Benefits:
* **Reduce database queries** - Cache complex query results
    * Cache more larger complex structures and do further processing on the client
    * Data may be out of sync with the server at some point in time

## Characteristics:
* **Asynchronous** - Doesn't block the UI
* **Large capacity** - Limited by disk space, not fixed size
* **Complex API** - More setup than localStorage
* **Persistent** - Survives browser restarts

**Documentation:** [MDN IndexedDB API](https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API)

---

# Service Workers: Offline Caching

## Service Worker Capabilities:
* **Intercept network requests** - Control what gets cached
* **Offline functionality** - App works without internet
* **Background processing** - Runs even when page is closed (sort of)
    * Event driven by push events or scheduled tasks

## Scalability Benefits:
* **Reduce server requests** - Cache static assets and API responses
    * Pushes the complexity of caching to the client
    * Possible to prevent more requests to the server that might otherwise result in 304s
    * Trade-off: more complexity on the client

## When to Use:
* **Offline-first apps** - Apps that need to work without network
* **Performance optimization** - Cache static assets
* **Progressive Web Apps** - Modern web app features

**Documentation:** [MDN Service Worker API](https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API)

---

# Decision Framework:

1. **How long should data persist?**
   - Session only → Session Storage
   - Across sessions → Local Storage / IndexedDB

2. **How large is the data?**
   - Small (< 5MB) → Local Storage
   - Large → IndexedDB

3. **Do you need offline access?**
   - Yes → Service Worker + Cache API
   - No → HTTP caching + localStorage

4. **Is it user-specific?**
   - Yes → Client-side storage
   - No → HTTP caching + CDN

---

# Key Takeaways:

* **Client-side caching reduces server load** - Fewer requests = better scalability
* **Choose the right tool** for your specific use case
* **Layer your caching** strategies for maximum performance
* **Consider offline scenarios** in modern web applications
* **Caching is a key scalability technique** - Essential for handling growth