#Part 3 - Middleware Pipeline

Elliot Forbes · Mar 20, 2026 · 10 min read

In the previous tutorial, we added routing to our gateway. Now it’s time to add middleware — the mechanism that lets us process requests and responses as they flow through the gateway.

Middleware is where the real power of an API gateway lives. Logging, authentication, rate limiting, header manipulation — these are all concerns that apply across services, and middleware lets us handle them in one place rather than duplicating logic in every backend service.

By the end of this tutorial, we’ll have a composable middleware pipeline and three working middleware implementations: request logging, CORS headers, and custom header injection.

What is Middleware?

Middleware is code that sits between receiving a request and sending the response. Each middleware gets a chance to inspect or modify the request before it reaches the backend, and inspect or modify the response before it goes back to the client.

Client Request
  → Logging Middleware (logs the request)
    → CORS Middleware (adds CORS headers)
      → Header Middleware (adds custom headers)
        → Proxy (forwards to backend)
      ← Header Middleware
    ← CORS Middleware
  ← Logging Middleware (logs the response)
Client Response

The key design goal is composability — you should be able to stack middleware like building blocks, and each middleware shouldn’t need to know about any other.

Designing with Traits

In Rust, the way we define shared behavior is through traits. If you’re coming from Go, traits are similar to interfaces. If you’re coming from TypeScript, they’re like interfaces too — but with a twist: Rust traits can have default implementations.

Let’s create a new file for our middleware system:

src/
├── main.rs
├── middleware/
│   ├── mod.rs        # The trait definition and pipeline
│   ├── logging.rs    # Logging middleware
│   ├── cors.rs       # CORS middleware
│   └── headers.rs    # Custom header middleware
├── proxy.rs
└── router.rs

First, create the src/middleware/ directory and define our trait in src/middleware/mod.rs:

src/middleware/mod.rs

pub mod cors;
pub mod headers;
pub mod logging;

use bytes::Bytes;
use http_body_util::combinators::BoxBody;
use hyper::{Request, Response};

/// The context that flows through the middleware pipeline
pub struct RequestContext {
    pub client_ip: String,
    pub start_time: std::time::Instant,
}

/// The Middleware trait - implement this to create new middleware
pub trait Middleware: Send + Sync {
    /// Called before the request is forwarded to the backend.
    /// Return the (possibly modified) request, or return a response
    /// to short-circuit the pipeline and skip the backend entirely.
    fn on_request(
        &self,
        req: Request<BoxBody<Bytes, hyper::Error>>,
        ctx: &RequestContext,
    ) -> Result<Request<BoxBody<Bytes, hyper::Error>>, Response<BoxBody<Bytes, hyper::Error>>>;

    /// Called after the response comes back from the backend.
    /// Return the (possibly modified) response.
    fn on_response(
        &self,
        resp: Response<BoxBody<Bytes, hyper::Error>>,
        ctx: &RequestContext,
    ) -> Response<BoxBody<Bytes, hyper::Error>> {
        // Default: pass through unchanged
        resp
    }

    /// A human-readable name for this middleware (used in logging)
    fn name(&self) -> &str;
}

Let’s unpack the important concepts here:

Send + Sync — These are Rust’s thread safety markers. Send means the type can be moved to another thread. Sync means it can be shared between threads via references. We need both because our middleware will be shared across async tasks. The compiler enforces this — if your middleware type isn’t thread-safe, it simply won’t compile. This is one of Rust’s superpowers: thread safety bugs become compile-time errors.

Result<Request, Response> — The on_request method returns a Result. The Ok variant means “here’s the (possibly modified) request, continue the pipeline.” The Err variant means “stop here and send this response back to the client.” This is how authentication middleware can reject requests without hitting the backend.

Default implementation — Notice that on_response has a body: it just returns the response unchanged. This is a default implementation — middleware that only care about requests can skip implementing on_response. Go interfaces can’t do this; Rust traits can.

The Middleware Pipeline

Now let’s build the pipeline that chains middleware together. Add this to the same file:

src/middleware/mod.rs

/// A pipeline of middleware that processes requests in order
pub struct MiddlewarePipeline {
    middlewares: Vec<Box<dyn Middleware>>,
}

impl MiddlewarePipeline {
    pub fn new() -> Self {
        MiddlewarePipeline {
            middlewares: Vec::new(),
        }
    }

    pub fn add(&mut self, middleware: Box<dyn Middleware>) {
        println!("  Middleware added: {}", middleware.name());
        self.middlewares.push(middleware);
    }

    /// Process a request through all middleware before forwarding
    pub fn process_request(
        &self,
        mut req: Request<BoxBody<Bytes, hyper::Error>>,
        ctx: &RequestContext,
    ) -> Result<Request<BoxBody<Bytes, hyper::Error>>, Response<BoxBody<Bytes, hyper::Error>>> {
        for mw in &self.middlewares {
            match mw.on_request(req, ctx) {
                Ok(modified_req) => req = modified_req,
                Err(response) => return Err(response),
            }
        }
        Ok(req)
    }

    /// Process a response through all middleware in reverse order
    pub fn process_response(
        &self,
        mut resp: Response<BoxBody<Bytes, hyper::Error>>,
        ctx: &RequestContext,
    ) -> Response<BoxBody<Bytes, hyper::Error>> {
        for mw in self.middlewares.iter().rev() {
            resp = mw.on_response(resp, ctx);
        }
        resp
    }
}

Box<dyn Middleware> — This is a trait object. Since different middleware types have different sizes (logging middleware might have different fields than CORS middleware), we can’t store them in a Vec directly — Rust needs to know the size of each element at compile time. Box allocates on the heap and gives us a fixed-size pointer, and dyn Middleware means “any type that implements the Middleware trait.” This is Rust’s equivalent of []Middleware in Go where Middleware is an interface.

Reverse order for responses — Notice that process_response iterates in reverse. This gives us the “onion” pattern: the first middleware to see the request is the last to see the response, just like the diagram at the top of this tutorial.

Implementing Logging Middleware

Let’s start with the most useful middleware — request logging:

src/middleware/logging.rs

use super::{Middleware, RequestContext};
use bytes::Bytes;
use http_body_util::combinators::BoxBody;
use hyper::{Request, Response};

pub struct LoggingMiddleware;

impl Middleware for LoggingMiddleware {
    fn on_request(
        &self,
        req: Request<BoxBody<Bytes, hyper::Error>>,
        ctx: &RequestContext,
    ) -> Result<Request<BoxBody<Bytes, hyper::Error>>, Response<BoxBody<Bytes, hyper::Error>>> {
        println!(
            "[{}] --> {} {} (from {})",
            chrono::Local::now().format("%Y-%m-%d %H:%M:%S"),
            req.method(),
            req.uri().path(),
            ctx.client_ip,
        );
        Ok(req)
    }

    fn on_response(
        &self,
        resp: Response<BoxBody<Bytes, hyper::Error>>,
        ctx: &RequestContext,
    ) -> Response<BoxBody<Bytes, hyper::Error>> {
        let elapsed = ctx.start_time.elapsed();
        println!(
            "[{}] <-- {} ({:.2}ms)",
            chrono::Local::now().format("%Y-%m-%d %H:%M:%S"),
            resp.status(),
            elapsed.as_secs_f64() * 1000.0,
        );
        resp
    }

    fn name(&self) -> &str {
        "logging"
    }
}

We’re using the chrono crate for timestamps, so add it to Cargo.toml:

Cargo.toml

[dependencies]
# ... existing deps ...
chrono = "0.4"

This middleware doesn’t modify anything — it just logs the request on the way in and the response (with timing) on the way out. The RequestContext carries the start time so we can calculate how long the whole request took.

Implementing CORS Middleware

Cross-Origin Resource Sharing (CORS) headers are essential if your API will be called from web browsers. Without the right headers, browsers will block requests from different domains.

src/middleware/cors.rs

use super::{Middleware, RequestContext};
use bytes::Bytes;
use http_body_util::{combinators::BoxBody, BodyExt, Full};
use hyper::{Request, Response};

pub struct CorsMiddleware {
    allowed_origins: Vec<String>,
    allowed_methods: String,
    allowed_headers: String,
}

impl CorsMiddleware {
    pub fn new(allowed_origins: Vec<String>) -> Self {
        CorsMiddleware {
            allowed_origins,
            allowed_methods: "GET, POST, PUT, DELETE, OPTIONS".to_string(),
            allowed_headers: "Content-Type, Authorization".to_string(),
        }
    }
}

impl Middleware for CorsMiddleware {
    fn on_request(
        &self,
        req: Request<BoxBody<Bytes, hyper::Error>>,
        _ctx: &RequestContext,
    ) -> Result<Request<BoxBody<Bytes, hyper::Error>>, Response<BoxBody<Bytes, hyper::Error>>> {
        // Handle preflight OPTIONS requests
        if req.method() == hyper::Method::OPTIONS {
            let origin = req
                .headers()
                .get("Origin")
                .and_then(|v| v.to_str().ok())
                .unwrap_or("*");

            let allowed_origin = if self.allowed_origins.contains(&"*".to_string()) {
                "*".to_string()
            } else if self.allowed_origins.contains(&origin.to_string()) {
                origin.to_string()
            } else {
                return Ok(req); // Not an allowed origin, skip CORS headers
            };

            let body = Full::new(Bytes::new())
                .map_err(|never| match never {})
                .boxed();

            let response = Response::builder()
                .status(204)
                .header("Access-Control-Allow-Origin", &allowed_origin)
                .header("Access-Control-Allow-Methods", &self.allowed_methods)
                .header("Access-Control-Allow-Headers", &self.allowed_headers)
                .header("Access-Control-Max-Age", "86400")
                .body(body)
                .unwrap();

            return Err(response); // Short-circuit: don't forward to backend
        }

        Ok(req)
    }

    fn on_response(
        &self,
        mut resp: Response<BoxBody<Bytes, hyper::Error>>,
        _ctx: &RequestContext,
    ) -> Response<BoxBody<Bytes, hyper::Error>> {
        // Add CORS headers to all responses
        let headers = resp.headers_mut();
        if !self.allowed_origins.is_empty() {
            let origin = if self.allowed_origins.contains(&"*".to_string()) {
                "*"
            } else {
                self.allowed_origins.first().map(|s| s.as_str()).unwrap_or("*")
            };
            headers.insert(
                "Access-Control-Allow-Origin",
                origin.parse().unwrap(),
            );
        }
        resp
    }

    fn name(&self) -> &str {
        "cors"
    }
}

This middleware demonstrates short-circuiting — when a browser sends a preflight OPTIONS request, we return a response immediately (Err(response)) without ever hitting the backend. The pipeline sees the Err and sends the response back to the client.

Implementing Custom Header Middleware

Finally, let’s add middleware that injects custom headers into every request or response:

src/middleware/headers.rs

use super::{Middleware, RequestContext};
use bytes::Bytes;
use http_body_util::combinators::BoxBody;
use hyper::{Request, Response};

pub struct AddHeadersMiddleware {
    request_headers: Vec<(String, String)>,
    response_headers: Vec<(String, String)>,
}

impl AddHeadersMiddleware {
    pub fn new() -> Self {
        AddHeadersMiddleware {
            request_headers: Vec::new(),
            response_headers: Vec::new(),
        }
    }

    pub fn add_request_header(mut self, name: &str, value: &str) -> Self {
        self.request_headers.push((name.to_string(), value.to_string()));
        self
    }

    pub fn add_response_header(mut self, name: &str, value: &str) -> Self {
        self.response_headers.push((name.to_string(), value.to_string()));
        self
    }
}

impl Middleware for AddHeadersMiddleware {
    fn on_request(
        &self,
        mut req: Request<BoxBody<Bytes, hyper::Error>>,
        _ctx: &RequestContext,
    ) -> Result<Request<BoxBody<Bytes, hyper::Error>>, Response<BoxBody<Bytes, hyper::Error>>> {
        for (name, value) in &self.request_headers {
            req.headers_mut().insert(
                hyper::header::HeaderName::from_bytes(name.as_bytes()).unwrap(),
                value.parse().unwrap(),
            );
        }
        Ok(req)
    }

    fn on_response(
        &self,
        mut resp: Response<BoxBody<Bytes, hyper::Error>>,
        _ctx: &RequestContext,
    ) -> Response<BoxBody<Bytes, hyper::Error>> {
        for (name, value) in &self.response_headers {
            resp.headers_mut().insert(
                hyper::header::HeaderName::from_bytes(name.as_bytes()).unwrap(),
                value.parse().unwrap(),
            );
        }
        resp
    }

    fn name(&self) -> &str {
        "add-headers"
    }
}

Notice the builder pattern in add_request_header and add_response_header — each method takes mut self (ownership) and returns Self, allowing you to chain calls like:

AddHeadersMiddleware::new()
    .add_request_header("X-Gateway", "ferroway")
    .add_response_header("X-Powered-By", "Ferroway/0.1")

This pattern is very common in Rust and gives a clean, fluent API.

Wiring It All Together

Now let’s update main.rs to use the middleware pipeline. We need to adjust how requests flow — they now go through middleware before and after proxying:

src/main.rs

mod middleware;
mod proxy;
mod router;

use bytes::Bytes;
use http_body_util::{combinators::BoxBody, BodyExt, Full};
use hyper::server::conn::http1;
use hyper::service::service_fn;
use hyper::{Request, Response};
use hyper_util::rt::TokioIo;
use middleware::{
    cors::CorsMiddleware, headers::AddHeadersMiddleware, logging::LoggingMiddleware,
    MiddlewarePipeline, RequestContext,
};
use router::Router;
use std::net::SocketAddr;
use std::sync::Arc;
use tokio::net::TcpListener;

struct Gateway {
    router: Router,
    pipeline: MiddlewarePipeline,
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Configure routes
    let mut router = Router::new();
    println!("Configuring routes...");
    router.add_route("/api/users", "http://127.0.0.1:8081");
    router.add_route("/api/orders", "http://127.0.0.1:8082");
    router.add_route("/", "http://127.0.0.1:8080");

    // Configure middleware
    let mut pipeline = MiddlewarePipeline::new();
    println!("Configuring middleware...");
    pipeline.add(Box::new(LoggingMiddleware));
    pipeline.add(Box::new(CorsMiddleware::new(vec!["*".to_string()])));
    pipeline.add(Box::new(
        AddHeadersMiddleware::new()
            .add_request_header("X-Gateway", "ferroway")
            .add_response_header("X-Powered-By", "Ferroway/0.1"),
    ));

    let gateway = Arc::new(Gateway { router, pipeline });

    let addr = SocketAddr::from(([127, 0, 0, 1], 3000));
    let listener = TcpListener::bind(addr).await?;
    println!("Ferroway API Gateway listening on http://{}", addr);

    loop {
        let (stream, remote_addr) = listener.accept().await?;
        let io = TokioIo::new(stream);
        let gateway = gateway.clone();
        let client_ip = remote_addr.ip().to_string();

        tokio::task::spawn(async move {
            if let Err(err) = http1::Builder::new()
                .serve_connection(
                    io,
                    service_fn(move |req| {
                        let gateway = gateway.clone();
                        let client_ip = client_ip.clone();
                        async move { handle_request(req, &gateway, &client_ip).await }
                    }),
                )
                .await
            {
                eprintln!("Error serving connection: {:?}", err);
            }
        });
    }
}

async fn handle_request(
    req: Request<hyper::body::Incoming>,
    gateway: &Gateway,
    client_ip: &str,
) -> Result<Response<BoxBody<Bytes, hyper::Error>>, hyper::Error> {
    let ctx = RequestContext {
        client_ip: client_ip.to_string(),
        start_time: std::time::Instant::now(),
    };

    let path = req.uri().path().to_string();

    // Convert the incoming body to a boxed body for middleware
    let (parts, body) = req.into_parts();
    let boxed_req = Request::from_parts(parts, body.boxed());

    // Run request middleware
    let boxed_req = match gateway.pipeline.process_request(boxed_req, &ctx) {
        Ok(req) => req,
        Err(response) => return Ok(gateway.pipeline.process_response(response, &ctx)),
    };

    // Route and proxy
    let response = match gateway.router.find_route(&path) {
        Some(route) => proxy::forward_request_boxed(boxed_req, route).await?,
        None => {
            let body = Full::new(Bytes::from("404 Not Found"))
                .map_err(|never| match never {})
                .boxed();
            Response::builder().status(404).body(body).unwrap()
        }
    };

    // Run response middleware
    Ok(gateway.pipeline.process_response(response, &ctx))
}

Testing the Middleware

Start the backend and gateway, then make a request:

$ curl -v http://localhost:3000/api/users

In the gateway terminal, you’ll see the logging middleware output:

[2026-03-20 10:30:15] --> GET /api/users (from 127.0.0.1)
[2026-03-20 10:30:15] <-- 200 OK (2.34ms)

And in the curl output, you’ll see the injected headers:

< X-Powered-By: Ferroway/0.1
< Access-Control-Allow-Origin: *

Test CORS preflight:

$ curl -X OPTIONS -v http://localhost:3000/api/users \
  -H "Origin: http://example.com"

You should get a 204 No Content response with the CORS headers — and the backend never sees the request. The middleware short-circuited it.

Conclusion

We’ve built a composable middleware system using Rust’s trait system. The key concepts we covered are traits as shared behavior contracts, trait objects (Box<dyn Middleware>) for heterogeneous collections, the builder pattern for fluent APIs, and short-circuiting with Result to handle cases like CORS preflight.

This pattern is powerful because adding new middleware is as simple as implementing the Middleware trait. The pipeline handles the orchestration.

Challenge - Create a RequestIdMiddleware that generates a unique UUID for each request and adds it as an X-Request-Id header to both the request (so the backend can log it) and the response (so the client can reference it). You’ll want the uuid crate for this.

Next Part

In Part 4 - Rate Limiting, we’ll protect our backend services by implementing a token bucket rate limiter that tracks requests per client IP.

Continue Learning

[rustgateway]

Part 6 - Configuration & Authentication

In the final part of this series, we move our gateway configuration into a YAML file and implement JWT-based authentication middleware to protect our backend services.

Mar 2026 9 min read

[rustgateway]

Part 5 - Load Balancing & Health Checks

In this tutorial, we add load balancing to our API gateway with round-robin distribution and background health checks that automatically remove unhealthy backends from the pool.

Mar 2026 8 min read

[rustgateway]

Part 4 - Rate Limiting

In this tutorial, we implement a token bucket rate limiter as middleware for our API gateway, protecting backend services from abuse by tracking requests per client IP.

Mar 2026 7 min read

[rustgateway]

Part 2 - Routing & Path Matching

In the second part of this series, we add a routing layer to our API gateway so it can forward requests to different backend services based on URL path patterns.

Mar 2026 9 min read

#Part 3 - Middleware Pipeline

##What is Middleware?

##Designing with Traits

##The Middleware Pipeline

##Implementing Logging Middleware

##Implementing CORS Middleware

##Implementing Custom Header Middleware

##Wiring It All Together

##Testing the Middleware

##Conclusion

##Next Part

Continue Learning

Part 6 - Configuration & Authentication

Part 5 - Load Balancing & Health Checks

Part 4 - Rate Limiting

Part 2 - Routing & Path Matching

What is Middleware?

Designing with Traits

The Middleware Pipeline

Implementing Logging Middleware

Implementing CORS Middleware

Implementing Custom Header Middleware

Wiring It All Together

Testing the Middleware

Conclusion

Next Part