Postel: Fast Async Transport for Rust

At Warlock Labs, we're excited to announce the release of postel, our high-performance async transport framework for Rust. Named after internet pioneer Jon Postel, this library embodies his principle of being "conservative in what you send, and liberal in what you accept" - a foundation of robust networked systems.

The Challenge of Modern Networking

Building modern networked services requires juggling multiple protocols, transport layers, and middleware components. Whether you're working with HTTP/2, gRPC, or raw TCP connections, each brings its own complexities around connection management, protocol negotiation, TLS integration, and performance optimization.

While excellent libraries like hyper, tokio, and tower exist, integrating them cohesively often requires significant boilerplate and careful consideration of edge cases. We found ourselves repeatedly implementing similar patterns across our services at Warlock Labs.

Enter Postel

postel provides a unified, high-performance framework for building networked services in Rust. It abstracts away the complexity of protocol handling while maintaining the flexibility to work with any tower-compatible service.

Key Features

• Unified Transport Layer: Abstract over TCP and TLS connections with a single interface
• HTTP/1.1 and HTTP/2 Support: Automatic protocol negotiation and optimization
• First-class TLS Support: Built-in rustls integration with modern cipher suites
• tower/hyper Integration: Seamless compatibility with axum, tonic, and other tower/hyper-based frameworks
• Performance Focused: Optimized for high-throughput and low-latency scenarios
• Graceful Shutdown: Built-in support for clean service termination

Getting Started

Here's a simple example of setting up a secure HTTP/2 tonic server with postel:

use std::net::SocketAddr;|
use std::sync::Arc;|

use hyper_util::rt::TokioExecutor;|
use hyper_util::server::conn::auto::Builder as HttpConnectionBuilder;|
use hyper_util::service::TowerToHyperService;|
use rustls::ServerConfig;|
use tokio::net::TcpListener;|
use tokio_stream::wrappers::TcpListenerStream;|
use tonic::server::NamedService;|
use tonic::transport::Server;|
use tower::ServiceExt;|
use tracing::{info, Level};|

use postel::{load_certs, load_private_key, serve_http_with_shutdown};|

// Define a service for demonstration purposes|
pub struct GreeterService;|

impl NamedService for GreeterService {|
    const NAME: &'static str = "greeter";|
}|

#[tokio::main]|
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {|
    // Initialize logging|
    tracing_subscriber::fmt()|
        .with_max_level(Level::DEBUG)|
        .init();|

    // Configure server address|
    let addr = SocketAddr::from(([127, 0, 0, 1], 8443));|

    let reflection_server = tonic_reflection::server::Builder::configure()|
        .register_encoded_file_descriptor_set(tonic_health::pb::FILE_DESCRIPTOR_SET)|
        .register_encoded_file_descriptor_set(tonic_types::pb::FILE_DESCRIPTOR_SET)|
        .register_encoded_file_descriptor_set(tonic_reflection::pb::v1alpha::FILE_DESCRIPTOR_SET)|
        .build_v1alpha()?;|

    // Set up gRPC health service|
    let (mut health_reporter, health_service) = tonic_health::server::health_reporter();|
    health_reporter.set_serving::<GreeterService>().await;|

    // Set up TCP listener|
    let listener = TcpListener::bind(addr).await?;|
    info!("Server listening on https://{}", addr);|
    let incoming = TcpListenerStream::new(listener);|

    // Create HTTP connection builder|
    let builder = HttpConnectionBuilder::new(TokioExecutor::new());|

    // Set up TLS configuration|
    rustls::crypto::aws_lc_rs::default_provider()|
        .install_default()|
        .expect("Failed to install rustls crypto provider");|

    let certs = load_certs("examples/sample.pem")?;|
    let key = load_private_key("examples/sample.rsa")?;|

    let mut config = ServerConfig::builder()|
        .with_no_client_auth()|
        .with_single_cert(certs, key)?;|

    // Configure ALPN protocols - required for gRPC|
    config.alpn_protocols = vec![b"h2".to_vec()];|
    let tls_config = Arc::new(config);|

    // Create a shutdown signal|
    let (shutdown_tx, shutdown_rx) = tokio::sync::oneshot::channel();|

    // Start server in the background|
    let server = tokio::spawn(async move {|
        info!("Server starting up...");|

        let svc = Server::builder()|
            .add_service(health_service)|
            .add_service(reflection_server)|
            .into_service()|
            .into_axum_router()|
            .into_service()|
            .boxed_clone();|

        let hyper_svc = TowerToHyperService::new(svc);|

        serve_http_with_shutdown(|
            hyper_svc,|
            incoming,|
            builder,|
            Some(tls_config),|
            Some(async {|
                shutdown_rx.await.ok();|
                info!("Shutdown signal received");|
            }),|
        )|
        .await|
        .expect("Server failed unexpectedly");|
    });|

    // Keep the main thread running until Ctrl+C|
    tokio::signal::ctrl_c().await?;|
    info!("Initiating graceful shutdown");|
    let _ = shutdown_tx.send(());|

    // Wait for server to shutdown|
    server.await?;|

    Ok(())|
}|

Performance Optimization

Performance is a first-class concern in postel. Our benchmarks show impressive results with microsecond latency for HTTP/2 requests and support for tens of thousands of concurrent connections. The library includes comprehensive benchmarks and flamegraph generation support to help users optimize their services.

Real-World Usage at Warlock Labs

At Warlock Labs, we use postel as the foundation for our gRPC services, handling everything from authentication to large file transfers. Some key use cases include:

• High-throughput data ingestion pipelines
• Low-latency API gateways
• WebSocket-based real-time services
• Mixed protocol services (HTTP/1.1, HTTP/2, gRPC)

Future Roadmap

We're actively developing postel with several exciting features planned:

Near-term

• Enhanced metrics and observability
• Additional protocol examples (WebSocket, HTTP/3)
• Extended middleware examples ecosystem

Long-term

• Kubernetes Gateway API integration
• Dynamic configuration management
• Advanced load balancing strategies

Installation

Add postel to your Cargo.toml:

[dependencies]|
postel = "0.7.1"|

Why Open Source?

At Warlock Labs, we believe in contributing back to the Rust ecosystem that has enabled our industry's success. postel represents our commitment to building robust, well-documented infrastructure components that benefit the entire community.

The challenges we solve in postel - high-performance networking, protocol interop, and graceful service management - are common across the industry. By open-sourcing our solution, we hope to:

• Reduce duplication of effort across teams
• Share our learning and best practices
• Engage with the broader Rust community
• Improve the library through diverse use cases

Contributing

postel follows the Rule of St. Benedict's principles in its community governance, emphasizing thoughtful communication, careful consideration of changes, and support for new contributors. We welcome contributions from the community and are committed to helping new contributors get started.

Get Started Today

We're excited to see how the community uses and extends postel. Whether you're building microservices, API gateways, or high-performance network applications, postel provides a solid foundation for your next project.

Check out our GitHub repository to get started, and don't hesitate to reach out with questions or feedback.

Written by the Warlock Labs Engineering Team