spvn

Rust ASGI bindings for python.

Features

Drop and go CLI for replacing transport layer handling
ASGI compliant protocols allowing direct import and use of python through rust
Fast

Sneak Peek

fastapi before uvicorn-fastapi-5000 fastapi after spvn-fastapi-5000

p99 @ 38x

Rationale & Goals

Relieve limits by python in networking applications
Safe python threadpooling
Uvicorn drops requests & stalls on IO > 7500 concurrent clients
Hypercorn drops requests & stalls on IO > 7500 concurrent clients

In both, we must horizontally scale to accomodate these limits in our systems. This is further accompanied by essentially a second layer of IO bound processes, which are evidently unable to maintain highly concurrent environments

What spvn does

Delegation of connection multiplex, stream, and IO processes into Rust, and autoinjection at runtime following standard ASGI protocol.

Performance

Tests performed using ali. See ali for methodology.

spvn @ 1000 reqs/s, 30s sustained

spvn-5000

app: fastapi
file: dotest.bit:app

spvn @ 5000 reqs/s

spvn-5000

app: custom
file: dotest.baz:app

spvn @ 10000 reqs/s

spvn-5000

app: custom
file: dotest.baz:app

uvicorn @ 1000 reqs/s

uvicorn-1000

app: custom
file: dotest.baz:app

uvicorn @ 5000 reqs/s (DDOS Success)

uvicorn-5000

app: custom
file: dotest.baz:app

hypercorn @ 1000 reqs/s

hypercorn-5000

app: custom
file: dotest.baz:app

hypercorn @ 5000 reqs/s (DDOS Success)

hypercorn-5000

app: custom
file: dotest.baz:app

Installation

spvn is offered as beta currently, keep in mind it has quirks and some features might be untested. for this reason, we dont recommend it for production use yet. if you find a bug, please submit an issue!

pypi

pip install spvn

crates

spvn
spvn_caller
spvn_lifespan

Developing

Project Status

Roughly in order of priority

[✅] Integrate standard import semantics
[✅] PyCaller
- [✅] (rust) Async safe integration
- [✅] Abstract (py fn) async / sync handle
- [✅] Caller pool [this will be revised, its too slow]
[✅] Standard asgi traits & structs
- [✅] ASGIScope
  - [✅] (rust) Async safe integration
  - [✅] Conversion from tower::Body -> dict
- [✅] ASGIVersion
- [✅] ASGIMessage
  - [✅] Lifecycle Scope
  - [✅] HTTP Lifecycle Scope
  - [🚧] Websockets (msg integration)
[✅] App listener
[✅] App dispatcher
- [✅] Async threadsafe
- [🚧] Lifecycle activation for caller objects (unstable)
[🚧] App scheduler
- [✅] Injectable awaitables (rust ptr -> python ptr)
- [✅] Async threadsafe
- [✅] Delayed py-fn call
- [🚧] Scheduler into py
[🚧] Live reloader
[🚧] Websockets

Pre-requisites

Python >= 3.9

Use virtualenv / venv

python3.10 -m (venv|virtualenv) env && \
        . ./env/bin/activate && \
        pip install maturin

Test bindings by running

maturin develop

Rust >= 1.69.0

Build CLI

cargo build

Run CLI

cargo cli serve dotest.foo:app

Configuration

Serve

Usage: spvn serve [OPTIONS] <py import>

Arguments:
  <py import>

Options:
      --bind <BIND> (bind target, ip:port format)
      --n-threads <N_THREADS> (concurrent servers)
      --cpu (use number of cpus for concurrency)
  -w, --watch (watch & reload, not implemented)
  -v, --verbose                        [env: SPVN_VERBOSE_PROC=] (enable logging, approx 75ms overhead)
      --ssl-cert-file <SSL_CERT_FILE>  [env: SPVN_SSL_CERT_FILE=] (enable tls)
      --ssl-key-file <SSL_KEY_FILE>    [env: SPVN_SSL_KEY_FILE=] (enable tls)
      --user <USER> (user, unimplemented)
      --proc-dir <PROC_DIR>            [env: PROC_DIR=] (process directory, important for FD limits on unix, unimplemented)
  -l, --lifespan (lifespan, unstable)
  -h, --help                           Print help

Preliminary Performance

Tests performed using ali.

Methodology

Request with 4 byte payload, and common ASGI app executing the following:


async def app(scope, receive, send):
    await receive() # added: receive the full payload
    await send({
        'type': 'http.response.start',
        'status': 200,
        'headers': [
            (b'content-type', b'text/plain'),
        ],
    }) # send a start callback
    await send({
        'type': 'http.response.body',
        'body': b'Hello, world!',
    }) # send a body

modified from uvicorn docs

Execution

50 req / s

ali http://127.0.0.1:8000 -b body --rate=50

500 req / s

ali http://127.0.0.1:8000 -b body --rate=500

1000 req / s

ali http://127.0.0.1:8000 -b body --rate=1000

5000 req / s

ali http://127.0.0.1:8000 -b body --rate=5000

Commands

uvicorn

uvicorn dotest.baz:app

hypercorn

hypercorn dotest.baz:app

spvn

spvn serve --target dotest.baz:app

Results

hypercorn-50 hypercorn @ 50 reqs/s

hypercorn-500 hypercorn @ 500 reqs/s

hypercorn-1000 hypercorn @ 1000 reqs/s

hypercorn-5000 hypercorn @ 5000 reqs/s (crash / ddos thresh)

uvicorn-50 uvicorn @ 50 reqs/s

uvicorn-500 uvicorn @ 500 reqs/s

uvicorn-1000 uvicorn @ 1000 reqs/s

uvicorn-5000 uvicorn @ 5000 reqs/s (crash / ddos thresh)

spvn-50 spvn @ 50 reqs/s

spvn-500 spvn @ 500 reqs/s

spvn-1000 spvn @ 1000 reqs/s

spvn-5000 spvn @ 5000 reqs/s

spvn-10000 spvn @ 10000 reqs/s (dropped requests but continued service <130ms P95)

POST Test

Source: https://github.com/serde-rs/json-benchmark/blob/master/data/canada.json