pickleDB · Lightning-Fast Key-Value Store for Python

Get Started in Seconds

pip install pickledb

Need the SQLite backend? Install the extra: pip install "pickledb[sqlite]"

Quick Start

Synchronous Example

from pickledb import PickleDB

# Bind to a JSON file; no I/O yet
db = PickleDB("data.json")
db.load()

db.set("username", "alice")
db.set("theme", {
    "color": "blue",
    "font": "sans-serif"
})

print(db.get("username"))  # → "alice"

db.save()  # atomically write to disk

Asynchronous Example

import asyncio
from pickledb import PickleDB

async def main():
    db = PickleDB("data.json")
    await db.load()

    await db.set("score", 42)
    value = await db.get("score")
    print(value)  # → 42

    await db.save()

asyncio.run(main())

Explicit I/O: The constructor PickleDB("data.json") never touches disk. load() and save() are explicit (or handled by context managers).

Design Philosophy

In-memory first: All data lives in memory while you work; JSON is used only for persistence
Method-only API: No dict syntax. Use set(), get(), remove(), etc.
Unified sync/async: Every core method works in both worlds via the same name
Atomic disk writes: Uses a temp file and os.replace to avoid partial writes
No hidden autosave: Nothing is written to disk unless you call save() or exit a context manager cleanly

⚠️ Thread & Process Safety: PickleDB (the default JSON-backed class) is not thread-safe or process-safe. Multiple threads or processes writing to the same JSON file will cause data corruption.

When you need concurrency on a single machine: Use PickleDBSQLite instead. SQLite provides file-level locking and ACID transactions, making it safe for concurrent access across threads and processes on the same host.

⚡ Performance trade-off: PickleDBSQLite is significantly slower than PickleDB because it reads/writes to disk on every operation instead of keeping everything in memory. Use it only when you need the safety guarantees.

For networked or multi-host workloads: Neither class is suitable. Consider Redis, PostgreSQL, MongoDB, or mongoKV.

Choosing the Right Backend

pickleDB offers two storage backends, each optimized for different use cases:

Feature	PickleDB (orJSON)	PickleDBSQLite
Storage Model	In-memory with JSON persistence	Disk-based SQLite with orjson encoding
Performance	Very fast (all operations in RAM)	Slower (disk I/O on every operation)
Thread Safety	❌ Not safe	✅ Safe (SQLite locking)
Process Safety	❌ Not safe (data corruption risk)	✅ Safe (file-level locking)
ACID Transactions	❌ No	✅ Yes
Memory Usage	High (entire DB in RAM)	Low (only active queries in RAM)
Best For	Single-threaded apps, scripts, fast prototypes	Multi-threaded/process apps, ASGI servers
Dependencies	`orjson` and `aiofiles`	`orjson`, `aiofiles` and `aiosqlite`

Rule of thumb: Use PickleDB (JSON) by default for maximum performance. Only switch to PickleDBSQLite when you need thread/process safety or ACID guarantees, and can accept the performance penalty.

Context Managers

Automatically load and save your database with context managers:

Synchronous Context

from pickledb import PickleDB

with PickleDB("data.json") as db:
    # On enter: db.load()
    db.set("foo", "bar")
    db.set("hello", "world")
    # On exit: db.save()

Asynchronous Context

import asyncio
from pickledb import PickleDB

async def main():
    async with PickleDB("data.json") as db:
        # On enter: await db.load()
        await db.set("foo", "bar")
        await db.set("hello", "world")
        # On exit: await db.save()

asyncio.run(main())

API Reference

pickleDB exposes two primary classes: PickleDB for JSON-on-disk, and PickleDBSQLite for an optional SQLite backend. All core methods share the same names in synchronous and asynchronous code — just add await when you're inside an async function.

class PickleDB(location: str)

In-memory JSON database backed by a single file on disk.

load() -> bool

sync async

Load (or reload) the JSON file at location into memory. If the file is missing or empty, the in-memory database becomes an empty dict.

Sync: db.load()
Async: await db.load()
Returns True on success.

save() -> bool

sync async

Atomically write the in-memory database to disk at location using a temporary file + os.replace().

Sync: db.save()
Async: await db.save()
Returns True on success.

set(key: Any, value: Any) -> bool

sync async

Store value under key in the in-memory database. Keys are coerced to str, and values must be JSON-serializable (via orjson).

Sync: db.set("name", "alice")
Async: await db.set("name", "alice")
Returns True after updating the in-memory store (does not write to disk until save()).

get(key: Any, default: Any | None = None) -> Any | None

sync async

Retrieve the value stored under key, or default if it doesn't exist.

Sync: db.get("name") or db.get("name", default="guest")
Async: await db.get("name")
Returns the stored value, or default (default is None).

remove(key: Any) -> bool

sync async

Remove key from the in-memory database.

Sync: db.remove("name")
Async: await db.remove("name")
Returns True if the key existed and was removed, False otherwise.

all() -> list[str]

sync async

Return a list of all keys currently stored in memory.

Sync: db.all()
Async: await db.all()
Returns a list of str keys. Order is not guaranteed.

purge() -> bool

sync async

Clear the in-memory database (equivalent to db.all() becoming an empty list). Does not touch disk until you call save().

Sync: db.purge()
Async: await db.purge()
Returns True.

class PickleDBSQLite(sqlite_path: str = "pickledb.sqlite3", table_name: str = "kv")

Optional SQLite-backed key-value store using the same sync/async method names.

Installation: PickleDBSQLite lives behind an optional dependency. Install aiosqlite yourself or use: pip install "pickledb[sqlite]".

Why use it? SQLite adds file-level locking and ACID transactions, so concurrent readers and serialized writers across multiple threads/processes on the same machine are handled for you. It's still not a distributed database, but it's much safer than multiple processes writing to a plain JSON file.

⚠️ Performance warning: Unlike PickleDB, this class does NOT store data in memory. Every read and write hits the disk, making it significantly slower. Only use PickleDBSQLite when thread/process safety is essential.

set(key: str | None, value: Any) -> str

sync async

Store value in a SQLite table as an orjson-encoded BLOB. If key is None, a new UUID key is generated and returned.

Sync: key = kv.set(None, {"foo": "bar"})
Async: key = await kv.set(None, {"foo": "bar"})
Returns the key used (either the provided string or the generated UUID).

get(key: str, default: Any = MISSING) -> Any

sync async

Look up a key in SQLite and deserialize it from orjson. If the key does not exist and no default is provided, a KeyError is raised.

Sync: kv.get(key) or kv.get("missing", default=None)
Async: await kv.get(key)
Returns the stored value, default, or raises KeyError.

remove(key: str) -> bool

sync async

Delete key from the SQLite table.

Sync: kv.remove("key")
Async: await kv.remove("key")
Returns True if a row was deleted, False otherwise.

all() -> list[str]

sync async

Return a list of all keys stored in the SQLite table.

Sync: kv.all()
Async: await kv.all()
Returns a list of key strings sorted by key.

purge() -> bool

sync async

Remove all rows from the SQLite table.

Sync: kv.purge()
Async: await kv.purge()
Returns True.

close() -> None

sync async

Close the underlying synchronous SQLite connection. In async code, returns a coroutine you should await.

Sync: kv.close()
Async: await kv.close()

Important: pickleDB is intentionally method-based only. Dict-style access like db["key"] or db["key"] = value is not supported.

Common Patterns

Storing Complex Data

# Store a dictionary
db.set("user", {"name": "Alice", "age": 30})

# Update it
user = db.get("user")
user["age"] += 1
db.set("user", user)

print(db.get("user"))
# {'name': 'Alice', 'age': 31}

Lists / Simple Queues

db.set("tasks", ["write", "test", "deploy"])

tasks = db.get("tasks", [])
tasks.append("celebrate")
db.set("tasks", tasks)

print(db.get("tasks"))
# ['write', 'test', 'deploy', 'celebrate']

Namespace Keys

db.set("user:1", {"name": "Alice"})
db.set("user:2", {"name": "Bob"})

def keys_with_prefix(db, prefix):
    return [k for k in db.all() if k.startswith(prefix)]

print(keys_with_prefix(db, "user:"))
# ['user:1', 'user:2']

Basic TTL Pattern

import time

def set_with_ttl(db, key, value, ttl_seconds):
    db.set(key, {
        "value": value,
        "expires_at": time.time() + ttl_seconds,
    })

def get_if_fresh(db, key):
    data = db.get(key)
    if not data:
        return None
    if time.time() < data.get("expires_at", 0):
        return data["value"]
    db.remove(key)
    return None

set_with_ttl(db, "session", "active", ttl_seconds=5)
time.sleep(3)
print(get_if_fresh(db, "session"))  # 'active'
time.sleep(3)
print(get_if_fresh(db, "session"))  # None

Performance Snapshot

Example timings loading, reading, and saving large JSON payloads using PickleDB (JSON) in async mode on a Dell XPS 9350, Ubuntu 24.04:

Entries	Load (into memory)	Bulk Read	Save (to disk)
1M	0.68 s	0.64 s	0.03 s
10M	7.48 s	7.27 s	0.22 s
50M	43.36 s	36.53 s	1.09 s

Benchmark Details: Full benchmark script available at this GitHub Gist.

PickleDBSQLite Performance: The SQLite backend is orders of magnitude slower than these numbers because it performs disk I/O on every operation. Use it only when you need thread/process safety.

When Not to Use pickleDB

PickleDB (JSON): Multi-threaded or multi-process applications (use PickleDBSQLite instead)
Both classes: ASGI web servers with multiple workers
Both classes: Your dataset is too large to comfortably fit in memory (JSON) or requires high-performance disk access (SQLite)
Both classes: You need distributed/multi-host concurrency
Both classes: You need rich querying, indexing, or joins

In those cases, consider Redis, PostgreSQL, MongoDB, or mongoKV.