A Safe pickle Alternative: Serialize Python with Type Hints

Why CocoIndex replaced pickle with serialization that reads Python type hints to choose msgspec, Pydantic or restricted pickle for each value.

EngineeringUpdated 10 min read
A safe pickle alternative from your type hints. A type hint such as -> list[float] picks the engine; an arrow points to 3 routing-byte rows: 0x01 msgspec (msgpack, the default), 0x02 Pydantic (validated on load) and 0x80 pickle (opt-in, restricted).
Enlarge cover

A Safe pickle Alternative: Serialize Python with Type Hints

Zoom in to see details.

A safe pickle alternative from your type hints. A type hint such as -> list[float] picks the engine; an arrow points to 3 routing-byte rows: 0x01 msgspec (msgpack, the default), 0x02 Pydantic (validated on load) and 0x80 pickle (opt-in, restricted).

Python’s built-in pickle can run code while it reads data back: a crafted payload executes arbitrary Python on the machine that loads it. It is also large: the integer 1024 takes 15 bytes in pickle and 3 bytes in msgpack. Our first fix for the security problem, an allowlist, needed 9 decorators and 4 rounds of run-fail-annotate in one of our own examples. This post explains how CocoIndex moved to serialization guided by type hints, and the union-type edge cases that remain.

CocoIndex is a framework for building incremental data pipelines. It has a Rust core for performance and exposes a Python SDK for users to define their pipelines. Under the hood, the engine needs to serialize and deserialize Python objects constantly (serde for short: turning an object into bytes and back). It does this to cache function results, persist pipeline state and track records for change detection. The serialized data crosses the Rust/Python boundary and is stored by the Rust core.

In CocoIndex v0, we had a closed type system: a fixed set of supported data types, with serialization handled entirely in Rust. It was safe and fast but rigid: users could not use their own data types without converting them into our type system first.

When we started building v1, we wanted to natively support any Python data type. Pickle was the first choice.

Three serialization designs side by side. Pickle, the prototype: handles almost any object, runs code while it deserializes, and encodes 1024 in 15 bytes. Allowlisted pickle, the security fix: rejects types not on the list, and needed 9 @coco.unpickle_safe decorators and 4 fix rounds. Type-guided serde, used in production and outlined in periwinkle: type hints pick the engine, 0 decorators in the example, and 1024 in 3 bytes.
Three serialization designsEnlarge

Three serialization designs

Zoom in to see details.

Three serialization designs side by side. Pickle, the prototype: handles almost any object, runs code while it deserializes, and encodes 1024 in 15 bytes. Allowlisted pickle, the security fix: rejects types not on the list, and needed 9 @coco.unpickle_safe decorators and 4 fix rounds. Type-guided serde, used in production and outlined in periwinkle: type hints pick the engine, 0 decorators in the example, and 1024 in 3 bytes.

Pickle runs code on load and wastes bytes

Pickle is Python’s built-in serialization. It handles almost any Python object out of the box: dataclasses, NamedTuples, Pydantic models, nested containers and custom classes. For early prototyping under prerelease, it needed one line of code.

Pickle has two problems that make it unsuitable for production.

Security. Pickle executes arbitrary code during deserialization. A crafted payload can run any Python code on your machine, for example os.system(...). In CocoIndex’s case, the serialized data lives in internal storage (a database managed by the framework), so the risk requires an attacker to have access to that storage, which is unlikely for most deployments. Defense in depth (more than one layer of protection) still matters: if the storage is ever exposed through a misconfiguration or a compromised database, pickle becomes an escalation path from data access to code execution. We did not want that risk in the foundation of our persistence layer.

Pickle escalation path in 3 steps. 01 write: an attacker with access to internal storage stores a crafted payload. 02 read: the framework loads its cache with pickle.loads(blob), a routine read with no validation. 03 execute, outlined in white: the blob runs as Python code, such as os.system(...) shown in red, on the host. A note says a format that cannot run code stops step 03 even if step 01 happens.
Pickle escalation pathEnlarge

Pickle escalation path

Zoom in to see details.

Pickle escalation path in 3 steps. 01 write: an attacker with access to internal storage stores a crafted payload. 02 read: the framework loads its cache with pickle.loads(blob), a routine read with no validation. 03 execute, outlined in white: the blob runs as Python code, such as os.system(...) shown in red, on the host. A note says a format that cannot run code stops step 03 even if step 01 happens.

Overhead. Pickle encodes type information into every value: each value is a sequence of opcodes that tells the deserializer what type to reconstruct and how. This makes pickle self-describing, at a cost. The integer 1024 takes 15 bytes in pickle, while msgpack (a compact binary format similar to JSON) encodes it in 3. Msgpack can be this compact because it relies on the deserializer already knowing the expected type:

ValuepicklemsgpackRatio
"" (empty string)15 bytes1 byte15x
102415 bytes3 bytes5x
"hello"20 bytes6 bytes3.3x
{"key": [1,2,3]}31 bytes9 bytes3.4x
Point(42, "origin")63 bytes16 bytes3.9x

For a pipeline that caches thousands of intermediate results, this overhead adds up.

An allowlist fixed security but added decorators

Our first attempt to fix the security problem was a restricted unpickler: a pickle reader that refuses any class that is not on a list. We maintained an allowlist of types that were safe to deserialize, and rejected everything else. Users could opt their types into the allowlist with a @coco.unpickle_safe decorator:

python
@coco.unpickle_safe
@dataclass
class DocumentChunk:
    text: str
    embedding: list[float]

This solved the security problem and introduced a new one: annotation burden.

Every type that appeared anywhere in a serialized object graph needed the decorator: the top-level type and every type nested inside it. A missed type causes a runtime error.

What you see. When a type is missing from the allowlist, the read fails with an error such as _pickle.UnpicklingError: Forbidden global during unpickling: myapp.models.Summary. You add the decorator, run again, and meet the next missing type.

We experienced this when building the conversation_to_knowledge example. It needed 9 @coco.unpickle_safe annotations across the codebase. Getting there took 4 rounds of run-fail-annotate-repeat: run the pipeline, hit an error about an unregistered type, add the decorator, run again, hit the next error.

That was our own example. A user pipeline with dozens of model types needs the decorator on each of them.

Type hints tell the decoder what to build

We looked for something that already describes the shape of users’ data, and found it in their code: type hints, the annotations such as -> list[float] that name the type of a value.

In CocoIndex, there are 3 places where data gets serialized and deserialized: memoized function return values (the most common case), memo state values for change detection, and target tracking records. A memoized function is one whose results CocoIndex caches, so that unchanged work can be skipped on later runs. For memoized return values, the type hint is in the function signature:

python
@coco.fn(memo=True)
async def embed_chunk(chunk: str) -> list[float]:
    return await embedding_model.embed(chunk)

The return type list[float] tells us exactly how to deserialize the cached result. No extra annotation is needed: the type hint that users already write for readability and IDE support doubles as the deserialization schema.

What you see. When embed_chunk is called again with the same chunk and its code has not changed, CocoIndex returns the cached list[float] instead of calling the embedding model. If a memoized function has no return annotation, cached values may come back as basic Python types such as dict or list instead of the original type.

This led us to a type-guided serialization system built on msgspec, a fast msgpack-based serialization library for Python. Msgspec natively handles dataclasses, NamedTuples, and standard types (str, int, list, dict, datetime, UUID, etc.), and it does so by relying on type information at deserialization time.

One routing byte picks the engine

Different types need different serialization engines. We use a single routing byte at the start of each payload to select the engine:

ByteEngineUsed for
0x01msgspec (msgpack)Dataclasses, NamedTuples, primitives, collections
0x02PydanticBaseModel subclasses
0x80PickleOpted-in types, legacy data

Serialization checks types in priority order: explicit pickle opt-in first (the user specifically requested it), then Pydantic models, then msgspec as the default. Deserialization reads the routing byte and dispatches accordingly.

Writing a value checks its type in order. 1, a type opted in to pickle, gets byte 0x80 and is pickled and read back by a restricted unpickler. 2, a Pydantic BaseModel, gets byte 0x02 and is written with model_dump(mode="json") into msgpack. 3, anything else, gets byte 0x01 and goes to msgspec (msgpack), the default. Reading goes the other way: the first byte selects the decoder, and msgspec and Pydantic decode against the type hint.
The routing byteEnlarge

The routing byte

Zoom in to see details.

Writing a value checks its type in order. 1, a type opted in to pickle, gets byte 0x80 and is pickled and read back by a restricted unpickler. 2, a Pydantic BaseModel, gets byte 0x02 and is written with model_dump(mode="json") into msgpack. 3, anything else, gets byte 0x01 and goes to msgspec (msgpack), the default. Reading goes the other way: the first byte selects the decoder, and msgspec and Pydantic decode against the type hint.

For Pydantic models, we serialize via model_dump(mode="json") into msgpack, and deserialize via TypeAdapter.validate_python(). This avoids pickle entirely while preserving Pydantic’s validation semantics.

For types that msgspec can’t handle natively, like numpy arrays or pathlib paths, CocoIndex uses pickle. These types get the pickle routing byte (0x80) and are still read through a restricted unpickler. Users opt their own types in with @coco.serialize_by_pickle. Current CocoIndex registers numpy arrays and pathlib paths itself, so they need no decorator; see the supported types.

Quick check: a memoized function returns a Pydantic BaseModel. Which routing byte does its cached payload start with, and how is it read back?

0x02. The model is written with model_dump(mode="json") into msgpack and read back with TypeAdapter.validate_python() for the function’s return type, so Pydantic’s validation runs on the way in.

What changed for users

The conversation_to_knowledge example dropped all 9 @coco.unpickle_safe annotations. Users define their data types with standard Python type hints, and serialization works without extra code:

python
@dataclass
class DocumentChunk:
    text: str
    embedding: list[float]
    metadata: dict[str, str]

No decorator or registration step is needed, so the run-fail-annotate cycle is gone. The framework reads the type hint and picks the serializer and deserializer.

Quick check: DocumentChunk is a dataclass nested inside another dataclass. Should you add @coco.serialize_by_pickle to it?

No. Dataclasses, NamedTuples and msgspec.Struct types are supported natively. When such a type is nested inside another supported type, msgspec encodes it directly and the decorator has no effect. If one field of the dataclass has an unsupported type, register that field’s type instead. See custom types.

Ambiguous unions do not round-trip

This approach does not round-trip every type. There are edge cases with union types (a field that accepts one of several types, such as date | str) that users should be aware of. They are not specific to CocoIndex: they come from how msgspec and Pydantic handle ambiguous types.

msgspec: loud failure

msgspec rejects ambiguous unions at construction time:

python
import msgspec.msgpack

# These all raise TypeError immediately:
msgspec.msgpack.Decoder(str | datetime.date)     # two str-like types
msgspec.msgpack.Decoder(list[int] | tuple[int, ...])  # two array-like types
msgspec.msgpack.Decoder(bytes | bytearray)        # two bytes-like types

This is the safer behavior: msgspec tells you up front that these types can’t be distinguished on the wire, rather than silently corrupting your data.

What you see in CocoIndex. When a memoized function’s return type contains such a union, reading its cached value fails with a DeserializationError. The message names the type hint and the function (source=return type of <function>()), and its cause is Cannot build msgspec Decoder for ....

Pydantic: silent surprise

Pydantic is more permissive: it accepts date | str, but the behavior after a serialization-deserialization round-trip can be surprising.

Consider a Pydantic model with a date | str field:

python
class Event(BaseModel):
    ts: datetime.date | str

e = Event(ts=datetime.date(2024, 1, 15))

If you serialize this to JSON (or msgpack), date becomes the string "2024-01-15". On deserialization, Pydantic sees a string input and needs to decide: is it a str or a date?

With Pydantic v2’s default smart mode, the answer is always str. Smart mode scores candidates by exactness: a string input is an exact match for str but only a lax match (requires coercion) for date. Exact wins. Ordering doesn’t matter: date | str and str | date both produce str.

What you see. The round-tripped Event has ts == "2024-01-15", a str, where the original had datetime.date(2024, 1, 15). No error is raised.

Left-to-right mode behaves differently. With date | str ordering, Pydantic tries date first, the lax coercion from "2024-01-15" succeeds, and you get your date back:

python
from pydantic import Field
from typing import Annotated

class Event(BaseModel):
    # Left-to-right mode: more specific type first
    ts: Annotated[datetime.date | str, Field(union_mode='left_to_right')]

This requires users to explicitly opt in to left-to-right mode and put the more specific type first.

A date | str field after a round-trip. The stored value date(2024, 1, 15) travels on the wire as the string "2024-01-15". Pydantic smart mode, the default, returns the string "2024-01-15" (str), outlined in red. Left_to_right mode with date first returns date(2024, 1, 15), outlined in green. A note says msgspec rejects str | datetime.date when the decoder is built, instead of guessing.
Union round-tripEnlarge

Union round-trip

Zoom in to see details.

A date | str field after a round-trip. The stored value date(2024, 1, 15) travels on the wire as the string "2024-01-15". Pydantic smart mode, the default, returns the string "2024-01-15" (str), outlined in red. Left_to_right mode with date first returns date(2024, 1, 15), outlined in green. A note says msgspec rejects str | datetime.date when the decoder is built, instead of guessing.

The wire format erases the type

The cause is the wire format (the bytes as stored), and it applies to both libraries. Msgpack (like JSON) has no date type: dates are serialized as strings. Once the type information is erased on the wire, no decoder can reliably recover it when multiple types map to the same wire representation.

The practical advice: avoid union types where members share a wire representation in data objects that go through serialization. Use unambiguous types, or use Pydantic’s discriminated unions with a literal tag field.

Quick check: a Pydantic model stores created: datetime.date | str with default settings. You cache date(2024, 1, 15). What type comes back?

str: the value is "2024-01-15". Smart mode prefers the exact match for a string input, whatever the order of the union. To get a date back, use one type for the field, a discriminated union with a literal tag, or union_mode='left_to_right' with date listed first.

What we learned

Type hints as a source of truth. Python developers already annotate their data types. Using those annotations for serialization adds no new concepts, decorators or registration APIs.

Ambiguity reported early. msgspec rejects ambiguous unions at construction time, which is better than silently producing wrong results. A serialization layer should fail early and clearly.

The wire format sets the limit. If the wire format can’t distinguish two types, your serialization layer can’t either. Design your data types around what the serialization format can express.

A staged design. Pickle was right for prototyping. The allowlisted pickle was right for the security fix. The type-guided approach was right for production. Each step taught us what the next step needed to be.

What to try next

  • Annotate memoized return types. Give each @coco.fn(memo=True) function a return type such as -> list[float] or -> DocumentChunk, so cached values come back as that type.
  • Check unions in cached data. Look for fields such as date | str whose members share a wire representation. Use one type, a Pydantic discriminated union with a literal tag field, or tagged msgspec.Struct variants (tag=True); the union types section shows the pattern.
  • Register types outside the supported list. Decorate your own class with @coco.serialize_by_pickle, or call coco.serialize_by_pickle(SomeType) for a third-party type.
  • If an upgrade shows Forbidden global during unpickling, older cached data references an unregistered type. Add @coco.serialize_by_pickle to it, or for a dataclass or NamedTuple add @coco.unpickle_safe until the cache is rebuilt; see troubleshooting.

Further reading:

Frequently asked questions

Is Python's pickle safe to use? What are the security risks?

No. Pickle is unsafe for data you do not fully control. Pickle executes arbitrary code during deserialization: a crafted payload can run any Python code on the host. Even when the serialized data lives in internal storage that is not directly user-facing, defense in depth matters. A misconfiguration or a compromised database can turn data access into code execution.

This is the reason CocoIndex moved off pickle as the default persistence format. See Pickle runs code on load and wastes bytes.

What are good alternatives to pickle in Python?

For typed data (dataclasses, NamedTuples, primitives, lists, dicts, datetimes, UUIDs), use msgspec, a fast msgpack-based library that uses type information at deserialization time. For Pydantic BaseModel subclasses, dump via model_dump(mode="json") into msgpack and reload via TypeAdapter.validate_python(). Reserve pickle for types that neither library handles natively (e.g. numpy arrays, pathlib paths) and route them through a restricted unpickler.

This is the layered approach CocoIndex landed on. See One routing byte picks the engine.

What is type-guided deserialization?

Type-guided deserialization uses the type hints already present in user code as the schema for serialization. Instead of asking developers to register or decorate their classes, the framework reads the type annotation (for example, the return type of a memoized function) and selects a deserializer that matches that shape. The wire format does not need to embed type metadata for every value because the consumer already knows the expected type.

See Type hints tell the decoder what to build.

Can Python type hints be used for serialization?

Yes. Libraries like msgspec rely on type hints at both serialization and deserialization time, so a single annotation, for example list[float] on a function's return type, doubles as the schema. CocoIndex uses the function signature directly: a memoized async function annotated with -> list[float] tells the framework exactly how to decode the cached result.

See Type hints tell the decoder what to build.

How do I safely deserialize untrusted data in Python?

Pick a format that does not execute code. msgspec (msgpack) and Pydantic's JSON pipeline are both safe by construction: they reconstruct values according to a declared schema, never by running pickle opcodes. If you must accept pickle for a small set of opted-in types, gate it behind a restricted unpickler that allowlists exactly the classes you expect; reject everything else.

CocoIndex's escape-hatch design keeps pickle behind both an explicit opt-in (@coco.serialize_by_pickle) and a restricted unpickler. See One routing byte picks the engine.

How do I replace pickle with msgpack or msgspec?

Declare your data shape with standard Python type hints, such as a @dataclass, a NamedTuple, or primitives, and let msgspec serialize and deserialize against that type. msgspec encodes values much more compactly than pickle because the deserializer already knows the type: 1024 takes 3 bytes in msgpack vs. 15 bytes in pickle, an empty string 1 byte vs. 15. For pipelines that cache thousands of intermediate results, that adds up quickly.

See the size comparison and rationale in Pickle runs code on load and wastes bytes.

How do I serialize dataclasses and Pydantic models together in one system?

Use a single-byte routing prefix at the start of every payload to select the engine, and dispatch by type at serialization time:

  • 0x01: msgspec (msgpack) for dataclasses, NamedTuples, primitives, and collections.
  • 0x02: Pydantic for BaseModel subclasses (dump via model_dump(mode="json"), reload via TypeAdapter.validate_python()).
  • 0x80: pickle for opted-in types and legacy data, behind a restricted unpickler.

Resolve types in priority order: explicit pickle opt-in first, then Pydantic, then msgspec as the default. On the read path, look at the routing byte and call the matching engine. See One routing byte picks the engine.

How do I build a restricted (whitelisted) unpickler in Python?

Subclass pickle.Unpickler, override find_class, and refuse any class that is not on a known allowlist. In CocoIndex's first iteration, types opted in by way of a @coco.unpickle_safe decorator and the unpickler raised on anything missing.

The catch: every nested type must also be allowlisted, not just the top-level one. CocoIndex's own conversation_to_knowledge example needed 9 annotations spread across the codebase, found over 4 rounds of run-fail-annotate. That annotation burden is what eventually pushed the framework toward type-guided dispatch. See An allowlist fixed security but added decorators.

How do I choose a serializer automatically based on a function's return type?

Inspect the function's return annotation, pick the engine that matches that type, and write a routing byte at the start of the payload so deserialization can reverse the decision. CocoIndex's resolver checks types in priority order: explicit pickle opt-in → Pydantic BaseModel → msgspec for everything else. For memoized return values, the return-type annotation in the function signature is the only metadata required. No decorators or registration are needed.

See Type hints tell the decoder what to build and What changed for users.

What are the edge cases when using type hints for serialization?

Union types whose members share a wire representation are the main hazard. msgspec rejects ambiguous unions like str | datetime.date or list[int] | tuple[int, ...] at construction time, which is the safer behavior. Pydantic accepts them but, in default smart mode, prefers the exact match, so a round-tripped date field typed as date | str comes back as str, regardless of declaration order. Left-to-right mode with the more specific type first restores the date, but it must be opted into explicitly.

The underlying constraint is the wire format: msgpack and JSON have no native date type, so once a date is on the wire it is a string. Prefer unambiguous types or Pydantic discriminated unions with a literal tag. See Ambiguous unions do not round-trip.

Build fresh context for your agents.

CocoIndex keeps derived data in sync with its sources. Only what changed re-runs.