Jelly RDF Binary Codec

The fastest RDF serialization format for Dart.
Streaming binary encoding based on Protocol Buffers — significantly smaller and faster than Turtle or N-Triples.

The Challenge

Text-based RDF formats like Turtle and N-Triples are human-readable, but they pay a steep performance price for that readability.

Text Formats Are Slow

  • Every IRI is repeated in full on every triple
  • UTF-8 parsing is CPU-intensive at scale
  • No streaming: must buffer the entire document
  • Large output size means more I/O overhead

Jelly Solves This

  • IRI lookup tables — referenced by compact integer ID
  • Protobuf wire format — minimal parsing overhead
  • Frame-level streaming — process data as it arrives
  • 75% of Turtle's output size on real datasets

On the large benchmark (17.2k triples), Jelly encodes in 81% of Turtle's time and decodes in just 7% of Turtle's time, while producing output that is 25% smaller. See the full benchmark results.

The Solution

A complete Dart implementation of the Jelly RDF specification — batch and streaming, graphs and datasets.

Batch API — drop-in replacement for text codecs

import 'package:locorda_rdf_jelly/jelly.dart';

// Encode a graph to compact binary Jelly format
final bytes = jellyGraph.encode(graph);

// Decode back
final decoded = jellyGraph.decode(bytes);

✓ Same API as Turtle and other codecs
✓ Plugs into RdfCore for content-type dispatch
✓ 82 conformance tests pass (RDF 1.1)

Getting Started

Install the package

dart pub add locorda_rdf_jelly locorda_rdf_core
// Quick start: batch encode/decode with the pre-configured global codecs
import 'dart:typed_data';
import 'package:locorda_rdf_core/core.dart';
import 'package:locorda_rdf_jelly/jelly.dart';

void main() {
  // Build a small graph
  final graph = RdfGraph(triples: [
    Triple(
      IriTerm('http://example.org/alice'),
      IriTerm('http://xmlns.com/foaf/0.1/name'),
      LiteralTerm.string('Alice'),
    ),
    Triple(
      IriTerm('http://example.org/alice'),
      IriTerm('http://xmlns.com/foaf/0.1/knows'),
      IriTerm('http://example.org/bob'),
    ),
  ]);

  // Encode to compact binary Jelly format
  final Uint8List bytes = jellyGraph.encode(graph);
  print('Encoded ${bytes.length} bytes (vs ~200 bytes as Turtle)');

  // Decode back
  final decoded = jellyGraph.decode(bytes);
  print('Decoded ${decoded.size} triples');
}
// Frame-level streaming — encode and decode a stream of triple batches
import 'dart:async';
import 'package:locorda_rdf_core/core.dart';
import 'package:locorda_rdf_jelly/jelly.dart';

Future<void> main() async {
  // A stream of triple batches (e.g. from a database or file in pages)
  final Stream<Iterable<Triple>> triplePages = Stream.fromIterable([
    [
      Triple(
        IriTerm('http://example.org/s1'),
        IriTerm('http://example.org/p'),
        LiteralTerm.string('first batch'),
      ),
    ],
    [
      Triple(
        IriTerm('http://example.org/s2'),
        IriTerm('http://example.org/p'),
        LiteralTerm.string('second batch'),
      ),
    ],
  ]);

  // Encode — lookup tables are shared across frames for better compression
  final encodedStream = JellyTripleFrameEncoder().bind(triplePages);

  // Collect the encoded frames
  final frames = await encodedStream.toList();
  print('Encoded ${frames.length} Jelly frames');

  // Decode — each frame emits a List<Triple>
  final byteStream = Stream.fromIterable(frames);
  final decoded =
      JellyTripleFrameDecoder().bind(byteStream).expand((frame) => frame);

  final triples = await decoded.toList();
  print('Decoded ${triples.length} triples across all frames');
}
// Integration with RdfCore for content-type-based dispatch
import 'package:locorda_rdf_core/core.dart';
import 'package:locorda_rdf_jelly/jelly.dart';

void main() {
  // Register Jelly alongside the built-in text codecs
  final rdfCore = RdfCore.withStandardCodecs(
    additionalBinaryGraphCodecs: [jellyGraph],
    additionalBinaryDatasetCodecs: [jelly],
  );

  // Suppose we already have a graph (e.g. parsed from Turtle)
  final graph = RdfGraph(triples: [
    Triple(
      IriTerm('http://example.org/s'),
      IriTerm('http://example.org/p'),
      LiteralTerm.string('hello'),
    ),
  ]);

  // Encode to Jelly via content-type dispatch
  final bytes = rdfCore.encodeBinary(graph, contentType: jellyMimeType);
  print('Jelly bytes: ${bytes.length}');

  // Decode back — codec selection is automatic
  final decoded = rdfCore.decodeBinary(bytes, contentType: jellyMimeType);
  print('Decoded ${decoded.size} triples');
}

Key Features

⚡ Fastest in the Suite

Direct protobuf wire-format writing — no intermediate GeneratedMessage allocation on either encode or decode. O(1) LRU lookup tables, IRI term caching, and repeated-term delta encoding compound to give Jelly its lead. See the benchmarks.

🌊 Frame-Level Streaming

JellyTripleFrameEncoder / JellyTripleFrameDecoder (and quad equivalents) are idiomatic StreamTransformerBase instances. Compose with .bind() and .expand() just like any other Dart stream transformer.

🔗 Cross-Frame Table Sharing

In streaming mode, lookup tables accumulate across frames. IRIs and datatypes seen in early frames are reused in later ones — giving better compression for continuous streams than independent per-frame encoding.

📊 Graphs & Datasets

Batch and streaming APIs for both single graphs (TRIPLES physical type) and full datasets (QUADS or GRAPHS physical type). Named graph boundaries are preserved in GRAPHS mode.

✅ Conformance Tested

82 official Jelly-RDF conformance tests pass (51 decode + 31 encode), executed via the jelly-protobuf test suite. All RDF 1.1 test cases pass.

🔌 RdfCore Plugin

Register JellyGraphCodec and JellyDatasetCodec with RdfCore.withStandardCodecs for unified content-type dispatch alongside Turtle, JSON-LD, and RDF/XML.

Use Cases

📡 High-Throughput Pipelines

Process millions of triples with minimal CPU and memory overhead. Jelly's binary format eliminates the bottlenecks of text parsing at scale.

💾 Compact Storage

Store RDF graphs in databases or files with up to 25% less space than Turtle. Lookup-table compression is especially effective for datasets with repeated IRIs.

🌊 Streaming Ingestion

Feed triples into a consumer as they are decoded, without buffering the entire dataset. Frame-level streaming keeps end-to-end latency low.

Ready to Go Fast?

Read the documentation or explore other RDF packages.