Skip to content

protobufjs/protobuf.js

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1,079 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

protobuf.js
protobuf.js

Protocol Buffers are a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more, originally designed at Google (see).

protobuf.js is a standalone JavaScript implementation of Protocol Buffers for Node.js and browsers. It is tuned for fast binary I/O, battle-tested at scale, can load .proto files directly, and supports runtime reflection as well as static code generation with strong TypeScript declarations.

If protobuf.js is important to your project or organization, especially if you depend on it commercially, consider supporting its ongoing maintenance.

Getting started

Install

npm install protobufjs

The command line utility for generating reflection bundles, static code and TypeScript declarations is published as an add-on package:

npm install --save-dev protobufjs-cli

The CLI is a JS-native protobuf.js toolchain that does not require setting up protoc. If you prefer a protoc-based workflow, it provides protoc-gen-pbjs as an option.

Browser builds

Canonical browser builds for each runtime variant are provided via the jsDelivr CDN, supporting CommonJS, AMD and global window.protobuf. Make sure to pin an exact version in production.

Usage

The examples below use this schema:

syntax = "proto3";

package awesomepackage;

message AwesomeMessage {
  string awesome_field = 1;
}

protobuf.js converts .proto field names to camelCase by default, so awesome_field is used as awesomeField in JavaScript. Use the keepCase option when loading or parsing .proto files to preserve field names as written.

Load a schema

const protobuf = require("protobufjs");

const root = await protobuf.load("awesome.proto");
const AwesomeMessage = root.lookupType("awesomepackage.AwesomeMessage");

Optionally use load() with a callback, or loadSync() for synchronous loading on Node.js.

Encode and decode

const payload = { awesomeField: "hello" };

// Optionally create a message instance from already valid data
const message = AwesomeMessage.create(payload);

const encoded = AwesomeMessage.encode(message).finish();
const decoded = AwesomeMessage.decode(encoded);

encode expects a message instance or equivalent plain object and does not verify input implicitly. Use create to create a message instance from already valid data when useful, verify for plain objects whose shape is not guaranteed, and fromObject when conversion from broader JavaScript objects is needed.

Plain objects can be encoded directly when they already use protobuf.js runtime types: numbers for 32-bit numeric fields, booleans for bool, strings for string, Uint8Array or Buffer for bytes, arrays for repeated fields, and plain objects for maps. Map keys are the string representation of the respective value or an 8-character hash string for 64-bit keys.

Unknown fields present on the wire are preserved by default in message.$unknowns and forwarded when the message is re-encoded. Unknown field data can be dropped from a decoded message with delete message.$unknowns, discarded during decode with reader.discardUnknown = true per reader, or disabled by default for subsequently created readers with Reader.discardUnknown = true.

Convert plain objects

Conversion is an explicit interoperability boundary. fromObject accepts common JavaScript inputs such as enum values by name, base64 bytes, decimal 64-bit strings, Long, and BigInt; toObject lets callers choose the output expected by their application or transport.

const message = AwesomeMessage.fromObject({ awesomeField: 42 });
const object = AwesomeMessage.toObject(message, {
  longs: String,
  enums: String,
  bytes: String
});

Common ConversionOptions are:

Option Effect
longs: BigInt Converts 64-bit values to bigint values
longs: String Converts 64-bit values to decimal strings
longs: Number Converts 64-bit values to JS numbers (may lose precision)
enums: String Converts enum values to names
bytes: String Converts bytes to base64 strings
defaults: true Includes default values for unset fields
arrays: true Includes empty arrays for repeated fields
objects: true Includes empty objects for map fields
oneofs: true Includes virtual oneof discriminator properties

Message API

Message types expose focused methods for validation, conversion, and binary I/O.

  • encode(message: Message | object, writer?: Writer): Writer
    Encodes a message or equivalent plain object. Call .finish() on the returned writer to obtain a buffer.

  • encodeDelimited(message: Message | object, writer?: Writer): Writer
    Encodes a length-delimited message.

  • decode(reader: Reader | Uint8Array): Message
    Decodes a message from protobuf binary data.

  • decodeDelimited(reader: Reader | Uint8Array): Message
    Decodes a length-delimited message.

  • create(properties?: object): Message
    Creates a message instance from already valid data.

  • verify(object: object): null | string
    Checks whether a plain object can be encoded as-is. Returns null if valid, otherwise an error message.

  • fromObject(object: object): Message
    Converts broader JavaScript input into a message instance.

  • toObject(message: Message, options?: ConversionOptions): object
    Converts a message instance to a configurable plain JavaScript object.

  • message.toJSON(): object
    Converts a message instance to JSON-compatible output using default conversion options.

Message instances provide runtime identity, so they can be tested with instanceof. Their toJSON method integrates them with JSON.stringify.

Length-delimited methods read and write a varint byte length before the message, which is useful for streams and framed protocols.

If required fields are missing while decoding proto2 data, decode throws protobuf.util.ProtocolError with the partially decoded message available as err.instance.

Runtimes

protobuf.js provides three runtime entry points, keeping parser and reflection support optional: Runtime .proto loading needs the parser, JSON/reflection bundles need reflection support, and generated static modules only need the minimal runtime.

Import Includes Use when
protobufjs Reflection, Parser You load .proto files at runtime
protobufjs/light.js Reflection You load JSON bundles or build schemas programmatically
protobufjs/minimal.js Static runtime You use generated static code

The full build includes the light build, and the light build includes the minimal runtime.

Code generation

Use protobufjs-cli to generate reflection bundles, static JavaScript code, and matching TypeScript declarations, either directly with pbjs or through the optional protoc-gen-pbjs plugin for protoc.

Reflection keeps schemas as JSON metadata and generates optimized functions at runtime. Static code emits schema-specific, reflection-free functions ahead of time. The main tradeoffs are how schemas are loaded, how bundle size scales with schema size, and whether reflection metadata should remain available at runtime.

Target Output Minimum Runtime
json JSON bundle protobufjs/light.js
json-module JSON bundle module protobufjs/light.js
static-module Static code module protobufjs/minimal.js

Module targets support --wrap default for CommonJS and AMD, plus esm, commonjs, amd, and closure; --wrap can also load a custom wrapper module.

Static modules

Static modules emit dedicated JavaScript for your schema, so they only need protobufjs/minimal.js at runtime.

npx pbjs -t static-module -w esm -o awesome.js --dts awesome.proto
import { awesomepackage } from "./awesome.js";

const message = awesomepackage.AwesomeMessage.create({ awesomeField: "hello" });

While static code is verbose by design, its repeated patterns compress well with Brotli or gzip, and it works in CSP-restricted environments that disallow unsafe-eval without sacrificing performance.

Reflection bundles

Reflection bundles store schemas as compact JSON metadata, avoiding .proto parsing at runtime and letting browsers load schema metadata in one request. While they require at least protobufjs/light.js, large schemas can produce smaller combined bundles than equivalent static modules because common code is shared through reflection.

npx pbjs -t json -o awesome.json awesome1.proto awesome2.proto ...
const bundle = require("./awesome.json");

const root = protobuf.Root.fromJSON(bundle);
const AwesomeMessage = root.lookupType("awesomepackage.AwesomeMessage");
npx pbjs -t json-module -w esm -o awesome.js --dts awesome.proto
import { awesomepackage } from "./awesome.js";

const AwesomeMessage = awesomepackage.AwesomeMessage;

JSON modules export the reflection root and, with -w esm, also provide top-level named exports that align with static modules. Their declarations mirror static-module typings, but because JSON modules are backed by reflection objects, message instances should be created with MyMessage.create(...) instead of constructors. Code using create(...) works with static modules as well.

TypeScript integration

protobuf.js works with TypeScript out of the box: the runtime API is typed, and generated JavaScript can be paired with strong TypeScript declarations in the same CLI invocation. Generated output is directly usable from JavaScript without a transpile step, and strongly typed in TypeScript projects, with type-checked oneofs and JavaScript-friendly plain-object input.

For example, given the oneof:

message Profile {
  oneof contact {
    string email = 1;
    string phone = 2;
  }
}

Generated declarations narrow both the contact oneof and the concrete values:

const profile = Profile.create({
  contact: "email",
  email: "hello@example.com"
});

if (profile.contact === "email") {
  profile.email; // string
}

const decoded = Profile.decode(bytes);
if (decoded.contact === "phone") {
  decoded.phone; // string
}

Plain objects can use the same narrowed shape through a collision-free scoped type:

const object: Profile.$Shape = {
  contact: "email",
  email: "hello@example.com"
};

Advanced usage

Programmatic schemas

The full and light builds can construct schemas directly through reflection:

const AwesomeMessage = new protobuf.Type("AwesomeMessage")
  .add(new protobuf.Field("awesomeField", 1, "string"));

const root = new protobuf.Root()
  .define("awesomepackage")
  .add(AwesomeMessage);

Custom message classes

A reflected type can use a custom class as its runtime constructor:

class AwesomeMessage extends protobuf.Message<AwesomeMessage> {
  awesomeField = "";

  constructor(properties?: protobuf.Properties<AwesomeMessage>) {
    super(properties);
    // ...
  }

  customInstanceMethod() {
    return this.awesomeField.toLowerCase();
  }
}

root.lookupType("awesomepackage.AwesomeMessage").ctor = AwesomeMessage;

const decoded = AwesomeMessage.decode(bytes);
decoded.customInstanceMethod(); // string

protobuf.js will populate the constructor with the usual static runtime methods and use it for decoded messages. In TypeScript, custom members are visible when using the custom class type in consuming code.

Services

protobuf.js supports service clients built from service definitions. The service API is transport-agnostic: provide an rpcImpl function to connect it to HTTP, WebSocket, gRPC, or another transport.

function myRpcImpl(method, requestData, callback) {
  // method.name
  // method.path
  // method.requestStream?
  // method.responseStream?
  performRequest(requestData, function(err, responseData) {
    callback(err, responseData);
  });
}

const myService = MyService.create(myRpcImpl/*, requestDelimited?, responseDelimited? */);

See examples/streaming-rpc.js for a streaming example.

Extensions

The following extensions provide descriptor conversion and text-based protobuf formats when reflection metadata is available. Most applications only need the binary APIs above.

Descriptors

protobuf.js uses a compact JSON-based reflection representation internally that is easy to embed and fast to parse, so schemas can be loaded directly without first decoding binary descriptor blobs or postprocessing their full JSON representation. See ext/descriptor for use cases that need conversion between reflected roots and protoc descriptor messages.

ProtoJSON

Protocol Buffers support a special ProtoJSON format to share data with systems that do not support the binary wire format, for example when implementing gateways. Spec-compliant ProtoJSON is supported via ext/protojson.

Text Format

Protocol Buffers Text Format is a special syntax for representing protobuf data in text form, which can be useful for configurations or tests. Spec-compliant Text Format is supported via ext/textformat.

Conformance

protobuf.js targets complete binary wire-format conformance for Proto2, Proto3 and Editions in both static and reflection modes, plus complete ProtoJSON and Text Format conformance with reflection metadata present. CI runs the official Protocol Buffers conformance suite for validation, with logs uploaded as artifacts.

Category Total Required Recommended
Binary 100.00% (2805/2805) 100.00% (1939/1939) 100.00% (866/866)
↳ Proto2 100.00% (703/703) 100.00% (485/485) 100.00% (218/218)
↳ Proto3 100.00% (698/698) 100.00% (482/482) 100.00% (216/216)
↳ Editions 100.00% (1404/1404) 100.00% (972/972) 100.00% (432/432)
ProtoJSON 100.00% (2762/2762) 100.00% (2328/2328) 100.00% (434/434)
TextFormat 100.00% (883/883) 100.00% (819/819) 100.00% (64/64)
Overall 100.00% (6450/6450) 100.00% (5086/5086) 100.00% (1364/1364)

Performance

In both reflection and reflection-free modes, protobuf.js builds specialized encoders and decoders instead of interpreting descriptors at runtime.

The repository includes a small benchmark. It compares protobuf.js reflection and static code against JSON encode/decode, protoc-gen-js, and protoc-gen-es. Results depend on hardware, Node.js version, and message shape, so they should be treated as indicative rather than absolute.

Benchmark run on AMD Ryzen 9 9950X3D with Node.js 24.15.0
benchmarking encode performance ...

protobuf.js reflect x 2,430,103 ops/sec ±0.62% (95 runs sampled)
protobuf.js static x 2,390,407 ops/sec ±0.42% (96 runs sampled)
JSON encode x 2,155,918 ops/sec ±0.63% (92 runs sampled)
protoc-gen-js x 995,429 ops/sec ±0.18% (98 runs sampled)
protoc-gen-es x 403,334 ops/sec ±0.14% (96 runs sampled)

    protobuf.js reflect was fastest
     protobuf.js static was 1.4% ops/sec slower (factor 1.0)
            JSON encode was 11.3% ops/sec slower (factor 1.1)
          protoc-gen-js was 58.9% ops/sec slower (factor 2.4)
          protoc-gen-es was 83.3% ops/sec slower (factor 6.0)

benchmarking decode performance ...

protobuf.js reflect x 6,440,387 ops/sec ±0.25% (97 runs sampled)
protobuf.js static x 6,463,283 ops/sec ±0.27% (101 runs sampled)
JSON decode x 1,409,923 ops/sec ±0.11% (97 runs sampled)
protoc-gen-js x 947,647 ops/sec ±0.15% (99 runs sampled)
protoc-gen-es x 731,819 ops/sec ±0.28% (98 runs sampled)

     protobuf.js static was fastest
    protobuf.js reflect was 0.3% ops/sec slower (factor 1.0)
            JSON decode was 78.2% ops/sec slower (factor 4.6)
          protoc-gen-js was 85.3% ops/sec slower (factor 6.8)
          protoc-gen-es was 88.7% ops/sec slower (factor 8.8)

benchmarking round-trip performance ...

protobuf.js reflect x 1,310,677 ops/sec ±0.21% (97 runs sampled)
protobuf.js static x 1,310,926 ops/sec ±0.26% (101 runs sampled)
JSON encode/decode x 741,714 ops/sec ±0.24% (99 runs sampled)
protoc-gen-js x 472,844 ops/sec ±0.09% (96 runs sampled)
protoc-gen-es x 254,044 ops/sec ±0.05% (101 runs sampled)

    protobuf.js reflect was fastest
     protobuf.js static was 0.0% ops/sec slower (factor 1.0)
     JSON encode/decode was 43.4% ops/sec slower (factor 1.8)
          protoc-gen-js was 63.9% ops/sec slower (factor 2.8)
          protoc-gen-es was 80.6% ops/sec slower (factor 5.2)

Run it locally with:

npm --prefix bench install
npm run bench

Compatibility

Supported runtimes are browsers, Node.js v12+, Deno and Bun. When using the CLI with Bun, Node.js must also be installed.

Security

Security-impacting reports are handled through coordinated GitHub Security Advisories where appropriate. See SECURITY.md for supported release lines and reporting instructions.

Development

git clone https://github.com/protobufjs/protobuf.js
cd protobuf.js
npm install
npm --prefix cli install

Running the tests:

npm test

Building the development and production versions with their respective source maps to dist/:

npm run build

Additional documentation

About

Protocol Buffers for JavaScript and TypeScript.

Topics

Resources

License

Contributing

Security policy

Stars

Watchers

Forks

Sponsor this project

 

Contributors