惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

The Last Watchdog
The Last Watchdog
C
Cyber Attacks, Cyber Crime and Cyber Security
L
LINUX DO - 热门话题
G
GRAHAM CLULEY
S
Schneier on Security
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
S
SegmentFault 最新的问题
IT之家
IT之家
阮一峰的网络日志
阮一峰的网络日志
Recorded Future
Recorded Future
I
Intezer
云风的 BLOG
云风的 BLOG
博客园 - Franky
月光博客
月光博客
大猫的无限游戏
大猫的无限游戏
T
Tenable Blog
The Hacker News
The Hacker News
T
The Blog of Author Tim Ferriss
Attack and Defense Labs
Attack and Defense Labs
D
DataBreaches.Net
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
N
News and Events Feed by Topic
有赞技术团队
有赞技术团队
Exploit-DB.com RSS Feed
Exploit-DB.com RSS Feed
N
News and Events Feed by Topic
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
S
Secure Thoughts
The Register - Security
The Register - Security
B
Blog
Security Archives - TechRepublic
Security Archives - TechRepublic
The Cloudflare Blog
Webroot Blog
Webroot Blog
W
WeLiveSecurity
H
Heimdal Security Blog
博客园 - 三生石上(FineUI控件)
V
Vulnerabilities – Threatpost
G
Google Developers Blog
O
OpenAI News
V
V2EX
罗磊的独立博客
博客园_首页
N
News | PayPal Newsroom
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
TaoSecurity Blog
TaoSecurity Blog
Cloudbric
Cloudbric
H
Hacker News: Front Page
博客园 - 叶小钗
T
Tor Project blog
AI
AI

Pierce Freeman

A browser for agents | Pierce Freeman The grey market of podcast appearances The way I travel | Pierce Freeman Fixing slow AWS uploads | Pierce Freeman Local tools should still use vaults We solved scratch content first Starting a podcast in 2025 Being late but still being early Automating our home video imports Adding my parents to tailscale A deep dive on agent sandboxes Language servers for AI | Pierce Freeman My simple home podcast studio We need centralized infrastructure | Pierce Freeman Coercing agents to follow conventions using AST validation My unified theory of social selling My personal backup strategy | Pierce Freeman July updates to the homelab How the KV Cache works httpx is the right way to do web requests in Python Reputation is becoming everything | Pierce Freeman Building a (kind of) invisible mac app Updated knowledge in language models Making an ascii animation | Pierce Freeman How speculative decoding works | Pierce Freeman Under the hood of Claude Code Doing things because they're easy, not hard Speeding up sideeffects with JIT in mountaineer Firehot for hot reloading in Python Misadventures in Python hot reloading How text diffusion works | Pierce Freeman The tenacity of modern LLMs The ergonomics of rails | Pierce Freeman How language servers work | Pierce Freeman Just add eggs | Pierce Freeman Unfortunately SEO still matters | Pierce Freeman The futility of human-only web requirements Setting up Input Leap | Pierce Freeman Checking in on Waymo | Pierce Freeman The react revolution | Pierce Freeman Speeding up many small transfers to a unifi nas Quick notes on swift libraries AI engineering is a different animal San Francisco | Pierce Freeman Debugging a mountaineer rendering segfault Local network config on macOS Building our home network | Pierce Freeman Introducing Envelope.dev | Pierce Freeman Legacy code and AI copilots Typehinting from day-zero | Pierce Freeman Generating database migrations with acyclic graphs Lofoten | Pierce Freeman Mountaineer v0.1: Webapps in Python and React Constraining LLM Outputs | Pierce Freeman Passthrough above all | Pierce Freeman Accuracy in kudos | Pierce Freeman How quick we are to adapt The curious case of LM repetition Costa Rica | Pierce Freeman Debugging chrome extensions with system-level logging Speeding up runpod | Pierce Freeman Inline footnotes with html templates Parsing Common Crawl in a day for $60 An era of rich CLI All or nothing with remote work The Next 10 Years | Pierce Freeman Adding wheels to flash-attention | Pierce Freeman LLMs as interdisciplinary agents | Pierce Freeman New Zealand | Pierce Freeman Representations in autoregressive models | Pierce Freeman Let's talk about Siri | Pierce Freeman Minimum viable public infrastructure | Pierce Freeman Reasoning vs. Memorization in LLMs Automatically migrate enums in alembic Greater sequence lengths will set us free On learning to ski | Pierce Freeman Dolomites | Pierce Freeman Opportunity years | Pierce Freeman Buzzword peaks and valleys | Pierce Freeman Buenos Aires | Pierce Freeman Network routing interaction on MacOS Independent work: November recap | Pierce Freeman Debugging slow pytorch training performance The provenance of copy and paste Debugging tips for neural network training Patagonia | Pierce Freeman Santiago | Pierce Freeman My 2022 digital travel kit AWS vs GCP - GPU Availability V2 Independent work: October recap | Pierce Freeman Planning Patagonia | Pierce Freeman Relationship modeling | Pierce Freeman The power of status updates A new chapter | Pierce Freeman Give my library a coffee shop AWS vs GCP - GPU Availability V1 Switzerland | Pierce Freeman Headfull browsers beat headless | Pierce Freeman Webcrawling tradeoffs | Pierce Freeman Copenhagen | Pierce Freeman
Using grpc with node and typescript
2023-02-16 · via Pierce Freeman

Documentation for using grpc within node is split between static generation and dynamic generation. Dynamic generation compiles protobuffer definition files at runtime through protobuf.js and typically looks like the following:

var PROTO_PATH = __dirname + '/../../../protos/api.proto';
var grpc = require('@grpc/grpc-js');
var protoLoader = require('@grpc/proto-loader');
var packageDefinition = protoLoader.loadSync(
    PROTO_PATH,
    {keepCase: true,
     longs: String,
     enums: String,
     defaults: true,
     oneofs: true
    });
var protoDescriptor = grpc.loadPackageDefinition(packageDefinition);

Most of the grpc docs use the dynamic approach - I assume for ease of getting started. The main pro to dynamic generation is faster prototyping if the underlying schema changes, since you can hot reload the server/client. But one key downside includes not being able to typehint anything during development or compilation. For production use compiling it down to static code is a must.

I've started using a pipeline that re-generates static compiled files and their typescript definitions automatically. Here's how to do it.

The standard protoc version that generates code for Go/Java/Python/etc. doesn't have support for generating javascript definition files.

Screenshot of protoc

Where's my javascript at??

JS generation is only supported through the grpc-tools library, which includes a wrapped version of protoc and a plugin to generate javascript code.1 The CLI interface is identical except now it offers javascript generation support.

Screenshot of node protoc

Ah, there you are.

So to get started you'll have to download grpc-tools and grpc_tools_node_protoc_ts as dependencies. The second dependency will support the typescript generation.

npm install -d grpc-tools grpc_tools_node_protoc_ts

The javascript-enabled protoc now lives within node modules, so to make sure it installed correctly you can call:

$ ./node_modules/.bin/grpc_tools_node_protoc --help

Now it's time to wire everything up. Start with generating the javascript code. The --js_out argument will generate the definition files for messages, enums, and the like. The --grpc_out will generate the stub files for client and server implementations. In most applications you'll need both. If bash is your speed:

build.sh:

(
    mkdir -p app/src/api \\
    && app/node_modules/.bin/grpc_tools_node_protoc \\
        --grpc_out=grpc_js:app/src/api \\
        --js_out=import_style=commonjs,binary:app/src/api \\
        protos/api.proto
)

To generate the typescript definitions:

build.sh:

(
    protoc \\
        --plugin=protoc-gen-ts=app/node_modules/.bin/protoc-gen-ts \\
        --ts_out=grpc_js:app/src/api \\
        protos/api.proto
)

At this point you should have a populated app/src/api/protos folder with all the definition files. To use in your main code, just import the appropriate classes and bask in the glow of your newfound typehints.

import { CommandMessage } from './api/protos/api_pb';
import { APIClient } from './api/protos/api_grpc_pb';

The one last step is to automate the rebuild process. I use chokidar to watch the protobuf directory and re-run this build script. I also add a final generation just for good measure before I build my typescript files down to javascript:

{
    ...
    "scripts": {
        "build": "./build.sh && tsc",
        "watch-protobuf": "./watch.js"
    }
}

Easy. And your dev workflow will thank you.

  1. It should be possible to use the plugin directly with the standard protoc install through the --plugin keyword arg, but it didn't seem to work as of libprotoc 3.21.8. ↩