惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

A
Arctic Wolf
T
The Blog of Author Tim Ferriss
月光博客
月光博客
Recent Announcements
Recent Announcements
V
V2EX
Microsoft Azure Blog
Microsoft Azure Blog
博客园 - 三生石上(FineUI控件)
P
Proofpoint News Feed
The Register - Security
The Register - Security
博客园 - 叶小钗
博客园 - Franky
The Cloudflare Blog
雷峰网
雷峰网
罗磊的独立博客
M
MIT News - Artificial intelligence
I
InfoQ
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
博客园 - 【当耐特】
Engineering at Meta
Engineering at Meta
N
Netflix TechBlog - Medium
爱范儿
爱范儿
博客园 - 司徒正美
Recorded Future
Recorded Future
酷 壳 – CoolShell
酷 壳 – CoolShell
Google DeepMind News
Google DeepMind News
Martin Fowler
Martin Fowler
Microsoft Security Blog
Microsoft Security Blog
F
Full Disclosure
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
B
Blog
大猫的无限游戏
大猫的无限游戏
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
腾讯CDC
WordPress大学
WordPress大学
小众软件
小众软件
K
Kaspersky official blog
Attack and Defense Labs
Attack and Defense Labs
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
Forbes - Security
Forbes - Security
aimingoo的专栏
aimingoo的专栏
IT之家
IT之家
The Last Watchdog
The Last Watchdog
N
News and Events Feed by Topic
B
Blog RSS Feed
S
Security @ Cisco Blogs
美团技术团队
量子位
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
Cloudbric
Cloudbric
Hacker News - Newest:
Hacker News - Newest: "LLM"

kmcd.dev

Exploring Protocol Buffers Interactively Introducing ProtoDocs Ghost in the Shell: The Manga Behind the Anime The Hidden Cost of google.protobuf.Value Why Networking Built Its Own Data Modeling Language Zero-Friction Demos with WASM Let's Learn About BGP ConnectRPC: Where is it now? Building APIs with Contracts The Case for Greppable Code Unknown Fields in Protobuf IRC Log: Reactionary Faking protobuf data in Go Y'all are Sleeping on Mise-en-Place IRC Log: Standup 2 HTTP/2 From Scratch: Part 4 IRC Log: rm -rf /var/opt/gitlab/postgresql/data HTTP/2 From Scratch: Part 3 Building a Live BGP Map HTTP/2 From Scratch: Part 2 IRC Log: The Cloud Scale Incident Visualizing the Internet (2026) Shell Log: Namaste HTTP/2 From Scratch: Part 1 IRC Log: Standup HTTP/1.1 From Scratch WHOIS is dead, long live RDAP Months Considered Harmful Encryption vs. Compression On Creating My Own Cover Art Traceroute Tool from Scratch in Go My Favorite Interview Question From JSON to Protobuf Breaking gRPC Morse Code Can You Hack a Phone with Your Voice? Visualizing the Internet (2025) HTTP QUERY and Go I made a daily word game Protovalidate: Can Input Validation Be This Easy? Behold! The Barcode Scanner Mixing CEL and Protobuf for Fun FauxRPC and Protovalidate The Call of the Monolithic Codebase FauxRPC + Test Containers Self-Documenting Connect Services gRPC Over HTTP/3: Followup JSON to Protobuf Conversion gRPC: The Ugly Parts Working with Protobuf in 2024 Introducing FauxRPC HTTP/1.0 From Scratch Y'all are sleeping on HTTP/3 HTTP/0.9 From Scratch What version of HTTP are you using? Texans in Denmark gRPC Over HTTP/3 gRPC: The Good Parts Leaving Texas for Greener Pastures gRPC: The Bad Parts Unit Testing ConnectRPC Servers Daily Prompts Adding chart.js to Hugo Why I'm Rebranding Benchmarking gRPC (golang) Blog Update gRPC From Scratch: Part 3 - Protobuf Encoding Tracking the Wins Visualizing the Internet (2024) RESTless: Web APIs After REST Introducing unknownconnect-go Making gRPC more approachable with ConnectRPC Inspecting Protobuf Messages Introducing protoc-gen-connect-openapi gRPC From Scratch: Part 2 - Server gRPC From Scratch: Part 1 - Client Why you should use gNMI over SNMP in 2026 The Rollercoaster of Productivity in Side Projects Lessons from a Decades-Long Project How I learned to code Economists with (virtual) Guns Visualizing the Internet (2023) softlayer-python: language bindings/CLI for a cloud company SwFTP: SFTP/FTP Server For Openstack Swift Video: Morning Copenhagen Commute Goodbye Evepraisal Visualizing the spectrum of the sun (Part 2) Visualizing the Internet (2022) Evepraisal: A price estimation tool for Eve Online Visualizing the spectrum of the sun
Dropping Unknown Fields in ConnectRPC
2024-04-02 · via kmcd.dev

gRPC, with its focus on performance and language neutrality, remains a popular choice for building microservices and APIs. But when exposing your gRPC service to the internet, there are a few security considerations to account for. Protobuf, the serialization format often used with gRPC, offers various encoding options that can significantly impact your service’s security posture.

One crucial optimization for internet-facing gRPC services is customizing the behavior towards unknown fields. I’ve talked about unknown fields in a previous post, so read that one if unknown fields are still a mystery to you and then come back here. By default, protobuf messages can contain fields that are not defined in the current version of the proto schema. While convenient for development and can help with forward compatibility, this poses a security risk in a public environment.

Here’s why you should consider dropping unknown fields when exposing gRPC to the internet:

  • Preventing Malicious Data: Unknown fields can be exploited by malicious actors to inject unexpected data into your service. This could lead to potential security vulnerabilities like code injection or unexpected behavior.
  • Ensuring Compatibility: Uncontrolled unknown fields can cause compatibility issues if your clients are using different versions of the proto schema. Dropping them enforces stricter adherence to the defined message format.
  • Improving Performance: Skipping unknown fields during message parsing can lead to performance gains, especially when dealing with large datasets.

How to Drop Unknown Fields

Here is how you can drop unknown fields while using the standard proto.UnmarshalOptions struct provided by the google.golang.org/protobuf/proto package. Here’s how to do it in your Go code:

import (
	"google.golang.org/protobuf/proto"
	...
)

// Configure unmarshalling options to discard unknown fields
opts := proto.UnmarshalOptions{
	DiscardUnknown: true,
}

// Use the options when unmarshalling incoming messages
msg := &MyMessage{}
err := proto.Unmarshal(data, msg, opts)
if err != nil {
	// Handle error
}

By setting the DiscardUnknown field to true in the proto.UnmarshalOptions struct before unmarshalling incoming messages, you ensure that any unknown fields are ignored. This helps mitigate the security risks associated with unknown fields while processing internet-facing gRPC requests.

How to Drop Unknown Fields in Connect RPC Servers

package main

import (
	"log"
	"net/http"

	"golang.org/x/net/http2"
	"golang.org/x/net/http2/h2c"
	"go.akshayshah.org/connectproto"
)

func main() {
	greeter := &GreetServer{}
	mux := http.NewServeMux()
	path, handler := greetv1connect.NewGreetServiceHandler(
		greeter,
		// Add an option that customizes protobuf marshalling/unmarshalling behavior
		connectproto.WithBinary(
			proto.MarshalOptions{},
			proto.UnmarshalOptions{DiscardUnknown: true},
		),
		// Add an option to customize JSON marshalling/unmachalling
		connectproto.WithJSON(
			protojson.MarshalOptions{},
			protojson.UnmarshalOptions{DiscardUnknown: true},
		)
	)
	mux.Handle(path, handler)
	log.Fatal(http.ListenAndServe(
		"localhost:9000",
		h2c.NewHandler(mux, &http2.Server{}),
	))
}

In this example, connectproto.WithBinary ensures only messages with defined fields are processed, enhancing the security of your gRPC service. connectproto.WithJSON does the same thing but with JSON.

Additional Considerations

While dropping unknown fields is a valuable security practice, it’s important to consider potential trade-offs:

  • Backward compatibility: Clients using older versions of the proto schema will encounter errors if they rely on previously defined unknown fields.
  • Logging and Debugging: Dropping unknown fields might make it harder to identify the source of unexpected behavior during development or debugging.

In such cases, it’s recommended to document these trade-offs and have a clear versioning policy for your gRPC service and client applications.

Conclusion

Exposing gRPC services to the internet requires careful security considerations. By customizing protobuf encoding options, specifically by dropping unknown fields using proto.UnmarshalOptions, you can significantly improve the security posture of your service. Remember to weigh the benefits against potential drawbacks and implement a solution that aligns with your specific needs.