惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

GbyAI
GbyAI
T
Tenable Blog
Webroot Blog
Webroot Blog
L
Lohrmann on Cybersecurity
S
Securelist
S
Schneier on Security
NISL@THU
NISL@THU
Know Your Adversary
Know Your Adversary
C
Cybersecurity and Infrastructure Security Agency CISA
T
The Exploit Database - CXSecurity.com
L
LINUX DO - 热门话题
C
CXSECURITY Database RSS Feed - CXSecurity.com
O
OpenAI News
I
Intezer
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
TaoSecurity Blog
TaoSecurity Blog
S
Secure Thoughts
Application and Cybersecurity Blog
Application and Cybersecurity Blog
P
Privacy International News Feed
H
Hacker News: Front Page
N
Netflix TechBlog - Medium
M
MIT News - Artificial intelligence
博客园 - Franky
PCI Perspectives
PCI Perspectives
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
Microsoft Azure Blog
Microsoft Azure Blog
MongoDB | Blog
MongoDB | Blog
L
LangChain Blog
P
Proofpoint News Feed
S
Security Affairs
WordPress大学
WordPress大学
The Last Watchdog
The Last Watchdog
S
SegmentFault 最新的问题
小众软件
小众软件
F
Full Disclosure
博客园 - 叶小钗
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
T
The Blog of Author Tim Ferriss
Simon Willison's Weblog
Simon Willison's Weblog
P
Palo Alto Networks Blog
Security Latest
Security Latest
P
Proofpoint News Feed
月光博客
月光博客
T
Tailwind CSS Blog
Scott Helme
Scott Helme
Hacker News - Newest:
Hacker News - Newest: "LLM"
Google Online Security Blog
Google Online Security Blog
T
Threat Research - Cisco Blogs
Help Net Security
Help Net Security
Project Zero
Project Zero

kmcd.dev

Exploring Protocol Buffers Interactively Introducing ProtoDocs Ghost in the Shell: The Manga Behind the Anime The Hidden Cost of google.protobuf.Value Why Networking Built Its Own Data Modeling Language Zero-Friction Demos with WASM Let's Learn About BGP ConnectRPC: Where is it now? Building APIs with Contracts The Case for Greppable Code Unknown Fields in Protobuf IRC Log: Reactionary Faking protobuf data in Go Y'all are Sleeping on Mise-en-Place IRC Log: Standup 2 HTTP/2 From Scratch: Part 4 IRC Log: rm -rf /var/opt/gitlab/postgresql/data HTTP/2 From Scratch: Part 3 Building a Live BGP Map HTTP/2 From Scratch: Part 2 IRC Log: The Cloud Scale Incident Visualizing the Internet (2026) Shell Log: Namaste HTTP/2 From Scratch: Part 1 IRC Log: Standup HTTP/1.1 From Scratch WHOIS is dead, long live RDAP Months Considered Harmful Encryption vs. Compression On Creating My Own Cover Art Traceroute Tool from Scratch in Go My Favorite Interview Question From JSON to Protobuf Breaking gRPC Morse Code Can You Hack a Phone with Your Voice? Visualizing the Internet (2025) HTTP QUERY and Go I made a daily word game Protovalidate: Can Input Validation Be This Easy? Behold! The Barcode Scanner Mixing CEL and Protobuf for Fun FauxRPC and Protovalidate The Call of the Monolithic Codebase FauxRPC + Test Containers Self-Documenting Connect Services gRPC Over HTTP/3: Followup JSON to Protobuf Conversion gRPC: The Ugly Parts Working with Protobuf in 2024 Introducing FauxRPC HTTP/1.0 From Scratch Y'all are sleeping on HTTP/3 HTTP/0.9 From Scratch What version of HTTP are you using? Texans in Denmark gRPC Over HTTP/3 gRPC: The Good Parts Leaving Texas for Greener Pastures gRPC: The Bad Parts Unit Testing ConnectRPC Servers Daily Prompts Adding chart.js to Hugo Why I'm Rebranding Benchmarking gRPC (golang) Blog Update gRPC From Scratch: Part 3 - Protobuf Encoding Tracking the Wins Visualizing the Internet (2024) RESTless: Web APIs After REST Introducing unknownconnect-go Making gRPC more approachable with ConnectRPC Inspecting Protobuf Messages Introducing protoc-gen-connect-openapi gRPC From Scratch: Part 2 - Server gRPC From Scratch: Part 1 - Client Why you should use gNMI over SNMP in 2026 The Rollercoaster of Productivity in Side Projects Lessons from a Decades-Long Project How I learned to code Economists with (virtual) Guns Visualizing the Internet (2023) softlayer-python: language bindings/CLI for a cloud company SwFTP: SFTP/FTP Server For Openstack Swift Video: Morning Copenhagen Commute Goodbye Evepraisal Visualizing the spectrum of the sun (Part 2) Visualizing the Internet (2022) Evepraisal: A price estimation tool for Eve Online Visualizing the spectrum of the sun
Dropping Unknown Fields in ConnectRPC
2024-04-02 · via kmcd.dev

gRPC, with its focus on performance and language neutrality, remains a popular choice for building microservices and APIs. But when exposing your gRPC service to the internet, there are a few security considerations to account for. Protobuf, the serialization format often used with gRPC, offers various encoding options that can significantly impact your service’s security posture.

One crucial optimization for internet-facing gRPC services is customizing the behavior towards unknown fields. I’ve talked about unknown fields in a previous post, so read that one if unknown fields are still a mystery to you and then come back here. By default, protobuf messages can contain fields that are not defined in the current version of the proto schema. While convenient for development and can help with forward compatibility, this poses a security risk in a public environment.

Here’s why you should consider dropping unknown fields when exposing gRPC to the internet:

  • Preventing Malicious Data: Unknown fields can be exploited by malicious actors to inject unexpected data into your service. This could lead to potential security vulnerabilities like code injection or unexpected behavior.
  • Ensuring Compatibility: Uncontrolled unknown fields can cause compatibility issues if your clients are using different versions of the proto schema. Dropping them enforces stricter adherence to the defined message format.
  • Improving Performance: Skipping unknown fields during message parsing can lead to performance gains, especially when dealing with large datasets.

How to Drop Unknown Fields

Here is how you can drop unknown fields while using the standard proto.UnmarshalOptions struct provided by the google.golang.org/protobuf/proto package. Here’s how to do it in your Go code:

import (
	"google.golang.org/protobuf/proto"
	...
)

// Configure unmarshalling options to discard unknown fields
opts := proto.UnmarshalOptions{
	DiscardUnknown: true,
}

// Use the options when unmarshalling incoming messages
msg := &MyMessage{}
err := proto.Unmarshal(data, msg, opts)
if err != nil {
	// Handle error
}

By setting the DiscardUnknown field to true in the proto.UnmarshalOptions struct before unmarshalling incoming messages, you ensure that any unknown fields are ignored. This helps mitigate the security risks associated with unknown fields while processing internet-facing gRPC requests.

How to Drop Unknown Fields in Connect RPC Servers

package main

import (
	"log"
	"net/http"

	"golang.org/x/net/http2"
	"golang.org/x/net/http2/h2c"
	"go.akshayshah.org/connectproto"
)

func main() {
	greeter := &GreetServer{}
	mux := http.NewServeMux()
	path, handler := greetv1connect.NewGreetServiceHandler(
		greeter,
		// Add an option that customizes protobuf marshalling/unmarshalling behavior
		connectproto.WithBinary(
			proto.MarshalOptions{},
			proto.UnmarshalOptions{DiscardUnknown: true},
		),
		// Add an option to customize JSON marshalling/unmachalling
		connectproto.WithJSON(
			protojson.MarshalOptions{},
			protojson.UnmarshalOptions{DiscardUnknown: true},
		)
	)
	mux.Handle(path, handler)
	log.Fatal(http.ListenAndServe(
		"localhost:9000",
		h2c.NewHandler(mux, &http2.Server{}),
	))
}

In this example, connectproto.WithBinary ensures only messages with defined fields are processed, enhancing the security of your gRPC service. connectproto.WithJSON does the same thing but with JSON.

Additional Considerations

While dropping unknown fields is a valuable security practice, it’s important to consider potential trade-offs:

  • Backward compatibility: Clients using older versions of the proto schema will encounter errors if they rely on previously defined unknown fields.
  • Logging and Debugging: Dropping unknown fields might make it harder to identify the source of unexpected behavior during development or debugging.

In such cases, it’s recommended to document these trade-offs and have a clear versioning policy for your gRPC service and client applications.

Conclusion

Exposing gRPC services to the internet requires careful security considerations. By customizing protobuf encoding options, specifically by dropping unknown fields using proto.UnmarshalOptions, you can significantly improve the security posture of your service. Remember to weigh the benefits against potential drawbacks and implement a solution that aligns with your specific needs.