惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

博客园 - 三生石上(FineUI控件)
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
月光博客
月光博客
博客园 - 【当耐特】
Hugging Face - Blog
Hugging Face - Blog
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
爱范儿
爱范儿
大猫的无限游戏
大猫的无限游戏
S
SegmentFault 最新的问题
博客园_首页
雷峰网
雷峰网
量子位
有赞技术团队
有赞技术团队
博客园 - 叶小钗
博客园 - 聂微东
V
V2EX
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
博客园 - 司徒正美
小众软件
小众软件
The Cloudflare Blog
阮一峰的网络日志
阮一峰的网络日志
Apple Machine Learning Research
Apple Machine Learning Research
Jina AI
Jina AI
人人都是产品经理
人人都是产品经理

kmcd.dev

Joining Buf Let The Gravity of Ashburn, Virginia The CPU Cost of Protobuf Varints in Go It Beating Go gRPC-Web Should Have Fixed gRPC Making Dynamic Protobuf Fast in Go Proxy, Record, and Mock gRPC APIs with FauxRPC Exploring Protocol Buffers Interactively Introducing ProtoDocs Ghost in the Shell: The Manga Behind the Anime The Hidden Cost of google.protobuf.Value Why Networking Built Its Own Data Modeling Language Zero-Friction Demos with WASM Let's Learn About BGP ConnectRPC: Where is it now? Building APIs with Contracts The Case for Greppable Code Unknown Fields in Protobuf IRC Log: Reactionary Faking protobuf data in Go Y'all are Sleeping on Mise-en-Place IRC Log: Standup 2 HTTP/2 From Scratch: Part 4 IRC Log: rm -rf /var/opt/gitlab/postgresql/data HTTP/2 From Scratch: Part 3 Building a Live BGP Map HTTP/2 From Scratch: Part 2 IRC Log: The Cloud Scale Incident
Dropping Unknown Fields in ConnectRPC
2024-04-02 · via kmcd.dev

gRPC, with its focus on performance and language neutrality, remains a popular choice for building microservices and APIs. But when exposing your gRPC service to the internet, there are a few security considerations to account for. Protobuf, the serialization format often used with gRPC, offers various encoding options that can significantly impact your service’s security posture.

One crucial optimization for internet-facing gRPC services is customizing the behavior towards unknown fields. I’ve talked about unknown fields in a previous post, so read that one if unknown fields are still a mystery to you and then come back here. By default, protobuf messages can contain fields that are not defined in the current version of the proto schema. While convenient for development and can help with forward compatibility, this poses a security risk in a public environment.

Here’s why you should consider dropping unknown fields when exposing gRPC to the internet:

  • Preventing Malicious Data: Unknown fields can be exploited by malicious actors to inject unexpected data into your service. This could lead to potential security vulnerabilities like code injection or unexpected behavior.
  • Ensuring Compatibility: Uncontrolled unknown fields can cause compatibility issues if your clients are using different versions of the proto schema. Dropping them enforces stricter adherence to the defined message format.
  • Improving Performance: Skipping unknown fields during message parsing can lead to performance gains, especially when dealing with large datasets.

How to Drop Unknown Fields

Here is how you can drop unknown fields while using the standard proto.UnmarshalOptions struct provided by the google.golang.org/protobuf/proto package. Here’s how to do it in your Go code:

import (
	"google.golang.org/protobuf/proto"
	...
)

// Configure unmarshalling options to discard unknown fields
opts := proto.UnmarshalOptions{
	DiscardUnknown: true,
}

// Use the options when unmarshalling incoming messages
msg := &MyMessage{}
err := proto.Unmarshal(data, msg, opts)
if err != nil {
	// Handle error
}

By setting the DiscardUnknown field to true in the proto.UnmarshalOptions struct before unmarshalling incoming messages, you ensure that any unknown fields are ignored. This helps mitigate the security risks associated with unknown fields while processing internet-facing gRPC requests.

How to Drop Unknown Fields in Connect RPC Servers

package main

import (
	"log"
	"net/http"

	"golang.org/x/net/http2"
	"golang.org/x/net/http2/h2c"
	"go.akshayshah.org/connectproto"
)

func main() {
	greeter := &GreetServer{}
	mux := http.NewServeMux()
	path, handler := greetv1connect.NewGreetServiceHandler(
		greeter,
		// Add an option that customizes protobuf marshalling/unmarshalling behavior
		connectproto.WithBinary(
			proto.MarshalOptions{},
			proto.UnmarshalOptions{DiscardUnknown: true},
		),
		// Add an option to customize JSON marshalling/unmachalling
		connectproto.WithJSON(
			protojson.MarshalOptions{},
			protojson.UnmarshalOptions{DiscardUnknown: true},
		)
	)
	mux.Handle(path, handler)
	log.Fatal(http.ListenAndServe(
		"localhost:9000",
		h2c.NewHandler(mux, &http2.Server{}),
	))
}

In this example, connectproto.WithBinary ensures only messages with defined fields are processed, enhancing the security of your gRPC service. connectproto.WithJSON does the same thing but with JSON.

Additional Considerations

While dropping unknown fields is a valuable security practice, it’s important to consider potential trade-offs:

  • Backward compatibility: Clients using older versions of the proto schema will encounter errors if they rely on previously defined unknown fields.
  • Logging and Debugging: Dropping unknown fields might make it harder to identify the source of unexpected behavior during development or debugging.

In such cases, it’s recommended to document these trade-offs and have a clear versioning policy for your gRPC service and client applications.

Conclusion

Exposing gRPC services to the internet requires careful security considerations. By customizing protobuf encoding options, specifically by dropping unknown fields using proto.UnmarshalOptions, you can significantly improve the security posture of your service. Remember to weigh the benefits against potential drawbacks and implement a solution that aligns with your specific needs.