惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

The GitHub Blog
The GitHub Blog
IT之家
IT之家
B
Blog RSS Feed
罗磊的独立博客
GbyAI
GbyAI
博客园 - Franky
Y
Y Combinator Blog
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Google DeepMind News
Google DeepMind News
博客园 - 聂微东
N
Netflix TechBlog - Medium
博客园 - 三生石上(FineUI控件)
人人都是产品经理
人人都是产品经理
U
Unit 42
博客园 - 叶小钗
Jina AI
Jina AI
MyScale Blog
MyScale Blog
雷峰网
雷峰网
B
Blog
Hugging Face - Blog
Hugging Face - Blog
Blog — PlanetScale
Blog — PlanetScale
Recent Announcements
Recent Announcements
腾讯CDC
酷 壳 – CoolShell
酷 壳 – CoolShell

kmcd.dev

Joining Buf Let The Gravity of Ashburn, Virginia The CPU Cost of Protobuf Varints in Go It Beating Go gRPC-Web Should Have Fixed gRPC Making Dynamic Protobuf Fast in Go Proxy, Record, and Mock gRPC APIs with FauxRPC Exploring Protocol Buffers Interactively Introducing ProtoDocs Ghost in the Shell: The Manga Behind the Anime The Hidden Cost of google.protobuf.Value Why Networking Built Its Own Data Modeling Language Zero-Friction Demos with WASM Let's Learn About BGP ConnectRPC: Where is it now? Building APIs with Contracts The Case for Greppable Code Unknown Fields in Protobuf IRC Log: Reactionary Faking protobuf data in Go Y'all are Sleeping on Mise-en-Place IRC Log: Standup 2 HTTP/2 From Scratch: Part 4 IRC Log: rm -rf /var/opt/gitlab/postgresql/data HTTP/2 From Scratch: Part 3 Building a Live BGP Map HTTP/2 From Scratch: Part 2 IRC Log: The Cloud Scale Incident
Dropping Unknown Fields in ConnectRPC
2024-04-02 · via kmcd.dev

gRPC, with its focus on performance and language neutrality, remains a popular choice for building microservices and APIs. But when exposing your gRPC service to the internet, there are a few security considerations to account for. Protobuf, the serialization format often used with gRPC, offers various encoding options that can significantly impact your service’s security posture.

One crucial optimization for internet-facing gRPC services is customizing the behavior towards unknown fields. I’ve talked about unknown fields in a previous post, so read that one if unknown fields are still a mystery to you and then come back here. By default, protobuf messages can contain fields that are not defined in the current version of the proto schema. While convenient for development and can help with forward compatibility, this poses a security risk in a public environment.

Here’s why you should consider dropping unknown fields when exposing gRPC to the internet:

  • Preventing Malicious Data: Unknown fields can be exploited by malicious actors to inject unexpected data into your service. This could lead to potential security vulnerabilities like code injection or unexpected behavior.
  • Ensuring Compatibility: Uncontrolled unknown fields can cause compatibility issues if your clients are using different versions of the proto schema. Dropping them enforces stricter adherence to the defined message format.
  • Improving Performance: Skipping unknown fields during message parsing can lead to performance gains, especially when dealing with large datasets.

How to Drop Unknown Fields

Here is how you can drop unknown fields while using the standard proto.UnmarshalOptions struct provided by the google.golang.org/protobuf/proto package. Here’s how to do it in your Go code:

import (
	"google.golang.org/protobuf/proto"
	...
)

// Configure unmarshalling options to discard unknown fields
opts := proto.UnmarshalOptions{
	DiscardUnknown: true,
}

// Use the options when unmarshalling incoming messages
msg := &MyMessage{}
err := proto.Unmarshal(data, msg, opts)
if err != nil {
	// Handle error
}

By setting the DiscardUnknown field to true in the proto.UnmarshalOptions struct before unmarshalling incoming messages, you ensure that any unknown fields are ignored. This helps mitigate the security risks associated with unknown fields while processing internet-facing gRPC requests.

How to Drop Unknown Fields in Connect RPC Servers

package main

import (
	"log"
	"net/http"

	"golang.org/x/net/http2"
	"golang.org/x/net/http2/h2c"
	"go.akshayshah.org/connectproto"
)

func main() {
	greeter := &GreetServer{}
	mux := http.NewServeMux()
	path, handler := greetv1connect.NewGreetServiceHandler(
		greeter,
		// Add an option that customizes protobuf marshalling/unmarshalling behavior
		connectproto.WithBinary(
			proto.MarshalOptions{},
			proto.UnmarshalOptions{DiscardUnknown: true},
		),
		// Add an option to customize JSON marshalling/unmachalling
		connectproto.WithJSON(
			protojson.MarshalOptions{},
			protojson.UnmarshalOptions{DiscardUnknown: true},
		)
	)
	mux.Handle(path, handler)
	log.Fatal(http.ListenAndServe(
		"localhost:9000",
		h2c.NewHandler(mux, &http2.Server{}),
	))
}

In this example, connectproto.WithBinary ensures only messages with defined fields are processed, enhancing the security of your gRPC service. connectproto.WithJSON does the same thing but with JSON.

Additional Considerations

While dropping unknown fields is a valuable security practice, it’s important to consider potential trade-offs:

  • Backward compatibility: Clients using older versions of the proto schema will encounter errors if they rely on previously defined unknown fields.
  • Logging and Debugging: Dropping unknown fields might make it harder to identify the source of unexpected behavior during development or debugging.

In such cases, it’s recommended to document these trade-offs and have a clear versioning policy for your gRPC service and client applications.

Conclusion

Exposing gRPC services to the internet requires careful security considerations. By customizing protobuf encoding options, specifically by dropping unknown fields using proto.UnmarshalOptions, you can significantly improve the security posture of your service. Remember to weigh the benefits against potential drawbacks and implement a solution that aligns with your specific needs.