惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

S
Schneier on Security
G
GRAHAM CLULEY
T
Threat Research - Cisco Blogs
Spread Privacy
Spread Privacy
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
NISL@THU
NISL@THU
Latest news
Latest news
N
News and Events Feed by Topic
量子位
爱范儿
爱范儿
M
MIT News - Artificial intelligence
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
GbyAI
GbyAI
A
Arctic Wolf
Google Online Security Blog
Google Online Security Blog
W
WeLiveSecurity
S
Security Affairs
Webroot Blog
Webroot Blog
B
Blog
N
Netflix TechBlog - Medium
Hugging Face - Blog
Hugging Face - Blog
I
InfoQ
博客园 - 司徒正美
H
Heimdal Security Blog
罗磊的独立博客
L
LINUX DO - 最新话题
Security Archives - TechRepublic
Security Archives - TechRepublic
博客园_首页
G
Google Developers Blog
Google DeepMind News
Google DeepMind News
Attack and Defense Labs
Attack and Defense Labs
The Hacker News
The Hacker News
Y
Y Combinator Blog
C
CERT Recently Published Vulnerability Notes
C
Cybersecurity and Infrastructure Security Agency CISA
Cyberwarzone
Cyberwarzone
C
Cisco Blogs
V
Vulnerabilities – Threatpost
T
The Exploit Database - CXSecurity.com
博客园 - 聂微东
Microsoft Azure Blog
Microsoft Azure Blog
人人都是产品经理
人人都是产品经理
The Register - Security
The Register - Security
PCI Perspectives
PCI Perspectives
The GitHub Blog
The GitHub Blog
Jina AI
Jina AI
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
O
OpenAI News
S
Security @ Cisco Blogs
Google DeepMind News
Google DeepMind News

Ivan on Containers, Kubernetes, and Server-Side

A grounded take on agentic coding for production environments Server-Side Playgrounds Reimagined: Build, Boot, and Network Your Own Virtual Labs [not a] Kubernetes 101 - Pods, Deployments, and Services As an Attempt To Automate Age-Old Infra Patterns JavaScript or TypeScript? How To Benefit From the Dichotomy On Software Design... and Good Writing Building a Firecracker-Powered Course Platform To Learn Docker and Kubernetes How To Publish a Port of a Running Container What Actually Happens When You Publish a Container Port A Visual Guide to SSH Tunnels: Local and Remote Port Forwarding Debugging Containers Like a Pro Docker: How To Debug Distroless And Slim Containers How To Extract Container Image Filesystem Using Docker | iximiuz Labs In Pursuit of Better Container Images: Alpine, Distroless, Apko, Chisel, DockerSlim, oh my! How To Start Programming In Go: Advice For Fellow DevOps Engineers Kubernetes Ephemeral Containers and kubectl debug Command How To Develop Kubernetes CLIs Like a Pro Docker Container Commands Explained: Understand, Don't Memorize | iximiuz Labs Learning Docker with Docker - Toying With DinD For Fun And Profit How To Extend Kubernetes API - Kubernetes vs. Django The Influence of Plumbing on Programming How To Call Kubernetes API from Go - Types and Common Machinery How To Call Kubernetes API using Simple HTTP Client Kubernetes API Basics - Resources, Kinds, and Objects OpenFaaS - Run Containerized Functions On Your Own Terms Learning Containers From The Bottom Up Docker Containers vs. Kubernetes Pods - Taking a Deeper Look | iximiuz Labs Learn-by-Doing Platforms for Dev, DevOps, and SRE Folks How HTTP Keep-Alive can cause TCP race condition How to Work with Container Images Using ctr | iximiuz Labs Multiple Containers, Same Port, no Reverse Proxy... Exploring Go net/http Package - On How Not To Set Socket Options Disposable Local Development Environments with Vagrant, Docker, and Arkade DevOps, SRE, and Platform Engineering My Choice of Programming Languages Prometheus Is Not a TSDB How to learn PromQL with Prometheus Playground Prometheus Cheat Sheet - Basics (Metrics, Labels, Time Series, Scraping) Rust - Writing Parsers With nom Parser Combinator Framework pq - parse and query log files as time series Prometheus Cheat Sheet - Moving Average, Max, Min, etc (Aggregation Over Time) Prometheus Cheat Sheet - How to Join Multiple Metrics (Vector Matching) The Need For Slimmer Containers Understanding Rust Privacy and Visibility Model Bridge vs. Switch: Takeaways from a Real Data Center Tour | iximiuz Labs From LAN to VXLAN: Networking Basics for Non-Network Engineers | iximiuz Labs KiND - How I Wasted a Day Loading Local Docker Images Go, HTTP handlers, panic, and deadlocks Exploring Kubernetes Operator Pattern Making Sense Out Of Cloud Native Buzz Service Discovery in Kubernetes: Combining the Best of Two Worlds How Container Networking Works: Building a Bridge Network From Scratch | iximiuz Labs Traefik: canary deployments with weighted load balancing Service Proxy, Pod, Sidecar, oh my! You Need Containers To Build Images You Don't Need an Image To Run a Container Not Every Container Has an Operating System Inside Working with container images in Go Master Go While Learning Containers Implementing Container Runtime Shim: Interactive Containers How to use Flask with gevent (uWSGI and Gunicorn editions) My 10 Years of Programming Experience Implementing Container Runtime Shim: First Code Implementing Container Runtime Shim: runc Kubernetes Repository On Flame Dealing with process termination in Linux (with Rust examples) conman - [the] Container Manager: Inception Journey From Containerization To Orchestration And Beyond Linux PTY - How docker attach and docker exec Commands Work Inside Illustrated introduction to Linux iptables From Docker Container to Bootable Linux Disk Image Пишем свой веб-сервер на Python: протокол HTTP 9001 способ создать веб-сервер на Python Explaining async/await in 200 lines of code Explaining event loop in 100 lines of code Save the day with gevent Пишем свой веб-сервер на Python: процессы, потоки и асинхронный I/O Truly optional scalar types in protobuf3 (with Go examples) Node.js Writable streams distilled Node.js Readable streams distilled How to on starting processes (mostly in Linux) Дайджест интересных ссылок – Июль 2016 Пишем свой веб-сервер на Python: сокеты Наследование в JavaScript Мастерить!
API Developers Never REST
Ivan Velichko · 2020-11-21 · via Ivan on Containers, Kubernetes, and Server-Side

Disclaimer: despite the controversial title, this article is not trying to show that RPC is a superior approach to REST, or GraphQL is superior to RPC. Instead, the goal of the article is to give you an overview of the approaches, their strengths and weaknesses. The final choice anyway will be a trade-off.

Even though HTTP is an application layer (i.e. L7), protocol, when it comes to API development, HTTP de facto plays the role of a lower-level transport mechanism.

There are multiple conceptually different approaches on how to implement an API on top of HTTP:

  • REST
  • RPC
  • GraphQL

...but the actual list of things an average API developer needs to be aware of is not limited by these three dudes. There are also JSON, gRPC, protobuf, and many other terms in the realm. Let's try to sort them out, once and for all!

What is REST? What is RPC? What is GraphQL? What is the difference between REST, RPC, and GraphQL?

Level up your server-side game — join 20,000 engineers getting insightful learning materials straight to their inbox.

Representational state transfer (REST)

First and foremost, REST is just a software architectural style. It's a set of design constraints, not a concrete protocol. REST relies on the idea of a resource. I.e. a REST API is comprised of a set of resources (nouns) and a fairly limited number of actions (verbs) to interact with these resources (fetch, create, update, delete, etc). It's ideologically very close to the original HTTP design heavily based on the concepts of resources (URLs) and methods (GET, POST, PUT, DELETE). Thus, from the implementation standpoint mapping of the REST model to the HTTP protocol is relatively straightforward:

# Create new book
POST http://myapi.com/books/ (author="R. Feynman", year=1975) -> book_id

# Get book with ID = 1
GET http://myapi.com/books/1 () -> (id=1, author="R. Feynman", year=1975)

# Update book with ID = 1
PUT http://myapi.com/books/1 (id=1, author="Richard Feynman", year=1975) -> (...)

# Delete book with ID = 1
DELETE http://myapi.com/books/1 () -> null

Probably every modern web framework provides all the needed facilities to build a RESTful web service out of the box. And from the client-standpoint, calling a REST API is pretty trivial - it just needs to be able to send the specified HTTP methods to the set of predefined URLs.

However, from the API designer standpoint, it may be not so obvious how to express a real-world domain model in terms of resources (i.e. nouns) keeping the number of allowed actions (i.e. verbs) limited. Surprisingly, in spite of a super-widespread (ab)use of the word REST nowadays, not so many contemporary APIs can be actually considered truly RESTful. Trying to play a purist while designing a RESTful web service can lead you to a really clunky API design.

Remote procedure call (RPC)

Similarly, RPC is also an API design technique. RPC focuses on the idea of actions (verbs) and that often makes the involved resources (nouns) fairly primitive and ad hoc. For every procedure or transaction in the business model an API designer simply adds an RPC endpoint:

# Create new book
createBook(author, year) -> book_id

# Get book by ID
getBook(book_id) -> book

# Change book's author
setBookAuthor(book_id, author) -> null

# Delete book by ID
deleteBook(book_id) -> null

Under the hood, there should be another layer mapping such procedure calls to the HTTP requests. E.g. setBookAuthor(1, "Richard Feynman") can be mapped to something like:

POST http://myapi.com/set-book-author/ (book_id=1, author="Richard Feynman") -> null

Now, let's compare the tasks of changing the author's name in the REST and RPC worlds. While the implementation seems trivial in the case of RPC, there is a bunch of controversial questions that need to be answered in case of the RESTful implementation. If the author's name is an attribute of the book, should we provide the full book object with the modified author field while sending PUT /books/1? What if there is no full book object around on the client-side? Should we fetch the book first or can we just send only the book ID and the new author name? But what's about the rest of the attributes on the server-side? Should they be zeroed out if an almost empty PUT request arrives (that would be disastrous but fully aligned with the REST principles decision)? Or maybe we should give up on the HTTP PUT method and start using PATCH (have you ever heard of it)?

Luckily, if we follow the RPC approach, we can simply omit most of the questions from above. But of course, there is always a downside - a typical RPC API usually consists of a pretty high number of custom procedures. Obviously, it makes the mapping of the RPC model to the HTTP layer a non-trivial task. The good thing is that API designers rarely need to think of this part at all. There are a lot of libraries implementing the RPC layer on top of HTTP and some of them are truly prominent (yes, I'm talking about Google's gRPC and Facebook's Thrift). But there is another, probably much more complicated problem with the RPC approach...

A dozen rich REST resources combined with 3-5 HTTP methods usually can cover a hundred use cases. REST API deliberately brings a regular structure to the domain model making its growth and evolution much more controllable. On contrary, RPC APIs grow organically. Introducing a new use case often require adding a few more API endpoints to the already bloated list. Since there is no entity-oriented structure enforced by the API design, over a relatively short period of time the number of RPC calls may exceed the maximum level of complexity a team can handle.

GraphQL

By this time, we already learned that neither RPC nor REST approach is ideal. REST suffers from over and under fetching problem and may lead to a significant mental overhead on the design phase. But if we try to avoid the entity-oriented design by replacing it with a hundred ad hoc RPC endpoints, over time the maintenance of the overgrown RPC API may become a nightmare.

GraphQL tries to address the weaknesses of both techniques. Assuming the entity-oriented domain model contributes to the peace of mind of developers in the long term, the GraphQL approach starts from defining a schema, i.e. set of resources (i.e. nouns) and their interconnections. Sounds like a graph, doesn't it? Having a formal scheme definition, GraphQL bumps a fairly complex server and client on top of it. The thick client allows querying (QL stands for query language) custom and compound resources. The thick server knows how to populate the response based on the client's query and the domain schema.

Thus, GraphQL API basically consists of a single endpoint. I.e. no more bloated APIs. At the same time, its super-flexible and tailored queries help to avoid fetching unnecessary data from the server. Additionally, developers still benefit from the formal entity-oriented domain schema. But magic always comes at a price (c). And the price in that case is the utter complexity of the GraphQL client and server sides. So, yeah, it's all about trade-offs...

What's up with protobuf, JSON, etc?

What could help with wrapping your head around the API development realm is the proper categorization of the moving parts. At this time we already know that REST and RPC are just architectural styles. And GraphQL, well... I tend to consider it as a style as well, even though technically it's a formal language backed by a runtime.

But if GraphQL is backed by a particular implementation, we probably could expect the same from RPC and REST. And indeed, there is a bunch of prominent RPC frameworks - gRPC, Apache Thrift, and Apache Avro to name a few. Surprisingly though, there seem to be no clear leaders among REST frameworks. Probably because REST is technically super-close to HTTP and almost every web framework already plays well with it.

Hm, but what's the deal with protobuf? Apparently, it's just one of the ways to serialize your data before it will be sent over the wire or stored somewhere. It just so happened that gRPC, a particular RPC framework, fully relies on protobuf. And since they often used hand in hand and have been released altogether, lots of people negligently use the terms gRPC and protobuf interchangeably. However, it's perfectly fine to use protobuf in your RESTful API as an encoding format and we do it currently at work for some of our services. Thus, protobuf should not be compared with REST or RPC frameworks but instead, it can be compared with XML or JSON.

To confuse you even further, Apache Thrift has its own serialization format called... Thrift! I.e. there is gRPC over protobuf and Thrift over Thrift 🙈

Summarizing, there are different architectural styles to design an HTTP API. The three most popular ones are REST, RPC, and GraphQL. Every style in turns can be implemented in many ways and there is a bunch of well-known frameworks such as gRPC or Apache Thrift. Under the hood, they rely on lower-level mechanisms like data serialization (protobuf, JSON, XML) or protocols (JSON-RPC, XML-RPC). And these lower-level bits sometimes can be reused between styles making the whole realm of API development looking pretty convoluted at first sight.

Further reading

Level up your server-side game — join 20,000 engineers getting insightful learning materials straight to their inbox: