惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
L
Lohrmann on Cybersecurity
aimingoo的专栏
aimingoo的专栏
V
V2EX
S
Security Affairs
T
Threatpost
C
CXSECURITY Database RSS Feed - CXSecurity.com
IT之家
IT之家
J
Java Code Geeks
The Register - Security
The Register - Security
U
Unit 42
C
CERT Recently Published Vulnerability Notes
月光博客
月光博客
A
About on SuperTechFans
H
Hackread – Cybersecurity News, Data Breaches, AI and More
T
The Blog of Author Tim Ferriss
Cisco Talos Blog
Cisco Talos Blog
Project Zero
Project Zero
S
Schneier on Security
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
D
DataBreaches.Net
博客园 - 司徒正美
V
Vulnerabilities – Threatpost
T
Tor Project blog
Security Latest
Security Latest
T
The Exploit Database - CXSecurity.com
T
Threat Research - Cisco Blogs
Scott Helme
Scott Helme
Application and Cybersecurity Blog
Application and Cybersecurity Blog
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
M
MIT News - Artificial intelligence
云风的 BLOG
云风的 BLOG
小众软件
小众软件
L
LangChain Blog
Attack and Defense Labs
Attack and Defense Labs
Recent Commits to openclaw:main
Recent Commits to openclaw:main
P
Palo Alto Networks Blog
A
Arctic Wolf
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
C
Cyber Attacks, Cyber Crime and Cyber Security
博客园 - 叶小钗
D
Darknet – Hacking Tools, Hacker News & Cyber Security
L
LINUX DO - 最新话题
MongoDB | Blog
MongoDB | Blog
Webroot Blog
Webroot Blog
H
Hacker News: Front Page
Know Your Adversary
Know Your Adversary
Spread Privacy
Spread Privacy
AWS News Blog
AWS News Blog
Engineering at Meta
Engineering at Meta

C0reFast记事本

一个virtio-blk设备IO大小的问题 使用单台高配服务器(部署你的业务) 一个Solidigm P41 Plus的冷数据问题 测试ARP/ND双发效果的小工具 硬件Bonding卸载场景下的ARP/ND双发 “无限”套娃,在WSL的Docker中使用YOLOv11做目标检测 在三条指令内实现闰年判断 一个中断关闭时间太长导致的网络延迟问题 一个Linux新老内核对于硬件中断的统计差异 IPIP隧道导致的DPDK收包RSS队列不均匀问题 在KVM虚拟机中开启TSC作为时钟源 x86平台的TSC(TIME-STAMP COUNTER) 一个UFO引发的惨案 在RHEL系统中快速抓取火焰图 DIY一个8盘位全闪NAS 分享一个记录LSI RAID卡芯片和各个OEM对应卡型号列表的神贴 在移动硬盘盒上启用SSD的Trim功能 SPDK的“Reduce”块压缩算法 实现一个Kubernetes的身份认证代理服务
反序列化AWS/阿里云样式的基于Query的API请求
陈孚 · 2023-07-13 · via C0reFast记事本

对于比较了解云计算的人来说,一定接触过AWS、阿里云的API接口,这两者的API调用方式很相似,当然具体谁参考谁这里就不深究了。以给EC2/ECS添加Tag这个接口为例:

AWS:

https://ec2.amazonaws.com/?Action=CreateTags
&ResourceId.1=ami-1a2b3c4d
&ResourceId.2=i-1234567890abcdef0
&Tag.1.Key=webserver
&Tag.1.Value=
&Tag.2.Key=stack
&Tag.2.Value=Production
&AUTHPARAMS

阿里云:

https://ecs.aliyuncs.com/?Action=TagResources
&RegionId=cn-hangzhou
&ResourceId.1=i-bp1j6qtvdm8w0z1o0****
&ResourceId.2=i-bp1j6qtvdm8w0z1oP****
&ResourceType=instance
&Tag.1.Key=TestKey
&Tag.1.Value=TestKey
&<公共请求参数>

这种样式的接口设计,其实没有什么复杂的,相对比较特殊的地方在于,如果需要传入一个数组,则需要使用类似下标一样的Tag.N.Key这种格式进行传递,这个传递方式,和已有的一些诸如google/go-querystring的传递方式都不太相同,总之是个很特殊的设计。

如果需要写一个类似的服务,使用和这两家相同的API格式的话,针对这种数组格式的请求反序列化是个挺麻烦的事,而且找了一圈也没有类似的开源项目做这个。

今天借助ChatGPT写了一个反序列化函数,专门用来实现服务端对类似形态API的反序列化,通过这个函数可以很方便的将Query反序列化成一个对应的Struct:

package main

import (
	"encoding/json"
	"fmt"
	"net/url"
	"reflect"
	"strconv"
	"strings"
)

type TagRequest struct {
	Action       string   `query:"Action"`
	RegionID     string   `query:"RegionId"`
	ResourceIds  []string `query:"ResourceId"`
	ResourceType string   `query:"ResourceType"`
	Tags         []Tag    `query:"Tag"`
}

type Tag struct {
	Key   string `query:"Key"`
	Value string `query:"Value"`
}

func Unmarshal(queryStr string, output interface{}) error {
	values, err := url.ParseQuery(queryStr)
	if err != nil {
		return err
	}

	return unmarshalData(values, output)
}

func unmarshalData(values url.Values, output interface{}) error {
	outputVal := reflect.ValueOf(output)
	if outputVal.Kind() != reflect.Ptr {
		return fmt.Errorf("output must be a pointer")
	}

	outputElem := outputVal.Elem()
	outputType := outputElem.Type()

	for i := 0; i < outputType.NumField(); i++ {
		field := outputType.Field(i)
		tag := field.Tag.Get("query")
		if tag == "" {
			continue
		}

		value := values.Get(tag)
		fieldVal := outputElem.FieldByName(field.Name)

		if field.Type.Kind() == reflect.Slice {
			elemType := field.Type.Elem()
			if elemType.Kind() != reflect.Struct {
				prefix := tag + "."
				arrIndex := 1
				for {
					currKey := prefix + fmt.Sprint(arrIndex)
					currValue := values.Get(currKey)
					if currValue == "" {
						break
					}

					currSliceVal := reflect.ValueOf(currValue)
					fieldVal.Set(reflect.Append(fieldVal, currSliceVal))
					arrIndex++
				}
			} else {
				prefix := tag + "."
				objIndex := 1
				outer := true
				for outer {
					innerValues := make(url.Values)
					for innerKey, innerValue := range values {
						if strings.HasPrefix(innerKey, prefix+strconv.Itoa(objIndex)+".") {
							innerValues.Set(strings.TrimPrefix(innerKey, prefix+strconv.Itoa(objIndex)+"."), innerValue[0])
						}
					}
					if len(innerValues) == 0 {
						break
					}

					newStructPtr := reflect.New(elemType)
					err := unmarshalData(innerValues, newStructPtr.Interface())
					if err != nil {
						return err
					}
					fieldVal.Set(reflect.Append(fieldVal, newStructPtr.Elem()))

					objIndex++
				}
			}
		} else {
			fieldVal.Set(reflect.ValueOf(value))
		}
	}

	return nil
}

func main() {
	queryStr := "?Action=TagResources&RegionId=cn-hangzhou&ResourceId.1=i-bp1j6qtvdm8w0z1o0&ResourceId.2=i-bp1j6qtvdm8w0z1oP&ResourceType=instance&Tag.1.Key=TestKey&Tag.1.Value=TestValue&Tag.2.Key=TestKey&Tag.2.Value=TestValue"

	req := TagRequest{}
	err := Unmarshal(strings.TrimLeft(queryStr, "?"), &req)
	if err != nil {
		fmt.Println("Error:", err)
		return
	}

	jsonOutput, _ := json.MarshalIndent(req, "", "  ")
	fmt.Println("Unmarshaled output:", string(jsonOutput))
}

测试一下:

% go run main.go
Unmarshaled output: {
  "Action": "TagResources",
  "RegionID": "cn-hangzhou",
  "ResourceIds": [
    "i-bp1j6qtvdm8w0z1o0",
    "i-bp1j6qtvdm8w0z1oP"
  ],
  "ResourceType": "instance",
  "Tags": [
    {
      "Key": "TestKey",
      "Value": "TestValue"
    },
    {
      "Key": "TestKey",
      "Value": "TestValue"
    }
  ]
}

嗯,ChatGPT牛逼!为了方便大家使用,我创建了一个项目c0refast/aws-querystring,可以方便地作为库使用:

package main

import (
	"encoding/json"
	"fmt"
	"net/url"

	"github.com/c0refast/aws-querystring/query"
)

type TagRequest struct {
	Action       string   `query:"Action"`
	RegionID     string   `query:"RegionId"`
	ResourceIds  []string `query:"ResourceId"`
	ResourceType string   `query:"ResourceType"`
	Tags         []Tag    `query:"Tag"`
}

type Tag struct {
	Key   string `query:"Key"`
	Value string `query:"Value"`
}

func main() {
	queryStr := "Action=TagResources&RegionId=cn-hangzhou&ResourceId.1=i-bp1j6qtvdm8w0z1o0&ResourceId.2=i-bp1j6qtvdm8w0z1oP&ResourceType=instance&Tag.1.Key=TestKey&Tag.1.Value=TestValue&Tag.2.Key=TestKey&Tag.2.Value=TestValue"
	urlValues, _ := url.ParseQuery(queryStr)
	req := TagRequest{}
	err := query.BindQuery(urlValues, &req)
	if err != nil {
		fmt.Println("Error:", err)
		return
	}

	jsonOutput, _ := json.MarshalIndent(req, "", "  ")
	fmt.Println("Unmarshaled output:", string(jsonOutput))
}