惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Engineering at Meta
Engineering at Meta
月光博客
月光博客
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
V
Visual Studio Blog
大猫的无限游戏
大猫的无限游戏
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
量子位
cs.CV updates on arXiv.org
cs.CV updates on arXiv.org
腾讯CDC
S
Securelist
Know Your Adversary
Know Your Adversary
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
博客园 - 【当耐特】
V2EX - 技术
V2EX - 技术
J
Java Code Geeks
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
Hugging Face - Blog
Hugging Face - Blog
Cisco Talos Blog
Cisco Talos Blog
酷 壳 – CoolShell
酷 壳 – CoolShell
阮一峰的网络日志
阮一峰的网络日志
美团技术团队
W
WeLiveSecurity
S
SegmentFault 最新的问题
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
宝玉的分享
宝玉的分享
博客园 - Franky
S
Security Affairs
博客园 - 司徒正美
V
V2EX
K
Kaspersky official blog
T
Threatpost
NISL@THU
NISL@THU
博客园 - 叶小钗
Help Net Security
Help Net Security
PCI Perspectives
PCI Perspectives
IT之家
IT之家
小众软件
小众软件
K
KPMG report finds enterprise disconnect between AI and its ROI | CIO
Last Week in AI
Last Week in AI
Jina AI
Jina AI
爱范儿
爱范儿
罗磊的独立博客
N
News and Events Feed by Topic
博客园 - 三生石上(FineUI控件)
有赞技术团队
有赞技术团队
P
Proofpoint News Feed
L
LINUX DO - 热门话题
Google Online Security Blog
Google Online Security Blog
雷峰网
雷峰网
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint

Posts on WKLKEN THINKING

apisix 中的 lrucache apisix 中的服务发现机制 apisix 中的负载均衡 apisix etcd机制 聊聊框架 关于 k8s 的 zero downtime deployment 一些建议 apisix 遇到的一些问题 关于在除夕前一天换了一个洗衣机的故事 Django DRF 性能优化 DRF 的一些实践 Part1: Serializer DRF继承关系图 Better Code: 关于接口的灵活性 新的仓库: wklken/naming 缓存使用的一些经验 Better Code: 抽象: 可扩展性与可维护性的抉择 Better Code: 异常时, 该提示用户哪些信息? Better Code: 更好的异常日志打印 Go: some libs Go: go-redis/cache升级的坑 Go: logrus性能提升 Go: gin validation 远程办公的一点总结 Go: 开发过程中的一些bug 项目管理实践: 风险驱动开发 Go: 一种error wrap调用链处理方式 漫谈技术选型 Go: 基于 apitest 做handler层单元测试 Go: go-sql-driver interpolateparams参数优化 [分享]深度工作 你需要更多的思考时间 Django项目重构小结 工作七年小结: 学习,生活及其他 [分享]bash日常: bash-utils 极客时间推广海报 2017总结: 予时光以意义 k8s APIServer源码: api注册主体流程 k8s APIServer源码: 服务启动 k8s APIServer源码: go-restful框架 重构 - 读书笔记(Python示例) 写给新人的沟通建议 vim 杂谈 - 关于快速编辑 vim 杂谈 - 关于移动 读书笔记-重构: 章11 处理概括关系 读书笔记-重构: 章10 简化函数调用 读书笔记-重构: 章9 简化表达式 读书笔记-重构: 章8 重新组织数据 读书笔记-重构: 章7 在对象之间搬移特性 读书笔记-重构: 章6 重新组织函数 Python 代码规范小结 [分享]关于vim ElasticSearch集群部署文档 Logstash+ElasticSearch处理mysql慢查询日志 [分享]关于代码调试DE那些事 Logstash+ElasticSearch+Kibana- 实现相对通用的数据收集分析 ELK维护的一些点(二) [分享]Python源码剖析-数据结构 一些Centos Python生产环境的部署命令 摘录<<6个月学会任何一种外语>> ELK 维护的一些点 也许是一个新的开始 一些vim的个性化配置 读书笔记-调试九法 这段时间的一些想法 Python 源码阅读 - 垃圾回收机制 我为什么要写博客 APUE笔记-第一章 UNIX基础知识 Python源码阅读-闭包的实现 Python源码阅读-内存管理机制(二) Python源码阅读-内存管理机制(一) Python-基础-数据结构小结 '活动'设计的一些trick 一些简单的Python测试题 我的tmux配置及说明【k-tmux】 Review and Restart 工作四周年小结 vim插件: surround & repeat[成对符号编辑] vim插件: gundo[时光机] vim插件: expand-region[区域选中] vim插件: quickrun[快速执行] vim插件: trailing-whitespace[行尾空格处理] vim插件: closetag[成对标签补全] vim插件: ctrlp[文件搜索] vim插件: airline[状态栏增强] vim插件: theme[主题] vim插件: tagbar[大纲式导航] vim插件: nerdcommenter[快速注释] vim插件: rainbow_parentheses[括号高亮] vim插件: syntastic[语法检查] vim插件: delimitmate[符号自动补全] vim插件: matchit[成对标签跳转] vim插件: easy-align[快速对齐] vim插件: multiple-cursors[多光标操作] vim插件: vim-signature[快速标记跳转] vim插件: easymotion[快速跳转] vim插件: vundle[管理插件] Elasticsearch几个问题的解决 分享一份 Vim 简介PPT k-vim 更新9.0版本 关于知识管理工具的思考 Logstash+ElasticSearch+Kibana处理nginx访问日志
k8s APIServer源码: api注册详细细节
2017-09-23 · via Posts on WKLKEN THINKING

基于版本 1.6.7

前面介绍了, api注册过程

问题: go-restful githubroute中, handlerpath是如何绑定在一起的? handler在哪里定义的?

/api为例

前面介绍到/api/apis分别注册加入到Container, 而最终, 二者调用installer.Install(ws). 这一步, 我们需要进一步了解细节.

webservice.add(route)

问题: 构建Route加入到WebService在哪里处理的?

  • vendor/k8s.io/apiserver/pkg/endpoints/installer.go
func (a *APIInstaller) Install(ws *restful.WebService) (apiResources []metav1.APIResource, errors []error) {
	// Register the paths in a deterministic (sorted) order to get a deterministic swagger spec.
	paths := make([]string, len(a.group.Storage))
	var i int = 0
	for path := range a.group.Storage {
		paths[i] = path
		i++
	}
	sort.Strings(paths)
	for _, path := range paths {
		apiResource, err := a.registerResourceHandlers(path, a.group.Storage[path], ws, proxyHandler)
	}
}
  • apiserver/pkg/endpoints/installer.go

仅摘录部分核心代码, 这里, 获取handler之后, 构建route, 然后加入到webservice

func (a *APIInstaller) registerResourceHandlers(path string, storage rest.Storage, ws *restful.WebService, proxyHandler http.Handler) (*metav1.APIResource, error) {
   ...
	creater, isCreater := storage.(rest.Creater)
	....
	actions = appendIf(actions, action{"POST", resourcePath, resourceParams, namer, false}, isCreater)
	case "POST":

	 // 获取handler
    handler = handlers.CreateResource(creater, reqScope, a.group.Typer, admit) // => next
    // 构建route,  action.Path -> handler
	  route := ws.POST(action.Path).To(handler).
				Doc(doc).
				Param(ws.QueryParameter("pretty", "If 'true', then the output is pretty printed.")).
				Operation("create"+namespaced+kind+strings.Title(subresource)+operationSuffix).
				Produces(append(storageMeta.ProducesMIMETypes(action.Verb), mediaTypes...)...).
				Returns(http.StatusOK, "OK", versionedObject).
				Reads(versionedObject).
				Writes(versionedObject)
		addParams(route, action.Params)
		// 添加route到webservice
		ws.Route(route)
}

create handler

  • vendor/k8s.io/apiserver/pkg/endpoints/handlers/rest.go
// CreateResource returns a function that will handle a resource creation.
func CreateResource(r rest.Creater, scope RequestScope, typer runtime.ObjectTyper, admit admission.Interface) restful.RouteFunction {
	return createHandler(&namedCreaterAdapter{r}, scope, typer, admit, false)
}

func createHandler(r rest.NamedCreater, scope RequestScope, typer runtime.ObjectTyper, admit admission.Interface, includeName bool) restful.RouteFunction {
	return func(req *restful.Request, res *restful.Response) {
		original := r.New()  // => here
		responsewriters.WriteObject(http.StatusCreated, scope.Kind.GroupVersion(), scope.Serializer, result, w, req.Request)
	}
}

即, 最终handler执行时, 调用的是rest.Creater.New()

这里的 Creater 是一个interface

# vendor/k8s.io/apiserver/pkg/registry/rest/rest.go

// Creater is an object that can create an instance of a RESTful object.
type Creater interface {
	// New returns an empty object that can be used with Create after request data has been put into it.
	// This object must be a pointer type for use with Codec.DecodeInto([]byte, runtime.Object)
	New() runtime.Object

	// Create creates a new version of a resource.
	Create(ctx genericapirequest.Context, obj runtime.Object) (runtime.Object, error)
}

回到最初的问题

最终, handler调用的是rest.Creater.New()

creater声明的位置

  • apiserver/pkg/endpoints/installer.go
	creater, isCreater := storage.(rest.Creater)

这里, 想要知道handler最终调用的是哪里定义的方法, 我们需要分析storage的来源

第一步: 链路分析

调用链

// pkg/master/master.go
// => got: apiGroupInfo 初始化
func (m *Master) InstallLegacyAPI(c *Config, restOptionsGetter generic.RESTOptionsGetter, legacyRESTStorageProvider corerest.LegacyRESTStorageProvider) {
	legacyRESTStorage, apiGroupInfo, err := legacyRESTStorageProvider.NewLegacyRESTStorage(restOptionsGetter)
	m.GenericAPIServer.InstallLegacyAPIGroup(genericapiserver.DefaultLegacyAPIPrefix, &apiGroupInfo)
}

// vendor/k8s.io/apiserver/pkg/server/genericapiserver.go
func (s *GenericAPIServer) InstallLegacyAPIGroup(apiPrefix string, apiGroupInfo *APIGroupInfo) error {
    s.installAPIResources(apiPrefix, apiGroupInfo)
}

// vendor/k8s.io/apiserver/pkg/server/genericapiserver.go
// NOTE => apigroup TO apigroupversion
// => got: apigroupversion
func (s *GenericAPIServer) installAPIResources(apiPrefix string, apiGroupInfo *APIGroupInfo) error {
	for _, groupVersion := range apiGroupInfo.GroupMeta.GroupVersions {
		apiGroupVersion := s.getAPIGroupVersion(apiGroupInfo, groupVersion, apiPrefix)
		apiGroupVersion.InstallREST(s.HandlerContainer.Container);	}
}

// vendor/k8s.io/apiserver/pkg/server/genericapiserver.go
// => got: APIGroupVersion.Storage = make(map[string]rest.Storage
//         APIGroupVersion.Storage[path] = apiGroupInfo.VersionedResourcesStorageMap[groupVersion.Version][path]
func (s *GenericAPIServer) getAPIGroupVersion(apiGroupInfo *APIGroupInfo, groupVersion schema.GroupVersion, apiPrefix string) *genericapi.APIGroupVersion {
	storage := make(map[string]rest.Storage)
	for k, v := range apiGroupInfo.VersionedResourcesStorageMap[groupVersion.Version] {
		storage[strings.ToLower(k)] = v
	}
	version := s.newAPIGroupVersion(apiGroupInfo, groupVersion)
	version.Root = apiPrefix
	version.Storage = storage
	return version
}

// vendor/k8s.io/apiserver/pkg/endpoints/groupversion.go
// => got: installer.group = APIGroupVersion
func (g *APIGroupVersion) InstallREST(container *restful.Container) error {
    installer := g.newInstaller()
    installer.Install(ws)
}

func (g *APIGroupVersion) newInstaller() *APIInstaller {
	prefix := path.Join(g.Root, g.GroupVersion.Group, g.GroupVersion.Version)
	installer := &APIInstaller{
		group:             g,           // group = APIGroupVersion
		prefix:            prefix,
		minRequestTimeout: g.MinRequestTimeout,
	}
	return installer
}

// vendor/k8s.io/apiserver/pkg/endpoints/installer.go
// got: a.group.Storage[path] = APIInstaller.group.Storage[path] = APIGroupVersion.Storage[path]
//      APIGroupVersion.Storage[path] = apiGroupInfo.VersionedResourcesStorageMap[groupVersion.Version][path]
func (a *APIInstaller) Install(ws *restful.WebService) (apiResources []metav1.APIResource, errors []error) {
	// Register the paths in a deterministic (sorted) order to get a deterministic swagger spec.
	paths := make([]string, len(a.group.Storage))
	var i int = 0
	for path := range a.group.Storage {
		paths[i] = path
		i++
	}
	sort.Strings(paths)
	for _, path := range paths {
		apiResource, err := a.registerResourceHandlers(path, a.group.Storage[path], ws, proxyHandler)
	}
}

// apiserver/pkg/endpoints/installer.go
// got: storage = apiGroupInfo.VersionedResourcesStorageMap[groupVersion.Version][path]
func (a *APIInstaller) registerResourceHandlers(path string, storage rest.Storage, ws *restful.WebService, proxyHandler http.Handler) (*metav1.APIResource, error) {

	creater, isCreater := storage.(rest.Creater)
}

到了这里, 其实有了一个清晰的结论

// apiGroupInfo.VersionedResourcesStorageMap
storage = apiGroupInfo.VersionedResourcesStorageMap[groupVersion.Version][path]
creater, isCreater := storage.(rest.Creater)

此时, 我们再反向寻找apiGroupInfo初始化的位置

第二步: apiGroupInfo 初始化

  • pkg/master/master.go
func (m *Master) InstallLegacyAPI(c *Config, restOptionsGetter generic.RESTOptionsGetter, legacyRESTStorageProvider corerest.LegacyRESTStorageProvider) {
	legacyRESTStorage, apiGroupInfo, err := legacyRESTStorageProvider.NewLegacyRESTStorage(restOptionsGetter) // => next
	m.GenericAPIServer.InstallLegacyAPIGroup(genericapiserver.DefaultLegacyAPIPrefix, &apiGroupInfo)
}
  • pkg/registry/core/rest/storage_core.go
func (c LegacyRESTStorageProvider) NewLegacyRESTStorage(restOptionsGetter generic.RESTOptionsGetter) (LegacyRESTStorage, genericapiserver.APIGroupInfo, error) {
  // 初始化: VersionedResourcesStorageMap
	apiGroupInfo := genericapiserver.APIGroupInfo{
		GroupMeta:                    *api.Registry.GroupOrDie(api.GroupName),
		VersionedResourcesStorageMap: map[string]map[string]rest.Storage{},
		Scheme:                      api.Scheme,
		ParameterCodec:              api.ParameterCodec,
		NegotiatedSerializer:        api.Codecs,
		SubresourceGroupVersionKind: map[string]schema.GroupVersionKind{},
	}
	// ......

	// 初始化了一个restStorage的map,然后赋值给APIGroupInfo.VersionedResourcesStorageMap["v1"]
	restStorageMap := map[string]rest.Storage{
		"pods":             podStorage.Pod,
		"pods/attach":      podStorage.Attach,
		"pods/status":      podStorage.Status,
		"services":        serviceRest.Service,
		"nodes":        nodeStorage.Node,
		.....
	}

	apiGroupInfo.VersionedResourcesStorageMap["v1"] = restStorageMap

	return restStorage, apiGroupInfo, nil
}

apiGroupInfo.VersionedResourcesStorageMap["v1"] = map[string]rest.Storage{
		"pods":             podStorage.Pod,
		"pods/attach":      podStorage.Attach,
		"pods/status":      podStorage.Status,
		"services":        serviceRest.Service,
		"nodes":        nodeStorage.Node,
		.....
	}

此时, 根据

// apiGroupInfo.VersionedResourcesStorageMap
storage = apiGroupInfo.VersionedResourcesStorageMap[groupVersion.Version][path]
creater, isCreater := storage.(rest.Creater)

我们可以得到

storage = apiGroupInfo.VersionedResourcesStorageMap["v1"]["pods"]
// equals
storage = podStorage.Pod
creater, isCreater := (podStorage.Pod).(rest.Creater)

然后, 我们再看下podStorage.Pod的实现

第三步: podStorage.Pod

  • pkg/registry/core/pod/storage/storage.go
type PodStorage struct {
	Pod         *REST
  ...
}

// REST implements a RESTStorage for pods
type REST struct {
	*genericregistry.Store     // => NOTE
	proxyTransport http.RoundTripper
}

即, PodStorage.Pod 类型是 REST, 而REST.genericregistry.Store, 其定义文件中存在

  • vendor/k8s.io/apiserver/pkg/registry/generic/registry/store.go
// New implements RESTStorage.New.
func (e *Store) New() runtime.Object {
	return e.NewFunc()
}

func (e *Store) Create(ctx genericapirequest.Context, obj runtime.Object) (runtime.Object, error) {
}

即,

storage = apiGroupInfo.VersionedResourcesStorageMap["v1"]["pods"]
// equals
storage = podStorage.Pod
creater, isCreater := (podStorage.Pod).(rest.Creater)

// equals
creater, isCreater := (REST).(rest.Creater)
creater, isCreater := (*genericregistry.Store).(rest.Creater)

第四步: creater.New()

  • vendor/k8s.io/apiserver/pkg/registry/generic/registry/store.go
// New implements RESTStorage.New.
func (e *Store) New() runtime.Object {
	return e.NewFunc()
}
  • pkg/registry/core/pod/storage/storage.go
func NewStorage(optsGetter generic.RESTOptionsGetter, k client.ConnectionInfoGetter, proxyTransport http.RoundTripper, podDisruptionBudgetClient policyclient.PodDisruptionBudgetsGetter) PodStorage {

	store := &genericregistry.Store{
		NewFunc:     func() runtime.Object { return &api.Pod{} },
		....
	}
}

// pkg/api/types.go
type Pod struct {
	metav1.TypeMeta
	// +optional
	metav1.ObjectMeta

	// Spec defines the behavior of a pod.
	// +optional
	Spec PodSpec

	// Status represents the current information about a pod. This data may not be up
	// to date.
	// +optional
	Status PodStatus
}

etcd相关的, 在后面介绍