首页 笔记 图片 查字 
所属分类:Flink
浏览:34
内容:

参考:

https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeLabel.html

https://baijiahao.baidu.com/s?id=1763775102728430102 



要点:
Setup following properties in yarn-site.xml

Property                                    |    Value
yarn.node-labels.fs-store.root-dir          |    hdfs://namenode:port/path/to/store/node-labels/
yarn.node-labels.enabled                    |    true
yarn.node-labels.configuration-type  |    Set configuration type for node labels. Administrators can specify “centralized”, “delegated-centralized” or “distributed”. Default value is “centralized”.

Add cluster node labels list:
yarn rmadmin -addToClusterNodeLabels "label_1(exclusive=true/false),label_2(exclusive=true/false)", exclusive will be true by default
yarn cluster --list-node-labels

Remove node labels from YARN:
yarn rmadmin -removeFromClusterNodeLabels "<label>[,<label>,...]"

Add/modify node-to-labels mapping to YARN:
yarn rmadmin -replaceLabelsOnNode “node1[:port]=label1 node2=label2” [-failOnUnknownNodes]

为队列绑定标签:
修改capacity-scheduler.xml配置文件:

root.default队列使用所有标签。
yarn.scheduler.capacity.root.default.accessible-node-labels = *
root.small队列使用label3。
yarn.scheduler.capacity.root.small.accessible-node-labels = label3
root.default队列可以使用label2的60%的资源。
yarn.scheduler.capacity.root.default.accessible-node-labels.label2.capacity = 60
root.small队列可以使用label3的80%的资源。
yarn.scheduler.capacity.root.small.accessible-node-labels.label3.capacity = 80
root.default队列默认使用label3。
yarn.scheduler.capacity.root.default.default-node-label-expression = label3

刷新配置:
yarn rmadmin -refreshQueues