参考:
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeLabel.html
https://baijiahao.baidu.com/s?id=1763775102728430102
要点:
Setup following properties in yarn-site.xml
Property | Value
yarn.node-labels.fs-store.root-dir | hdfs://namenode:port/path/to/store/node-labels/
yarn.node-labels.enabled | true
yarn.node-labels.configuration-type | Set configuration type for node labels. Administrators can specify “centralized”, “delegated-centralized” or “distributed”. Default value is “centralized”.
Add cluster node labels list:
yarn rmadmin -addToClusterNodeLabels "label_1(exclusive=true/false),label_2(exclusive=true/false)", exclusive will be true by default
yarn cluster --list-node-labels
Remove node labels from YARN:
yarn rmadmin -removeFromClusterNodeLabels "<label>[,<label>,...]"
Add/modify node-to-labels mapping to YARN:
yarn rmadmin -replaceLabelsOnNode “node1[:port]=label1 node2=label2” [-failOnUnknownNodes]
为队列绑定标签:
修改capacity-scheduler.xml配置文件:
root.default队列使用所有标签。
yarn.scheduler.capacity.root.default.accessible-node-labels = *
root.small队列使用label3。
yarn.scheduler.capacity.root.small.accessible-node-labels = label3
root.default队列可以使用label2的60%的资源。
yarn.scheduler.capacity.root.default.accessible-node-labels.label2.capacity = 60
root.small队列可以使用label3的80%的资源。
yarn.scheduler.capacity.root.small.accessible-node-labels.label3.capacity = 80
root.default队列默认使用label3。
yarn.scheduler.capacity.root.default.default-node-label-expression = label3
刷新配置:
yarn rmadmin -refreshQueues