Datax往hdfs写数据配置HA高可用
问题描述:
datax往hdfs写数据如何配置高可用
问题解析:
datax官网有相关配置,但是太过于笼统不够详细
解决方案:(nameservice1)
"defaultFS":"hdfs://nameservice1",
"hadoopConfig":{
"dfs.nameservices": "nameservice1",
"dfs.ha.namenodes.nameservice1": "namenode142,namenode35",
"dfs.namenode.rpc-address.nameservice1.namenode142": "hadoop-prod-datanode7:8020",
"dfs.namenode.rpc-address.nameservice1.namenode35": "hadoop-prod-namenode:8020",
"dfs.client.failover.proxy.provider.nameservice1": "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider"
},
每个参数分别对用hdfs-site.xml文件配置信息