running beyond virtual memory limits 错误


------------恢复内容开始------------

1.在练习hive跑任务过程出现如下错误:

2.根据报错信息:

    hadoop运行时使用的虚拟内存不足,从机上运行的Container想用更多的虚拟内存,而被NodeManager kill ,致使任务失败

3.解决方法:

   进入hadoop安装目录/etc/hadoop/

  3.1.在yarn-site.xml文件中添加如下配置: 



  yarn.nodemanager.vmem-check-enabled
  false
  Whether virtual memory limits will be enforced for containers



  yarn.nodemanager.vmem-pmem-ratio
  4
  Ratio between virtual memory to physical memory when setting memory limits for containers

3.2.在mapred-site.xml文件中添加如下配置:修改map和reduce任务时内存配置(value需要根据自己的虚拟机内存及应用修改


mapreduce.map.memory.mb
  2048


  mapreduce.reduce.memory.mb
  2048

------------恢复内容结束------------