hadoop distributedcache 应该是共享的

[cjyu]

hadoop用distributedcache发布数据应该是共享的。 每次task开始前会初始化，这里面会包括distributedcache下载数据。 这个操作在每个node上只能顺序进行。多个task的话只能一个一个等。 如果已经下载过了就不会重复下载。

[JerryLead]

多谢指出，还有几个问题，等过些天整理文档的时候再请教你

[lxyscls]

Then, before a task is run, the node manager copies the files from the distributed filesystem to a local disk—the cache—so the task can access the files. The files are said to be localized at this point. From the task’s point of view, the files are just there, symbolically linked from the task’s working directory.

<Hadoop: The Definitive Guide, Fourth Edition> ch9 distributed cache

我个人的理解是nodemanager帮忙下载，用软链接的方式链接到task的工作目录，所以不会存在多个同样的distribtued cache文件。