ThreadedMapBenchmark.java 文件源码

java
阅读 19 收藏 0 点赞 0 评论 0

项目:hadoop 作者:
public void map(WritableComparable key, 
                Writable value,
                OutputCollector<BytesWritable, BytesWritable> output, 
                Reporter reporter) throws IOException {
  int itemCount = 0;
  while (numBytesToWrite > 0) {
    int keyLength = minKeySize 
                    + (keySizeRange != 0 
                       ? random.nextInt(keySizeRange) 
                       : 0);
    randomKey.setSize(keyLength);
    randomizeBytes(randomKey.getBytes(), 0, randomKey.getLength());
    int valueLength = minValueSize 
                      + (valueSizeRange != 0 
                         ? random.nextInt(valueSizeRange) 
                         : 0);
    randomValue.setSize(valueLength);
    randomizeBytes(randomValue.getBytes(), 0, randomValue.getLength());
    output.collect(randomKey, randomValue);
    numBytesToWrite -= keyLength + valueLength;
    reporter.incrCounter(Counters.BYTES_WRITTEN, 1);
    reporter.incrCounter(Counters.RECORDS_WRITTEN, 1);
    if (++itemCount % 200 == 0) {
      reporter.setStatus("wrote record " + itemCount + ". " 
                         + numBytesToWrite + " bytes left.");
    }
  }
  reporter.setStatus("done with " + itemCount + " records.");
}
评论列表
文章目录


问题


面经


文章

微信
公众号

扫码关注公众号