Sznajder ForMailingList | 23 Oct 10:27 2014

Similar Column names in creation of HBase table


Is it allowed to provide two similar family names during the creation of an
HBase table?

I was expecting that it is forbidden, **but** this call does not throw any

        String[] families = {"cf", "cf"};
        HTableDescriptor desc = new HTableDescriptor(name);
        for (String cf : families) {
          HColumnDescriptor coldef = new HColumnDescriptor(cf);
        try {
        } catch (TableExistsException e) {
            throw new IOException("table \'" + name + "\' already exists");
Hayden Marchant | 23 Oct 07:57 2014

Orphaned aborted snapshot

Hi all,

I am running HBase 0.94.6 on a 20 node cluster, and am taking daily snapshots of our single table (only
keeping snapshots for the last 3 days. Yesterday, I started seeing the following messages in one of the
region servers that had to be restarted:

2014-10-22 08:29:19,982 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 1 on 60020: starting
2014-10-22 08:29:19,982 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 2 on 60020: starting
2014-10-22 08:29:19,986 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Serving as
njhdslave40,60020,1413980958234, RPC listening on njhdslave40/, sessionid=0x2482ca09984b22a
2014-10-22 08:29:19,986 INFO org.apache.hadoop.hbase.regionserver.SplitLogWorker:
SplitLogWorker njhdslave40,60020,1413980958234 starting
2014-10-22 08:29:19,988 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Registered
RegionServer MXBean
2014-10-22 08:29:20,024 INFO org.apache.hadoop.hbase.procedure.ProcedureMember: Received abort
on procedure with no local subprocedure upd-2014_10_19, ignoring it.
org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable via
timer-java.util.Timer <at> 2cef133c:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
org.apache.hadoop.hbase.errorhandling.TimeoutException: Timeout elapsed! Source:Timeout
caused Foreign Exception Start:1413728408736, End:1413728468758, diff:60022, max:60000 ms
    at org.apache.hadoop.hbase.errorhandling.ForeignException.deserialize(
    at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.abort(
    at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.watchForAbortedProcedures(
    at org.apache.hadoop.hbase.procedure.ZKProcedureMemberRpcs.start(
    at org.apache.hadoop.hbase.regionserver.snapshot.RegionServerSnapshotManager.start(
Caused by: org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
org.apache.hadoop.hbase.errorhandling.TimeoutException: Timeout elapsed! Source:Timeout
caused Foreign Exception Start:1413728408736, End:1413728468758, diff:60022, max:60000 ms
(Continue reading)

ch huang | 23 Oct 03:34 2014

issue about snapshot migration


       i have two hbase cluster ,one is version 0.94.6 another is 0.98.1 ,i
try to mv data of table A from 0.94.6 to 0.98.1 ,i make the snapshot of A
,then want to use ExportSnapshot to mv snapshot to hdfs cluster which hbase
0.98.1 on,but failed ,why?

# hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot demo_shot
-copy-to hdfs://
Exception in thread "main" Failed on local exception: Message missing
required fields: callId, status; Host Details : local host is: "ch11/"; destination host is: "hzih231":8020;
        at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.lang.reflect.Method.invoke(
        at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
(Continue reading)

krish_571 | 23 Oct 02:50 2014

setup method in Mapreduce Program

I understand that setup and cleanup methods of Mapper/Reducer  are called
for each task, so if you have 20 mappers running, the setup / cleanup will
be called for each one.

Is there a setup like method to for the entire mapreduce program.  Basically
i want to open only  1  connectionto a HBase table for the entire Mapreduce
job so that all map tasks need not open a separate connection.

View this message in context:
Sent from the HBase User mailing list archive at

jeevi tesh | 22 Oct 13:14 2014

java.lang.IllegalArgumentException: Path must start with / character

I'm using hbase0.94.3, hadoop-2.2.0 and jdk 1.7.67 version..
In hbase shell if i issue any command i'm getting
ERROR: java.lang.IllegalArgumentException: Path must start with / character
hbase-site.xml file content



kindly help me to resolve the issue
Junegunn Choi | 22 Oct 03:17 2014

Performance impact of io.file.buffer.size parameter of HDFS


I've read that on a modern hardware I should increase the value of
io.file.buffer.size parameter of HDFS, up to 128kb or so. [1] Does this
advice still hold true in the context of HBase? We've done a series of
performance benchmarks with the different values of it, but couldn't
observe a noticeable difference in performance. Is the default 4kb good
enough? What is the recommended value, or maybe it doesn't matter?

Thanks in advance.

[1]: Book: Hadoop - The Definitive Guide
Kevin | 21 Oct 19:26 2014

HBase client machine with multiple interfaces


I have connected a client machine with two network interfaces to an
internal, isolated HBase cluster and an external network. The HBase cluster
is on its own private LAN, away from the external network. After installing
and updating the Hadoop and HBase configuration files on the client
machine, I can run the HBase shell and look at tables, but
TableInputFormatBase can't find any servers using its DNS checks. The
machine is on the DNS for the external network so, from my understanding,
the HBase client API will use this DNS to try and find the HBase cluster. I
know there are master and regionserver DNS interface configurations, but
are there anything for client libraries? My use case is running Pig scripts
from the client machine. Oddly enough, sometimes a Pig script works, other
times (most times) I get the NullPointerException from the call.

Any ideas?

Serega Sheypak | 21 Oct 17:34 2014

Why snapshots are disabled by default?

Hi, I tried to create a snapshot and it's not enabled

: java.lang.UnsupportedOperationException: To use
snapshots, You must add to the hbase-site.xml of the HBase Master:
'hbase.snapshot.enabled' property with value 'true'.

Is it by-default? If yes, then why? What is the reason?
Jan Lukavský | 21 Oct 17:23 2014

Delete.deleteColumns not working with HFileOutputFormat?

Hi all,

we are using HBase version 0.94.6-cdh4.3.1 and I have a suspicion that a 
Delete written to hbase through HFileOutputFormat might be ignored (and 
not delete any data) in the following scenario:
  * a Delete object is used to delete the data at the client side
  * call to "deleteColumn" instead of "deleteColumns" is used, which 
means that the underlaying KeyValue will not have an associated 
timestamp (will have HConstants.LATEST_TIMESTAMP)
  * the Delete object is then converted to KeyValues and these are 
written into the output format's record writer

I think (our systems seems to behave this way) the problem is in the way 
the KeyValue is processed in the RegionServer, even though I was not 
able to track the problem in the source code. Can anyone else confirm 
this? When using Delete#deleteColumns everything seems to be working 
fine (the KeyValues have different type). Is this expected or should it 
be considered a bug? And if so, where it should be fixed? I think it 
could be on the side of the record writer (maybe by throwing an 
exception), or in the region server (if possible, this might be 
non-trivial, because of the Delete#deleteColumn semantics).

Any opinions?


Krishna Kalyan | 21 Oct 14:29 2014

Duplicate Value Inserts in HBase

I have a HBase table which is populated from pig using PigStorage.
While inserting, suppose for rowkey i have a duplicate value.
Is there a way to prevent an update?.
I want to maintain the version history for my values which are unique.

Qiang Tian | 21 Oct 11:28 2014

YCSB load data stucked

Hi Guys,
I am running YCSB 0.1.4 against hbase 0.98.5,

"bin/ycsb load hbase -P workloads/workloada -p columnfamily=f1 -p
recordcount=1000 -p threadcount=4 -s | tee -a workloada.dat" stucked as

 10 sec: 0 operations;
 20 sec: 0 operations;
 30 sec: 0 operations;
 40 sec: 0 operations;
 50 sec: 0 operations;

stacktrace shows YCSB is locating meta region:

at java/lang/Object.wait(
 (entered lock:
org/apache/hadoop/hbase/zookeeper/RootRegionTracker <at> 0x00000000E01A02B0,
entry count: 1)
(Continue reading)