raymond | 20 Oct 04:27 2014
Picon

multiple hbase cluster share a zookeeper cluster?

Hi

How can I setup several hbase cluster using one single standalone zookeeper cluster? Can someone give me
some clue? Is there any best practice on such kinds of usage? say. run multiple same services against
single zookeeper cluster.

Thanks!

Raymond
Ravindranath Akila | 18 Oct 12:35 2014
Picon

Archive Files

Is there any approach HBASE can store archive like rarely used files on
cheap storage?

That's a vague question. If I may elaborate...

Our current office stores terabytes of well structured log data on s3 to
save cost. The other day I was asked to process all these files. These
files are still used for Analytics and other decision making. The logs come
from a RTB (Real Time Bidding) system.

Now ideally these files would have been on HDFS, but would incur large
storage costs over time since they are only occasionally used but the
servers need to be up and running to store them.

By context of Big Data, aren't these files big date files? If so is there a
cheap way of storing them on HBASE? For example, write  a storage adapter
of sorts.

I'm really sorry if this isn't the right place to ask this. Thanks in
advance :)

--

-- 
R. A.
BTW, there is a website called* Thank God it's Friday!*
It tells you fun things to do in your area over the weekend.
*See here: http://www.ThankGodItIsFriday.com
<http://www.ThankGodItIsFriday.com>*
S Ahmed | 17 Oct 22:18 2014
Picon

RE: getting into the industry, is theory enough?

Hello!

How do you guys suggest someone get into the hbase/hadoop industry with a
focus on the software development side (as oppose to ops) ?

If all you have done is read a few books, played around with hbase and
maybe cloudera's packages how exactly would that result in getting some
kind of employment in the industry?

I'm not sure how mature the market is so I am weary if it is a good idea or
not to focus in this domain.

Do most people just fall into the industry b/c their company provides
training and real-life problems to solve?  i.e. you have to be in a company
that at some point adopts hadoop/hbase and you are fortunate enough to get
on the project.
Enis Söztutar | 17 Oct 20:29 2014
Picon

[ANNOUNCE] HBase 0.99.1 (developer preview release) is now available for download

The HBase Team is pleased to announce the immediate release of HBase 0.99.1.
Download it from your favorite Apache mirror [1] or maven repository.

THIS RELEASE IS NOT INTENDED FOR PRODUCTION USE, and does not contain any
backwards or forwards compatibility guarantees (even within minor versions
of
0.99.x). Please refrain from deploying this over important data. Use latest
0.98.x release instead. HBase 0.99.1 is a "developer preview" release, and
an odd-numbered release as defined in [2].

A series of of 0.99.x releases are planned in preparation for 1.0.0 release
which will be the next stable and supported release. Please use this
release as
a test bed for the upcoming HBase-1.0 release. Report any encountered
problems
or features that you think need fixing before 1.0. This release also
contains
some API changes, and deprecation of older APIs which won't be supported in
2.0 series. Please give them a try and let us know what you think. All
contribution in terms of testing, benchmarking, checking API / source /wire
compatibility, checking out documentation and further code contribution is
highly appreciated. 1.0 will be the first series in the 1.x line of
releases
which are expected to keep compatibility with previous 1.x releases. Thus
it
is very important to check the client side and server side APIs for
compatibility and maintainability concerns for future releases.

0.99.1 builds on top of all the changes that is in the 0.99.0 release (an
overview can be found at [1]). The theme of (eventual) 1.0 release is to
(Continue reading)

Andrew Purtell | 17 Oct 15:27 2014
Picon

[ANNOUNCE] HBase 0.98.7 is now available for download

Apache HBase 0.98.
​7
is now available for download. Get it from an Apache mirror [1] or Maven
repository.

The list of changes in this release can be found in the release notes [2]
or following this announcement.

Thanks to all who contributed to this release.

Best,
The HBase Dev Team

1. http://www.apache.org/dyn/closer.cgi/hbase/
2. http://s.apache.org/KmZ

HBASE-8808 Use Jacoco to generate Unit Test coverage reports
HBASE-8936 Fixing TestSplitLogWorker while running Jacoco tests.
HBASE-9473 Change UI to list 'system tables' rather than 'catalog tables'.
HBASE-10153 improve VerifyReplication to compute BADROWS more accurately
HBASE-10748 hbase-daemon.sh fails to execute with 'sh' command
HBASE-11136 Add permission check to roll WAL writer
HBASE-11331 [blockcache] lazy block decompression
HBASE-11405 Multiple invocations of hbck in parallel disables balancer
permanently
HBASE-11445 TestZKProcedure#testMultiCohortWithMemberTimeoutDuringPrepare
is flaky
HBASE-11772 Bulk load mvcc and seqId issues with native hfiles
HBASE-11796 Add client support for atomic checkAndMutate
HBASE-11798 TestBucketWriterThread may hang due to WriterThread stopping
(Continue reading)

Ron van der Vegt | 17 Oct 10:22 2014

Re: Trying to upgrade from hbase-0.94.20 to hbase-0.98.6.1-hadoop1

Hi all,

I just tried the same with version 0.98.7, but have the same result.
Does the compilation error still exists? Do you have ticket number of 
this problem?

Thanks in advice,

Ron

On ma, okt 6, 2014 at 4:16 , Ron van der Vegt 
<ron.van.der.vegt@...> wrote:
> Thanks!
> 
> On ma, okt 6, 2014 at 4:13 , Ted Yu <yuzhihong@...> wrote:
>> There was a compilation error in 0.98.6.1 for Hadoop-1
>> 
>> Please use 0.98.5 or wait for 0.98.7 (to be released this week)
>> 
>> Cheers
>> 
>> On Oct 6, 2014, at 5:39 AM, Ron van der Vegt 
>> <ron.van.der.vegt@...> wrote:
>> 
>>>  Hi,
>>>   I'm trying to upgrade from hbase-0.94.20 to 
>>> hbase-0.98.6.1-hadoop1 by following the steps of:
>>>   http://hbase.apache.org/book/ch03s04.html
>>>  http://hbase.apache.org/book/upgrade0.96.html
>>>   My 0.94.x cluster is running and then I try to run the upgrade 
(Continue reading)

ch huang | 17 Oct 08:32 2014
Picon

can not enable snapshot function on hbase 0.94.6

hi,maillist :
         i installed CDH4.4 with hbase version 0.94.6 ,(no cloudera manager
involved) but when i test snapshot function ,i get error like this
 ,acturally , i add the following info into my
/etc/hbase/conf/hbase-site.xml (each node) ,and restart hbase cluster,still
same error,anyone know why?

<property>
        <name>hbase.snapshot.enabled</name>
        <value>true</value>
</property>

hbase(main):002:0> snapshot 'demo','demo_2014'

ERROR: java.io.IOException: java.io.IOException:
java.lang.UnsupportedOperationException: To use snapshots, You must add to
the hbase-site.xml of the HBase Master: 'hbase.snapshot.enabled' property
with value 'true'.
        at
org.apache.hadoop.hbase.master.HMaster.snapshot(HMaster.java:2008)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
        at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1428)
(Continue reading)

ch huang | 17 Oct 07:48 2014
Picon

issue about migrate hbase table from 0.94 to 0.98

hi,maillist:

 how can do do data migrate from hbase 0.94 to 0.96 ,copy file from old
hadoop cluster to new hadoop cluster is OK ,any one can help me?
THORMAN, ROBERT D | 16 Oct 20:35 2014
Picon

hbase-client Put serialization exception

Anyone had a problem using hbase-client on hadoop2?  Seems like the Put class is missing a method (either
default public constructor or some init method) and throws an exception when my MR jobs starts up.

I’m using:
HDP 2.1 with
hbase-client.0.98.0.2.1.4.0-632-hadoop2.jar

Stack trace:

hadoop com.att.bdcoe.platform.persistence.mapreduce.jobs.GeoAnalyticFormatBulkLoader
/user/hbase/scada /user/hbase/output
14/10/16 13:25:46 INFO impl.TimelineClientImpl: Timeline service address: http://dn02.platform.bigtdata.io:8188/ws/v1/timeline/
14/10/16 13:25:46 INFO client.RMProxy: Connecting to ResourceManager at dn02.platform.bigtdata.io/172.16.6.27:8050
14/10/16 13:25:47 INFO input.FileInputFormat: Total input paths to process : 1
14/10/16 13:25:47 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/hbase/.staging/job_1410461150931_0063
14/10/16 13:25:47 ERROR jobs.GeoAnalyticFormatBulkLoader: java.lang.NoSuchMethodException: org.apache.hadoop.hbase.client.Put.<init>()
14/10/16 13:25:47 WARN hdfs.DFSClient: DataStreamer
Exception
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException):
No lease on /user/hbase/.staging/job_1410461150931_0063/job.split: File does not exist. Holder
DFSClient_NONMAPREDUCE_-1551666999_1 does not have any open files.
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2952)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2772)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2680)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:590)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
(Continue reading)

Liu, Ming (HPIT-GADSC | 16 Oct 09:18 2014
Picon

when will hbase create the zookeeper znode 'root-region-server’ is created? Hbase 0.94

Hello,

I am trying to debug a coprocessor code on hbase 0.94.24, which seems to work well on 0.94.5, but I cannot make
it work on 0.94.24.

Here is the copy of some coprocessor init code:

public class TestEndpoint implements TestIface, HTableWrapper {
 …
   <at> Override
  public void start(CoprocessorEnvironment env) throws IOException {
    this.env = env;
conf = env.getConfiguration();
HBaseAdmin admin = new HBaseAdmin(conf);
if (!admin.tableExists(SOME_TABLE)) {
   //do something if this table is not there
…

When hbase starts, in the log file I noticed that when regionserver load this coprocessor, it will hang
there inside the admin.tableExists() function. That API will try to access the zookeeper znode
‘root-region-server’, so I start the ‘hbase zkcli’ and run ‘ls /hbase’ at that time, and I
found the znode ‘root-region-server’ is not created. Since the coprocessor want to access a table,
it must look up the –ROOT- region, which location is saved in that znode, but that znode is not there. Then
it hangs there for ever. If I disable this coprocessor, hbase can start good and I can see
‘root-region-server’ znode created there.

This coprocessor code is claimed to work well on 0.94.5, so I am wondering if there is something changed
about the sequence of ‘load coprocessor’ and the ‘create root-region-server znode’ in hbase
0.94 serials after 0.94.5?

(Continue reading)

Ramon Wang | 16 Oct 09:03 2014

Pig 0.11 cannot work with HBase 0.98

Hi Guys

We are recently trying to upgrade our HBase from 0.94 to 0.98, our original
PIG setup seems not work with the new HBase anymore, it throws the
following class not found exception:

java.lang.NoClassDefFoundError:
org/apache/hadoop/hbase/filter/WritableByteArrayComparable
at java.lang.Class.forName0(Native Method)

I checked the source code of HBase 0.98, there seems no such a class
WritableByteArrayComparable for 0.98, any thoughts? Thanks!

Cheers
Ramon

Gmane