Picon

[jira] [Created] (HBASE-12593) Tags and Tag dictionary to work with BB

ramkrishna.s.vasudevan created HBASE-12593:
----------------------------------------------

             Summary: Tags and Tag dictionary to work with BB
                 Key: HBASE-12593
                 URL: https://issues.apache.org/jira/browse/HBASE-12593
             Project: HBase
          Issue Type: Sub-task
            Reporter: ramkrishna.s.vasudevan
            Assignee: ramkrishna.s.vasudevan

Adding the subtask so that we don't forget it. Came up while reviewing the items required for this parent task.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

stack (JIRA | 27 Nov 12:12 2014
Picon

[jira] [Created] (HBASE-12592) Fix the truncate feature or purge it.

stack created HBASE-12592:
-----------------------------

             Summary: Fix the truncate feature or purge it.
                 Key: HBASE-12592
                 URL: https://issues.apache.org/jira/browse/HBASE-12592
             Project: HBase
          Issue Type: Umbrella
            Reporter: stack
            Assignee: stack

While working on our move to unmanaged connections, I ran into issues in
TestAccessController.testTruncatePerms; the acl permissions were not being restored when the table
was restored.  Digging, I ended up in the truncate 'handler'. Its a subclass of the delete table handler. 
For the create table part of trunctate, it does not call create table handler.  Instead it copy/pastes what
is going on over there only the copy/paste has rotted and needs recalibrating so acl perms get restored at
least; better would be removing the copy/paste and using the create table handler.  Or, we could just drop
the truncate feature as just not appropriate to a store like this.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jiajia Li (JIRA | 27 Nov 07:46 2014
Picon

[jira] [Created] (HBASE-12591) Ignore the count of mob compaction metrics when there is issue

Jiajia Li created HBASE-12591:
---------------------------------

             Summary: Ignore the count of mob compaction metrics when there is issue
                 Key: HBASE-12591
                 URL: https://issues.apache.org/jira/browse/HBASE-12591
             Project: HBase
          Issue Type: Sub-task
          Components: regionserver
    Affects Versions: hbase-11339
            Reporter: Jiajia Li
            Assignee: Jiajia Li
            Priority: Minor
             Fix For: hbase-11339

In mob compaction, the metrics of "mobCompactedFromMobCellsCount" and
"mobCompactedFromMobCellsSize" should not be count when there is issue when retrieve the mob cell from
the mob file.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Weichen Ye (JIRA | 27 Nov 02:59 2014
Picon

[jira] [Created] (HBASE-12590) A solution for data skew in HBase-Mapreduce Job

Weichen Ye created HBASE-12590:
----------------------------------

             Summary: A solution for data skew in HBase-Mapreduce Job 
                 Key: HBASE-12590
                 URL: https://issues.apache.org/jira/browse/HBASE-12590
             Project: HBase
          Issue Type: Improvement
          Components: mapreduce
    Affects Versions: 2.0.0
            Reporter: Weichen Ye

1, Motivation
In production environment, data skew is a very common case. A HBase table always contains a lot of small
regions and several large regions. Small regions waste a lot of computing resources. If we use a job to scan
a table with 3000 small regions, we need a job with 3000 mappers. Large regions always block the job. If in a
100-region table, one region is far larger then the other 99 regions. When we run a job with the table as
input, 99 mappers will be completed very quickly, and we need to wait for the last mapper for a long time.

2, Configuration
Add two new configuration. 
hbase.mapreduce.split.autobalance = true means enabling the “auto balance” in HBase-MapReduce
jobs. The default value is false. 
hbase.mapreduce.split.targetsize = 1073741824 (default 1GB). The target size of mapreduce splits. 
If a region size is large than the target size, cut the region into two split.If the sum of several small
continuous region size less than the target size, combine these regions into one split.

Example:
In attachment

(Continue reading)

Jeffrey Zhong (JIRA | 27 Nov 01:56 2014
Picon

[jira] [Resolved] (HBASE-12053) SecurityBulkLoadEndPoint set 777 permission on input data files


     [
https://issues.apache.org/jira/browse/HBASE-12053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jeffrey Zhong resolved HBASE-12053.
-----------------------------------
      Resolution: Fixed
    Hadoop Flags: Reviewed

Thanks for the comments! I've integrated the fix into 0.98,0.99 & master branches.

> SecurityBulkLoadEndPoint set 777 permission on input data files 
> ----------------------------------------------------------------
>
>                 Key: HBASE-12053
>                 URL: https://issues.apache.org/jira/browse/HBASE-12053
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jeffrey Zhong
>            Assignee: Jeffrey Zhong
>             Fix For: 2.0.0, 0.98.9, 0.99.2
>
>         Attachments: HBASE-12053.patch
>
>
> We have code in SecureBulkLoadEndpoint#secureBulkLoadHFiles
> {code}
>               LOG.trace("Setting permission for: " + p);
>               fs.setPermission(p, PERM_ALL_ACCESS);
> {code}
(Continue reading)

stack (JIRA | 27 Nov 00:39 2014
Picon

[jira] [Reopened] (HBASE-12581) TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 addendum)


     [
https://issues.apache.org/jira/browse/HBASE-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack reopened HBASE-12581:
---------------------------

Reopening. I just saw this again after thinking it fixed. The below is different class but same failure
type.  We are not getting notification of the acl update probably since we went to unmanaged
connections... we are missing watches?

{code}
org.apache.hadoop.hbase.security.access.TestCellACLs.testCellPermissions

Failing for the past 1 build (Since Failed#5827 )
Took 1.1 sec.
add description
Error Message

Failed 1 action: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
permissions (user=owner, scope=testCellPermissions, family=f1:q1, action=WRITE)
 at org.apache.hadoop.hbase.security.access.AccessController.preBatchMutate(AccessController.java:1493)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$35.call(RegionCoprocessorHost.java:891)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1573)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1648)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1605)
 at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preBatchMutate(RegionCoprocessorHost.java:887)
 at org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2668)
 at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2450)
 at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2401)
(Continue reading)

stack (JIRA | 26 Nov 23:44 2014
Picon

[jira] [Resolved] (HBASE-12589) Forward-port fix for TestFromClientSideWithCoprocessor.testMaxKeyValueSize


     [
https://issues.apache.org/jira/browse/HBASE-12589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack resolved HBASE-12589.
---------------------------
       Resolution: Fixed
    Fix Version/s: 2.0.0

Pushed to master

> Forward-port fix for TestFromClientSideWithCoprocessor.testMaxKeyValueSize
> --------------------------------------------------------------------------
>
>                 Key: HBASE-12589
>                 URL: https://issues.apache.org/jira/browse/HBASE-12589
>             Project: HBase
>          Issue Type: Task
>          Components: test
>            Reporter: stack
>            Assignee: stack
>             Fix For: 2.0.0
>
>         Attachments: 12589.txt
>
>
> Trunk just failed with TestFromClientSideWithCoprocessor.testMaxKeyValueSize  Issue is that test
has a Connection and subsequently, we set configuration to reject big value only we are setting it on a
different conf instance.  Let me forward-port the fix from branch-1.

(Continue reading)

stack (JIRA | 26 Nov 23:40 2014
Picon

[jira] [Created] (HBASE-12589) Forward-port fix for TestFromClientSideWithCoprocessor.testMaxKeyValueSize

stack created HBASE-12589:
-----------------------------

             Summary: Forward-port fix for TestFromClientSideWithCoprocessor.testMaxKeyValueSize
                 Key: HBASE-12589
                 URL: https://issues.apache.org/jira/browse/HBASE-12589
             Project: HBase
          Issue Type: Task
          Components: test
            Reporter: stack
            Assignee: stack

Trunk just failed with TestFromClientSideWithCoprocessor.testMaxKeyValueSize  Issue is that test has
a Connection and subsequently, we set configuration to reject big value only we are setting it on a
different conf instance.  Let me forward-port the fix from branch-1.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

משפחת ינקוביץ | 26 Nov 22:54 2014
Picon

.oldlogs arent being cleaned

Hi,

We have a cluster participating in cycle replication.

We noticed that the .oldlogs folder takes 4.4T (4.4 T    /hbase/.oldlogs).

The  replication is working fine and snapshots are used well right now, but
I noticed that the last oldlogs file was added at October 2014.

Does anyone have any idea why they aren’t being cleaned? Can I delete them?

I am running hbase 0.94.6 (CDH 4.5).

Thanks,

Jeffrey Zhong (JIRA | 26 Nov 22:40 2014
Picon

[jira] [Created] (HBASE-12588) Need to fail writes when row lock can't be acquired

Jeffrey Zhong created HBASE-12588:
-------------------------------------

             Summary: Need to fail writes when row lock can't be acquired
                 Key: HBASE-12588
                 URL: https://issues.apache.org/jira/browse/HBASE-12588
             Project: HBase
          Issue Type: Bug
    Affects Versions: 0.99.1, 0.98.8
            Reporter: Jeffrey Zhong
            Assignee: Jeffrey Zhong

Currently we don't fail write operations when can't acquiring row locks as shown below in
HRegion#doMiniBatchMutation. 
{code}
...
        RowLock rowLock = null;
        try {
          rowLock = getRowLock(mutation.getRow(), shouldBlock);
        } catch (IOException ioe) {
          LOG.warn("Failed getting lock in batch put, row="
            + Bytes.toStringBinary(mutation.getRow()), ioe);
        }
        if (rowLock == null) {
          // We failed to grab another lock
          assert !shouldBlock : "Should never fail to get lock when blocking";
          break; // stop acquiring more rows for this batch
        } else {
          acquiredRowLocks.add(rowLock);
        }
(Continue reading)

stack (JIRA | 26 Nov 21:52 2014
Picon

[jira] [Created] (HBASE-12587) TestReplicationTrackerZKImpl.testPeerRemovedEvent timedout

stack created HBASE-12587:
-----------------------------

             Summary: TestReplicationTrackerZKImpl.testPeerRemovedEvent timedout
                 Key: HBASE-12587
                 URL: https://issues.apache.org/jira/browse/HBASE-12587
             Project: HBase
          Issue Type: Task
          Components: test
            Reporter: stack

Test timedout in branch-1: https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.0/508/testReport/junit/org.apache.hadoop.hbase.replication/TestReplicationTrackerZKImpl/testPeerListChangedEvent/

{code}
Error Message

test timed out after 30000 milliseconds
Stacktrace

java.lang.Exception: test timed out after 30000 milliseconds
	at java.lang.Thread.sleep(Native Method)
	at org.apache.hadoop.hbase.replication.TestReplicationTrackerZKImpl.testPeerListChangedEvent(TestReplicationTrackerZKImpl.java:170)

{code}

Looking at code, zk count run before we are set up to listen on event.

Second test failure is symptom of this fail.

I can't repro locally but happened on trunk and branch-1 just now.
(Continue reading)


Gmane