Chao Wang (JIRA | 4 Jan 18:41 2010
Picon

Commented: (PIG-1167) [zebra] Zebra does not support Hadoop Globs


    [
https://issues.apache.org/jira/browse/PIG-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796259#action_12796259
] 

Chao Wang commented on PIG-1167:
--------------------------------

Patch looks good +1.

> [zebra] Zebra does not support Hadoop Globs
> -------------------------------------------
>
>                 Key: PIG-1167
>                 URL: https://issues.apache.org/jira/browse/PIG-1167
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Olga Natkovich
>            Assignee: Yan Zhou
>             Fix For: 0.6.0, 0.7.0
>
>         Attachments: PIG-1167.patch
>
>
> Pssing the following path to Zebra causing error but works with Hadoop directly: /projects/FETL/sample/ABF1/{2009120204}

--

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
(Continue reading)

Yan Zhou (JIRA | 4 Jan 20:35 2010
Picon

Updated: (PIG-1167) [zebra] Zebra does not support Hadoop Globs


     [
https://issues.apache.org/jira/browse/PIG-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yan Zhou updated PIG-1167:
--------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

Committed to both Apache trunk and 6.0 branch.

> [zebra] Zebra does not support Hadoop Globs
> -------------------------------------------
>
>                 Key: PIG-1167
>                 URL: https://issues.apache.org/jira/browse/PIG-1167
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Olga Natkovich
>            Assignee: Yan Zhou
>             Fix For: 0.6.0, 0.7.0
>
>         Attachments: PIG-1167.patch
>
>
> Pssing the following path to Zebra causing error but works with Hadoop directly: /projects/FETL/sample/ABF1/{2009120204}

--

-- 
This message is automatically generated by JIRA.
(Continue reading)

Pradeep Kamath (JIRA | 4 Jan 20:39 2010
Picon

Commented: (PIG-1094) Fix unit tests corresponding to source changes so far


    [
https://issues.apache.org/jira/browse/PIG-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796316#action_12796316
] 

Pradeep Kamath commented on PIG-1094:
-------------------------------------

+1 to PIG-1094_6.patch , patch committed - thanks Thejas!

Here is the output of test-patch for the same:

  [exec]
     [exec] +1 overall.
     [exec]
     [exec]     +1  <at> author.  The patch does not contain any  <at> author tags.
     [exec]
     [exec]     +1 tests included.  The patch appears to include 6 new or modified tests.
     [exec]
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec]
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec]
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec]
     [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
     [exec]
     [exec]
     [exec]

(Continue reading)

Richard Ding (JIRA | 4 Jan 21:33 2010
Picon

Updated: (PIG-1090) Update sources to reflect recent changes in load-store interfaces


     [
https://issues.apache.org/jira/browse/PIG-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Richard Ding updated PIG-1090:
------------------------------

    Attachment: PIG-1090-9.patch

This patch replaced msStorage with a Configuration object in LOLoad and fixed corresponding test cases.

The results of "test-patch" run:

{code}
     [exec] +1 overall.  
     [exec] 
     [exec]     +1  <at> author.  The patch does not contain any  <at> author tags.
     [exec] 
     [exec]     +1 tests included.  The patch appears to include 15 new or modified tests.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.

{code}

(Continue reading)

Daniel Dai (JIRA | 4 Jan 23:14 2010
Picon

Commented: (PIG-1090) Update sources to reflect recent changes in load-store interfaces


    [
https://issues.apache.org/jira/browse/PIG-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796386#action_12796386
] 

Daniel Dai commented on PIG-1090:
---------------------------------

+1 for PIG-1090-8.patch

> Update sources to reflect recent changes in load-store interfaces
> -----------------------------------------------------------------
>
>                 Key: PIG-1090
>                 URL: https://issues.apache.org/jira/browse/PIG-1090
>             Project: Pig
>          Issue Type: Sub-task
>            Reporter: Pradeep Kamath
>            Assignee: Pradeep Kamath
>         Attachments: PIG-1090-2.patch, PIG-1090-3.patch, PIG-1090-4.patch, PIG-1090-6.patch,
PIG-1090-7.patch, PIG-1090-8.patch, PIG-1090-9.patch, PIG-1090.patch, PIG-1190-5.patch
>
>
> There have been some changes (as recorded in the Changes Section, Nov 2 2009 sub section of
http://wiki.apache.org/pig/LoadStoreRedesignProposal) in the load/store interfaces - this jira is
to track the task of making those changes under src. Changes under test will be addresses in a different jira.

--

-- 
This message is automatically generated by JIRA.
-
(Continue reading)

Alan Gates (JIRA | 4 Jan 23:32 2010
Picon

Commented: (PIG-1172) PushDownForeachFlatten shall not push ForEach below Join if the flattened fields is used in Join


    [
https://issues.apache.org/jira/browse/PIG-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796394#action_12796394
] 

Alan Gates commented on PIG-1172:
---------------------------------

Changes look good, +1.

The patch lists a new hadoop20.jar.  Is this intentional?

> PushDownForeachFlatten shall not push ForEach below Join if the flattened fields is used in Join
> ------------------------------------------------------------------------------------------------
>
>                 Key: PIG-1172
>                 URL: https://issues.apache.org/jira/browse/PIG-1172
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.6.0
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.6.0
>
>         Attachments: PIG-1172-1.patch
>
>
> Currently the following script will push B below D. But we will use fattened column in the join, we cannot
push that.
(Continue reading)

Daniel Dai (JIRA | 4 Jan 23:40 2010
Picon

Updated: (PIG-1172) PushDownForeachFlatten shall not push ForEach below Join if the flattened fields is used in Join


     [
https://issues.apache.org/jira/browse/PIG-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai updated PIG-1172:
----------------------------

    Attachment: PIG-1172-2.patch

hadoop20.jar should not be in patch. I reattched the patch. Thanks.

> PushDownForeachFlatten shall not push ForEach below Join if the flattened fields is used in Join
> ------------------------------------------------------------------------------------------------
>
>                 Key: PIG-1172
>                 URL: https://issues.apache.org/jira/browse/PIG-1172
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.6.0
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.6.0
>
>         Attachments: PIG-1172-1.patch, PIG-1172-2.patch
>
>
> Currently the following script will push B below D. But we will use fattened column in the join, we cannot
push that.
> A = load '1.txt' as (bg:bag{t:tuple(a0,a1)});
(Continue reading)

Pradeep Kamath (JIRA | 5 Jan 00:08 2010
Picon

Commented: (PIG-1090) Update sources to reflect recent changes in load-store interfaces


    [
https://issues.apache.org/jira/browse/PIG-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796420#action_12796420
] 

Pradeep Kamath commented on PIG-1090:
-------------------------------------

Committed PIG-1090-8.patch and PIG-1090-9.patch to branch.

> Update sources to reflect recent changes in load-store interfaces
> -----------------------------------------------------------------
>
>                 Key: PIG-1090
>                 URL: https://issues.apache.org/jira/browse/PIG-1090
>             Project: Pig
>          Issue Type: Sub-task
>            Reporter: Pradeep Kamath
>            Assignee: Pradeep Kamath
>         Attachments: PIG-1090-2.patch, PIG-1090-3.patch, PIG-1090-4.patch, PIG-1090-6.patch,
PIG-1090-7.patch, PIG-1090-8.patch, PIG-1090-9.patch, PIG-1090.patch, PIG-1190-5.patch
>
>
> There have been some changes (as recorded in the Changes Section, Nov 2 2009 sub section of
http://wiki.apache.org/pig/LoadStoreRedesignProposal) in the load/store interfaces - this jira is
to track the task of making those changes under src. Changes under test will be addresses in a different jira.

--

-- 
This message is automatically generated by JIRA.
-
(Continue reading)

Alan Gates (JIRA | 5 Jan 01:24 2010
Picon

Commented: (PIG-1174) Creation of output path should be done by storage function


    [
https://issues.apache.org/jira/browse/PIG-1174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796449#action_12796449
] 

Alan Gates commented on PIG-1174:
---------------------------------

Delegating creation of the output path to the storage function is not trivial.  The storage function is
invoked on every reducer (or every mapper for map only jobs).  So delaying creation until the storage
function will create a race condition that the storage functions will handle.  And if the solution is just
to let the first one win and all the rest error out and ignore the error, for a large job this will still
bombard the namenode with hundreds or thousands of bogus mkdir requests.  It also has the problem that all
the storage functions that get an error can't tell if it's really an error (there's old data there they are
overwriting) versus they just lost the race and another function has already created it.

We are reworking the way load and store function interact with InputFormats and OutputFormats (see
PIG-966 for full details).  This will push the responsibility of file creation onto the OutputFormat. 
This may partially address your concerns.

> Creation of output path should be done by storage function
> ----------------------------------------------------------
>
>                 Key: PIG-1174
>                 URL: https://issues.apache.org/jira/browse/PIG-1174
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Bill Graham
>
> When executing a STORE command, Pig creates the output location before the storage function gets called.
(Continue reading)

Apache Hudson Server | 5 Jan 03:21 2010
Picon

Hudson build is back to normal: Pig-trunk #658

See <http://hudson.zones.apache.org/hudson/job/Pig-trunk/658/changes>


Gmane