headers
Chao Wang (JIRA | 4 Jan 2010 18:41
Picon
Favicon

Commented: (PIG-1167) [zebra] Zebra does not support Hadoop Globs


    [
https://issues.apache.org/jira/browse/PIG-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796259#action_12796259
] 

Chao Wang commented on PIG-1167:
--------------------------------

Patch looks good +1.

> [zebra] Zebra does not support Hadoop Globs
> -------------------------------------------
>
>                 Key: PIG-1167
>                 URL: https://issues.apache.org/jira/browse/PIG-1167
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Olga Natkovich
>            Assignee: Yan Zhou
>             Fix For: 0.6.0, 0.7.0
>
>         Attachments: PIG-1167.patch
>
>
> Pssing the following path to Zebra causing error but works with Hadoop directly: /projects/FETL/sample/ABF1/{2009120204}

--

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Yan Zhou (JIRA | 4 Jan 2010 20:35
Picon
Favicon

Updated: (PIG-1167) [zebra] Zebra does not support Hadoop Globs


     [
https://issues.apache.org/jira/browse/PIG-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yan Zhou updated PIG-1167:
--------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

Committed to both Apache trunk and 6.0 branch.

> [zebra] Zebra does not support Hadoop Globs
> -------------------------------------------
>
>                 Key: PIG-1167
>                 URL: https://issues.apache.org/jira/browse/PIG-1167
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Olga Natkovich
>            Assignee: Yan Zhou
>             Fix For: 0.6.0, 0.7.0
>
>         Attachments: PIG-1167.patch
>
>
> Pssing the following path to Zebra causing error but works with Hadoop directly: /projects/FETL/sample/ABF1/{2009120204}

--

-- 
This message is automatically generated by JIRA.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Pradeep Kamath (JIRA | 4 Jan 2010 20:39
Picon
Favicon

Commented: (PIG-1094) Fix unit tests corresponding to source changes so far


    [
https://issues.apache.org/jira/browse/PIG-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796316#action_12796316
] 

Pradeep Kamath commented on PIG-1094:
-------------------------------------

+1 to PIG-1094_6.patch , patch committed - thanks Thejas!

Here is the output of test-patch for the same:

  [exec]
     [exec] +1 overall.
     [exec]
     [exec]     +1  <at> author.  The patch does not contain any  <at> author tags.
     [exec]
     [exec]     +1 tests included.  The patch appears to include 6 new or modified tests.
     [exec]
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec]
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec]
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec]
     [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
     [exec]
     [exec]
     [exec]
(Continue reading)

Permalink | Reply | 0 comments |
headers
Richard Ding (JIRA | 4 Jan 2010 21:33
Picon
Favicon

Updated: (PIG-1090) Update sources to reflect recent changes in load-store interfaces


     [
https://issues.apache.org/jira/browse/PIG-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Richard Ding updated PIG-1090:
------------------------------

    Attachment: PIG-1090-9.patch

This patch replaced msStorage with a Configuration object in LOLoad and fixed corresponding test cases.

The results of "test-patch" run:

{code}
     [exec] +1 overall.  
     [exec] 
     [exec]     +1  <at> author.  The patch does not contain any  <at> author tags.
     [exec] 
     [exec]     +1 tests included.  The patch appears to include 15 new or modified tests.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.

{code}
(Continue reading)

Permalink | Reply | 0 comments |
headers
Daniel Dai (JIRA | 4 Jan 2010 23:14
Picon
Favicon

Commented: (PIG-1090) Update sources to reflect recent changes in load-store interfaces


    [
https://issues.apache.org/jira/browse/PIG-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796386#action_12796386
] 

Daniel Dai commented on PIG-1090:
---------------------------------

+1 for PIG-1090-8.patch

> Update sources to reflect recent changes in load-store interfaces
> -----------------------------------------------------------------
>
>                 Key: PIG-1090
>                 URL: https://issues.apache.org/jira/browse/PIG-1090
>             Project: Pig
>          Issue Type: Sub-task
>            Reporter: Pradeep Kamath
>            Assignee: Pradeep Kamath
>         Attachments: PIG-1090-2.patch, PIG-1090-3.patch, PIG-1090-4.patch, PIG-1090-6.patch,
PIG-1090-7.patch, PIG-1090-8.patch, PIG-1090-9.patch, PIG-1090.patch, PIG-1190-5.patch
>
>
> There have been some changes (as recorded in the Changes Section, Nov 2 2009 sub section of
http://wiki.apache.org/pig/LoadStoreRedesignProposal) in the load/store interfaces - this jira is
to track the task of making those changes under src. Changes under test will be addresses in a different jira.

--

-- 
This message is automatically generated by JIRA.
-
(Continue reading)

Permalink | Reply | 0 comments |
headers
Alan Gates (JIRA | 4 Jan 2010 23:32
Picon
Favicon

Commented: (PIG-1172) PushDownForeachFlatten shall not push ForEach below Join if the flattened fields is used in Join


    [
https://issues.apache.org/jira/browse/PIG-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796394#action_12796394
] 

Alan Gates commented on PIG-1172:
---------------------------------

Changes look good, +1.

The patch lists a new hadoop20.jar.  Is this intentional?

> PushDownForeachFlatten shall not push ForEach below Join if the flattened fields is used in Join
> ------------------------------------------------------------------------------------------------
>
>                 Key: PIG-1172
>                 URL: https://issues.apache.org/jira/browse/PIG-1172
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.6.0
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.6.0
>
>         Attachments: PIG-1172-1.patch
>
>
> Currently the following script will push B below D. But we will use fattened column in the join, we cannot
push that.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Daniel Dai (JIRA | 4 Jan 2010 23:40
Picon
Favicon

Updated: (PIG-1172) PushDownForeachFlatten shall not push ForEach below Join if the flattened fields is used in Join


     [
https://issues.apache.org/jira/browse/PIG-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai updated PIG-1172:
----------------------------

    Attachment: PIG-1172-2.patch

hadoop20.jar should not be in patch. I reattched the patch. Thanks.

> PushDownForeachFlatten shall not push ForEach below Join if the flattened fields is used in Join
> ------------------------------------------------------------------------------------------------
>
>                 Key: PIG-1172
>                 URL: https://issues.apache.org/jira/browse/PIG-1172
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.6.0
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.6.0
>
>         Attachments: PIG-1172-1.patch, PIG-1172-2.patch
>
>
> Currently the following script will push B below D. But we will use fattened column in the join, we cannot
push that.
> A = load '1.txt' as (bg:bag{t:tuple(a0,a1)});
(Continue reading)

Permalink | Reply | 0 comments |
headers
Pradeep Kamath (JIRA | 5 Jan 2010 00:08
Picon
Favicon

Commented: (PIG-1090) Update sources to reflect recent changes in load-store interfaces


    [
https://issues.apache.org/jira/browse/PIG-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796420#action_12796420
] 

Pradeep Kamath commented on PIG-1090:
-------------------------------------

Committed PIG-1090-8.patch and PIG-1090-9.patch to branch.

> Update sources to reflect recent changes in load-store interfaces
> -----------------------------------------------------------------
>
>                 Key: PIG-1090
>                 URL: https://issues.apache.org/jira/browse/PIG-1090
>             Project: Pig
>          Issue Type: Sub-task
>            Reporter: Pradeep Kamath
>            Assignee: Pradeep Kamath
>         Attachments: PIG-1090-2.patch, PIG-1090-3.patch, PIG-1090-4.patch, PIG-1090-6.patch,
PIG-1090-7.patch, PIG-1090-8.patch, PIG-1090-9.patch, PIG-1090.patch, PIG-1190-5.patch
>
>
> There have been some changes (as recorded in the Changes Section, Nov 2 2009 sub section of
http://wiki.apache.org/pig/LoadStoreRedesignProposal) in the load/store interfaces - this jira is
to track the task of making those changes under src. Changes under test will be addresses in a different jira.

--

-- 
This message is automatically generated by JIRA.
-
(Continue reading)

Permalink | Reply | 0 comments |
headers
Alan Gates (JIRA | 5 Jan 2010 01:24
Picon
Favicon

Commented: (PIG-1174) Creation of output path should be done by storage function


    [
https://issues.apache.org/jira/browse/PIG-1174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796449#action_12796449
] 

Alan Gates commented on PIG-1174:
---------------------------------

Delegating creation of the output path to the storage function is not trivial.  The storage function is
invoked on every reducer (or every mapper for map only jobs).  So delaying creation until the storage
function will create a race condition that the storage functions will handle.  And if the solution is just
to let the first one win and all the rest error out and ignore the error, for a large job this will still
bombard the namenode with hundreds or thousands of bogus mkdir requests.  It also has the problem that all
the storage functions that get an error can't tell if it's really an error (there's old data there they are
overwriting) versus they just lost the race and another function has already created it.

We are reworking the way load and store function interact with InputFormats and OutputFormats (see
PIG-966 for full details).  This will push the responsibility of file creation onto the OutputFormat. 
This may partially address your concerns.

> Creation of output path should be done by storage function
> ----------------------------------------------------------
>
>                 Key: PIG-1174
>                 URL: https://issues.apache.org/jira/browse/PIG-1174
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Bill Graham
>
> When executing a STORE command, Pig creates the output location before the storage function gets called.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Apache Hudson Server | 5 Jan 2010 03:21
Picon
Favicon

Hudson build is back to normal: Pig-trunk #658

See <http://hudson.zones.apache.org/hudson/job/Pig-trunk/658/changes>
Permalink | Reply | 0 comments |

Gmane