Jerry Cwiklik (JIRA | 3 Sep 22:25 2015
Picon

[jira] [Closed] (UIMA-4585) DUCC Agent CPU percentage is wrong when JP spawn child procs


     [
https://issues.apache.org/jira/browse/UIMA-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jerry Cwiklik closed UIMA-4585.
-------------------------------
    Resolution: Fixed

Modified to extract CPU from stat file for each process in a cgroup

> DUCC Agent CPU percentage is wrong when JP spawn child procs
> ------------------------------------------------------------
>
>                 Key: UIMA-4585
>                 URL: https://issues.apache.org/jira/browse/UIMA-4585
>             Project: UIMA
>          Issue Type: Bug
>          Components: DUCC
>    Affects Versions: 2.0.0-Ducc
>            Reporter: Jerry Cwiklik
>            Assignee: Jerry Cwiklik
>             Fix For: 2.1.0-Ducc
>
>
> Looks like an Agent is not calculating CPU percentage correctly when running with cgroups and JP spawns
child processes. It correctly iterates over all processes in the cgroup but it keeps collecting CPU from
the JP stat file in the /proc file system.
> To fix this, while iterating over procs in the cgroup, open process stat file to read the CPU. Make sure to
close the file when done.

(Continue reading)

Apache Jenkins Server | 3 Sep 17:55 2015
Picon

Jenkins build became unstable: UIMA-DUCC » Apache UIMA DUCC: uima-ducc-container #547

See <https://builds.apache.org/job/UIMA-DUCC/org.apache.uima$uima-ducc-container/547/>

Apache Jenkins Server | 3 Sep 17:55 2015
Picon

Jenkins build became unstable: UIMA-DUCC #547

See <https://builds.apache.org/job/UIMA-DUCC/547/changes>

Burn Lewis (JIRA | 3 Sep 16:34 2015
Picon

[jira] [Updated] (UIMA-4584) ducc-mon should warn logged-out users why files can't be read


     [
https://issues.apache.org/jira/browse/UIMA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Burn Lewis updated UIMA-4584:
-----------------------------
    Affects Version/s: 2.0.0-Ducc
        Fix Version/s:     (was: 2.0.0-Ducc)

> ducc-mon should warn logged-out users why files can't be read
> -------------------------------------------------------------
>
>                 Key: UIMA-4584
>                 URL: https://issues.apache.org/jira/browse/UIMA-4584
>             Project: UIMA
>          Issue Type: Improvement
>          Components: DUCC
>    Affects Versions: 2.0.0-Ducc
>            Reporter: Burn Lewis
>            Priority: Minor
>
> Currently just says "No data found" --- could add "Try logging in" if not already logged in

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jim Challenger (JIRA | 3 Sep 16:25 2015
Picon

[jira] [Created] (UIMA-4587) DUCC SM race: rapid unregister/deregister can corrupt state.

Jim Challenger created UIMA-4587:
------------------------------------

             Summary: DUCC SM race: rapid unregister/deregister can corrupt state.
                 Key: UIMA-4587
                 URL: https://issues.apache.org/jira/browse/UIMA-4587
             Project: UIMA
          Issue Type: Bug
          Components: DUCC
    Affects Versions: 2.0.0-Ducc
            Reporter: Jim Challenger
            Assignee: Jim Challenger

If an unregister followed by a very quick register for a service with the same endpoint is done, the internal
tables managing these regs can get corrupted.  The problem appears to be missing 'synchronized' around
the CLI-based methods.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Lou DeGenaro (JIRA | 3 Sep 14:21 2015
Picon

[jira] [Commented] (UIMA-4586) DUCC Job Driver (JD) should provide individual work item performance breakdowns


    [
https://issues.apache.org/jira/browse/UIMA-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728935#comment-14728935
] 

Lou DeGenaro commented on UIMA-4586:
------------------------------------

Code is delivered.

To be considered: replace environment variable DUCC_WORK_ITEM_PERFORMANCE with a formal job submit
flag in the CLI/API.

> DUCC Job Driver (JD) should provide individual work item performance breakdowns
> -------------------------------------------------------------------------------
>
>                 Key: UIMA-4586
>                 URL: https://issues.apache.org/jira/browse/UIMA-4586
>             Project: UIMA
>          Issue Type: Improvement
>          Components: DUCC
>            Reporter: Lou DeGenaro
>            Assignee: Lou DeGenaro
>             Fix For: 2.1.0-Ducc
>
>
> Presently, JD provides a combined performance breakdown for all work items.  It is sometimes useful to
have performance breakdowns for individual work items and JD should provide same.
> Initially, enable/disable of this capability will be via an environment variable:
DUCC_WORK_ITEM_PERFORMANCE=true specified with the user's Job submission.
(Continue reading)

Lou DeGenaro (JIRA | 3 Sep 14:01 2015
Picon

[jira] [Updated] (UIMA-4586) DUCC Job Driver (JD) should provide individual work item performance breakdowns


     [
https://issues.apache.org/jira/browse/UIMA-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Lou DeGenaro updated UIMA-4586:
-------------------------------
    Summary: DUCC Job Driver (JD) should provide individual work item performance breakdowns  (was: DUCC Job
Driver (JD) should provide individual work item performace breakdowns)

> DUCC Job Driver (JD) should provide individual work item performance breakdowns
> -------------------------------------------------------------------------------
>
>                 Key: UIMA-4586
>                 URL: https://issues.apache.org/jira/browse/UIMA-4586
>             Project: UIMA
>          Issue Type: Improvement
>          Components: DUCC
>            Reporter: Lou DeGenaro
>            Assignee: Lou DeGenaro
>             Fix For: 2.1.0-Ducc
>
>
> Presently, JD provides a combined performance breakdown for all work items.  It is sometimes useful to
have performance breakdowns for individual work items and JD should provide same.
> Initially, enable/disable of this capability will be via an environment variable:
DUCC_WORK_ITEM_PERFORMANCE=true specified with the user's Job submission.
> When enabled, the JD will create subdirectory work-item-performance in the user specified log
directory and will place an individual json.gz file for each work item successfully processed
comprising its performance breakdown.

(Continue reading)

Lou DeGenaro (JIRA | 3 Sep 14:01 2015
Picon

[jira] [Work started] (UIMA-4586) DUCC Job Driver (JD) should provide individual work item performance breakdowns


     [
https://issues.apache.org/jira/browse/UIMA-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on UIMA-4586 started by Lou DeGenaro.
------------------------------------------
> DUCC Job Driver (JD) should provide individual work item performance breakdowns
> -------------------------------------------------------------------------------
>
>                 Key: UIMA-4586
>                 URL: https://issues.apache.org/jira/browse/UIMA-4586
>             Project: UIMA
>          Issue Type: Improvement
>          Components: DUCC
>            Reporter: Lou DeGenaro
>            Assignee: Lou DeGenaro
>             Fix For: 2.1.0-Ducc
>
>
> Presently, JD provides a combined performance breakdown for all work items.  It is sometimes useful to
have performance breakdowns for individual work items and JD should provide same.
> Initially, enable/disable of this capability will be via an environment variable:
DUCC_WORK_ITEM_PERFORMANCE=true specified with the user's Job submission.
> When enabled, the JD will create subdirectory work-item-performance in the user specified log
directory and will place an individual json.gz file for each work item successfully processed
comprising its performance breakdown.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
(Continue reading)

Lou DeGenaro (JIRA | 3 Sep 13:50 2015
Picon

[jira] [Created] (UIMA-4586) DUCC Job Driver (JD) should provide individual work item performace breakdowns

Lou DeGenaro created UIMA-4586:
----------------------------------

             Summary: DUCC Job Driver (JD) should provide individual work item performace breakdowns
                 Key: UIMA-4586
                 URL: https://issues.apache.org/jira/browse/UIMA-4586
             Project: UIMA
          Issue Type: Improvement
          Components: DUCC
            Reporter: Lou DeGenaro
            Assignee: Lou DeGenaro
             Fix For: 2.1.0-Ducc

Presently, JD provides a combined performance breakdown for all work items.  It is sometimes useful to have
performance breakdowns for individual work items and JD should provide same.

Initially, enable/disable of this capability will be via an environment variable:
DUCC_WORK_ITEM_PERFORMANCE=true specified with the user's Job submission.

When enabled, the JD will create subdirectory work-item-performance in the user specified log directory
and will place an individual json.gz file for each work item successfully processed comprising its
performance breakdown.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jerry Cwiklik (JIRA | 2 Sep 21:00 2015
Picon

[jira] [Created] (UIMA-4585) DUCC Agent CPU percentage is wrong when JP spawn child procs

Jerry Cwiklik created UIMA-4585:
-----------------------------------

             Summary: DUCC Agent CPU percentage is wrong when JP spawn child procs
                 Key: UIMA-4585
                 URL: https://issues.apache.org/jira/browse/UIMA-4585
             Project: UIMA
          Issue Type: Bug
          Components: DUCC
    Affects Versions: 2.0.0-Ducc
            Reporter: Jerry Cwiklik
            Assignee: Jerry Cwiklik
             Fix For: 2.1.0-Ducc

Looks like an Agent is not calculating CPU percentage correctly when running with cgroups and JP spawns
child processes. It correctly iterates over all processes in the cgroup but it keeps collecting CPU from
the JP stat file in the /proc file system.
To fix this, while iterating over procs in the cgroup, open process stat file to read the CPU. Make sure to
close the file when done.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Apache Jenkins Server | 2 Sep 20:56 2015
Picon

Jenkins build became unstable: UIMA-DUCC » Apache UIMA DUCC: uima-ducc-orchestrator #544

See <https://builds.apache.org/job/UIMA-DUCC/org.apache.uima$uima-ducc-orchestrator/544/changes>


Gmane