headers
Dmitri Gribenko | 2 Jan 2005 17:07

Re: Stats RSS Feed?

Hello!

> Is there any provided functionality like this that would pull in your
> RC5 and/or OGR stats for display?
Yes, there is! You can fetch "raw" stats (blocks flushed on some day) with 
this URL:
http://stats.distributed.net/participant/phistory_raw.php?project_id=PROJECT_ID&id=YOUR_PARTICIPIANT_ID
PROJECT_ID is 8 for RC5-72, 25 for OGR-P2
YOUR_PARTICIPANT_ID is your participant id :)
Then, you can parse that page with some Perl script.

> Or perhaps someone has built something like this that I could look at
> and modify? My web page is running PostNuke if that helps. I am just
> looking to add a block that would display my stats for each contest. I
> do not run any sort of personal proxy to report against.
I've written a Perl script that downloads raw RC5-72 stats, parses them and 
creates some pretty plots with gnuplot. The script is covered by GPL, so you 
can modify it. You can leave the parser and the computations, but change the 
gnuplot script output with HTML output. You can get the script here:
http://gribozavr.narod.ru/rc5_stats.pl

Dmitri Gribenko

--

-- 
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/M/S d++ s+:- a---- C++++>$ UL+++>++++$ P+++>$
L+++>++++$ E--- W+++ N o? K? w--- O+ !M V? PS++ PE- Y+ PGP>++
t? 5? X R- tv- b+ !DI D- G++ e- h!(*) r !y
------END GEEK CODE BLOCK------
(Continue reading)

Permalink | Reply | 9 comments |
headers
Jim C. Nasby | 6 Jan 2005 02:12

Re: CPU/OS Distribution stats page question

If you look through the complete bug history of these issues, there is
an open bug/TODO item to create a system that relates client
version/os/cpu into 'real' os/cpu. The OS and CPU id fields as reported
by clients have sometimes been reused, etc, so it's impossible to
accurately correlate this information without taking client version into
account as well.

On Tue, Dec 28, 2004 at 02:56:47PM +0100, Didier Levet wrote:
> 
> Le 28 d?c. 04, ? 14:31, Dmitri Gribenko a ?crit :
> 
> >Hello everyone!
> >
> >>It's not Mac OS X but Darwin/x86. It turns out an OS ID was created 
> >>for
> >>Darwin (x86
> >>and PPC), and this ID is now meant to be Mac OS X since several users
> >>complained
> >>about Darwin/PPC being no longer relevant (there's a bug filed in
> >>bugzilla on this topic).
> >So, the statspage's text should be changed: "Mac OS X/x86" --> 
> >"Darwin/x86".
> >
> >>In short, the same OS ID applies for Darwin (old stuff) and Mac OS X.
> >This must be fixed in the client's source.
> 
> What must be fixed, in this case, is the stats page. Since there's no 
> Darwin/x86
> client, there's no need to keep this entry. Using another OS ID for Mac 
> OS X is
(Continue reading)

Permalink | Reply | 0 comments |
headers
Jim C. Nasby | 6 Jan 2005 02:17

Re: Stats RSS Feed?

We would also like to offer XML versions of all stats pages, and any
help on that would be much appreciated. The eventual idea is:

database --(PHP? Java?)--> XML --(XSLT)--> .html

That way, most (if not all) stats pages would be available in XML as
well as .html. We should also be able to offer .csv and other formats
easily.

On Sun, Jan 02, 2005 at 06:07:05PM +0200, Dmitri Gribenko wrote:
> Hello!
> 
> > Is there any provided functionality like this that would pull in your
> > RC5 and/or OGR stats for display?
> Yes, there is! You can fetch "raw" stats (blocks flushed on some day) with 
> this URL:
> http://stats.distributed.net/participant/phistory_raw.php?project_id=PROJECT_ID&id=YOUR_PARTICIPIANT_ID
> PROJECT_ID is 8 for RC5-72, 25 for OGR-P2
> YOUR_PARTICIPANT_ID is your participant id :)
> Then, you can parse that page with some Perl script.
> 
> > Or perhaps someone has built something like this that I could look at
> > and modify? My web page is running PostNuke if that helps. I am just
> > looking to add a block that would display my stats for each contest. I
> > do not run any sort of personal proxy to report against.
> I've written a Perl script that downloads raw RC5-72 stats, parses them and 
> creates some pretty plots with gnuplot. The script is covered by GPL, so you 
> can modify it. You can leave the parser and the computations, but change the 
> gnuplot script output with HTML output. You can get the script here:
> http://gribozavr.narod.ru/rc5_stats.pl
(Continue reading)

Permalink | Reply | 8 comments |
headers
Christopher Hicks | 6 Jan 2005 14:38

Re: Stats RSS Feed?

On Wed, 5 Jan 2005, Jim C. Nasby wrote:
> We would also like to offer XML versions of all stats pages, and any
> help on that would be much appreciated. The eventual idea is:
>
> database --(PHP? Java?)--> XML --(XSLT)--> .html
>
> That way, most (if not all) stats pages would be available in XML as
> well as .html. We should also be able to offer .csv and other formats
> easily.

I totally agree with your ultimate goal here, but I think your idea of 
using the XML to generate the HTML breaks down on a few levels:

(1) How do you generate the CSV and other formats?  If XSLT isn't going to 
help you do that then the procedural code is going to have to generate it 
somehow so just let it generate all the resultant formats.  While I can 
certainly see situations where XSLT might be useful (and I have actually 
written XSLT) I certainly wouldn't go out of my way to force myself to 
write tons of code in it.

(2) It seems more effective conceiving of this as an MVC where the model 
is the database, the procedural code is the controller, and the views are 
the different formats you want to generate.  The easier you make it for 
people to add new views the more views that will utimately exist which 
should also exclude XSLT since its not widely known.

(3) I do this sort of thing regularly for Intranets.  In these cases the 
targest end up being PDF, Excel, and HTML.  I've found Perl with 
Apache::ASP and Spreadsheet::WriteExcel and a handful of other modules
makes the actual coding that has to be done pretty painless, but the same
(Continue reading)

Permalink | Reply | 7 comments |
headers
Ben Gavin | 6 Jan 2005 18:12
Picon

Re: Stats RSS Feed?

In general, XSLT is going to be easier than the other formats, and
provides exactly what you are looking for.  XSLT itself is not
terribly difficult and I would argue is probably easier [given the
simple schema that the XML format uses] to learn than a myriad of
Perl/PHP/etc solutions.  There shouldn't be any problems generating
pretty much any format you need from the XML itself, and since we've
already got pages that generate XML for the team member [and team
summary, IIRC] pages, it should be trivial to bring that functionality
to the other pages as well.  There was a [fairly one-sided, since
nobody seemed interested at the time] discussion of this on the
stats-dev mailing list a long time ago.

Now, if we wanted to avoid that completely and write individual pages
for each output type, that's fine, but that seems like just continuing
along the path we're already on and doesn't really improve the
maintenance factor longterm...  In either case, no matter what we do
we're maintaining something, either a bunch of XSLT stylesheets or a
bunch of PHP pages so it's probably a wash either way...

Of course, if we go XML for everything, then we need to figure out
some way of getting XML out of the database layer itself [without an
intervening PHP page], or we're introducing an XML translation step
for every page in the system, which is probably sub-optimal [even with
caching of the XML output].  I don't know that PostgreSQL has an XML
output method, so this may not be an option at this time.  We've
already got classes which generate all the lists we need, so maybe
introducing the XML step for everything at this time is a bad idea...

Ben
(Continue reading)

Permalink | Reply | 3 comments |
headers
Jim C. Nasby | 7 Jan 2005 23:04

Re: Stats RSS Feed?

On Thu, Jan 06, 2005 at 08:38:49AM -0500, Christopher Hicks wrote:
> On Wed, 5 Jan 2005, Jim C. Nasby wrote:
> >We would also like to offer XML versions of all stats pages, and any
> >help on that would be much appreciated. The eventual idea is:
> >
> >database --(PHP? Java?)--> XML --(XSLT)--> .html
> >
> >That way, most (if not all) stats pages would be available in XML as
> >well as .html. We should also be able to offer .csv and other formats
> >easily.
> 
> I totally agree with your ultimate goal here, but I think your idea of 
> using the XML to generate the HTML breaks down on a few levels:
> 
> (1) How do you generate the CSV and other formats?  If XSLT isn't going to 
> help you do that then the procedural code is going to have to generate it 
> somehow so just let it generate all the resultant formats.  While I can 
> certainly see situations where XSLT might be useful (and I have actually 
> written XSLT) I certainly wouldn't go out of my way to force myself to 
> write tons of code in it.

Generating CSV or any other format with XSLT would be trivial. The code
to do this would also be highly portable from one page to another.

> (2) It seems more effective conceiving of this as an MVC where the model 
> is the database, the procedural code is the controller, and the views are 
> the different formats you want to generate.  The easier you make it for 
> people to add new views the more views that will utimately exist which 
> should also exclude XSLT since its not widely known.
MVC?
(Continue reading)

Permalink | Reply | 2 comments |
headers
Jim C. Nasby | 7 Jan 2005 23:07

Re: Stats RSS Feed?

On Thu, Jan 06, 2005 at 11:12:06AM -0600, Ben Gavin wrote:
> Of course, if we go XML for everything, then we need to figure out
> some way of getting XML out of the database layer itself [without an
> intervening PHP page], or we're introducing an XML translation step
> for every page in the system, which is probably sub-optimal [even with
> caching of the XML output].  I don't know that PostgreSQL has an XML
> output method, so this may not be an option at this time.  We've
> already got classes which generate all the lists we need, so maybe
> introducing the XML step for everything at this time is a bad idea...

I'm not sure why you feel this is the case. There's no reason we can't
use PHP or Java or what-have-you as a middle-tier that talks to the
database and generates XML data. This XML data could then be converted
into any format we needed.
--

-- 
Jim C. Nasby, Database Consultant           decibel@...
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"
_______________________________________________
rc5 mailing list
rc5@...
http://lists.distributed.net/mailman/listinfo/rc5
Permalink | Reply | 2 comments |
headers
Ben Gavin | 7 Jan 2005 23:46
Picon

Re: Stats RSS Feed?

On Fri, 7 Jan 2005 16:07:43 -0600, Jim C. Nasby <decibel@...> wrote:
> On Thu, Jan 06, 2005 at 11:12:06AM -0600, Ben Gavin wrote:
> > Of course, if we go XML for everything, then we need to figure out
> > some way of getting XML out of the database layer itself [without an
> > intervening PHP page], or we're introducing an XML translation step
> > for every page in the system, which is probably sub-optimal [even with
> > caching of the XML output].  I don't know that PostgreSQL has an XML
> > output method, so this may not be an option at this time.  We've
> > already got classes which generate all the lists we need, so maybe
> > introducing the XML step for everything at this time is a bad idea...
> 
> I'm not sure why you feel this is the case. There's no reason we can't
> use PHP or Java or what-have-you as a middle-tier that talks to the
> database and generates XML data. This XML data could then be converted
> into any format we needed.

Right, and I agree with you that eventually it would be nice to get
there [after all, I suggested it eons ago ;), and I probably wasn't
the first ], however the point has been brought up before, and I think
it still has merit, that we're basically introducing another
translation layer for rather minimal [in the end] gain.  If we were
able to get the translation costs minimized to be negligible as
compared to the total page processing cost, then I'd say we should go
that direction.  However, I've been involved in a number of projects
that utilized an XML based middle-tier, and unfortunately a number of
times we had to pay a pretty significant price to generate that XML
from the underlying dataset.

We also talked about caching once upon a time, and implementing
caching [at the expense of disk space] would certainly go a long way
(Continue reading)

Permalink | Reply | 1 comment |
headers
Christopher Hicks | 8 Jan 2005 01:06

Re: Stats RSS Feed?

On Fri, 7 Jan 2005, Jim C. Nasby wrote:
> Generating CSV or any other format with XSLT would be trivial. The code
> to do this would also be highly portable from one page to another.

I'd be surprised.

> MVC?

Model-View-Controller -- its a design pattern.

> With the (probable) expection of Excel, it is easy to generate all those 
> formats using a small amount of XSLT.

Then maybe you can generate oocalc files so that people can easily open it 
in a standard spread sheet.

>> Is the database schema and sample data easily available if somebody wanted
>> to take a whack at this?
>
> There isn't a test dataset, but the code is available it
> http://cvs.distributed.net. I've also done some work on allowing the
> stats-code to run against pproxy logs, in which case you'd need
> stats-proc, stats-sql, and stats-html.

I'll play with that.

--

-- 
</chris>

"There are four boxes to be used in defense of liberty:
(Continue reading)

Permalink | Reply | 1 comment |
headers
Jim C. Nasby | 8 Jan 2005 01:18

Re: Stats RSS Feed?

On Fri, Jan 07, 2005 at 04:46:19PM -0600, Ben Gavin wrote:
> On Fri, 7 Jan 2005 16:07:43 -0600, Jim C. Nasby <decibel@...> wrote:
> > On Thu, Jan 06, 2005 at 11:12:06AM -0600, Ben Gavin wrote:
> > > Of course, if we go XML for everything, then we need to figure out
> > > some way of getting XML out of the database layer itself [without an
> > > intervening PHP page], or we're introducing an XML translation step
> > > for every page in the system, which is probably sub-optimal [even with
> > > caching of the XML output].  I don't know that PostgreSQL has an XML
> > > output method, so this may not be an option at this time.  We've
> > > already got classes which generate all the lists we need, so maybe
> > > introducing the XML step for everything at this time is a bad idea...
> > 
> > I'm not sure why you feel this is the case. There's no reason we can't
> > use PHP or Java or what-have-you as a middle-tier that talks to the
> > database and generates XML data. This XML data could then be converted
> > into any format we needed.
> 
> Right, and I agree with you that eventually it would be nice to get
> there [after all, I suggested it eons ago ;), and I probably wasn't
> the first ], however the point has been brought up before, and I think
> it still has merit, that we're basically introducing another
> translation layer for rather minimal [in the end] gain.  If we were

I think the gain in code simplicity is well worth it. If it was so easy
to generate all these different output formats just out of PHP then it'd
be done by now. But of course the first priority is to get XML output
for everything, which has it's own set of benefits.

> able to get the translation costs minimized to be negligible as
> compared to the total page processing cost, then I'd say we should go
(Continue reading)

Permalink | Reply | 0 comments |

Gmane