Anthony Eden | 25 May 13:11
Picon
Gravatar

Mailing List Moved to Google Groups

I hereby declare this mailing list defunct. Please use the Google
Group for ActiveWarehouse from this point on:

http://groups.google.com/group/activewarehouse-discuss

Thanks!

SIncerely,
Anthony Eden

--

-- 
GMU/IT d- s: a32 C++(++++)$ UL@ P--- L+(++) !E W+++$ !N o? K? w--- !O
M++ V PS+ PE Y PGP t+ !5 X- R tv b++ DI+ D++ G- e++ h---- r+++ y++++**

http://anthony.mp
Carlo Cabanilla | 24 May 16:40
Picon
Gravatar

append vs append_rows

Hi,

I found a bug in the documentation
(http://activewarehouse.rubyforge.org/docs/activewarehouse-etl.html#load):

In the load section, it says to set :append_rows to append to the
output file rather than overwrite it, when :append_rows actually takes
an array of hashes to add to the destination in addition to the
processed rows. To have the destination append to the output file
rather than overwrite it, you must set the :append option to true, not
:append_rows. This would only work for file destinations.

Sidenote: might be worth making the aw docs a wiki so that anyone can
contribute documentation. I feel like there's tons of awesome but
undocumented/partially documented functionality that I'm missing out
on.

--

-- 

.Carlo
http://syntacticbayleaves.com
John Dell | 22 May 05:27
Picon
Gravatar

documentation outdated

Since everything has moved to github, might be a good idea to update reference links on the rubyforge pages, etc.

I added AW on rdoc.info and now there are fresh auto-generated docs from the github project.  Also, it will auto-update when new commits are made, or you can just click the update button.

http://rdoc.info/projects/aeden/activewarehouse

John

_______________________________________________
Activewarehouse-discuss mailing list
Activewarehouse-discuss@...
http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
John Dell | 22 May 04:53
Picon
Gravatar

Re: demo or some help?

Paco,

You're a saint!  Thank you!  I got the DateDimension setup and loaded successfully, and will start working on the other bits tomorrow.  When I get everything working, I'll post a blog (I'm keeping notes).  Call it AW for noobs :-)

One other question.  How does the view stuff work (high level)?  I saw one of the commits on github referring to Yahoo UI charts/datatable.  Do I need Yui4Rails to help with views? Or does AW generate view HTML by itself?

Thanks again for the quick response.

Regards,
John

On Thu, May 21, 2009 at 4:52 PM, Paco Viramontes <kidpollo-Re5JQEeQqe8@public.gmane.orgm> wrote:
I am running on 2.3.2 with no problem

You need to add the date dimension class in you model dir as such

class DateDimension < ActiveWarehouse::DateDimension
  define_hierarchy :anual, [:calendar_year,:calendar_quarter,:calendar_month_name,:calendar_week,:day_in_week]
  
  define_hierarchy :trimestral, [:calendar_quarter,:calendar_month_name,:calendar_week,:day_in_week]
  define_hierarchy :mensual, [:calendar_month_name,:calendar_week,:day_in_week]
  define_hierarchy :semanal, [:calendar_week]
  define_hierarchy :diario, [:day_in_week]
  
  set_order :sql_date_stamp
end

--------------------
then make your fact class, 

# model/sales_fact.rb
class SalesFact < ActiveWarehouse::Fact
  aggregate :total, :label => "Total", :order => 'Total'
   
  dimension :date
  dimension :location
  
  prejoin :date => [:sql_date_stamp]
  prejoin :location  => [:zip]
  
end

other dimension classes, 

# model/location_dimension.rb
class LocationDimension < ActiveWarehouse::Dimension
  define_hierarchy :region, [:region, :state, :zip]
  define_hierarchy : zip, [: zip]
end

and cube defioition class

# model/sales_cube.rb
class SalesCube < ActiveWarehouse::Cube
  reports_on :robo_negocios
  pivots_on :date, :lugar, :robo_negocio
end

in your conrtoller index

<at> report = ActiveWarehouse::Report::TableReport.new    
<at> view = <at> report.view(params, options)

and finally in your view


<%= render_report_from( <at> view) %>

and voila,

of course there is much more work than that but it is an approximation, obviously you would have to have data in your database populated that matched the dimension and fact fiels I considered 

Cheers 

Hope it helps

On May 21, 2009, at 6:23 PM, John Dell wrote:

Hi,

I'm trying out AW for a rails (2.3.2) project.  I've used BI software (siebel analytics) in the past so I'm famiilar with warehousing concepts in general.

I've found several links to the tutorial http://anthonyeden.com/2006/12/20/activewarehouse-example-with-rails-svn-logs but it is gone and others posts suggest it is very out of date.

So how else to get started with AW?  Or is there a copy of that tutorial floating around somewhere? 

I installed AW from the git repo as a plugin, added the gem dependencies and no errors.

Then I ran the generator for a fact table, and populated that fact table with some test data.  No problems.

So, what is the next steps to get a basic date drill-down?

I tried to setup the standard date dimension table 'rake warehouse:build_date_dimension' but get a unitialized constant DateDimension.  Looking through the plugin, I can see builder/date_dimension_builder.rb is in the require path/chain from init.rb

Suggestions?  Is anyone running AW with rails 2.3.2?

Thanks!
John
_______________________________________________
Activewarehouse-discuss mailing list
Activewarehouse-discuss <at> rubyforge.org
http://rubyforge.org/mailman/listinfo/activewarehouse-discuss


_______________________________________________
Activewarehouse-discuss mailing list
Activewarehouse-discuss@...
http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
John Dell | 22 May 01:23
Picon
Gravatar

demo or some help?

Hi,

I'm trying out AW for a rails (2.3.2) project.  I've used BI software (siebel analytics) in the past so I'm famiilar with warehousing concepts in general.

I've found several links to the tutorial http://anthonyeden.com/2006/12/20/activewarehouse-example-with-rails-svn-logs but it is gone and others posts suggest it is very out of date.

So how else to get started with AW?  Or is there a copy of that tutorial floating around somewhere? 

I installed AW from the git repo as a plugin, added the gem dependencies and no errors.

Then I ran the generator for a fact table, and populated that fact table with some test data.  No problems.

So, what is the next steps to get a basic date drill-down?

I tried to setup the standard date dimension table 'rake warehouse:build_date_dimension' but get a unitialized constant DateDimension.  Looking through the plugin, I can see builder/date_dimension_builder.rb is in the require path/chain from init.rb

Suggestions?  Is anyone running AW with rails 2.3.2?

Thanks!
John

_______________________________________________
Activewarehouse-discuss mailing list
Activewarehouse-discuss@...
http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
Thibaut Barrère | 17 May 17:37
Picon
Gravatar

How to do upserts ? (update+insert)

Hi,

a question I have received a number of times is: what's the proper(s) way(s) of doing upserts when working with AW-ETL ?

Is there some way to avoid doing updates records by records (like a bulk update) ?

Any hint will be most welcome,

regards,

Thibaut
--
[blog] http://evolvingworker.com - tools for a better day
[blog] http://blog.logeek.fr - about writing software

_______________________________________________
Activewarehouse-discuss mailing list
Activewarehouse-discuss@...
http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
Thibaut Barrère | 14 May 23:31
Picon
Gravatar

Pushing a couple of ETL helpers on GitHub

Hi guys,


in case that's useful to others - I took some time to push out a few tools that I'm using all around the place:


  • CsvHelper – allows to:
    • extract csv columns names in a stable fashion
    • returns all the values of a specific column
  • FixedWidthFileDestination : if you’re outputting stuff to COBOL machines
  • DimensionMigrationHelper : useful for ActiveRecord migrations
  • French(Date/Time)DimensionBuilder : fine-grained date/time dimensions in french
  • EnsureFieldsPresenceProcessor : raises error as soon as one field is missing from a row
  • EscapeCSVProcessor : replaces \" by "" in hope FasterCSV will be able to munge your CSV afterwards
  • CleanUpTransform : like AW-ETL DecodeTransform, but leave the original value if no match is found

cheers,

Thibaut Barrère
--
LoGeek
[blog] http://evolvingworker.com - tools for a better day
[blog] http://blog.logeek.fr - about writing software
_______________________________________________
Activewarehouse-discuss mailing list
Activewarehouse-discuss@...
http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
Thibaut Barrère | 8 May 09:06
Picon
Gravatar

Re: AW-ETL and Rails ?

Bonjour Jérémy,

> I have absolutely no idea how I have to or should organize my stuff ; where to put .ctl files, source and
destination files, database.yml config, …

I'll get back to you later on this week-end.

An even more comfortable option would be if you live in Paris: we
could actually meet so that I can show you the code of a real-life
application with AW-ETL, tips and tricks I've used along its life etc.

cheers,

Thibaut Barrère
--
LoGeek
[blog] http://evolvingworker.com - tools for a better day
[blog] http://blog.logeek.fr - about writing software
Bernardo Telles | 22 Apr 00:18
Picon
Gravatar

Adding and removing fields with activewarehouse-etl

Hi there,
I'm pretty new to activewarehouse, and was wondering what techniques are used to add and remove fields from the destination file. Right now it appears I have to delete fields by omitting them from the bulk import section, and add them by either renaming them on the fly or copying them. (Please let me know if this is not what activewarehouse is for :-).

Here's the goal:
take a list of employees and their phone numbers, and place each phone number on a different row with a corresponding phone number type like so:

original format and sample row:
=> id, first_name, last_name, phone, fax
=>10,ben,tom,4444444,5555555

final format and TWO rows:
=>id,number,phone_category
=>1,4444444,phone
=>2,5555555,fax

Notice I removed the first and last name columns...here are the beginnings of a row_processor (clearly written incorrectly or against conventions):
      def process(row)
        bp = row[:business_phone]
        fp = row[:fax_phone]
        rows = []
        biz = {:business_phone => bp, :first_name => 'biz'}
        fax = {:business_phone => fp, :first_name => 'fax'}
        rows =[biz,fax]
      end

_______________________________________________
Activewarehouse-discuss mailing list
Activewarehouse-discuss@...
http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
Andrew Kuklewicz | 7 Apr 20:29
Picon
Gravatar

rolap aggregate and pipelined rollup aggregate?

Hey folks,

Looking through some old code, I found reference to _rollup tables,
which lead me to find the old rolap aggregate implementations,
rolap_aggregate.rb and pipelined_rolap_aggregate.rb.
I see it was removed from the source tree awhile back (May 2007):
http://www.codenotifier.com/projects/249/commits/758

the pipelined_rolap_aggregate.rb looks like exactly what I was looking
for, so I have to ask why this was removed, and if there is something
wrong with it other than not being well maintained since 2007?

In the meantime, I have working (but much less reusable) code that is
generating rollups for me, but this seems like a better way to go
about it, so I'm wondering if maybe I should try to use what I have
learned creating my own rollups to update this aggregate strategy.

Cheers,
Andrew Kuklewicz
Paco Viramontes | 3 Apr 08:41
Picon
Gravatar

change calculated field

Hi!

I am trying to figure out what would be the best way to calculate the  
difference between the result of two different time periods

Suppose you have a table report that shows amount of sales in January  
2008 compared against the number of sales in January 2007

Any ideas to implement this in AW?

Thanks

PAco

Gmane