Darren Oakley | 1 Sep 10:58 2009
Picon

Biomart API for Ruby

Hi All,

How's things going?

Just thought i'd let you all know about a pleasant by-product of some  
of my recent work - I've just packaged up and released a Biomart  
library/api for Ruby.  I was writing a lot of code that queried  
various marts via the web services and ended up copying around lots of  
boiler plate code, so it made sense to modularize it. :)  It's quite  
basic for the moment and only does single dataset queries, but it will  
be quite easy to build upon the base from here.

The source is available from github (http://github.com/dazoakley/ 
biomart), and the docs are on rdoc.info (http://rdoc.info/projects/dazoakley/biomart 
).

Fingers crossed this might be helpful to others! :)

Thanks,

Daz

--

-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 

Arek Kasprzyk | 1 Sep 13:53 2009
Picon

Re: Biomart API for Ruby


On 01/09/09 4:58 AM, "Darren Oakley" <do2 <at> sanger.ac.uk> wrote:

> Hi All,
> 
> How's things going?
> 
> Just thought i'd let you all know about a pleasant by-product of some
> of my recent work - I've just packaged up and released a Biomart
> library/api for Ruby.  I was writing a lot of code that queried
> various marts via the web services and ended up copying around lots of
> boiler plate code, so it made sense to modularize it. :)  It's quite
> basic for the moment and only does single dataset queries, but it will
> be quite easy to build upon the base from here.
> 
> The source is available from github (http://github.com/dazoakley/
> biomart), and the docs are on rdoc.info
> (http://rdoc.info/projects/dazoakley/biomart
> ).
> 
> Fingers crossed this might be helpful to others! :)
> 

Great stuff Daz :) I'll get in touch with bioruby developers to let them
know that this exists :)

A.

> Thanks,
> 
(Continue reading)

Ayton Meintjes | 1 Sep 15:41 2009
Picon

Re: FW: Dynamic attribute set

Hi Gnani, Christina

Thanks for the quick response (and for providing an example), this is exactly what I was looking for. Got it working on a test dataset already.

regards
Ayton


Having some issues still though. The structure is being created but the tables are empty

2009/8/28 Saravanamuttu Gnaneshan <Saravanamuttu.Gnaneshan <at> oicr.on.ca>
Hello Ayton
 
We have now built a test mart to satisfy your requirements with some fake data as shown in your mail.
 
 
If the above link expires by the time you see this mail please use this url:  http://dcc-dev.res.oicr.on.ca:9030
 
 
This mart was built as explained in Christina's email and if you need to see the XML files created by the martbuilder and marteditor please email me offline (not through the mart-dev list).
 
If you have any queries on this please feel free to contact us.
 
Best wishes
 
Gnani
 

Saravanamuttu Gnaneshan
Scientist, Computational Biologist

Ontario Institute for Cancer Research
MaRS Centre, South Tower
101 College Street, Suite 800
Toronto, Ontario, Canada M5G 0A3
     
Tel:         647-260-7990
Toll-free: 1-866-678-6427
www.oicr.on.ca

 

 

This message and any attachments may contain confidential and/or privileged information for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this message in error, please contact the sender and delete all copies. Opinions, conclusions or other information contained in this message may not be that of the organization.


From: owner-mart-dev <at> ebi.ac.uk [mailto:owner-mart-dev <at> ebi.ac.uk] On Behalf Of Christina Yung
Sent: August 27, 2009 9:53 AM
To: Ayton Meintjes
Cc: mart-dev <at> ebi.ac.uk
Subject: RE: [mart-dev] Dynamic attribute set

Hi Ayton:
 
I think a mart with partititioned tables in the following structure will serve your needs:
 
Marker [id, rsid, allele_A, alleleB]
CEU_AlleleFrequency [fk_marker_id, CEU_freq]
JPT_AlleleFrequency [fk_marker_id, JPT_freq]
YRI_AlleleFrequency [fk_marker_id, YRI_freq]
 
In this mart, Marker is the main table where *_AlleleFrequency are the dimension tables.  This structure allows the addition of new populations as dimension tables.  Keep in mind that when multiple *_freq attributes are selected, multiple tables are joined.  Hence, scalability may become an issue if the number of populations is large.
 
To arrive at this mart, you may wish to use MartBuilder.  First, specify Marker as the main table.  MartBuilder will merge Population and AlleleFrequency into one dimension table [id, fk_population_id, fk_marker_id, freq, population_name].  The next step is to partition this dimension table by either fk_population_id or population_name.
 
Hope this helps.
 
Christina
 

From: owner-mart-dev <at> ebi.ac.uk [mailto:owner-mart-dev <at> ebi.ac.uk] On Behalf Of Ayton Meintjes
Sent: Wednesday, August 26, 2009 11:37 AM
To: mart-dev <at> ebi.ac.uk
Subject: [mart-dev] Dynamic attribute set

I have the following three tables in a custom database:

Population [id, name]
Marker [id, rsid, allele_A, allele_B]
AlleleFrequency [id, fk_population_id, fk_marker_id, freq]

How would I go about creating a mart that the user can query for markers and get results in the form:

rsid  | CEU_freq | JPT_freq | YRI_freq
rs12 |    0.05     |    0.55     |  0.75    
rs34 |    0.95     |    0.65     |  0.05    

In other words, the number of attributes available depends on how many unique populations there are? The automate push action and partitioning seem to suggest this sort of structure might be possible.
 
Thanks in advance

Saravanamuttu Gnaneshan | 1 Sep 16:58 2009
Picon

RE: FW: Dynamic attribute set

Hi Ayton
 
Glad to hear that you have built a mart based on our solution.
 
On the tables being empty issue,  I am sending you (offline to  this list) two sql dumps for you to create a normalized table structure and to create the mart based on it.
 
These are the data dumps from the example we have given.
 
If you have any other issue please feel free to contact us.
 
Gnani

From: Ayton Meintjes [mailto:aytonm <at> gmail.com]
Sent: September 1, 2009 9:41 AM
To: Saravanamuttu Gnaneshan
Cc: mart-dev <at> ebi.ac.uk; Christina Yung
Subject: Re: FW: [mart-dev] Dynamic attribute set

Hi Gnani, Christina

Thanks for the quick response (and for providing an example), this is exactly what I was looking for. Got it working on a test dataset already.

regards
Ayton


Having some issues still though. The structure is being created but the tables are empty

2009/8/28 Saravanamuttu Gnaneshan <Saravanamuttu.Gnaneshan <at> oicr.on.ca>
Hello Ayton
 
We have now built a test mart to satisfy your requirements with some fake data as shown in your mail.
 
 
If the above link expires by the time you see this mail please use this url:  http://dcc-dev.res.oicr.on.ca:9030
 
 
This mart was built as explained in Christina's email and if you need to see the XML files created by the martbuilder and marteditor please email me offline (not through the mart-dev list).
 
If you have any queries on this please feel free to contact us.
 
Best wishes
 
Gnani
 

Saravanamuttu Gnaneshan
Scientist, Computational Biologist

Ontario Institute for Cancer Research
MaRS Centre, South Tower
101 College Street, Suite 800
Toronto, Ontario, Canada M5G 0A3
     
Tel:         647-260-7990
Toll-free: 1-866-678-6427
www.oicr.on.ca

 

 

This message and any attachments may contain confidential and/or privileged information for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this message in error, please contact the sender and delete all copies. Opinions, conclusions or other information contained in this message may not be that of the organization.


From: owner-mart-dev <at> ebi.ac.uk [mailto:owner-mart-dev <at> ebi.ac.uk] On Behalf Of Christina Yung
Sent: August 27, 2009 9:53 AM
To: Ayton Meintjes
Cc: mart-dev <at> ebi.ac.uk
Subject: RE: [mart-dev] Dynamic attribute set

Hi Ayton:
 
I think a mart with partititioned tables in the following structure will serve your needs:
 
Marker [id, rsid, allele_A, alleleB]
CEU_AlleleFrequency [fk_marker_id, CEU_freq]
JPT_AlleleFrequency [fk_marker_id, JPT_freq]
YRI_AlleleFrequency [fk_marker_id, YRI_freq]
 
In this mart, Marker is the main table where *_AlleleFrequency are the dimension tables.  This structure allows the addition of new populations as dimension tables.  Keep in mind that when multiple *_freq attributes are selected, multiple tables are joined.  Hence, scalability may become an issue if the number of populations is large.
 
To arrive at this mart, you may wish to use MartBuilder.  First, specify Marker as the main table.  MartBuilder will merge Population and AlleleFrequency into one dimension table [id, fk_population_id, fk_marker_id, freq, population_name].  The next step is to partition this dimension table by either fk_population_id or population_name.
 
Hope this helps.
 
Christina
 

From: owner-mart-dev <at> ebi.ac.uk [mailto:owner-mart-dev <at> ebi.ac.uk] On Behalf Of Ayton Meintjes
Sent: Wednesday, August 26, 2009 11:37 AM
To: mart-dev <at> ebi.ac.uk
Subject: [mart-dev] Dynamic attribute set

I have the following three tables in a custom database:

Population [id, name]
Marker [id, rsid, allele_A, allele_B]
AlleleFrequency [id, fk_population_id, fk_marker_id, freq]

How would I go about creating a mart that the user can query for markers and get results in the form:

rsid  | CEU_freq | JPT_freq | YRI_freq
rs12 |    0.05     |    0.55     |  0.75    
rs34 |    0.95     |    0.65     |  0.05    

In other words, the number of attributes available depends on how many unique populations there are? The automate push action and partitioning seem to suggest this sort of structure might be possible.
 
Thanks in advance

Christina Yung | 3 Sep 22:54 2009
Picon

RE: Biomart API for Ruby

Hi Daz:

A link to the API for Ruby is now avaiable on the Biomart homepage (www.biomart.org), under "Third Party
Software with BioMart Plugin".  Thank you for your contribution!  

Christina

> -----Original Message-----
> From: owner-mart-dev <at> ebi.ac.uk 
> [mailto:owner-mart-dev <at> ebi.ac.uk] On Behalf Of Darren Oakley
> Sent: Tuesday, September 01, 2009 4:59 AM
> To: mart-dev <at> ebi.ac.uk
> Subject: [mart-dev] Biomart API for Ruby
> 
> Hi All,
> 
> How's things going?
> 
> Just thought i'd let you all know about a pleasant by-product 
> of some of my recent work - I've just packaged up and 
> released a Biomart library/api for Ruby.  I was writing a lot 
> of code that queried various marts via the web services and 
> ended up copying around lots of boiler plate code, so it made 
> sense to modularize it. :)  It's quite basic for the moment 
> and only does single dataset queries, but it will be quite 
> easy to build upon the base from here.
> 
> The source is available from github 
> (http://github.com/dazoakley/biomart), and the docs are on 
> rdoc.info (http://rdoc.info/projects/dazoakley/biomart).
> 
> Fingers crossed this might be helpful to others! :)
> 
> Thanks,
> 
> Daz
> 
> 
> --
>  The Wellcome Trust Sanger Institute is operated by Genome 
> Research  Limited, a charity registered in England with 
> number 1021457 and a  company registered in England with 
> number 2742969, whose registered  office is 215 Euston Road, 
> London, NW1 2BE. 
> 

Darren Oakley | 4 Sep 10:09 2009
Picon

Re: Biomart API for Ruby

Cool! Thanks! :)

On 3 Sep 2009, at 21:54, Christina Yung wrote:

> Hi Daz:
>
> A link to the API for Ruby is now avaiable on the Biomart homepage (www.biomart.org 
> ), under "Third Party Software with BioMart Plugin".  Thank you for  
> your contribution!
>
> Christina
>
>> -----Original Message-----
>> From: owner-mart-dev <at> ebi.ac.uk
>> [mailto:owner-mart-dev <at> ebi.ac.uk] On Behalf Of Darren Oakley
>> Sent: Tuesday, September 01, 2009 4:59 AM
>> To: mart-dev <at> ebi.ac.uk
>> Subject: [mart-dev] Biomart API for Ruby
>>
>> Hi All,
>>
>> How's things going?
>>
>> Just thought i'd let you all know about a pleasant by-product
>> of some of my recent work - I've just packaged up and
>> released a Biomart library/api for Ruby.  I was writing a lot
>> of code that queried various marts via the web services and
>> ended up copying around lots of boiler plate code, so it made
>> sense to modularize it. :)  It's quite basic for the moment
>> and only does single dataset queries, but it will be quite
>> easy to build upon the base from here.
>>
>> The source is available from github
>> (http://github.com/dazoakley/biomart), and the docs are on
>> rdoc.info (http://rdoc.info/projects/dazoakley/biomart).
>>
>> Fingers crossed this might be helpful to others! :)
>>
>> Thanks,
>>
>> Daz
>>
>>
>> --
>> The Wellcome Trust Sanger Institute is operated by Genome
>> Research  Limited, a charity registered in England with
>> number 1021457 and a  company registered in England with
>> number 2742969, whose registered  office is 215 Euston Road,
>> London, NW1 2BE.
>>

--

-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 

Syed Haider | 7 Sep 16:17 2009
Picon
Picon

Re: Features visible in MartEditor, but they are not displayed in the web-interface

Hi Joachim,

could you please write a details on which tables you have added, names 
please ?

do these tables have at least 1 data row ?

what did you do to add this info in MartEditor? please write the series 
of steps.

thanks
Syed

Joachim Baran wrote:
> Hello!
> 
> I have added some custom tables to the functional_genomics_mart_55 and
> changed the mart's meta-information in MartEditor accordingly. While
> this worked beforehand when I modified VEGA marts and Ensembl marts,
> the attributes/filters regarding my extra data are not displayed in
> the web-interface. However, when I import/export the
> functional_genomics_mart_55 annotated_feature dataset, I can clearly
> see my extra data.
> 
> Is there an easy explanation to this?
> 
> Thanks,
> Joachim
> 

Joachim Baran | 7 Sep 15:23 2009
Picon

Features visible in MartEditor, but they are not displayed in the web-interface

Hello!

I have added some custom tables to the functional_genomics_mart_55 and
changed the mart's meta-information in MartEditor accordingly. While
this worked beforehand when I modified VEGA marts and Ensembl marts,
the attributes/filters regarding my extra data are not displayed in
the web-interface. However, when I import/export the
functional_genomics_mart_55 annotated_feature dataset, I can clearly
see my extra data.

Is there an easy explanation to this?

Thanks,
Joachim

--

-- 
B.1079 Michael Smith Building
Faculty of Life Sciences
The University of Manchester
Oxford Road
Manchester
M13 9PT
United Kingdom

Rhoda Kinsella | 7 Sep 15:57 2009
Picon
Picon

Re: Features visible in MartEditor, but they are not displayed in the web-interface

Hi Joachim,
Can you send me your meta tables (gzipped) so I can have a look at  
what you have modified in the configuration? Also send me a  
description of the data you wish to add and I will take a look at it.
Regards,
Rhoda

On 7 Sep 2009, at 14:23, Joachim Baran wrote:

> Hello!
>
> I have added some custom tables to the functional_genomics_mart_55 and
> changed the mart's meta-information in MartEditor accordingly. While
> this worked beforehand when I modified VEGA marts and Ensembl marts,
> the attributes/filters regarding my extra data are not displayed in
> the web-interface. However, when I import/export the
> functional_genomics_mart_55 annotated_feature dataset, I can clearly
> see my extra data.
>
> Is there an easy explanation to this?
>
> Thanks,
> Joachim
>
> -- 
> B.1079 Michael Smith Building
> Faculty of Life Sciences
> The University of Manchester
> Oxford Road
> Manchester
> M13 9PT
> United Kingdom

Rhoda Kinsella Ph.D.
Ensembl Bioinformatician,
European Bioinformatics Institute (EMBL-EBI),
Wellcome Trust Genome Campus,
Hinxton
Cambridge CB10 1SD,
UK.

Joachim Baran | 7 Sep 17:03 2009
Picon

Re: Features visible in MartEditor, but they are not displayed in the web-interface

Hello Syed,

First of all, I mean of course annotated_feature template config in my
previous post.

2009/9/7 Syed Haider <syed <at> ebi.ac.uk>:
> could you please write a details on which tables you have added, names
> please ?
I have added a table 'mscript_ann_fea_blast_cdna' table to the various
species, which links to the annotated features by its column
'annotated_feature_id'. I then run MartBuilder, update all schemas,
build & execute the SQL. There is plenty of data in the human and
mouse mart-tables, but the fruit-fly's table
dmelangoaster_regulatory_feature__mscript_ref_fea_blast_cdna__dm is
empty.

> what did you do to add this info in MartEditor? please write the series of
> steps.
In MartEditor I import the meta-info from martdb.ensembl.org and
export it to my mart, then do the following:
1. connect to my mart
2. run 'Update All'
3. 'Import' where I choose annotated_feature
4. have a look at the attributes to see if there is a new collection
with new attributes
5. 'Export'

Then I run 'perl bin/configure.pl --clean -r conf/myMart.xml'.

It is peculiar that the web-interface will not change even when I edit
entries in MartEditor which are displayed in the browser. I wonder
where the data comes from..

Thanks,
Joachim

--

-- 
B.1079 Michael Smith Building
Faculty of Life Sciences
The University of Manchester
Oxford Road
Manchester
M13 9PT
United Kingdom


Gmane