Grant Farnsworth | 6 Aug 02:23 2013

RSQLite and transparent compression

I frequently create, write to, and read from large SQLite databases
from within R using RSQLite, sometimes across the network.  As
size/performance can be a consideration for me, I find myself wishing
I could compress my sqlite database files and interact with them
directly, the way one can interact with a gzipped text file using
read.table() or scan().  In case anyone wasn't aware, you can read
from a gzipped  text file without any additional arguments

G <- read.table("myfile.txt.gz")

and so forth.

The syntax doesn't have to be that simple, but does anyone know of a
workaround to allow me to read from an SQLite database that has been
compressed?   Or if you have a different solution/workaround, I would
love to hear it.  Thanks!

R-sig-DB mailing list -- R Special Interest Group

Simon Zehnder | 16 Jul 10:39 2013

Serialize data.frame to database

Dear all,

I need a very fast and reliable database solution so I try to serialize a data.frame (to binary data) and to
store this data to an SQLite database (I might be wrong with this approach and a data.frame can be
efficiently and faster stored and fetched in a database row in a different manner - please feel free to make
suggestions. It is just what I know from Java how to store a matrix). 

This is what I tried to do:

con <- dbDriver("SQLite")
db <- dbConnect(con, "test")
dbSendQuery(db, 'CREATE TABLE frames("simID" INT, "data" BLOB)')
data.bin <- serialize(iris, NULL, ascii = FALSE)
dbSendQuery(db, paste("INSERT INTO frames VALUES(1, X'", data.bin, "')", sep = ""))
data.bin2 <- dbGetQuery(db, "SELECT DATA FROM frames WHERE simID = 1")
1   58

So, only the first entry of data.bin is saved to the database. I tried to first convert the binary data to raw data:
data.raw <- rawToChar(data.bin)
Error in rawToChar(data.bin) :
 embedded nul in string: 'X\n\0\0\0\002\0\003\0\001\0\002\003\0\0\0\003\023\0\0\0\005\0\0\0\016\0\0\0\x96 <at> \024ffffff <at> \023\x99\x99\x99\x99\x99\x9a <at> \022\xcc\xcc\xcc\xcc\xcc\xcd <at> \022ffffff <at> \024\0\0\0\0\0\0 <at> \025\x99\x99\x99\x99\x99\x9a <at> \022ffffff <at> \024\0\0\0\0\0\0 <at> \021\x99\x99\x99\x99\x99\x9a <at> \023\x99\x99\x99\x99\x99\x9a <at> \025\x99\x99\x99\x99\x99\x9a <at> \023333333 <at> \023333333 <at> \021333333 <at> \027333333 <at> \026\xcc\xcc\xcc\xcc\xcc\xcd <at> \025\x99\x99\x99\x99\x99\x9a <at> \024ffffff <at> \026\xcc\xcc\xcc\xcc\xcc\xcd <at> \024ffffff <at> \025\x99\x99\x99\x99\x99\x9a <at> \024ffffff <at> \022ffffff <at> \024ffffff <at> \023333333 <at> \024\0\0\0\0\0\0 <at> \024\0\0\0\0\0\0 <at> \024\xcc\xcc\xcc\xcc\xcc\xcd <at> \024\xcc\xcc\xcc\xcc\xcc\xcd <at> \022\xcc\xcc\xcc\xcc\xcc\xcd <at> \023333333 <at> \025\x99\x99\x99\x99\x99\x9a <at> \024\xcc\xcc\xcc\xcc\xcc\xcd <at> \026\0\0\0\0\0\0 <at> \023\x99\x99\x99\x99
 \x99\x9a <at> \024\0\0\0\0\0\0 <at> \026\0\0\0\0\0\0 <at> \023\x99\x99\x99\x99\x99\x9a <at> \021\x99\x99\x99\x99\x99\x9a <at> \024ffffff <at> \024\0\0\0\0\0\0 <at> \022\0\0\0\0\0\0 <at> \021\x99\x99\x99\x99\x99\x9a <at> \024\0\0\0\0\0!

I don't know what this error should tell me. Then I tried to use the ASCII format

data.ascii <- serialize(iris, NULL, ascii = TRUE)
(Continue reading)

Daniel Reis | 11 Jul 21:29 2013

[RODBC] Error (ORA-12560 ) when trying to connect to Oracle database


I installed the Oracle 11g client on my machine, and tested the
connection to my target Oracle database through MS Access. All seems to
be working fine.

When I try to connect to the very same DB using R with:

channel <- odbcConnect(dsn="MyDSN", uid="XXXXX", pwd="XXXXXXXX",

I get the following error:

In odbcDriverConnect("DSN=MyDSN;UID=XXXX;PWD=XXX",  :

[RODBC] ERROR: state HY000, code 12560, message
[Oracle][ODBC][Ora]ORA-12560: TNS:protocol adapter error

Am I missing the installation of some component/driver? I read something
about installing the Oracle Instant Client (free) but thought the Oracle
11g client would be sufficient. What am I missing?


	[[alternative HTML version deleted]]

R-sig-DB mailing list -- R Special Interest Group
(Continue reading)

Chris Colburn | 9 May 14:48 2013

RODBC error when connecting to postgres

Hello All,

I’m trying to connect via ODBC to a postgres database. Although I’ve been
able to test my ODBC configurations using isql I am now getting the
following error in R.  This seems like a memory issue, but cannot tell if
the ‘calloc’ error is a red herring.  Let me know if you need any more
information.  Many thanks in advance!

> library(RODBC)> myconn <- odbcConnect("postgres_dsn", uid="test",
> sqlQuery(myconn,"select count(*) from test_table;")
Error in odbcQuery(channel, query, rows_at_time) :  'Calloc' could not
allocate memory (18446744073709551616 of 22816 bytes)

> versionplatform       x86_64-suse-linux-gnu
arch           x86_64
os             linux-gnu
system         x86_64, linux-gnu
major          3
minor          0.0
year           2013
month          04
day            03
svn rev        62481
language       R
version.string R version 3.0.0 (2013-04-03)
nickname       Masked Marvel

Any help that you can provide is greatly appreciate, and I will try to
(Continue reading)

Dirk Eddelbuettel | 23 Apr 15:22 2013

Re: GSoC R project (Port pymssql core to R)

Hi Taras,

On 23 April 2013 at 16:05, Taras Murzhiev wrote:
| Hi Dirk,
| My name is Taras Murzhiev, I am a last-year student from Ukraine and want to
| participate in Google Summer of Code this year.
| I have experience in programming (C++, Python, less Matlab and R) and math with

For this project, you may to look a little at the R packages (all on CRAN)

    DBI	 	  -- generic database interface

    RSQLite	  -- use DBI to access SQLite
    RMySQL	  --  "   "   "    "   MySQL
    RPostgreSQL   --  "   "   "    "   PostgreSQL
    ROracle       --  "   "   "    "   Oracle

depending on what backend you have. SQLite does NOT need a server and is
easiest to get going.

| theory background.
| I'm interested in working on project 'Port pymssql core to R and DBI as
| RMSSql' that you proposed on GSoC R wiki page.
| It would be very pleasant if you could briefly describe main goals, expectation
| and usage aspect of this project and your vision of techical side and
| implementation details that give me ability to go forward.

(Continue reading)

Paul Gilbert | 5 Apr 19:32 2013

RMySQL cannot allocate a new connection

  I have recently started getting error messages

Error: RS-DBI driver: (d/mysqld.sockcannot allocate a new connection -- 
of 16 connections already opened)

The server has a max connections set to 500, which is verified from the 
R session with

    mysqlQuickSQL(ets, "show variables;")

so the error message does not seem to be reflecting the real problem. I 
am also getting

Warning in mysqlQuickSQL(conn, statement, ...) : pending rows
Warning in mysqlFetch(res, n, ...) :
   RS-DBI driver warning: (error while fetching rows)

I am using  RMySQL 0.9-3 (on Ubuntu)  and am now using R-3.0.0 but the 
problem started before the new R version.  The same code did work in the 
past (circa Dec 2012), but I'm not certain about the version numbers 
then. I've probably upgraded my server since then too.

Does anyone know how to deal with this, or what the cause is?

Is anyone else having a similar problem?


(Continue reading)

Mark Nadel | 2 Apr 19:09 2013

RODBC Filemaker connection on a Mac

I am trying to communicate through RODBC with a remote Filemaker database.
Searching the web has turned up a number of recipes, but I have not been
able to make any of them work.
Does anyone currently have RODBC working successfully with a remote
Filemaker database? If so, I would appreciate knowing  what driver you are
using and how you do the necessary setup. The more detailed and explicit
the directions, the better.

Thanks in advance,



*Mark Nadel*
*Principal Scientist
Nabsys Inc.
60 Clifford Street
Providence, RI  02903

Phone   401-276-9100 x204
Fax 401-276-9122

	[[alternative HTML version deleted]]

R-sig-DB mailing list -- R Special Interest Group

(Continue reading)

Sean Davis | 20 Mar 19:37 2013

neo4j graph database for R

Dirk got this started with his interest in a mssql R interface.  Is anyone
aware of an R interface to the neo4j graph database?

The main interface to the server is REST over http, so the infrastructure
can be native R, potentially.  Interoperability between igraph and graph
packages with neo4j might be interesting as well.  Would this be an
appropriate Google Summer of Code endeavor?


	[[alternative HTML version deleted]]

R-sig-DB mailing list -- R Special Interest Group

Dirk Eddelbuettel | 19 Mar 23:26 2013

R port of pymssql

I just came across

Maybe someone wants to run with this and create 'RMSSql' using the DBI

The Google Summer of Code framework served us well when Sameer started
RPostgreSQL a few years ago (which I had suggested and mentored).  Now, I am
not volunteering to mentor again (particularly as I, luckily, get by without
having to use mssql).  But in case someone a) needs mssql and b) has some
capacity to (co-)mentor this...



Dirk Eddelbuettel | edd@... |

R-sig-DB mailing list -- R Special Interest Group

Elliot Bernstein | 11 Mar 18:35 2013

Microsoft SQL Server Temp Tables and RODBC

I've noticed that RODBC has some issues with MS SQL Server's syntax for temp tables. For example:

> temp <- data.frame(x = 1:10)
> sqlSave(conn, temp, "#rtemp")
Error in sqlColumns(channel, tablename) :
  '#rtemp': table not found on channel
> sqlQuery(conn, "create table #rtemp (x integer)")
[1] "42S01 2714 [Microsoft][ODBC SQL Server Driver][SQL Server]There is already an object named
'#rtemp' in the database."
[2] "[RODBC] ERROR: Could not SQLExecDirect 'create table #rtemp (x integer)'"
> sqlSave(conn, temp, "#rtemp", append = TRUE)
Error in sqlSave(conn, temp, "#rtemp", append = TRUE) :
  42S01 2714 [Microsoft][ODBC SQL Server Driver][SQL Server]There is already an object named '#rtemp' in
the database.
[RODBC] ERROR: Could not SQLExecDirect 'CREATE TABLE "#rtemp"  ("rownames" varchar(255), "x" int)'
> sqlQuery(conn, "select * from #rtemp")
[1] rownames x
<0 rows> (or 0-length row.names)
> for (i in 1:nrow(temp)) { sqlQuery(conn, sprintf("insert into #rtemp (x) values (%d)", temp$x[i])) }
> sqlQuery(conn, "select * from #rtemp")
   rownames  x
1        NA  1
2        NA  2
3        NA  3
4        NA  4
5        NA  5
6        NA  6
7        NA  7
8        NA  8
9        NA  9
(Continue reading)

Joran Elias | 11 Mar 16:27 2013

ROracle loads from command line, but not RGUI/RStudio in OSX

I'm having some trouble getting ROracle up and running completely on my
computer (OS X 10.7.5, R 2.15.2).

Compiling ROracle was a little bit of an adventure, but I finally did get
it to compile by mostly following the Linux instructions with some
modifications pieced together from the web. When I run R from the command
line, ROracle loads fine, I can connect to the db and run queries.
Everything works. (I've also installed the Oracle utility sqlplus which
works as well; connecting, querying, etc.)

But if I try to load ROracle from either the R GUI or RStudio, I get this

Loading required package: DBI
Error in dyn.load(file, DLLpath = DLLpath, ...) :
  unable to load shared object

6): Library not loaded:
  Referenced from:
  Reason: image not found
Error: package/namespace load failed for ‘ROracle’

I examined how the environment variables are set between running R from the
command line and RGUI/RStudio, and noticed some differences, so I tried
making them the same via setting:

(Continue reading)