open_via_path / close file - WIndows CRT problems

Hi all,
when trying to trace a bug in one of my externals i came across what could be an issue in the PD API.

The header m_pd.h exposes the function open_via_path which can be used in externals to search file in the PD
search path.
On success it opens the file and returns a file handle, which can in turn be used by the external code.
This doesn't seem to be a problem on Unix-based systems.
On Windows, under certain circumstances [1], PD and the external use separate copies of the CRT, meaning
that passing the file handle and using POSIX-style file functions between dynamically linked binaries
is not really possible. The file handle would not be valid in the external.
This happens when at least one party is linked with the threaded static CRT, which is the default with PD and
most externals.

Now, i'm wondering how this problem can have been hidden for so long. Are there no externals using
open_via_path and running under Windows?
This must be a problem for all script loader externals relying on the PD path, like my py/pyext external.
Am i doing something wrong, or is there a known solution/workaround etc.?

Thanks and all the best,
Thomas

[1] http://msdn.microsoft.com/en-us/library/ms235460(v=vs.80).aspx

--
Thomas Grill
http://grrrr.org

Re: open_via_path / close file - WIndows CRT problems

On 2012-07-03 15:19, Thomas Grill wrote:
> Hi all, when trying to trace a bug in one of my externals i came
> across what could be an issue in the PD API.
> 
> The header m_pd.h exposes the function open_via_path which can be
> used in externals to search file in the PD search path. On success
> it opens the file and returns a file handle, which can in turn be
> used by the external code. This doesn't seem to be a problem on
> Unix-based systems. On Windows, under certain circumstances [1], PD
> and the external use separate copies of the CRT, meaning that
> passing the file handle and using POSIX-style file functions
> between dynamically linked binaries is not really possible. The
> file handle would not be valid in the external. This happens when
> at least one party is linked with the threaded static CRT, which is
> the default with PD and most externals.
> 
> Now, i'm wondering how this problem can have been hidden for so
> long. Are there no externals using open_via_path and running under
> Windows? This must be a problem for all script loader externals
> relying on the PD path, like my py/pyext external. Am i doing
> something wrong, or is there a known solution/workaround etc.?

Pd-0.43 introduced sys_close() in order to have the same CRT
implementation open and close the file.

prior version of Pd lack this function and therefore there a number of
file-handle leakage bugs were reported.

fgadsrm

Re: open_via_path / close file - WIndows CRT problems

On 2012-07-03 15:40, IOhannes m zmoelnig wrote:
> 
> Pd-0.43 introduced sys_close() in order to have the same CRT 
> implementation open and close the file.
> 
> prior version of Pd lack this function and therefore there a number
> of file-handle leakage bugs were reported.
> 

sp the workaround is actually to use open_via_path() to get the full
patch of the file you want to open, sys_close() the filehandle and
then use the full-path to open the file yourself.

fgmasdr
IOhannes

_______________________________________________
Pd-dev mailing list
Pd-dev@...
http://lists.puredata.info/listinfo/pd-dev

Re: open_via_path / close file - WIndows CRT problems

It's a workaround, yes.. but it's frequently necessary even within Pd
where "open" wasn't really the desired function but one still needed to
travers the search path to find a readable file.  There ought to be a
better way but I don't know what it is.  An example is
binbuf_read_via_canvas().

On another note, open_via_path isn't threadsafe - if one thread is changing
the path it's unsafe for another to chase it.  This happend in sfread~/
sfwrite~ and I'm still trying to think of a good fix.

cheers
Miller

On Tue, Jul 03, 2012 at 03:49:00PM +0200, IOhannes m zmoelnig wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 2012-07-03 15:40, IOhannes m zmoelnig wrote:
> > 
> > Pd-0.43 introduced sys_close() in order to have the same CRT 
> > implementation open and close the file.
> > 
> > prior version of Pd lack this function and therefore there a number
> > of file-handle leakage bugs were reported.
> > 
> 
> sp the workaround is actually to use open_via_path() to get the full
> patch of the file you want to open, sys_close() the filehandle and
> then use the full-path to open the file yourself.
> 

Re: open_via_path / close file - WIndows CRT problems

I see, thanks! That's perfect - sorry for not being up to date.
gr~~~

Am 03.07.2012 um 15:49 schrieb IOhannes m zmoelnig:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 2012-07-03 15:40, IOhannes m zmoelnig wrote:
>> 
>> Pd-0.43 introduced sys_close() in order to have the same CRT 
>> implementation open and close the file.
>> 
>> prior version of Pd lack this function and therefore there a number
>> of file-handle leakage bugs were reported.
>> 
> 
> sp the workaround is actually to use open_via_path() to get the full
> patch of the file you want to open, sys_close() the filehandle and
> then use the full-path to open the file yourself.
> 
> fgmasdr
> IOhannes
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.12 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> 
> iEYEARECAAYFAk/y+EkACgkQkX2Xpv6ydvTkCwCfRvO9WVlNjg2/AZ0unQgs6qYs
> xLkAnAwTrGl7B/uqClNFrJP0o1VDzEpc
> =oSId

Re: [PD] [PD-announce] pd 0.43-3 released

On 2012-07-03 23:11, Miller Puckette wrote:
> Hi all,
> 
> Pd version 0.43-3 is available on
> http://crca.ucsd.edu/~msp/software.htm

cool.

> 
> I'm ready to start hacking on 0.44.  The most urgent thing seems to
> be for me to go back and work on the audio system (and perhaps
> tweak MIDI a bit more too).

i have a project lying on my hardisk for >1 year (iirc, shortly before
the 0.32.0 release), that is related to that: separating the audio
system(s) from Pd-core.

background:
currently Pd supports a number of audio-backends.
 while these backends are mainly separated into several source files
(e.g. s_audio_alsa.c for ALSA), there is a lot of theoretically
backend-agnostic code (s_audio, s_midi, s_main) cluttered with "#ifdef
USEAPI_ALSA" and the like.
this makes the current code badly readable and thus maintainable, it
also makes it hard to add new audio systems as backends (and the more
you add, the more complicated the codebase gets).

more background:
currently all audio backends are linked to the main "Pd" binary.

Re: [PD] [PD-announce] pd 0.43-3 released

Le 04/07/2012 09:51, IOhannes m zmoelnig a écrit :

> furthermore, Pd supports outdated/deprecated backends like OSS (both
> audio and midi).
> in practice i haven't used OSS-midi for a number of years (and i doubt
> whether it is actually still supported by the mainline kernels), and i
> used OSS-audio only seldomly in very specific setups.

i think oss-midi is still in ubuntu 12.04 and i'll still using it.
oss-audio has been removed, but i'm still using it on older ubuntu release : i've got lot's better
performance than alsa, and it's more plug and play than jack (obviously in some simple (stereo) setup).

cheers
Cyrille

Re: [PD] [PD-announce] pd 0.43-3 released

Also... I think OSS/MIDI is the only API now that lets you spit out arbitrary
byes over the MIDI line -- all the others 'protect' you.

cheers
Miller

On Wed, Jul 04, 2012 at 08:43:50PM +0200, Cyrille Henry wrote:
> 
> 
> Le 04/07/2012 09:51, IOhannes m zmoelnig a écrit :
> 
> >furthermore, Pd supports outdated/deprecated backends like OSS (both
> >audio and midi).
> >in practice i haven't used OSS-midi for a number of years (and i doubt
> >whether it is actually still supported by the mainline kernels), and i
> >used OSS-audio only seldomly in very specific setups.
> 
> i think oss-midi is still in ubuntu 12.04 and i'll still using it.
> oss-audio has been removed, but i'm still using it on older ubuntu release : i've got lot's better
performance than alsa, and it's more plug and play than jack (obviously in some simple (stereo) setup).
> 
> cheers
> Cyrille
> 
> _______________________________________________
> Pd-dev mailing list
> Pd-dev@...
> http://lists.puredata.info/listinfo/pd-dev

Re: [PD] [PD-announce] pd 0.43-3 released

On 2012-07-04 21:12, Miller Puckette wrote:
> Also... I think OSS/MIDI is the only API now that lets you spit out
> arbitrary byes over the MIDI line -- all the others 'protect' you.

i'm not saying that i want to remove OSS-MIDI.
i'm saying that i want to allow the user to remove OSS-MIDI, if they
don't care for it.

to repeat: my patch is about adding more audio systems without
touching the Pd-core. it's not about removing unused backends (though
about making them optional)

fgamsdr
IOhannes

_______________________________________________
Pd-dev mailing list
Pd-dev@...
http://lists.puredata.info/listinfo/pd-dev

Compilation failure on Lucid 64

_______________________________________________
Pd-dev mailing list
Pd-dev@...
http://lists.puredata.info/listinfo/pd-dev