headers
Leif Asbrink | 4 Jan 03:15

Re: Network standards for SDR

Hi All,

Alberto wrote on the Winrad mailing list:
> The network model is very general, and could allow 
> even to have Winrad and MAP65 on different PCs, 
> maybe one with Linux and the other with Windows. 
> But this could be a long range target. 

Thanks to the assistance from ON4IY Linrad now
has a network that can send data in seven different
formats and number of bits:
raw 16
raw 18
raw 24
fft1 32
timf2 16 or 32
fft2 16 or 32

The first four formats can be used for input to Linrad. 
timf2, the input to fft2 and fft2 will be sent as 16 bit 
integers if the user selected the MMX implmentations of the 
FFT algorithms, otherwise they are 32 bit floating point. 
Besides broadcasting one or more of these formats the computer 
that owns the hardware, the master, has a simple server to 
which slaves can connect and request calibration data as well 
as post what frequency the operator is focussing on for all 
other computers on the network to know.

Right now, only the raw data broadcasts and the client server 
communication with calibration data works. I see no problems
(Continue reading)

Permalink | Reply | 0 comments |
headers
Leif Asbrink | 4 Jan 03:15

Re: Network standards for SDR

Hi All,

Alberto wrote on the Winrad mailing list:
> The network model is very general, and could allow 
> even to have Winrad and MAP65 on different PCs, 
> maybe one with Linux and the other with Windows. 
> But this could be a long range target. 

Thanks to the assistance from ON4IY Linrad now
has a network that can send data in seven different
formats and number of bits:
raw 16
raw 18
raw 24
fft1 32
timf2 16 or 32
fft2 16 or 32

The first four formats can be used for input to Linrad. 
timf2, the input to fft2 and fft2 will be sent as 16 bit 
integers if the user selected the MMX implmentations of the 
FFT algorithms, otherwise they are 32 bit floating point. 
Besides broadcasting one or more of these formats the computer 
that owns the hardware, the master, has a simple server to 
which slaves can connect and request calibration data as well 
as post what frequency the operator is focussing on for all 
other computers on the network to know.

Right now, only the raw data broadcasts and the client server 
communication with calibration data works. I see no problems
(Continue reading)

Permalink | Reply | 0 comments |
headers
J.D. Bakker | 4 Jan 04:02
Picon

Re: Network standards for SDR

>// Structure for multicasting receive data on the network.
>#define NET_MULTICAST_PAYLOAD 1024
>typedef struct {
>char buf[NET_MULTICAST_PAYLOAD];
>double passband_center;
>float userx_freq;
>unsigned int block_no;
>unsigned char userx_no;
>char passband_direction;
>} NET_RX_STRUCT;

Very interesting ! A couple of observations:

- I would want a timestamp in there somewhere. It might be derived 
from  block_no, but why not make it explicit ?
- how is the sampling rate communicated ?
- using float/double makes it much harder for dedicated hardware 
receivers to act as server.
- if you are not doing so already, please please _please_ use the 
functions htons() / ntohs() and friends to convert between host byte 
order and network byte order (or forever determine that linrad 
communicates with either little endian (IA32) or big endian (Alpha, 
PowerPC etc) byte order. I would want to be able to use a PC as the 
server and my PowerBook as the client, for instance.

JDB.
--

-- 
LART. 250 MIPS under one Watt. Free hardware design files.
http://www.lartmaker.nl/
(Continue reading)

Permalink | Reply | 0 comments |
headers
Leif Asbrink | 4 Jan 05:08

Re: Network standards for SDR

On Thu, 4 Jan 2007 04:02:10 +0100
"J.D. Bakker" <jdb@...> wrote:

> >// Structure for multicasting receive data on the network.
> >#define NET_MULTICAST_PAYLOAD 1024
> >typedef struct {
> >char buf[NET_MULTICAST_PAYLOAD];
> >double passband_center;
> >float userx_freq;
> >unsigned int block_no;
> >unsigned char userx_no;
> >char passband_direction;
> >} NET_RX_STRUCT;
> 
> Very interesting ! A couple of observations:
> 
> - I would want a timestamp in there somewhere. It might be derived 
> from  block_no, but why not make it explicit ?
I do not see what it would be good for. Why do you want the clock
from the master while there is another one in the slave? 

Surely I could add this, but there is a cost to it.
The packages are sent out at maybe 1 kHz. The corresponding
time resolution is 1 ms. There is an appreciable time jitter 
however and it is not obvious what time to put into
a specific package. Presumably it should be the time
when the first sample within the package was taken from 
the hardware. What would it be good for? It is not trivial
to evaluate.
(Continue reading)

Permalink | Reply | 0 comments |
headers
J.D. Bakker | 4 Jan 06:00
Picon

Re: Network standards for SDR

A general point: in virtually all communications protocols the 
(descriptive) header comes before the data block, since the receiver 
usually needs to decode the header to be sure what to do with the 
data. This also makes it possible to vary the length of the data 
block, if desired (for instance, to tune to FFT block sizes or 
sampling hardware word length).

>  > - I would want a timestamp in there somewhere. It might be derived
>>  from  block_no, but why not make it explicit ?
>I do not see what it would be good for. Why do you want the clock
>from the master while there is another one in the slave?

Array processing. It would be very useful for a situation where you 
have multiple masters on one network (either during a contest, or -in 
my case- with a few servers each connected to an antenna+receiver). 
Time sync is not hard over either GPS/TAC or ntp.

Even in one-master situations it could be useful: with timestamps, it 
is very easy to make something similar to the Time Machine.

>  > - how is the sampling rate communicated ?
>The slave(client) asks the server for the meaning of the data.
>Number of channels, nominal sampling rate, whether the format is
>real or complex etc.

That, too, makes it harder for dedicated hardware receivers; ideally 
these would not need _any_ communication from the slave to the 
master. As I see it, encoding this information in the header of each 
package is a low-overhead way to reduce ambiguity, too.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Joe Taylor | 4 Jan 15:51
Picon
Favicon

Re: Network standards for SDR

Hi Leif and all,

>>- I would want a timestamp in there somewhere. It might be derived 
>>from  block_no, but why not make it explicit ?
> 
> I do not see what it would be good for. Why do you want the clock
> from the master while there is another one in the slave? 
> 
> Surely I could add this, but there is a cost to it.
> The packages are sent out at maybe 1 kHz. The corresponding
> time resolution is 1 ms. There is an appreciable time jitter 
> however and it is not obvious what time to put into
> a specific package. Presumably it should be the time
> when the first sample within the package was taken from 
> the hardware. What would it be good for? It is not trivial
> to evaluate.

I agree that a timestamp will be useful.  For what I am 
thinking about, very high precision and high accuracy are 
not required.  JT65 wants to know the UTC of a data block to 
within a second or so.  (Relative timing among successive 
blocks is of course maintained by the fixed and nominally 
known sample rate.)  Of course the slave computer could use 
its own time, but that will add another level of jitter.

As a minimum, I suggest that each packet include as a 32-bit 
integer the number of seconds since the Unix epoch, 
according to the master computer's system clock.  So much 
the better if you include a number of milliseconds as a 
second number, or combine both into a double.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Leif Asbrink | 4 Jan 15:58

Re: Network standards for SDR

Hi JDB and all,

> A general point: in virtually all communications protocols the 
> (descriptive) header comes before the data block, since the receiver 
> usually needs to decode the header to be sure what to do with the 
> data. This also makes it possible to vary the length of the data 
> block, if desired (for instance, to tune to FFT block sizes or 
> sampling hardware word length).

My primary goal is radio and experimenting with radio receivers.
Today we can do many things inside computers and I want to
make it easy for newcomers to enter radio experimenting from 
the digital side. That is why I have placed the fixed size data 
block in front of the header. 

The newcomer who wants to write his own software does not have 
to know anything about the header, he can just use the 1024 
bytes of data and ignore whatever has been appended. Having a 
header which has to be properly decoded in order to extract
the data builds a threshold that makes it more difficult to 
get started. (Processing simple .wav files has a pretty high
threshold in decoding the header. Common practice between
amateurs has been to just dicard the header and read the actual
data using the information supplied with the file. Those 
who worked with the UNKN422 challenge were adviced to do
so for example.)  

> >  > - I would want a timestamp in there somewhere. It might be derived
> >>  from  block_no, but why not make it explicit ?
> >I do not see what it would be good for. Why do you want the clock
(Continue reading)

Permalink | Reply | 0 comments |
headers
Leif Asbrink | 4 Jan 17:47

Re: Network standards for SDR

Hi Joe and all,

> I agree that a timestamp will be useful.  For what I am 
> thinking about, very high precision and high accuracy are 
> not required.  JT65 wants to know the UTC of a data block to 
> within a second or so.  (Relative timing among successive 
> blocks is of course maintained by the fixed and nominally 
> known sample rate.)  Of course the slave computer could use 
> its own time, but that will add another level of jitter.
> 
> As a minimum, I suggest that each packet include as a 32-bit 
> integer the number of seconds since the Unix epoch, 
> according to the master computer's system clock.  So much 
> the better if you include a number of milliseconds as a 
> second number, or combine both into a double.
> 
> Never mind about jitter; the receiving program would need to 
> know that jitter in the time values will exist, and behave 
> accordingly.
> 
> Together with the block number, these approaches will 
> suffice very nicely for JT65, anyway.  They will work better 
> and more reliably than having the slave computer use its own 
> system clock.
OK. This is what it looks like under Windows:

int lir_get_epoch_seconds(void)
{
// Here we have to add a calendar to add the number
// of seconds from todays (year, month, day) to Jan 1 1970.
(Continue reading)

Permalink | Reply | 0 comments |
headers
J.D. Bakker | 4 Jan 18:07
Picon

Re: Network standards for SDR

>The newcomer who wants to write his own software does not have
>to know anything about the header, he can just use the 1024
>bytes of data and ignore whatever has been appended. Having a
>header which has to be properly decoded in order to extract
>the data builds a threshold that makes it more difficult to
>get started. (Processing simple .wav files has a pretty high
>threshold in decoding the header. Common practice between
>amateurs has been to just dicard the header and read the actual
>data using the information supplied with the file. Those
>who worked with the UNKN422 challenge were adviced to do
>so for example.)

WAV is an example of a file format where *everyone* added their own 
custom headers/chunks, without any planning. As a result, no program 
can read all existing WAV files; WAV is considered an example how 
*not* to do a file format.

Would you consider a very simple, easy to parse header like this:

typedef struct {
   short header_len;  // This field is always present, and always first
   short data_len;    // This field is always present, and always second
   [ header contents, including header version, type, etc. goes here ]
   char data[];
} NET_RX_STRUCT;

This is still easy to parse, since all a user needs to do is something like

   struct NET_RX_STRUCT *rx_packet;
   char *my_data;
(Continue reading)

Permalink | Reply | 0 comments |
headers
Joe Taylor | 4 Jan 18:41
Picon
Favicon

Re: Network standards for SDR

Leif and all,

> Would you agree on milliseconds since midnight? From JDB I
> learned that a double with seconds since Unix epoch 
> would be a bad idea since conversion may be difficult on 
> non-PC platforms. (It is the internal time format within
> Linrad however) 

Yes, milliseconds since 0000 UTC would be OK.  Maybe you 
should send BOTH this quantity AND a double with seconds 
since Unix epoch (which I would actually prefer).  I don't 
see the conversion issue as a big deal; little-endian to 
big-endian copnversion is trivial, and doesn't nearly 
everybody use IEEE floating point these days?

	-- Joe

#############################################################
This message is sent to you because you are subscribed to
  the mailing list <linrad@...>.
To unsubscribe, E-mail to: <linrad-off@...>
To switch to the DIGEST mode, E-mail to <linrad-digest@...>
To switch to the INDEX mode, E-mail to <linrad-index@...>
Send administrative queries to  <linrad-request@...>
Permalink | Reply | 0 comments |

Gmane