4 Feb 15:27
Cat on (RAM) steroids
Hi all, I need a simple command-line program, which works as a plain FIFO stream buffer with a huge RAM. Something I can do: $ fatcat -b 256M /dev/datasource | ./my_shaky_data_sink The idea is that "fatcat" reads data whenever available and stores it to non-swappable RAM. It then pushes the data to stdout. So it's just like good old "cat", only with a potentially large tummy. Rationale: The (kernel) device /dev/datasource has limited RAM it can allocate in kernel space. If data arrives at a high constant rate (say, 100 MB/sec), the kernel buffers (4 MB total?) overflow rather quickly if not constantly drained. A real-life data sink may write the data to a disk with sufficient *average* rate, but with breaks every now and then to mind its own business. So if data is loaded into a huge RAM array (what is 256 MB these days?) by a high-priority thread (dare I say "real-time"?), there's a good chance the kernel buffers are emptied quick enough. It's rather straightforward to write a program doing this with POSIX threads, but why bother if it's already available? After all, I can't be the first one in the world with this need. My own search lead me at best to cstream, which doesn't look like it's tailored for solving realtime issues. Ideas, anyone?(Continue reading)
RSS Feed