changeset 1:aa9d023a61a2 draft

ckpt
author Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
date Fri, 22 May 2020 00:10:47 -0400
parents 10fecf6362e1
children 63dfe6ae95f6
files notes.md
diffstat 1 files changed, 24 insertions(+), 8 deletions(-) [+]
line wrap: on
line diff
--- a/notes.md	Wed May 20 14:33:16 2020 -0400
+++ b/notes.md	Fri May 22 00:10:47 2020 -0400
@@ -4,7 +4,7 @@
 There are three concepts necessary to reason about containers:
 
 * packet: A variable length chunk of data.  The only "metadata" associated
-  with in is a logical timestamp of when the first byte of the data was
+  with it is a logical timestamp of when the first byte of the data was
   produced.  The type of the packet is *not* stored by the packet itself.
 * stream: An ordered sequence of packets.  Each stream is identified by a
   unique id (a random integer).  In general, all streams are handled
@@ -13,15 +13,24 @@
 * session: A set of streams representing a recording session.  E.g., all the
   data (I/Q samples) and metadata (timestamps, device info, center
   frequency, bandwidth, etc.) captured between when the recording started
-  until it was stopped.
+  and when it was stopped.
 
-Note that there are no other structures.  Any data that is to be stored,
+Note that there are no other structures.  Any data that is to be stored
 must be stored in a packet in a stream.
 
+These concepts are rather generic.  The next section talks about how DSP-
+related data and metadata can be mapped to them.
+
 
 DSP
 ===
 
+A typical DSP application reads and writes a sequence of samples making up a
+signal.  The metadata associated with the data (e.g., center frequency,
+sample rate) 
+
+
+
 TODO: describe how DSP workload maps to these concepts
 TODO: what's in a packet? what's in a stream? what streams are there?
 
@@ -34,7 +43,7 @@
 2. I/Q metadata
 
 There is no reason to limit the format to just one I/Q stream.  Multiple I/Q
-streams can be generated with phase coherent multi-antenna setups.  (I'm
+streams can be generated with phase-coherent multi-antenna setups.  (I'm
 thinking things like KerberosSDR or other/fancier setups.)  In such a setup,
 we might see something like this:
 
@@ -50,6 +59,7 @@
 
 TODO: Note that the ordering of the streams does not matter.
 
+
 Mapping to Existing Container Formats
 =====================================
 
@@ -65,7 +75,7 @@
 therefore may be a better choice.  Both are equally simple to implement.)
 
 * packet -> file with filename `<stream unique>-<logical timestamp>`
-* stream -> files with filenames `<stream unique>-*`, in lexographic order
+* stream -> files with filenames `<stream unique>-*`, in lexicographic order
 * session -> the directory/tar/cpio containing the files
 
 
@@ -84,7 +94,7 @@
 
 It may be beneficial to start with a hybrid approach where the I/Q samples
 are appended to a single large file (as is the case today) and all the other
-streams are serialized into a richer container.  For example, suppose that a
+streams are serialized into a richer container.  For example, suppose that
 a recording session produces two files:
 
 ```
@@ -94,10 +104,16 @@
 
 The `.raw` contains the samples as before.  The `.ogg` file contains the
 remaining streams serialized into an Ogg format.  This would yield all the
-benefits of having a richer format but only produce 
+benefits of having a richer format without the initial incompatibility with
+other software.
 
 
 Other Existing Container Formats
 --------------------------------
 
-* RIFF: has 32-bit file offsets, RF64 extends it to 64-bit
+Here are some other existing container file formats that I looked at and
+decided that they are too cumbersome for some reason.
+
+* RIFF: 32-bit file sizes, very messy spec
+* RF64: 64-bit version of RIFF, very messy spec
+* MKV: requires EBML implementation