changeset 2:63dfe6ae95f6 draft

ckpt
author Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
date Sat, 23 May 2020 00:30:17 -0400
parents aa9d023a61a2
children 52b73912bf0b
files notes.md
diffstat 1 files changed, 72 insertions(+), 21 deletions(-) [+]
line wrap: on
line diff
--- a/notes.md	Fri May 22 00:10:47 2020 -0400
+++ b/notes.md	Sat May 23 00:30:17 2020 -0400
@@ -26,37 +26,88 @@
 ===
 
 A typical DSP application reads and writes a sequence of samples making up a
-signal.  The metadata associated with the data (e.g., center frequency,
-sample rate) 
+signal.  The some of the relevant metadata is often serialized into the
+filename, while other metadata is silently discarded.  This is unfortunate.
+
+What streams should there be?
+-----------------------------
+
+We certainly want some global metadata - and because of the "everything is a
+stream" approach, this means we want a global metadata stream.
+
+It is rather clear that the sequence of I/Q samples should be considered a
+stream.  If we are handling multiple sequences of I/Q samples (e.g., we have
+a phase-coherent multi-antenna setup), each sequence should be a separate
+stream.
+
+Finally, for each I/Q stream, we want a metadata stream with information
+such as the center frequency, sample rate, and so on.
+
+It is also possible to include other streams with data that is application
+specific.  For example, meteorological observations or raw GNSS packets.
+
+Global Metadata Stream
+----------------------
+
+The global metadata stream contains information that is valid for the entire
+container.  Namely:
+
+* metadata format version number (e.g., 0)
+* generator software ID string (e.g., "SigDigger v0.1.0-23-g828657f")
+* date time of when the container file was created (not necessarily the same
+  as when the I/Q samples were recorded)
+* hostname?
+* explicit list of metadata stream unique IDs?
+
+I/Q Data Streams
+----------------
 
+These are just I/Q samples without any additional headers.
+
+Per-I/Q Stream Metadata Streams
+-------------------------------
+
+* mime type of the I/Q stream (always "application/octet-stream")
+* I/Q data stream unique id
+* I/Q data format (e.g., "cf32" or "cu8")
+* I/Q center frequency in Hz (e.g., 137500000)
+* I/Q sample rate in S/s (e.g., 2560000)
+* I/Q device (e.g., "RTL-SDR v3 00000...")
+* I/Q antenna (e.g., "RTL-SDR dipole")
+* LNA device
+* LNB device
+
+* date time of when the packet of samples was created (time of first sample)
+* lat/lon/alt
+* ground speed? azimuth?
+* antenna orientation?
+* LNA gain?
+* LNB freq?
+
+Misc Data Streams
+-----------------
+
+These are opaque byte streams with no particular format.  The interpretation
+of this data stream is left up to the application.  The associated metadata
+stream (see below) identifies the MIME type of this data.
+
+Per-Misc Data Stream Metadata Streams
+-------------------------------------
+
+* mime type of the data stream (e.g., XXX)
+* optional application specific key-value pairs
+
+* optional application specific key-value pairs
 
 
-TODO: describe how DSP workload maps to these concepts
 TODO: what's in a packet? what's in a stream? what streams are there?
 
-A typical session will contain several streams, each made up of one or more
-packets.  For example, a 5 minute long recording of complex float32 samples
-at 1MS/s will produce at least the following streams:
+A typical session will contain only three streams:
 
 0. session metadata
 1. I/Q samples
 2. I/Q metadata
 
-There is no reason to limit the format to just one I/Q stream.  Multiple I/Q
-streams can be generated with phase-coherent multi-antenna setups.  (I'm
-thinking things like KerberosSDR or other/fancier setups.)  In such a setup,
-we might see something like this:
-
-0. session metadata
-1. I/Q samples
-2. I/Q metadata (for stream 1)
-3. I/Q samples
-4. I/Q metadata (for stream 3)
-5. I/Q samples
-6. I/Q samples
-7. I/Q metadata (for stream 5)
-8. I/Q metadata (for stream 6)
-
 TODO: Note that the ordering of the streams does not matter.