#include <das2/io.h>

Detailed Description

Tracks input and output operations for das2 stream headers and data.

Members of this class handle overall stream operations reading writing packets, checking packet lengths, passing XML string data off to descriptor object constructors, triggering processing callbacks and most other general Das2 Stream IO tasks.

Public Member Functions
DasIO *	new_DasIO_cfile (const char sProg, FILE file, const char *mode)
	Create a new DasIO object from a standard C FILE. More...

DasIO *	new_DasIO_cmd (const char sProg, const char sCmd)
	Create a new DasIO object from a shell command. More...

DasIO *	new_DasIO_file (const char sProg, const char sFile, const char *mode)
	Create a new DasIO object from a disk file. More...

DasIO *	new_DasIO_socket (const char sProg, int nSockFd, const char mode)
	Create a new DasIO object from a socket. More...

DasIO *	new_DasIO_ssl (const char sProg, void pSsl, const char *mode)
	Create a new DasIO object using an encripted connection. More...

int	DasIO_addProcessor (DasIO pThis, StreamHandler pProc)
	Add a packet processor to be invoked during I/O operations. More...

int	DasIO_readAll (DasIO *pThis)
	Starts the processing of the stream read from FILE* infile. More...

DasErrCode	DasIO_writeStreamDesc (DasIO pThis, StreamDesc pSd)
	Writes the data describing the stream to the output channel (e.g. More...

DasErrCode	DasIO_writePktDesc (DasIO pThis, PktDesc pd)
	Writes the data describing a packet type to the output channel (e.g. More...

DasErrCode	DasIO_writePktData (DasIO pThis, PktDesc pPd)
	Sends the data packet on to the stream after checking validity. More...

DasErrCode	DasIO_writeException (DasIO pThis, OobExcept pSe)
	Output an exception structure.

DasErrCode	DasIO_writeComment (DasIO pThis, OobComment pSc)
	Output a StreamComment Stream comments are generally messages interpreted only by humans and may change without affecting processes. More...

DasErrCode	DasIO_sendLog (DasIO pThis, int level, char msg,...)
	Send a log message onto the stream at the given log level. More...

void	DasIO_setLogLvl (DasIO *pThis, int minLevel)
	Set the minimum log level that will be transmitted on the stream. More...

int	DasIO_getLogLvl (const DasIO *pThis)
	Get logging verbosity level. More...

DasErrCode	DasIO_setTaskProgress (DasIO *pThis, int progress)
	Place rate-limited progress comments on an output stream. More...

void	DasIO_throwException (DasIO pThis, StreamDesc pSd, const char type, char msg)
	Set the logging level. More...

void	DasIO_close (DasIO *pThis)
	Normal stream close with no unusual condiditons Closes the output file descriptor, flushes a gzip buffer, etc. More...

int	DasIO_printf (DasIO pThis, const char format,...)
	Print a string with a format specifier (Low-level API) This works similar to the C printf function. More...

size_t	DasIO_write (DasIO pThis, const char data, int length)
	Anolog of fwrite (Low-level API)

int	DasIO_read (DasIO pThis, DasBuf pBuf, size_t nBytes)
	Analog of fread (Low-level API)

int	DasIO_getc (DasIO *pThis)
	Analog of getc (Low-level API)

Member Function Documentation

DasIO * new_DasIO_cfile	(	const char *	sProg,
		FILE *	file,
		const char *	mode
	)

Create a new DasIO object from a standard C FILE.

Create a DasIO object that reads or writes to C standard IO files. The C file object's read or write state should be compatible with the DasIO state. For example:

* DasIO* dio = new_DasIO_cfile(stdin, "myprogram", "rc");

*

is invalid, if standard input is not compressed, but the library has no way to tell until read operations start.

Parameters

sProg	A spot to store the name of the program creating the file this is useful for automatically generated error and log messages
file	a C standard IO file object.
mode	A string containing the mode, one of: 'r' read 'w' write uncompressed 'wc' write compressed

DasIO * new_DasIO_cmd	(	const char *	sProg,
		const char *	sCmd
	)

Create a new DasIO object from a shell command.

Create a DasIO object that reads from a sub command. The sub command is run in the shell '/bin/sh' on POSIX systems

* DasIO* dio = new_DasIO_cmd("/my/das2/reader 2017-01-01 2017-01-02", "wc");

*

is invalid, but the library has no way to tell until operations start.

Parameters

sProg	A spot to store the name of the program running the command this is useful for automatically generated error and log messages
sCmd	the command line to run, will be started via popen

DasIO * new_DasIO_file	(	const char *	sProg,
		const char *	sFile,
		const char *	mode
	)

Create a new DasIO object from a disk file.

Parameters

sProg	A spot to store the name of the program creating the file this is useful for automatically generated error and log messages
sFile	the name of a file on disk. If the file doesn't exist it is created.
mode	A string containing the mode, one of: 'r' read (reads compressed and uncompressed files) 'w' write uncompressed 'wc' write compressed

DasIO * new_DasIO_socket	(	const char *	sProg,
		int	nSockFd,
		const char *	mode
	)

Create a new DasIO object from a socket.

Parameters

sProg	A spot to store the name of the program creating the file this is useful for automatically generated error and log messages
nSockFd	The socket file descriptor used for recv/write calls
mode	A string containing the mode, one of: 'r' read (reads compressed and uncompressed files) 'w' write uncompressed 'wc' write compressed

Returns: A new DasIO object allocated on the heap

DasIO * new_DasIO_ssl	(	const char *	sProg,
		void *	pSsl,
		const char *	mode
	)

Create a new DasIO object using an encripted connection.

This class does not handle communications setup but can take over after a connection has been established and all needed headers have been sent and or read.

It is also assumed that the SSL connection has been established on a socket using BLOCKING I/O and is not sutiable for use as an action for a select() or poll() statement. To handle multiple connections at once in a single program create more than one DasIO object.

You should also setup the SSL connection with the SSL_MODE_AUTO_RETRY option to SSL_set_mode or SSL_CTX_set_mode to let the openssl library handle session renegotiation transparently. The http_getBodySocket() call does initialize any SSL structures it generates in auto-retry mode.

Parameters

sProg	A spot to store the name of the program creating the file this is useful for automatically generated error and log messages
pSsl	A pointer to OpenSSL SSL structure referencing an open connection that has been initialized in BLOCKING mode.
mode	A string containing the mode, one of: 'r' read (reads compressed and uncompressed files) 'w' write uncompressed 'wc' write compressed

Returns

int DasIO_addProcessor	(	DasIO *	pThis,
		StreamHandler *	pProc
	)

Add a packet processor to be invoked during I/O operations.

A DasIO object may have 0 - DAS2_MAX_PROCESSORS packet processors attached to it. Each processor contains one or more callback functions that will be invoked and provided with DasDesc objects.

During stream read operations, processors are invoked after the header or data have been serialized from disk. During stream write operations processors are invoked before the write operation is completed.

Parameters

pThis	The DasIO object to receive the processor.
pProc	The StreamProc object to add to the list of processors.

Returns: The number of packet processors attached to this DasIO object or a negative error value if there is a problem.

int DasIO_readAll ( DasIO * pThis )

Starts the processing of the stream read from FILE* infile.

This function does not return until all input has been read or an exception condition has occurred. If a StreamHandler has been set with DasIO_setProcessor() then as each header packet and each data packet is read callbacks specified in the stream handler will be invoked. Otherwise all data is merely parsed and then discarded.

Returns: a status code. 0 indicates that all processing ended with no errors, otherwise a non-zero value is returned.

DasErrCode DasIO_writeStreamDesc	(	DasIO *	pThis,
		StreamDesc *	pSd
	)

Writes the data describing the stream to the output channel (e.g.

File* ).

This serializes the descriptor structure into XML and writes it out.

Parameters

pThis	The IO object, must be set up in a write mode
pSd	The stream descriptor to serialize

Returns: 0 on success a positive integer error code other wise.

DasErrCode DasIO_writePktDesc	(	DasIO *	pThis,
		PktDesc *	pd
	)

Writes the data describing a packet type to the output channel (e.g.

File* ).

This serializes the descriptor structure into XML and writes it out. The packet type will be assigned a number on the das2Stream, and data packets will be tagged with this number.

DasErrCode DasIO_writePktData	(	DasIO *	pThis,
		PktDesc *	pPd
	)

Sends the data packet on to the stream after checking validity.

This check insures that all planes have been set via setDataPacket.

Parameters

pThis	the IO object to use for sending the data
pPd	a Packet Descriptor loaded with values for output.

Returns: 0 if the operation was successful or an positive value on an error.

DasErrCode DasIO_writeComment	(	DasIO *	pThis,
		OobComment *	pSc
	)

Output a StreamComment Stream comments are generally messages interpreted only by humans and may change without affecting processes.

Progress messages are sent via stream comments.

DasErrCode DasIO_sendLog	(	DasIO *	pThis,
		int	level,
		char *	msg,
			...
	)

Send a log message onto the stream at the given log level.

Note that messages sent at a level greater than INFO may be thrown out if performance would be degraded.

void DasIO_setLogLvl	(	DasIO *	pThis,
		int	minLevel
	)

Set the minimum log level that will be transmitted on the stream.

The point of the DasIO logging functions is not to handle debugging, but to inform a client program what is going on by embedding messages in an output stream. Since the point of a stream is to transmit data, placing too many comments in the stream will degrade performance.

Unless this function is called, the default logging level is LOGLEVEL_WARNING

Parameters

pThis	the DasIO object who's log level will be altered.
minLevel	A logging level, one of: LOGLVL_ERROR - Only output messages that indicate improper operation. LOGLVL_WARNING - Messages that indicate a problem but processing can continue LOGLVL_INFO - Extra stuff the operator may want to know LOGLVL_CONFIG - Include basic setup information as well

int DasIO_getLogLvl ( const DasIO * pThis )

Get logging verbosity level.

Parameters

pThis a DasIO object to query

Returns: A logging level as defined in DasIO_setLogLevel()

DasErrCode DasIO_setTaskProgress	(	DasIO *	pThis,
		int	progress
	)

Place rate-limited progress comments on an output stream.

Messages are decimated dynamically to try to limit the stream comment rate to about 10 per second (see targetUpdateRateMilli). If Note the tacit assumption that is the stream is reread, it will be reread at a faster rate than it was written. Processes reducing a stream should take care to call setTaskProgress themselves instead of simply forwarding the progress comments, so that the new stream is not burdened by excessive comments. See setTaskSize().

void DasIO_throwException	(	DasIO *	pThis,
		StreamDesc *	pSd,
		const char *	type,
		char *	msg
	)

Set the logging level.

Close and free an output stream with an exception.

Call this to notify stream consumers that there was an exceptional condition that prevents completion of the stream. For example, this might be used to indicate that no input files were available. This is a convenience function for the calls:

Parameters

pThis	the output object to receive the exception packet
pSd	The stream descriptor. All Das2 Streams have to start with a stream header. If this header hasn't been output then this object will be serialized to create a stream header.
type	the type of exception may be one of the pre-defined strings: DAS2_EXCEPT_NO_DATA_IN_INTERVAL DAS2_SERVER_ERROR or some other short type string of your choice.
msg	a longer message explaining what went wrong.

void DasIO_close ( DasIO * pThis )

Normal stream close with no unusual condiditons Closes the output file descriptor, flushes a gzip buffer, etc.

May possibly send a stream termination comment as well.

Parameters

pThis

int DasIO_printf	(	DasIO *	pThis,
		const char *	format,
			...
	)

Print a string with a format specifier (Low-level API) This works similar to the C printf function.

Parameters

pThis
format
...	The variables to print

Returns: The number of characters printed.

The documentation for this struct was generated from the following file:

das2/io.h

Detailed Description

Public Member Functions

Member Function Documentation