Catalogs

Provides utilities for reading remote data source catalogs and caching their contents in RAM. More...

Detailed Description

Provides utilities for reading remote data source catalogs and caching their contents in RAM.

There are two basic methods of catalog object acquisition, finding a node by it's path from the global root of the federated das2 catalog system: (ex: tag:das2.org,2012:site:/swri/mars_express/aspera3/sciels) or by an loading an explicit URL (ex: http://das2.org/catalog/das/site/uiowa/cassini/rpws/survey.json).

When describing the das2 catalog C-API there are two meanings for the word Root.

1) Memory Roots - A node can be the in-memory root of a linked list of nodes.

2) Global Root - Root can also refer to the global das2 top-level un-named node which resides at http://das2.org/catalog/index.json and it's mirrors.

Any node loaded using new_RootNode() or new_RootNode_url() is an in memory root node. It might not be the root of a federated catalog system, but it is still a top level memory item that manages a memory cache for it's subnodes. If new_RootNode() happens to be called with a URI of 'NULL' then the in-memory root corresponds to the federated catalog root. The reason for splitting the concept of the in-memory root from the global catalog root is to provide for isolated catalogs that are not part of the global system.

When you instantiate a node from a random URL you have to give it a name. Just like files on a disk, catalog nodes do not know their name, it is derived from the name provided by the higher level catalog entries (up to root, which has no name).

An Example of getting a data source interface definition follows:

// Get the site root and safely cast the output.  Any particular point in
// the catalog can be a root node for the purposes of sub-item queries and
// memory management.  By calling new_RootNode() the returned item is a local
// root object for memory management purposes.  The higher level nodes that
// were traversed to get to the requested item are deleted from memory.

DasNode* pRoot = new_RootNode("tag:das2.org,2012:site:/uiowa", NULL, NULL);
if(!DasNode_isCatalog(pNode)) handle_error();

// Use the site catalog to get the Cassini Saturn SLS2 data source description

const char* sRelPath = "uiowa/cassini/ephemeris/saturn_sls2";
DasNode* pNode = DasNode_subNode(pRoot, sRelPath, NULL, NULL);

if(!(DasNode_isStreamSource(pNode)) handle_error();

// Get the HTTP get Query interface definition

const DasJdo* pDef = DasNode_getJdo(pNode, "SOURCE/QUERY_PARAMS");
if(pDef == NULL) handle_error();
 
// Freeing the root node will delete all lower nodes

del_DasNode(pRoot);

You now have a JSON object providing the query definition see http://das2.org/datasource.html for more information on the content of das2 data source definitions.

For isolated sites, or for testing purposes catalog nodes can be loaded directly. The following example acquires a local test root and then a specific test node.

// Assume this site uses a password since it is a testing area.  Create a 
// credentials manager and pass a basic auth hash so we aren't bothered during
// testing.

DasCredMngr* pMngr = new_CredMngr(NULL);
das_credential cred;
das_cred_init(&cred, "my.domain.com", "Regression Test Catalog", NULL, 
             "VVNFUk5BTUU6UEFTU1dPUkQ=");
CredMngr_addCred(pMngr, &cred);

const char* sTestUrl = "http://my.domain.com/das2/regtest.json";
DasNode* pRoot = new_RootNode_url(sTestUrl, pMngr, NULL);
if(!DasNode_isCatalog(pNode)) handle_error();

// At this point sub-items can be loaded from the local root as above.

DasNode* pNode = DasNode_subNode("MarsExpress/aspera/els", pMngr, NULL);

Data Structures
struct	das_json_str
	A JSON string value. More...
struct	das_json_str_ex
	A JSON string value (extended) More...
struct	das_json_num
	A JSON number value. More...
struct	das_json_dict_el
	An element of a JSON dictionary. More...
struct	das_json_dict
	a JSON dictionary payload More...
struct	das_json_ary_el
	an element of a JSON array More...
struct	das_json_ary
	a JSON array value More...
struct	DasJdo
	JSON Dom Element. More...
struct	das_json_val_ex
	a JSON value (extended) More...
struct	das_json_parse_result_s
	error report from json_parse_ex() More...
struct	DasNode
	Base type for das2 catalog nodes. More...

Enumerations
enum	das_json_type_e
	The various types JSON values can be. More...
enum	das_json_parse_flags_e { ,   das_jparse_flags_allow_trailing_comma = 0x1, das_jparse_flags_allow_unquoted_keys = 0x2, das_jparse_flags_allow_global_object = 0x4, das_jparse_flags_allow_equals_in_object = 0x8,   das_jparse_flags_allow_no_commas = 0x10, das_jparse_flags_allow_c_style_comments = 0x20 , das_jparse_flags_allow_location_information = 0x80, das_jparse_flags_allow_single_quoted_strings = 0x100,   das_jparse_flags_allow_hexadecimal_numbers = 0x200, das_jparse_flags_allow_leading_plus_sign = 0x400, das_jparse_flags_allow_leading_or_trailing_decimal_point = 0x800, das_jparse_flags_allow_inf_and_nan = 0x1000,   das_jparse_flags_allow_multi_line_strings = 0x2000, das_jparse_flags_allow_simplified_json, das_jparse_flags_allow_json5 }
	Flag useed by dasj_parse() and dasj_parse_ex() to alter parsing behavior. More...
enum	das_jparse_error_e {   das_jparse_error_none = 0, das_jparse_error_expected_comma_or_closing_bracket, das_jparse_error_expected_colon, das_jparse_error_expected_opening_quote,   das_jparse_error_invalid_string_escape_sequence, das_jparse_error_invalid_number_format, das_jparse_error_invalid_value, das_jparse_error_premature_end_of_buffer,   das_jparse_error_invalid_string, das_jparse_error_allocator_failed, das_jparse_error_unexpected_trailing_characters, das_jparse_error_unknown }
	JSON parsing error codes. More...

Functions
DasDesc *	dsdf_parse (const char *sFileName)
	Parse a DSDF file into a descriptor object. More...
double *	dsdf_valToArray (const char *sValue, size_t *pLen)
	Helper function to parse a DSDF value as a double array. More...
const char *	dsdf_setIdlBin (const char *sIdlBin)
	Set the location of the IDL binary. More...
char *	dsdf_valToNormParam (const char *sRawParam, char *sNormParam, size_t uNormLen)
	Normalize a general reader command line parameter set. More...
const char *	json_parse_error_info (const struct das_json_parse_result_s *pRes, char *sTmp, size_t uLen)
	Provide error string describing a parsing error result.
DasJdo *	das_json_parse_ex (const void *src, size_t src_size, size_t flags_bitset, void *(*alloc_func_ptr)(void *, size_t), void *user_data, struct das_json_parse_result_s *result)
	Parse a JSON text file, returning a pointer to the root of the JSON structure. More...
DasJdo *	das_json_parse (const void *src, size_t src_size)
	Parse a JSON text file with default options and without detailed error reporting. More...
const das_json_dict_el *	DasJdo_dictFirst (const DasJdo *pThis)
	Get the first dictionary element from a JSON dictionary.
const das_json_ary_el *	DasJdo_aryFirst (const DasJdo *pThis)
	Get the first array element from a JSON array.
const char *	DasJdo_string (const DasJdo *pThis)
	Get a string value from a JSON DOM element. More...
void *	DasJdo_writePretty (const DasJdo *pThis, const char *indent, const char *newline, size_t *out_size)
	Write out a pretty JSON utf-8 string. More...
const DasJdo *	DasJdo_get (const DasJdo *pThis, const char *sRelPath)
	Given a DOM path retrieve a JSON element. More...
void *	DasJdo_writeMinified (const DasJdo *pThis, size_t *out_size)
	Write out a minified JSON utf-8 string. More...

Enumeration Type Documentation

enum das_json_type_e

The various types JSON values can be.

Used to identify what a value is

enum das_json_parse_flags_e

Flag useed by dasj_parse() and dasj_parse_ex() to alter parsing behavior.

Enumerator
das_jparse_flags_allow_trailing_comma	allow trailing commas in objects and arrays. For example, both [true,] and {"a" : null,} would be allowed with this option on.
das_jparse_flags_allow_unquoted_keys	allow unquoted keys for objects. For example, {a : null} would be allowed with this option on.
das_jparse_flags_allow_global_object	allow a global unbracketed object. For example, a : null, b : true, c : {} would be allowed with this option on.
das_jparse_flags_allow_equals_in_object	allow objects to use '=' instead of ':' between key/value pairs. For example, a = null, b : true would be allowed with this option on.
das_jparse_flags_allow_no_commas	allow that objects don't have to have comma separators between key/value pairs.
das_jparse_flags_allow_c_style_comments	allow c-style comments (// or /* *\/) to be ignored in the input JSON file.
das_jparse_flags_allow_location_information	record location information for each value.
das_jparse_flags_allow_single_quoted_strings	allow strings to be 'single quoted'
das_jparse_flags_allow_hexadecimal_numbers	allow numbers to be hexadecimal
das_jparse_flags_allow_leading_plus_sign	allow numbers like +123 to be parsed
das_jparse_flags_allow_leading_or_trailing_decimal_point	allow numbers like .0123 or 123. to be parsed
das_jparse_flags_allow_inf_and_nan	allow Infinity, -Infinity, NaN, -NaN
das_jparse_flags_allow_multi_line_strings	allow multi line string values
das_jparse_flags_allow_simplified_json	allow simplified JSON to be parsed. Simplified JSON is an enabling of a set of other parsing options.
das_jparse_flags_allow_json5	allow JSON5 to be parsed. JSON5 is an enabling of a set of other parsing options.

enum das_jparse_error_e

JSON parsing error codes.

Enumerator
das_jparse_error_none	no error occurred (huzzah!)
das_jparse_error_expected_comma_or_closing_bracket	expected either a comma or a closing '}' or ']' to close an object or array!
das_jparse_error_expected_colon	colon separating name/value pair was missing!
das_jparse_error_expected_opening_quote	expected string to begin with '"'!
das_jparse_error_invalid_string_escape_sequence	invalid escaped sequence in string!
das_jparse_error_invalid_number_format	invalid number format!
das_jparse_error_invalid_value	invalid value!
das_jparse_error_premature_end_of_buffer	reached end of buffer before object/array was complete!
das_jparse_error_invalid_string	string was malformed!
das_jparse_error_allocator_failed	a call to malloc, or a user provider allocator, failed
das_jparse_error_unexpected_trailing_characters	the JSON input had unexpected trailing characters that weren't part of the JSON value
das_jparse_error_unknown	catch-all error for everything else that exploded (real bad chi!)

Function Documentation

DasDesc* dsdf_parse

(

const char *

sFileName

)

Parse a DSDF file into a descriptor object.

This DSDF parser supports IDL continuation characters, which are '$' immediately followed by a newline (
). The total line buffer across all continuation lines is 22499 bytes long.

Parameters

sFileName

- The DSDF file to open and parse

Returns

an Descriptor object allocated on the heap, or NULL if there was a parsing error

double* dsdf_valToArray	(	const char *	sValue,
		size_t *	pLen
	)

Helper function to parse a DSDF value as a double array.

Certian Das1 DSDF values such as the y_coordinate contained executable IDL code instead of a simple array of values. If a global IDL executable has been set via dsdf_setIdlBin, then any arrays this function cannot parse will be handed by an IDL subprocess.

Parameters

[in]	sValue	- The DSDF value to parse
[out]	pLen	- a pointer to a size_t that will receive the length of the returned double array

Returns

- A pointer to an array of doubles allocated on the heap, or NULL if parsing failed.

const char* dsdf_setIdlBin

(

const char *

sIdlBin

)

Set the location of the IDL binary.

By default the library does not know how to find IDL, use this function to set the location of the idl startup program. Note that IDL is not needed when parsing Das 2.2 (or higher) compliant DSDF files. Only programs that read older Das1 DSDF files may have the need to call IDL.

char* dsdf_valToNormParam	(	const char *	sRawParam,
		char *	sNormParam,
		size_t	uNormLen
	)

Normalize a general reader command line parameter set.

The normalization rules are as follows:

0. If the raw parameter string is NULL or empty the string '_noparam' is copied into the norm-param buffer and the function returns.

1. The params are broken on whitespace into a set of ordered tokens.
2. If the token starts with a '-' and the next token doe snot start with a '-' then the two tokens are merged with a '-' separator.
3. All tokens are sorted alphabetically and then merged via '_' separators.

Parameters

[in]	sRawParam	- The parameter string to normalize
[out]	sNormParam	- A buffer to hold the normalized parameter string
[in]	uNormLen	- The length of the sNormParam buffer

Returns

A pointer to sNormParam which will contain a null terminated string if not NULL and uLen is at least 2

DasJdo* das_json_parse_ex	(	const void *	src,
		size_t	src_size,
		size_t	flags_bitset,
		void *(*)(void *, size_t)	alloc_func_ptr,
		void *	user_data,
		struct das_json_parse_result_s *	result
	)

Parse a JSON text file, returning a pointer to the root of the JSON structure.

Parameters

src	a pointer to a utf-8 encoded JSON document, type is void* to emphasize that the data may contain byte values > 127.
src_size	The size in bytes of the document string to parse
flags_bitset	values from enum das_jparse_flags_e OR'ed together to configure parsing preferences.
alloc_func_ptr	May be use to allocate memory via some method other than malloc.
user_data	A user data pointer pto be passed as the first argument to alloc_func_ptr, if alloc_func_ptr is not NULL.
result	(if not NULL) will explain the type of error, and the location in the input it occurred.

Returns

A pointer to the entire malloc'ed memory. Use free() on the return value when it's no longer needed. Returns NULL if an error occured (malformed JSON input, or malloc failed)

DasJdo* das_json_parse	(	const void *	src,
		size_t	src_size
	)

Parse a JSON text file with default options and without detailed error reporting.

This is just shorthand for das_json_parse_ex(src, size, 0, NULL, NULL, NULL);

const char* DasJdo_string

(

const DasJdo *

pThis

)

Get a string value from a JSON DOM element.

Parameters

pThis

The element in question

Returns

NULL if the object type is not a string or if pThis is NULL, a null terminated utf-8 string otherwise

void* DasJdo_writePretty	(	const DasJdo *	pThis,
		const char *	indent,
		const char *	newline,
		size_t *	out_size
	)

Write out a pretty JSON utf-8 string.

This string is encoded such that the resultant JSON is pretty in that it is easily human readable. The indent and newline parameters allow a user to specify what kind of indentation and newline they want (two spaces / three spaces / tabs? ,
,
?). Both indent and newline can be NULL, indent defaults to two spaces (" "), and newline defaults to linux newlines ('
' as the newline character).

DasJdo_writePretty performs 1 call to malloc for the entire encoding.

Returns

NULL if an error occurred (malformed JSON input, or malloc failed). The out_size parameter is optional as the utf-8 string is null terminated.

const DasJdo * DasJdo_get	(	const DasJdo *	pThis,
		const char *	sRelPath
	)

Given a DOM path retrieve a JSON element.

Parameters

pThis

sRelPath

Pointer to a utf-8 string providing a relative path to the sub object. This has the form: [STRING | INTEGER] [ / [STRING | INTEGER] [ / [STRING | INTEGER]]] For example: "cassini/ephemeris/CONTACTS/0/EMAIL". For Arrays sub items are denoted by ascii integers 0 - size, for dictionaries sub items can be denoted by either keys strings or by ascii integers. Objects of type string, number, true, false and null have no sub items

Returns

NULL if no object exists at the specified relative path or if the object is atomic and has no sub items.

void * DasJdo_writeMinified	(	const DasJdo *	pThis,
		size_t *	out_size
	)

Write out a minified JSON utf-8 string.

This string is an encoding of the minimal string characters required to still encode the same data. DasJdo_writeMinified performs 1 call to malloc for the entire encoding.

Parameters

pThis

The top level document object to write, all sub-objects will be written as well.

Returns

A new buffer containing utf-8 string data, or NULL if an error occurred (malformed JSON input, or malloc failed),

The out_size parameter is optional as the utf-8 string is null terminated.