uri.h File Reference

Finding files whose names encode coordinate values, via URI templates. More...

#include <stdbool.h>
#include <stdint.h>
#include <das3/defs.h>
#include <das3/time.h>
#include <das3/datum.h>

Include dependency graph for uri.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Data Structures
struct	DasUriField
	One sub-field within a named coordinate type. More...
struct	DasUriSegDef
	Definition of one coordinate type for use by the URI template parser. More...
struct	das_range
	A constraint on one coordinate (or sub-field) used to select matching files. More...

Macros
#define	DURI_MAX_PATH   2048
	Maximum length of a rendered URI / file path.

Enumerations
enum	DasUriProto
	Transport protocol detected from the leading scheme of a URI template. More...

Functions
DAS_API DasErrCode	das_range_fromUtc (das_range *pRng, const char *sBeg, const char *sEnd)
	Initialize a das_range for the "time" coordinate from ISO-8601 UTC strings.
DAS_API DasErrCode	das_range_fromTime (das_range *pRng, const das_time *tBeg, const das_time *tEnd)
	Initialize a das_range for the "time" coordinate from das_time structs.
DAS_API DasErrCode	das_range_fromInt (das_range *pRng, const char *sCoord, int64_t nBeg, int64_t nEnd)
	Initialize a das_range for a named integer coordinate or sub-field.
DAS_API DasErrCode	das_range_fromDatum (das_range *pRng, const char *sCoord, const das_datum *dmBeg, const das_datum *dmEnd)
	Initialize a das_range from pre-built das_datum values.
DAS_API const DasUriSegDef *	das_time_uridef (void)
	Return the built-in coordinate definition for the time coordinate.
DAS_API char **	das_uri_list (const char *sTemplate, const DasUriSegDef *pDef, int nRanges, const das_range *pRanges, size_t *pCount)
	Collect all paths yielded by a template and coordinate ranges into a heap array.

Detailed Description

Finding files whose names encode coordinate values, via URI templates.

Core concept: coordinates in filenames

Scientific file archives commonly encode one or more coordinate values in each filename to locate that file in a parameter space. A URI template describes how those coordinates appear in a path so that, given a coordinate range, the matching set of files can be found by directory scan.

Field tokens: short form and long form

Each sub-field of a coordinate has two interchangeable token forms:

$X single-character short token (syntactic sugar) $(coord.field) qualified long token, usable anywhere a short token is $(coord.field;modifier=value;...) qualified long token with modifiers

Short tokens are a general feature: any DasUriField with a non-zero cShort character gets one. For example, a Voyager spacecraft-clock coordinate registered with fields cShort='P' (partition), cShort='M' (mod64k), cShort='S' (mod60) would be used in a template as:

P$P/V1P$P_$x/C$M$S.DAT

The built-in time coordinate (see das_time_uridef()) provides the familiar short tokens $Y $m $d $j $H $M $S as syntactic sugar for its sub-fields; they carry no special status beyond being pre-registered.

Coordinate definitions

The engine is coordinate-agnostic. Any DasUriSegDef can be registered with DasUriTplt_register() before calling DasUriTplt_pattern(). The library ships one pre-built definition — das_time_uridef() — for the common time coordinate. Other coordinate types are user-supplied.

Examples of coordinates the design accommodates (user must supply the DasUriSegDef). These use the scalar shorthand form which is valid when the named coordinate has exactly one sub-field:

Monotonically increasing orbit counter. Common in planetary missions (Mars Express, Juno, Cassini). Orbit numbers are NOT time; no orbit-to-time mapping is assumed or required by this module.

Geodetic latitude/longitude — useful for camera archives named by instrument pointing rather than observation time.

A $() token whose coordinate or sub-field name is not recognised from any registered DasUriSegDef is a hard error. Scientific data programmers should fix their templates rather than silently losing coordinate constraints.

Directory listing is always available

This implementation does not attempt to guess or generate file paths speculatively. A directory listing (or equivalent) is always performed when needed. Templates whose file stores cannot provide directory listings are out of scope.

Relationship to the Autoplot URI template specification

Since Autoplot is a very common (and useful) program in space-physics, its URI field codes are used here whenever possible. In most common cases a DasUriTplt will look identical to its Autoplot counterpart. Some of the ways in which this implementation differs from Autoplot are noted below.

• Directory listing is always assumed available; speculative URL generation (the AP "shoot-in-the-dark" mode) is not supported.
• The coordinate concept is generalised beyond time.
• Version selection ($v) uses numeric comparison, not just lexicographic.
• Variable-duration files (where the encoded coordinate is the start point and file length is non-uniform) are handled by directory scan and backward search, not by fixed-cadence stepping.

Protocols

Templates may begin with a protocol prefix. If no prefix is present, local filesystem access is assumed.

file:// Explicit local filesystem. The prefix is stripped before any filesystem call so the returned path is usable directly with fopen(), CDFopenCDF(), etc.

http:// Remote HTTP. DasUriIter_next() returns the full URL; the https:// caller is responsible for downloading before opening.

Wildcard and version scanning ($x, $v) requires directory listing. For file:// this is done with opendir/readdir. Directory listing over HTTP is not yet supported; using $x or $v in an http:// or https:// template is an error detected at iterator initialisation time.

The built-in time coordinate (see das_time_uridef()) provides short tokens $Y $m $d $j $H $M $S and their qualified long-form equivalents.

Wildcard and version tokens

$x Unstructured wildcard. Matched portion is treated as an opaque string; the lexicographically last match in the directory is used. Not a coordinate — carries no query-relevant information. Requires file:// or implicit local protocol.

$v Version wildcard. Like $x but the matched portion is interpreted as a version identifier; the numerically greatest version is used. The comparison strategy is set by the type= modifier (see below). Not a coordinate — carries no query-relevant information. Requires file:// or implicit local protocol.

Long-form field syntax

The $() form is the explicit, self-describing alternative to a short token. Two variants are supported:

$(coord.field) qualified form — primary $(coord.field;modifier=value;...) qualified form with modifiers scalar shorthand — only valid when the named coordinate has exactly one sub-field

The qualified form uses the coordinate name (DasUriSegDef.sCoord) and the sub-field long name (DasUriField.sLong) separated by a dot. This mirrors the dotted key used in das_range.sCoord, so the same names appear in both the template and the constraint:

template: /data/$(time.year)/$(time.yday)/file_$(time.year)$(time.yday).dat range key: "time.year", "time.yday"

An unrecognised name in $() is a hard error. Unlike a silent wildcard, an error is preferable in scientific data programming where the user base is small and there is always a programmer on hand to fix the problem.

Note: Autoplot's extended modifier form $(Y;pad=none) uses the same single-character codes as the short token form, not English names. If Autoplot compatibility is needed, use the short-token form ($Y, etc.) with modifiers, not the qualified long form.

Supported general modifiers

delta=N Coverage hint. File covers approximately N units of the field's own unit (default 1). Used to scan backward when looking for variable-coverage files that may start before the query start range.

pad=none Suppress leading-zero padding (default is zero-padded).

Supported version field modifiers

type=sep (default for $v) Split the matched string on '.' and compare each component as a non-negative integer left-to-right. Handles semantic versions such as 0.5.20 and 0.7.1 correctly regardless of component width.

type=int Parse the entire matched string as a single non-negative integer. Handles zero-padded counters such as v01 and v02.

type=alpha Lexicographic comparison, identical to $x behaviour. Useful when the version token is already fixed-width and lexicographic order matches version order.

Version collision warning

When type=sep or type=int is in use, two files may resolve to the same numeric version through different string representations. For example:

data_$Y$m$d_$(v;type=int).cdf

could match both "data_20250930_v1.cdf" and "data_20250930_v01.cdf" in the same directory, both evaluating to version 1. Since this is usually unintentional, a warning is issued on standard error to help you find such cases in your datasets.

Enumeration Type Documentation

DasUriProto

enum DasUriProto

Transport protocol detected from the leading scheme of a URI template.

Stored on DasUriTplt for fast access; also carried by the first segment when an explicit scheme is present (DURI_PROTOCOL segment).

Function Documentation

das_range_fromUtc()

DAS_API DasErrCode das_range_fromUtc	(	das_range *	pRng,
		const char *	sBeg,
		const char *	sEnd
	)

Initialize a das_range for the "time" coordinate from ISO-8601 UTC strings.

Sets sCoord to "time" and parses sBeg / sEnd via das_datum_fromStr(). Any format accepted by that function works here: "2025-10-01", "2025-288", "2025-10-15T06:00", etc. The range is half-open: [sBeg, sEnd).

Parameters

pRng	Storage to initialise; all fields are overwritten.
sBeg	ISO-8601 range begin (inclusive).
sEnd	ISO-8601 range end (exclusive).

Returns

DAS_OKAY on success, a positive error code on parse failure.

das_range_fromTime()

DAS_API DasErrCode das_range_fromTime	(	das_range *	pRng,
		const das_time *	tBeg,
		const das_time *	tEnd
	)

Initialize a das_range for the "time" coordinate from das_time structs.

Sets sCoord to "time" and copies the two time values. Both pointers may address stack-allocated structs; contents are copied before return. Passing by pointer avoids a 32-byte struct copy on each call.

Parameters

pRng	Storage to initialise; all fields are overwritten.
tBeg	Inclusive range begin.
tEnd	Exclusive range end.

Returns

DAS_OKAY on success.

das_range_fromInt()

DAS_API DasErrCode das_range_fromInt	(	das_range *	pRng,
		const char *	sCoord,
		int64_t	nBeg,
		int64_t	nEnd
	)

Initialize a das_range for a named integer coordinate or sub-field.

Sets sCoord (stored lower-cased) and stores nBeg / nEnd as dimensionless datums. The range is half-open: [nBeg, nEnd). When nBeg > nEnd the iterator interprets the pair as a rollover crossing — see the das_range documentation for details.

Use this for coordinates such as "sclk.mod64k", "sclk.partition", or any user-registered coordinate whose sub-fields are plain integers.

Parameters

pRng	Storage to initialise; all fields are overwritten.
sCoord	Coordinate or sub-field name, e.g. "sclk.mod64k". Stored lower-cased; must fit in 31 chars.
nBeg	Inclusive range begin.
nEnd	Exclusive range end (or rollover end when nBeg > nEnd).

Returns

DAS_OKAY on success, a positive error code if sCoord is too long.

das_range_fromDatum()

DAS_API DasErrCode das_range_fromDatum	(	das_range *	pRng,
		const char *	sCoord,
		const das_datum *	dmBeg,
		const das_datum *	dmEnd
	)

Initialize a das_range from pre-built das_datum values.

Sets sCoord (stored lower-cased) and copies dmBeg / dmEnd. Both pointers may address stack-allocated datums; struct copy is safe per datum.h. The datum value types must be consistent with sCoord: vtTime for "time", das_vt_isint() for all other sub-field coordinates.

Parameters

pRng	Storage to initialise; all fields are overwritten.
sCoord	Coordinate or sub-field name. Stored lower-cased.
dmBeg	Inclusive begin datum; caller retains ownership.
dmEnd	Exclusive end datum; caller retains ownership.

Returns

DAS_OKAY on success, a positive error code if sCoord is too long.

das_time_uridef()

DAS_API const DasUriSegDef * das_time_uridef

(

void

)

Return the built-in coordinate definition for the time coordinate.

Returns a pointer to a file-scope static DasUriSegDef pre-loaded with the following sub-fields:

short long name width range qualified token

---

$Y year 4 1678–2262 $(time.year) $m month 2 01–12 $(time.month) $d mday 2 01–31 $(time.mday) $j yday 3 001–366 $(time.yday) $H hour 2 00–23 $(time.hour) $M minute 2 00–59 $(time.minute) $S second 2 00–60 $(time.second)

No allocation takes place; the returned pointer is always valid. Pass it to DasUriTplt_register() to enable time field parsing.

das_uri_list()

DAS_API char ** das_uri_list	(	const char *	sTemplate,
		const DasUriSegDef *	pDef,
		int	nRanges,
		const das_range *	pRanges,
		size_t *	pCount
	)

Collect all paths yielded by a template and coordinate ranges into a heap array.

Convenience wrapper around new_DasUriTplt() + new_DasUriIter() for callers that need the complete list up front (unit tests, small utilities). For production streaming use prefer the iterator directly so that files can be processed as they are found without buffering the full path list.

The returned array is NULL-terminated; the count is also written to *pCount if pCount is not NULL. The entire structure — pointer array and all path strings — is a single contiguous heap allocation. Release it with a single free() call:

char** ppPaths = das_uri_list(...);
// ... use ppPaths ...
free(ppPaths);

Returns NULL (and sets *pCount to 0) if the template cannot be parsed, all ranges are empty, or no files are found.

Example — list daily CDF files over two days:

das_range r;
das_range_fromUtc(&r, "2025-288", "2025-290");
 
size_t nCount = 0;
char** ppPaths = das_uri_list(
    "/data/$Y/$j/instrument_$Y$j_$v.cdf",
    das_time_uridef(),
    1, &r, &nCount
);
for(size_t i = 0; i < nCount; ++i)
    printf("%s\n", ppPaths[i]);
free(ppPaths);

For templates that use a user-defined coordinate (e.g. spacecraft clock), pass the matching DasUriSegDef instead of das_time_uridef(). To use more than one coordinate definition, use the full iterator API.

Parameters

sTemplate	URI template string, same syntax as DasUriTplt_pattern().
pDef	Coordinate definition to register before parsing; typically das_time_uridef() for time-based templates. May be NULL for literal or $x/$v-only templates.
nRanges	Number of entries in pRanges.
pRanges	Array of coordinate range constraints.
pCount	If not NULL, receives the number of entries in the array.

Returns

NULL-terminated, NULL-safe-to-free array of path strings, or NULL on error or empty result.