Supported Datasets

Currently, support is included for files from the following datasets:

These routines may be used as a guide to write routines for other datasets. A general data loading routine is also provided for ASCII files. These routines are part of the ocbpy.instruments module.

General

General loading routines for data files

Functions

test_file(filename)
Test to see whether file exists and is small enough to load
load_ascii_data(filename, hlines, kwargs)
Load time-sorted ascii data file
ocbpy.instruments.general.load_ascii_data(filename, hlines, miss=None, fill=nan, hsplit=None, inline_comment=None, invalid_raise=False, datetime_cols=[], datetime_fmt=None, int_cols=[], str_cols=[], max_str_length=50, header=[])[source]

Load an ascii data file into a dict of numpy array.

Parameters:
filename : (str)

data file name

hlines : (int)

number of lines in header. If zero, must include header.

miss : (str, sequence, or dict)

Denotes missing value options (default=None)

fill : (value, sequence, or dict)

fill value (default=NaN)

hsplit : (str, NoneType)

character seperating data labels in header. None splits on all whitespace characters. (default=None)

inline_comment : (str or NoneType)

If there are comments inline, denote the charater that indicates it has begun. If there are no comments inline, leave as the default. (default=None)

invalid_raise : (bool)

Should the routine fail if a row of data with a different number of columns is encountered? If false, these lines will be skipped and all other lines will be read in. (default=False)

datetime_cols : (list of ints)

If there are date strings or values that should be converted to a datetime object, list them in order here. Not processed as floats. (default=[])

datetime_fmt : (str or NoneType)

Format needed to convert the datetime_cols entries into a datetime object. Special formats permitted are: ‘YEAR SOY’, ‘YYDDD’, ‘SOD’. ‘YEAR SOY’ must be used together; ‘YYDDD’ indicates years since 1900 and day of year, and may be used with any time format; ‘SOD’ indicates seconds of day, and may be used with any date format (default=None)

int_cols : (list of ints)

Data that should be processed as integers, not floats. (default=[])

str_cols : (list of ints)

Data that should be processed as strings, not floats. (default=[])

max_str_length : (int)

Maximum allowed string length. (default=50)

header : (list of str)

Header string(s) where the last line contains whitespace separated data names (default=list())

Returns:
header : (list of strings)

Contains all specified header lines

out : (dict of numpy.arrays)

The dict keys are specified by the header data line, the data for each key are stored in the numpy array

Notes

Data is assumed to be float unless otherwise stated.

ocbpy.instruments.general.test_file(filename)[source]

Test to ensure the file is small enough to read in. Python can only allocate 2GB of data without crashing

Parameters:
filename : (str)

Filename to test

Returns:
good_flag : (bool)

True if good, bad if false

SuperMAG

Perform OCB gridding for SuperMAG data

Functions

supermag2ascii_ocb(smagfile, outfile, kwargs)
Write and ASCII file with SuperMAG data and the OCB coordinates for each data point
load_supermag_ascii_data(filename)
Load SuperMAG ASCII data files

Data

SuperMAG data available at: http://supermag.jhuapl.edu/

ocbpy.instruments.supermag.load_supermag_ascii_data(filename)[source]

Open a SuperMAG ASCII data file and load it into a dictionary of nparrays

Parameters:
filename : (str)

SuperMAG ASCI data file name

Returns:
out : (dict of numpy.arrays)

The dict keys are specified by the header data line, the data for each key are stored in the numpy array

ocbpy.instruments.supermag.supermag2ascii_ocb(smagfile, outfile, ocb=None, ocbfile=None, max_sdiff=600, min_sectors=7, rcent_dev=8.0, max_r=23.0, min_r=10.0, min_j=0.15)[source]

Coverts the location of SuperMAG data into a frame that is relative to the open-closed field-line boundary (OCB) as determined from a circle fit to the poleward boundary of the auroral oval

Parameters:
smagfile : (str)

file containing the required SuperMAG file sorted by time

outfile : (str)

filename for the output data

ocb : (OCBoundary or NoneType)

OCBoundary object with data loaded from an OC boundary data file. If None, looks to ocbfile

ocbfile : (str or NoneType)

file containing the required OC boundary data sorted by time, or None to use IMAGE WIC or to pass in an OCBoundary object (default=None)

max_sdiff : (int)

maximum seconds between OCB and data record in sec (default=600)

min_sectors : (int)

Minimum number of MLT sectors required for good OCB (default=7).

rcent_dev : (float)

Maximum number of degrees between the new centre and the AACGM pole (default=8.0).

max_r : (float)

Maximum radius for open-closed field line boundary in degrees default=23.0).

min_r : (float)

Minimum radius for open-closed field line boundary in degrees (default=10.0).

min_j : (float)

Minimum unitless current magnitude scale difference (default=0.15)

Returns:
Void

SuperDARN Vorticity

Perform OCB gridding for SuperDARN vorticity data

Functions

vort2ascii_ocb(vortfile, outfile, kwargs)
Write and ASCII file with SuperDARN data and the OCB coordinates for each data point
load_vorticity_ascii_data(filename, save_all=False)
Load vorticity block ASCII data files

Data

Specialised SuperDARN data product, available from: gchi@bas.ac.uk

ocbpy.instruments.vort.load_vorticity_ascii_data(vortfile, save_all=False)[source]

Load SuperDARN vorticity data files.

Parameters:
vortfile : (str)

SuperDARN vorticity file in block format

save_all : (bool)

Save all data from the file (True), or only data needed to calculate the OCB coordinates and normalised vorticity (False). (default=False)

Returns:
vdata : (dict)

Dictionary of numpy arrays

ocbpy.instruments.vort.vort2ascii_ocb(vortfile, outfile, ocb=None, ocbfile=None, max_sdiff=600, save_all=False, min_sectors=7, rcent_dev=8.0, max_r=23.0, min_r=10.0, min_j=0.15)[source]

Coverts the location of vorticity data in AACGM coordinates into a frame that is relative to the open-closed field-line boundary (OCB) as determined from a circle fit to the poleward boundary of the auroral oval

Parameters:
vortfile : (str)

file containing the required vorticity file sorted by time

outfile : (str)

filename for the output data

ocb : (ocbpy.ocboundary.OCBoundary or NoneType)

Object containing open closed boundary data or None to load from file

ocbfile : (str or NoneType)

file containing the required OC boundary data sorted by time, or None to use ocb object or IMAGE WIC file (default=None)

max_sdiff : (int)

maximum seconds between OCB and data record in sec (default=600)

save_all : (bool)

Save all data (True), or only that needed to calcuate OCB and vorticity (False). (default=False)

min_sectors : (int)

Minimum number of MLT sectors required for good OCB. (default=7)

rcent_dev : (float)

Maximum number of degrees between the new centre and the AACGM pole (default=8.0).

max_r : (float)

Maximum radius for open-closed field line boundary in degrees. (default=23.0)

min_r : (float)

Minimum radius for open-closed field line boundary in degrees (default=10.0)

min_j : (float)

Minimum unitless current magnitude scale difference (default=0.15)

Returns:
Void

Notes

Input header or col_names must include the names in the default string.