io

This module contains functions to read spectra and save deltas. Can be imported without a need for MPI.

get_spectra_reader_function(input_dir, arms_to_keep, mock_analysis, skip_resomat, read_true_continuum, is_tile, read_exposures, program='dark')[source]

Returns a callable object (function) that returns a list of Spectrum objects for a given catalog of a single healpix. Essentially, a wrapper for the following functions:

If for data, SURVEY column must be present and sorted when calling the function this returns in the healpix grouping. Tile grouping requires TILEID and PETAL_LOC.

Parameters:

  • input_dir (str) – Input directory

  • arms_to_keep (list(str)) – Must only contain B, R and Z

  • mock_analysis (bool) – Reads for mock data if true.

  • skip_resomat (bool) – If true, do not read resomat.

  • read_true_continuum (bool) – If true, reads the true continuum for mock analysis.

  • is_tile (bool) – If true, reads for tile grouping

  • read_exposures (bool) – If true, reads exposures for healpix format only.

  • program (str, default: "dark") – Always use dark program.

Returns:

readerFunction – Call this function with a catalog of quasars in a single healpix.

Return type:

Callable

Raises:

RuntimeWarning – If number of quasars in the healpix file does not match the catalog.

read_resolution_matrices_onehealpix_data(catalog_hpx, input_dir, spectra_list, program='dark')[source]

Returns a list of Spectrum objects for a given catalog of a single healpix.

Parameters:

  • catalog_hpx (ndarray) – Catalog of quasars in a single healpix. If for data ‘SURVEY’ column must be present and sorted.

  • input_dir (str) – Input directory

  • spectra_list (list(Spectrum)) – List of spectra to read resolution matrix

  • program (str, default: "dark") – Always use dark program.

Returns:

spectra_list

Return type:

list(Spectrum)

Raises:

RuntimeWarning – If number of quasars in the healpix file does not match the catalog.

read_deltas(fname)[source]

Returns a list of all Delta objects in a file.

Parameters:

fname (str) – FITS file name

Returns:

deltas_list

Return type:

list(Delta)

Raises:

RuntimeError – If the file is missing certain columns and keys. See Delta.

save_deltas(spectra_list, outdir, save_by_hpx=False, mpi_rank=None)[source]

Saves given list of spectra as deltas. NO coaddition of arms. Each arm is saved separately. Only valid spectra are saved.

Parameters:

  • spectra_list (list(Spectrum)) – Continuum fitted spectra objects. All must be valid!

  • outdir (str) – Output directory. Does not save if empty of None

  • save_by_hpx (bool, default: False) – Saves by healpix if True. Has priority over mpi_rank

  • mpi_rank (int, default: None) – Rank of the MPI process. Save by mpi_rank if passed.

Raises:

  • QsonicException – If both mpi_rank and save_by_hpx is None/False.

  • QsonicException – If blinding is not set.

_read_true_continuum(targetids, fspec)[source]

Read true continuum as dictionary from file.

Parameters:

  • targetids (ndarray) – Target IDs.

  • fspec (str) – Filename to open.

Returns:

cont_data – This has keys for w1, w2, dwave and data, where data is ndarray and others are floats.

Return type:

dict( ndarray )

Raises:

RuntimeError – If number of quasars in TRUE_CONT does not match the input.

_read_onehealpix_file(targetids_by_survey, fspec, arms_to_keep, skip_resomat, fbrmap_columns=['TARGETID'])[source]

Common function to read a single FITS file in DESI healpix grouping.

Parameters:

  • targetids_by_survey (ndarray) – Targetids_by_survey (used to be catalog). If data, split by survey and contains only one survey.

  • fspec (str) – Filename to open.

  • arms_to_keep (list(str)) – Must only contain B, R and Z.

  • skip_resomat (bool) – If true, do not read resomat.

  • fbrmap_columns (list(str), default: ['TARGETID']) – Columns to read from FIBERMAP. For coadded files, it should be ['TARGETID']. For uncoadded files (spectra-), you should read at least [TARGETID, PETAL_LOC, FIBER, NIGHT, EXPID, TILEID].

Returns:

  • data (dict( ndarray )) – Only quasar spectra are read into keywords wave, flux etc. Resolution is read if present.

  • idx_cat (ndarray) – Indices in targetids_by_survey there were succesfully read.

  • fbrmap (ndarray) – Catalog with columns from the FIBERMAP. Note the size will not be equal to targetids_by_survey.size when reading exposures.

Raises:

RuntimeWarning – If number of quasars in the healpix file does not match the catalog.

_read_onehealpix_file_onlyreso(targetids_by_survey, fspec, arms_to_keep, spectra_list)[source]

Function to read forest region resolution matrix for data. Note reading resolution matrix is already fast for mocks.

Parameters:

  • targetids_by_survey (ndarray) – Targetids_by_survey (used to be catalog). If data, split by survey and contains only one survey.

  • fspec (str) – Filename to open.

  • arms_to_keep (list(str)) – Must only contain B, R and Z.

  • spectra_list (list(Spectrum)) – List of spectra that forest region is set. Modified in place and also returned.

Returns:

spectra_list – List of spectra where forestreso is set.

Return type:

list(Spectrum)

Raises:

RuntimeWarning – If number of quasars in the healpix file does not match the catalog.

read_onehealpix_file_data_coadd(catalog_hpx, input_dir, arms_to_keep, skip_resomat, program='dark')[source]

Read FITS files for all surveys needed for data.

Parameters:

  • catalog_hpx (ndarray) – Catalog for a healpix. Ordered by SURVEY and TARGETID.

  • input_dir (str) – Input directory.

  • arms_to_keep (list(str)) – Must only contain B, R and Z.

  • skip_resomat (bool) – If true, do not read resomat.

  • program (str, default: "dark") – Program.

Returns:

spectra_list

Return type:

list(Spectrum)

Raises:

RuntimeWarning – If number of quasars in the healpix file does not match the catalog.

read_onehealpix_file_data_uncoadd(catalog_hpx, input_dir, arms_to_keep, skip_resomat, program='dark')[source]

Read uncoadded spectra from FITS files for all surveys needed for data.

Parameters:

  • catalog_hpx (ndarray) – Catalog for a healpix. Ordered by SURVEY and TARGETID.

  • input_dir (str) – Input directory.

  • arms_to_keep (list(str)) – Must only contain B, R and Z.

  • skip_resomat (bool) – If true, do not read resomat.

  • program (str, default: "dark") – Program.

Returns:

spectra_list

Return type:

list(Spectrum)

read_onetile_coaddfile_data(catalog_tile, input_dir, arms_to_keep, skip_resomat)[source]

Read all petal coadd FITS files for a given tile.

Parameters:

  • catalog_tile (ndarray) – Catalog for a tile. Ordered by PETAL_LOC and TARGETID.

  • input_dir (str) – Input directory.

  • arms_to_keep (list(str)) – Must only contain B, R and Z.

  • skip_resomat (bool) – If true, do not read resomat.

Returns:

spectra_list

Return type:

list(Spectrum)

Raises:

RuntimeWarning – If number of quasars in the tile file does not match the catalog.

read_onehealpix_file_mock(catalog_hpx, input_dir, arms_to_keep, skip_resomat, read_true_continuum, nside=16)[source]

Read a single FITS file for mocks.

Parameters:

  • catalog_hpx (ndarray) – Catalog for a single healpix. Ordered by TARGETID.

  • input_dir (str) – Input directory.

  • arms_to_keep (list(str)) – Must only contain B, R and Z.

  • skip_resomat (bool) – If true, do not read resomat.

  • read_true_continuum (bool) – If true, reads the true continuum for mock analysis.

  • nside (int, default: 16) – NSIDE for healpix.

Returns:

spectra_list

Return type:

list(Spectrum)

Raises:

RuntimeWarning – If number of quasars in the healpix file does not match the catalog.

Arguments

add_io_parser(parser=None)[source]

Adds IO related arguments to parser. These arguments are grouped under ‘Input/output parameters and selections’. --input-dir and --catalog are required.

Parameters:

parser (argparse.ArgumentParser, default: None) –

Returns:

parser

Return type:

argparse.ArgumentParser

usage: foo [-h] --input-dir INPUT_DIR --catalog CATALOG [--tile-format]
           [--mock-analysis] [--keep-surveys KEEP_SURVEYS [KEEP_SURVEYS ...]]
           [--skip-resomat] [--arms {B,R,Z} [{B,R,Z} ...]] [--outdir OUTDIR]
           [--coadd-arms {before,after,disable}]
           [--exposures {before,after,disable}] [--save-by-hpx]

Input options

--input-dir, -i

Input directory.

--catalog

Catalog filename

--tile-format

Read tile coadd-*.fits files in tiles/cumulative directory.

Default: False

--mock-analysis

Input folder is mock. Uses nside=16

Default: False

--keep-surveys

Surveys to keep.

Default: [‘main’]

--skip-resomat

Skip reading resolution matrix for 3D.

Default: False

--arms

Possible choices: B, R, Z

Arms to read.

Default: [‘B’, ‘R’]

Output options

--outdir, -o

Output directory to save deltas.

--coadd-arms

Possible choices: before, after, disable

Coadds arms before or after continuum fitting or not at all.

Default: “before”

--exposures

Possible choices: before, after, disable

Reads exposures before or after continuum fitting and saves. Tile format not supported. Related function qsonic.scripts.qsonic_fit.mpi_read_exposures_after().

Default: “disable”

--save-by-hpx

Save by healpix. If not, saves by MPI rank.

Default: False