| Title: | Observations, Footprints and Receptors ('rtorf') and HYSPLIT Configuration Tools |
|---|---|
| Description: | Ingest, process, and visualize atmospheric observation data, specifically tailored for the NOAA Global Monitoring Laboratory (GML) ObsPack GLOBALView+ products <https://gml.noaa.gov/ccgg/obspack/index.html>. Includes specialized routines for filtering trace gas time-series data based on meteorological and quality control parameters. Additionally, streamlines the generation of configuration files for the Hybrid Single-Particle Lagrangian Integrated Trajectory (HYSPLIT) model <https://www.ready.noaa.gov/HYSPLIT.php>, supporting high-throughput parallel processing of trajectory simulations. |
| Authors: | Sergio Ibarra-Espinosa [aut, cre] (ORCID: <https://orcid.org/0000-0002-3162-1905>), Lei Hu [ctb] (ORCID: <https://orcid.org/0000-0002-5624-3832>) |
| Maintainer: | Sergio Ibarra-Espinosa <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 4.3.2 |
| Built: | 2026-06-10 10:50:52 UTC |
| Source: | https://github.com/atmoschem/rtorf |
file extension
fex(x)fex(x)
x |
character vector giving file paths. |
A character vector of the same length as x containing the file
extensions without the leading dot. Files without an extension return an
empty string ("").
source tools::file_ext
tools::file_ext()
Other helpers:
obs_footname(),
obs_format(),
obs_freq(),
obs_list.dt(),
obs_out(),
obs_rbind(),
obs_read_csvy(),
obs_roundtime(),
obs_trunc(),
obs_write_csvy(),
rtorf-deprecated,
sr()
## Not run: # Do not run ## End(Not run)## Not run: # Do not run ## End(Not run)
get_nt check the number of threads in this machine
get_nt()get_nt()
Integer with the max number of threads
{ get_nt() }{ get_nt() }
Calculate an approximation of local hour
obs_addltime( dt, timeUTC = "timeUTC", utc2lt = "site_utc2lst", longitude = "longitude", tz = "UTC", timeonly = FALSE )obs_addltime( dt, timeUTC = "timeUTC", utc2lt = "site_utc2lst", longitude = "longitude", tz = "UTC", timeonly = FALSE )
dt |
data.table |
timeUTC |
Character indicating the Time column as POSIXct |
utc2lt |
Character indicating the integer column to convert to local time if available |
longitude |
Character indicating the column with the lingitude |
tz |
Timezone, default "UTC" |
timeonly |
return only local_time column |
data.table with local time columns
time depending n longitude is by John Miller (GML/NOAA)
Other time:
obs_addmtime(),
obs_addstime(),
obs_addtime()
## Not run: # Do not run ## End(Not run)## Not run: # Do not run ## End(Not run)
This function add timeUTC as POSIX class, local time and ending sampling time
obs_addmtime(time, origin = "1970-01-01", tz = "UTC")obs_addmtime(time, origin = "1970-01-01", tz = "UTC")
time |
numeric time from MATLAB |
origin |
default "1970-01-01" |
tz |
Timezone, default "UTC" |
return the same data.frame adding solar time
Other time:
obs_addltime(),
obs_addstime(),
obs_addtime()
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) dt <- obs_addtime(dt) ## End(Not run)## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) dt <- obs_addtime(dt) ## End(Not run)
This function add timeUTC as POSIX class, local time and ending sampling time
obs_addstime(dt, tz = "UTC")obs_addstime(dt, tz = "UTC")
dt |
obspack data.table |
tz |
Timezone, default "UTC" |
return the same data.frame adding solar time
Other time:
obs_addltime(),
obs_addmtime(),
obs_addtime()
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) dt <- obs_addtime(dt) ## End(Not run)## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) dt <- obs_addtime(dt) ## End(Not run)
This function add timeUTC as POSIX class, local time and ending sampling time
obs_addtime(dt, verbose = TRUE, tz = "UTC", timeonly = FALSE)obs_addtime(dt, verbose = TRUE, tz = "UTC", timeonly = FALSE)
dt |
data.table |
verbose |
Logical, to show more info |
tz |
Timezone, default "UTC" |
timeonly |
return only timeUTC column |
A data.frame with with an index obspack.
timeUTC is calculated based on the field column start_time, the timeUTC_end is calculated using this approach: 1. If the column time_interval is not found, proceed with the calculation using midpoint_time 2. Else, use column time_interval
Other time:
obs_addltime(),
obs_addmtime(),
obs_addstime()
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) dt <- obs_addtime(dt) ## End(Not run)## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) dt <- obs_addtime(dt) ## End(Not run)
This function add aggregate cols by 'key_time'
obs_agg( dt, cols = c("year", "month", "day", "hour", "minute", "second", "time", "time_decimal", "value", "latitude", "longitude", "altitude", "pressure", "u", "v", "temperature", "type_altitude"), by = c("key_time", "site_code", "altitude_final", "type_altitude", "lab_1_abbr", "dataset_calibration_scale"), fn = "mean", verbose = TRUE )obs_agg( dt, cols = c("year", "month", "day", "hour", "minute", "second", "time", "time_decimal", "value", "latitude", "longitude", "altitude", "pressure", "u", "v", "temperature", "type_altitude"), by = c("key_time", "site_code", "altitude_final", "type_altitude", "lab_1_abbr", "dataset_calibration_scale"), fn = "mean", verbose = TRUE )
dt |
data.table |
cols |
Character which defines columns to be aggregated |
by |
String with columns to be aggregated |
fn |
Function to be applied to the columns, default mean |
verbose |
logical to show more information |
A data.frame with with an index.
By default add column timeUTC based on the input column 'key_time'
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) ## End(Not run)## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) ## End(Not run)
This function returns a arrays (or list of arrays) of convolved footpritns with flux
obs_convolve( foot_path = "AAA", name_foot = "foot1", flon = "foot1lon", flat = "foot1lat", time_foot, flux = "CTCO2", df_flux, flux_format = "%Y%m%d.nc", name_var_flux = 1, factor = 1e+09, fn = NULL, flip = FALSE, as_list = FALSE, verbose = TRUE )obs_convolve( foot_path = "AAA", name_foot = "foot1", flon = "foot1lon", flat = "foot1lat", time_foot, flux = "CTCO2", df_flux, flux_format = "%Y%m%d.nc", name_var_flux = 1, factor = 1e+09, fn = NULL, flip = FALSE, as_list = FALSE, verbose = TRUE )
foot_path |
path for footprint (length 1). |
name_foot |
name of the footprint variable in the NetCDF file. |
flon |
name of lons |
flat |
name of lats |
time_foot |
Time of the footprints (in the name file) or third dimension |
flux |
String, default NOAA "CTCO2". Implies bio, ocn, fossil, fire. |
df_flux |
data.table with columns f (full path) and nf file name 'Ymd.nc' |
flux_format |
string with date format 'Ymd.nc' |
name_var_flux |
string or numeric of position for the name var in the flux format. e.g.: If missing, asusmes date ("2020-01-01"), if string something like ("flux"), if numeric, identify the var names and you select which of them (first, second, etc) |
factor |
number to multiply fluxes. |
fn |
string with function to aggregate convolved fluxes, e.g. 'mean', 'sum', 'max', etc. |
flip |
Logical, to vertically flip emissions, or not |
as_list |
Logical, to return list of arrays |
verbose |
Logical, to display more information |
An array of footprints and convolved footprints, or a list of footprints, convolved fooprints, latitudes, longitudes and time.
main assumption is that fluxes have same spatial dimensions as footprints when flux is "monthly", "daily" or "yearly", it assumes flux variable has the same name as "
## Not run: # do not run ## End(Not run)## Not run: # do not run ## End(Not run)
This function creates a data.frame with the expected footprints in .nc and compare with the actual footprints. The idea is verify is there are missing footprints.
obs_find_receptors( path, year, month, day, hour, minute, lat, lon, alt, out = paste0(tempfile(), ".csvy"), verbose = FALSE )obs_find_receptors( path, year, month, day, hour, minute, lat, lon, alt, out = paste0(tempfile(), ".csvy"), verbose = FALSE )
path |
String with path hwere are stored the footprints |
year |
numeric number |
month |
numeric number |
day |
numeric number |
hour |
numeric number |
minute |
numeric number |
lat |
numeric number |
lon |
numeric number |
alt |
numeric number |
out |
outfile path. |
verbose |
logical to show more information |
A data.frame with with expected footprints
## Not run: # do not run p <- "/path/to/continuous/" # here we have year/month/hysplit*.nc x <- dt dt <- obs_find_receptors(p, year, month....) ## End(Not run)## Not run: # do not run p <- "/path/to/continuous/" # here we have year/month/hysplit*.nc x <- dt dt <- obs_find_receptors(p, year, month....) ## End(Not run)
This function returns a arrays (or list of arrays) of convolved footpritns with flux
obs_foot( foot_path = "AAA", name_foot = "foot1", flon = "foot1lon", flat = "foot1lat", time_foot, fn = NULL, as_list = FALSE, verbose = TRUE )obs_foot( foot_path = "AAA", name_foot = "foot1", flon = "foot1lon", flat = "foot1lat", time_foot, fn = NULL, as_list = FALSE, verbose = TRUE )
foot_path |
path for footprint (length 1). |
name_foot |
name of the footprint variable in the NetCDF file. |
flon |
name of lons |
flat |
name of lats |
time_foot |
Time of the footprints (in the name file) or third dimension |
fn |
string with function to aggregate convolved fluxes, e.g. 'mean', 'sum', 'max', etc. |
as_list |
Logical, to return list of arrays |
verbose |
Logical, to display more information |
An array of footprints, or a list of footprints, latitudes, longitudes and time.
## Not run: # do not run ## End(Not run)## Not run: # do not run ## End(Not run)
Reorders the axes of a 3-D footprint array produced by
obs_traj_foot so that its dimension order matches the
CF-convention expected by obs_nc / terra::rast(),
or converts back in the opposite direction.
obs_foot_flip(arr, from = c("traj", "cf"), flip_lat = FALSE, flip_lon = FALSE)obs_foot_flip(arr, from = c("traj", "cf"), flip_lat = FALSE, flip_lon = FALSE)
arr |
A 3-D numeric array. Two layouts are accepted:
|
from |
Character string, either |
flip_lat |
Logical. If |
flip_lon |
Logical. If |
Why this is needed.
obs_traj_foot returns an array with dimensions
[lat, lon, time] (R row-major: row = latitude index).
NetCDF files written with obs_nc preserve that order, but
terra::rast() follows the CF / GIS convention where the
first spatial dimension is longitude (x-axis). Reading the file
back with terra therefore transposes the spatial plane, which is
why a t() call was needed on the raster.
Calling obs_foot_flip(arr) before passing the array to
obs_nc (or after reading it back) resolves the mismatch without
manual transposing.
A 3-D array with reordered (and optionally flipped) axes.
## Not run: # foot.arr comes from obs_traj_foot: dims [lat, lon, time] foot_cf <- obs_foot_flip(foot.arr) # now [lon, lat, time] obs_nc( lat = lats, lon = lons, time_nc = time_vec, vars_out = "foot1", units_out = "(ppb/nanomol m-2 s-1)", nc_out = "output.nc", larrays = list(foot1 = foot_cf) ) # terra::rast("output.nc") now reads correctly without t() ## End(Not run)## Not run: # foot.arr comes from obs_traj_foot: dims [lat, lon, time] foot_cf <- obs_foot_flip(foot.arr) # now [lon, lat, time] obs_nc( lat = lats, lon = lons, time_nc = time_vec, vars_out = "foot1", units_out = "(ppb/nanomol m-2 s-1)", nc_out = "output.nc", larrays = list(foot1 = foot_cf) ) # terra::rast("output.nc") now reads correctly without t() ## End(Not run)
return the expected name for the NetCDF footprint
obs_footname( time = NULL, year, month, day, hour, minute, second, lat, lon, alt, fullpath = FALSE, out, ... )obs_footname( time = NULL, year, month, day, hour, minute, second, lat, lon, alt, fullpath = FALSE, out, ... )
time |
POSIXct time to extract time variblaes |
year |
numeric number |
month |
numeric number |
day |
numeric number |
hour |
numeric number |
minute |
numeric number |
second |
numeric number, optional |
lat |
numeric number |
lon |
numeric number |
alt |
numeric number |
fullpath |
Logical, to add or not YYYY/MO/hysplit to id |
out |
outfile path. |
... |
data.table::fwrite arguments. |
A concatenation of characters representing a spatio-temporal point.
source https://stackoverflow.com/a/47015304/2418532
# IMPORTANT!!! # This function will generate the expected NetCDF file name. # It assumes that the name was generated under the following considerations: # time variables (year, month, day, etc) have a format of two digits, eg "0.1" # latitude and longitude have been round with 4 decimals # The format for latitude is 2 integers, a point and 4 decimals # The format for longitude is 3 integers, a point and 4 decimals
Other helpers:
fex(),
obs_format(),
obs_freq(),
obs_list.dt(),
obs_out(),
obs_rbind(),
obs_read_csvy(),
obs_roundtime(),
obs_trunc(),
obs_write_csvy(),
rtorf-deprecated,
sr()
## Not run: # Do not run obs_footname(time = Sys.time(), second = data.table::second(Sys.Time()), lat = 0, lon = 0, alt = 0) obs_footname(year = 2020, month = 12, day = 30, hour = 9, minute = 54, lat = 3.2133, lon = 30.9131, alt = 497, fullpath = TRUE) obs_footname(year = 2020, month = 12, day = 30, hour = 9, minute = 54, lat = 1, lon = -130.9131, alt = 497, fullpath = TRUE) ## End(Not run)## Not run: # Do not run obs_footname(time = Sys.time(), second = data.table::second(Sys.Time()), lat = 0, lon = 0, alt = 0) obs_footname(year = 2020, month = 12, day = 30, hour = 9, minute = 54, lat = 3.2133, lon = 30.9131, alt = 497, fullpath = TRUE) obs_footname(year = 2020, month = 12, day = 30, hour = 9, minute = 54, lat = 1, lon = -130.9131, alt = 497, fullpath = TRUE) ## End(Not run)
Formatting data
obs_format()obs_format()
formatted characters
Other helpers:
fex(),
obs_footname(),
obs_freq(),
obs_list.dt(),
obs_out(),
obs_rbind(),
obs_read_csvy(),
obs_roundtime(),
obs_trunc(),
obs_write_csvy(),
rtorf-deprecated,
sr()
## Not run: # do not run ## End(Not run)## Not run: # do not run ## End(Not run)
return numeric vector in intervals
obs_freq(x, freq, ...)obs_freq(x, freq, ...)
x |
numeric, longer than 'freq' |
freq |
numeric vector |
... |
findInterval arguments |
numeric vector
Other helpers:
fex(),
obs_footname(),
obs_format(),
obs_list.dt(),
obs_out(),
obs_rbind(),
obs_read_csvy(),
obs_roundtime(),
obs_trunc(),
obs_write_csvy(),
rtorf-deprecated,
sr()
## Not run: # Do not run ## End(Not run)## Not run: # Do not run ## End(Not run)
add columns of
obs_grid( min.x, max.x, min.y, max.y, numpix.x, numpix.y, coarse.factor = 1, adaptive = TRUE )obs_grid( min.x, max.x, min.y, max.y, numpix.x, numpix.y, coarse.factor = 1, adaptive = TRUE )
min.x |
grid indices |
max.x |
grid indices |
min.y |
grid indices |
max.y |
grid indices |
numpix.x |
number pixels x |
numpix.y |
number pixels y |
coarse.factor |
integer to coarse the grid resolution |
adaptive |
logical, to use adaptive gridding |
return footprint
Other helpers legacy:
obs_id2pos(),
obs_info2id(),
obs_julian(),
obs_normalize_dmass(),
obs_traj_foot()
obs_grid(1, 10, 1, 10, 100, 100, 1)obs_grid(1, 10, 1, 10, 100, 100, 1)
Compares the total foot between the outputs of
obs_grid_simple and obs_grid_kernel
and prints a brief summary to the console. A relative difference
larger than 5% triggers a warning with diagnostic
suggestions.
obs_grid_check(simple, kernel)obs_grid_check(simple, kernel)
simple |
Output list from |
kernel |
Output list from |
The kernel method loses a small amount of foot at the ±3
sigma boundary of the truncated Gaussian. Larger differences
usually indicate a grid extent that is too tight (edge particles
are discarded) or a bandwidth that is large relative to the domain.
Invisibly returns a named numeric vector with three
elements: simple (total foot from bin-and-sum),
kernel (total foot from kernel method), and
relative_diff_pct (absolute percentage difference
relative to simple).
obs_grid_simple, obs_grid_kernel
## Not run: bs <- obs_grid_simple(d, lon_min, lat_min, lon_max, lat_max) gk <- obs_grid_kernel(d, lon_min, lat_min, lon_max, lat_max, bandwidth = 0.2) chk <- obs_grid_check(bs, gk) chk["relative_diff_pct"] ## End(Not run)## Not run: bs <- obs_grid_simple(d, lon_min, lat_min, lon_max, lat_max) gk <- obs_grid_kernel(d, lon_min, lat_min, lon_max, lat_max, bandwidth = 0.2) chk <- obs_grid_check(bs, gk) chk["relative_diff_pct"] ## End(Not run)
Bins particles into a 3D grid: longitude, latitude, and time. Supports two methods: "simple" (bin-and-sum) and "kernel" (Gaussian smoothing).
obs_grid_cube( x, lon_min, lat_min, lon_max, lat_max, res = 0.1, nt = 240L, t0 = 0, dt = 60, method = c("simple", "kernel"), bandwidth = res, use_haversine = FALSE, n_threads = 1L, npar = 1L )obs_grid_cube( x, lon_min, lat_min, lon_max, lat_max, res = 0.1, nt = 240L, t0 = 0, dt = 60, method = c("simple", "kernel"), bandwidth = res, use_haversine = FALSE, n_threads = 1L, npar = 1L )
x |
A |
lon_min, lat_min, lon_max, lat_max
|
Grid extent (degrees). |
res |
Spatial resolution (degrees). |
nt |
Number of time layers in the output cube. |
t0 |
Reference time for the first layer (minutes). Default |
dt |
Time step per layer (minutes). Default |
method |
Character. Either |
bandwidth |
Kernel standard deviation for |
use_haversine |
Logical. If |
n_threads |
Number of OpenMP threads. |
npar |
Normalization factor (total particles released). |
A named list:
gridNumeric 3D array of dimension [ny x nx x nt].
lon, latCell-centre coordinates.
timeStart time of each bin (minutes back from t0).
Computes a gridded footprint by spreading each particle's
foot value over neighbouring cells using a 2-D isotropic
Gaussian kernel. The kernel integrates to 1 (cell area
lon_res * lat_res is included in the weight), so the
total foot is approximately conserved — the small
residual arises from truncation at ±3 sigma.
obs_grid_kernel( x, lon_min, lat_min, lon_max, lat_max, lon_res = 0.1, lat_res = 0.1, bandwidth = lon_res, use_haversine = FALSE, n_threads = 1L, npar = 1L )obs_grid_kernel( x, lon_min, lat_min, lon_max, lat_max, lon_res = 0.1, lat_res = 0.1, bandwidth = lon_res, use_haversine = FALSE, n_threads = 1L, npar = 1L )
x |
A |
lon_min |
Western edge of the output grid (degrees). |
lat_min |
Southern edge of the output grid (degrees). |
lon_max |
Eastern edge of the output grid (degrees). |
lat_max |
Northern edge of the output grid (degrees). |
lon_res |
Cell width in longitude (degrees). Default |
lat_res |
Cell height in latitude (degrees). Default |
bandwidth |
Kernel standard deviation. Units depend on
Default |
use_haversine |
Logical. If |
n_threads |
Number of OpenMP threads. Default |
npar |
Number of particles for normalization. Default |
Use this function when particle density is too low to fill every
grid cell, or for display purposes. For analyses that require
exact conservation of the total sensitivity, prefer
obs_grid_simple.
A named list with the same structure as
obs_grid_simple:
gridNumeric matrix [nlon x nlat] of
kernel-smoothed foot.
lonCell-centre longitude vector.
latCell-centre latitude vector.
The Fortran routine skips particles with foot == 0
before entering the kernel loop, so sparse footprints with many
zero-weight particles incur no unnecessary computation.
obs_grid_simple for the unsmoothed primary
output, obs_grid_check to verify conservation
between the two methods.
## Not run: library(data.table) dat <- fread("PARTICLE.DAT") d <- dat[time %in% sort(unique(time))[1:4]] # Cartesian mode (degrees, fast) gk <- obs_grid_kernel( x = d, lon_min = floor(min(d$lon)) - 0.1, lat_min = floor(min(d$lat)) - 0.1, lon_max = ceiling(max(d$lon)) + 0.1, lat_max = ceiling(max(d$lat)) + 0.1, lon_res = 0.1, lat_res = 0.1, bandwidth = 0.2, # 2-cell sigma in degrees use_haversine = FALSE, n_threads = 4L ) image(gk$lon, gk$lat, log1p(gk$grid), main = "Gaussian kernel") # Haversine mode (metres, accurate at high latitudes) gk_hav <- obs_grid_kernel( x = d, lon_min = -130, lat_min = 55, lon_max = -100, lat_max = 75, lon_res = 0.1, lat_res = 0.1, bandwidth = 15000, # 15 km use_haversine = TRUE, n_threads = 4L ) ## End(Not run)## Not run: library(data.table) dat <- fread("PARTICLE.DAT") d <- dat[time %in% sort(unique(time))[1:4]] # Cartesian mode (degrees, fast) gk <- obs_grid_kernel( x = d, lon_min = floor(min(d$lon)) - 0.1, lat_min = floor(min(d$lat)) - 0.1, lon_max = ceiling(max(d$lon)) + 0.1, lat_max = ceiling(max(d$lat)) + 0.1, lon_res = 0.1, lat_res = 0.1, bandwidth = 0.2, # 2-cell sigma in degrees use_haversine = FALSE, n_threads = 4L ) image(gk$lon, gk$lat, log1p(gk$grid), main = "Gaussian kernel") # Haversine mode (metres, accurate at high latitudes) gk_hav <- obs_grid_kernel( x = d, lon_min = -130, lat_min = 55, lon_max = -100, lat_max = 75, lon_res = 0.1, lat_res = 0.1, bandwidth = 15000, # 15 km use_haversine = TRUE, n_threads = 4L ) ## End(Not run)
Bins STILT/HYSPLIT particles into a regular longitude/latitude
grid and sums the foot sensitivity values per cell.
This is the primary scientific output: no smoothing is applied,
no foot is redistributed spatially, and the grid total exactly
equals the particle total.
obs_grid_simple( x, lon_min, lat_min, lon_max, lat_max, res = 0.1, n_threads = 1L, npar = 1L )obs_grid_simple( x, lon_min, lat_min, lon_max, lat_max, res = 0.1, n_threads = 1L, npar = 1L )
x |
A |
lon_min |
Western edge of the output grid (degrees). |
lat_min |
Southern edge of the output grid (degrees). |
lon_max |
Eastern edge of the output grid (degrees). |
lat_max |
Northern edge of the output grid (degrees). |
res |
Cell size in degrees. Both longitude and latitude use
the same value (square cells). Default |
n_threads |
Number of OpenMP threads passed to the Fortran
routine. Default |
npar |
Number of particles used for normalization. The output
grid is divided by this value to return the average footprint
sensitivity. Default |
A named list with three elements:
gridNumeric matrix of dimension
[nlon x nlat] containing the summed foot
per cell, divided by npar. Row i corresponds
to lon[i], column j to lat[j].
lonNumeric vector of cell-centre longitudes
(length nlon).
latNumeric vector of cell-centre latitudes
(length nlat).
Particles whose coordinates fall outside the rectangle
[lon_min, lon_max) x [lat_min, lat_max) are silently
discarded. Add a small buffer to the grid extent if edge
particles matter.
obs_grid_kernel for a smoothed
alternative, obs_grid_check to compare totals.
## Not run: library(data.table) dat <- fread("PARTICLE.DAT") d <- dat[time %in% sort(unique(time))[1:4]] # first hour bs <- obs_grid_simple( x = d, lon_min = floor(min(d$lon)) - 0.1, lat_min = floor(min(d$lat)) - 0.1, lon_max = ceiling(max(d$lon)) + 0.1, lat_max = ceiling(max(d$lat)) + 0.1, res = 0.1, n_threads = 4L ) image(bs$lon, bs$lat, log1p(bs$grid), main = "Bin and sum") ## End(Not run)## Not run: library(data.table) dat <- fread("PARTICLE.DAT") d <- dat[time %in% sort(unique(time))[1:4]] # first hour bs <- obs_grid_simple( x = d, lon_min = floor(min(d$lon)) - 0.1, lat_min = floor(min(d$lat)) - 0.1, lon_max = ceiling(max(d$lon)) + 0.1, lat_max = ceiling(max(d$lat)) + 0.1, res = 0.1, n_threads = 4L ) image(bs$lon, bs$lat, log1p(bs$grid), main = "Bin and sum") ## End(Not run)
This function creates a ASCDATA.CFG file for HYSPLIT model.
obs_hysplit_ascdata( llc = c(-90, -180), spacing = c(1, 1), n = c(180, 360), landusecat = 2, rough = 0.2, bdyfiles = "../bdyfiles/", ascdata = "ASCDATA.CFG" )obs_hysplit_ascdata( llc = c(-90, -180), spacing = c(1, 1), n = c(180, 360), landusecat = 2, rough = 0.2, bdyfiles = "../bdyfiles/", ascdata = "ASCDATA.CFG" )
llc |
Lower left corner, default c(-90.0, -180.0) |
spacing |
spacing in degress, default c(1.0, 1.0) |
n |
number of data points, default c(180, 360) |
landusecat |
land use category, default 2 |
rough |
default roughness length (meters), default 0.2 |
bdyfiles |
directory location of the data files, default '../bdyfiles/' |
ascdata |
file, default ASCDATA.CFG |
A ASCDATA.CFG file
{ # Do not run ascdata_file <- tempfile() obs_hysplit_ascdata(ascdata = ascdata_file) cat(readLines(ascdata_file), sep = "\n") }{ # Do not run ascdata_file <- tempfile() obs_hysplit_ascdata(ascdata = ascdata_file) cat(readLines(ascdata_file), sep = "\n") }
This function creates a CONTROL file for HYSPLIT model. It uses inputs from a data.frame with the receptor information.
obs_hysplit_control( df, year, month, day, hour, minute, lat, lon, alt, start_time = NULL, nlocations = 1, duration = -240, vertical_motion = 5, top_model_domain = 10000, met = c("hrrr", "nams", "gfs0p25"), nmet = abs(duration/24) + 1, metpath = c("/work/noaa/lpdm/metfiles/hrrr/", "/work/noaa/lpdm/metfiles/nams/", "/work/noaa/lpdm/metfiles/gfs0p25/"), metformat = c(hrrr = "%Y%m%d_hrrr", nams = "%Y%m%d_hysplit.t00z.namsa", gfs0p25 = "%Y%m%d_gfs0p25", era5 = "ERA5_%Y%m%d.ARL"), ngases = 1, gas = "Foot", emissions_rate = 0, hour_emissions = 0.01, release_start = NULL, nsim_grids = 1, center_conc_grids = c(0, 0), grid_spacing = c(0.1, 0.1), grid_span = c(30, 30), nconc = "cdump", nvert_levels = 1, height_vert_levels = 50, sampling_start_time = c(0, 0, 0, 0, 0), sampling_end_time = c(0, 0, 0, 0, 0), sampling_interval_type = c(0, abs(duration), 0), npol_depositing = 1, particle_params = c(0, 0, 0), dmwsrdre = c(0, 0, 0, 0, 0), wrhcicbc = c(0, 0, 0), radiactive_decay = 0, pol_res = 0, control = "CONTROL", verbose = FALSE )obs_hysplit_control( df, year, month, day, hour, minute, lat, lon, alt, start_time = NULL, nlocations = 1, duration = -240, vertical_motion = 5, top_model_domain = 10000, met = c("hrrr", "nams", "gfs0p25"), nmet = abs(duration/24) + 1, metpath = c("/work/noaa/lpdm/metfiles/hrrr/", "/work/noaa/lpdm/metfiles/nams/", "/work/noaa/lpdm/metfiles/gfs0p25/"), metformat = c(hrrr = "%Y%m%d_hrrr", nams = "%Y%m%d_hysplit.t00z.namsa", gfs0p25 = "%Y%m%d_gfs0p25", era5 = "ERA5_%Y%m%d.ARL"), ngases = 1, gas = "Foot", emissions_rate = 0, hour_emissions = 0.01, release_start = NULL, nsim_grids = 1, center_conc_grids = c(0, 0), grid_spacing = c(0.1, 0.1), grid_span = c(30, 30), nconc = "cdump", nvert_levels = 1, height_vert_levels = 50, sampling_start_time = c(0, 0, 0, 0, 0), sampling_end_time = c(0, 0, 0, 0, 0), sampling_interval_type = c(0, abs(duration), 0), npol_depositing = 1, particle_params = c(0, 0, 0), dmwsrdre = c(0, 0, 0, 0, 0), wrhcicbc = c(0, 0, 0), radiactive_decay = 0, pol_res = 0, control = "CONTROL", verbose = FALSE )
df |
data.frame of receptor information. Must include, "year", "month", "day", "hour" (0:23), "minute", "latitude", "longitude", "altitude" |
year |
year, if missing df. |
month |
month, if missing df. |
day |
day, if missing df. |
hour |
hour, if missing df. |
minute |
minute, if missing df. |
lat |
latitude, if missing df. |
lon |
longitude, if missing df. |
alt |
altitude, if missing df. |
start_time |
String to be added, default derived from df. |
nlocations |
number of locations. |
duration |
number of hours of release. (Negative is backwards in time). |
vertical_motion |
Vertical motion option. (0:data 1:isob 2:isen 3:dens 4:sigma 5:diverg 6:msl2agl 7:average 8:damped). . The default "data" selection will use the meteorological model's vertical velocity fields; other options include isobaric, isentropic, constant density, constant internal sigma coordinate, computed from the velocity divergence, a special transformation to correct the vertical velocities when mapped from quasi-horizontal surfaces (such as relative to MSL) to HYSPLIT's internal terrain following sigma coordinate, and a special option (7) to spatially average the vertical velocity. The averaging distance is automatically computed from the ratio of the temporal frequency of the data to the horizontal grid resolution. Default 5 |
top_model_domain |
altitude above ground level (m). |
met |
meteorological models to be used. |
nmet |
Number of days for the meteorological files. Default is number of days in duration plus two days. nmet is the number of simultaneous input meteorological files. For instance, 11 means that for each meteorological grid, 11 files are expected. Usually, the files are daily. Note that the same number of files are required for each grid in this approach. Hysplit expects something like 2 11, which means 2 meteorological grids with 11 files each. The number 2 comes from the length of met files. |
metpath |
paths for each meteorological model output. |
metformat |
format to applied to the meteorological daily file |
ngases |
Default 1. |
gas |
default "Foot". |
emissions_rate |
Default 0 |
hour_emissions |
hour release, depend of type of release, instantaneous 0.01, continuous 1. |
release_start |
derived from df. |
nsim_grids |
Number of simulated grids. |
center_conc_grids |
center coordinates for conc grid. |
grid_spacing |
grid spacing, default is 0.1 degree. |
grid_span |
model extension in degrees by dimension. |
nconc |
name of the concentration file, default is "cdump" |
nvert_levels |
number of vertical levels |
height_vert_levels |
hright vertical levels (50) |
sampling_start_time |
2 digits year, month, day, hour, minute |
sampling_end_time |
2 digits year, month, day, hour, minute |
sampling_interval_type |
type, hour, minure |
npol_depositing |
number of pollutant depositing |
particle_params |
Particle diameter, density, and shape |
dmwsrdre |
DepVel, MW, SurfReRa, DifRa, EHenry |
wrhcicbc |
Wet removal: HC, incloud, belowcloud |
radiactive_decay |
days |
pol_res |
Pollutant Resuspension (1/m) |
control |
name of the file, default "CONTROL" |
verbose |
logical, to show more information |
A CONTROL file
{ # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) df <- dt[1] control_file <- tempfile() obs_hysplit_control(df, control = control_file) ff <- readLines(control_file) cat(ff, sep = "\n") }{ # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) df <- dt[1] control_file <- tempfile() obs_hysplit_control(df, control = control_file) ff <- readLines(control_file) cat(ff, sep = "\n") }
This function creates a CONTROL file for HYSPLIT model. It uses inputs from a data.frame with the receptor information.
obs_hysplit_control_read(control = "CONTROL")obs_hysplit_control_read(control = "CONTROL")
control |
CONTROL text file |
A list with CONTROL information
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) df <- dt[1] control_file <- tempfile() obs_hysplit_control(df, control = control_file) ff <- readLines(control_file) cat(ff, sep = "\n") obs_hysplit_control_read(control_file) ## End(Not run)## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) df <- dt[1] control_file <- tempfile() obs_hysplit_control(df, control = control_file) ff <- readLines(control_file) cat(ff, sep = "\n") obs_hysplit_control_read(control_file) ## End(Not run)
This function creates a SETUP.CFG file for HYSPLIT model.
obs_hysplit_setup( idsp = 2, capemin = 500, vscales = -1, kbls = 1, kblt = 5, kmixd = 0, initd = 0, veght = 0.5, kmix0 = 150, numpar = 500, maxpar = 500, ichem = 8, krand = 4, varsiwant = c("time", "indx", "lati", "long", "zagl", "zsfc", "foot", "samt", "temp", "dswf", "mlht", "dens", "dmas", "sigw", "tlgr"), ivmax = length(varsiwant), outdt = 15, extra_params, bypass_params, setup = "SETUP.CFG" )obs_hysplit_setup( idsp = 2, capemin = 500, vscales = -1, kbls = 1, kblt = 5, kmixd = 0, initd = 0, veght = 0.5, kmix0 = 150, numpar = 500, maxpar = 500, ichem = 8, krand = 4, varsiwant = c("time", "indx", "lati", "long", "zagl", "zsfc", "foot", "samt", "temp", "dswf", "mlht", "dens", "dmas", "sigw", "tlgr"), ivmax = length(varsiwant), outdt = 15, extra_params, bypass_params, setup = "SETUP.CFG" )
idsp |
particle dispersion scheme 1:HYSPLIT 2:STILT |
capemin |
-1 no convection; -2 Grell convection scheme; -3 extreme convection; >0 enhanced vertical mixing when CAPE exceeds this value (J/kg) |
vscales |
vertical Lagrangian time scale (sec) for stable PBL |
kbls |
boundary layer stability derived from 1:fluxes 2:wind_temperature |
kblt |
boundary layer turbulence parameterizations 1:Beljaars 2:Kanthar 3:TKE 4:Measured 5:Hanna |
kmixd |
mixed layer obtained from 0:input 1:temperature 2:TKE 3:modified Richardson |
initd |
initial distribution, particle, puff, or combination |
veght |
Height below which particle's time is spent is tallied to calculate footprint for PARTICLE_STILT.DAT less than or equal to 1. 0: fraction of PBL height; greater than 1.0: heightAGL (m) |
kmix0 |
minimum mixing depth |
numpar |
number of puffs or particles to released per cycle |
maxpar |
maximum number of particles carried in simulation |
ichem |
chemistry conversion modules 0:none 1:matrix 2:convert 3:dust 4: conc grid equal to met grid, 5: divide output mass by air density (kg/m3) to sum as mixing ration, 7: transport deposited particles on the ocean surface, 8: stilt mode mixing ratio and varying layer, 9: set concentration layer one to a fraction of the boundary layer, 10: restructure concentration grid into time-varying transfer matrix, 11: enable daughter produyct calculation. |
krand |
0 method to calculate random number 0=precompute if NUMPAR greater than 5000 or dynamic if NUMPAR less than or equal to 5000; 1=precalculated; 2=calculated in pardsp; 3=none; 4=random initial seed number and calculated in pardsp; 10=same as 0 with random initial seed for non-dispersion applications; 11=same as 1 with random initial seed for non-dispersion applications; 12=same as 2 with random initial seed for non-dispersion applications; 13=same as 3 with random initial seed for non-dispersion applications |
varsiwant |
='TIME','INDX','LONG','LATI','ZAGL','SIGW','TLGR','ZSFC', TEMP', 'SAMT','FOOT','SHTF','DMAS','DENS','RHFR','SPHU','DSWF','WOUT','MLHT','PRES' variables written to PARTICLE_STILT.DAT. However, the default in this case is: c('time','indx', 'lati','long','zagl', 'zsfc','foot','samt', 'temp', 'dswf','mlht','dens','dmas','sigw','tlgr' |
ivmax |
0 number of variables written to PARTICLE_STILT.DAT. Must equal the number of variables listed for variable VARSIWANT |
outdt |
Default 15. defines the output frequency in minutes of the endpoint positions in the PARTICLE.DAT file when the STILT emulation mode is configured. The default value of 0 results in output each time step while the positive value gives the output frequency in minutes. A negative value disables output. The output frequency should be an even multiple of the time step and be evenly divisible into 60. In STILT emulation mode, the time step is forced to one minute. |
extra_params |
more parameters |
bypass_params |
named vector of characters to bypass all other arguments. If this list is available, only the content of this list will be used to write SETUP.CFG file and not other arguments. |
setup |
Default SETUP.CFG |
A SETUP.CFG file
The var description comes from hysplit 5.3 manual page 214 From the hysplit user guide: STILT mode The STILT model incorporates the variation of HYSPLIT developed by Lin et al. (2003 - JGR, VOL. 108, NO. D16, 4493, doi:10.1029/2002JD003161) that can be used to estimate upwind surface fluxes from atmospheric measurements. Two changes are introduced; the mass summation is divided by air density resulting in a mixing ratio output field (ICHEM=6) and the lowest concentration summation layer (concentration layer top depth) is permitted to vary with the mixed layer depth (ICHEM=9). The ICHEM=8 switch turns on both density and varying layer depth. Two text files of particle position information (PARTICLE.DAT and PARTICLE_STILT.DAT) at each time step will also be created unless the namelist parameter OUTDT defining the output interval (min) is changed. PARTICLE_STILT.DAT follows the same format as STILT. The footprint output in PARTICLE.DAT represents particles that were below 50 particles that were below a user defined height (VEGHT).
{ # Do not run # default setup_file <- tempfile() obs_hysplit_setup(setup = setup_file) cat(readLines(setup_file),sep = "\n") # bypass setup_file <- tempfile() obs_hysplit_setup(bypass_params = c(lala = 1), setup = setup_file) cat(readLines(setup_file),sep = "\n") }{ # Do not run # default setup_file <- tempfile() obs_hysplit_setup(setup = setup_file) cat(readLines(setup_file),sep = "\n") # bypass setup_file <- tempfile() obs_hysplit_setup(bypass_params = c(lala = 1), setup = setup_file) cat(readLines(setup_file),sep = "\n") }
return ndata.frame based on footprint/receptor id
obs_id2pos(id, sep = "x", asdf = FALSE)obs_id2pos(id, sep = "x", asdf = FALSE)
id |
string |
sep |
string |
asdf |
Logical, to return as data.frame or not |
Helpers Legacy
data.frame with time and location based on input
Other helpers legacy:
obs_grid(),
obs_info2id(),
obs_julian(),
obs_normalize_dmass(),
obs_traj_foot()
## Not run: # Do not run id <- '2002x08x03x10x45.00Nx090.00Ex00030' (obs_id2pos(id, asdf = TRUE) -> dx) id <- c('2002x08x03x10x00x45.000Nx090.000Ex00030', '2002x08x03x10x55x45.335Sx179.884Wx00030') (obs_id2pos(id) -> dx) (obs_id2pos(id, asdf = TRUE) -> dx) (obs_id2pos(rep(id, 2)) -> dx) (obs_id2pos(rep(id, 2), asdf = TRUE) -> dx) ## End(Not run)## Not run: # Do not run id <- '2002x08x03x10x45.00Nx090.00Ex00030' (obs_id2pos(id, asdf = TRUE) -> dx) id <- c('2002x08x03x10x00x45.000Nx090.000Ex00030', '2002x08x03x10x55x45.335Sx179.884Wx00030') (obs_id2pos(id) -> dx) (obs_id2pos(id, asdf = TRUE) -> dx) (obs_id2pos(rep(id, 2)) -> dx) (obs_id2pos(rep(id, 2), asdf = TRUE) -> dx) ## End(Not run)
return footprint/receptor id
obs_info2id(yr, mo, dy, hr, mn = 0, lat, lon, alt, sep = "x", long = TRUE)obs_info2id(yr, mo, dy, hr, mn = 0, lat, lon, alt, sep = "x", long = TRUE)
yr |
year |
mo |
month |
dy |
day |
hr |
hour |
mn |
minute |
lat |
latitude |
lon |
longitude |
alt |
altitude above ground level |
sep |
character, default"x" |
long |
Logical, to add minute, and rounded with 2 decimals, instead of 4. default TRUE |
a string with the receptor id
Other helpers legacy:
obs_grid(),
obs_id2pos(),
obs_julian(),
obs_normalize_dmass(),
obs_traj_foot()
## Not run: # Do not run obs_info2id(yr = 2002, mo = 8, dy = 3, hr = 10, mn = 0, lat = 42, lon = -90, alt = 1) [1] ## End(Not run)## Not run: # Do not run obs_info2id(yr = 2002, mo = 8, dy = 3, hr = 10, mn = 0, lat = 42, lon = -90, alt = 1) [1] ## End(Not run)
return numeric
return footprint/receptor id
obs_julian(y, m, d, origin., legacy = FALSE, verbose = TRUE) obs_julian(y, m, d, origin., legacy = FALSE, verbose = TRUE)obs_julian(y, m, d, origin., legacy = FALSE, verbose = TRUE) obs_julian(y, m, d, origin., legacy = FALSE, verbose = TRUE)
y |
year |
m |
month |
d |
day |
origin. |
string |
legacy |
logical, to use legacy code |
verbose |
logical, to show more messages |
Helpers Legacy
returns day since 1/1/1960
a string with the receptor id
Other helpers legacy:
obs_grid(),
obs_id2pos(),
obs_info2id(),
obs_normalize_dmass(),
obs_traj_foot()
Other helpers legacy:
obs_grid(),
obs_id2pos(),
obs_info2id(),
obs_normalize_dmass(),
obs_traj_foot()
## Not run: # Do not run obs_julian(1, 2020, 1) ## End(Not run) ## Not run: # Do not run obs_julian(y = 2002, m = 8, d = 3, legacy = TRUE) obs_julian(y = 2002, m = 8, d = 3, legacy = FALSE) ## End(Not run)## Not run: # Do not run obs_julian(1, 2020, 1) ## End(Not run) ## Not run: # Do not run obs_julian(y = 2002, m = 8, d = 3, legacy = TRUE) obs_julian(y = 2002, m = 8, d = 3, legacy = FALSE) ## End(Not run)
Some treatments for list of data.frames
obs_list.dt(ldf, na, verbose = TRUE)obs_list.dt(ldf, na, verbose = TRUE)
ldf |
list of data.frames |
na |
common names in the final data.frame |
verbose |
Logical to show more information |
long data.table
1. Filter out empty data.frames 2. identify common names 3. rbindlist and return data.table
Other helpers:
fex(),
obs_footname(),
obs_format(),
obs_freq(),
obs_out(),
obs_rbind(),
obs_read_csvy(),
obs_roundtime(),
obs_trunc(),
obs_write_csvy(),
rtorf-deprecated,
sr()
## Not run: # Do not run ## End(Not run)## Not run: # Do not run ## End(Not run)
Read obspack metadata
obs_meta( index, verbose = TRUE, n_site_code = 15, n_site_name = 15, n_site_latitude = 18, n_site_longitude = 19, n_site_country = 18, n_dataset_project = 21, n_lab = 16, n_scales = 31, n_site_elevation = 20, n_altitude_comment = 22, n_utc = 18, fill_value = -1e+34, as_list = FALSE )obs_meta( index, verbose = TRUE, n_site_code = 15, n_site_name = 15, n_site_latitude = 18, n_site_longitude = 19, n_site_country = 18, n_dataset_project = 21, n_lab = 16, n_scales = 31, n_site_elevation = 20, n_altitude_comment = 22, n_utc = 18, fill_value = -1e+34, as_list = FALSE )
index |
data.table |
verbose |
Logical to show more information |
n_site_code |
number of characters extraced from metadata after search |
n_site_name |
number of characters extraced from metadata after search |
n_site_latitude |
number of characters extraced from metadata after search |
n_site_longitude |
number of characters extraced from metadata after search |
n_site_country |
number of characters extraced from metadata after search |
n_dataset_project |
number of characters extraced from metadata after search |
n_lab |
number of characters extraced from metadata after search |
n_scales |
number of characters extraced from metadata after search |
n_site_elevation |
number of characters extraced from metadata after search |
n_altitude_comment |
number of characters extraced from metadata after search |
n_utc |
number of characters extraced from metadata after search |
fill_value |
fill value. Appeared in aoa_aircraft-flask_19_allvalid.txt |
as_list |
Logical to return as list |
A data.frame with with an index obspack.
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_meta(index) ## End(Not run)## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_meta(index) ## End(Not run)
Creates NetCDF based on dimension from another NetCDF with custom dimensions, and attributes
obs_nc( lat, lon, time_nc, vars_out = c("total", "bio", "ocn", "fossil", "fire"), units_out, nc_out, larrays, verbose = FALSE )obs_nc( lat, lon, time_nc, vars_out = c("total", "bio", "ocn", "fossil", "fire"), units_out, nc_out, larrays, verbose = FALSE )
lat |
lats array |
lon |
longs arrau |
time_nc |
Times. |
vars_out |
names for the variables to be created in the NetCDF. |
units_out |
units for the NetCDF variables to be created. NO DEFAULT. |
nc_out |
path for the created NetCDF. |
larrays |
list of arrays, length equal to vars_out. |
verbose |
Logical, to display more information. |
A NetCDF file with the custom attributes and units
# nc_path <- paste0("Z:/footprints/aircraft/flask/2018", # "/04/hysplit2018x04x08x15x15x38.7459Nx077.5584Wx00594.nc") # foot <- obs_nc_get(nc_path, all = TRUE) # nco <- paste0(tempfile(), "2.nc") # file.remove(nco) # obs_nc(lat = foot$lat, # lon = foot$lon, # time_nc = ISOdatetime(2018, 4, 8, 15, 15, 38), # units_out = "(ppb/nanomol m-2 s-1)*nanomol m-2 s-1", # vars_out = c("a", "b"), # nc_out = nco, # larrays = list(a = foot, b = foot), # verbose = TRUE)# nc_path <- paste0("Z:/footprints/aircraft/flask/2018", # "/04/hysplit2018x04x08x15x15x38.7459Nx077.5584Wx00594.nc") # foot <- obs_nc_get(nc_path, all = TRUE) # nco <- paste0(tempfile(), "2.nc") # file.remove(nco) # obs_nc(lat = foot$lat, # lon = foot$lon, # time_nc = ISOdatetime(2018, 4, 8, 15, 15, 38), # units_out = "(ppb/nanomol m-2 s-1)*nanomol m-2 s-1", # vars_out = c("a", "b"), # nc_out = nco, # larrays = list(a = foot, b = foot), # verbose = TRUE)
Reads NetCDF var and returns the spatial array
obs_nc_get( nc_path = "AAA", nc_name = "foot1", nc_lat = "foot1lat", nc_lon = "foot1lon", verbose = FALSE, all = FALSE )obs_nc_get( nc_path = "AAA", nc_name = "foot1", nc_lat = "foot1lat", nc_lon = "foot1lon", verbose = FALSE, all = FALSE )
nc_path |
String pointing to the target NetCDF |
nc_name |
String indicating the spatial array |
nc_lat |
String to extract latitude. |
nc_lon |
String to extract longitude |
verbose |
Logical, to display more information. |
all |
Logical, if TRUE, return list of array, lon and lats |
Array of convolved footprints, or lis tof convolved fluxes and lat lon
#nc_path <- paste0("Z:/footprints/aircraft/flask/2018/04", #"/hysplit2018x04x08x15x15x38.7459Nx077.5584Wx00594.nc") #f <- obs_nc_get(nc_path = nc_path)#nc_path <- paste0("Z:/footprints/aircraft/flask/2018/04", #"/hysplit2018x04x08x15x15x38.7459Nx077.5584Wx00594.nc") #f <- obs_nc_get(nc_path = nc_path)
add columns of
obs_normalize_dmass(part = NULL)obs_normalize_dmass(part = NULL)
part |
particle data.table |
return footprint
Other helpers legacy:
obs_grid(),
obs_id2pos(),
obs_info2id(),
obs_julian(),
obs_traj_foot()
## Not run: # Do not run ## End(Not run) #' # Do not run## Not run: # Do not run ## End(Not run) #' # Do not run
Just the opposite of intersect
obs_out(x, y)obs_out(x, y)
x |
vector |
y |
vector |
Helpers
vector opposite of intersect
Other helpers:
fex(),
obs_footname(),
obs_format(),
obs_freq(),
obs_list.dt(),
obs_rbind(),
obs_read_csvy(),
obs_roundtime(),
obs_trunc(),
obs_write_csvy(),
rtorf-deprecated,
sr()
## Not run: #do not run ## End(Not run)## Not run: #do not run ## End(Not run)
Read obspack metadata
obs_plot( dt, time, y = "value", yfactor = 1, colu = "site_code", type = "p", n = if (length(unique(dt[[colu]])) == 1) unique(dt[[colu]]) else unique(dt[[colu]])[1:2], pal = cptcity::find_cpt("qual")[6], verbose = TRUE, xlab = "time", ylab = "value", xlim = range(dt[[time]], na.rm = TRUE), ylim = range(dt[[y]], na.rm = TRUE), ... )obs_plot( dt, time, y = "value", yfactor = 1, colu = "site_code", type = "p", n = if (length(unique(dt[[colu]])) == 1) unique(dt[[colu]]) else unique(dt[[colu]])[1:2], pal = cptcity::find_cpt("qual")[6], verbose = TRUE, xlab = "time", ylab = "value", xlim = range(dt[[time]], na.rm = TRUE), ylim = range(dt[[y]], na.rm = TRUE), ... )
dt |
data.table |
time |
x axis column (time) |
y |
y axis column, default "value" |
yfactor |
factor of y |
colu |
column to plot by color, default site_code |
type |
type of plot, default "p" |
n |
Character indicating 'colu' to subset, for instance, if you want to plot "site_code", here include all the site_code that you want to plot |
pal |
Color palette name see cpt, default "cb_qual_Accent_08" |
verbose |
Logical to show more information |
xlab |
Character, xlab |
ylab |
Character, ylab |
xlim |
x limits |
ylim |
y limits |
... |
plot arguments |
Plot
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) obs_plot(dt, time = "time") ## End(Not run)## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) obs_plot(dt, time = "time") ## End(Not run)
return rbind obs data.tables
obs_rbind(dt1, dt2, verbose = TRUE)obs_rbind(dt1, dt2, verbose = TRUE)
dt1 |
first data.table |
dt2 |
second data.table |
verbose |
logical to show more information |
numeric vector
Other helpers:
fex(),
obs_footname(),
obs_format(),
obs_freq(),
obs_list.dt(),
obs_out(),
obs_read_csvy(),
obs_roundtime(),
obs_trunc(),
obs_write_csvy(),
rtorf-deprecated,
sr()
## Not run: # Do not run ## End(Not run)## Not run: # Do not run ## End(Not run)
Each obspack file has a header with metadata and this function reads selected fields from the metadata and add them as columns. This new columns are used later to be filtered
obs_read( index, categories = "flask", expr = NULL, verbose = TRUE, n_site_code = 15, n_site_latitude = 18, n_site_longitude = 19, n_site_name = 15, n_site_country = 18, n_dataset_project = 21, n_lab = 16, n_scales = 31, n_site_elevation = 20, n_altitude_comment = 22, n_utc = 18, fill_value = -1e+34, as_list = FALSE )obs_read( index, categories = "flask", expr = NULL, verbose = TRUE, n_site_code = 15, n_site_latitude = 18, n_site_longitude = 19, n_site_name = 15, n_site_country = 18, n_dataset_project = 21, n_lab = 16, n_scales = 31, n_site_elevation = 20, n_altitude_comment = 22, n_utc = 18, fill_value = -1e+34, as_list = FALSE )
index |
data.table |
categories |
character; ONE category : of c("aircraft-pfp", "aircraft-insitu", "surface-insitu", "tower-insitu", "aircore", "surface-pfp", "shipboard-insitu", "flask"). |
expr |
String expressions to filter data.table internally |
verbose |
Logical to show more information |
n_site_code |
number of characters extratced from metadata after search |
n_site_latitude |
number of characters extracted from metadata after search |
n_site_longitude |
number of characters extracted from metadata after search |
n_site_name |
number of characters extracted from metadata after search |
n_site_country |
number of characters extracted from metadata after search |
n_dataset_project |
number of characters extracted from metadata after search |
n_lab |
number of characters extracted from metadata after search |
n_scales |
number of characters extracted from metadata after search |
n_site_elevation |
number of characters extracted from metadata after search |
n_altitude_comment |
number of characters extracted from metadata after search |
n_utc |
number of characters extracted from metadata after search |
fill_value |
fill value. Appeared in aoa_aircraft-flask_19_allvalid.txt |
as_list |
Logical to return as list |
A data.frame with with an index obspack.
The identification of the altitude and type is critical. The approach used here consists of: 1. Identify agl from the name of the tile. 2. If magl not present, search dill_values used in elevation and transform them into NA (not available) 3. If magl is not present, agl = altitude - elevation 4. If there are some NA in elevation, will result some NA in agl 5. A new column is added named 'altitude_final' to store agl or asl 6. Another column named 'type_altitude' is added to identify "magl" or "masl" 7. If there is any case NA in 'altitude_final', 'type_altitude' is "not available"
Then, the relationship with hysplit is:
| type_altitude | hysplit | |
| magl | agl | |
| masl | asl | |
| not available | f-PBL |
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) obs_read(index, expr = "altitude_final == '5800'") ## End(Not run)## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) obs_read(index, expr = "altitude_final == '5800'") ## End(Not run)
Reads CSVY, print YAML and fread file.
obs_read_csvy(f, n = 100, ...)obs_read_csvy(f, n = 100, ...)
f |
path to csvy file |
n |
number of files to search for "—" yaml |
... |
extra data.table arguments |
a data.table from a csvy file
Other helpers:
fex(),
obs_footname(),
obs_format(),
obs_freq(),
obs_list.dt(),
obs_out(),
obs_rbind(),
obs_roundtime(),
obs_trunc(),
obs_write_csvy(),
rtorf-deprecated,
sr()
## Not run: # Do not run df <- data.frame(a = rnorm(n = 10), time = Sys.time() + 1:10) f <- paste0(tempfile(), ".csvy") notes <- c("notes", "more notes") obs_write_csvy(dt = df, notes = notes, out = f) s <- obs_read_csvy(f) s # or readLines(f) data.table::fread(f) ## End(Not run)## Not run: # Do not run df <- data.frame(a = rnorm(n = 10), time = Sys.time() + 1:10) f <- paste0(tempfile(), ".csvy") notes <- c("notes", "more notes") obs_write_csvy(dt = df, notes = notes, out = f) s <- obs_read_csvy(f) s # or readLines(f) data.table::fread(f) ## End(Not run)
Each obspack file has a header with metadata and this function reads selected fields from the metadata and add them as columns. This new columns are used later to be filtered
obs_read_nc( index, categories = "flask", att = FALSE, expr = NULL, solar_time = if (grepl("aircraft", categories)) FALSE else TRUE, as_list = FALSE, verbose = FALSE, warnings = FALSE )obs_read_nc( index, categories = "flask", att = FALSE, expr = NULL, solar_time = if (grepl("aircraft", categories)) FALSE else TRUE, as_list = FALSE, verbose = FALSE, warnings = FALSE )
index |
data.table |
categories |
character; ONE category : of c("aircraft-pfp", "aircraft-insitu", "surface-insitu", "tower-insitu", "aircore", "surface-pfp", "shipboard-insitu", "flask"). |
att |
Logical, if global attributes should be added to data.table |
expr |
String expressions to filter data.table internally |
solar_time |
Logical, add solar time? Default: if categories include aircraft, FALSE, otherwise, TRUE |
as_list |
Logical to return as list |
verbose |
Logical to show more information |
warnings |
Logical to show warnings when reading NetCDF, especially global attributes |
A data.frame with with an index obspack.
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) ## End(Not run)## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) ## End(Not run)
Each obspack file has a header with metadata and this function reads selected fields from the metadata and add them as columns. This new columns are used later to be filtered
obs_read_nc_att(index, as_list = FALSE, verbose = FALSE, warnings = FALSE)obs_read_nc_att(index, as_list = FALSE, verbose = FALSE, warnings = FALSE)
index |
data.table |
as_list |
Logical to return as list |
verbose |
Logical to show more information |
warnings |
Logical to show warnings when reading NetCDF, especially global attributes |
A data.frame with with an index obspack.
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) ## End(Not run)## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) ## End(Not run)
return rounded seconds from time
obs_roundtime(x, n = 10)obs_roundtime(x, n = 10)
x |
time as "POSIXct" "POSIXt" |
n |
factor |
numeric vector
Other helpers:
fex(),
obs_footname(),
obs_format(),
obs_freq(),
obs_list.dt(),
obs_out(),
obs_rbind(),
obs_read_csvy(),
obs_trunc(),
obs_write_csvy(),
rtorf-deprecated,
sr()
## Not run: # Do not run x <- Sys.time() + seq(1, 55, 1) paste0(x," ", obs_roundtime(x), " ", obs_freq(data.table::second(x), seq(0, 55, 10))) ## End(Not run)## Not run: # Do not run x <- Sys.time() + seq(1, 55, 1) paste0(x," ", obs_roundtime(x), " ", obs_freq(data.table::second(x), seq(0, 55, 10))) ## End(Not run)
This function select by closest time. Good for aircrafts
obs_select_sec( dt, origin = "1970-01-01", tz = "UTC", seconds = 30, verbose = TRUE )obs_select_sec( dt, origin = "1970-01-01", tz = "UTC", seconds = 30, verbose = TRUE )
dt |
data.table |
origin |
a date-time object, default "1970-01-01" |
tz |
time zone string |
seconds |
seocnds target to select and filter data |
verbose |
logical to show more information |
A filtered data.frame.
By default add column timeUTC based on the input column 'key_time'
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) dx <- obs_read(index, expr = "altitude_final == '5800'") dx <- obs_addtime(dx[, -"site_code"]) dy <- obs_select_sec(dx) ## End(Not run)## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) dx <- obs_read(index, expr = "altitude_final == '5800'") dx <- obs_addtime(dx[, -"site_code"]) dy <- obs_select_sec(dx) ## End(Not run)
This function returns a data.frame index of all files available in ObsPack.
This function returns a data.frame index of all files available in obspack.
obs_summary( obs, categories = c("aircraft-pfp", "aircraft-insitu", "surface-insitu", "tower-insitu", "aircore", "surface-pfp", "shipboard-insitu", "flask"), lnchar = 11, out = paste0(tempfile(), "_index.csv"), verbose = TRUE, aslist = FALSE ) obs_index( obs, categories = c("aircraft-pfp", "aircraft-insitu", "surface-insitu", "tower-insitu", "aircore", "surface-pfp", "shipboard-insitu", "flask"), lnchar = 11, out = paste0(tempfile(), "_index.csv"), verbose = TRUE )obs_summary( obs, categories = c("aircraft-pfp", "aircraft-insitu", "surface-insitu", "tower-insitu", "aircore", "surface-pfp", "shipboard-insitu", "flask"), lnchar = 11, out = paste0(tempfile(), "_index.csv"), verbose = TRUE, aslist = FALSE ) obs_index( obs, categories = c("aircraft-pfp", "aircraft-insitu", "surface-insitu", "tower-insitu", "aircore", "surface-pfp", "shipboard-insitu", "flask"), lnchar = 11, out = paste0(tempfile(), "_index.csv"), verbose = TRUE )
obs |
Path to the Obspack GLOBALview txt data |
categories |
character; default are c("aircraft-pfp", "aircraft-insitu", "surface-insitu", "tower-insitu", "aircore", "surface-pfp", "shipboard-insitu", "flask"). The ideia is that,as the file names include these words, this function identifies which files has these words and add them as columns. |
lnchar |
Integer; last nchar, default = 11. |
out |
Path to the Obspack index output |
verbose |
Logical to show more information |
aslist |
Logical to return list of index and summary |
A data.frame with with an index of the obspack Globalview.
A data.frame with with an index of the obspack Globalview.
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) ## End(Not run) { ## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) ## End(Not run) }## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) ## End(Not run) { ## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) ## End(Not run) }
This function reads the obsPack directory providing a summary for the columns: "value", "time", "time_decimal", "latitude" and "longitude". These summary are made by the columns "name", "sector", "site_name", "site_country", "type_altitude", "lab_1_abbr" and "site_utc2lst"
obs_table( df, cols = c("value", "time", "time_decimal", "latitude", "longitude") )obs_table( df, cols = c("value", "time", "time_decimal", "latitude", "longitude") )
df |
data.table |
cols |
String of columns to be summarized. |
A data.frame with with an index obspack.
## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) dx <- obs_table(dt) ## End(Not run)## Not run: # Do not run obs <- system.file("data-raw", package = "rtorf") index <- obs_summary(obs) dt <- obs_read(index) dx <- obs_table(dt) ## End(Not run)
return trajectory
obs_traj_foot( ident, part = NULL, timelabel = NULL, pathname = "", foottimes = 0:240, zlim = c(0, 0), coarse = 1, dmassTF = TRUE, numpix.x = 70, numpix.y = 70, lon.ll = -145, lat.ll = 11, lon.res = 1, lat.res = 1, npar = 500, eps.global = 0.1, adaptive = TRUE, center = FALSE, terminal_background = TRUE )obs_traj_foot( ident, part = NULL, timelabel = NULL, pathname = "", foottimes = 0:240, zlim = c(0, 0), coarse = 1, dmassTF = TRUE, numpix.x = 70, numpix.y = 70, lon.ll = -145, lat.ll = 11, lon.res = 1, lat.res = 1, npar = 500, eps.global = 0.1, adaptive = TRUE, center = FALSE, terminal_background = TRUE )
ident |
foot id |
part |
data.table with PARTICLE.DAT information |
timelabel |
timelabel |
pathname |
pathname |
foottimes |
vector of times between which footprint or influence will be integrated |
zlim |
zlim |
coarse |
default 1 |
dmassTF |
default TRUE weighting by accumulated mass due to violation of mass conservation in met fields |
numpix.x |
number of pixels in x directions in grid |
numpix.y |
number of pixels in y directions in grid |
lon.ll |
lower left lon |
lat.ll |
lower left lat |
lon.res |
resolution |
lat.res |
resolution |
npar |
default 500, number of particles hysplit was run with (required in order to account for those cases where a thinned particle table that may not contain all particle indices is used) |
eps.global |
default 0.01 |
adaptive |
logical, if TRUE (default), uses adaptive gridding resolution in time. |
center |
logical, if TRUE (default FALSE), lon.ll and lat.ll are treated as cell centers and adjusted to edges internally. |
terminal_background |
logical, if TRUE (default), particles entering the background area are removed from the simulation from that point onward. |
return footprint
Other helpers legacy:
obs_grid(),
obs_id2pos(),
obs_info2id(),
obs_julian(),
obs_normalize_dmass()
## Not run: # Do not run ## End(Not run)## Not run: # Do not run ## End(Not run)
Trunc numbers with a specified number of decimals.
obs_trunc(n, dec)obs_trunc(n, dec)
n |
Numeric number |
dec |
Integer, number of decimals |
truncated number
source https://stackoverflow.com/a/47015304/2418532
Other helpers:
fex(),
obs_footname(),
obs_format(),
obs_freq(),
obs_list.dt(),
obs_out(),
obs_rbind(),
obs_read_csvy(),
obs_roundtime(),
obs_write_csvy(),
rtorf-deprecated,
sr()
## Not run: # Do not run # in bash: # printf "%07.4f" 72.05785 # results in 72.0578 # but: formatC(72.05785, digits = 4, width = 8, format = "f", flag = "0") # results in "072.0579" # the goal is to obtain the same trunc number as using bash, then: formatC(obs_trunc(72.05785, 4), digits = 4, width = 8, format = "f", flag = "0") ## End(Not run)## Not run: # Do not run # in bash: # printf "%07.4f" 72.05785 # results in 72.0578 # but: formatC(72.05785, digits = 4, width = 8, format = "f", flag = "0") # results in "072.0579" # the goal is to obtain the same trunc number as using bash, then: formatC(obs_trunc(72.05785, 4), digits = 4, width = 8, format = "f", flag = "0") ## End(Not run)
Add YAML header info and writes a the data.frame into disk. The YAML section includes notes and str(dt).
obs_write_csvy( dt, notes, out = paste0(tempfile(), ".csvy"), sep = ",", nchar.max = 80, ... )obs_write_csvy( dt, notes, out = paste0(tempfile(), ".csvy"), sep = ",", nchar.max = 80, ... )
dt |
data.table |
notes |
notes. |
out |
outfile path. |
sep |
The separator between columns. Default is ",". |
nchar.max |
Max nchar for str. |
... |
extra data.table arguments |
a csvy file to disk
Other helpers:
fex(),
obs_footname(),
obs_format(),
obs_freq(),
obs_list.dt(),
obs_out(),
obs_rbind(),
obs_read_csvy(),
obs_roundtime(),
obs_trunc(),
rtorf-deprecated,
sr()
## Not run: # Do not run df <- data.frame(a = rnorm(n = 10), time = Sys.time() + 1:10) f <- paste0(tempfile(), ".csvy") notes <- c("notes", "more notes") obs_write_csvy(dt = df, notes = notes, out = f) readLines(f) data.table::fread(f, h = TRUE) ## End(Not run)## Not run: # Do not run df <- data.frame(a = rnorm(n = 10), time = Sys.time() + 1:10) f <- paste0(tempfile(), ".csvy") notes <- c("notes", "more notes") obs_write_csvy(dt = df, notes = notes, out = f) readLines(f) data.table::fread(f, h = TRUE) ## End(Not run)
file extension
sr(x, n)sr(x, n)
x |
a character vector. |
n |
integer. |
last n characters
Other helpers:
fex(),
obs_footname(),
obs_format(),
obs_freq(),
obs_list.dt(),
obs_out(),
obs_rbind(),
obs_read_csvy(),
obs_roundtime(),
obs_trunc(),
obs_write_csvy(),
rtorf-deprecated
## Not run: # do not run ## End(Not run)## Not run: # do not run ## End(Not run)