oce (version 0.8-9)

read.adv: Read an ADV data file

Description

Read an ADV data file, producing an object of type adv.

Usage

read.adv(file, from=1, to, by=1, tz=getOption("oceTz"),
  type=c("nortek", "sontek", "sontek.adr", "sontek.text"),
  header=TRUE,
  latitude=NA, longitude=NA,
  start, deltat,
  debug=getOption("oceDebug"), monitor=FALSE, processingLog)
read.adv.nortek(file, from=1, to, by=1, tz=getOption("oceTz"), 
  header=TRUE,
  latitude=NA, longitude=NA,
  haveAnalog1=FALSE, haveAnalog2=FALSE,
  debug=getOption("oceDebug"), monitor=FALSE, processingLog)
read.adv.sontek.serial(file, from=1, to, by=1, tz=getOption("oceTz"),
  latitude=NA, longitude=NA,
  start, deltat,
  debug=getOption("oceDebug"), monitor=FALSE, processingLog)
read.adv.sontek.adr(file, from=1, to, by=1, tz=getOption("oceTz"),
  header=TRUE, 
  latitude=NA, longitude=NA,
  debug=getOption("oceDebug"), monitor=FALSE, processingLog)
read.adv.sontek.text(basefile, from=1, to, by=1, tz=getOption("oceTz"),
  originalCoordinate="xyz",
  transformationMatrix,
  latitude=NA, longitude=NA,
  debug=getOption("oceDebug"), monitor=FALSE, processingLog)

Arguments

file

a connection or a character string giving the name of the file to load. It is also possible to give file as a vector of filenames, to handle the case of data split into files by a data logger. In the multi-file case, header

basefile

character string naming the base of filenames to load (used only by read.adv.sontek.text). The actual filenames are constructed by appending ".hd1" and ".ts1" to the base name.

from

index number of the first profile to be read, or the time of that profile, as created with as.POSIXct (hint: use tz="UTC"). This argument is ignored if header==FALSE. S

haveAnalog1

optional argument, only for Nortek devices: indicates whether to extract the ``analog1'' (8-bit) data.

haveAnalog2

optional argument, only for Nortek devices: indicates whether to extract the ``analog2'' (16-bit) data.

to

indication of the last profile to read, in a format matching that of from. This is ignored if header==FALSE.

by

an indication of the stride length to use while walking through the file. This is ignored if header==FALSE. Otherwise, if this is an integer, then by-1 profiles are skipped between each pair of profiles that is read.

header

a boolean indicating whether the file contains a header at the start. (This will not be the case for files that are created by data loggers that chop the raw data up into a series of sub-files, e.g. once per hour.)

latitude

optional signed number indicating the latitude in degrees North.

longitude

optional signed number indicating the longitude in degrees East.

start

the time of the first sample, typically created with as.POSIXct. This is mandatory if header is FALSE. This may be a vector of times, if filename is a vec

deltat

the time between samples. (This is mandatory if header=FALSE.)

originalCoordinate

character string indicating coordinate system, one of "beam", "xyz", "enu" or "other". (This is needed for the case of multiple files that were created by a data logger, because the header infor

transformationMatrix

transformation matrix to use in converting beam coordinates to xyz coordinates. This will over-ride the matrix in the file header, if there is one. An example is rbind(c(2.710, -1.409, -1.299), c(0.071, 2.372, -2.442), c(0.344, 0.344, 0.34

type

character string indicating type of file, and used by read.adv to dispatch to one of the speciality functions.

tz

character string indicating time zone to be assumed in the data.

debug

a flag that turns on debugging. The value indicates the depth within the call stack to which debugging applies. For example, read.adv.nortek() calls read.header.nortek(), so that read.adv.nortek(...,debug=2)

monitor

boolean, set to TRUE to provide an indication of every data burst read.

processingLog

if provided, the action item to be stored in the log. This parameter is typically only provided for internal calls; the default that it provides is better for normal calls by a user.

Value

• An object of class "adv", which contains measurements made with an ADV device. This is a list containing lists named metadata, data, and processingLog. The metadata contains information as given in the following table. The ``Nortek name'' is the name used in the Nortek System Integrator Guide [reference 1] and the ``Sontek name'' is the name used in the relevant Sontek documentation. References are given in square brackets.

  llll{ metadata name Nortek name Sontek name Meaning manufacturer - - Either "nortek" or "sontek" instrumentType - - Either "vector" or "adv" filename - - Name of data file(s) latitude - - Latitude of mooring (if applicable) longitude - - Longitude of mooring (if applicable) numberOfSamples - - Number of data samples in file numberOfBeams NBeams [1 p18] - Number of beams (always 3) numberOfBeamSequencesPerBurst NPings - number of beam sequences per burst measurementInterval MeasInterval [1 p31] - samplingRate 512/(AvgInterval) [1 p30; 4] - }

  The data list contains items with names corresponding to adp objects, with an exception for Nortek data. Nortek instruments report some things at a time interval that is longer than the velocity sampling, and these are stored in data as timeSlow, headingSlow, pitchSlow, rollSlow, and temperatureSlow; if burst sampling was used, there will also be items recordsBurst and timeBurst.

  The processingLog is in the standard format.

Special considerations for NorTek files

The data format is inferred from the System Integrator Guide [1]. This document lacks clarity in spots, and so read.adv.nortek contains some assumptions that are noted here, so that users will be aware of possible problems.

A prominent example is the specification of the sampling rate, stored in metadata$sampingRate in the return value. Repeated examination of the System Integrator Guide [1] failed to indicate where this value is stored in the various headers contained in Vector datasets. After some experimentation with a few data files, read.adv.nortek was set up to calculate metadata$samplingRate as 512/AvgInterval where AvgInterval is a part of the ``User Configuration'' header [1 p30], where the explanation is ``average interval in seconds''). This formula was developed through trial and error, but it was confirmed later on the Nortek discussion group, and it should appear in upcoming versions of [1].

Another basic issue is the determination of whether an instrument had recorded in continuous mode or burst mode. One might infer that TimCtrlReg in the ``User Configuration'' header [1 p30] determines this, in bits 1 and 2. However, this was the case in test files available to the author. For this reason, read.adv.nortek infers the mode by reverse engineering of data files of known configuration. The present version of read.adv.nortek determines the sampling mode from the ``NRecords'' item of the ``Vector Velocity Data'' header, which seems to be 0 for data collected continuously, and non-zero for data collected in bursts.

Taking these things together, we come upon the issue of how to infer sampling times for Nortek instruments. There do not seem to be definitive documents on this, and so read.adv.nortek is based partly on information (of unknown quality) found on Nortek discusson boards. The present version of read.adv.nortek infers the times of velocity observations differently, depending on whether the instrument was set to record in burst mode or continuous mode. For burst mode, times stated in the burst headers are used, but for continous mode, times stated in the ``vector system data'' are used. On the advice found on a Nortek discussion board, the burst-mode times are offset by 2 seconds to allow for the instrument warm-up period.

Special considerations for Sontek files

The binary format is inferred from Appendix 2.2.3 of the Sontek ADV operation Manual [3], with the following exceptions and notes.

1. The documentation says sampling rate is in units of 0.1Hz, but a test file indicates that it is in 0.01 Hz.
2. Bursts are recognized by byte sequences [2 p95]. In each case, a signalling byte is to be followed by a certain number of bytes, and so this code checks for two-byte sequences. The are as follows:
   • c(0x81,0x12)for an ADV with no optional sensors installed.
   • c(0x83,0x18)if a compass/tilt sensor is installed, but no temperature or pressure sensors.
   • c(0x85,0x16)if temperature and/or pressure sensors are installed, but no compass/tilt sensor.
   • c(0x87,0x1c)if a compass/tilt sensor is installed in addition to temperature and/or pressure sensors.
   Bug:only the second-last of these is handled in the present version of the package.

Bugs

The by argument only has an effect on quickly-varying variables, such as the fast timescale, velocity, backscatter, and amplitude It has no effect on slowly-varying variables, such as heading, temperature, etc. And, for the Nortek case, it also has no effect on the burst information. The reason for all of this is that it is not altogether clear what by should mean, for those variables. Indeed, this is an argument for deleting by from the argument list, and this may be done in a future version of read.adv.

Details

Reads a binary-format ADV file. This is straightforward for files with headers, since the headers contain all the information required for further processing.

Files without headers may be created in experiments in which a data logger was set up to monitor the serial data stream from an instrument. The lack of header information places a burden on the user, who must supply such basic information as the times of observations, the instrument orientation, the instrument coordinate system, etc. Example 3 below shows how to deal with such files. Three things should be noted.

1. The use must choose the appropriateread.advvariant corresponding to the instrument in question. (This is necessary becausemagic, which is used by the genericread.oceroutine, cannot determine the type of instrument by examining a file that lacks a header.)
2. The call to thereadfunction must include a start time (start) and the number of seconds between data (deltat), again, because the instrument data stream may lack those things when the device is set to a serial mode. Also, of course, it is necessary to setheader=FALSEin the function call.
3. Once the file has been read in, the user will be obliged to specify other information, for the object to be well-formed. For example, thereadfunction will have no way of knowing the instrument orientation, the coordinate system being used, the transformation matrix to go from"beam"to"xyz"coordinates, or the instrument heading, pitch, and roll, to go from"xyz"coordinates to"enu"coordinates. Such things are illustrated in example 3 below.

In ADV data files, velocities are coded to signed 2-byte integers, with a scale factor being used to convert to velocity in metres per second. These two facts control the maximum recordable velocity and the velocity resolution, values that may be retrieved for an ADV object name d with d[["velocityMaximum"]] and d[["velocityResolution"]].

References

1. Nortek AS. System Integrator Guide (paradopp family of products). June 2008. (Doc No: PSI00-0101-0608). (Users may find it helpful to also examine newer versions of the guide.)

2. SonTek/YSI ADVField/Hydra Acoustic Doppler Velocimeter (Field) Technical Documentation (Sept 1, 2001).

3. Appendix 2.2.3 of the Sontek ADV operation Manual, Firmware Version 4.0 (Oct 1997).

4. Nortek Knowledge Center http://www.nortekusa.com/en/knowledge-center

See Also

The documentation for adv-class explains the structure of ADV objects, and also outlines the other functions dealing with them.

Examples

Run this code

library(oce)
# A nortek Vector file
d <- read.oce("/data/archive/sleiwex/2008/moorings/m05/adv/nortek_1943/raw/adv_nortek_1943.vec",
              from=as.POSIXct("2008-06-26 00:00:00", tz="UTC"),
              to=as.POSIXct("2008-06-26 00:00:10", tz="UTC"))
plot(d, which=c(1:3,15))

Run the code above in your browser using DataLab