Acoustic analysis of one or more sounds: pitch tracking, basic spectral
characteristics, formants, estimated loudness (see
getLoudness), roughness (see modulationSpectrum),
novelty (see ssm), etc. The default values of arguments are
optimized for human non-linguistic vocalizations. For high-precision work,
first extract and manually correct pitch contours with
pitch_app, PRAAT, or whatever, and then run
analyze(pitchManual = ...) with these manual contours. For more
information, see https://cogsci.se/soundgen/acoustic_analysis.html
analyze(
x,
samplingRate = NULL,
scale = NULL,
from = NULL,
to = NULL,
dynamicRange = 80,
silence = 0.04,
windowLength = 50,
step = NULL,
overlap = 50,
wn = "gaussian",
zp = 0,
cutFreq = NULL,
nFormants = 3,
formants = list(),
loudness = list(SPL_measured = 70),
roughness = list(msType = "1D", specMethod = "spectrum", amRes = 1, specFun_pars =
list(windowLength = 25, step = 2)),
novelty = list(specFun = "melspec", kernelLen = 1000),
pitchMethods = c("dom", "autocor"),
pitchManual = NULL,
pitchFloor = 75,
pitchCeiling = 1000,
priorMean = 300,
priorSD = 6,
priorAdapt = TRUE,
nCands = 1,
minVoicedCands = NULL,
pitchDom = list(domThres = 0.1, domSmooth = 220),
pitchAutocor = list(autocorThres = 0.7, autocorSmooth = 7, autocorUpsample = 25,
autocorBestPeak = 0.975, interpol = "sinc"),
pitchCep = list(cepThres = 0.75, cepZp = 0),
pitchSpec = list(specThres = 0.05, specPeak = 0.25, specHNRslope = 0.8, specSmooth =
150, specMerge = 0.1, specSinglePeakCert = 0.4, specRatios = 3),
pitchHps = list(hpsNum = 5, hpsThres = 0.1, hpsNorm = 2, hpsPenalty = 2),
pitchZc = list(zcThres = 0.1, zcWin = 5),
harmHeight = list(harmThres = 3, harmTol = 0.25, harmPerSel = 5),
subh = list(method = c("cep", "pitchCands", "harm")[1], nSubh = 5, tol = 0.05, nHarm =
5, harmThres = 12, harmTol = 0.25),
flux = list(thres = 0.15),
amRange = c(10, 60),
fmRange = NULL,
shortestSyl = 20,
shortestPause = 60,
interpolPitch = list(win = 75, tol = 0.3, cert = 0.3),
certWeight = 0.5,
smooth = 1,
smoothVars = c("pitch", "dom"),
summaryFun = c("mean", "median", "sd"),
invalidArgAction = c("adjust", "abort", "ignore"),
reportEvery = NULL,
cores = 1,
plot = FALSE,
osc = c("linear", "dB", "none"),
showLegend = TRUE,
savePlots = FALSE,
embed = FALSE,
pitchPlot = list(col = rgb(0, 0, 1, 0.75), lwd = 3, showPrior = TRUE),
extraContour = NULL,
ylim = NULL,
xlab = "Time",
ylab = NULL,
main = NULL,
width = 900,
height = 500,
units = "px",
res = NA,
...
)A list with $detailed frame-by-frame descriptives and a
$summary with one row per file, as determined by summaryFun
(e.g., mean / median / SD of each acoustic variable across all STFT frames). Output measures include:
total duration, s
duration from the beginning of the first
non-silent STFT frame to the end of the last non-silent STFT frame, s
(NB: depends strongly on windowLength and silence
settings)
time of the middle of each frame (ms)
frequency (Hz), purity (0 to 1), and depth (0 to 100) of amplitude modulation estimated from a smoothed amplitude envelope
frequency (Hz) and purity (dB) of amplitude
modulation estimated via modulationSpectrum
root mean square of amplitude per frame, calculated as sqrt(mean(frame ^ 2))
same as ampl, but ignoring silent frames
Cepstral Peak Prominence, dB (a measure of pitch quality, the ratio of the highest peak in the cepstrum to the regression line drawn through it)
lowest dominant frequency band (Hz) (see "Pitch tracking methods / Dominant frequency" in the vignette)
Wiener entropy of the spectrum of the current frame (=spectral flatness). Close to 0: pure tone or tonal sound with nearly all energy in harmonics; close to 1: white noise
Normalized Shannon entropy of the spectrum of the current frame: 0 = pure tone, 1 = white noise
the frequency and bandwidth of the first nFormants formants per STFT frame, as calculated by phonTools::findformants
strength of low-frequency modulation at ~4 Hz (0.25-30
Hz), calculated from a modulation spectrum as a complement to
psychoacoustic roughness; see modulationSpectrum
feature-based flux, the rate of change in acoustic features
such as pitch, HNR, etc. (0 = none, 1 = max); "epoch" is an audio segment
between two peaks of flux that exceed a threshold of
flux = list(thres = ...) (listed in output$detailed only)
frequency of frequency modulation (FM) such as vibrato or jitter, Hz
depth of FM, semitones
purity or dominance of the main FM frequency (fmFreq), 0 to 1
the amount of energy in upper harmonics, namely the ratio of total spectral mass above 1.25 x F0 to the total spectral mass below 1.25 x F0 (dB)
how high harmonics reach in the spectrum, based on the best guess at pitch (or the manually provided pitch values)
harmonics-to-noise ratio (dB), a measure of harmonicity (see "Pitch tracking methods / Autocorrelation"). If HNR = 0 dB, there is as much energy in harmonics as in noise
subjective loudness, in sone, corresponding to the chosen
SPL_measured - see getLoudness
spectral novelty - a measure of how variable the spectrum is
on a particular time scale, as estimated by ssm
the frequency with maximum spectral power (Hz)
post-processed pitch contour based on all F0 estimates
the 25th, 50th, and 75th quantiles of the spectrum of voiced frames (Hz)
the amount of amplitude modulation in the roughness range, see
modulationSpectrum and Anikin 2025
psychoacoustic sharpness: related to spectral centroid,
but calculated from a psychoacoustic loudness model, see
getLoudness
the center of gravity of the frame's spectrum, first spectral moment (Hz)
the slope of linear regression fit to the spectrum below cutFreq (dB/kHz)
estimated depth of subharmonics per frame: 0 = none, 1 = as strong as f0. NB: this depends critically on accurate pitch tracking
the ratio of f0 to subharmonics frequency with strength subDep: 2 = period doubling, 3 = f0 / 3, etc.
is the current STFT frame voiced? TRUE / FALSE
path to a folder, one or more wav or mp3 files c('file1.wav', 'file2.mp3'), Wave object, numeric vector, or a list of Wave objects or numeric vectors
sampling rate of x (only needed if x is a
numeric vector)
maximum possible amplitude of input, used to normalize the input
vector (only needed if x is a numeric vector)
if NULL (default), analyzes the whole sound, otherwise from...to (s)
regions under -dynamicRange dB are treated as silent
(0 to 1 as proportion of max amplitude of the anayzed sound)
frames with RMS amplitude below silence * max_ampl adjusted by scale
are not analyzed at all
length of the analysis window, ms
step between successive windows, ms; if provided, overrides
overlap; because digital audio is sampled at discrete time intervals
of 1/samplingRate, the actual step and thus the time stamps of STFT frames
may be slightly different - e.g., 24.98866 instead of 25.0 ms
overlap between successive windows, %
window type accepted by winFun: character string or
function
window length after zero padding, samples. No padding is performed
if zp < analysis window length in samples. If NULL, the signal is
padded to a good number for speeding up the FFT
if specified, spectral descriptives (peakFreq, specCentroid,
specSlope, and quartiles) are calculated only between cutFreq[1] and
cutFreq[2], Hz. If a single number is given, analyzes frequencies
from 0 to cutFreq. For ex., when analyzing recordings with varying
sampling rates, set to half the lowest sampling rate to make the spectra
more comparable.
the number of formants to extract per STFT frame (0 = no formant analysis, NULL = as many as possible)
a list of arguments passed to
findformants for LPC analysis
a list of parameters passed to getLoudness for
measuring subjective loudness, namely SPL_measured, spreadSpectrum,
sharpnessMethod. NULL = skip loudness analysis
a list of parameters passed to
modulationSpectrum for measuring roughness and fluctuation
strength. NULL = skip roughness analysis
a list of parameters passed to ssm for measuring
spectral novelty. NULL = skip novelty analysis
methods of pitch estimation to consider for determining pitch contour: 'autocor' = autocorrelation (~PRAAT), 'cep' = cepstral, 'spec' = spectral (~BaNa), 'dom' = lowest dominant frequency band, 'hps' = harmonic product spectrum, 'zc' = zero crossings, NULL = no pitch analysis
manually corrected pitch contour. For a single sound,
provide a numeric vector of any length. For multiple sounds, provide a
dataframe with columns "file" and "pitch" (or path to a csv file) as
returned by pitch_app, ideally with the same windowLength and
step as in current call to analyze. A named list with pitch vectors per
file is also accepted - e.g., as returned by pitch_app
absolute bounds for pitch candidates (Hz)
specifies the mean (Hz) and standard deviation
(semitones) of gamma distribution describing our prior knowledge about the
most likely pitch values for this file. For ex., priorMean = 300,
priorSD = 6 gives a prior with mean = 300 Hz and SD = 6 semitones (half
an octave). NULL = no priors used at all; NA = no priors in the first pass,
adaptive priors in the second pass if priorAdapt = TRUE
adaptive second-pass prior: if TRUE, optimal pitch contours
are estimated first with a prior determined by priorMean,priorSD, and
then with a new prior adjusted according to this first-pass pitch contour
maximum number of pitch candidates per method, normally 1 to 4
(except for dom and hps, which return at most one candidate
per frame)
minimum number of pitch candidates that have to be
defined to consider a frame voiced (if NULL, defaults to 2 if dom is
among other candidates and 1 otherwise)
a list of control parameters for pitch tracking using the lowest dominant frequency band or "dom" method
a list of control parameters for pitch tracking using the autocorrelation or "autocor" method
a list of control parameters for pitch tracking using the cepstrum or "cep" method
a list of control parameters for pitch tracking using the BaNa or "spec" method
a list of control parameters for pitch tracking using the harmonic product spectrum or "hps" method
a list of control parameters for pitch tracking based on zero crossings in bandpass-filtered audio or "zc" method
a list of control parameters for estimating how high harmonics reach in the spectrum
a list of control parameters for estimating the strength of subharmonics per frame - that is, spectral energy at integer fractions of f0: f0/2, f0/3, etc.
a list of control parameters for calculating feature-based flux (not spectral flux)
target range of frequencies for amplitude modulation
(amFreq, Hz): a vector of length 2, defaults to c(10, 60).
Affects both amMsFreq and amEnvFreq. NB: f0 should not fall
into this range, or AM will treat glottal cycles as modulation
target range of frequencies for analyzing frequency
modulation (fmFreq, Hz): a vector of length 2, defaults to
c(5, 1000 / step / 2)
the smallest length of a voiced segment (ms) that constitutes a voiced syllable (shorter segments will be replaced by NA, as if voiceless)
the smallest gap between voiced syllables (ms): large value = interpolate and merge, small value = treat as separate syllables separated by a voiceless gap; shortestPause < step disables any gap tolerance (a single voiceless frame terminates the syllable)
a list of parameters (currently win, tol, cert) for
interpolating missing pitch candidates (NULL = no interpolation)
(0 to 1) in pitch postprocessing, specifies how much we prioritize the certainty of pitch candidates vs. pitch jumps / the internal tension of the resulting pitch curve
if smooth is a positive number, outliers of
the variables in smoothVars are adjusted with median smoothing.
smooth of 1 corresponds to a window of ~100 ms and tolerated
deviation of ~4 semitones. To disable, set smooth = 0
functions used to summarize each acoustic characteristic,
eg c('mean', 'sd'); user-defined functions are fine (see examples);
NAs are omitted automatically for mean/median/sd/min/max/range/sum,
otherwise take care of NAs yourself
what to do if an argument is invalid or outside the permitted range: 'adjust' = reset to default value, 'abort' = stop execution, 'ignore' = throw a warning and continue (may crash)
when processing multiple inputs, report estimated time
left every reportEvery iterations (NULL = default, NA = don't
report); see reportTime
number of cores for parallel processing
if TRUE, produces a spectrogram with pitch contour overlaid
"linear" = on the original scale (default); "none" = no oscillogram; "dB" = in decibels
if TRUE, adds a legend with pitch tracking methods
if TRUE, creates a subdirectory in the input directory (if input is a file or folder) or in the working directory (if input is a vector etc), named after the function (eg "spectrogram/"). All plots and audio files (if any) are saved in this new directory. If there are multiple inputs, an html notebook is also created for easy viewing and listening
if TRUE and savePlots is set and there are multiple inputs, all saved images and audio (if any) are embedded in the exported html notebook for easy sharing; if FALSE, the html file links to separate images and audio files (but separate files are still saved). NB: for this to work, package "base64enc" must be installed
a list of graphical parameters for displaying the final
pitch contour. Set to list(type = 'n') to suppress
name of an output variable to overlap on the pitch
contour plot, eg 'peakFreq' or 'loudness'; can also be a list with extra
graphical parameters, eg extraContour = list(x = 'harmHeight', col =
'red')
frequency range to plot, kHz (defaults to 0 to Nyquist frequency). NB: still in kHz, even if yScale = bark, mel, or ERB
plotting parameters
graphical parameters for saving plots passed to
png
other graphical parameters passed to spectrogram
Each pitch tracker is controlled by its own list of settings, as follows:
pitchDom (lowest dominant frequency band)domThres (0 to 1) to find the lowest dominant frequency band, we
do short-term FFT and take the lowest frequency with amplitude at least
domThres
domSmooth the width of smoothing interval (Hz) for
finding dom
pitchAutocor (autocorrelation)autocorThres voicing threshold (unitless, ~0 to 1)
autocorSmooth the width of smoothing interval (in bins) for
finding peaks in the autocorrelation function. Defaults to 7 for sampling
rate 44100 and smaller odd numbers for lower values of sampling rate
autocorUpsample upsamples acf to this resolution (Hz) to improve
accuracy in high frequencies
autocorBestPeak amplitude of the lowest best candidate relative
to the absolute max of the acf
interpol method of interpolating the ACF: "sinc" for
maximum precision, "none" for speed
pitchCep (cepstrum)cepThres voicing threshold (unitless, ~0 to 1)
cepZp zero-padding of the spectrum used for cepstral pitch
detection (final length of spectrum after zero-padding in points, e.g. 2 ^ 13)
pitchSpec (ratio of harmonics - BaNa algorithm)specThres voicing threshold (unitless, ~0 to 1)
specPeak,specHNRslope when looking for putative harmonics in the
spectrum, the threshold for peak detection is calculated as
specPeak * (1 - HNR * specHNRslope)
specSmooth the width of window for detecting peaks in the
spectrum, Hz
specMerge pitch candidates within specMerge semitones are
merged with boosted certainty
specSinglePeakCert (0 to 1) if F0 is calculated based on a single
harmonic ratio (as opposed to several ratios converging on the same
candidate), its certainty is taken to be specSinglePeakCert
specMethod "commonFactor" = highest common factor of
putative harmonics, "BaNa" = ratio of putative harmonics
specRatios for method = "commonFactor", the number of
harmonics and integer fractions to consider
pitchHps (harmonic product spectrum)hpsNum the number of times to downsample the spectrum
hpsThres voicing threshold (unitless, ~0 to 1)
hpsNorm the amount of inflation of hps pitch certainty (0 = none)
hpsPenalty the amount of penalizing hps candidates in low
frequencies (0 = none)
pitchZc (zero crossings)zcThres pitch candidates with certainty below this value are
treated as noise and set to NA (0 = nothing discarded, 1 = pitch must be
perfectly stable over zcWin)
zcWin certainty in pitch candidates depends on how stable pitch
is over zcWin glottal cycles (odd integer > 3)
Each of these lists also accepts graphical parameters that affect how pitch
candidates are plotted, eg pitchDom = list(domThres = .5, col = 'yellow').
Other arguments that are lists of subroutine-specific settings include:
harmHeight (finding how high harmonics reach in the spectrum)harmThres minimum height of spectral peak, dB
harmPerSel the number of harmonics per sliding selection
harmTol maximum tolerated deviation of peak frequency from
multiples of f0, proportion of f0
Anikin, A. (2025) Acoustic estimation of voice roughness. Attention, Perception, & Psychophysics 87: 1771–1787.
pitch_app getLoudness
segment getRMS
# Detailed documentation: https://cogsci.se/soundgen/acoustic_analysis.html
sound = soundgen(sylLen = 300, pitch = c(500, 400, 600),
noise = list(time = c(0, 300), value = c(-40, 0)),
temperature = 0.001,
addSilence = 50) # NB: always have some silence before and after!!!
# playme(sound, 16000)
a = analyze(sound, samplingRate = 16000, plot = TRUE)
str(a$detailed) # frame-by-frame
a$summary # summary per sound
if (FALSE) {
# For maximum processing speed (just basic spectral descriptives):
a = analyze(sound, samplingRate = 16000,
plot = FALSE, # no plotting
pitchMethods = NULL, # no pitch tracking
loudness = NULL, # no loudness analysis
novelty = NULL, # no novelty analysis
roughness = NULL, # no roughness analysis
nFormants = 0 # no formant analysis
)
# Fancy plotting options:
a = analyze(sound, samplingRate = 44100, plot = TRUE,
xlab = 'Time, ms', colorTheme = 'seewave', yScale = 'ERB',
contrast = .5, ylim = c(0.05, 8), main = 'My plot',
pitchMethods = c('dom', 'autocor', 'spec', 'hps', 'cep'),
priorMean = NA, # no prior info at all
pitchDom = list(col = 'red', domThres = .25),
pitchPlot = list(col = 'black', pch = 9, lty = 3, lwd = 3),
extraContour = list(x = 'peakFreq', type = 'b', pch = 4, col = 'brown'),
osc = 'dB', heights = c(2, 1))
# Analyze an entire folder in one go, saving spectrograms with pitch contours
# plus an html file for easy access
s2 = analyze('~/Downloads/temp',
savePlots = TRUE, # save the spectrograms with pitch contours
showLegend = TRUE, yScale = 'bark',
width = 20, height = 12,
units = 'cm', res = 300, ylim = c(0, 5),
cores = 4) # use multiple cores to speed up processing
s2$summary[, 1:5]
# Analyzing ultrasounds (slow but possible, just adjust pitchCeiling)
s = soundgen(sylLen = 100, addSilence = 10,
pitch = c(25000, 35000, 30000),
formants = NA, rolloff = -12, rolloffKHz = 0,
pitchSamplingRate = 350000, samplingRate = 350000, windowLength = 5,
pitchCeiling = 45000, invalidArgAction = 'ignore',
plot = TRUE)
# s is a bat-like ultrasound inaudible to humans
a = analyze(
s, 350000, plot = TRUE,
pitchFloor = 10000, pitchCeiling = 90000, priorMean = NA,
pitchMethods = c('autocor', 'spec'),
# probably shouldn't use pitchMethods = "dom" b/c of likely low-freq noise
windowLength = 5, step = 2.5,
shortestSyl = 10, shortestPause = 10, # again, very short sounds
interpolPitch = list(win = 10), # again, very short sounds
smooth = 0.1, # might need less smoothing if very rapid f0 changes
nFormants = 0, loudness = NULL, roughness = NULL, novelty = NULL)
# NB: ignore formants and loudness estimates for such non-human sounds
}
Run the code above in your browser using DataLab