grep
Pattern Matching and Replacement
grep
, grepl
, regexpr
, gregexpr
and
regexec
search for matches to argument pattern
within
each element of a character vector: they differ in the format of and
amount of detail in the results.
sub
and gsub
perform replacement of the first and all
matches respectively.
Usage
grep(pattern, x, ignore.case = FALSE, perl = FALSE, value = FALSE,
fixed = FALSE, useBytes = FALSE, invert = FALSE)grepl(pattern, x, ignore.case = FALSE, perl = FALSE,
fixed = FALSE, useBytes = FALSE)
sub(pattern, replacement, x, ignore.case = FALSE, perl = FALSE,
fixed = FALSE, useBytes = FALSE)
gsub(pattern, replacement, x, ignore.case = FALSE, perl = FALSE,
fixed = FALSE, useBytes = FALSE)
regexpr(pattern, text, ignore.case = FALSE, perl = FALSE,
fixed = FALSE, useBytes = FALSE)
gregexpr(pattern, text, ignore.case = FALSE, perl = FALSE,
fixed = FALSE, useBytes = FALSE)
regexec(pattern, text, ignore.case = FALSE, perl = FALSE,
fixed = FALSE, useBytes = FALSE)
Arguments
- pattern
character string containing a regular expression (or character string for
fixed = TRUE
) to be matched in the given character vector. Coerced byas.character
to a character string if possible. If a character vector of length 2 or more is supplied, the first element is used with a warning. Missing values are allowed except forregexpr
andgregexpr
.- x, text
a character vector where matches are sought, or an object which can be coerced by
as.character
to a character vector. Long vectors are supported.- ignore.case
if
FALSE
, the pattern matching is case sensitive and ifTRUE
, case is ignored during matching.- perl
logical. Should Perl-compatible regexps be used?
- value
if
FALSE
, a vector containing the (integer
) indices of the matches determined bygrep
is returned, and ifTRUE
, a vector containing the matching elements themselves is returned.- fixed
logical. If
TRUE
,pattern
is a string to be matched as is. Overrides all conflicting arguments.- useBytes
logical. If
TRUE
the matching is done byte-by-byte rather than character-by-character. See ‘Details’.- invert
logical. If
TRUE
return indices or values for elements that do not match.- replacement
a replacement for matched pattern in
sub
andgsub
. Coerced to character if possible. Forfixed = FALSE
this can include backreferences"\1"
to"\9"
to parenthesized subexpressions ofpattern
. Forperl = TRUE
only, it can also contain"\U"
or"\L"
to convert the rest of the replacement to upper or lower case and"\E"
to end case conversion. If a character vector of length 2 or more is supplied, the first element is used with a warning. IfNA
, all elements in the result corresponding to matches will be set toNA
.
Details
Arguments which should be character strings or character vectors are coerced to character if possible.
Each of these functions operates in one of three modes:
fixed = TRUE
: use exact matching.perl = TRUE
: use Perl-style regular expressions.fixed = FALSE, perl = FALSE
: use POSIX 1003.2 extended regular expressions (the default).
See the help pages on regular expression for details of the different types of regular expressions.
The two *sub
functions differ only in that sub
replaces
only the first occurrence of a pattern
whereas gsub
replaces all occurrences. If replacement
contains
backreferences which are not defined in pattern
the result is
undefined (but most often the backreference is taken to be ""
).
For regexpr
, gregexpr
and regexec
it is an error
for pattern
to be NA
, otherwise NA
is permitted
and gives an NA
match.
The main effect of useBytes
is to avoid errors/warnings about
invalid inputs and spurious matches in multibyte locales, but for
regexpr
it changes the interpretation of the output.
It inhibits the conversion of inputs with marked encodings, and is
forced if any input is found which is marked as "bytes"
(see Encoding
).
Caseless matching does not make much sense for bytes in a multibyte
locale, and you should expect it only to work for ASCII characters if
useBytes = TRUE
.
regexpr
and gregexpr
with perl = TRUE
allow
Python-style named captures, but not for long vector inputs.
Invalid inputs in the current locale are warned about up to 5 times.
Caseless matching with perl = TRUE
for non-ASCII characters
depends on the PCRE library being compiled with ‘Unicode
property support’.
Value
grep(value = FALSE)
returns a vector of the indices
of the elements of x
that yielded a match (or not, for
invert = TRUE
). This will be an integer vector unless the input
is a long vector, when it will be a double vector.
grep(value = TRUE)
returns a character vector containing the
selected elements of x
(after coercion, preserving names but no
other attributes).
grepl
returns a logical vector (match or not for each element of
x
).
sub
and gsub
return a character vector of the same
length and with the same attributes as x
(after possible
coercion to character). Elements of character vectors x
which
are not substituted will be returned unchanged (including any declared
encoding). If useBytes = FALSE
a non-ASCII substituted result
will often be in UTF-8 with a marked encoding (e.g., if there is a
UTF-8 input, and in a multibyte locale unless fixed = TRUE
).
Such strings can be re-encoded by enc2native
.
regexpr
returns an integer vector of the same length as
text
giving the starting position of the first match or
\(-1\) if there is none, with attribute "match.length"
, an
integer vector giving the length of the matched text (or \(-1\) for
no match). The match positions and lengths are in characters unless
useBytes = TRUE
is used, when they are in bytes (as they are
for an ASCII-only matching: in either case an attribute
useBytes
with value TRUE
is set on the result). If
named capture is used there are further attributes
"capture.start"
, "capture.length"
and
"capture.names"
.
gregexpr
returns a list of the same length as text
each
element of which is of the same form as the return value for
regexpr
, except that the starting positions of every (disjoint)
match are given.
regexec
returns a list of the same length as text
each
element of which is either \(-1\) if there is no match, or a
sequence of integers with the starting positions of the match and all
substrings corresponding to parenthesized subexpressions of
pattern
, with attribute "match.length"
a vector
giving the lengths of the matches (or \(-1\) for no match). The
interpretation of positions and length and the attributes follows
regexpr
.
Where matching failed because of resource limits (especially for PCRE) this is regarded as a non-match, usually with a warning.
Warning
The POSIX 1003.2 mode of gsub
and gregexpr
does not
work correctly with repeated word-boundaries (e.g.,
pattern = "\b"
).
Use perl = TRUE
for such matches (but that may not
work as expected with non-ASCII inputs, as the meaning of
‘word’ is system-dependent).
Performance considerations
If you are doing a lot of regular expression matching, including on
very long strings, you will want to consider the options used.
Generally PCRE will be faster than the default regular expression
engine, and fixed = TRUE
faster still (especially when each
pattern is matched only a few times).
If you are working in a single-byte locale and have marked UTF-8 strings that are representable in that locale, convert them first as just one UTF-8 string will force all the matching to be done in Unicode, which attracts a penalty of around \(3\times{}\) for the default POSIX 1003.2 mode.
If you can make use of useBytes = TRUE
, the strings will not be
checked before matching, and the actual matching will be faster.
Often byte-based matching suffices in a UTF-8 locale since byte
patterns of one character never match part of another.
PCRE-based matching by default puts additional effort into
‘studying’ the compiled pattern when x
/text
has
length at least 10. As from R 3.4.0 that study may use the PCRE JIT
compiler on platforms where it is available (see
pcre_config
). The details are controlled by
options
PCRE_study
and PCRE_use_JIT
.
(Some timing comparisons can be seen by running file
tests/PCRE.R
in the R sources (and perhaps installed).)
People working with PCRE and very long strings can adjust the maximum
size of the JIT stack by setting environment variable
R_PCRE_JIT_STACK_MAXSIZE
before JIT is used to a value between
1
and 1000
in MB: the default is 64
. (Then it
would usually be wise to set the option
PCRE_limit_recursion
.)
References
Becker, R. A., Chambers, J. M. and Wilks, A. R. (1988)
The New S Language.
Wadsworth & Brooks/Cole (grep
)
See Also
regular expression (aka regexp
) for the details
of the pattern specification.
regmatches
for extracting matched substrings based on
the results of regexpr
, gregexpr
and regexec
.
glob2rx
to turn wildcard matches into regular expressions.
agrep
for approximate matching.
charmatch
, pmatch
for partial matching,
match
for matching to whole strings,
startsWith
for matching of initial parts of strings.
tolower
, toupper
and chartr
for character translations.
apropos
uses regexps and has more examples.
grepRaw
for matching raw vectors.
Options PCRE_limit_recursion
, PCRE_study
and
PCRE_use_JIT
.
extSoftVersion
for the versions of regex and PCRE
libraries in use, pcre_config
for more details for
PCRE.
Examples
library(base)
# NOT RUN {
grep("[a-z]", letters)
txt <- c("arm","foot","lefroo", "bafoobar")
if(length(i <- grep("foo", txt)))
cat("'foo' appears at least once in\n\t", txt, "\n")
i # 2 and 4
txt[i]
## Double all 'a' or 'b's; "\" must be escaped, i.e., 'doubled'
gsub("([ab])", "\\1_\\1_", "abc and ABC")
txt <- c("The", "licenses", "for", "most", "software", "are",
"designed", "to", "take", "away", "your", "freedom",
"to", "share", "and", "change", "it.",
"", "By", "contrast,", "the", "GNU", "General", "Public", "License",
"is", "intended", "to", "guarantee", "your", "freedom", "to",
"share", "and", "change", "free", "software", "--",
"to", "make", "sure", "the", "software", "is",
"free", "for", "all", "its", "users")
( i <- grep("[gu]", txt) ) # indices
stopifnot( txt[i] == grep("[gu]", txt, value = TRUE) )
## Note that in locales such as en_US this includes B as the
## collation order is aAbBcCdDe ...
(ot <- sub("[b-e]",".", txt))
txt[ot != gsub("[b-e]",".", txt)]#- gsub does "global" substitution
txt[gsub("g","#", txt) !=
gsub("g","#", txt, ignore.case = TRUE)] # the "G" words
regexpr("en", txt)
gregexpr("e", txt)
## Using grepl() for filtering
## Find functions with argument names matching "warn":
findArgs <- function(env, pattern) {
nms <- ls(envir = as.environment(env))
nms <- nms[is.na(match(nms, c("F","T")))] # <-- work around "checking hack"
aa <- sapply(nms, function(.) { o <- get(.)
if(is.function(o)) names(formals(o)) })
iw <- sapply(aa, function(a) any(grepl(pattern, a, ignore.case=TRUE)))
aa[iw]
}
findArgs("package:base", "warn")
## trim trailing white space
str <- "Now is the time "
sub(" +$", "", str) ## spaces only
## what is considered 'white space' depends on the locale.
sub("[[:space:]]+$", "", str) ## white space, POSIX-style
## what PCRE considered white space changed in version 8.34: see ?regex
sub("\\s+$", "", str, perl = TRUE) ## PCRE-style white space
## capitalizing
txt <- "a test of capitalizing"
gsub("(\\w)(\\w*)", "\\U\\1\\L\\2", txt, perl=TRUE)
gsub("\\b(\\w)", "\\U\\1", txt, perl=TRUE)
txt2 <- "useRs may fly into JFK or laGuardia"
gsub("(\\w)(\\w*)(\\w)", "\\U\\1\\E\\2\\U\\3", txt2, perl=TRUE)
sub("(\\w)(\\w*)(\\w)", "\\U\\1\\E\\2\\U\\3", txt2, perl=TRUE)
## named capture
notables <- c(" Ben Franklin and Jefferson Davis",
"\tMillard Fillmore")
# name groups 'first' and 'last'
name.rex <- "(?<first>[[:upper:]][[:lower:]]+) (?<last>[[:upper:]][[:lower:]]+)"
(parsed <- regexpr(name.rex, notables, perl = TRUE))
gregexpr(name.rex, notables, perl = TRUE)[[2]]
parse.one <- function(res, result) {
m <- do.call(rbind, lapply(seq_along(res), function(i) {
if(result[i] == -1) return("")
st <- attr(result, "capture.start")[i, ]
substring(res[i], st, st + attr(result, "capture.length")[i, ] - 1)
}))
colnames(m) <- attr(result, "capture.names")
m
}
parse.one(notables, parsed)
## Decompose a URL into its components.
## Example by LT (http://www.cs.uiowa.edu/~luke/R/regexp.html).
x <- "http://stat.umn.edu:80/xyz"
m <- regexec("^(([^:]+)://)?([^:/]+)(:([0-9]+))?(/.*)", x)
m
regmatches(x, m)
## Element 3 is the protocol, 4 is the host, 6 is the port, and 7
## is the path. We can use this to make a function for extracting the
## parts of a URL:
URL_parts <- function(x) {
m <- regexec("^(([^:]+)://)?([^:/]+)(:([0-9]+))?(/.*)", x)
parts <- do.call(rbind,
lapply(regmatches(x, m), `[`, c(3L, 4L, 6L, 7L)))
colnames(parts) <- c("protocol","host","port","path")
parts
}
URL_parts(x)
## There is no gregexec() yet, but one can emulate it by running
## regexec() on the regmatches obtained via gregexpr(). E.g.:
pattern <- "([[:alpha:]]+)([[:digit:]]+)"
s <- "Test: A1 BC23 DEF456"
lapply(regmatches(s, gregexpr(pattern, s)),
function(e) regmatches(e, regexec(pattern, e)))
# }
Community examples
[Exercise file for LinkedIn Learning:](https://linkedin-learning.pxf.io/rweekly_grep) ```r haystack <- c("red", "blue", "green", "blue", "green forest") grep("green", haystack) grep("r", haystack) # returns position grep("r", haystack, value = TRUE) # returns value grepl("r", haystack) # returns boolean sub("e", "+", haystack) # replaces pattern with replacement (once) gsub("e", "+", haystack) # replaces pattern with replacement (global) ```