stri_enc_detect(str, filter_angle_brackets = FALSE)raw vectorsstr.
Each list element is a list with the following three named
vectors representing all guesses: Encoding-- string; guessed encodings;NAon
failure,Language-- string; guessed
languages;NAif the language could not be
determined (e.g. in case of UTF-8),Confidence-- numeric in [0,1]; the higher the value, the more
confidence there is in the match;NAon failure.str and
filter_angle_brackets.This is, at best, an imprecise operation using statistics and heuristics. Because of this, detection works best if you supply at least a few hundred bytes of character data that's mostly in a single language. However, Because the detection only looks at a limited amount of the input byte data, some of the returned charsets may fail to handle the all of input data. Note that in some cases, the language can be determined along with the encoding.
Several different techniques are used for character set detection. For multi-byte encodings, the sequence of bytes is checked for legal patterns. The detected characters are also check against a list of frequently used characters in that encoding. For single byte encodings, the data is checked against a list of the most commonly occurring three letter groups for each language that can be written using that encoding.
The detection process can be configured to optionally ignore HTML or XML style markup (using ICU's internal facilities), which can interfere with the detection process by changing the statistics.
This function should most often be used for byte-marked
input strings, especially after loading them from text
files and before the main conversion with
stri_encode. The input encoding is of course
not taken into account here, even if marked.
The following table shows all the encodings that can be detected:
If you have some initial guess on language and encoding,
try with stri_enc_detect2.
stri_enc_detect2;
stri_enc_isascii;
stri_enc_isutf16be,
stri_enc_isutf16le,
stri_enc_isutf32be,
stri_enc_isutf32le;
stri_enc_isutf8;
stringi-encodingf <- rawToChar(readBin("test.txt", "raw", 1024))
stri_enc_detect(f)Run the code above in your browser using DataLab