Lrexlib provides bindings of three regular expression library APIs (POSIX, PCRE and Oniguruma) to Lua 5.1.
Lrexlib builds into shared libraries called by default rex_posix.so, rex_pcre.so and rex_onig.so, which can be used with require.
Lrexlib is copyright Reuben Thomas 2000-2008 and copyright Shmuel Zeigerman 2004-2008, and is released under the MIT license.
Most functions and methods in Lrexlib have mandatory and optional arguments. There are no dependencies between arguments in Lrexlib's functions and methods. Any optional argument can be supplied as nil (or omitted if it is a trailing argument), the library will then use the default value for that argument.
This document uses the following syntax for optional arguments: they are bracketed separately, and commas are left outside brackets, e.g.:
MyFunc (arg1, arg2, [arg3], [arg4])
Throughout this document (unless it causes ambiguity), the identifier rex is used in place of either rex_posix, rex_pcre or rex_onig, that are the default namespaces for the corresponding libraries.
All functions that take a regular expression pattern as an argument will generate an error if that pattern is found invalid by the used POSIX / PCRE / Oniguruma library.
All functions that take a string-type regex argument accept a compiled regex too. In this case, the cf, locale and syntax arguments are ignored (should be either supplied as nils or omitted).
The default value for compilation flags (cf) that Lrexlib uses when the parameter is not supplied or nil is:
- REG_EXTENDED for POSIX regex library
- 0 for PCRE
- ONIG_OPTION_NONE for Oniguruma
PCRE, Oniguruma: cf may also be supplied as a string, whose characters stand for compilation flags. Combinations of the following characters (case sensitive) are supported:
Character
PCRE flag
Oniguruma flag
i
PCRE_CASELESS
ONIG_OPTION_IGNORECASE
m
PCRE_MULTILINE
ONIG_OPTION_NEGATE_SINGLELINE
s
PCRE_DOTALL
ONIG_OPTION_MULTILINE
x
PCRE_EXTENDED
ONIG_OPTION_EXTEND
U
PCRE_UNGREEDY
n/a
X
PCRE_EXTRA
n/a
The default value for execution flags (ef) that Lrexlib uses when the parameter is not supplied or nil, is:
- 0 for standard POSIX regex library
- REG_STARTEND for those POSIX regex libraries that support it, e.g. Spencer's.
- 0 for PCRE
- 0 for Oniguruma
PCRE: parameter locale (lo) can be either a string (e.g., "French_France.1252"), or a userdata obtained from a call to maketables. The default value, used when the parameter is not supplied or nil, is the built-in PCRE set of character tables.
Oniguruma: this parameter (which actually should be named "encoding" rather than "locale") must be one of the predefined strings that are formed from the ONIG_ENCODING_xxx identifiers defined in oniguruma.h, by means of omitting the ONIG_ENCODING_ part. For example, ONIG_ENCODING_UTF8 becomes "UTF8" on the Lua side. The default value, used when the parameter is not supplied or nil, is "ASCII".
If the caller-supplied value of this parameter is not one of the predefined "encoding" string set, an error is raised.
Oniguruma: parameter syntax (syn) must be one of the predefined strings that are formed from the ONIG_SYNTAX_xxx identifiers defined in oniguruma.h, by means of omitting the ONIG_SYNTAX_ part. For example, ONIG_SYNTAX_JAVA becomes "JAVA" on the Lua side. The default value, used when the parameter is not supplied or nil, is either "RUBY" (at the start-up), or the value set by the last setdefaultsyntax call.
If the caller-supplied value of syntax parameter is not one of the predefined "syntax" string set, an error is raised.
rex.match (subj, patt, [init], [cf], [ef], [lo], [syn])
or
r:match (subj, [init], [ef])
The function searches for the first match of the regexp patt in the string subj, starting from offset init, subject to flags cf and ef.
Parameter Description Type Default Value r regex object produced by new userdata n/a subj subject string n/a patt regular expression pattern string or userdata n/a [init] start offset in the subject (can be negative) number 1 [cf] compilation flags (bitwise OR) number cf [ef] execution flags (bitwise OR) number ef [lo] [PCRE, Oniguruma] locale string or userdata locale [syn] [Oniguruma] syntax string syntax
rex.find (subj, patt, [init], [cf], [ef], [lo], [syn])
or
r:find (subj, [init], [ef])
The function searches for the first match of the regexp patt in the string subj, starting from offset init, subject to flags cf and ef.
Parameter Description Type Default Value r regex object produced by new userdata n/a subj subject string n/a patt regular expression pattern string or userdata n/a [init] start offset in the subject (can be negative) number 1 [cf] compilation flags (bitwise OR) number cf [ef] execution flags (bitwise OR) number ef [lo] [PCRE, Oniguruma] locale string or userdata locale [syn] [Oniguruma] syntax string syntax
rex.gmatch (subj, patt, [cf], [ef], [lo], [syn])
The function is intended for use in the generic for Lua construct. It returns an iterator for repeated matching of the pattern patt in the string subj, subject to flags cf and ef.
Parameter Description Type Default Value subj subject string n/a patt regular expression pattern string or userdata n/a [cf] compilation flags (bitwise OR) number cf [ef] execution flags (bitwise OR) number ef [lo] [PCRE, Oniguruma] locale string or userdata locale [syn] [Oniguruma] syntax string syntax
The iterator function is called by Lua. On every iteration (that is, on every match), it returns all captures in the order they appear in the pattern (or the entire match if the pattern specified no captures). The iteration will continue till the subject fails to match.
rex.gsub (subj, patt, repl, [n], [cf], [ef], [lo], [syn])
This function searches for all matches of the pattern patt in the string subj and replaces them according to the parameters repl and n (see details below).
Parameter Description Type Default Value subj subject string n/a patt regular expression pattern string or userdata n/a repl substitution source string, function or table n/a [n] maximum number of matches to search for, or control function, or nil number or function nil [cf] compilation flags (bitwise OR) number cf [ef] execution flags (bitwise OR) number ef [lo] [PCRE, Oniguruma] locale string or userdata locale [syn] [Oniguruma] syntax string syntax
The parameter repl can be either a string, a function or a table. On each match made, it is converted into a value repl_out that may be used for the replacement.
repl_out is generated differently depending on the type of repl:
- if X represents a digit, then each %X occurence is substituted by the value of the X-th submatch (capture), with the following cases handled specially:
- each %0 is substituted by the entire match
- if the pattern contains no captures, then each %1 is substituted by the entire match
- any other %X where X is greater than the number of captures in the pattern will generate an error ("invalid capture index")
- if the pattern does contain a capture with number X but that capture didn't participate in the match, then %X is substituted by an empty string
- if X is any non-digit character then %X is substituted by X
All parts of repl other than %X are copied to repl_out verbatim.
- if it is a string or a number (coerced to a string), then the replacement value is that string;
- if it is a nil or a false, then no replacement is to be done;
Note: Under some circumstances, the value of repl_out may be ignored; see below.
gsub behaves differently depending on the type of n:
If n is a function, then it is called on each match, after repl_out is produced (so if repl is a function, it will be called prior to the n call).
n receives 3 arguments and returns 2 values. Its arguments are:
- The start offset of the match (a number)
- The end offset of the match (a number)
- repl_out
The type of its first return controls the replacement produced by gsub for the current match:
- true -- replace/don't replace, according to repl_out;
- nil/false -- don't replace;
- a string (or a number coerced to a string) -- replace by that string;
The type of its second return controls gsub behavior after the current match is handled:
- nil/false -- no changes: n will be called on the next match;
- true -- search for an unlimited number of matches; n will not be called again;
- a number -- maximum number of matches to search for, beginning from the next match; n will not be called again;
rex.split (subj, sep, [cf], [ef], [lo], [syn])
The function is intended for use in the generic for Lua construct. It is used for splitting a subject string subj into parts (sections). The sep parameter is a regular expression pattern representing separators between the sections.
The function returns an iterator for repeated matching of the pattern sep in the string subj, subject to flags cf and ef.
Parameter Description Type Default Value subj subject string n/a sep separator (regular expression pattern) string or userdata n/a [cf] compilation flags (bitwise OR) number cf [ef] execution flags (bitwise OR) number ef [lo] [PCRE, Oniguruma] locale string or userdata locale [syn] [Oniguruma] syntax string syntax
On every iteration pass, the iterator returns:
- A subject section (can be an empty string), followed by
- All captures in the order they appear in the sep pattern (or the entire match if the sep pattern specified no captures). If there is no match (this can occur only in the last iteration), then nothing is returned after the subject section.
The iteration will continue till the end of the subject. Unlike gmatch, there will always be at least one iteration pass, even if there are no matches in the subject.
rex.flags ([tb])
This function returns a table containing the numeric values of the constants defined by the used regex library, with the keys being the (string) names of the constants. If the table argument tb is supplied then it is used as the output table, otherwise a new table is created.
The constants contained in the returned table can then be used in most functions and methods where compilation flags or execution flags can be specified. They can also be used for comparing with return codes of some functions and methods for determining the reason of failure. For details, see the POSIX, PCRE and Oniguruma documentation.
Parameter Description Type Default Value [tb] a table for placing results into table nil
Notes: The keys in the tb table are formed from the names of the corresponding constants in the used library. They are formed as follows:
rex.new (patt, [cf], [lo], [syn])
The functions compiles regular expression patt into a regular expression object whose internal representation is corresponding to the library used. The returned result then can be used by the methods, e.g. tfind, exec, etc. Regular expression objects are automatically garbage collected.
Parameter Description Type Default Value patt regular expression pattern string n/a [cf] compilation flags (bitwise OR) number cf [lo] [PCRE, Oniguruma] locale string or userdata locale [syn] [Oniguruma] syntax string syntax
r:tfind (subj, [init], [ef])
The method searches for the first match of the compiled regexp r in the string subj, starting from offset init, subject to execution flags ef.
Parameter Description Type Default Value r regex object produced by new userdata n/a subj subject string n/a [init] start offset in the subject (can be negative) number 1 [ef] execution flags (bitwise OR) number ef
r:exec (subj, [init], [ef])
The method searches for the first match of the compiled regexp r in the string subj, starting from offset init, subject to execution flags ef.
Parameter Description Type Default Value r regex object produced by new userdata n/a subj subject string n/a [init] start offset in the subject (can be negative) number 1 [ef] execution flags (bitwise OR) number ef
[PCRE 6.0 and later. See pcre_dfa_exec in the PCRE docs.]
r:dfa_exec (subj, [init], [ef], [ovecsize], [wscount])
The method matches a compiled regular expression r against a given subject string subj, using a DFA matching algorithm.
Parameter Description Type Default Value r regex object produced by new userdata n/a subj subject string n/a [init] start offset in the subject (can be negative) number 1 [ef] execution flags (bitwise OR) number ef [ovecsize] size of the array for result offsets number 100 [wscount] number of elements in the working space array number 50
[See pcre_maketables in the PCRE docs.]
rex_pcre.maketables ()
Creates a set of character tables corresponding to the current locale and returns it as a userdata. The returned value can be passed to any Lrexlib function accepting the locale parameter.
[PCRE 4.0 and later. See pcre_config in the PCRE docs.]
rex_pcre.config ([tb])
This function returns a table containing the values of the configuration parameters used at PCRE library build-time. Those parameters (numbers) are keyed by their names (strings). If the table argument tb is supplied then it is used as the output table, else a new table is created.
Parameter Description Type Default Value [tb] a table for placing results into table nil
[See pcre_version in the PCRE docs.]
rex_pcre.version ()
This function returns a string containing the version of the used PCRE library and its release date.
rex_onig.setdefaultsyntax (syntax)
This function sets the default syntax for the Oniguruma library, according to value of the string syntax. The specified syntax will be further used for interpreting string regex patterns by all relevant functions, unless syntax argument is passed to those functions explicitly.
Returns: nothing
Examples:
- rex_onig.setdefaultsyntax ("ASIS") -- use plain text syntax as the default
- rex_onig.setdefaultsyntax ("PERL") -- use PERL regex syntax as the default
[See onig_version in the Oniguruma docs.]
rex_onig.version ()
This function returns a string containing the version of the used Oniguruma library.
rex.plainfind (subj, patt, [init], [ci])
The function searches for the first match of the string patt in the subject subj, starting from offset init.
- The string patt is not regular expression, all its characters stand for themselves.
- Both strings subj and patt can have embedded zeros.
- The flag ci specifies case-insensitive search (current locale is used).
- This function uses no regex library.
Parameter Description Type Default Value subj subject string n/a patt text to find string n/a [init] start offset in the subject (can be negative) number 1 [ci] case insensitive search boolean false
Incompatibilities between the versions 2.0 and 1.19:
- Lua 5.1 is required
- Functions newPCRE and newPOSIX renamed to new
- Functions flagsPCRE and flagsPOSIX renamed to flags
- Function versionPCRE renamed to version
- Method match renamed to tfind
- Method gmatch removed (similar functionality is provided by function gmatch)
- Methods tfind and exec: 2 values are returned on failure
- (PCRE) exec: the returned table may additionally contain named subpatterns
Incompatibilities between the versions 2.1 and 2.0:
Incompatibilities between the versions 2.2 and 2.1: