Regex: Update PCRE to v8.35.
I was über lazy at first, so took libs from SM. But actually it's quite easy to compile, so let's update to latest version \o/.
This commit is contained in:
@ -81,33 +81,36 @@ strings. This optimization is also disabled for partial matching.
|
||||
<br><a name="SEC2" href="#TOC1">PARTIAL MATCHING USING pcre_exec() OR pcre[16|32]_exec()</a><br>
|
||||
<P>
|
||||
A partial match occurs during a call to <b>pcre_exec()</b> or
|
||||
<b>pcre[16|32]_exec()</b> when the end of the subject string is reached successfully,
|
||||
but matching cannot continue because more characters are needed. However, at
|
||||
least one character in the subject must have been inspected. This character
|
||||
need not form part of the final matched string; lookbehind assertions and the
|
||||
\K escape sequence provide ways of inspecting characters before the start of a
|
||||
matched substring. The requirement for inspecting at least one character exists
|
||||
because an empty string can always be matched; without such a restriction there
|
||||
would always be a partial match of an empty string at the end of the subject.
|
||||
<b>pcre[16|32]_exec()</b> when the end of the subject string is reached
|
||||
successfully, but matching cannot continue because more characters are needed.
|
||||
However, at least one character in the subject must have been inspected. This
|
||||
character need not form part of the final matched string; lookbehind assertions
|
||||
and the \K escape sequence provide ways of inspecting characters before the
|
||||
start of a matched substring. The requirement for inspecting at least one
|
||||
character exists because an empty string can always be matched; without such a
|
||||
restriction there would always be a partial match of an empty string at the end
|
||||
of the subject.
|
||||
</P>
|
||||
<P>
|
||||
If there are at least two slots in the offsets vector when a partial match is
|
||||
returned, the first slot is set to the offset of the earliest character that
|
||||
was inspected. For convenience, the second offset points to the end of the
|
||||
subject so that a substring can easily be identified.
|
||||
subject so that a substring can easily be identified. If there are at least
|
||||
three slots in the offsets vector, the third slot is set to the offset of the
|
||||
character where matching started.
|
||||
</P>
|
||||
<P>
|
||||
For the majority of patterns, the first offset identifies the start of the
|
||||
partially matched string. However, for patterns that contain lookbehind
|
||||
assertions, or \K, or begin with \b or \B, earlier characters have been
|
||||
inspected while carrying out the match. For example:
|
||||
For the majority of patterns, the contents of the first and third slots will be
|
||||
the same. However, for patterns that contain lookbehind assertions, or begin
|
||||
with \b or \B, characters before the one where matching started may have been
|
||||
inspected while carrying out the match. For example, consider this pattern:
|
||||
<pre>
|
||||
/(?<=abc)123/
|
||||
</pre>
|
||||
This pattern matches "123", but only if it is preceded by "abc". If the subject
|
||||
string is "xyzabc12", the offsets after a partial match are for the substring
|
||||
"abc12", because all these characters are needed if another match is tried
|
||||
with extra characters added to the subject.
|
||||
string is "xyzabc12", the first two offsets after a partial match are for the
|
||||
substring "abc12", because all these characters were inspected. However, the
|
||||
third offset is set to 6, because that is the offset where matching began.
|
||||
</P>
|
||||
<P>
|
||||
What happens when a partial match is identified depends on which of the two
|
||||
@ -303,6 +306,16 @@ not retain the previously partially-matched string. It is up to the calling
|
||||
program to do that if it needs to.
|
||||
</P>
|
||||
<P>
|
||||
That means that, for an unanchored pattern, if a continued match fails, it is
|
||||
not possible to try again at a new starting point. All this facility is capable
|
||||
of doing is continuing with the previous match attempt. In the previous
|
||||
example, if the second set of data is "ug23" the result is no match, even
|
||||
though there would be a match for "aug23" if the entire string were given at
|
||||
once. Depending on the application, this may or may not be what you want.
|
||||
The only way to allow for starting again at the next character is to retain the
|
||||
matched part of the subject and try a new complete match.
|
||||
</P>
|
||||
<P>
|
||||
You can set the PCRE_PARTIAL_SOFT or PCRE_PARTIAL_HARD options with
|
||||
PCRE_DFA_RESTART to continue partial matching over multiple segments. This
|
||||
facility can be used to pass very long subject strings to the DFA matching
|
||||
@ -334,10 +347,9 @@ processing time is needed.
|
||||
<P>
|
||||
<b>Note:</b> If the pattern contains lookbehind assertions, or \K, or starts
|
||||
with \b or \B, the string that is returned for a partial match includes
|
||||
characters that precede the partially matched string itself, because these must
|
||||
be retained when adding on more characters for a subsequent matching attempt.
|
||||
However, in some cases you may need to retain even earlier characters, as
|
||||
discussed in the next section.
|
||||
characters that precede the start of what would be returned for a complete
|
||||
match, because it contains all the characters that were inspected during the
|
||||
partial match.
|
||||
</P>
|
||||
<br><a name="SEC9" href="#TOC1">ISSUES WITH MULTI-SEGMENT MATCHING</a><br>
|
||||
<P>
|
||||
@ -356,12 +368,35 @@ includes the effect of PCRE_NOTEOL.
|
||||
offsets that are returned for a partial match. However a lookbehind assertion
|
||||
later in the pattern could require even earlier characters to be inspected. You
|
||||
can handle this case by using the PCRE_INFO_MAXLOOKBEHIND option of the
|
||||
<b>pcre_fullinfo()</b> or <b>pcre[16|32]_fullinfo()</b> functions to obtain the length
|
||||
of the largest lookbehind in the pattern. This length is given in characters,
|
||||
not bytes. If you always retain at least that many characters before the
|
||||
partially matched string, all should be well. (Of course, near the start of the
|
||||
subject, fewer characters may be present; in that case all characters should be
|
||||
retained.)
|
||||
<b>pcre_fullinfo()</b> or <b>pcre[16|32]_fullinfo()</b> functions to obtain the
|
||||
length of the longest lookbehind in the pattern. This length is given in
|
||||
characters, not bytes. If you always retain at least that many characters
|
||||
before the partially matched string, all should be well. (Of course, near the
|
||||
start of the subject, fewer characters may be present; in that case all
|
||||
characters should be retained.)
|
||||
</P>
|
||||
<P>
|
||||
From release 8.33, there is a more accurate way of deciding which characters to
|
||||
retain. Instead of subtracting the length of the longest lookbehind from the
|
||||
earliest inspected character (<i>offsets[0]</i>), the match start position
|
||||
(<i>offsets[2]</i>) should be used, and the next match attempt started at the
|
||||
<i>offsets[2]</i> character by setting the <i>startoffset</i> argument of
|
||||
<b>pcre_exec()</b> or <b>pcre_dfa_exec()</b>.
|
||||
</P>
|
||||
<P>
|
||||
For example, if the pattern "(?<=123)abc" is partially
|
||||
matched against the string "xx123a", the three offset values returned are 2, 6,
|
||||
and 5. This indicates that the matching process that gave a partial match
|
||||
started at offset 5, but the characters "123a" were all inspected. The maximum
|
||||
lookbehind for that pattern is 3, so taking that away from 5 shows that we need
|
||||
only keep "123a", and the next match attempt can be started at offset 3 (that
|
||||
is, at "a") when further characters have been added. When the match start is
|
||||
not the earliest inspected character, <b>pcretest</b> shows it explicitly:
|
||||
<pre>
|
||||
re> "(?<=123)abc"
|
||||
data> xx123a\P\P
|
||||
Partial match at offset 5: 123a
|
||||
</PRE>
|
||||
</P>
|
||||
<P>
|
||||
3. Because a partial match must always contain at least one character, what
|
||||
@ -465,9 +500,9 @@ Cambridge CB2 3QH, England.
|
||||
</P>
|
||||
<br><a name="SEC11" href="#TOC1">REVISION</a><br>
|
||||
<P>
|
||||
Last updated: 24 June 2012
|
||||
Last updated: 02 July 2013
|
||||
<br>
|
||||
Copyright © 1997-2012 University of Cambridge.
|
||||
Copyright © 1997-2013 University of Cambridge.
|
||||
<br>
|
||||
<p>
|
||||
Return to the <a href="index.html">PCRE index page</a>.
|
||||
|
Reference in New Issue
Block a user