two question about python regular expressions

Question

Q1. why we can not use word boundary and back reference without using r at start of regex? e.g. '\b[a-z]{5}\d{3}\b' this not works but this r'\b[a-z]{5}\d{3}\b' works

Q2. why python does not support variable length negative look behind assertions while it supports variable length negative look ahead assertion, c# support both and i think it is an excellent feature to have also variable length negative look behind in python.

please clear these two concepts. thanks

Q1: replace '\b' with '\\b' within your regex and it will work without raw string — Artsiom Rudzenka, Aug 18 '11 at 11:15
don't ask two question in one posting, don't ask questions that cannot really be answered. — , Aug 18 '11 at 11:22
@Hop He didn't know the question couldn't really be answered. The questions are also closely related, I think they're perfectly appropriate for one question. — agf, Aug 18 '11 at 11:32

agf · Accepted Answer · 2011-08-18T11:30:45.810

4

It does work without raw strings:

'\\b[a-z]{5}\\d{3}\\b'

You just need to double escape the special chars (actually, what you do is escape all backslashes).

Variable length assertions are one of those features that some implementations support and some don't. Check out the regex module on PyPI for a version with more features and better unicode support, which may eventually replace the standard library re.

Edit: To make the version from your comment work without raw strings, use:

re.sub('[a-z]+(\d+)', '\\1', string)

Again, Python interprets backslashes. it thinks \1 means a byte value of 1. If you actually mean \1, you need to escape the backslash by doing \\1, or use raw strings.

Edit 2: Adding the link from @Nate's comment to the list of Python escape sequences.

edited Aug 18 '11 at 11:30

answered Aug 18 '11 at 11:15

agf

171,228
44
289
238

so raw string is basically escaping special characters?? so why back refrencing dont work? e.g. this dont work `re.sub('[a-z](\d+)', '\1', string)` but this works `re.sub('[a-z]+(\d+)', r'\1', string)` – Aamir Rind Aug 18 '11 at 11:20
I've added the example from your comment to my answer. – agf Aug 18 '11 at 11:24
@agf - Not **strictly** true; I'm pretty sure python only replaces the escape sequences in [this list](http://docs.python.org/reference/lexical_analysis.html#string-literals) (though I only just checked a couple). Not that I'm trying to confuse Aamir - please, use raw strings - but the language doesn't require it for all escape sequences. – Nate Aug 18 '11 at 11:25
Yes, I know, I was generalizing. It doesn't hurt to be consistent and escape all backslashes. He has already noticed that `\d` works fine. – agf Aug 18 '11 at 11:27
@agf: this is issue with only word boundary, so when you can use `\w+` or `\d+` without explicitly defining `r` at start then why we dont double escape the `\w+`?? – Aamir Rind Aug 18 '11 at 11:27
Yes, as @Nate said, some of them will work. See the link in his comment which I've added to my post. However, it will __never hurt__ to escape backslashes, so if you're not using raw strings, you should __always__ do it, instead of just where necessary, or you might miss one where it's required. – agf Aug 18 '11 at 11:29
@agf: @nate: and what about `variable length look behind`? why python does not support that? – Aamir Rind Aug 18 '11 at 11:33
I already answered that in my answer -- no particular reason, Python's `re` module isn't that great. I linked to the more advanced `regex` module in my answer, which has many more features, and may replace `re` eventually. – agf Aug 18 '11 at 11:46

score 1 · Answer 2 · edited May 23 '17 at 09:59

In regards to your first question, this is because the r designates a "raw string". Without this r, your backslashes are interpreted as escape codes. If you don't want to use raw strings, you can use '\\b[a-z]{5}\\d{3}\\b', although this is far less readable. You can read more detail about raw strings here.

In regards to your second question, you should take a look at this excellent question, which discusses the differences between various flavors of regular expression used by different languages (namely C#, Java, and Python).

score 0 · Answer 3 · answered Aug 18 '11 at 11:26

Almost all information you can find in tutorial - which is your the best friend:

Regular expressions use the backslash character ('\') to indicate special forms or to allow special characters to be used without invoking their special meaning. This collides with Python’s usage of the same character for the same purpose in string literals; for example, to match a literal backslash, one might have to write '\\' as the pattern string, because the regular expression must be \, and each backslash must be expressed as \ inside a regular Python string literal.

The solution is to use Python’s raw string notation for regular expression patterns; backslashes are not handled in any special way in a string literal prefixed with 'r'. So r"\n" is a two-character string containing '\' and 'n', while "\n" is a one-character string containing a newline. Usually patterns will be expressed in Python code using this raw string notation.

It is quite hard to answer on your second question - i think that authors are implementing only those features which they think are required. They try to add code that is useful for the most of user but it is impossible to implement all the features fast.

two question about python regular expressions

3 Answers3