Regular Expressions (Regex) - Practical Handbook
Regex is a family of pattern languages, not one identical standard in every tool. grep/POSIX, JavaScript, Python and PCRE are similar but differ in details, so advanced expressions should be written with the target engine in mind.
Related topics: Shell Scripting, Visual Studio Code, vi / Vim / gVim / Neovim and Python.
1. What regex is
A regular expression is a pattern used to search, validate and transform text.
2. Simplest pattern
hello
Matches the literal text hello.
3. Special characters
Common metacharacters:
. ^ $ * + ? { } [ ] \ | ( )
Escape a metacharacter with a backslash when you want its literal meaning.
4. Dot
a.c
Matches abc, a-c, a c, etc., depending on engine and flags.
5. Start and end
^start
end$
6. Character classes
[abc]
[a-z]
[A-Z0-9]
[^0-9]
7. Shortcuts
Typical Perl-style classes:
\d
\w
\s
\D
\W
\S
Exact behavior can vary by regex engine and Unicode settings.
8. Quantifiers
a*
a+
a?
a{3}
a{2,5}
Meaning:
*zero or more,+one or more,?zero or one,{n}exactly n,{n,m}between n and m.
9. Groups
(ab)+
Groups let you apply quantifiers or capture submatches.
10. Alternation
cat|dog
11. Grouping alternation
^(cat|dog)$
12. Capturing groups
(\d{4})-(\d{2})-(\d{2})
Captures year, month and day separately.
13. Non-capturing group
(?:cat|dog)
14. Named groups
Syntax varies by engine.
JavaScript/Python commonly support forms such as:
(?<year>\d{4})
or in Python:
(?P<year>\d{4})
15. Greedy and lazy
Greedy:
".*"
Lazy:
".*?"
16. Lookahead
Positive:
foo(?=bar)
Negative:
foo(?!bar)
17. Lookbehind
Positive:
(?<=USD )\d+
Support differs by engine/version.
18. Flags
Common flags include:
icase-insensitive,mmultiline,sdot matches newlines,gglobal search in engines such as JavaScript.
19. JavaScript
const re = /error/i;
re.test("ERROR");
"abc123".match(/\d+/);
"abc123".replace(/\d+/, "X");
20. Python
import re
re.search(r"\d+", "abc123")
re.findall(r"\w+", "hello world")
re.sub(r"\s+", " ", text)
Use raw strings for most Python regexes.
21. grep
Basic:
grep 'error' file.log
Extended regex:
grep -E 'error|warning' file.log
Recursive:
grep -R -E 'TODO|FIXME' .
22. sed
sed -E 's/[[:space:]]+/ /g' file.txt
23. VS Code / Vim
VS Code search supports regex mode.
Vim search:
/pattern
Substitution:
:%s/old/new/g
Remember: regex dialects differ.
24. Email validation
Do not try to implement the full email RFC with one giant regex unless there is a real need.
For normal forms, use a practical syntax check and verify ownership by sending a confirmation message.
25. Regex and HTML
Regex can search simple HTML fragments, but it is not a reliable parser for arbitrary nested HTML.
Use an HTML parser for structural work.
26. Readability
Prefer simple expressions over clever one-liners.
Break complex validation into steps when possible.
27. Testing
Test against:
- expected matches,
- expected non-matches,
- empty input,
- long input,
- Unicode if relevant,
- malformed edge cases.
28. ReDoS
Some regexes can cause catastrophic backtracking on specially crafted input.
Be cautious with nested ambiguous quantifiers such as:
(a+)+$
especially on untrusted long input.
29. Examples
IPv4-like shape, not full semantic validation:
^(\d{1,3}\.){3}\d{1,3}$
Date shape:
^\d{4}-\d{2}-\d{2}$
Hex color:
^#[0-9A-Fa-f]{6}$
Simple identifier:
^[A-Za-z_][A-Za-z0-9_]*$
30. What you should know
You should understand:
- literals,
- character classes,
- anchors,
- quantifiers,
- groups,
- alternation,
- greedy vs lazy,
- lookarounds,
- engine differences,
- basic use in JavaScript, Python, grep and editors.
The main rule: use the simplest regex that solves the problem.
Reference sources
- POSIX regular expressions: https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap09.html
- MDN regular expressions: https://developer.mozilla.org/docs/Web/JavaScript/Guide/Regular_expressions
- Python re: https://docs.python.org/3/library/re.html