Tech Handbook Null Yard

Regular Expressions (Regex) - Practical Handbook

Regex is a family of pattern languages, not one identical standard in every tool. grep/POSIX, JavaScript, Python and PCRE are similar but differ in details, so advanced expressions should be written with the target engine in mind.

Related topics: Shell Scripting, Visual Studio Code, vi / Vim / gVim / Neovim and Python.

1. What regex is

A regular expression is a pattern used to search, validate and transform text.

2. Simplest pattern

hello

Matches the literal text hello.

3. Special characters

Common metacharacters:

. ^ $ * + ? { } [ ] \ | ( )

Escape a metacharacter with a backslash when you want its literal meaning.

4. Dot

a.c

Matches abc, a-c, a c, etc., depending on engine and flags.

5. Start and end

^start
end$

6. Character classes

[abc]
[a-z]
[A-Z0-9]
[^0-9]

7. Shortcuts

Typical Perl-style classes:

\d
\w
\s
\D
\W
\S

Exact behavior can vary by regex engine and Unicode settings.

8. Quantifiers

a*
a+
a?
a{3}
a{2,5}

Meaning:

  • * zero or more,
  • + one or more,
  • ? zero or one,
  • {n} exactly n,
  • {n,m} between n and m.

9. Groups

(ab)+

Groups let you apply quantifiers or capture submatches.

10. Alternation

cat|dog

11. Grouping alternation

^(cat|dog)$

12. Capturing groups

(\d{4})-(\d{2})-(\d{2})

Captures year, month and day separately.

13. Non-capturing group

(?:cat|dog)

14. Named groups

Syntax varies by engine.

JavaScript/Python commonly support forms such as:

(?<year>\d{4})

or in Python:

(?P<year>\d{4})

15. Greedy and lazy

Greedy:

".*"

Lazy:

".*?"

16. Lookahead

Positive:

foo(?=bar)

Negative:

foo(?!bar)

17. Lookbehind

Positive:

(?<=USD )\d+

Support differs by engine/version.

18. Flags

Common flags include:

  • i case-insensitive,
  • m multiline,
  • s dot matches newlines,
  • g global search in engines such as JavaScript.

19. JavaScript

const re = /error/i;

re.test("ERROR");
"abc123".match(/\d+/);
"abc123".replace(/\d+/, "X");

20. Python

import re

re.search(r"\d+", "abc123")
re.findall(r"\w+", "hello world")
re.sub(r"\s+", " ", text)

Use raw strings for most Python regexes.

21. grep

Basic:

grep 'error' file.log

Extended regex:

grep -E 'error|warning' file.log

Recursive:

grep -R -E 'TODO|FIXME' .

22. sed

sed -E 's/[[:space:]]+/ /g' file.txt

23. VS Code / Vim

VS Code search supports regex mode.

Vim search:

/pattern

Substitution:

:%s/old/new/g

Remember: regex dialects differ.

24. Email validation

Do not try to implement the full email RFC with one giant regex unless there is a real need.

For normal forms, use a practical syntax check and verify ownership by sending a confirmation message.

25. Regex and HTML

Regex can search simple HTML fragments, but it is not a reliable parser for arbitrary nested HTML.

Use an HTML parser for structural work.

26. Readability

Prefer simple expressions over clever one-liners.

Break complex validation into steps when possible.

27. Testing

Test against:

  • expected matches,
  • expected non-matches,
  • empty input,
  • long input,
  • Unicode if relevant,
  • malformed edge cases.

28. ReDoS

Some regexes can cause catastrophic backtracking on specially crafted input.

Be cautious with nested ambiguous quantifiers such as:

(a+)+$

especially on untrusted long input.

29. Examples

IPv4-like shape, not full semantic validation:

^(\d{1,3}\.){3}\d{1,3}$

Date shape:

^\d{4}-\d{2}-\d{2}$

Hex color:

^#[0-9A-Fa-f]{6}$

Simple identifier:

^[A-Za-z_][A-Za-z0-9_]*$

30. What you should know

You should understand:

  • literals,
  • character classes,
  • anchors,
  • quantifiers,
  • groups,
  • alternation,
  • greedy vs lazy,
  • lookarounds,
  • engine differences,
  • basic use in JavaScript, Python, grep and editors.

The main rule: use the simplest regex that solves the problem.

Reference sources

  • POSIX regular expressions: https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap09.html
  • MDN regular expressions: https://developer.mozilla.org/docs/Web/JavaScript/Guide/Regular_expressions
  • Python re: https://docs.python.org/3/library/re.html