Bash String Manipulation - Complete Reference Guide

Quick Answer: What Is String Manipulation in Bash?

String manipulation means extracting, replacing, or transforming text in variables. The fastest method is parameter expansion using ${variable:offset:length} for extraction and ${variable/old/new} for replacement—no external commands needed.

Quick Comparison: Which Method Should You Use?

Method	Speed	Best For	Complexity
Parameter expansion	Fastest	Simple extraction/replacement	Very simple
sed	Very fast	Complex patterns, regex	Moderate
tr	Fast	Character translation	Simple
awk	Fast	Field-based manipulation	Moderate
grep/cut	Fast	Filtering/extracting	Simple

Bottom line: Start with parameter expansion. Use sed when you need regex patterns.

---

String manipulation is fundamental to Bash scripting. This guide covers parameter expansion, replacement, extraction, and transformation techniques—the skills you’ll use constantly in real scripts.

Table of Contents

1. Parameter Expansion Basics
2. Substring Extraction
3. String Replacement
4. Case Conversion
5. String Trimming
6. String Testing
7. Advanced Techniques
8. Performance Tips
9. Frequently Asked Questions

---

Parameter Expansion Basics

Parameter expansion is the Swiss Army knife of string manipulation. It’s pure Bash—no external commands, which makes it blazingly fast. Every syntax below uses the ${} syntax which allows you to do more than just access the variable’s value.

Basic Syntax

These patterns form the foundation of string manipulation in Bash:

${parameter}              # Simple expansion
${parameter:-word}        # Use default if unset
${parameter:=word}        # Assign default if unset
${parameter:?message}     # Display error if unset
${parameter:+word}        # Use alternate if set

Each pattern solves a different problem. The :- operator is for defensive programming—when a variable might not exist. The :+ operator is useful when you want to use one value only if another exists. The :? operator makes your scripts fail fast with a clear error message.

---

Substring Extraction

Extract portions of strings using positions and lengths. This is perfect for parsing filenames, URLs, or formatted data.

Extract from Position

Substring extraction uses the syntax ${string:offset:length}. The offset starts at 0, and length tells Bash how many characters to grab.

text="Hello, World!"
echo ${text:0:5}    # Output: Hello (5 chars from position 0)
echo ${text:7}      # Output: World! (from position 7 to end)

Without the length parameter, extraction continues to the end of the string. This is one of the most common string operations in Bash scripts.

Extract Last N Characters

To extract characters from the end of a string, use a negative offset. Note the space before the minus sign—without it, Bash might interpret it as a subtraction operator.

filename="document.pdf"
echo ${filename: -4}    # Output: .pdf (last 4 chars)

This is perfect for file extensions or getting the last part of a path. The space before the minus sign is crucial—${filename:-4} means something different (it’s the default value operator).

Remove Prefix/Suffix

Pattern removal is useful for parsing paths, URLs, or extracting file extensions. The # operator removes from the beginning; % removes from the end. Single symbol removes the shortest match; double removes the longest.

path="/home/user/file.txt"
echo ${path#*/}         # Output: home/user/file.txt (remove shortest prefix)
echo ${path##*/}        # Output: file.txt (remove longest prefix)
echo ${path%/*}         # Output: /home/user (remove shortest suffix)
echo ${path%%/*}        # Output: (remove longest suffix)

This is perfect for extracting filenames from paths (${path##*/}) or removing file extensions (${path%.*}). The pattern uses glob-style wildcards, not regex.

When to Use Substring Extraction

Use substring extraction when:

• You need part of a string (extension, prefix, portion)
• Parsing structured data (paths, URLs, formatted strings)
• You want maximum speed (parameter expansion is fastest)
• The position is known or easy to calculate

Don’t use when:

• You need regex pattern matching (use grep or sed instead)
• Working with file operations (use basename/dirname)

---

String Replacement

Replace parts of strings efficiently using parameter expansion. This is faster than piping to sed for simple replacements.

Replace First Occurrence

Parameter expansion replacement uses a single slash for the first match, double slash for all occurrences. Single slash is perfect when you only want to fix one instance.

text="cat in the hat"
echo ${text/cat/dog}    # Output: dog in the hat

Only the first “cat” becomes “dog”—the “hat” stays unchanged. This is useful when multiple occurrences exist but you want to change only the first.

Replace All Occurrences

Double slashes replace every occurrence in the string. This is the workhorse replacement pattern you’ll use most often.

text="cat cat cat"
echo ${text//cat/dog}   # Output: dog dog dog

The // tells Bash “replace all occurrences, not just the first.” This is much faster than looping or piping to sed for simple literal string replacement.

Remove Pattern

To remove all occurrences of a pattern, use parameter expansion with an empty replacement.

text="hello world"
echo ${text//o}         # Output: hell wrld (remove all 'o')

You can also remove prefix/suffix using the # and % operators shown earlier.

When to Use String Replacement

Use parameter expansion replacement when:

• Doing simple literal string replacement
• Speed matters (faster than sed for strings already in memory)
• The text is already in a variable
• No regex patterns needed

Use sed when:

• You need regex pattern matching
• Editing files directly
• Case-insensitive matching required
• Working with complex patterns

---

Case Conversion

Transform string case quickly without external commands. Case conversion requires Bash 4 or later.

Uppercase

Convert entire string or just the first character to uppercase.

text="hello"
echo ${text^^}          # Output: HELLO

# Only first char
echo ${text^}           # Output: Hello

The ^^ operator converts the entire string to uppercase, while the single ^ capitalizes only the first character. This is much faster than piping to tr or sed.

Lowercase

Convert to lowercase using the ,, operator, or just the first character with single comma.

text="HELLO"
echo ${text,,}          # Output: hello

# Only first char
echo ${text,}           # Output: hELLO

The ,, (double comma) converts everything to lowercase, while single , converts only the first character. This is faster than external commands.

When to Use Case Conversion

Use parameter expansion case conversion when:

• You need quick case transformation
• Bash 4+ is available
• Converting variables already in memory
• No special locale handling needed

Use tr or sed when:

• Bash 3 compatibility required
• Complex locale-specific conversions
• Working with file content

---

String Trimming

Remove leading and trailing whitespace—a common requirement when processing user input or parsing files.

Using xargs (Simplest)

The easiest way to trim whitespace is piping through xargs, which automatically strips leading and trailing space.

text="  hello world  "
trimmed=$(echo "$text" | xargs)
echo "[$trimmed]"   # Output: [hello world]

This is concise and readable. If you absolutely need to avoid spawning a subprocess, you can use parameter expansion, but it’s more complex.

Using Parameter Expansion (Pure Bash)

For pure Bash trimming without external commands:

text="  hello world  "
# Remove leading and trailing
trimmed=${text#${text%%[![:space:]]*}}
trimmed=${trimmed%${trimmed##*[![:space:]]}}
echo "[$trimmed]"   # Output: [hello world]

This works but is harder to read. Use it when subprocess overhead matters or when xargs isn’t available.

---

String Testing

Test string properties and contents. These checks are essential for validation and defensive programming.

Length

Get the length of a string instantly using the # operator. This is useful for validation (is the input long enough?).

text="Bash"
echo ${#text}       # Output: 4

This is pure Bash, no external commands needed.

Empty Check

Always validate that a string exists and has content before using it. The -z test checks if empty, -n checks if not empty.

if [ -z "$text" ]; then
  echo "String is empty"
fi

if [ -n "$text" ]; then
  echo "String is not empty"
fi

This is defensive programming—catching missing or empty input before it causes problems downstream.

Contains Substring

Check if a string contains another string using glob pattern matching with ==.

text="hello world"
if [[ $text == *"world"* ]]; then
  echo "Contains 'world'"
fi

The * wildcards match any characters before and after. Use double brackets [[ for this to work properly. For case-insensitive matching, use grep or sed instead.

---

Quick Reference

# Substring extraction
${text:0:5}             # First 5 chars
${text:7}               # From position 7 to end
${text: -4}             # Last 4 chars

# String replacement
${text/old/new}         # Replace first
${text//old/new}        # Replace all

# Remove prefix/suffix
${text#pattern}         # Remove shortest prefix
${text##pattern}        # Remove longest prefix
${text%pattern}         # Remove shortest suffix
${text%%pattern}        # Remove longest suffix

# Case conversion
${text^^}               # Uppercase
${text,,}               # Lowercase
${text^}                # Capitalize first

# Testing
${#text}                # Length
[[ $text = *pattern* ]] # Contains substring

# Default values
${text:-default}        # Use default if unset
${text:=default}        # Set default if unset

---

Advanced Techniques

Indirect Expansion

var="greeting"
greeting="Hello"
echo ${!var}        # Output: Hello

Pattern Matching

text="file123.txt"
if [[ $text == file[0-9]*.txt ]]; then
  echo "Matches pattern"
fi

Default Values

# Use default if unset
name=${1:-"guest"}
echo "$name"

# Set and use default
config=${2:="default.conf"}
echo "$config"

---

Performance Tips

Fastest to Slowest:

1. Parameter expansion - Built-in, no external commands
2. tr command - Single external process
3. sed command - Regex processing, slightly slower
4. awk command - Pattern matching, more overhead

# Fast (built-in)
result=${text//old/new}

# Slower (external)
result=$(echo "$text" | sed 's/old/new/g')

---

Practical Examples

Extract File Extension

file="script.bash"
ext=${file##*.}
echo "$ext"         # Output: bash

Get Directory from Path

path="/home/user/documents/file.txt"
dir=${path%/*}
echo "$dir"         # Output: /home/user/documents

Capitalize Words

text="hello world"
echo ${text^}       # Output: Hello world

Build URLs

protocol="https"
domain="example.com"
path="/api/users"
url="${protocol}://${domain}${path}"
echo "$url"         # Output: https://example.com/api/users

---

Frequently Asked Questions

Q: What’s the difference between ${var} and $var?

A: Both work the same. ${var} is explicit syntax, useful when variable name is ambiguous.

Q: How do I handle special characters?

A: Always quote: "${var}". Quotes preserve special characters.

Q: Can I use regex in parameter expansion?

A: No, use [[ ]] with =~ for regex matching. Parameter expansion uses glob patterns.

Q: Is parameter expansion faster than sed?

A: Yes, parameter expansion is built-in and much faster (no external process).

---

Next Steps

Explore related topics:

• Bash Arrays - Work with string collections
• Bash Text Processing - Process files line-by-line
• Bash Regular Expressions - Pattern matching in detail