How to Generate CSV from Data

Quick Answer: Generate CSV from Data in Bash

To create a CSV file, use echo to write rows: echo "Header1,Header2,Header3" > file.csv then echo "value1,value2,value3" >> file.csv. For complex data, use printf with field values separated by commas. Always quote values containing commas or newlines.

Quick Comparison: CSV Generation Methods

Method	Syntax	Best For	Complexity
echo pipes	echo "col1,col2" >> file.csv	Simple data	Very simple
printf format	printf "%s,%s\n" $v1 $v2	Formatted output	Simple
awk -v OFS	awk '{print $1, $2}'	Field processing	Moderate
Custom function	Reusable format	Complex logic	Moderate

Bottom line: Use echo for simple CSVs; use printf for formatted output with quoting.

---

Create CSV files from command output, arrays, and data sources. Learn generating headers, formatting data, and handling special characters.

Method 1: Basic CSV Generation

# Simple CSV with echo
echo "Name,Age,City" > people.csv
echo "John,30,New York" >> people.csv
echo "Jane,25,Boston" >> people.csv

Generate CSV from Command Output

# Convert ps output to CSV
ps aux | awk '{print $1","$2","$3","$4","$11}' > processes.csv

CSV with Proper Header

#!/bin/bash

output="$1"

if [ -z "$output" ]; then
  echo "Usage: $0 <output.csv>"
  exit 1
fi

# Write header
echo "ID,Name,Email,Status" > "$output"

# Append data
echo "1,John Doe,john@example.com,Active" >> "$output"
echo "2,Jane Smith,jane@example.com,Inactive" >> "$output"
echo "3,Bob Wilson,bob@example.com,Active" >> "$output"

echo "Created: $output"

Generate CSV from Array

#!/bin/bash

# Create CSV from array data

output="results.csv"

# Define data arrays
names=("John" "Jane" "Bob" "Alice")
ages=(30 25 35 28)
cities=("NYC" "Boston" "LA" "Chicago")

# Write header
echo "Name,Age,City" > "$output"

# Write data rows
for i in "${!names[@]}"; do
  echo "${names[$i]},${ages[$i]},${cities[$i]}" >> "$output"
done

echo "Generated: $output"

Output (results.csv):

Name,Age,City
John,30,NYC
Jane,25,Boston
Bob,35,LA
Alice,28,Chicago

Generate CSV from Directory Listing

#!/bin/bash

# Create CSV from file listing

directory="${1:-.}"
output="filelist.csv"

echo "Filename,Size,Modified" > "$output"

for file in "$directory"/*; do
  if [ -f "$file" ]; then
    filename=$(basename "$file")
    size=$(stat -c%s "$file" 2>/dev/null || stat -f%z "$file" 2>/dev/null)
    modified=$(stat -c%y "$file" 2>/dev/null | cut -d' ' -f1 || stat -f %Sm -t "%Y-%m-%d" "$file" 2>/dev/null)

    echo "$filename,$size,$modified" >> "$output"
  fi
done

echo "Created: $output"

Generate CSV from Database

#!/bin/bash

# Generate CSV from MySQL/SQLite

output="data.csv"

# Example: MySQL query
mysql -u user -p password database -e \
  "SELECT id, name, email, status FROM users;" \
  | sed 's/\t/,/g' > "$output"

echo "Created: $output"

Practical Example: System Report CSV

#!/bin/bash

# File: generate_system_report.sh

output="system_report.csv"

echo "=== Generating System Report ==="

# Header
echo "Metric,Value" > "$output"

# System information
hostname=$(hostname)
kernel=$(uname -r)
uptime=$(uptime -p 2>/dev/null || uptime)
cpu_count=$(nproc)
memory=$(free -h | grep Mem | awk '{print $2}')
disk=$(df -h / | tail -1 | awk '{print $2}')

# Add metrics
echo "Hostname,$hostname" >> "$output"
echo "Kernel,$kernel" >> "$output"
echo "CPU Cores,$cpu_count" >> "$output"
echo "Memory,$memory" >> "$output"
echo "Disk,$disk" >> "$output"

echo ""
echo "✓ Report created: $output"
echo ""
echo "Content:"
cat "$output"

Output:

Metric,Value
Hostname,myserver
Kernel,5.10.0-8-generic
CPU Cores,8
Memory,16G
Disk,500G

Handle Special Characters in CSV

#!/bin/bash

# Properly escape CSV special characters

csv_escape() {
  # Escape double quotes by doubling them
  local text="$1"
  text="${text//\"/\"\"}"

  # Wrap in quotes if contains comma, newline, or quote
  if [[ "$text" == *","* ]] || [[ "$text" == *$'\n'* ]] || [[ "$text" == *'"'* ]]; then
    echo "\"$text\""
  else
    echo "$text"
  fi
}

# Usage
output="data.csv"
echo "Name,Description" > "$output"

name="John Doe"
desc="Hello, \"World\"!"

echo "$(csv_escape "$name"),$(csv_escape "$desc")" >> "$output"

Output:

Name,Description
John Doe,"Hello, ""World""!"

Generate CSV from JSON

#!/bin/bash

# Convert JSON to CSV

json_data='[
  {"name":"John","age":30,"city":"NYC"},
  {"name":"Jane","age":25,"city":"Boston"}
]'

output="from_json.csv"

echo "Name,Age,City" > "$output"

echo "$json_data" | jq -r '.[] | [.name, .age, .city] | @csv' >> "$output"

echo "Created: $output"

Generate Large CSV Batch

#!/bin/bash

# Generate large CSV file with many rows

output="large_data.csv"
rows="${1:-1000}"

echo "Generating $rows rows..."

# Header
echo "ID,Value,Status" > "$output"

# Generate data rows
for i in $(seq 1 "$rows"); do
  value=$((RANDOM * 100))
  status=$((i % 2 == 0 ? "Active" : "Inactive"))
  echo "$i,$value,$status" >> "$output"
done

echo "Created: $output ($(wc -l < "$output") lines)"

Merge Multiple CSVs

#!/bin/bash

# Combine multiple CSV files

output="merged.csv"

# Write header from first file
head -1 "$1" > "$output"

# Append all data rows
for file in "$@"; do
  tail -n +2 "$file" >> "$output"
done

echo "Merged into: $output"

CSV with Different Delimiters

#!/bin/bash

# Generate CSV-like file with different delimiter

output="data.tsv"  # Tab-separated

echo -e "Name\tAge\tCity" > "$output"
echo -e "John\t30\tNYC" >> "$output"
echo -e "Jane\t25\tBoston" >> "$output"

echo "Created: $output (tab-delimited)"

Validate CSV Generation

#!/bin/bash

# Verify CSV integrity

csv_file="$1"

if [ ! -f "$csv_file" ]; then
  echo "ERROR: File not found"
  exit 1
fi

# Check header exists
header=$(head -1 "$csv_file")
field_count=$(echo "$header" | awk -F"," '{print NF}')

echo "Header: $header"
echo "Expected fields: $field_count"
echo ""

# Count rows
line_count=$(wc -l < "$csv_file")
echo "Total lines: $line_count"
echo "Data rows: $((line_count - 1))"
echo ""

# Check field count consistency
echo "Checking field count consistency..."
awk -F"," -v expected="$field_count" '
  NR > 1 {
    actual = NF
    if (actual != expected) {
      print "ERROR: Row " NR " has " actual " fields (expected " expected ")"
    }
  }' "$csv_file"

echo "✓ Validation complete"

Common Mistakes

1. Not escaping commas in values - wrap in quotes if contains comma
2. Not escaping quotes - double them: " becomes ""
3. Inconsistent field count - each row should have same number of fields
4. Forgotten newlines - use echo or ensure proper line endings
5. Not handling special characters - test with real-world data

Performance Tips

• Use >> for appending (opens/closes file each time)
• For large files, collect data then write once
• Use printf for faster output in loops
• Consider using awk to generate multiple lines at once

Key Points

• Write header first
• Escape commas and quotes properly
• Use consistent field count
• Validate output before use
• Keep newlines consistent

Summary

Generating CSV files is straightforward in Bash. Always include headers, properly escape special characters, and validate the output. Use pipes and command output to generate data dynamically.