How to use SED to manipulate text files (Part 2: pattern, regular expression)

This is part 2 (hopefully final) of my SED cheatsheet and here, I will explain pattern matching and regular expression in SED. So let’s jump straight into it.

Let’s consider the same file we used in part 1. The link to part 1 is given here.

Pattern:

Case-sensitive:

sed -n ‘/lammps/p’ foo.data

• /lammps/ = lines having a pattern/word lammps in them
• p = print the lines matching pattern

Case-Insensitive:
What if we want to do a case-insensitive search?

sed -n ‘/lammps/Ip’ foo.data

• I = case-insensitive search

sed ‘/lammps/d’ foo.data

• d = delete lines having lammps in it

We can see the line containing lammps is deleted.

sed -n ‘/lammps/d’ foo.data

• This will not print the file contents. Also, to delete the line from the main file, we need to use -i

sed -i ‘/lammps/d’ foo.data

Lines between two patterns:

Also, we can get a range of lines between two patterns:

sed -n ‘/software/,/computers/p’ foo.data

This command prints out the lines between two patterns (starting with the line having ‘software’ and ending with the line having ‘computers’).

Ignoring lines between two patterns:

sed -n ‘/software/,/computers/!p’ foo.data

• It will print out the lines that are not between these two patterns ( ‘software’ and ‘computers’).

sed ‘$!d’ foo.data

• This will delete all lines except the last line. This edit is only terminal only, to make changes in the original text file, we need to use:
  sed -i ‘$!d’ foo.data

sed -n ‘$!p’ foo.data

• This will print all lines except the last line.

Replace:

sed ‘s/lammps/VMD/Ig’ foo.data

• This code snippet will replace lammps with VMD with the case-insensitive and global search:
  s = substitute
  Ig = case-insensitive and global search
• g is for global; this means all the instances in each line.
  if sed ‘s/lammps/VMD/I2’ foo.data was used, then only the 2nd ‘lammps’ in each line would be supplanted. Try it out yourself, it’s super easy.
• sed ‘3s/lammps/VMD/Ig’ foo.data will replace only the 3rd line’s contents.
• sed ‘3,$s/lammps/VMD/Ig’ foo.data will start replacing from 3rd line up to the last line.

Using variables:

variable=”lammps”
sed “s/$variable/VMD/Ig” foo.data

Or using pipe tool,

$ variable=”lammps” | sed “s/$variable/VMD/Ig” foo.data

For variables, we need to use the double quote “ “ instead of single quote ‘ ‘ as POSIX shells do not read variables well inside a single quote.

Others:

sed -n ‘s/[aeiou]/\$/gp’ foo.data

• This will replace all vowels with $. $ means the last line but we have to escape it with \ to make sure we want SED to think of it not as the last line but as a symbol.

sed ‘s/[^aeiou]/*/g’ foo.data

• This will replace all consonants (not vowels = [^aeiou]) with an asterisk *

In part 3, I will show how to use sed and awk to manipulate log files of LAMMPS. That’s all for now and I hope you had a nice time reading this article. Thank you!!