sed examples

Simple search and replace

Detailed examples for substitute command will be convered in later sections, syntax is

s/REGEXP/REPLACEMENT/FLAGS

The / character is idiomatically used as delimiter character. See also Using different delimiter for REGEXP

editing stdin

$ seq 10 | paste -sd,
1,2,3,4,5,6,7,8,9,10

$ # change only first ',' to ' : '
$ seq 10 | paste -sd, | sed 's/,/ : /'
1 : 2,3,4,5,6,7,8,9,10

$ # change all ',' to ' : ' by using 'g' modifier
$ seq 10 | paste -sd, | sed 's/,/ : /g'
1 : 2 : 3 : 4 : 5 : 6 : 7 : 8 : 9 : 10

Note: As a good practice, all examples use single quotes around arguments to prevent shell interpretation. See Shell substitutions section on use of double quotes

editing file input

• By default newline character is the line separator
• See Regular Expressions section for qualifying search terms
  • for example to distinguish between 'hi', 'this', 'his', 'history', etc

$ cat greeting.txt 
Hi there
Have a nice day

$ # change first 'Hi' in each line to 'Hello'
$ sed 's/Hi/Hello/' greeting.txt
Hello there
Have a nice day

$ # change first 'nice day' in each line to 'safe journey'
$ sed 's/nice day/safe journey/' greeting.txt
Hi there
Have a safe journey

$ # change all 'e' to 'E' and save changed text to another file
$ sed 's/e/E/g' greeting.txt > out.txt
$ cat out.txt 
Hi thErE
HavE a nicE day

Inplace file editing

• In previous section, the output from sed was displayed on stdout or saved to another file
• To write the changes back to original file, use -i option

Note:

• Refer to man sed for details of how to use the -i option. It varies with different sed implementations. As mentioned at start of this chapter, sed (GNU sed) 4.2.2 is being used here
• See this Q&A when working with symlinks

With backup

• When extension is given, the original input file is preserved with name changed according to extension provided

$ # '.bkp' is extension provided
$ sed -i.bkp 's/Hi/Hello/' greeting.txt

$ # original file gets preserved in 'greeting.txt.bkp'
Hi there
Have a nice day

$ # output from sed gets written to 'greeting.txt'
$ cat greeting.txt
Hello there
Have a nice day

Without backup

• Use this option with caution, changes made cannot be undone

$ sed -i 's/nice day/safe journey/' greeting.txt

$ # note, 'Hi' was already changed to 'Hello' in previous example
$ cat greeting.txt
Hello there
Have a safe journey

Multiple files

• Multiple input files are treated individually and changes are written back to respective files

$ cat f1
I ate 3 apples
$ cat f2
I bought two bananas and 3 mangoes

$ # -i can be used with or without backup
$ sed -i 's/3/three/' f1 f2
$ cat f1
I ate three apples
$ cat f2
I bought two bananas and three mangoes

Prefix backup name

• A * in argument given to -i will get expanded to input filename
• This way, one can add prefix instead of suffix for backup

$ cat var.txt 
foo
bar
baz

$ sed -i'bkp.*' 's/foo/hello/' var.txt 
$ cat var.txt 
hello
bar
baz

$ cat bkp.var.txt 
foo
bar
baz

Place backups in directory

• * also allows to specify an existing directory to place the backups instead of current working directory

$ mkdir bkp_dir
$ sed -i'bkp_dir/*' 's/bar/hi/' var.txt 
$ cat var.txt 
hello
hi
baz

$ cat bkp_dir/var.txt
hello
bar
baz

$ # extensions can be added as well
$ # bkp_dir/*.bkp for suffix
$ # bkp_dir/bkp.* for prefix
$ # bkp_dir/bkp.*.2017 for both and so on

Line filtering options

• By default, sed acts on entire file. Often, one needs to extract or change only specific lines based on text search, line numbers, lines between two patterns, etc
• This filtering is much like using grep, head and tail commands in many ways and there are even more features
  • Use sed for inplace editing, the filtered lines to be transformed etc. Not as substitute for grep, head and tail

Print command

• It is usually used in conjunction with -n option
• By default, sed prints every input line, including any changes made by commands like substitution
  • printing here refers to line being part of sed output which may be shown on terminal, redirected to file, etc
• Using -n option and p command together, only specific lines needed can be filtered
• Examples below use the /REGEXP/ addressing, other forms will be seen in sections to follow

$ cat poem.txt 
Roses are red,
Violets are blue,
Sugar is sweet,
And so are you.

$ # all lines containing the string 'are'
$ # same as: grep 'are' poem.txt 
$ sed -n '/are/p' poem.txt 
Roses are red,
Violets are blue,
And so are you.

$ # all lines containing the string 'so are'
$ # same as: grep 'so are' poem.txt 
$ sed -n '/so are/p' poem.txt 
And so are you.

• Using print and substitution together

$ # print only lines on which substitution happens
$ sed -n 's/are/ARE/p' poem.txt 
Roses ARE red,
Violets ARE blue,
And so ARE you.

$ # if line contains 'are', perform given command
$ # print only if substitution succeeds
$ sed -n '/are/ s/so/SO/p' poem.txt 
And SO are you.

• Duplicating every input line

$ # note, -n is not used and no filtering applied
$ seq 3 | sed 'p'
1
1
2
2
3
3

Delete command

• By default, sed prints every input line, including any changes like substitution
• Using the d command, those specific lines will NOT be printed

$ # same as: grep -v 'are' poem.txt 
$ sed '/are/d' poem.txt 
Sugar is sweet,

$ # same as: seq 5 | grep -v '3'
$ seq 5 | sed '/3/d'
1
2
4
5

• Modifier I allows to filter lines in case-insensitive way
• See Regular Expressions section for more details

$ # /rose/I means match the string 'rose' irrespective of case
$ sed '/rose/Id' poem.txt 
Violets are blue,
Sugar is sweet,
And so are you.

Quit commands

• Exit sed without processing further input

$ # same as: seq 23 45 | head -n5
$ # remember that printing is default action if -n is not used
$ seq 23 45 | sed '5q'
23
24
25
26
27

• Q is similar to q but won't print the matching line

$ seq 23 45 | sed '5Q'
23
24
25
26

$ # useful to print from beginning of file up to but not including line matching REGEXP
$ sed '/is/Q' poem.txt 
Roses are red,
Violets are blue,

• Use tac to get all lines starting from last occurrence of search string

$ # all lines from last occurrence of '7'
$ seq 50 | tac | sed '/7/q' | tac
47
48
49
50

$ # all lines from last occurrence of '7' excluding line with '7'
$ seq 50 | tac | sed '/7/Q' | tac
48
49
50

Note

• This way of using quit commands won't work for inplace editing with multiple file input
• See this Q&A for alternate solution as well using gawk and perl instead

Negating REGEXP address

• Use ! to invert the specified address

$ # same as: sed -n '/so are/p' poem.txt
$ sed '/so are/!d' poem.txt
And so are you.

$ # same as: sed '/are/d' poem.txt
$ sed -n '/are/!p' poem.txt 
Sugar is sweet,

Combining multiple REGEXP

• See also sed manual - Multiple commands syntax for more details
• See also sed scripts section to use a file for multiple commands

$ # each command as argument to -e option
$ sed -n -e '/blue/p' -e '/you/p' poem.txt 
Violets are blue,
And so are you.

$ # each command separated by ;
$ # not all commands can be specified so
$ sed -n '/blue/p; /you/p' poem.txt 
Violets are blue,
And so are you.

$ # each command separated by literal newline character
$ # might depend on whether the shell allows such multiline command
$ sed -n '
/blue/p
/you/p
' poem.txt
Violets are blue,
And so are you.

• Use {} command grouping for logical AND

$ # same as: grep 'are' poem.txt | grep 'And'
$ # space between /REGEXP/ and {} is optional
$ sed -n '/are/ {/And/p}' poem.txt 
And so are you.

$ # same as: grep 'are' poem.txt | grep -v 'so'
$ sed -n '/are/ {/so/!p}' poem.txt 
Roses are red,
Violets are blue,

$ # same as: grep -v 'red' poem.txt | grep -v 'blue'
$ sed -n '/red/!{/blue/!p}' poem.txt 
Sugar is sweet,
And so are you.
$ # many ways to do it, use whatever feels easier to construct
$ # sed -e '/red/d' -e '/blue/d' poem.txt 
$ # grep -v -e 'red' -e 'blue' poem.txt

• Different ways to do same things. See also Alternation and Control structures

$ # multiple commands can lead to duplicatation
$ sed -n '/blue/p; /t/p' poem.txt 
Violets are blue,
Violets are blue,
Sugar is sweet,
$ # in such cases, use regular expressions instead
$ sed -nE '/blue|t/p;' poem.txt 
Violets are blue,
Sugar is sweet,

$ sed -nE '/red|blue/!p' poem.txt 
Sugar is sweet,
And so are you.

$ sed -n '/so/b; /are/p' poem.txt
Roses are red,
Violets are blue,

Filtering by line number

• Exact line number can be specified to be acted upon
• As a special case, $ indicates last line of file
• See also sed manual - Multiple commands syntax

$ # here, 2 represents the address for print command, similar to /REGEXP/p
$ # same as: head -n2 poem.txt | tail -n1
$ sed -n '2p' poem.txt 
Violets are blue,

$ # print 2nd and 4th line
$ # for `p`, `d`, `s` etc multiple commands can be specified separated by ;
$ sed -n '2p; 4p' poem.txt 
Violets are blue,
And so are you.

$ # same as: tail -n1 poem.txt
$ sed -n '$p' poem.txt 
And so are you.

$ # delete only 3rd line
$ sed '3d' poem.txt 
Roses are red,
Violets are blue,
And so are you.

• For large input files, combine p with q for speedy exit
• sed would immediately quit without processing further input lines when q is used

$ seq 3542 4623452 | sed -n '2452{p;q}'
5993

$ seq 3542 4623452 | sed -n '250p; 2452{p;q}'
3791
5993

$ # here is a sample time comparison
$ time seq 3542 4623452 | sed -n '2452{p;q}' > /dev/null 

real    0m0.003s
user    0m0.000s
sys     0m0.000s
$ time seq 3542 4623452 | sed -n '2452p' > /dev/null 

real    0m0.334s
user    0m0.396s
sys     0m0.024s

• mimicking head command using q

$ # same as: seq 23 45 | head -n5
$ # remember that printing is default action if -n is not used
$ seq 23 45 | sed '5q'
23
24
25
26
27

Print only line number

$ # gives both line number and matching line
$ grep -n 'blue' poem.txt 
2:Violets are blue,

$ # gives only line number of matching line
$ sed -n '/blue/=' poem.txt 
2

$ sed -n '/are/=' poem.txt 
1
2
4

• If needed, matching line can also be printed. But there will be newline separation

$ sed -n '/blue/{=;p}' poem.txt 
2
Violets are blue,

$ # or
$ sed -n '/blue/{p;=}' poem.txt 
Violets are blue,
2

Address range

• So far, we've seen how to filter specific line based on REGEXP and line numbers
• sed also allows to combine them to enable selecting a range of lines
• Consider the sample input file for this section

$ cat addr_range.txt 
Hello World

Good day
How are you

Just do-it
Believe it

Today is sunny
Not a bit funny
No doubt you like it too

Much ado about nothing
He he he

• Range defined by start and end REGEXP
• For other cases like getting lines without the line matching start and/or end, unbalanced start/end, when end REGEXPdoesn't match, etc see Lines between two REGEXPs section

$ sed -n '/is/,/like/p' addr_range.txt 
Today is sunny
Not a bit funny
No doubt you like it too

$ sed -n '/just/I,/believe/Ip' addr_range.txt 
Just do-it
Believe it

$ # the second REGEXP will always be checked after the line matching first address
$ sed -n '/No/,/No/p' addr_range.txt 
Not a bit funny
No doubt you like it too

$ # all the matching ranges will be printed
$ sed -n '/you/,/do/p' addr_range.txt 
How are you

Just do-it
No doubt you like it too

Much ado about nothing

• Range defined by start and end line numbers

$ # print lines numbered 3 to 7
$ sed -n '3,7p' addr_range.txt 
Good day
How are you

Just do-it
Believe it

$ # print lines from line number 13 to last line
$ sed -n '13,$p' addr_range.txt 
Much ado about nothing
He he he

$ # delete lines numbered 2 to 13
$ sed '2,13d' addr_range.txt 
Hello World
He he he

• Range defined by mix of line number and REGEXP

$ sed -n '3,/do/p' addr_range.txt 
Good day
How are you

Just do-it

$ sed -n '/Today/,$p' addr_range.txt 
Today is sunny
Not a bit funny
No doubt you like it too

Much ado about nothing
He he he

• Negating address range, just add ! to end of address range

$ # same as: seq 10 | sed '3,7d'
$ seq 10 | sed -n '3,7!p'
1
2
8
9
10

$ # same as: sed '/Today/,$d' addr_range.txt
$ sed -n '/Today/,$!p' addr_range.txt
Hello World

Good day
How are you

Just do-it
Believe it

Relative addressing

• Prefixing + to a number for second address gives relative filtering
• Similar to using grep -A<num> --no-group-separator 'REGEXP' but grep merges adjacent groups while sed does not

$ # line matching 'is' and 2 lines after
$ sed -n '/is/,+2p' addr_range.txt 
Today is sunny
Not a bit funny
No doubt you like it too

$ # note that all matching ranges will be filtered
$ sed -n '/do/,+2p' addr_range.txt 
Just do-it
Believe it

No doubt you like it too

Much ado about nothing

• The first address could be number too
• Useful when using Shell substitutions

$ sed -n '3,+4p' addr_range.txt 
Good day
How are you

Just do-it
Believe it

• Another relative format is i~j which acts on ith line and i+j, i+2j, i+3j, etc
  • 1~2 means 1st, 3rd, 5th, 7th, etc (i.e odd numbered lines)
  • 5~3 means 5th, 8th, 11th, etc

$ # match odd numbered lines
$ # for even, use 2~2
$ seq 10 | sed -n '1~2p'
1
3
5
7
9

$ # match line numbers: 2, 2+2*2, 2+3*2, etc
$ seq 10 | sed -n '2~4p'
2
6
10

• If ~j is specified after , then meaning changes completely
• After the matching line based on number or REGEXP of start address, the closest line number multiple of j will mark end address

$ # 2nd line is start address
$ # closest multiple of 4 is 4th line
$ seq 10 | sed -n '2,~4p'
2
3
4
$ # closest multiple of 4 is 8th line
$ seq 10 | sed -n '5,~4p'
5
6
7
8

$ # line matching on `Just` is 6th line, so ending is 10th line
$ sed -n '/Just/,~5p' addr_range.txt 
Just do-it
Believe it

Today is sunny
Not a bit funny

Using different delimiter for REGEXP

• / is idiomatically used as the REGEXP delimiter
  • See also a bit of history on why / is commonly used as delimiter
• But any character other than \ and newline character can be used instead
• This helps to avoid/reduce use of \

$ # instead of this
$ echo '/home/learnbyexample/reports' | sed 's/\/home\/learnbyexample\//~\//'
~/reports

$ # use a different delimiter
$ echo '/home/learnbyexample/reports' | sed 's#/home/learnbyexample/#~/#'
~/reports

• For REGEXP used in address matching, syntax is a bit different \<char>REGEXP<char>

$ printf '/foo/bar/1\n/foo/baz/1\n'
/foo/bar/1
/foo/baz/1

$ printf '/foo/bar/1\n/foo/baz/1\n' | sed -n '\;/foo/bar/;p'
/foo/bar/1

Regular Expressions

• By default, sed treats REGEXP as BRE (Basic Regular Expression)
• The -E option enables ERE (Extended Regular Expression) which in GNU sed's case only differs in how meta characters are used, no difference in functionalities
  • Initially GNU sed only had -r option to enable ERE and man sed doesn't even mention -E
  • Other sed versions use -E and grep uses -E as well. So -r won't be used in examples in this tutorial
  • See also sed manual - BRE-vs-ERE
• See sed manual - Regular Expressions for more details

Line Anchors

• Often, search must match from beginning of line or towards end of line
• For example, an integer variable declaration in C will start with optional white-space, the keyword int, white-space and then variable(s)
  • This way one can avoid matching declarations inside single line comments as well
• Similarly, one might want to match a variable at end of statement

Consider the input file and sample substitution without using any anchoring

$ cat anchors.txt 
cat and dog
too many cats around here
to concatenate, use the cmd cat
catapults laid waste to the village
just scat and quit bothering me
that is quite a fabricated tale
try the grape variety muscat

$ # without anchors, substitution will replace whereever the string is found
$ sed 's/cat/XXX/g' anchors.txt 
XXX and dog
too many XXXs around here
to conXXXenate, use the cmd XXX
XXXapults laid waste to the village
just sXXX and quit bothering me
that is quite a fabriXXXed tale
try the grape variety musXXX

• The meta character ^ forces REGEXP to match only at start of line

$ # filtering lines starting with 'cat'
$ sed -n '/^cat/p' anchors.txt 
cat and dog
catapults laid waste to the village

$ # replace only at start of line
$ # g modifier not needed as there can only be single match at start of line
$ sed 's/^cat/XXX/' anchors.txt
XXX and dog
too many cats around here
to concatenate, use the cmd cat
XXXapults laid waste to the village
just scat and quit bothering me
that is quite a fabricated tale
try the grape variety muscat

$ # add something to start of line
$ echo 'Have a good day' | sed 's/^/Hi! /'
Hi! Have a good day

• The meta character $ forces REGEXP to match only at end of line

$ # filtering lines ending with 'cat'
$ sed -n '/cat$/p' anchors.txt 
to concatenate, use the cmd cat
try the grape variety muscat

$ # replace only at end of line
$ sed 's/cat$/YYY/' anchors.txt 
cat and dog
too many cats around here
to concatenate, use the cmd YYY
catapults laid waste to the village
just scat and quit bothering me
that is quite a fabricated tale
try the grape variety musYYY

$ # add something to end of line
$ echo 'Have a good day' | sed 's/$/. Cya later/'
Have a good day. Cya later

Word Anchors

• A word character is any alphabet (irrespective of case) or any digit or the underscore character
• The word anchors help in matching or not matching boundaries of a word
  • For example, to distinguish between par, spar and apparent
• \b matches word boundary
  • \ is meta character and certain combinations like \b and \B have special meaning
• One can also use these alternatives for \b
  • \< for start of word
  • \> for end of word

$ # words ending with 'cat'
$ sed -n 's/cat\b/XXX/p' anchors.txt 
XXX and dog
to concatenate, use the cmd XXX
just sXXX and quit bothering me
try the grape variety musXXX

$ # words starting with 'cat'
$ sed -n 's/\bcat/YYY/p' anchors.txt 
YYY and dog
too many YYYs around here
to concatenate, use the cmd YYY
YYYapults laid waste to the village

$ # only whole words
$ sed -n 's/\bcat\b/ZZZ/p' anchors.txt 
ZZZ and dog
to concatenate, use the cmd ZZZ

$ # word is made up of alphabets, numbers and _
$ echo 'foo, foo_bar and foo1' | sed 's/\bfoo\b/baz/g'
baz, foo_bar and foo1

• \B is opposite of \b, i.e it doesn't match word boundaries

$ # substitute only if 'cat' is surrounded by word characters
$ sed -n 's/\Bcat\B/QQQ/p' anchors.txt 
to conQQQenate, use the cmd cat
that is quite a fabriQQQed tale

$ # substitute only if 'cat' is not start of word
$ sed -n 's/\Bcat/RRR/p' anchors.txt 
to conRRRenate, use the cmd cat
just sRRR and quit bothering me
that is quite a fabriRRRed tale
try the grape variety musRRR

$ # substitute only if 'cat' is not end of word
$ sed -n 's/cat\B/SSS/p' anchors.txt 
too many SSSs around here
to conSSSenate, use the cmd cat
SSSapults laid waste to the village
that is quite a fabriSSSed tale