Introduction to Linux - A Hands on Guide | Linux Bible | Linux From Scratch | A Newbie's Getting Started Guide to Linux | Linux Command Line Cheat Sheet | More Linux eBooks



Monday, 20 April 2015

Sed Command in Linux - Delete Lines from a File

    This is the third article of the "Super sed' Series", in which we will learn how to delete lines from a file using line numbers and regular expressions. In the first two articles, we have learned, how to print lines using sed and how to append and insert lines in a file using sed. If you are familiar with sed print syntaxes, then this article would be pretty easy to understand.



For those learners who are new to sed, let's have a brief introduction of the Super sed:
  • sed stand for Stream EDitor and it being based on the ed editor, it borrows most of the commands from the ed. It was developed by Lee E. McMahon of Bell Labs.
  • sed offers large range of text transformations that include printing lines, deleting lines, editing line in-place, search and replace, appending and inserting lines, etc.
  • sed is useful whenever you need to perform common editing operations on multiple lines without using 'vi' editor.
  • Whenever sed is executed on an input file or on the contents from stdin, sed reads the file line-by-line and after removing the trailing newline, places it in the "Pattern space", where the commands are executed on them after conditions (as in case of regex matching) are verified, and then printed on the stdout.

Deleting Lines from a File using sed

Before we start, just remember two points:
  1. sed "d" command lets us print specific lines based on the line number or regex provided.
  2. When ^ means beginning of the line and $ denotes end of the line, ^$ makes a "Blank Line", very useful while removing empty lines from a file.
For our better understanding, let us have a file sedtest.txt with contents as follows:

$ cat sedtest.txt
This is line #1
It is line #2
That is line #3
While, this is line #4
It's line #5
I am line #6
Myself line #7
It's me, line #8
Hello, I am line #9
Last line, line #10

A. sed - Delete Lines with Line Number

1. Delete 'N'th line

This will remove 'N'th line in the FILE.txt.

Syntax:

sed 'N d' FILE.txt
Example:
To delete 1st line,

$ sed '1 d' sedtest.txt
It is line #2
That is line #3
While, this is line #4
It's line #5
I am line #6
Myself line #7
It's me, line #8
Hello, I am line #9
Last line, line #10
While, to remove last line,

$ sed '$ d' sedtest.txt
This is line #1
It is line #2
That is line #3
While, this is line #4
It's line #5
I am line #6
Myself line #7
It's me, line #8
Hello, I am line #9
If you run above commands and inspect the file sedtest.txt, you would find that, the original contents of that file would not change. In case you want to remove lines in the file and save the changes (i.e. edit the file in place), you will have to use the option -i as shown in below example:

$ sed -i '5 d' sedtest.txt
$ cat sedtest.txt
This is line #1
It is line #2
That is line #3
While, this is line #4
I am line #6
Myself line #7
It's me, line #8
Hello, I am line #9
Last line, line #10

2. Delete all Lines starting from 'M'th up to 'N'th

This will remove the block of lines starting at line number M and ending at line number N.

Syntax:

sed 'M,N d' FILE.txt
Example:
To delete block of lines from 3rd line to 8th line.

$ sed '3,8 d' sedtest.txt
This is line #1
It is line #2
Hello, I am line #9
Last line, line #10
Similarly, in order to delete lines starting from 5th up to last line, you would run-

$ sed '5,$ d' sedtest.txt
This is line #1
It is line #2
That is line #3
While, this is line #4

3. Delete Every 'M'th Line Starting from 'N'th Line

This will start from Nth line and delete every Mth line coming after that. Note that, Nth line will also be deleted.

Syntax:

sed 'N~M d' FILE.txt
Example:
To delete every alternate line staring from 2nd one.

sed '2~2 d' sedtest.txt
This is line #1
That is line #3
It's line #5
Myself line #7
Hello, I am line #9

B. sed - Delete Lines using Regular Expression/Pattern

1. Delete lines containing a specific Pattern

This will delete all those lines that contain the pattern provided.

Syntax:

sed '/PATTERN/ d' FILE.txt
Example:

$ sed '/It/ d' sedtest.txt
This is line #1
That is line #3
While, this is line #4
I am line #6
Myself line #7
Hello, I am line #9
Last line, line #10

2. Delete all those lines not containing the Pattern

This will delete all those lines which do not include the pattern provided.

Syntax:

sed '/PATTERN/ !d' FILE.txt
Example:

$ sed '/It/ !d' sedtest.txt
It is line #2
It's line #5
It's me, line #8

3. Delete block of lines starting from pattern matching line

This will remove the lines from the one where pattern matches, till 'N'th line.

Syntax:

sed '/PATTERN/,N d' FILE.txt
Example:

$ sed '/5/,8 d' sedtest.txt
This is line #1
It is line #2
That is line #3
While, this is line #4
Hello, I am line #9
Last line, line #10
Similarly, so as to delete all the lines starting from pattern matching line till the end, you would use '/PATTERN/,$ d' as follows -

$ sed '/5/,$ d' sedtest.txt
This is line #1
It is line #2
That is line #3
While, this is line #4

4. Delete block of lines starting from Nth and ending at pattern matching line

This will delete the lines starting from the 'N'th line, till the one where pattern matches.

Syntax:

sed 'N,/PATTERN/ d' FILE.txt
Example:

$ sed '4,/Hello/ d' sedtest.txt
This is line #1
It is line #2
That is line #3
Last line, line #10

5. Delete block of lines between two Pattern matches

This will start deleting lines from 1st pattern match till 2nd pattern match.

Syntax:

sed '/PATTERN1/,/PATTERN2/ d' FILE.txt
Example:

$ sed  '/While/,/Myself/ d' sedtest.txt
This is line #1
It is line #2
That is line #3
It's me, line #8
Hello, I am line #9
Last line, line #10

6. Deleting all blank lines

Syntax:

sed '^$ d' FILE.txt
7. Deleting lines containing either of two patterns
Syntax:

sed '/PATTERN1\|PATTERN2/ d' FILE.txt
Example:

$ sed  '/While\|Myself/ d' sedtest.txt
This is line #1
It is line #2
That is line #3
It's line #5
I am line #6
It's me, line #8
Hello, I am line #9
Last line, line #10
    That was all about the third article on sed command. More articles on sed are coming soon. So, stay tuned. Of course, do not forget to share your feedback in the comment section below.

7 comments:

  1. thanks a lot.really useful.

    ReplyDelete
  2. Thank you very much. I find it very educational, I like to have patience to explain.

    Sincerely,
    Carlos Pons Fumanal - Spain

    ReplyDelete
  3. why sed -i '1d' file.text and sed '^$ d' file.text is not working in my unix os?

    ReplyDelete
  4. How to delete the next line (it is a blank line) matching a pattern?

    ReplyDelete
  5. Very straightforward and helpful! Thx!

    ReplyDelete
  6. Thanks. Very nice tutorial. Is there a way to delete specific lines counting backwards? what i mean is, if a file has n lines, how can we delete, let's say, lines no. n-2 and n-4 ? i tried sed -i '$-2 d ; $-4 d ' filename but it doesn't work.

    ReplyDelete