Unix is an operating system analogous to DOS and Windows, supporting multiple concurrent users. Linux is an open-source operating system modelled on UNIX.
Ubuntu is a free and open-source Linux distribution based on Debian. Ubuntu is officially released in three editions: Desktop, Server, and Core for the internet of things devices and robots. All the editions can run on the computer alone, or in a virtual machine. Wikipedia
The Linux command line is a text interface to your computer. Often referred to as the shell, terminal, console, prompt or various other names. A Terminal is interface to the underlying operating system via a shell, usually bash. It is a command line.
Let us understand Terminal Command Prompt Example of Ubuntu Operating System.
Open Terminal, after opening terminal in Ubuntu OS, sdbhosale@sdbhosale-Inspiron-N5010:~$ such thing will be displayed on the screen in which before :~& represents your computer/laptop name.
let us type echo "Hello World" and press Enter Key
sdbhosale@sdbhosale-Inspiron-N5010:~$ echo "Hello World"
Hello World
clear command will clean up the terminal.
type clear and press enter key.
sdbhosale@sdbhosale-Inspiron-N5010:~$ clear
exit command will close the terminal or help to quit from terminal
sdbhosale@sdbhosale-Inspiron-N5010:~$ exit
pwd command will help to print working director (pwd)
sdbhosale@sdbhosale-Inspiron-N5010:~$ pwd
/home/sdbhosale
cd command will help to change directory.
ls command in home directory will help to list all files and folders from our home directory
mkdir command will help to create new folder/directory in home directory.
sdbhosale@sdbhosale-Inspiron-N5010:~$ mkdir myfolder
cd folder/directory_name and press enter key - will help to change directory.
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd myfolder/
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd myfolder/
sdbhosale@sdbhosale-Inspiron-N5010:~/myfolder$
By using pwd command we can check our current working directory.
sdbhosale@sdbhosale-Inspiron-N5010:~/myfolder$ pwd
/home/sdbhosale/myfolder
cd command without any arguemnts will help to jump from working directory to home directory.
sdbhosale@sdbhosale-Inspiron-N5010:~/myfolder$ cd
sdbhosale@sdbhosale-Inspiron-N5010:~$
sdbhosale@sdbhosale-Inspiron-N5010:~$ pwd
/home/sdbhosale
We are back to our home directory now.
cd .. command and enter key:
If suppose we are in myfolder of home directory then after pressing cd .. we can jump to above folder/directory to myfolder i.e. working directory.
.. represents directory above working directory.
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd myfolder/
sdbhosale@sdbhosale-Inspiron-N5010:~/myfolder$ cd ..
sdbhosale@sdbhosale-Inspiron-N5010:~$ pwd
/home/sdbhosale
cd ~/ Press Tab (once/twice system to sytem it may change) will help to list all files and folders in directory
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd ~/
cd ~/M and press tab key it will prompt cd ~/Music/ after pressing enter key will help to jump into Music directory.
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd ~/Music/
sdbhosale@sdbhosale-Inspiron-N5010:~/Music$
Let's create new directory Code in home directory
sdbhosale@sdbhosale-Inspiron-N5010:~$ mkdir Code
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd Code
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$
It means folder is empty.
Now let us create text file in Code directory.
touch command and file_name will help to create file in working directory.
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ touch 1.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt
ls -l Command and Enter key
sdbhosale@sdbhosale-Inspiron-N5010:~$ ls -l
drwxrwxr-x 2 sdbhosale sdbhosale 4096 May 9 22:50 Code
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd Code
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls -l
total 0
-rw-rw-r-- 1 sdbhosale sdbhosale 0 May 9 22:50 1.txt
d represents directory from drwxrwxr-x 2 sdbhosale sdbhosale 4096 May 9 22:50 Code
- represents fiel from -rw-rw-r-- 1 sdbhosale sdbhosale 0 May 9 22:50 1.txt
wc and file_name will help us to count number of lines and words from 1.txt file.
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ wc 1.txt
0 0 0 1.txt
cat command
nano editor in terminal :
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd Code
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ nano 1.txt
//Into NANO EDITOR
Hello,
Let's learn to use nano text editor
ctrl+x
Y
We will jump back to terminal by typing and saving two lines in 1.txt file.
By using nano we can create file in directory too like touch command.
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd Code
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ nano 1.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ nano 2.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt 2.txt
Let's create new directory in Code directory:
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd Code
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ mkdir Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt 2.txt Journal
Let's move 1.txt file into Journal Directory which is present in Code directory:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ mv 1.txt Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
2.txt Journal
Let's check it:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ cd Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ ls
1.txt
cp command in another way:
Copying 2.txt file which is present in Journa directoy to Code directory. Journal directory is present in Code directory:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ cd ..
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
2.txt Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ cd Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ ls
1.txt 2.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ cp -r 1.txt ~/Code
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ cd ..
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt 2.txt Journal
rm command:
rm command will help to delete files permanently from computer. Here let's try to remove file 2.txt from Code and Journal directory:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt 2.txt Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ cd Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ ls
1.txt 2.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ rm 2.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ ls
1.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ cd ..
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt 2.txt Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ rm 2.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt Journal
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd Code
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ mkdir Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt 2.txt Journal
Let's move 1.txt file into Journal Directory which is present in Code directory:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ mv 1.txt Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
2.txt Journal
Let's check it:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ cd Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ ls
1.txt
Let's understand cp command:
In Journal directory, there is 1.txt file, let us make copy of 1.txt file and save it in 2.txt file:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ cp 1.txt 2.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ ls
1.txt 2.txt
In Journal directory, there is 1.txt file, let us make copy of 1.txt file and save it in 2.txt file:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ cp 1.txt 2.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ ls
1.txt 2.txt
cp command in another way:
Copying 2.txt file which is present in Journa directoy to Code directory. Journal directory is present in Code directory:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ cd ..
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
2.txt Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ cd Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ ls
1.txt 2.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ cp -r 1.txt ~/Code
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ cd ..
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt 2.txt Journal
rm command:
rm command will help to delete files permanently from computer. Here let's try to remove file 2.txt from Code and Journal directory:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt 2.txt Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ cd Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ ls
1.txt 2.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ rm 2.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ ls
1.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code/Journal$ cd ..
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt 2.txt Journal
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ rm 2.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls
1.txt Journal
man command_name will help to display entire information for that command in terminal.
sdbhosale@sdbhosale-Inspiron-N5010:~$ man ls
Pressing q will bring us back to terminal.
apropos editor command will help to display editors which are present in terminal.
sdbhosale@sdbhosale-Inspiron-N5010:~$ apropos editor
atobm (1) - bitmap editor and converter utilities for the X Win...
audacity (1) - Graphical cross-platform audio editor
bitmap (1) - bitmap editor and converter utilities for the X Win...
bmtoa (1) - bitmap editor and converter utilities for the X Win...
ed (1) - line-oriented text editor
editor (1) - Nano's ANOther editor, an enhanced free Pico clone
editres (1) - a dynamic resource editor for X Toolkit applications
ex (1) - Vi IMproved, a programmers text editor
gedit (1) - text editor for the GNOME Desktop
gnome-text-editor (1) - text editor for the GNOME Desktop
nano (1) - Nano's ANOther editor, an enhanced free Pico clone
nm-connection-editor (1) - network connection editor for NetworkManager
openshot (1) - Non-Linear Video Editor
pico (1) - Nano's ANOther editor, an enhanced free Pico clone
psed (1) - a stream editor
red (1) - line-oriented text editor
rnano (1) - Restricted mode for Nano's ANOther editor, an enhan...
rview (1) - Vi IMproved, a programmers text editor
rvim (1) - Vi IMproved, a programmers text editor
s2p (1) - a stream editor
sed (1) - stream editor for filtering and transforming text
select-editor (1) - select your default sensible-editor from all instal...
sensible-editor (1) - sensible editing, paging, and web browsing
software-properties-gtk (1) - Software Sources List editor
texmaker (1) - A Cross-Platform LaTeX Editor
vi (1) - Vi IMproved, a programmers text editor
view (1) - Vi IMproved, a programmers text editor
vim (1) - Vi IMproved, a programmers text editor
xedit (1) - simple text editor for X
sdbhosale@sdbhosale-Inspiron-N5010:~$
sdbhosale@sdbhosale-Inspiron-N5010:~$ man ls
Pressing q will bring us back to terminal.
apropos editor command will help to display editors which are present in terminal.
sdbhosale@sdbhosale-Inspiron-N5010:~$ apropos editor
atobm (1) - bitmap editor and converter utilities for the X Win...
audacity (1) - Graphical cross-platform audio editor
bitmap (1) - bitmap editor and converter utilities for the X Win...
bmtoa (1) - bitmap editor and converter utilities for the X Win...
ed (1) - line-oriented text editor
editor (1) - Nano's ANOther editor, an enhanced free Pico clone
editres (1) - a dynamic resource editor for X Toolkit applications
ex (1) - Vi IMproved, a programmers text editor
gedit (1) - text editor for the GNOME Desktop
gnome-text-editor (1) - text editor for the GNOME Desktop
nano (1) - Nano's ANOther editor, an enhanced free Pico clone
nm-connection-editor (1) - network connection editor for NetworkManager
openshot (1) - Non-Linear Video Editor
pico (1) - Nano's ANOther editor, an enhanced free Pico clone
psed (1) - a stream editor
red (1) - line-oriented text editor
rnano (1) - Restricted mode for Nano's ANOther editor, an enhan...
rview (1) - Vi IMproved, a programmers text editor
rvim (1) - Vi IMproved, a programmers text editor
s2p (1) - a stream editor
sed (1) - stream editor for filtering and transforming text
select-editor (1) - select your default sensible-editor from all instal...
sensible-editor (1) - sensible editing, paging, and web browsing
software-properties-gtk (1) - Software Sources List editor
texmaker (1) - A Cross-Platform LaTeX Editor
vi (1) - Vi IMproved, a programmers text editor
view (1) - Vi IMproved, a programmers text editor
vim (1) - Vi IMproved, a programmers text editor
xedit (1) - simple text editor for X
sdbhosale@sdbhosale-Inspiron-N5010:~$
ls first_few_letter/letters* will help us to list all files which starts with first_few_letter/letters of the files.
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd Code
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls 1*
1.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ touch 100.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ touch 111.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls 1*
100.txt 111.txt 1.txt
ls *file_extension will help us to list all files which are of file extension which we mentioned.
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls *.txt
100.txt 111.txt 1.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls *0*
100.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls *0.*
100.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls *1.*
111.txt 1.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls *1*
100.txt 111.txt 1.txt
1. Wildcards can represent many kinds and numbers of characters.
2. The star wildcard (*) represents zero or more of any character.
3. We can use wildcards on the command line in order to work with multiple files and folders.
Let's save following states in txt file in directory and give name to states.txt:
Alabama
Alaska
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
Florida
Georgia
Hawaii
Idaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisiana
Maine
Maryland
Massachusetts
Michigan
Minnesota
Mississippi
Missouri
Montana
Nebraska
Nevada
New Hampshire
New Jersey
New Mexico
New York
North Carolina
North Dakota
Ohio
Oklahoma
Oregon
Pennsylvania
Rhode Island
South Carolina
South Dakota
Tennessee
Texas
Utah
Vermont
Virginia
Washington
West Virginia
Wisconsin
Wyoming
wc filename.txt will give us, there are 50 lines, but it’s interesting that there are 60 total words. Let’s a take a peak at the beginning of the file:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ wc states.txt
50 60 472 states.txt
head filename.txt will help us to
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ head states.txt
Alabama
Alaska
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
Florida
Georgia
grep "character" filename.txt will help us to give states which have x alphabet in the name of the states.
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ grep "x" states.txt
New Mexico
Texas
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ grep "q" states.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$
Nothing is printed to the console because the letter “q” isn’t in any of the state names.
grep "characters" filename.txt will display all states in which nia is present in the name of states.
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ grep "nia" states.txt
California
Pennsylvania
Virginia
West Virginia
The real power of regular expressions come from using metacharacters. Remember that metacharacters are characters that can be used to represent other characters.
The first metacharacter we should discuss is the "." (period) metacharacter, which represents any character. If for example I wanted to search states.txt for the character “i”, followed by any character, followed by the character “g” I could do so with the following command:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "i.g" states.txt
Virginia
Washington
West Virginia
Wyoming
Besides characters that can represent other characters, there are also metacharacters called quantifiers which allow us to specify the number of times a particular regular expression should appear in a string. One of the most basic quantifiers is "+" (plus) which represents one or more occurrences of the preceeding expression. For example the regular expression “s+as” means: one or more “s” followed by “as”. Let’s see if any of the state names match this expression:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "s+as" states.txt
Arkansas
Kansas
Besides the plus metacharacter there’s also the "*" (star) metacharacter which represents zero or more occurrences of the preceding expression. Let’s see what happens if we change "s+as" to "s*as":
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "s*as" states.txt
Alaska
Arkansas
Kansas
Massachusetts
Nebraska
Texas
Washington
Take note that the regular expression "s{2}" is equivalent to the regular expression "ss".
We could also search for state names that have between two and three adjacent occurrences of the letter “s” with the regular expression "s{2,3}":
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "s{2,3}" states.txt
Massachusetts
Mississippi
Missouri
Tennessee
Let us add two more states in states.txt file:
sssome
ssssome
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "s{2,3}" states.txt
Massachusetts
Mississippi
Missouri
Tennessee
sssome
ssssome
Some more practice:
Let us copy followin code and paste in terminal in our Code directory:
touch small.txt
echo "abcdefghijklmnopqrstuvwxyz" >> small.txt
echo "ABCDEFGHIJKLMNOPQRSTUVWXYZ" >> small.txt
echo "0123456789" >> small.txt
echo "aa bb cc" >> small.txt
echo "rhythms" >> small.txt
echo "xyz" >> small.txt
echo "abc" >> small.txt
echo "tragedy + time = humor" >> small.txt
echo "http://www.jhsph.edu/" >> small.txt
echo "#%&-=***=-&%#" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ touch small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "abcdefghijklmnopqrstuvwxyz" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "ABCDEFGHIJKLMNOPQRSTUVWXYZ" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "0123456789" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "aa bb cc" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "rhythms" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "xyz" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "abc" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "tragedy + time = humor" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "http://www.jhsph.edu/" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "#%&-=***=-&%#" >> small.txt
cat filename.txt will help us to display content of text file.
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ cat small.txt
abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ
0123456789
aa bb cc
rhythms
xyz
abc
tragedy + time = humor
http://www.jhsph.edu/
#%&-=***=-&%#
In addition to quantifiers there are also regular expressions for describing sets of characters. The \w metacharacter corresponds to all “word” characters, the \d metacharacter corresponds to all “number” characters, and the \s metacharacter corresponds to all “space” characters. Let’s take a look at using each of these metacharacters on small.txt:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "\w" small.txt
abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ
0123456789
aa bb cc
rhythms
xyz
abc
tragedy + time = humor
http://www.jhsph.edu/
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "\d" small.txt
abcdefghijklmnopqrstuvwxyz
tragedy + time = humor
http://www.jhsph.edu/
Note: output should be displayed as
0123456789 as per coursera exercise.
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "\s" small.txt
aa bb cc
tragedy + time = humor
We can see the compliment of this grep by adding the -v flag to the command:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep -v "\w" small.txt
#%&-=***=-&%#
In addition to general character sets we can also create specific character sets using square brackets ([ ]) and then including the characters we wish to match in the square brackets. For example the regular expression for the set of vowels is [aeiou]. You can also create a regular expression for the compliment of a set by including a caret (^) in the beginning of a set. For example the regular expression [^aeiou] matches all characters that are not vowels. Let’s test both on small.txt:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "[aeiou]" small.txt
abcdefghijklmnopqrstuvwxyz
aa bb cc
abc
tragedy + time = humor
http://www.jhsph.edu/
Notice that the word “rhythms” does not appear in the result (it’s the longest word without any vowels that I could think of).
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "[^aeiou]" small.txt
abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ
0123456789
aa bb cc
rhythms
xyz
abc
tragedy + time = humor
http://www.jhsph.edu/
#%&-=***=-&%#
Every line in the file is printed, because every line contains at least one non-vowel! If you want to specify a range of characters you can use a hyphen (-) inside of the square brackets. For example the regular expression [e-q] matches all of the lowercase letters between “e” and “q” in the alphabet inclusively. Case matters when you’re specifying character sets, so if you wanted to only match uppercase characters you’d need to use [E-Q]. To ignore the case of your match you could combine the character sets with the [e-qE-Q] regex (short for regular expression), or you could use the -i flag with grep to ignore the case. Note that the -i flag will work for any provided regular expression, not just character sets. Let’s take a look at some examples using the regular expressions that we just described:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "[e-q]" small.txt
abcdefghijklmnopqrstuvwxyz
rhythms
tragedy + time = humor
http://www.jhsph.edu/
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "[E-Q]" small.txt
ABCDEFGHIJKLMNOPQRSTUVWXYZ
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "[e-qE-Q]" small.txt
abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ
rhythms
tragedy + time = humor
http://www.jhsph.edu/
1. grep and egrep can be used along with regular expressions to search for patterns of text in a file.
2. Metacharacters are used in regular expressions to describe patterns of characters.
3. find can be used to search for the names of files in a directory.
sdbhosale@sdbhosale-Inspiron-N5010:~$ cd Code
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls 1*
1.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ touch 100.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ touch 111.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls 1*
100.txt 111.txt 1.txt
ls *file_extension will help us to list all files which are of file extension which we mentioned.
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls *.txt
100.txt 111.txt 1.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls *0*
100.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls *0.*
100.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls *1.*
111.txt 1.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ ls *1*
100.txt 111.txt 1.txt
1. Wildcards can represent many kinds and numbers of characters.
2. The star wildcard (*) represents zero or more of any character.
3. We can use wildcards on the command line in order to work with multiple files and folders.
Let's save following states in txt file in directory and give name to states.txt:
Alabama
Alaska
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
Florida
Georgia
Hawaii
Idaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisiana
Maine
Maryland
Massachusetts
Michigan
Minnesota
Mississippi
Missouri
Montana
Nebraska
Nevada
New Hampshire
New Jersey
New Mexico
New York
North Carolina
North Dakota
Ohio
Oklahoma
Oregon
Pennsylvania
Rhode Island
South Carolina
South Dakota
Tennessee
Texas
Utah
Vermont
Virginia
Washington
West Virginia
Wisconsin
Wyoming
wc filename.txt will give us, there are 50 lines, but it’s interesting that there are 60 total words. Let’s a take a peak at the beginning of the file:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ wc states.txt
50 60 472 states.txt
head filename.txt will help us to
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ head states.txt
Alabama
Alaska
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
Florida
Georgia
grep "character" filename.txt will help us to give states which have x alphabet in the name of the states.
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ grep "x" states.txt
New Mexico
Texas
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ grep "q" states.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$
Nothing is printed to the console because the letter “q” isn’t in any of the state names.
grep "characters" filename.txt will display all states in which nia is present in the name of states.
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ grep "nia" states.txt
California
Pennsylvania
Virginia
West Virginia
The real power of regular expressions come from using metacharacters. Remember that metacharacters are characters that can be used to represent other characters.
The first metacharacter we should discuss is the "." (period) metacharacter, which represents any character. If for example I wanted to search states.txt for the character “i”, followed by any character, followed by the character “g” I could do so with the following command:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "i.g" states.txt
Virginia
Washington
West Virginia
Wyoming
Besides characters that can represent other characters, there are also metacharacters called quantifiers which allow us to specify the number of times a particular regular expression should appear in a string. One of the most basic quantifiers is "+" (plus) which represents one or more occurrences of the preceeding expression. For example the regular expression “s+as” means: one or more “s” followed by “as”. Let’s see if any of the state names match this expression:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "s+as" states.txt
Arkansas
Kansas
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "s*as" states.txt
Alaska
Arkansas
Kansas
Massachusetts
Nebraska
Texas
Washington
Take note that the regular expression "s{2}" is equivalent to the regular expression "ss".
We could also search for state names that have between two and three adjacent occurrences of the letter “s” with the regular expression "s{2,3}":
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "s{2,3}" states.txt
Massachusetts
Mississippi
Missouri
Tennessee
Let us add two more states in states.txt file:
sssome
ssssome
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "s{2,3}" states.txt
Massachusetts
Mississippi
Missouri
Tennessee
sssome
ssssome
We could combine more quantifiers and capturing groups to dream up even more complicated regular expressions. For example, the following regular expression describes three occurrences of an “i” followed by two of any character:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "(i.{2}){3}" states.txt
Mississippi
Some more practice:
Let us copy followin code and paste in terminal in our Code directory:
touch small.txt
echo "abcdefghijklmnopqrstuvwxyz" >> small.txt
echo "ABCDEFGHIJKLMNOPQRSTUVWXYZ" >> small.txt
echo "0123456789" >> small.txt
echo "aa bb cc" >> small.txt
echo "rhythms" >> small.txt
echo "xyz" >> small.txt
echo "abc" >> small.txt
echo "tragedy + time = humor" >> small.txt
echo "http://www.jhsph.edu/" >> small.txt
echo "#%&-=***=-&%#" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ touch small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "abcdefghijklmnopqrstuvwxyz" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "ABCDEFGHIJKLMNOPQRSTUVWXYZ" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "0123456789" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "aa bb cc" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "rhythms" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "xyz" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "abc" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "tragedy + time = humor" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "http://www.jhsph.edu/" >> small.txt
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ echo "#%&-=***=-&%#" >> small.txt
cat filename.txt will help us to display content of text file.
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ cat small.txt
abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ
0123456789
aa bb cc
rhythms
xyz
abc
tragedy + time = humor
http://www.jhsph.edu/
#%&-=***=-&%#
In addition to quantifiers there are also regular expressions for describing sets of characters. The \w metacharacter corresponds to all “word” characters, the \d metacharacter corresponds to all “number” characters, and the \s metacharacter corresponds to all “space” characters. Let’s take a look at using each of these metacharacters on small.txt:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "\w" small.txt
abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ
0123456789
aa bb cc
rhythms
xyz
abc
tragedy + time = humor
http://www.jhsph.edu/
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "\d" small.txt
abcdefghijklmnopqrstuvwxyz
tragedy + time = humor
http://www.jhsph.edu/
Note: output should be displayed as
0123456789 as per coursera exercise.
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "\s" small.txt
aa bb cc
tragedy + time = humor
We can see the compliment of this grep by adding the -v flag to the command:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep -v "\w" small.txt
#%&-=***=-&%#
In addition to general character sets we can also create specific character sets using square brackets ([ ]) and then including the characters we wish to match in the square brackets. For example the regular expression for the set of vowels is [aeiou]. You can also create a regular expression for the compliment of a set by including a caret (^) in the beginning of a set. For example the regular expression [^aeiou] matches all characters that are not vowels. Let’s test both on small.txt:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "[aeiou]" small.txt
abcdefghijklmnopqrstuvwxyz
aa bb cc
abc
tragedy + time = humor
http://www.jhsph.edu/
Notice that the word “rhythms” does not appear in the result (it’s the longest word without any vowels that I could think of).
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "[^aeiou]" small.txt
abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ
0123456789
aa bb cc
rhythms
xyz
abc
tragedy + time = humor
http://www.jhsph.edu/
#%&-=***=-&%#
Every line in the file is printed, because every line contains at least one non-vowel! If you want to specify a range of characters you can use a hyphen (-) inside of the square brackets. For example the regular expression [e-q] matches all of the lowercase letters between “e” and “q” in the alphabet inclusively. Case matters when you’re specifying character sets, so if you wanted to only match uppercase characters you’d need to use [E-Q]. To ignore the case of your match you could combine the character sets with the [e-qE-Q] regex (short for regular expression), or you could use the -i flag with grep to ignore the case. Note that the -i flag will work for any provided regular expression, not just character sets. Let’s take a look at some examples using the regular expressions that we just described:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "[e-q]" small.txt
abcdefghijklmnopqrstuvwxyz
rhythms
tragedy + time = humor
http://www.jhsph.edu/
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "[E-Q]" small.txt
ABCDEFGHIJKLMNOPQRSTUVWXYZ
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "[e-qE-Q]" small.txt
abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ
rhythms
tragedy + time = humor
http://www.jhsph.edu/
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "\+" small.txt
tragedy + time = humor
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "\." small.txt
http://www.jhsph.edu/
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "^M" states.txt
Maine
Maryland
Massachusetts
Michigan
Minnesota
Mississippi
Missouri
Montana
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "North|South" states.txt
North Carolina
North Dakota
South Carolina
South Dakota
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "North|South|East|West" states.txt
North Carolina
North Dakota
South Carolina
South Dakota
West Virginia
Just two more notes on grep: you can display the line number that a match occurs on using the -n flag:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep -n "t$" states.txt
7:Connecticut
45:Vermont
We can also grep multiple files at once by providing multiple file arguments:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "New" states.txt canada.txt
states.txt:New Hampshire
states.txt:New Jersey
states.txt:New Mexico
states.txt:New York
egrep: canada.txt: No such file or directory
Metacharacter->Meaning :
. Any Character
\w A Word
\W Not a Word
\d A Digit
\D Not a Digit
\s Whitespace
\S Not Whitespace
[def] A Set of Characters
[^def] Negation of Set
[e-q] A Range of Characters
^ Beginning of String
$ End of String
\n Newline
+ One or More of Previous
* Zero or More of Previous
? Zero or One of Previous
| Either the Previous or the Following
{6} Exactly 6 of Previous
{4, 6} Between 4 and 6 or Previous
{4, } More than 4 of Previous
If you want to experiment with writing regular expressions before you use them I highly recommend playing around with http://regexr.com/
tragedy + time = humor
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "\." small.txt
http://www.jhsph.edu/
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "^M" states.txt
Maine
Maryland
Massachusetts
Michigan
Minnesota
Mississippi
Missouri
Montana
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "North|South" states.txt
North Carolina
North Dakota
South Carolina
South Dakota
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "North|South|East|West" states.txt
North Carolina
North Dakota
South Carolina
South Dakota
West Virginia
Just two more notes on grep: you can display the line number that a match occurs on using the -n flag:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep -n "t$" states.txt
7:Connecticut
45:Vermont
We can also grep multiple files at once by providing multiple file arguments:
sdbhosale@sdbhosale-Inspiron-N5010:~/Code$ egrep "New" states.txt canada.txt
states.txt:New Hampshire
states.txt:New Jersey
states.txt:New Mexico
states.txt:New York
egrep: canada.txt: No such file or directory
Metacharacter->Meaning :
. Any Character
\w A Word
\W Not a Word
\d A Digit
\D Not a Digit
\s Whitespace
\S Not Whitespace
[def] A Set of Characters
[^def] Negation of Set
[e-q] A Range of Characters
^ Beginning of String
$ End of String
\n Newline
+ One or More of Previous
* Zero or More of Previous
? Zero or One of Previous
| Either the Previous or the Following
{6} Exactly 6 of Previous
{4, 6} Between 4 and 6 or Previous
{4, } More than 4 of Previous
If you want to experiment with writing regular expressions before you use them I highly recommend playing around with http://regexr.com/
1. grep and egrep can be used along with regular expressions to search for patterns of text in a file.
2. Metacharacters are used in regular expressions to describe patterns of characters.
3. find can be used to search for the names of files in a directory.
No comments:
Post a Comment