Contents
- cat --- for creating and displaying short files
- chmod --- change permissions
- cd --- change directory
- cp --- for copying files
- date --- display date
- echo --- echo argument
- ftp --- connect to a remote machine to download or upload files
- grep --- search file
- head --- display first part of file
- ls --- see what files you have
- lpr --- standard print command (see also print )
- more --- use to read files
- mkdir --- create directory
- mv --- for moving and renaming files
- ncftp --- especially good for downloading files via anonymous ftp.
- print --- custom print command (see also lpr )
- pwd --- find out what directory you are in
- rm --- remove a file
- rmdir --- remove directory
- rsh --- remote shell
- setenv --- set an environment variable
- sort --- sort file
- tail --- display last part of file
- tar --- create an archive, add or extract files
- telnet --- log in to another machine
- wc --- count characters, words, lines
Unix Commands
1. $pwd - Gives the Present working directory
2. mkdir - create a directory
Eg : mkdir dir1
Lets say you want to create more than one directory instead of invoking mkdir multiple(three) times-like.
Eg :
mkdir dir2
mkdir dir2/dir3
mkdir dir2/dir3/dir4
rmdir: Remove a directory
3. cd : Change Directory
Eg : cd dir2
Cd .. comeback from the directory
Cd\ come back to root directory
4. cal : to check calendar
5. date: Displays the system date and time.
Syntax: date [+format]
Example: Display the date in dd/mm/yy format
clear
clear
date +%d%m%y
6. Creating a file in unix :
a) Using Cat Command
cat > abc.txt -- Creates a file and press control d to exit.
CtrlC
Cat filename – to display the contents of file
Cat>>filename to append file.
b ) touch file2.txt -- Creates an empty file name file2.txt
touch File1_name File2_name File3_name -- Touch command can be used to create the multiple numbers of emplty files at the same time
cat filename : Display Content of a File
cat -b filename : Display the line numbers by using the -b
7.ls : Listing Directories and Files
All data in Unix is organized into files. All files are organized into directories. These directories are organized into a tree-like structure called the filesystem.
ls command to list out all the files or directories available in a directory.
Eg :
ls -l list with long format - show permissions
ls -a ist all files including hidden file starting with '.' (Hidden Files)
ls -R Recursive directory tree list
ls -t sort by time & date
ls -r list in reverse order
ls -ls list with long format with file size
ls -lrt
We use * to match 0 or more characters, a question mark (?) matches with a single character.
ls file* -- Displays all the files, the names of which start with file
ls *.txt -- display all the files ending with just .txt
8. whoami : Displays the user id of the currently logged-in user
9. who : Displays the list of users currently logged in
10. wc : Counting Words in a File
Eg : wc file_name
wc filename1 filename2 filename3 -- can give multiple files and get information about those files at a time.
wc -l state.txt prints the number of lines present in a file
wc -l state.txt capital.txt With more than one file name
wc -w state.txt prints the number of words present in a file
wc -c state.txt displays count of bytes present in a file
wc -m state.txt displays count of characters from a file.
wc -L demo_file to find the length of longest line in the file
11. cp -- Copying Files
syntax: cp source_file destination_file
Eg : cp filename1 filename2 : Copy the contents of filename1 to filename2
cp abc.txt /root/infa_shared/SrcFiles/def.txt
12. mv --Renaming Files
syntax: mv old_file new_file
Eg: mv filename newfile -- will rename the existing file filename to newfile.
13. rm -- Deleting Files
Eg : rm filename -- completely remove the existing file filename.
rm filename1 filename2 filename3 -- can remove multiple files at a time4
14. chmod --- (change mode) To change the file or the directory permissions,
4 stands for "read",
2 stands for "write",
1 stands for "execute", and
0 stands for "no permission."
To set the read and write permission for other users, execute the below command: chmod o+w *.txt
To Remove thre write permission for other users, execute the below command : chmod o-w *.txt
To set execute permission for all users : chmod a+x File1
chmod 777 testfile: to give all permisions
15: Sending Email : mail [-s subject] [-c cc-addr] [-b bcc-addr] to-addr
mail -s "Test Meawssage" admin@yahoo.com -- to send a test message to admin@yahoo.com.
You can connect two commands together so that the output from one program becomes the input of the next program. Two or more commands connected in this way form a pipe (|)
16 : ps -- display currently running processes.
17 : kill -- kill the current process.
18 : man: (Manual) - Interface for working with the online reference manuals. (HELP)
Syntax: man [-s section] item
Example: Show manual page for the ‘cat’ command
$ man cat
19. find: Used to search for files and directories as mentioned in the ‘expression’
Syntax: find [starting-point] [expression]
Eg :
$ find -- List all files found in the current directory and its hierarchy
$find . -name cust.dat -- Find all the files whose name is cust.dat from all directories(.)
$ find /root/infa_shared/SrcFiles -name File1.dat -- Find all the files under /root/infa_shared/SrcFiles directory with the name File1.dat
$find /root/infa_shared/SrcFiles -iname File1.dat -- Find all the files whose name is file1.dat and contains both capital and small letters in /root/infa_shared/SrcFiles directory.
find / -type d -name dir10 -- Find all directories whose name is dir10 in / (root) directory.
find . -type f -empty -- Find all the Empty Files in all directories
More about Find command read the below page : https://www.tecmint.com/35-practical-examples-of-linux-find-command/
20. du: (disk usage) Estimate disk usage is blocks
syntax: du [options] [file]
Example: $ du - Show number of blocks occupied by files in the current directory
du -sh * - summary of directories (-s) in human-readable format (-h : Byte, Kilobyte, Megabyte, Gigabyte, Terabyte and Petabyte):
du -sk * -- summary of directories (-s) in kilobytes (-k)
https://www.geeksforgeeks.org/du-command-linux-examples/
21: df: (disk free) Show number of free blocks for mounted file system
Syntax: df [options] [file]
Example: $ df -l -- Show number of free blocks in local file systems
https://www.geeksforgeeks.org/df-command-in-linux-with-examples/
Unix Filter Commands :
grep: Find lines in stdin that match a pattern and print them to stdout.
sort: Sort the lines in stdin, and print the result to stdout.
uniq: Read from stdin and print unique (that are different from the adjacent line) to stdout.
cat: Read lines from stdin (and more files), and concatenate them to stdout.
more: Read lines from stdin, and provide a paginated view to stdout.
cut: Cut specified byte, character or field from each line of stdin and print to stdout.
paste: Read lines from stdin (and more files), and paste them together line-by-line to stdout.
head: Read the first few lines from stdin (and more files) and print them to stdout.
tail: Read the last few lines from stdin (and more files) and print them to stdout.
wc: Read from stdin, and print the number of newlines, words, and bytes to stdout.
tr: Translate or delete characters read from stdin and print to stdout.
22 . grep Command -- search for string in the file. – it will show all the lines which contains the string.
searches a file or files for lines that have a certain pattern .
g/re/p which means --Gobally search for a regular expression and print all lines containing it.
Eg :
grep "this" demo_file -- Search for the given string in a single file
grep "this" demo_* -- Checking for the given string in multiple files.
grep -i "this" demo_file -- Case insensitive search using grep -i ’ (Both upper and lower case)
grep -r "this" * -- Searching in all files recursively using grep -r
(look for the string “this” in all the files in the current directory and all it’s subdirectory.)
grep -l this demo_* -- Display only the file names which matches the given pattern using grep -l
grep -n "this" demo_file -- Show line number while displaying the output using grep -n
grep “[a-e]” file1 -- Match all lines that contain any of the letters ‘a’, ‘b’, ‘c’, ‘d’ or ‘e’.
grep “[^aeiou]” file1 -- Match all lines that do not contain a vowel
grep “^hello” file1 -- Match all lines that start with ‘hello’. E.g: “hello there”
grep “done$” file1 -- Match all lines that end with ‘done’. E.g: “well done”
$grep -e "Agarwal" –e "Aggarwal" –e "Agrawal" geekfile.txt
^ -- exclude “[^aeiou]” or (start)
$ -- ending
https://www.geeksforgeeks.org/grep-command-in-unixlinux/
23 : sort -- command arranges lines of text alphabetically or numerically
Assume the below initial contents of file1.txt for the following examples
01 Priya
04 Shreya
03 Tuhina
02 Tushar
$ sort file1.txt -- Default ordering
01 Priya
02 Tushar
03Tuhina
04 Shreya
$ sort -r file1.txt -- Sort in reverse ordering:
04 Shreya
03Tuhina
02 Tushar
01 Priya
$ sort -k 2 file1.txt -- Sort by the second field:
01 Priya
04 Shreya
03 Tuhina
02 Tushar
24) uniq : command line utility that reports or filters out the repeated lines in a file.
( Displays the unique lines in the file)
Eg :
uniq kt.txt
uniq -c kt.txt -- It tells the number of times a line was repeated. (count)
uniq -d kt.txt -- It only prints the repeated lines. ( -d dupliacate)
uniq -u kt.txt -- only show lines that are not repeated ( unique)
delete duplicate lines : -- Important interview question
sort file.txt | uniq -u -- remove duplicate line in a file in unix
sort file.txt | uniq -u | cat file.txt -- remove and display the contents of the file
https://www.cyberciti.biz/faq/unix-linux-shell-removing-duplicate-lines/
25) head Command : head by default, prints the first 10 lines of each FILE to standard output
head file1.txt -- displays first 10 lines in the file1.txt
Check more options below.
https://www.geeksforgeeks.org/head-command-linux-examples/
26) tail Command : tail by default last 10 lines of each FILE to standard output
tail file1.txt -- displays last 10 lines in the file1.txt
https://www.geeksforgeeks.org/tail-command-linux-examples/
head -10 filename | tail -1 filename -- to display 10th line of the file
head -n filename | tail -1 -- to display nth line of the file
27) diff commad : This command is used to display the differences in the files by comparing the files line by line. (Compare two or more files)
Eg : diff a.txt b.txt
28) tr command (translate) : tr command in UNIX is a command line utility for translating or deleting characters.
Syntax : $ tr [OPTION] SET1 [SET2] (Translate)
Eg : $cat greekfile | tr “[a-z]” “[A-Z]” --convert lower case to upper case
$cat greekfile | tr “[A-Z]” “[a-z]” --convert upper case to lower case
https://www.geeksforgeeks.org/tr-command-in-unix-linux-with-examples/
29) cut command : which is used to extract sections from each line of input
(Cut the fileds from a file)
Syntax: cut OPTION... [FILE]...
Eg : cut -c 2,5,7 state.txt -- command prints second, fifth and seventh character from each line of the file.
https://www.geeksforgeeks.org/cut-command-linux-examples/
30) zip command : a command-line utility that helps you create Zip archives.
Eg : zip archivename.zip filename1 filename2 filename3
31) unzip command : unzip command extracts all files from the specified ZIP archive to the current directory.
eg: unzip archivename.zip
32) echo command : used to display line of text/string that are passed as an argument
eg : echo Hello world
https://linuxize.com/post/echo-command-in-linux-with-examples/
33) rev command : used to reverse the lines characterwise
eg : rev file1.txt -- it reverse the contenets of the file.
https://www.geeksforgeeks.org/rev-command-in-linux-with-examples/
How to reverse a string in unix?
echo "java" | rev -- it will display avaj
34 ) sed command (stream Editor) : it can perform lot’s of function on file like, searching, find and replace, insertion or deletion.
Syntax: sed OPTIONS... [SCRIPT] [INPUTFILE...]
Eg : sed 's/unix/linux/' geekfile.txt -- replaces the word “unix” with “linux” in the file.
https://www.geeksforgeeks.org/sed-command-in-linux-unix-with-examples/
Syntax:
sed OPTIONS... [SCRIPT] [INPUTFILE...]
Example:
Consider the below text file as an input.
$cat > geekfile.txt
unix is great os. unix is opensource. unix is free os.
learn operating system.
unix linux which one you choose.
unix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.
Sample Commands
1. Replacing or substituting string : Sed command is mostly used to replace the text in a file. The below simple sed command replaces the word “unix” with “linux” in the file.
2. $sed 's/unix/linux/' geekfile.txt
Output :
linux is great os. unix is opensource. unix is free os.
learn operating system.
linux linux which one you choose.
linux is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.
Here the “s” specifies the substitution operation. The “/” are delimiters. The “unix” is the search pattern and the “linux” is the replacement string.
By default, the sed command replaces the first occurrence of the pattern in each line and it won’t replace the second, third…occurrence in the line.
3. Replacing the nth occurrence of a pattern in a line : Use the /1, /2 etc flags to replace the first, second occurrence of a pattern in a line. The below command replaces the second occurrence of the word “unix” with “linux” in a line.
4. $sed 's/unix/linux/2' geekfile.txt
Output:
unix is great os. linux is opensource. unix is free os.
learn operating system.
unix linux which one you choose.
unix is easy to learn.linux is a multiuser os.Learn unix .unix is a powerful.
5. Replacing all the occurrence of the pattern in a line : The substitute flag /g (global replacement) specifies the sed command to replace all the occurrences of the string in the line.
6. $sed 's/unix/linux/g' geekfile.txt
Output :
linux is great os. linux is opensource. linux is free os.
learn operating system.
linux linux which one you choose.
linux is easy to learn.linux is a multiuser os.Learn linux .linux is a powerful.
7. Replacing from nth occurrence to all occurrences in a line : Use the combination of /1, /2 etc and /g to replace all the patterns from the nth occurrence of a pattern in a line. The following sed command replaces the third, fourth, fifth… “unix” word with “linux” word in a line.
8. $sed 's/unix/linux/3g' geekfile.txt
Output:
unix is great os. unix is opensource. linux is free os.
learn operating system.
unix linux which one you choose.
unix is easy to learn.unix is a multiuser os.Learn linux .linux is a powerful.
9. Parenthesize first character of each word : This sed example prints the first character of every word in parenthesis.
10. $ echo "Welcome To The Geek Stuff" | sed 's/\(\b[A-Z]\)/\(\1\)/g'
Output:
(W)elcome (T)o (T)he (G)eek (S)tuff
11. Replacing string on a specific line number : You can restrict the sed command to replace the string on a specific line number. An example is
12. $sed '3 s/unix/linux/' geekfile.txt
Output:
unix is great os. unix is opensource. unix is free os.
learn operating system.
linux linux which one you choose.
unix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.
The above sed command replaces the string only on the third line.
13. Duplicating the replaced line with /p flag : The /p print flag prints the replaced line twice on the terminal. If a line does not have the search pattern and is not replaced, then the /p prints that line only once.
14. $sed 's/unix/linux/p' geekfile.txt
Output:
linux is great os. unix is opensource. unix is free os.
linux is great os. unix is opensource. unix is free os.
learn operating system.
linux linux which one you choose.
linux linux which one you choose.
linux is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.
linux is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.
15. Printing only the replaced lines : Use the -n option along with the /p print flag to display only the replaced lines. Here the -n option suppresses the duplicate rows generated by the /p flag and prints the replaced lines only one time.
16. $sed -n 's/unix/linux/p' geekfile.txt
Output:
linux is great os. unix is opensource. unix is free os.
linux linux which one you choose.
linux is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.
If you use -n alone without /p, then the sed does not print anything.
17. Replacing string on a range of lines : You can specify a range of line numbers to the sed command for replacing a string.
18. $sed '1,3 s/unix/linux/' geekfile.txt
Output:
linux is great os. unix is opensource. unix is free os.
learn operating system.
linux linux which one you choose.
unix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.
Here the sed command replaces the lines with range from 1 to 3. Another example is
$sed '2,$ s/unix/linux/' geekfile.txt
Output:
unix is great os. unix is opensource. unix is free os.
learn operating system.
linux linux which one you choose.
linux is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful
Here $ indicates the last line in the file. So the sed command replaces the text from second line to last line in the file.
19. Deleting lines from a particular file : SED command can also be used for deleting lines from a particular file. SED command is used for performing deletion operation without even opening the file
Examples:
1. To Delete a particular line say n in this example
20. Syntax:
21. $ sed 'nd' filename.txt
22. Example:
23. $ sed '5d' filename.txt
2. To Delete a last line
Syntax:
$ sed '$d' filename.txt
3. To Delete line from range x to y
Syntax:
$ sed 'x,yd' filename.txt
Example:
$ sed '3,6d' filename.txt
4. To Delete from nth to last line
Syntax:
$ sed 'nth,$d' filename.txt
Example:
$ sed '12,$d' filename.txt
5. To Delete pattern matching line
Syntax:
$ sed '/pattern/d' filename.txt
Example:
$ sed '/abc/d' filename.txt
35) awk command :
Awk is a scripting language used for manipulating data and generating reports.The awk command programming language requires no compiling, and allows the user to use variables, numeric functions, string functions, and logical operators.
1. AWK Operations:
(a) Scans a file line by line
(b) Splits each input line into fields
(c) Compares input line/fields to pattern
(d) Performs action(s) on matched lines
2. Useful For:
(a) Transform data files
(b) Produce formatted reports
3. Programming Constructs:
(a) Format output lines
(b) Arithmetic and string operations
(c) Conditionals and loops
Syntax:
awk options 'selection _criteria {action }' input-file > output-file
Options:
-f program-file : Reads the AWK program source from the file
program-file, instead of from the
first command line argument.
-F fs : Use fs for the input field separator
Sample Commands
Example:
Consider the following text file as the input file for all cases below:
$cat > employee.txt
ajay manager account 45000
sunil clerk account 25000
varun manager sales 50000
amit manager account 47000
tarun peon sales 15000
deepak clerk sales 23000
sunil peon sales 13000
satvik director purchase 80000
1. Default behavior of Awk: By default Awk prints every line of data from the specified file.
$ awk '{print}' employee.txt
Output:
ajay manager account 45000
sunil clerk account 25000
varun manager sales 50000
amit manager account 47000
tarun peon sales 15000
deepak clerk sales 23000
sunil peon sales 13000
satvik director purchase 80000
In the above example, no pattern is given. So the actions are applicable to all the lines. Action print without any argument prints the whole line by default, so it prints all the lines of the file without failure.
2. Print the lines which match the given pattern.
$ awk '/manager/ {print}' employee.txt
Output:
ajay manager account 45000
varun manager sales 50000
amit manager account 47000
In the above example, the awk command prints all the line which matches with the ‘manager’.
3. Splitting a Line Into Fields : For each record i.e line, the awk command splits the record delimited by whitespace character by default and stores it in the $n variables. If the line has 4 words, it will be stored in $1, $2, $3 and $4 respectively. Also, $0 represents the whole line.
$ awk '{print $1,$4}' employee.txt
Output:
ajay 45000
sunil 25000
varun 50000
amit 47000
tarun 15000
deepak 23000
sunil 13000
satvik 80000
In the above example, $1 and $4 represents Name and Salary fields respectively.
Built-In Variables In Awk
Awk’s built-in variables include the field variables—$1, $2, $3, and so on ($0 is the entire line) — that break a line of text into individual words or pieces called fields.
· NR: NR command keeps a current count of the number of input records. Remember that records are usually lines. Awk command performs the pattern/action statements once for each record in a file.
· NF: NF command keeps a count of the number of fields within the current input record.
· FS: FS command contains the field separator character which is used to divide fields on the input line. The default is “white space”, meaning space and tab characters. FS can be reassigned to another character (typically in BEGIN) to change the field separator.
· RS: RS command stores the current record separator character. Since, by default, an input line is the input record, the default record separator character is a newline.
· OFS: OFS command stores the output field separator, which separates the fields when Awk prints them. The default is a blank space. Whenever print has several parameters separated with commas, it will print the value of OFS in between each parameter.
· ORS: ORS command stores the output record separator, which separates the output lines when Awk prints them. The default is a newline character. print automatically outputs the contents of ORS at the end of whatever it is given to print.
Examples:
Use of NR built-in variables (Display Line Number)
$ awk '{print NR,$0}' employee.txt
Output:
1 ajay manager account 45000
2 sunil clerk account 25000
3 varun manager sales 50000
4 amit manager account 47000
5 tarun peon sales 15000
6 deepak clerk sales 23000
7 sunil peon sales 13000
8 satvik director purchase 80000
In the above example, the awk command with NR prints all the lines along with the line number.
Use of NF built-in variables (Display Last Field)
$ awk '{print $1,$NF}' employee.txt
Output:
ajay 45000
sunil 25000
varun 50000
amit 47000
tarun 15000
deepak 23000
sunil 13000
satvik 80000
In the above example $1 represents Name and $NF represents Salary. We can get the Salary using $NF , where $NF represents last field.
Another use of NR built-in variables (Display Line From 3 to 6)
$ awk 'NR==3, NR==6 {print NR,$0}' employee.txt
Output:
3 varun manager sales 50000
4 amit manager account 47000
5 tarun peon sales 15000
6 deepak clerk sales 23000
More Examples
For the given text file:
$cat > geeksforgeeks.txt
A B C
Tarun A12 1
Man B6 2
Praveen M42 3
1) To print the first item along with the row number(NR) separated with ” – “ from each line in geeksforgeeks.txt:
$ awk '{print NR "- " $1 }' geeksforgeeks.txt
1 - A
2 - Tarun
3 – Manav
4 - Praveen
2) To return the second column/item from geeksforgeeks.txt:
The question should be:- To return the second column/item from geeksforgeeks.txt:
$ awk '{print $2}' geeksforgeeks.txt
B
A12
B6
M42
3) To print any non empty line if present
$ awk 'NF < 0' geeksforgeeks.txt
here NF should be 0 not less than and the user have to print the line number also:
correct answer : awk ‘NF == 0 {print NR}’ geeksforgeeks.txt
OR
awk ‘NF <= 0 {print NR}’ geeksforgeeks.txt
0
4) To find the length of the longest line present in the file:
$ awk '{ if (length($0) > max) max = length($0) } END { print max }' geeksforgeeks.txt
13
5) To count the lines in a file:
$ awk 'END { print NR }' geeksforgeeks.txt
3
6) Printing lines with more than 10 characters:
$ awk 'length($0) > 10' geeksforgeeks.txt
Tarun A12 1
Praveen M42 3
7) To find/check for any string in any specific column:
$ awk '{ if($3 == "B6") print $0;}' geeksforgeeks.txt
8) To print the squares of first numbers from 1 to n say 6:
$ awk 'BEGIN { for(i=1;i<=6;i++) print "square of", i, "is",i*i; }'
square of 1 is 1
square of 2 is 4
square of 3 is 9
square of 4 is 16
square of 5 is 25
square of 6 is 36
eg : awk '{print}' employee.txt -- prints every line of data from the specified file.
awk '/manager/ {print}' employee.txt -- prints all the line which matches with the ‘manager’.
https://www.geeksforgeeks.org/awk-command-unixlinux-examples/
36) history command : to give the enitre history of the comamnds which we have used.
Hisotry>a.txt
37) vi editor : https://www.tutorialspoint.com/unix/unix-vi-editor.htm
- Esc – switch to command mode
- :w – write out changes that were made
- :q – exit Vim
- :q! – exit Vim and discard any changes
- :wq – saves the changes, and exits Vim
- :x – save the changes made, and exits Vim
No comments:
Post a Comment