AWK COMMAND AND QUESTIONS

TECH WOLF
4 min readAug 25, 2020

SOME COMMON AWK COMMANDS

print the content of the file using awk

awk ‘{print}' filename.txt

Adding a new line of word at top of the file

awk ‘BEGIN{printf "This is new line"}{print}’ filename.txt

Assigning a value to the variable before execution

awk -v name="Jack" ‘BEGIN{printf "Name = $s\n", name}’

Print a specific column or field of the file

awk ‘BEGIN{FS=" "}{print $3 "\t" $4}' filename.txt

For reference this is the text file we will be using:

1) Amit Physics 80

2) Rahul Maths 90

3) Shyam Biology 87

4) Kedar English 85

5) Hari History 89

Lets name it marks.txt — — →

print the line matching the pattern —

awk ‘/a/ {print $0}’marks.txt

This will print all line with a in it. For instance, in the file above it will print line 2,3,4,5

If some specific field or column of the matched line is required then it can be done in the similar fashion as for the normal awk withou matching regex

awk ‘/a/ {print $2 "\t" $3}' marks.txt

Similarly all the other patterns can be checked for its existence in the sentence.

Similarly we can find the count of the line containing a in the file.

awk ‘/a/ {++cnt} END {print "Count =" cnt}' marks.txt

Print lines with more than n(here 18) characters

awk ‘length($0) > 18' marks.txt

Standard AWK variables

awk ‘BEGIN{print "Commands=" ARGC}’ one two three four

This prints command = 5

ARGC, calculates he the no of lines commands available, Here[ awk,one,two,three,four]

User→ FS for file seperator

awk ‘BEGIN{FS=" " }’ marks.txt

We can get the number of fields in the column using NF command

awk ‘NF > 2’marks.txt

print all the lines with more than two fields

NR represents the number of present record

awk ‘NR < 3’ marks.txt

This will print the first two lines of the file

Similarly,

Output field Seperator(OFS)

awk ‘BEGIN{OFS=”|”}{print $1,$2}’

Similarly, we can set Output Row Seperator (ORS), by default the ORS is “\n”, But can be change as per requirement, for instance:

wk ‘BEGIN{ORS=”-”}{print $0}’ marks.txt
1) Amit Physics 80–2) Rahul Maths 90–3) Shyam Biology 87–4) Kedar English 85–5) Hari History 89-

The output row is seperated by “-”

match and RLENGTH

match checks if a particular substring is present in the string or not.

RLENGTH prints the length of the matched string

awk ‘BEGIN{FS=" "}{if (match($3,"Ph")){print $2}} marks.txt’`

This will print 2 as output

RSTART → This gives the position of the first match in the string

awk ‘BEGIN{FS=” “}{if (match($3,”Ph”)){print RSTART}}’ marks.txt

IGNORECASE → Search for a substring ignoring case

awk ‘BEGIN{IGNORECASE=1} /amit/’ marks.txt
1) Amit Physics 80

Arithematic Operator

awk ‘BEGIN{a=2; b=3; print “SUM=”,a+b;print “SUBS=”,a-b;print “MUL=”,a*b;print “DIV=”,a/b;print “MODULUS=a%b”;}’
SUM= 5
SUBS= -1
MUL= 6
DIV= 0.666667
MODULUS=a

Increment and Decrement Operator

awk ‘BEGIN{a=5; print a++; print a;}’

The increment and decrement operator works in the similar fashion as the programming of C or Java. Above is one such illustration.

Assignment Operator and Shorthands

assignment: → =

addition shorthand: → a += b — — → a = a + b

multiplication → a*= b

divison → a/=b

exponential: → a^=b or a**=b

Relational Operator

The signs are same as of C,C++ or Java

Logical operator are also the same

Exponential Operator

^,**

String concatenation operator

space is used for string concatentaion

awk ‘BEGIN{a="Hello ";b="World";print a b}’

will print Hello World`

Array Membership operator

awk 'BEGIN { 
arr[0] = 1; arr[1] = 2; arr[2] = 3; for (i in arr) printf "arr[%d] = %d\n", i, arr[i]
}'

Regular Expression Operator

Match

It is represented as ~. It looks for a field that contains the match string. For instance, the following example prints the lines that contain the pattern 9.

Example

[jerry]$ awk '$0 ~ 9' marks.txt

On executing this code, you get the following result −

Output

2) Rahul   Maths    90
5) Hari History 89

Not Match

It is represented as !~. It looks for a field that does not contain the match string. For instance, the following example prints the lines that do not contain the pattern 9.

Example

[jerry]$ awk '$0 !~ 9' marks.txt

On executing this code, you get the following result −

Output

1) Amit     Physics   80
3) Shyam Biology 87
4) Kedar English 85

Arrays in AWK

array_name[“index_name”] = value

Deleting an array element

delete array_name[“index_name”]

Next like all other programming language awk also got work flow. So now lets us explore the work flow with example:

AWK have if-else if -else conditional operation, it has for, while and do while like c, similarly ith has case operation like c

IF

e.g →

awk ‘BEGIN{FS=" ";count=0}{if($3 > 50){count++;}} END {print count}’

The above program will count all occurence of $3>50 and will print accordingly.

Similarly, else — if ladder can be formed lets see it how.

awk ‘BEGIN{FS=" ";count=0;count1=0}{if($3 ==50){count++;}else if($3 == 60){count1++;}else{print "Not counting";}} END {print count "\n" count1;}

In the above code, whenever 50 is found the count increases and similarly when 60 found count1 increases. and similarly adder continues as clear from the example.

FOR

awk ‘BEGIN{FS=" ";OFS=" ";i=0}{arr[i++]=$3} END {for(j=0;j<i;j++){for(k=0;k<i-j-1;k++){if(arr[k] > arr[k+1]){temp = arr[k];arr[k] = arr[k+1];arr[k+1]=temp;}}} for(j=0;j<i;j++){print arr[j]}}’`

The above program is a bubble sort illustrating foe loop in awk. In the first step we build up array by fetching data from the file. Next we ran for loop in the same fashion as we run in c,c++ or java to get our work done.

WHILE

The format of while also is same as in c or java. Lets see how:

awk ‘BEGIN{FS=" ";count=0;i=0}{while(i<=4){count = count + 1;i++;}} END {print count;}’`

This is a program illustrating the use how to use count using while loop. Similarly do — while can be performed. The skeleton of do while is shown below:

do{some_action}while(certain_condition_reached)

That’s it, I think we are done playing syntax and illustration regarding awk. For further practice I would suggest you using hacker rank platform.

Inside HackerRank search for → UNIX and pratice questions of awk over there. This will broaden your concept.

Additional Useful Resources:

Follow me on github @

--

--