Agfdhyk

I have a following file:

userID PWD_HASH

test 1234

admin 1234

user 6789

abcd 5555

efgh 6666

root 1234

Using AWK,
I need to find both original lines and their duplicates with row numbers,
so that get the output like:

NR $0

1 test 1234

2 admin 1234

6 root 1234

I have tried the following, but it does not print the correct row number with NR :

awk 'n=x[$2]{print NR" "n;print NR" "$0;} {x[$2]=$0;}' file.txt

Any help would be appreciated!

$ awk '

($2 in a) {          # look for duplicates in $2

    if(a[$2]) {      # if found

        print a[$2]  # output the first, stored one

        a[$2]=""     # mark it outputed

    }

    print NR,$0      # print the duplicated one

    next             # skip the storing part that follows

}

{

    a[$2]=NR OFS $0  # store the first of each with NR and full record

}' file

Output (with the header in file):

2 test 1234

3 admin 1234

7 root 1234

Using GAWK, you can do this by below construct : -

awk '

{

    NR>1

    {

       a[$2][NR-1 " " $0];

    }

}

END {

    for (i in a)

       if(length(a[i]) > 1)

          for (j in a[i])

             print j;

}

' Input_File.txt   

Create a 2-dimensional array.

In first dimension, store PWD_HASH and in second dimension, store line number(NR-1) concatenated with whole line($0).

To display only duplicate ones, you can use length(a[i] > 1) condition.

Could you please try following.

awk '

FNR==NR{

  a[$2]++

  b[$2,FNR]=FNR==1?FNR:(FNR-1) OFS $0

  next

}

a[$2]>1{

  print b[$2,FNR]

}

'  Input_file  Input_file

Output will be as follows.

1 test 1234

2 admin 1234

6 root 1234

Explanation: Following is the explanation for above code.

awk '                                        ##Starting awk program here.

FNR==NR{                                     ##Checking condition here FNR==NR which will be TRUE when first time Input_file is being read.

  a[$2]++                                    ##Creating an array named a whose index is $1 and incrementing its value to 1 each time it sees same index.

  b[$2,FNR]=FNR==1?FNR:(FNR-1) OFS $0        ##Creating array b whose index is $2,FNR and concatenating its value to its own.

  next                                       ##Using next for skipping all further statements from here.

}

a[$2]>1{                                     ##Checking condition where value of a[$2] is greater than 1, this will be executed when 2nd time Input_file read.

  print b[$2,FNR]                            ##Printing value of array b whose index is $2,FNR here.

}

'  Input_file  Input_file                    ##Mentioning Input_file(s) names here 2 times.

Without using awk, but GNU coretutils tools:

tail -n+2 file | nl | sort -k3n | uniq -D -f2

tail remove the first line.
nl add line number.
sort based on the 3rd field.
uniq only prints duplicate based on the 3rd field.