Take the 2-minute tour ×

I am very new to Bash scripting. I am trying to write a script that works with two files. Each line of the files looks like this:

INST <_variablename_> = <_value_>;  

The two files share many variables, but they are in a different order, so I can't just diff them. What I want to do is go through the files and find all the variables that have different values, or all the variables that are specified in one file but not the other.

Here is my script so far. Again, I'm very new to Bash so please go easy on me, but also feel free to suggest improvements (I appreciate it).

while read LINE
  search_var=`echo $LINE | awk '{print $2}'`
  result_line=`grep -w $search_var file2`
  if [ $? -eq 1 ]
    echo "$line_no: not found [ $search_var ]"
    value=`echo $LINE | awk '{print $4}'`
    result_value=`echo $result_line | awk '{print $4}'`
    if [ "$value" != "$result_value" ]
      echo "$line_no: mismatch [ $search_var , $value , $result_value ]" 
  line_no=`expr $line_no + 1`
done < file1

Now here's an example of some of the output that I'm getting:

111: mismatch [ TXAREFBIASSEL , TRUE; , "TRUE"; ]
 , 4'b1100; ] [ TXTERMTRIM , 4'b1100;
113: not found [ VREFBIASMODE ]
 , 2'b00; ]ch [ CYCLE_LIMIT_SEL , 2'b00;
 , 3'b100; ]h [ FDET_LCK_CAL , 3'b101;

The first line is what I would expect (I'll deal with the quotes later). On the second, fourth, and fifth line, it looks like the final value is overwriting the "line_no: mismatch" part. And furthermore, on the second and fourth line, the values DO match--it shouldn't print anything at all!

I asked my friend about this, and his suggestion was "Do it in Perl." So I'm learning Perl right now, but I'd still like to know what's going on and why this is happening.

Thank you!


Sigh. I figured out the problem. One of the files had Unix line breaks, and the other had DOS line breaks. I actually thought this might be the case, but I also thought that vi was supposed to display some character if it opened a dos-ended file. Since they looked the same, I assumed that they were the same.

Thanks for your help and suggestions everybody!

share|improve this question
read up on the comm utility. Good luck. –  shellter Feb 26 '12 at 21:48
add comment

2 Answers

up vote 4 down vote accepted

Rather than simply replacing the Bash language with Perl, how about a paradigm shift?

diff -w <(sort file1) <(sort file2)

This will sort both files, so that the variables will appear in the same order in each, and will diff the results (ignoring whitespace differences, just for fun).

This may give you more or less what you need, without any "code" per se. Note that you could also sort the files into intermediate files and run diff on those if you find that easier...I happen to like doing it with no temporary files.

share|improve this answer
Simple and elegant, I like it. But can I expand it to deal with the TRUE vs. "TRUE" case? I guess I could just edit the files...Anyway, thank you, I might do just that. However, why is my script doing this? I'd still like to know. –  Cory G. Feb 26 '12 at 22:00
add comment

What about this? 2 is avaliable in both files and same value. other values can be parsed easily.

sort 1.txt 2.txt | uniq -c
      2 a = 10
      1 b = 20
      1 b = 40
      1 c = 10
      1 c = 30
      1 e = 50

or like this get your key and values.

sed 's|INST \(.*\) = \(.*\)|\1 = \2|' 1.txt 2.txt | sort | uniq -c
      2 a = 10
      1 b = 20
      1 b = 40
      1 c = 10
      1 c = 30
      1 e = 50
share|improve this answer
Interesting, but I don't think it worked. I got stuff like this as a result: 1 TXPRE_PRDRV_DAC = 3'b111; 1 TXPRE_PRDRV_DAC = 3'b111; 1 TXPRE_TAP_DAC = 5'b00000; 1 TXPRE_TAP_DAC = 5'b00000; Shouldn't those be combined since they match? –  Cory G. Feb 27 '12 at 0:11
put your output and run uniq on it worked. cat dnm.txt | uniq -c 2 TXPRE_PRDRV_DAC = 3'b111; 2 TXPRE_TAP_DAC = 5'b00000; –  Selman Ulug Feb 27 '12 at 0:19
But we're already piping into uniq? Why do we need to do it again? –  Cory G. Feb 27 '12 at 0:37
I want to show you uniq working with your output. I think you must check sed regexp and sort steps for your input. –  Selman Ulug Feb 27 '12 at 0:42
It works now. See my edit. Thank you! –  Cory G. Feb 27 '12 at 0:46
add comment

Your Answer


