It should also be noted that binary is base 2, means counting in binary is something like:
001 = 1
010 = 2
011 = 3
100 = 4
101 = 5
...
see the pattern? This is why an AND with the number 001 will properly test if the number is odd. You can extend this to any number of bits... if the last bit is 1, it is an odd number.
And as Bekar said, dealing with bits is what a processor was made to do, so the and operation is a basic building block for not only processor instruction sets, but all digital circuit theory as well.
