But to say life isn't fair implies life has problems, or can be improved. And that (in my opinion at least) says life is significant.
And who is to say anything in life matters. Or in other words - the fact that living organisms find themselves significant doesn't mean they are. It doesn't matter whether you existed or not, just as much as stuff goes around in the universe or not.