Is õ actually the lower-case version of Õ (just like o is the lower-case version of O)?
Exactly.
õ is lower-case version of Õ
õ is lower-case, Õ is upper-case
ä is lower-case, Ä is upper-case
ö is lower-case, Ö is upper-case
ü is lower-case, Ü is upper-case