The 50V is the maximum rated working voltage. The ceramic capacitor may behave a little differently depending on voltage, but that won't affect its use here.
I would not put a tantalum capacitor on the voltage regulator input. They are notorious sensitive to low impedance voltage changes, and usually fail by shorting out.
How much trouble would it be to wire three 0.1µF ceramic capacitors in parallel?