Symmetries at causal boundaries in 2D and 3D gravity