On Rigidity of 3d Asymptotic Symmetry Algebras