The TT, TB, EB and BB correlations in anisotropic inflation