The sign bit on immediates relied on the value of csr_imm_en from the previous instruction. This fixes by gating with csr_imm_en after it has been latched instead of before