Optical memories with long-term stability at high temperatures have long been pursued in azopolymers with photoinduced birefringence. In this study, we show that the residual birefringence in layer-by-layer (LbL) films made with poly[1-[4-(3-carboxy-4 hydroxyphenylazo)benzene sulfonamido]-1,2-ethanediyl, sodium salt] (PAZO) alternated with poly(allylamine hydrochloride) (PAH) can be tuned by varying the extent of electrostatic interactions with film fabrication at different pHs for PAH. The dynamics of both writing and relaxation processes could be explained with a two-stage mechanism involving the orientation of the chromophores per se and the chain movement. Upon calculating the activation energies for these processes, we demonstrate semiquantitatively that reduced electrostatic interactions in films prepared at higher pH, for which PAH is less charged, are responsible for the longer stability at high temperatures. This is attributed to orientation of PAZO chromophores via cooperative aggregation, where the presence of counterions hindered relaxation.