There has been a renewed interest in generative modeling/unsupervised learning for language for downstream natural language understanding tasks. In this thesis, we explore the augmentation of standard language models with latent variables. In the ...