In this paper, we investigate the association of hidden Markov models and convolutional neural networks for handwritten word recognition. The convolutional neural networks have been successfully applied to various computer vision tasks, including handwritten character recognition. In this work, we show that they can replace Gaussian mixtures to compute emission probabilities in hidden Markov models (hybrid association), or serve as feature extractor for a standard Gaussian HMM system. The proposed systems outperform a basic HMM based on either decorrelated pixels or handcrafted features. We validated the approach on two publicly available databases, and we report up to 60% (Rimes) and 35% (IAM) relative improvement compared to a Gaussian HMM based on pixel values. The final systems give comparable to recurrent neural networks, which are the best systems since 2009.