DNA-binding proteins (DBPs) are fundamental to many key cellular processes, possessing distinct binding domains, differential affinities for single- and double-stranded DNA structures, and playing roles in fundamental biological functions such as DNA replication and gene regulation. They are intimately linked to the pathological mechanisms of diseases like neurodegenerative disorders and cancers, making their prediction pivotal for unraveling protein function and disease mechanisms. However, conventional experimental techniques for DBP identification are temporally inefficient, labor-intensive, material-intensive and pricey. Existing DNA-binding protein prediction models either lack integration with pre-trained protein language models, primarily relying on manually constructed features, or despite utilizing pre-trained language models, fail to extract sufficiently effective information from the features generated by these models. Hence, a pressing imperative persists to devise robust computational frameworks capable of precise and efficient DBP delineation. In this study, we propose a novel deep learning-based method named CNNCaps-DBP for the accurate prediction of DBPs from primary sequence information. Our methodology incorporates the pre-trained protein language model ESM C and enhances the embeddings via an attention augmented convolution module. The extracted features are then passed through a hybrid deep learning framework consisting of Capsule network and MLP to construct the final predictive model. To optimize the model's training process, we applied a dynamic learning rate scheduler utilized for lessening the risk of premature convergence and enhance the robustness of the learning process. Experimental results show that CNNCaps-DBP significantly outperforms previous models in terms of predictive performance. To further validate the robustness of the proposed model, we evaluated it on additional independent datasets, where CNNCaps-DBP consistently outperformed state-of-the-art methods. In addition, we conducted two case studies to interpret the predictions of our model, which demonstrates the strong predictive capability for DBP identification. The source code used in this study is available at: https://github.com/YZYAlex/CNNCaps-DBP.