To address security vulnerabilities arising from third-party libraries, security researchers maintain databases monitoring and curating vulnerability reports. Application developers can identify vulnerable libraries by directly querying the databases with their used libraries. However, the querying results of vulnerable libraries are not reliable due to the incompleteness of vulnerability reports. Thus, current approaches model the task of identifying vulnerable libraries as a named-entity-recognition (NER) task or an extreme multi-label learning (XML) task. These approaches suffer from highly inaccurate results in identifying vulnerable libraries with complex and similar names, e.g., Java libraries. To address these limitations, in this paper, we propose VulLibMiner, the first to identify vulnerable libraries from textual descriptions of both vulnerabilities and libraries, together with VulLib, a Java vulnerability dataset with their affected libraries. VulLibMiner consists of a TF-IDF matcher to efficiently screen out a small set of candidate libraries and a BERT-FNN model to identify vulnerable libraries from these candidates effectively. We evaluate VulLibMiner using four state-of-the-art/practice approaches of identifying vulnerable libraries on both their dataset named VeraJava and our VulLib dataset. Our evaluation results show that VulLibMiner can effectively identify vulnerable libraries with an average F1 score of 0.657 while the state-of-the-art/practice approaches achieve only 0.521.