Unstructured data(e.g., images, videos, PDF files, etc.) contain semantic information, for example, the facial feature of a person and the plate number of a vehicle. There could be semantic relationships between data items which are not explicitly represented. For example, a person's face may appear in two irrelevant photos. Also, much information is represented as structured data(e.g., the person's name and age). End-users prefer to query the semantic information from unstructured data together with structured data based on the potential relationships among them. However, due to the lack of a unified database system for structured and unstructured data, developers have to comprise multiple systems and runtime together to answer these queries. In this work, we build an open-source graph database named PandaDB to consistently manage and query structured and unstructured data. We first introduce a graph data model to manage structured and unstructured data, then propose a new query language to understand the semantics of the unstructured data in the graph. Next, we develop a new cost model and related query optimization techniques to speed up the unstructured data processing pipeline. Finally, we optimize the unstructured data storage and provide the index to speed up the query processing over unstructured data. PandaDB is widely used in industrial applications like FinTech, Knowledge Graph, and Recommendation System. The results show PandaDB can support a large scale of unstructured data query processing in a graph.