LLMs, trained on a diverse range of resources, internalize a substantial amount of knowledge. Understanding this internalized knowledge is crucial for assessing the accuracy and reliability of their outputs, guiding improvements in model design, and addressing potential biases. In this chapter, we explore two primary forms of knowledge within LLMs: taxonomy and factual knowledge. We delve into the nature of internalized knowledge, examining how these models store and utilize information. Additionally, we introduce CRAG, a comprehensive benchmark for evaluating both the internalized knowledge of LLMs and their performance in retrieved-augmented generation tasks.