This paper presents an exploratory study that investigates the use of various Large Language Models (LLMs) for the task of taxonomy expansion. Our objective is to enhance the taxonomical structure by querying LLMs for (1) child taxons and (2) alternative labels of existing taxons. Beginning with an incomplete taxonomy, we explore the most effective ways to prompt LLMs exploiting explicit and shared knowledge captured in manually curated taxonomies to provide context for the task at hand. We experiment with different prompting templates, well-recognized taxonomies (EuroVoc, STW, UNESCO), and popular language models (Claude, Claude3, Llama2). Our results suggest feasibility of solving of the proposed task with the modern LLMs and human oversight. Moreover, we observe certain patterns and trends in the performance of the models, noting that it was not possible to identify a single best configuration that would fit all models.