In the NLP domain, Aspect-Based Sentiment Analysis (ABSA) has gained significant attention in recent years due to its ability to perform fine-grained sentiment analysis. A challenging task in ABSA is Aspect Sentiment Quadruplet Extraction (ASQE), which involves the extraction of aspect terms and their associated opinion terms, sentiment polarities, and categories in the form of quadruplets. However, existing studies have ignored the strong dependence among the multiple subtasks involved in ASQE. In this paper, we propose a novel Enhanced Machine Reading Comprehension (EMRC) method and formalize ASQE task as a multi-turn MRC task. Our EMRC effectively learns and utilizes the relationships among different subtasks by incorporating previously generated query answers into the current queries. We design a hierarchical category classification strategy to perform the category prediction in a structured manner, enabling the model to tackle intricate categories with ease. Furthermore, we employ the bi-directional attention mechanism, i.e., context-to-query and query-to-context attentions, to map the context into a task-aware representation. We conduct extensive experiments on two benchmark datasets. The results demonstrate that EMRC outperforms the state-of-art baselines. The source code is publicly available at unmapped: uri https://github.com/Little-Yeah/EMCR.