This paper explores key success factors for the development and implementation of a Common Data Model (CDM) for Rare Diseases (RDs) focusing on the European context. Several challenges hinder RD care and research in diagnosis, treatment, and research, including data fragmentation, lack of standardisation, and Interoperability (IOP) issues within healthcare information systems. We identify key issues and recommendations for an RD-CDM, drawing on international guidelines and existing infrastructure, to address organisational, consensus, interoperability, usage, and secondary use challenges. Based on these, we analyse the importance of balancing the scope and IOP of a CDM to cater to the unique requirements of RDs while ensuring effective data exchange and usage across systems. In conclusion, a well-designed RD-CDM can bridge gaps in RD care and research, enhance patient care and facilitate international collaborations.