As a guest user you are not logged in or recognized by your IP address. You have
access to the Front Matter, Abstracts, Author Index, Subject Index and the full
text of Open Access publications.
This study presents an efficient Extract, Transform, Load (ETL) process for synchronizing data between NoSQL time series databases and relational data warehouses in IoT environments, addressing challenges of maintaining data consistency while supporting real-time and historical analytics. Our methodology uses Azure CosmosDB for NoSQL storage and Azure Synapse for warehousing. The ETL process is optimized for high-velocity, high-volume IoT data, focusing on data modeling, transformation, and loading. Key optimizations include incremental loading, parallel processing, and data compression. Query times improved by over 99%. The system manages out-of-order data arrival through staging, windowing, and merge operations. This approach balances real-time accessibility with powerful analytics, enhancing IoT data processing across domains beyond smart homes.
This website uses cookies
We use cookies to provide you with the best possible experience. They also allow us to analyze user behavior in order to constantly improve the website for you. Info about the privacy policy of IOS Press.
This website uses cookies
We use cookies to provide you with the best possible experience. They also allow us to analyze user behavior in order to constantly improve the website for you. Info about the privacy policy of IOS Press.