Overview & Challenge
The Client is a telecommunication provider offering television, internet, and telephone services. They have been operating on the market for more than 30 years. Having a fast and reliable connection, and outstanding personal service have led them to be the second-largest fiber optic provider in their country of operating. Their mission is to ensure that the latest technology developments are accessible to everyone and the services they provide are of top-notch quality.
Establishing efficient mechanisms to extract data from various and heterogeneous enterprise data sources (Oracle, MySQL, SQL Server), perform complex data transformations and calculations, and load the data into Amazon Redshift as a central enterprise data warehouse. BI tools should also use the warehouse to build a wide spectrum of reports that meet the requirements of a diverse set of business users.
Solution
The solution quickly ingests, prepares, and delivers big data in Amazon Redshift, consisting of different submodules to more effectively manage the entire lifecycle. The solution is composed of optimized ETL pipelines implemented by using SnapLogic. The common functionalities include complex query logic applicable for all integrated data types from heterogeneous systems, consuming data warehouse dimensional/fact model with highly optimized bulk operations applied.