Developing an architecture for data migration from several databases to a single, streamlined source with automated workflows, improved data analytics, execution of complex business rules, and cost savings.
Client
NOT NAMED IN PPT
Goal
Enable data migration from multiple sources to a streamlined source with automated workflows and improved data analytics
Tools and Technologies
NOT LISTED SEPARATELY
Business Challenge
The client used a limited number of applications to source data from multiple types of databases. The outdated system slowed responses to critical requests. It also faced challenges in executing complex business rules. Owing to the lack of a centralized data warehouse, its data analytics could not deliver meaningful insights. Maintenance costs were also high.
The goal was to streamline the number of data sources, create a single store for data analytics, and reduce overall costs.
Solution
- Developed a customizable ETL architecture to pull data from any type of data source
- Created fully automated workflows using AWS Glue to transfer data between Stage, ODS and DWH instances
- Built a single source of information for data analytics
- Developed data analytics dashboards in Power BI
- Enabled system to perform complex business rules on the fly by integrating AWS Glue with stored procedures
Outcomes
- Optimized and consolidated multiple servers into three or fewer
- Reduced risk involved in maintenance of unsupported and dated database servers
- Improved data processing and data transfer by automating workflows
- Enhanced quality by using a single source of information for data analytics and applications
