Data Engineering for Analytics to Solve Business Challenges

In today's world, data is generated at an ever-increasing rate. The analytic platforms need to match this pace of generated data, digest it, and generate useful insights. The best decisions are made with informed data and as it changes, one needs to follow the signals and indicators embedded in the data. The technology space is evolving rapidly and choosing the right technology fit for the data at hand is an important decision. The next decision is to select the best architecture to provide the solution for technical challenges and helps the business improve its growth, revenue, and time to market. Spark provides a swiss army knife to handle the entire data life cycle, from ingestion to consumption. Newer offerings from the open source community around Delta and MLFlow help strengthen the data platform by making it performant, reliable, and repeatable. Often, innovation is left in proof of concept stages and does not see production because of the lack of foundational architectural components necessary for hardened and mature enterprise-grade deployments. This lost innovation translates to lost revenue and missed opportunities. This course helps students to appreciate the power of technology and skillfully apply it in practical situations in the real world. It leverages the Databricks platform on Amazon web services (AWS) to simplify the cluster setup so that students can focus on the data engineering aspects of getting the data ready for analytics.