ZS Associates is a global professional services firm providing strategy, analytics, and technology consulting to various clients across industries. Learn how ZS Associates have built a robust self-service, secure and cost-efficient data engineering platform on AWS hosting hundreds of microservices, processing millions of API and hundreds of data access requests per month.
– The architecture leverages Amazon EKS for hosting the microservices for automation workflows, UI portals & reusable data engineering jobs.
– The platform portal is leveraged to launch a variety of infrastructures for applications like batch ETL pipelines, real-time streaming pipelines, custom analytics applications, etc.
– Their automated access management workflows use Amazon S3 for storing data, Glue as a central metadata repository, IAM to control access to data sets, and Apache Ranger for fine grain access.
– The self-service data engineering platform built on AWS helped reduce the duration of their data governance workflows from weeks to a few hours, Infrastructure provisioning to a few minutes & development effort of building new data engineering pipelines by 30%.
Check out more resources for architecting in the #AWS cloud:
http://amzn.to/2ZIbygO
#AWS #AmazonWebServices #CloudComputing #ThisIsMyArchitecture
