Job Description
ADG is hiring a senior data engineer who will work within ADG’s Data and AI practice and build scalable data platforms for clients and internal business use cases. ADG is seeking a candidate with advanced knowledge of Google Cloud Platform (GCP), Python, SQL, Apache Spark, and distributed computing to design and deploy robust batch and real-time streaming pipelines. A senior data engineer at ADG will design and build cloud-native data platforms, orchestrate complex deliveries from a technical perspective, and combine state-of-the-art data engineering skills with cutting-edge generative AI technologies.
Experience: 7+ Years
Qualification: Bachelor’s degree in computer science or equivalent/experience.
Apply: Click Here
Responsibilities
- Create and implement scalable batch and streaming pipelines for data processing in cloud-native settings.
- Design and maintain data engineering solutions using GCP offerings like BigQuery, Cloud Dataflow, Cloud Storage, Pub/Sub, and Cloud Dataproc.
- Lead technical initiatives from concept design to production deployment, keeping in mind quality and taking responsibility.
- Understand business requirements and build the design for the data architecture, taking into account how generative AI could support.
- Mentor junior engineers and initiate discussions on PoC areas related to Data & AI.
Qualifications
- At least 7 years of professional experience in the field of data engineering, data architecture, and enterprise software systems.
- Bachelor’s degree in computer science or equivalent/experience.
- Professional experience working on Google Cloud Platform (GCP) data engineering solutions such as BigQuery, Cloud Dataflow, Cloud Storage, Pub/Sub, Cloud Dataproc, Cloud Composer, and Dataplex.
- Professional experience with Python, SQL, Apache Spark, and data management.
- Knowledge of distributed computing, Hadoop, cloud-native databases, enterprise data lakes, data lakehouses, advanced analytics, and generative AI technology.
Preferred Qualifications
- Knowledge of the practical implementation of batch and stream processing resilient pipelines on the Google Cloud Platform.
- Experience working with big data platforms such as the Hadoop ecosystem, Apache Spark, and distributed computing systems.
- Knowledge of cloud computing systems, data lakehouses, and cloud databases.
- Knowledge of the generative AI framework and implementation of AI in data engineering.
- Leadership skills in technical projects, concept validation, coaching, and technical solutions to the business.