NiFi Data Engineer
Atlanta, GA 30303
Seeking a data engineer experience developing and deploying Data Flow pipelines and Data Extractions
This individual should be self-motivated to drive solutions and proactively create (in sufficient detail) the documentation required to support and describe technical solutions.
The candidate will work with cloud architects to establish pipeline to Azure data stores using either NiFi. Candidate will build templates for engineering processes in cloud. NiFi experience preferred.
- Develop roadmap of end to end Data Flow
- Define Data Extraction best practice and methodologies
- Develop a framework for ETL
- Define a automated testing process
- Data Validations of the extracts
2--3 years SQL, Teradata SQL, Azure, NiFi (https://nifi.apache.org/)
- The data engineer will be responsible for development, deployment, maintenance, diagnostics and support of ETL flow using NiFi.
- Responsible to configuration, CICD and create parameterized development patterns in NiFi
- Develop error handling, audit balancing in data flow using NiFi
- Have knowledge of using NiFi to process data into Azure
- Experience in Load Balancing and Performance tuning in NiFi
- Code control in NiFi using Bitbucket or git.