Cloud Data Engineer
Arlington, VA 22203
- Design and develop moderate to complex data pipelines for reusability using AWS Glue
- Integrate AWS Glue jobs with AWS data lake technologies including S3, Step Functions, Lambda and Kinesis
- Integrate AWS Glue jobs with supporting AWS services including SQS, SMS, DynamoDB
- Setup repeatable patterns for migration of data to AWS Data Lake from on-premises, third parties, and other cloud providers
- Work with AWS tools like Athena, QuickSight, and Sagemaker to provide platforms for Cyber Fraud and Analytics team to perform necessary analytics and machine learning
- Perform technical analysis on existing code in Unix/Datastage for migration to AWS
- Prepare detailed design, architecture, mapping sheet and other ETL documents
- Prepare and validate end to end business logic and mappings.
- Prepare unit test cases, test data and perform unit testing for the jobs developed
- Use standard version control tools to check in and perform code reviews with Data Integration SME
- Use standard CI/CD pipelines to roll and deploy the code.