HUNTER Technical Resources
http://cdn.haleymarketing.com/templates/61747/logos/logo-200x200.png
http://www.htrjobs.com
http://www.htrjobs.com
Position: Big Data Engineer
Location: Atlanta, GA
Duration: 2 + year contract
Skills and Experience:
Required:
Preferred (Not Required):
Big Data Engineer
1200 Peachtree St NE , Box 227 Atlanta, GA 30309
Posted: 01/28/2021
2021-01-28
2021-05-14
Employment Type:
Contract
Category: Big Data
Job Number: 54939
Job Description
Position: Big Data Engineer
Location: Atlanta, GA
Duration: 2 + year contract
Skills and Experience:
Required:
- 6+ years of overall IT experience
- 3+ years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark Streaming
- Experience with real-time data processing and streaming techniques using Spark structured streaming and Kafka
- Deep knowledge of troubleshooting and tuning Spark applications
- 3+ years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV
- 3+ years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or Cloud platforms e.g. AWS, GCP, etc.
- 3+ years of experience building, testing, and optimizing ‘ Big Data’ data ingestion pipelines, architectures and data sets
- 2+ years of experience with Python (and/or Scala) and PySpark
- 2+ years of experience with NoSQL databases, including HBASE and/or Cassandra
- Knowledge of Unix/Linux platform and shell scripting is a must
- Strong analytical and problem solving skills
Preferred (Not Required):
- Experience with Cloudera/Hortonworks HDP and HDF platforms
- Experience with NIFI, Schema Registry, NIFI Registry
- Strong SQL skills with ability to write intermediate complexity queries
- Strong understanding of Relational & Dimensional modeling
- Experience with GIT code versioning software
- Experience with REST API and Web Services
- Good business analyst and requirements gathering/writing skills