Big Data Engineer

Atlanta, GA 30309

Posted: 01/28/2021 Employment Type: Contract Category: Big Data Job Number: 54939

Job Description


Position: Big Data Engineer

Location: Atlanta, GA

Duration: 2 + year contract

 

Skills and Experience:

              Required:
  • 6+ years of overall IT experience
  • 3+ years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark Streaming
    • Experience with real-time data processing and streaming techniques using Spark structured streaming and Kafka
    • Deep knowledge of troubleshooting and tuning Spark applications
  • 3+ years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV
  • 3+ years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or Cloud platforms e.g. AWS, GCP, etc.
  • 3+ years of experience building, testing, and optimizing ‘ Big Data’ data ingestion pipelines, architectures and data sets
  • 2+ years of experience with Python (and/or Scala) and PySpark
  • 2+ years of experience with NoSQL databases, including HBASE and/or Cassandra
  • Knowledge of Unix/Linux platform and shell scripting is a must
  • Strong analytical and problem solving skills

 

              Preferred (Not Required):  
  • Experience with Cloudera/Hortonworks HDP and HDF platforms
  • Experience with NIFI, Schema Registry, NIFI Registry
  • Strong SQL skills with ability to write intermediate complexity queries
  • Strong understanding of Relational & Dimensional modeling
  • Experience with GIT code versioning software
  • Experience with REST API and Web Services
  • Good business analyst and requirements gathering/writing skills
Apply Online
Apply with LinkedIn Apply with Facebook Apply with Twitter

Send an email reminder to:

Share This Job:

Related Jobs:

Login to save this search and get notified of similar positions.