Lead Big Data Engineer




Chicago, Illinois

Full-Time/Regular

RITM0019174

 

 

Job Description

 

 

Cars.com is seeking an Lead Big Data Engineer to join our organization in providing leadership for the overall, design, development and deployment of a Hadoop based platform. A leader who can efficiently execute our efforts to scale our Machine Learning products and advanced analytical products using spark as a part of Big Data team

#LI-SR1

 

 

Required Skills

 


● Experience designing and implementing large, scalable distributed systems
● Strong understanding of OO and functional programming (preferably in Java/Scala)
● Understanding of Hadoop Ecosystem (HDFS, Yarn, MapReduce, Spark, Hive, Flume,
Impala) and should be able to coach the other members of the team
● Experience with effectively modeling and storing data in HDFS and using sqoop to
import/export data out of HDFS
● Ability to write crisp and clear technical documents on data architect and execution
pipeline
● Experience with writing optimized (for memory and process time) spark jobs to process
streaming data from Kafka
● Have a good understanding of various file formats and compression techniques in HDFS
and can establish and guide patterns
● Experience with building and or using state of the art methods for scaling machine learnt
predictive models
● Proficient in scripting on Linux based operating systems.
Bonuses if you have:
● Experience working with MPP databases like Teradata, NoSql databases like
Couchbase
● Tuning HIVE and Impala queries
● Strong fundamentals in Machine Learning and have used Spark ML/R and related tools
● Developed big data applications in cloud (AWS, Azure, Google Cloud)

 

 

Required Experience

 

  • Bachelor’s or Master’s Degree in a technology related field (e.g. Engineering, Computer
    Science, etc.) required.
  •  

  • 7+ years of experience as a Java developer.
  •  

  • 1.5+ years of hands on experience with Apache Spark.
  •  

  • 3+ years of experience in implementing Hadoop solutions in data analytics space
  •