Lead Big Data Engineer

2 weeks ago


O'Fallon, United States ShiftCode Analytics Full time

Interview : Coding test + Video

This is hybrid from day-1

Visa : USC, GC, GC EAD, H4, L2

Description :

Financial services experience preferred

  • Experience dealing with large volumes of data, from various sources, both structured and unstructured.
  • Ability to triage and talk through performance / scaling issues of dealing with data at scale.
  • Good understanding of how data will be read (file formats, partitioning, bucketing).
  • Extensive experience writing testable jobs using Spark (or equivalent) framework.
  • Programming & Scripting Languages: Java EE, Scala, Spark, SQL, Bash.
  • Web services & API standards: REST, OAuth, JSON.
  • Software Architectures (micro-services, event driven, peer-to-peer).
  • Application Security.
  • Asynchronous Pub-Sub and Point to Point Messaging Systems.
  • Advantage, if you have experience working in ETL and Hadoop Ecosystem: HBase, Solr, Spark Streaming, Kudu, Spring Boot, Spring Context, Spring Data Rest, General Cloudera experience.
  • Streaming within the Hadoop ecosystem is a plus.


  • O'Fallon, United States Diverse Lynx Full time

    Diverse Lynx LLC is a forward-thinking company that seeks an experienced Data Engineering Lead to join our team. With a focus on innovation and excellence, we aim to create cutting-edge solutions that drive business growth.Salary:$150,000 - $180,000 per annumJob Description:As a key member of our engineering team, the successful candidate will be responsible...

  • Lead Data Engineer

    7 days ago


    O'Fallon, United States ShiftCode Analytics Full time

    ShiftCode Analytics is seeking a talented Lead Data Engineer - Financial Services to drive our big data engineering efforts. The ideal candidate will have extensive experience working with big data technologies, including Hadoop, Spark, and NoSQL databases.The successful candidate will design and implement scalable big data architectures using cloud-based...


  • O'Fallon, United States MasterCard Full time

    Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation,...


  • O'Fallon, United States Diverse Lynx Full time

    Are you an experienced data professional looking for a new challenge? Do you have a passion for designing and developing innovative data solutions? We're seeking a talented Big Data Solutions Architect to join our team at Diverse Lynx LLC.Salary:$140,000 - $170,000 per annumJob Description:In this role, you will be responsible for architecting and...


  • O'Fallon, United States ShiftCode Analytics Full time

    About Our TeamWe are a dynamic team at ShiftCode Analytics, seeking a talented Data Engineering Team Lead to join us. As a key member of our team, you will be responsible for leading the development of software engineering projects and creating enhanced software solutions.About the RoleThe successful candidate will have a strong background in ETL...


  • O'Fallon, United States Motion Recruitment Full time

    About Motion RecruitmentWe are a leading recruitment agency specializing in technical placements. We are currently seeking a highly skilled Data Center Technical Lead to join our team.The ideal candidate will have 5-10+ years of experience in Power Applications Engineering, preferably with data center experience. A B.S. in Engineering or equivalent...

  • Lead, Bizops Engineer

    4 weeks ago


    O'Fallon, United States MasterCard Full time

    Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation,...


  • O'Fallon, United States ShiftCode Analytics Full time

    ShiftCode Analytics is seeking an experienced Senior Data Scientist - Hadoop and Spark to lead our big data initiatives. The ideal candidate will have a strong background in machine learning, statistical modeling, and data analysis using big data tools like Hadoop and Spark.The successful candidate will develop and implement advanced data analytics and...


  • O'Fallon, United States ShiftCode Analytics Full time

    Job DescriptionWe are looking for an experienced Lead Data Integration Developer to join our team at ShiftCode Analytics. As a key member of our data integration team, you will be responsible for designing and implementing cutting-edge ETL solutions.About the RoleThe successful candidate will have a strong background in ETL development, with experience...

  • Lead ETL Developer

    2 weeks ago


    O'Fallon, United States ShiftCode Analytics Full time

    Interview : Coidng test + Video Visa : USC, GC, GC EAD, H4, L2 This is hybrid from day-1 Description : Technical requirements: SQL and PL/SQL ETL Development (They use SSIS but Informatica is also fine) Big Plus: Financial services experience Big Data exp: Spark, Nifi pipelines, data bricks Working with Power BI and Tableau Python Job Overview: Leads...


  • O'Fallon, United States duvari group Full time

    **Job Overview:**About Us:Duvari Group is a data-driven company that values teamwork, creativity, and innovation. We are seeking a Data Solutions Engineer to join our team and help us drive business success through data insights.Estimated Salary: $110,000 - $170,000 per year**Key Qualifications:**Strong experience in SQL development and database...


  • O'Fallon, Missouri, United States T-Rex Solutions Full time

    Job OverviewT-Rex Solutions LLC is a leading business combining innovative Next Generation IT capabilities and enabling skills to provide industry-leading solutions to the federal government, focusing on cloud infrastructure and engineering, cybersecurity, and large systems integration.We are part of the winning team awarded a program to support the United...


  • O'Fallon, United States O'Fallon Casting Full time

    Engineering Quality Control LeadEstimated Salary: $90,000 - $130,000 per yearAbout the JobO'Fallon Casting is seeking an experienced Engineering Quality Control Lead to oversee our engineering quality control efforts. This role requires a strong technical background, with experience in leading cross-functional teams and implementing quality improvements.You...


  • O'Fallon, United States MasterCard Full time

    Company Overview:At Mastercard, we work to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart, and accessible. Using secure data and networks, partnerships, and passion, our innovations and solutions help individuals, financial institutions, governments, and businesses realize their...


  • O'Fallon, United States T-Rex Solutions Full time

    Job DescriptionJob DescriptionT-Rex Solutions LLC is a leading business combining innovative Next Generation IT ("NGIT") capabilities and enabling skills to provide industry-leading solutions to the federal government with a focus on cloud infrastructure and engineering, cybersecurity and large systems integration. T-Rex is part of the winning team awarded a...

  • Cloud Network Engineer

    2 months ago


    O'Fallon, United States T-Rex Solutions Full time

    Job DescriptionJob DescriptionT-Rex Solutions LLC is a leading business combining innovative Next Generation IT ("NGIT") capabilities and enabling skills to provide industry-leading solutions to the federal government with a focus on cloud infrastructure and engineering, cybersecurity and large systems integration. T-Rex is part of the winning team awarded a...

  • Sales Executive

    3 weeks ago


    O'Fallon, United States Secure Data Technologies Inc Full time

    Enterprise Sales Executive - Pure Hunter ExtraordinaireSecure Data Technologies is hunting for a fearless Enterprise Sales Executive - Someone who thrives on the thrill of the chase, lives for the close, and is relentless in pursuit of new logos This role is for the rare breed of pure hunter who eats, breathes, and dreams about winning. As the vanguard of...

  • Sales Executive

    4 weeks ago


    O'Fallon, United States Secure Data Technologies Inc Full time

    Enterprise Sales Executive - Pure Hunter Extraordinaire!Secure Data Technologies is hunting for a fearless Enterprise Sales Executive - Someone who thrives on the thrill of the chase, lives for the close, and is relentless in pursuit of new logos! This role is for the rare breed of pure hunter who eats, breathes, and dreams about winning. As the vanguard of...

  • Sales Executive

    4 weeks ago


    O'Fallon, United States Secure Data Technologies Inc Full time

    Enterprise Sales Executive - Pure Hunter Extraordinaire!Secure Data Technologies is hunting for a fearless Enterprise Sales Executive - Someone who thrives on the thrill of the chase, lives for the close, and is relentless in pursuit of new logos! This role is for the rare breed of pure hunter who eats, breathes, and dreams about winning. As the vanguard of...


  • O'Fallon, United States O'Fallon Casting Full time

    Career OpportunityO'Fallon Casting invites applications from qualified Engineering Quality Leads to join our team.The estimated salary for this role is $160,000 - $220,000 per year, based on experience.Key Responsibilities:Quality System Development: Create and maintain quality systems, policies, and procedures to ensure compliance with AS9100...