Sr Data Engineer

3 days ago

O'Fallon, United States Tricon Solutions Full time

********* W2 ONLY ROLE NO C2C Only qualified candidates located near the OFallon MO area to be considered due to the position requiring an onsite presence *********

Job Title: Senior Data Engineer

Location: OFallon MO

Type: 9 months contract on W2

Role Overview:

Seeking an enthusiastic and adaptable Senior Data Engineer responsible for developing and managing data pipelines and assisting various datadriven requests across the company. The position involves a wide variety of technical skills from developing high throughput Spark jobs to tuning the performance of Big Data solutions. The ideal candidate will have experience working with largescale data and building automated data ETL processes. Data warehousing experience would be ideal.

Qualifications:

  • Excellent problemsolving skills with a solid understanding of data engineering concepts.

  • Proficient in Apache Spark with Python and related technologies.

  • Strong knowledge of SQL and performance tuning.

  • Experience in Big Data technologies like Hadoop and Oracle Exadata.

  • Solid knowledge of Linux environments and proficiency with bash scripting.

  • Effective verbal and written communication skills.

Nice to Have:

  • Knowledge or prior experience with Apache Kafka Apache Spark with Scala.

  • Orchestration with Apache Nifi Apache Airflow.

  • Java development and microservices architecture.

  • Build tools like Jenkins.

  • Log analysis and monitoring using Splunk.

  • Experience with Databricks AWS.

  • Working with large data sets with terabytes of data.

Key Responsibilities:

  • Build and maintain big data technologies environments and applications seeking opportunities for improvements and efficiencies.

  • Perform ETL (Extract Transform Load) processes based on business requirements using Apache Spark and data ingestion from Apache Kafka.

  • Work with various data platforms including Apache Hadoop Apache Ozone AWS S3 Delta Lake Apache Iceberg.

  • Utilize orchestration tools like Apache NiFi for managing and scheduling data flows efficiently.

  • Write performant SQL statements to analyze data with Hive/Impala/Oracle.

  • Full application development lifecycle (SDLC) from design to deployment.

  • Work with multiple stakeholders across teams to fulfill adhoc investigations including largescale data extraction transformation and analyses.

NOTES FROM HIRING MANAGER

  • Looking for Data engineer
  • Do you have experience in Spark
  • Do you have experience in Python
  • Do you have experience in SQL
  • Are you familiar with Hadoop

ETL,Spark,Python,SQL,Data Engineering,Hadoop

  • Data Engineer

    1 week ago

    O'Fallon, Missouri, United States Tricon Solutions Full time

    Job Title: Data EngineerLocation: O Fallon MO 63366Type: 6 months contract on W2Description:We are seeking a highly skilled Data Engineer to join our team at Tricon Solutions. As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems using Apache Spark and related...

  • Senior Data Engineer

    1 week ago

    O'Fallon, Missouri, United States MasterCard Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Mastercard. As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data pipelines and architectures to support our business operations.Your primary focus will be on building and maintaining big data technologies, environments, and...

  • Data Engineer

    1 week ago

    O'Fallon, Missouri, United States Red River Full time

    Position Overview:We are seeking a highly skilled and motivated Data Engineer to support the Defense Information Systems Agency (DISA) at Scott Air Force Base. The successful candidate will play a crucial role in designing, developing, and maintaining data infrastructure and pipelines to support the federation of structured and unstructured data across the...

  • Cloud Data Engineering Lead

    5 days ago

    O'Fallon, United States MasterCard Full time

    Job Title: Cloud Data Engineering LeadJob Summary:We are seeking a highly skilled Cloud Data Engineering Lead to drive Predictive and Prescriptive Analytics solutions for our Loyalty Rewards & Segments Program. The successful candidate will design and implement new data pipelines, transfers, and compliance infrastructure to support cloud-based data...

  • Senior Data Engineer

    2 weeks ago

    O'Fallon, Missouri, United States MasterCard Full time

    Job SummaryWe are seeking a highly skilled Senior Data Engineer to join our Customer Intelligence Data Strategy and Enablement team. The successful candidate will be responsible for designing and surfacing critical customer data visualizations and insights to our internal and external stakeholders.The ideal candidate will have experience with data modeling,...

  • Big Data Software Engineer

    4 days ago

    O'Fallon, Missouri, United States Tricon Solutions Full time

    Job Title: Software EngineerLocation: O Fallon MO 63366Type: 6 months contract on W2Description: Tricon Solutions is seeking a skilled Software Engineer to design, code, and maintain large data processing systems primarily in the Spark framework using Scala. The ideal candidate will have in-depth knowledge of algorithms and processing complexity, as well as...

  • Senior Data Processing Engineer

    1 week ago

    O'Fallon, Missouri, United States TalTeam Full time

    Job Description:At TalTeam Inc., we are seeking a highly skilled Senior Data Processing Engineer to join our team. As a key member of our engineering team, you will be responsible for designing, coding, and maintaining large data processing systems, primarily in the Spark framework using Scala. Our ideal candidate will have a deep understanding of the...

  • Senior Principal Software Engineer, Data Platforms and Engineering Services

    4 days ago

    O'Fallon, Missouri, United States MasterCard Full time

    About the RoleWe are seeking a highly skilled Senior Principal Software Engineer to join our Data Platforms and Engineering Services team. As a key member of this team, you will be responsible for driving the technology architecture, design, implementation, operation, and support of our data technology and platforms.The Mastercard Data Warehouse is the...

  • Cooperative Engineer

    3 weeks ago

    O'Fallon, Missouri, United States Rawlings Sporting Goods Company Inc Full time

    Job SummaryWe are seeking a highly motivated and detail-oriented Cooperative Engineer to join our team at Rawlings Sporting Goods Company Inc. This is an exciting opportunity to split time between Quality Assurance and Product Development, working closely with our Quality Assurance Director and Sr. Manager, Product Development.Key ResponsibilitiesPerform...

  • Big Data Software Engineer

    2 weeks ago

    O'Fallon, United States Tricon Solutions Full time

    Job Title: Software Engineer Location: O Fallon MO 63366 Type: 6 months contract on W2 Description: Seeking a Software Engineer who thrives on designing coding and maintaining large data processing systems primarily in the Spark framework using Scala. Our ideal candidate will understand the challenges of working with ...

  • Big Data Software Engineer

    6 days ago

    O'Fallon, United States Tricon Solutions Full time

    Job Title: Software Engineer Location: O Fallon MO 63366 Type: 6 months contract on W2 Description: Seeking a Software Engineer who thrives on designing coding and maintaining large data processing systems primarily in the Spark framework using Scala. Our ideal candidate will understand the challenges of working with ...

  • Principal Data Architect

    4 days ago

    O'Fallon, Missouri, United States MasterCard Full time

    Job Title: Principal Data ArchitectWe are seeking a highly skilled Principal Data Architect to lead our data engineering efforts and drive the development of advanced analytics solutions. As a key member of our team, you will be responsible for designing and implementing scalable data architectures, leading cross-functional teams, and collaborating with...

  • Operational Risk Sr Officer

    4 weeks ago

    O'Fallon, United States Hispanic Technology Executive Council Full time

    Job Title: Operational Risk Sr OfficerThe Operational Risk Sr Officer is a strategic professional who closely follows the latest trends in their field and adapts them for application within their job and the business. This role requires excellent communication skills to negotiate internally, often at a senior level, and developed communication and diplomacy...

  • Big Data Strategist

    1 month ago

    O'Fallon, Missouri, United States Accord Technologies Inc. Full time

    Job Title: Lead Big Data ConsultantWe are seeking a highly skilled Lead Big Data Consultant to join our team at Accord Technologies Inc.Job SummaryThe successful candidate will lead the implementation of machine learning production systems, develop production machine learning pipelines, and have extensive experience in data warehousing/data lake development...

  • Big Data Specialist

    5 days ago

    O'Fallon, Missouri, United States Diverse Lynx Full time

    Job Role: Big Data SpecialistJob Type: ContractLocation: RemoteJob Description:We are seeking a highly skilled Big Data Specialist to join our team at Diverse Lynx LLC. As a Big Data Specialist, you will be responsible for leading collaborative work in complex engineering projects in an Agile setting.Key Responsibilities:Develop and configure dashboards to...

  • Big Data Strategist

    1 month ago

    O'Fallon, Missouri, United States Accord Technologies Inc. Full time

    Job Title: Lead Big Data ConsultantWe are seeking a highly skilled Lead Big Data Consultant to join our team at Accord Technologies Inc.Job SummaryThe successful candidate will lead the implementation of machine learning production systems, develop production machine learning pipelines, and drive data-driven decision making across the organization.Key...

  • Senior Principal Data Architect

    3 days ago

    O'Fallon, Missouri, United States MasterCard Full time

    About the RoleWe are seeking a highly skilled Senior Principal Data Architect to join our Data Platforms and Engineering Services team at Mastercard. As a key member of our team, you will be responsible for driving the technical vision and architecture for our data platforms and engineering services ecosystem.Key ResponsibilitiesSet and drive the technical...

  • Senior Firmware Engineer

    20 hours ago

    O'Fallon, Missouri, United States Talentporte Full time

    Job DescriptionTalentporte is seeking a highly skilled Sr. Firmware Engineer to join our team. As a key member of our embedded systems team, you will be responsible for the development and maintenance of firmware for a wide range of products.ResponsibilitiesDevelop firmware against system requirements for embedded systems applied to refrigeration...

  • Sr. Technical Product Manager

    2 weeks ago

    O'Fallon, United States Adbakx Full time

    Role: Sr. Technical Product ManagerType of Employment: Full TimeLocation: O’Fallon, MO (Day 1 Onsite)Requirements:Payment domain experienceResponsible for developing inspiring platform/product visions, derived from business needs, market opportunities, and technology trendsUsing the Working Backwards method, decomposes business requirements and customer...

  • Test Engineer

    1 week ago

    O'Fallon, United States EPITEC Full time

    Graph & Tools team is looking for a Software Engineer II in Test who thrives on testing and maintaining large data processing systems, primarily in the Spark framework using Scala. Our ideal candidate will understand the challenges of working with data at scale and have a firm knowledge of algorithms and processing complexity.As a Software Engineer in Test...