Kafka Administrator/Data Platform Engineer

4 weeks ago


Dallas, United States CareerBuilder Full time

Copart is looking for a talented Lead or Senior Data Platform Engineer/Kafka Administrator to join our DevOps team to support our data, search and messaging platforms and other related technologies, as well as partner with the product team to ensure data integrity in the entire system. An ideal candidate brings curiosity, passion for data, and a deep understanding of the technologies behind data pipelines, big data, analytics, and deep understanding of the software development life cycle.

Responsibilities:

Lead, Design and Develop all Coparts Data Platforms
Deploy, manage, and monitor data platforms and data movement solutions.
Build scalability, fault-tolerance, security, and performance into our data platforms to meet the growing needs of our customers.
Partner with agile data development teams and use DevOps methodologies to create automated, efficient CI/CD processes to reduce the time to promote, test and deploy code.
Partner with Information Security professionals to ensure data is secure both at-rest and in-flight.
Develop, test, deploy and maintain efficient reusable patterns of streaming and batch data ingestion pipeline architectures for our data engineers to use.
Document and maintain key architecture and coding standards for supported platforms.
Act as first line support for data platforms to maintain peak operating efficiency and ensure maximum uptime.
Ensure data platforms remain current to take advantage of the latest features and support.
Communicate status of assigned work to management and follow agile practices and procedures covered by precedents, standards, or policies.
Seek guidance when direction is needed and speak up about technology risks identified and own addressing them with the team.
Influence peers on the team with questions and insight
Be open to learn, train and develop as a team.

Experience required minimum 8 years.
Extensive Experience with Kafka and related technologies (Hive, Hadoop, Spark, Storm, Zookeeper)
Experience managing Kafka with Confluent Platform and Confluent Cloud
Experience with Solr and related technologies (Solr, Elastic Search, Lucene API, etc.)
Experience with deploying, configuring, scaling, and troubleshooting data and search infrastructure.
Experience with alerting, monitoring, and auto-remediation in a large-scale distributed environment
Programming experience in Java, Python, or similar languages
Experience in support of stream processing solutions in Big Data and Kafka, Kinesis
Strong Knowledge of messaging/events architecture Concepts and PUB/SUB Pattern
Solid knowledge of Java SE, Java EE, XML, XML Schema, XSD, XSLT/XPath and JSON technologies
Experience in building and deploying Micro Services on containers such as Pivotal Foundry Cloud, Docker, or Kubernetes etc.
Experience with designing and supporting Enterprise Redis Platforms
Experience with Source control/Bug Tracking/Automated Build tools Jira, Jenkins, and Git
Experience with http web servers and load balancers
Experience with Reporting and ETL platforms

#J-18808-Ljbffr



  • Dallas, United States SSi People Full time

    Sr. Kafka engineer strong in confluent with heavy experience with on prem installations and configuration. Linux RedHat V8, Ansible, Artifactory and Bit Bucket.Key responsibilities:Jira use to manage/track activitiesInstall/configure/troubleshoot Confluent KafkaRespond to MIS teams having issues with KafkaHandle MIS requests for Kafka Pipeline processMust...


  • Dallas, United States SSi People Full time

    Sr. Kafka engineer strong in confluent with heavy experience with on prem installations and configuration. Linux RedHat V8, Ansible, Artifactory and Bit Bucket.Key responsibilities:Jira use to manage/track activitiesInstall/configure/troubleshoot Confluent KafkaRespond to MIS teams having issues with KafkaHandle MIS requests for Kafka Pipeline processMust...

  • Kafka Developer

    2 days ago


    Dallas, United States Synechron Full time

    Our ChallengeWe are seeking an experienced Senior Kafka Engineer to guide the development and implementation of robust and scalable solutions.The RoleResponsibilities:Responsible for designing and developing scalable and high-performance systems using Core Java with Confluent or Apache KafkaIntegrate Apache Kafka with various systems and applications to...

  • Kafka Developer

    1 day ago


    Dallas, United States Synechron Full time

    Our ChallengeWe are seeking an experienced Senior Kafka Engineer to guide the development and implementation of robust and scalable solutions.The RoleResponsibilities:Responsible for designing and developing scalable and high-performance systems using Core Java with Confluent or Apache KafkaIntegrate Apache Kafka with various systems and applications to...

  • Kafka Developer

    24 hours ago


    Dallas, United States Synechron Full time

    Our ChallengeWe are seeking an experienced Senior Kafka Engineer to guide the development and implementation of robust and scalable solutions.The RoleResponsibilities:Responsible for designing and developing scalable and high-performance systems using Core Java with Confluent or Apache KafkaIntegrate Apache Kafka with various systems and applications to...


  • Dallas, United States G-Research Full time

    Do you want to tackle the biggest questions in finance with near infinite compute power at your fingertips?G-Research is a leading quantitative research and technology firm, with offices in London and Dallas. We are proud to employ some of the best people in their field and to nurture their talent in a dynamic, flexible and highly stimulating culture where...


  • Dallas, United States G-Research Full time

    Do you want to tackle the biggest questions in finance with near infinite compute power at your fingertips? G-Research is a leading quantitative research and technology firm, with offices in London and Dallas. We are proud to employ some of the best people in their field and to nurture their talent in a dynamic, flexible and highly stimulating culture where...


  • Dallas, United States Tekstrom Inc Full time

    Job DescriptionJob DescriptionBig Data/ Hadoop Engineer - (Ideal candidate will be a Sr. java Developer currently working in Big Data and AWS cloud platform) Responsibilities:Work with big data analytics engineers to develop automated testing systems.Develop testing software that will generate and validate test data for big data systems.Develop testing...

  • Snowflake Data Lead

    7 hours ago


    Dallas, United States Triunity Software Full time

    Must Haves:10+ years of experience within data engineeringPython experienceSnowflake experience (developing ETL pipelines)IDMC (intelligent data management cloud) experienceKnowledge of AWSFluent in English (written and verbal)Job Description:This role will be part of a team focused on cloud transformation, modernizing analytics platforms and improving...


  • Dallas, TX, United States G-Research Full time

    Do you want to tackle the biggest questions in finance with near infinite compute power at your fingertips?G-Research is a leading quantitative research and technology firm, with offices in London and Dallas. We are proud to employ some of the best people in their field and to nurture their talent in a dynamic, flexible and highly stimulating culture where...


  • Dallas, United States Grid Dynamics Full time

    Description Position at Grid Dynamics Our customer is an American multinational technology company headquartered in Cupertino, California. Our customer is one of the world's largest technology companies based in Silicon Valley with operations all over the world. On this project, we are working with bleeding-edge big data technologies to develop a...


  • Dallas, United States TrinityRail Full time

    Trinity Industries is searching for a Senior Data Architect at our Dallas, TX HQ! Trinity Industries is at the forefront of expanding data infrastructure and analytics capabilities. In the role of Senior Data Architect, you will be instrumental in architecting, implementing, and optimizing our data processing and analytics platform based on Databricks for...

  • Sr Data Engineer

    2 weeks ago


    DALLAS, United States S&S Tech Solutions LLC Full time

    JOB DESCRIPTION S & S Tech solutions is looking for a Senior Data Engineer to join our Data Engineering Team and build world-class data solutions and applications that power crucial business decisions throughout the organization. If you are a seasoned Senior Big Data Software Engineer looking for a challenging and rewarding opportunity, we invite you to...


  • Dallas, United States Cloud Resources LLC Full time

    Senior Cloud Data EngineerLong TermREMOTE, but Candidate will have to go onsite at Dallas, TX for one week per quarter as per client requestClient: Southwest AirlinesNeed: 10+ yrsVisa : GC, GC EAD, H4 EAD, H1B, L2S, USCJob Description:Experience on data modeling and data engineering experience Experience on Python, Kafka, AWS EMR, AWS Glue, Apache Hudi on...


  • Dallas, United States Cloud Resources LLC Full time

    Senior Cloud Data EngineerLong TermREMOTE, but Candidate will have to go onsite at Dallas, TX for one week per quarter as per client requestClient: Southwest AirlinesNeed: 10+ yrsVisa : GC, GC EAD, H4 EAD, H1B, L2S, USCJob Description:Experience on data modeling and data engineering experience Experience on Python, Kafka, AWS EMR, AWS Glue, Apache Hudi on...


  • Dallas, United States Cloud Resources LLC Full time

    Senior Cloud Data Engineer Scroll down the page to see all associated job requirements, and any responsibilities successful candidates can expect. Long Term REMOTE, but Candidate will have to go onsite at Dallas, TX for one week per quarter as per client request Client: Southwest Airlines Need: 10+ yrs Visa : GC, GC EAD, H4 EAD, H1B, L2S, USC Job...

  • Lead Data Engineer

    1 day ago


    Dallas, United States Employer Direct Healthcare Full time

    About Employer Direct HealthcareEmployer Direct healthcare (EDH) is on a mission to transform access to excellent care for the moments that matter most. As the leading provider of surgical benefits for self-insured employers, we bring guided access to high-quality, affordable care for employers, their employees and the family members. We understand and have...

  • Lead Data Engineer

    24 hours ago


    Dallas, United States Employer Direct Healthcare Full time

    About Employer Direct HealthcareEmployer Direct healthcare (EDH) is on a mission to transform access to excellent care for the moments that matter most. As the leading provider of surgical benefits for self-insured employers, we bring guided access to high-quality, affordable care for employers, their employees and the family members. We understand and have...


  • Dallas, United States Request Technology, LLC Full time

    ***Hybrid, 3 days onsite, 2 days remote*** ***We are unable to sponsor as this is a permanent full-time role*** A prestigious company is looking for a Principal, Data Architecture. This principal will focus on data architecture and design to support all IT departments throughout the company. These responsibilities include design of data lakes, data...


  • Dallas, United States Sogeti Full time

    This is a full-time opportunity with Sogeti.Location - Dallas/TX, Onsite full-timeOnshore Platforms Administrator Lead• Lead the Data Program's platform Administration for Informatica EDC, AXON, IDQ, and Informatica Cloud IDMC platforms• Lead platform administration of Azure for data engineering• Lead platform administration of Power BI for...