Kafka Administrator/Data Platform Engineer
4 weeks ago
Copart is looking for a talented Lead or Senior Data Platform Engineer/Kafka Administrator to join our DevOps team to support our data, search and messaging platforms and other related technologies, as well as partner with the product team to ensure data integrity in the entire system. An ideal candidate brings curiosity, passion for data, and a deep understanding of the technologies behind data pipelines, big data, analytics, and deep understanding of the software development life cycle.
Responsibilities:
Lead, Design and Develop all Coparts Data Platforms
Deploy, manage, and monitor data platforms and data movement solutions.
Build scalability, fault-tolerance, security, and performance into our data platforms to meet the growing needs of our customers.
Partner with agile data development teams and use DevOps methodologies to create automated, efficient CI/CD processes to reduce the time to promote, test and deploy code.
Partner with Information Security professionals to ensure data is secure both at-rest and in-flight.
Develop, test, deploy and maintain efficient reusable patterns of streaming and batch data ingestion pipeline architectures for our data engineers to use.
Document and maintain key architecture and coding standards for supported platforms.
Act as first line support for data platforms to maintain peak operating efficiency and ensure maximum uptime.
Ensure data platforms remain current to take advantage of the latest features and support.
Communicate status of assigned work to management and follow agile practices and procedures covered by precedents, standards, or policies.
Seek guidance when direction is needed and speak up about technology risks identified and own addressing them with the team.
Influence peers on the team with questions and insight
Be open to learn, train and develop as a team.
Experience required minimum 8 years.
Extensive Experience with Kafka and related technologies (Hive, Hadoop, Spark, Storm, Zookeeper)
Experience managing Kafka with Confluent Platform and Confluent Cloud
Experience with Solr and related technologies (Solr, Elastic Search, Lucene API, etc.)
Experience with deploying, configuring, scaling, and troubleshooting data and search infrastructure.
Experience with alerting, monitoring, and auto-remediation in a large-scale distributed environment
Programming experience in Java, Python, or similar languages
Experience in support of stream processing solutions in Big Data and Kafka, Kinesis
Strong Knowledge of messaging/events architecture Concepts and PUB/SUB Pattern
Solid knowledge of Java SE, Java EE, XML, XML Schema, XSD, XSLT/XPath and JSON technologies
Experience in building and deploying Micro Services on containers such as Pivotal Foundry Cloud, Docker, or Kubernetes etc.
Experience with designing and supporting Enterprise Redis Platforms
Experience with Source control/Bug Tracking/Automated Build tools Jira, Jenkins, and Git
Experience with http web servers and load balancers
Experience with Reporting and ETL platforms
#J-18808-Ljbffr
-
Kafka Confluent Admin
1 week ago
Dallas, United States SSi People Full timeSr. Kafka engineer strong in confluent with heavy experience with on prem installations and configuration. Linux RedHat V8, Ansible, Artifactory and Bit Bucket.Key responsibilities:Jira use to manage/track activitiesInstall/configure/troubleshoot Confluent KafkaRespond to MIS teams having issues with KafkaHandle MIS requests for Kafka Pipeline processMust...
-
Kafka Confluent Admin
1 week ago
Dallas, United States SSi People Full timeSr. Kafka engineer strong in confluent with heavy experience with on prem installations and configuration. Linux RedHat V8, Ansible, Artifactory and Bit Bucket.Key responsibilities:Jira use to manage/track activitiesInstall/configure/troubleshoot Confluent KafkaRespond to MIS teams having issues with KafkaHandle MIS requests for Kafka Pipeline processMust...
-
Kafka Developer
2 days ago
Dallas, United States Synechron Full timeOur ChallengeWe are seeking an experienced Senior Kafka Engineer to guide the development and implementation of robust and scalable solutions.The RoleResponsibilities:Responsible for designing and developing scalable and high-performance systems using Core Java with Confluent or Apache KafkaIntegrate Apache Kafka with various systems and applications to...
-
Kafka Developer
1 day ago
Dallas, United States Synechron Full timeOur ChallengeWe are seeking an experienced Senior Kafka Engineer to guide the development and implementation of robust and scalable solutions.The RoleResponsibilities:Responsible for designing and developing scalable and high-performance systems using Core Java with Confluent or Apache KafkaIntegrate Apache Kafka with various systems and applications to...
-
Kafka Developer
24 hours ago
Dallas, United States Synechron Full timeOur ChallengeWe are seeking an experienced Senior Kafka Engineer to guide the development and implementation of robust and scalable solutions.The RoleResponsibilities:Responsible for designing and developing scalable and high-performance systems using Core Java with Confluent or Apache KafkaIntegrate Apache Kafka with various systems and applications to...
-
Analytics Platform Engineer
3 weeks ago
Dallas, United States G-Research Full timeDo you want to tackle the biggest questions in finance with near infinite compute power at your fingertips?G-Research is a leading quantitative research and technology firm, with offices in London and Dallas. We are proud to employ some of the best people in their field and to nurture their talent in a dynamic, flexible and highly stimulating culture where...
-
Analytics Platform Engineer
1 month ago
Dallas, United States G-Research Full timeDo you want to tackle the biggest questions in finance with near infinite compute power at your fingertips? G-Research is a leading quantitative research and technology firm, with offices in London and Dallas. We are proud to employ some of the best people in their field and to nurture their talent in a dynamic, flexible and highly stimulating culture where...
-
Big Data/Hadoop Engineer
4 weeks ago
Dallas, United States Tekstrom Inc Full timeJob DescriptionJob DescriptionBig Data/ Hadoop Engineer - (Ideal candidate will be a Sr. java Developer currently working in Big Data and AWS cloud platform) Responsibilities:Work with big data analytics engineers to develop automated testing systems.Develop testing software that will generate and validate test data for big data systems.Develop testing...
-
Snowflake Data Lead
7 hours ago
Dallas, United States Triunity Software Full timeMust Haves:10+ years of experience within data engineeringPython experienceSnowflake experience (developing ETL pipelines)IDMC (intelligent data management cloud) experienceKnowledge of AWSFluent in English (written and verbal)Job Description:This role will be part of a team focused on cloud transformation, modernizing analytics platforms and improving...
-
Analytics Platform Engineer
1 week ago
Dallas, TX, United States G-Research Full timeDo you want to tackle the biggest questions in finance with near infinite compute power at your fingertips?G-Research is a leading quantitative research and technology firm, with offices in London and Dallas. We are proud to employ some of the best people in their field and to nurture their talent in a dynamic, flexible and highly stimulating culture where...
-
Senior Big Data Engineer
6 days ago
Dallas, United States Grid Dynamics Full timeDescription Position at Grid Dynamics Our customer is an American multinational technology company headquartered in Cupertino, California. Our customer is one of the world's largest technology companies based in Silicon Valley with operations all over the world. On this project, we are working with bleeding-edge big data technologies to develop a...
-
Senior Data Architect
6 days ago
Dallas, United States TrinityRail Full timeTrinity Industries is searching for a Senior Data Architect at our Dallas, TX HQ! Trinity Industries is at the forefront of expanding data infrastructure and analytics capabilities. In the role of Senior Data Architect, you will be instrumental in architecting, implementing, and optimizing our data processing and analytics platform based on Databricks for...
-
Sr Data Engineer
2 weeks ago
DALLAS, United States S&S Tech Solutions LLC Full timeJOB DESCRIPTION S & S Tech solutions is looking for a Senior Data Engineer to join our Data Engineering Team and build world-class data solutions and applications that power crucial business decisions throughout the organization. If you are a seasoned Senior Big Data Software Engineer looking for a challenging and rewarding opportunity, we invite you to...
-
Senior Cloud Data Engineer
2 weeks ago
Dallas, United States Cloud Resources LLC Full timeSenior Cloud Data EngineerLong TermREMOTE, but Candidate will have to go onsite at Dallas, TX for one week per quarter as per client requestClient: Southwest AirlinesNeed: 10+ yrsVisa : GC, GC EAD, H4 EAD, H1B, L2S, USCJob Description:Experience on data modeling and data engineering experience Experience on Python, Kafka, AWS EMR, AWS Glue, Apache Hudi on...
-
Senior Cloud Data Engineer
2 weeks ago
Dallas, United States Cloud Resources LLC Full timeSenior Cloud Data EngineerLong TermREMOTE, but Candidate will have to go onsite at Dallas, TX for one week per quarter as per client requestClient: Southwest AirlinesNeed: 10+ yrsVisa : GC, GC EAD, H4 EAD, H1B, L2S, USCJob Description:Experience on data modeling and data engineering experience Experience on Python, Kafka, AWS EMR, AWS Glue, Apache Hudi on...
-
Senior Cloud Data Engineer
5 days ago
Dallas, United States Cloud Resources LLC Full timeSenior Cloud Data Engineer Scroll down the page to see all associated job requirements, and any responsibilities successful candidates can expect. Long Term REMOTE, but Candidate will have to go onsite at Dallas, TX for one week per quarter as per client request Client: Southwest Airlines Need: 10+ yrs Visa : GC, GC EAD, H4 EAD, H1B, L2S, USC Job...
-
Lead Data Engineer
1 day ago
Dallas, United States Employer Direct Healthcare Full timeAbout Employer Direct HealthcareEmployer Direct healthcare (EDH) is on a mission to transform access to excellent care for the moments that matter most. As the leading provider of surgical benefits for self-insured employers, we bring guided access to high-quality, affordable care for employers, their employees and the family members. We understand and have...
-
Lead Data Engineer
24 hours ago
Dallas, United States Employer Direct Healthcare Full timeAbout Employer Direct HealthcareEmployer Direct healthcare (EDH) is on a mission to transform access to excellent care for the moments that matter most. As the leading provider of surgical benefits for self-insured employers, we bring guided access to high-quality, affordable care for employers, their employees and the family members. We understand and have...
-
Principal, Data Architecture
7 hours ago
Dallas, United States Request Technology, LLC Full time***Hybrid, 3 days onsite, 2 days remote*** ***We are unable to sponsor as this is a permanent full-time role*** A prestigious company is looking for a Principal, Data Architecture. This principal will focus on data architecture and design to support all IT departments throughout the company. These responsibilities include design of data lakes, data...
-
Informatica Administrator Lead
7 days ago
Dallas, United States Sogeti Full timeThis is a full-time opportunity with Sogeti.Location - Dallas/TX, Onsite full-timeOnshore Platforms Administrator Lead• Lead the Data Program's platform Administration for Informatica EDC, AXON, IDQ, and Informatica Cloud IDMC platforms• Lead platform administration of Azure for data engineering• Lead platform administration of Power BI for...