Senior Big Data Infrastructure Engineer

3 weeks ago

Dallas, United States G-Research Full time

Do you want to tackle the biggest questions in finance with near infinite compute power at your fingertips?

G-Research is a leading quantitative research and technology firm, with offices in London and Dallas. We are proud to employ some of the best people in their field and to nurture their talent in a dynamic, flexible and highly stimulating culture where world-beating ideas are cultivated and rewarded.

This is a hybrid role based in our new Dallas infrastructure hub where we work on the latest technologies in a cutting-edge environment.

The role

G-Research is seeking a DevOps focused Infrastructure Engineer, with deep experience in HDFS, YARN and the Hadoop ecosystem, to join its growing Big Data Platform Engineering team.

Big Data Platform Engineering manages a significant distributed Cloudera HDFS storage footprint, has recently adopted Airflow and has also started to explore other Open Source offerings, such as Ozone. In addition, there is also an overlap in technology support within the team, in products such as Trino, Spark, Grafana, Hive, Nifi and HBase to name a few.

As a Senior Big Data Infrastructure Engineer, you will help us deliver automation, feature enhancements and best practice security on our existing hybrid cloud platforms, as well as explore new technologies and integration with other Open Source products within our ecosystem.

There is a significant focus on configuration management, container orchestration, Infrastructure as Code (IaC) and CI/CD. In return, you will gain exposure to the latest hardware and software technologies in a forward-thinking company, which values innovation, personal development and training.

Key responsibilities of the role include:

Developing, deploying and maintaining big data solutions within the Hadoop ecosystem, such as HDFS, YARN, Ranger and Hive.Delivering engineering solutions ensuring they are scalable, stable, performant, and operationally efficientProviding metrics, documentation, and self-service infrastructure to help our customers work at pace and get the most out of the platformsProactively identifying and addressing system performance bottlenecks, security vulnerabilities, and areas for improvement to ensure systems are stable and robustDeploying and maintaining AirflowParticipating in the out of hours rota, providing support and maintenance for our critical systems across the BigData ecosystem

Who are we looking for?

Whatever your technical background, good coding ability is essential for this role.

The ideal candidate will have the following skills and experience:

Strong HDFS, YARN and Hadoop ecosystem knowledge, preferably on the Cloudera Distribution but also OSS DistributionsExcellent coding skills, preferably PythonExperience working with CI\CD pipelines, such as Jenkins and ArgoCDExperience working with Configuration Management tools, such as Ansible and TerraformStrong Linux, Hardware and Networking knowledgeExperience deploying and running applications on Docker and Kubernetes, including the creation of Helm chartsExperience working with distributed systems

Highly desirable but not essential:

Experience working with OzoneExperience working with other Big Data technologies, such as Spark, Trino, Hive, Airflow, Ranger, Oozie etcData Engineering experience (Spark application development, SQL)Team Lead experience

Why should you apply?

Market-leading compensation plus annual discretionary bonus

Informal dress code and excellent work/life balance

Excellent paid time off allowance of 25 days

Sick days, military leave, and family and medical leave

Generous 401(k) plan

16-weeks' fully paid parental leave

Medical and Prescription, Dental, and Vision insurance

Life and Accidental Death & Dismemberment (AD&D) insurance

Employee Assistance and Wellness programs

Generous relocation allowance and support

Great selection of office snacks, and hot and cold drinks

On-site gym and car parking #J-18808-Ljbffr

  • Senior Big Data Engineer

    1 week ago

    Dallas, United States Resource Informatics Group Full time

    Position: Senior Big Data Engineer Location: Dallas, TX Duration: 12+ Months Job Description: Must be highly hands-on lead, writing code, leading team, reviewing other developers code, working closely with the architect to implement the proposed design 5+ years of experience writing enterprise grade Java or Scala or Python code (should be highly...

  • Senior Big Data Engineer

    6 days ago

    Dallas, United States Resource Informatics Group Full time

    Position: Senior Big Data Engineer Location: Dallas, TX Duration: 12+ Months Job Description: • Must be highly hands-on lead, writing code, leading team, reviewing other developers code, working closely with the architect to implement the proposed design • 5+ years of experience writing enterprise grade Java or Scala or Python code (should be highly...

  • Senior Data Technical Lead

    3 weeks ago

    Dallas, Texas, United States NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Senior Data Technical Lead (Life/Annuity exp, Databricks Expertise) to join our team in Dallas, Texas (US-TX),...

  • Senior Data Technical Lead

    3 weeks ago

    Dallas, Texas, United States NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Senior Data Technical Lead (Life/Annuity exp, Databricks Expertise) to join our team in Dallas, Texas (US-TX),...

  • Big Data Architect

    4 weeks ago

    Dallas, United States Resource Informatics Group Full time

    Job Description Job Description Role: Senior Big Data Architect Location: Dallas, TX Duration: 3 months Rates: All-Inclusive Locals prefer Another requirement should be a solid understanding of traditional DW/BI environments and how to migrate them to a Cloudera Hadoop based data lake environment. Big Data Architect Skills Experience in working with Cloudera...

  • Senior Big Data Engineer

    3 weeks ago

    Dallas, United States Resource Informatics Group Full time

    Position: Senior Big Data EngineerLocation: Dallas, TXDuration: 12+ Months Job Description:• Must be highly hands-on lead, writing code, leading team, reviewing other developers code, working closely with the architect to implement the proposed design• 5+ years of experience writing enterprise grade Java or Scala or Python code (should be highly...

  • Senior Data Engineer

    2 months ago

    Dallas, United States tlnt Full time

    Job DescriptionJob DescriptionThe Role& We are seeking a highly experienced Senior Data Engineer with 5+ years of expertise in Azure data engineering. This role is responsible for the development and management of data pipelines, data modeling, and supporting organizational data analytics and insights initiatives. The ideal candidate will be proficient in...

  • Big Data Architect

    2 months ago

    Dallas, United States Resource Informatics Group Inc Full time

    Job DescriptionJob DescriptionRole: Senior Big Data ArchitectLocation: Dallas, TXDuration: 3 months Rates: All-InclusiveLocals prefer Another requirement should be a solid understanding of traditional DW/BI environments and how to migrate them to a Cloudera Hadoop based data lake environment. Big Data Architect SkillsExperience in working with Cloudera Big...

  • Big Data Architect

    2 months ago

    Dallas, United States Resource Informatics Group Inc Full time

    Job DescriptionJob DescriptionRole: Senior Big Data ArchitectLocation: Dallas, TXDuration: 3 months Rates: All-InclusiveLocals prefer Another requirement should be a solid understanding of traditional DW/BI environments and how to migrate them to a Cloudera Hadoop based data lake environment. Big Data Architect SkillsExperience in working with Cloudera Big...

  • Big Data Architect

    2 weeks ago

    Dallas, United States Resource Informatics Group Inc Full time

    Job DescriptionJob DescriptionRole: Senior Big Data ArchitectLocation: Dallas, TXDuration: 3 months Rates: All-InclusiveLocals prefer Another requirement should be a solid understanding of traditional DW/BI environments and how to migrate them to a Cloudera Hadoop based data lake environment. Big Data Architect SkillsExperience in working with Cloudera Big...

  • Senior Data Technical Lead

    2 weeks ago

    Dallas, United States NTT DATA Full time

    Req ID: 279236NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Data Technical Lead (Life/Annuity exp, Databricks Expertise) to join our team in Dallas, Texas (US-TX),...

  • Senior Data Technical Lead

    2 weeks ago

    Dallas, United States NTT DATA Full time

    Req ID: 279237NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Data Technical Lead (Life/Annuity exp, Databricks Expertise) to join our team in Dallas, Texas (US-TX),...

  • Senior Big Data Developer

    3 weeks ago

    Dallas, United States JobRialto Full time

    Description Must have's: Python (or maybe Scala) SQL Big data Cloud (can be other cloud although AWS is nice) Space:Multiple teams are hiring-example, surveillance algorithm team , ETL heavy team, etc. The work is similar. Interview process: L1 (45+min: Will be technical) > L2 Panel (will talk to 2 other engineers. Will be very technical.)...

  • Senior Big Data Developer

    2 weeks ago

    Dallas, United States JobRialto Full time

    Description Must have's: Python (or maybe Scala) SQL Big data Cloud (can be other cloud although AWS is nice) Space:Multiple teams are hiring-example, surveillance algorithm team , ETL heavy team, etc. The work is similar. Interview process: L1 (45+min: Will be technical) > L2 Panel (will talk to 2 other engineers. Will be very technical.)...

  • Senior/Lead Big Data Engineer

    3 weeks ago

    Dallas, United States Broadbase Human Resources Full time

    Key Responsibilities This role can sit remote- but the preference is in the Dallas, TX location ⦁ Executes complex and occasionally highly complex functional work tracks for the team. Drives the execution of operational/technical objectives for data analytic outputs and business solutions. ⦁ Partners with technology teams on Big Data efforts. ⦁...

  • Big Data/Hadoop Engineer

    2 months ago

    Dallas, United States Tekstrom Inc Full time

    Job DescriptionJob DescriptionBig Data/ Hadoop Engineer - (Ideal candidate will be a Sr. java Developer currently working in Big Data and AWS cloud platform) Responsibilities:Work with big data analytics engineers to develop automated testing systems.Develop testing software that will generate and validate test data for big data systems.Develop testing...

  • Big Data/Hadoop Engineer

    2 weeks ago

    Dallas, United States Tekstrom Inc Full time

    Job DescriptionJob DescriptionBig Data/ Hadoop Engineer - (Ideal candidate will be a Sr. java Developer currently working in Big Data and AWS cloud platform) Responsibilities:Work with big data analytics engineers to develop automated testing systems.Develop testing software that will generate and validate test data for big data systems.Develop testing...

  • Senior Data Technical Lead

    3 weeks ago

    Dallas, United States NTT DATA Services Full time

    Req ID: 279236    NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.   We are currently seeking a Senior Data Technical Lead (Life/Annuity exp, Databricks Expertise) to join our...

  • Senior Data Technical Lead

    1 week ago

    Dallas, United States NTT DATA Services Full time

    Req ID: 279236    NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.   We are currently seeking a Senior Data Technical Lead (Life/Annuity exp, Databricks Expertise) to join our...

  • Senior Data Technical Lead

    3 weeks ago

    Dallas, United States NTT DATA Services Full time

    Req ID: 279237    NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.   We are currently seeking a Senior Data Technical Lead (Life/Annuity exp, Databricks Expertise) to join our...