Senior Data Engineer

2 months ago

Dallas, United States Anblicks Full time

The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.

Responsibilities:

Responsible for design and development of big data solutions. Partner with domain experts, product managers, analyst, and data scientists to develop Big Data pipelines in Hadoop

Responsible for moving all legacy workloads to cloud platform

Work with data scientist to build Client pipelines using heterogeneous sources and provide engineering services for data science applications

Ensure automation through CI/CD across platforms both in cloud and on-premises

Define needs around maintainability, testability, performance, security, quality and usability for data platform

Drive implementation, consistent patterns, reusable components, and coding standards for data engineering processes

Convert SAS based pipelines into languages like PySpark, Scala to execute on Hadoop and non-Hadoop ecosystems

Tune Big data applications on Hadoop and non-Hadoop platforms for optimal performance

Evaluate new IT developments and evolving business requirements and recommend appropriate systems alternatives and/or enhancements to current systems by analyzing business processes, systems and industry standards.

Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinates and contributes to the objectives of the entire function.

Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/taken.

Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.

Qualifications:

8+ years of total IT experience

5+ years of experience with Hadoop (Cloudera)/big data technologies

Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)

Experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Java or Scala or Python.

Experience with Spark programming (pyspark or scala or java)

Expert level building pipelines using Apache Spark Familiarity with core provider services from AWS, Azure or GCP, preferably having supported deployments on one or more of these platforms

Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required;

Exposure to containerization and related technologies (e.g. Docker, Kubernetes)

Exposure to aspects of DevOps (source control, continuous integration, deployments, etc.)

Proficient in programming in Java or Python with prior Apache Beam/Spark experience a plus.

System level understanding - Data structures, algorithms, distributed storage & compute

Can-do attitude on solving complex business problems, good interpersonal and teamwork skills

Possess team management experience and have led a team of data engineers and analysts.

Experience in Snowflake is a plus.

Education:

Bachelor’s degree/University degree or equivalent experience

  • Data Engineer

    1 month ago

    Dallas, Texas, United States NTT DATA Full time

    Job SummaryNTT DATA Services is seeking a highly skilled Data Engineer to join our team in Dallas, Texas. As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data systems to support our business operations.Key Responsibilities:Develop and maintain data pipelines to extract, transform, and load data from diverse...

  • Data Engineer

    5 months ago

    Dallas, United States NTT DATA Full time

    Job Description NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer (Onsite) to join our team in Dallas, Texas (US-TX), United States (US).Job Responsibilities...

  • Senior Data Engineer

    2 days ago

    Dallas, United States InfoVision Inc. Full time

    Senior Data EngineerDallas, TXLong term contractJob Description:We are looking for a Data Engineer to join our Advanced Analytics team under the Wholesale organization. This engineer will design and deliver data solutions for the analytics and data science teams in the consumer organization. The engineer will be able to work with all types of interesting...

  • Senior Linux Systems Engineer

    4 weeks ago

    Dallas, Texas, United States NTT DATA Full time

    We are seeking a Senior Linux Systems Engineer to join our team in Irving, Texas. This is a permanent, full-time, salaried position with benefits.The Grid/HPC engineer will work as a key member of the Symphony Grid Engineering team, engineering and supporting technical solutions for the Grid platform, leveraging modern and innovative technologies to deliver...

  • Data Science Engineering Lead

    4 weeks ago

    Dallas, Texas, United States Data Axle Full time

    Job SummaryData Axle is seeking a highly experienced Principal Data Engineer to lead the design, implementation, and optimization of our modern data architecture.This role requires a strong background in data engineering, expertise in Python and SQL programming, and a passion for creating scalable solutions to support business needs.The ideal candidate will...

  • Senior Linux Engineer

    2 weeks ago

    Dallas, United States NTT DATA Full time

    Req ID: 299982 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Linux Engineer - Irving, Texas to join our team in Irving, Texas (US-TX), United States (US).NTT Data Americas is...

  • Senior Data Engineering Manager

    4 weeks ago

    Dallas, Texas, United States Glocomms Full time

    **Senior Data Engineering Manager**Glocomms is seeking a seasoned Senior Data Engineering Manager to lead our dynamic team in Dallas. As a key member of our organization, you will be responsible for overseeing the development and implementation of data engineering projects using top-notch technology within the Azure data factory environment.*Key...

  • Senior Data Engineer

    2 months ago

    dallas, United States TopShelf Talent Group Full time

    Our client is a well-known investment management firm in the Dallas area and they're seeking a Senior Data Engineer to join their team.They need a Senior Data Engineer with 8+ years of hands-on expertise in data engineering, with a focus on creating reliable, efficient, and scalable systems. This role is pivotal in shaping the data architecture and pipelines...

  • Senior Data Engineer

    1 month ago

    dallas, United States TopShelf Talent Group Full time

    Our client is a well-known investment management firm in the Dallas area and they're seeking a Senior Data Engineer to join their team.They need a Senior Data Engineer with 8+ years of hands-on expertise in data engineering, with a focus on creating reliable, efficient, and scalable systems. This role is pivotal in shaping the data architecture and pipelines...

  • Senior Data Engineer

    2 months ago

    Dallas, United States TopShelf Talent Group Full time

    Our client is a well-known investment management firm in the Dallas area and they're seeking a Senior Data Engineer to join their team.They need a Senior Data Engineer with 8+ years of hands-on expertise in data engineering, with a focus on creating reliable, efficient, and scalable systems. This role is pivotal in shaping the data architecture and pipelines...

  • Senior Linux Systems Engineer

    4 weeks ago

    Dallas, Texas, United States NTT DATA Full time

    We are seeking a Senior Linux Engineer to join our team in Irving, Texas (US-TX), United States (US).This position will work closely with the vendor and internal clients to engineer and support technical solutions for the Grid platform, leveraging modern and innovative technologies to deliver the optimal stability and performance.The ideal candidate should...

  • Senior Data Engineer

    1 month ago

    Dallas, Texas, United States MSys Inc Full time

    Job Title: Senior Data Engineer - OnsiteJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team on an onsite basis. The ideal candidate will have 5-8+ years of experience in data engineering or a related role and be able to design, build, and manage ETL data pipelines using AWS services such as Glue, Athena, and Lambda.Key...

  • Senior Data Engineer

    4 weeks ago

    Dallas, Texas, United States UT Southwestern Medical Center Full time

    About the Role:The Department of Information Resources at UT Southwestern Medical Center is seeking a highly skilled Senior Data Engineer to join their team. As a key member of the Enterprise Data Services team, you will be responsible for designing, developing, and implementing large-scale data engineering solutions using cloud-based platforms.Key...

  • Senior Cloud Data Engineer

    4 weeks ago

    Dallas, Texas, United States Abode Techzone LLC Full time

    Job Title:Senior Cloud Data Engineer - Air flowLocation: RemoteProject Duration: 12 months ContractInterview: Three rounds of Video interviewJob Description :Primary Skills:Air Flow, Big Query, Data Proc , Spark), Hadoop. Lead the Team. E2E DeliverySecondary Skills:Java, Unix Scripts, Oozie, Google Cloud Big Query & Big Table, Google Data Proc, Google Data...

  • Senior Data Systems Engineer

    1 month ago

    Dallas, Texas, United States Shtudy Full time

    Job Title: Senior Data Systems EngineerWe are seeking a highly skilled Senior Data Systems Engineer to join our dynamic team at Shtudy. This role requires a strong background in data engineering, cloud technologies, and quality assurance.Key Responsibilities:Automate ETL pipelines using Python to streamline data integration, transformation, and migration...

  • Senior Scrum Master

    3 weeks ago

    Dallas, United States Tiva's Client Full time

    Senior Scrum Master (Data Analytics and Data Engineering Teams)We are looking for a Senior Scrum Master who has driven projects/programs that are data heavy projects around Data Analytics, Data Engineering, Data Curation, Data Ingestion, Cloud Data etc because they are supporting Data Engineering and Data Analytics Team.ResponsibilitiesFacilitating daily...

  • Senior Scrum Master

    2 weeks ago

    dallas, United States Tiva's Client Full time

    Senior Scrum Master (Data Analytics and Data Engineering Teams)We are looking for a Senior Scrum Master who has driven projects/programs that are data heavy projects around Data Analytics, Data Engineering, Data Curation, Data Ingestion, Cloud Data etc because they are supporting Data Engineering and Data Analytics Team.ResponsibilitiesFacilitating daily...

  • Senior Scrum Master

    3 weeks ago

    dallas, United States Tiva's Client Full time

    Senior Scrum Master (Data Analytics and Data Engineering Teams)We are looking for a Senior Scrum Master who has driven projects/programs that are data heavy projects around Data Analytics, Data Engineering, Data Curation, Data Ingestion, Cloud Data etc because they are supporting Data Engineering and Data Analytics Team.ResponsibilitiesFacilitating daily...

  • Senior Data Ingestion Engineer

    1 month ago

    dallas, United States Tiva's Client Full time

    Senior Data Ingestion EngineerMost important for this role (key skills)Kafka (data streaming)Python/PySparkAWSDatabricks Cloud Platform Unity CatalogRole Overview:This role will be responsible for conducting data ingestion and platform management on the Databricks platform. This role requires a deep understanding of:Data Lake ingestion processes and best...

  • Senior Data Engineer

    3 weeks ago

    dallas, United States Talent Groups Full time

    Direct Hire - the client is unable to offer sponsorship at this time. Hybrid: Mostly remote role in Dallas, TX. Must be able to interview onsite and work onsite 1x a month. Client is located near 75/635. Our client is seeking a Senior Data Engineer from a larger-scale healthcare production environment of 500 KB - up to 5 TB....