Senior Big Data Engineer

5 days ago

Irving, United States Anblicks Full time

Description:

The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.


Responsibilities:

  • Responsible for design and development of big data solutions. Partner with domain experts, product managers, analyst, and data scientists to develop Big Data pipelines in Hadoop
  • Responsible for moving all legacy workloads to cloud platform
  • Work with data scientist to build Client pipelines using heterogeneous sources and provide engineering services for data science applications
  • Ensure automation through CI/CD across platforms both in cloud and on-premises
  • Define needs around maintainability, testability, performance, security, quality and usability for data platform
  • Drive implementation, consistent patterns, reusable components, and coding standards for data engineering processes
  • Convert SAS based pipelines into languages like PySpark, Scala to execute on Hadoop and non-Hadoop ecosystems
  • Tune Big data applications on Hadoop and non-Hadoop platforms for optimal performance
  • Evaluate new IT developments and evolving business requirements and recommend appropriate systems alternatives and/or enhancements to current systems by analyzing business processes, systems and industry standards.
  • Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinates and contributes to the objectives of the entire function.
  • Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/taken.
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.

Qualifications:

  • 8+ years of total IT experience
  • 5+ years of experience with Hadoop (Cloudera)/big data technologies
  • Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)
  • Experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Java or Scala or Python.
  • Experience with Spark programming (pyspark or scala or java)
  • Expert level building pipelines using Apache Spark Familiarity with core provider services from AWS, Azure or GCP, preferably having supported deployments on one or more of these platforms
  • Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required;
  • Exposure to containerization and related technologies (e.g. Docker, Kubernetes)
  • Exposure to aspects of DevOps (source control, continuous integration, deployments, etc.)
  • Proficient in programming in Java or Python with prior Apache Beam/Spark experience a plus.
  • System level understanding - Data structures, algorithms, distributed storage & compute
  • Can-do attitude on solving complex business problems, good interpersonal and teamwork skills
  • Possess team management experience and have led a team of data engineers and analysts.
  • Experience in Snowflake is a plus.

Education:

  • Bachelor’s degree/University degree or equivalent experience

  • Senior Big Data Engineer

    3 weeks ago

    Irving, United States Anblicks Full time

    Description: The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for design and development of big data...

  • Big Data Engineer

    3 weeks ago

    Irving, Texas, United States Manpower Group Inc. Full time

    Job Title: Big Data EngineerWe are seeking a highly skilled Big Data Engineer to join our team. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Key Responsibilities:Design and develop data processing systems using Hadoop, Spark, and other big data technologies.Develop and maintain...

  • Big Data Engineer

    3 weeks ago

    Irving, Texas, United States Manpower Group Inc. Full time

    Job Title: Big Data EngineerWe are seeking a highly skilled Big Data Engineer to join our team. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Key Responsibilities:Design and develop data processing systems using Starburst/TRINO, Impala, Hive, and SparkSQL.Work with Hadoop...

  • Big Data Engineer

    4 weeks ago

    Irving, United States Newt Global Full time

    Greeting from Newt Global LLC!!!We are currently looking for Big Data Engineer to join our team for long term project with our Banking client. Interested with the below job description, please share your resume with expected rate/hr.Direct Client: Citibank Big Data Engineer Irving, TX (Hybrid – 2 Days a Week)Contract Type: W2 Long TermExp: 8+ yrs Banking...

  • Big Data Engineer

    4 weeks ago

    Irving, United States Newt Global Full time

    Greeting from Newt Global LLC!!!We are currently looking for Big Data Engineer to join our team for long term project with our Banking client. Interested with the below job description, please share your resume with expected rate/hr.Direct Client: Citibank Big Data Engineer Irving, TX (Hybrid – 2 Days a Week)Contract Type: W2 Long TermExp: 8+ yrs Banking...

  • Big Data Engineer

    2 days ago

    Irving, United States Newt Global Full time

    Greeting from Newt Global LLC!!!We are currently looking for Big Data Engineer to join our team for long term project with our Banking client. Interested with the below job description, please share your resume with expected rate/hr. Big Data Engineer Irving, TX (Hybrid – 2 Days a Week)Contract Type: W2 or SCorp (No Third Party Resume)Long TermExp: 8+ yrs...

  • Big Data Engineer

    3 weeks ago

    Irving, United States Motion Recruitment Full time

    Grow your career as a Big Data Engineer with an innovative global bank in Irving, TX. Contract role with strong possibility of extension. Will require working a hybrid schedule 2-3 days onsite per week. Join one of the world's most renowned global banks and trusted brand with over 200 years of continuously evolving financial services worldwide. You will...

  • Big Data Engineer

    2 weeks ago

    Irving, United States Motion Recruitment Partners, LLC Full time

    Grow your career as a Big Data Engineer with an innovative global bank in Irving, TX. Contract role with strong possibility of extension. Will require working a hybrid schedule 2-3 days onsite per week. Join one of the world's most renowned global banks and trusted brand with over 200 years of continuously evolving financial services worldwide. You will...

  • Big Data Engineer

    2 weeks ago

    Irving, United States Rose International Full time

    •**Only qualified Big Data Engineer candidates located near the Irving, TX area to be considered due to the position requiring an onsite presence***Qualifications: 5+ years of experience with Starburst/TRINO, Impala, Hive, SparkSQL Good knowledge with Hadoop environment and distribution system Strong data analysis, data evaluation and...

  • Lead / Sr. Big Data Engineer

    2 days ago

    Irving, United States InfoVision Inc. Full time

    Job Title: Lead / Sr. Big Data EngineerLocation: Irving, TX (Hybrid 2-3 days/week)Duration: 12+ Months Required Skills: Big data expert with 9+ years experience in Hadoop Big data ecosystemSpark - Batch Streaming (Python, Scala)Experience in cloud environment specially Google Cloud PlatformExperience in developing both batch and real-time streaming data...

  • Big Data Engineer

    2 weeks ago

    Irving, Texas, United States Motion Recruitment Full time

    Job Title: Big Data DeveloperJoin Motion Recruitment in Irving, TX, as a Big Data Developer and grow your career with an innovative global bank. This contract role offers a strong possibility of extension and requires a hybrid schedule with 2-3 days onsite per week.Key Responsibilities:Design, develop, and enhance SQL queries to meet business...

  • Lead / Sr. Big Data Engineer--HYBRID

    3 days ago

    Irving, United States BTPSUSA, LLC Full time

    Hi All, Hope you are doing well!Job Title: Lead / Sr. Big Data EngineerLocation: Irving, TX (Hybrid 2-3 days/week)Duration: 12+ Months This role can play the hand-on dev role along with tech lead skills Required Skills:Big data expert with 9+ years experience in Hadoop Big data ecosystemSpark - Batch Streaming (Python, Scala)Experience in cloud environment...

  • Senior Data Engineer

    2 weeks ago

    Irving, Texas, United States Acunor Full time

    Job Title: Senior Data EngineerLocation: Irving, TX / OnsiteFull Time OpportunityJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team at Acunor. As a Senior Data Engineer, you will be responsible for designing and building scalable data systems, developing data pipelines, and ensuring data quality and integrity.Required...

  • Senior Data Engineer

    1 month ago

    Irving, Texas, United States IVID TEK INC Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our team at IVID TEK INC. As a Senior Data Engineer, you will be responsible for designing, developing, and deploying large-scale data systems and architectures.Key ResponsibilitiesDesign and Develop Data Systems: Design and develop scalable, efficient, and automated data systems and...

  • Senior Data Engineer

    3 weeks ago

    Irving, Texas, United States Acunor Full time

    Job Title: Senior Data EngineerLocation: Irving, TX / OnsiteFull Time OpportunityJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team at Acunor. As a key member of our data engineering team, you will be responsible for designing and building scalable data pipelines, architectures, and systems.Key Responsibilities:Design and...

  • Big Data Specialist

    1 week ago

    Irving, United States Motion Recruitment Full time

    Job Title: Big Data EngineerDuration: 13+ months (Hybrid and Contract)Location: Irving TX (Only local candidates to TX Allowed)Job Type: W2 and only USC/GC/EAD Required (No Sponsorship or visa transfer allowed)Job Description:Grow your career as a Big Data Engineer with an innovative global bank in Irving, TX. Contract role with strong possibility of...

  • Big Data Specialist

    1 week ago

    Irving, United States Motion Recruitment Full time

    Job Title: Big Data EngineerDuration: 13+ months (Hybrid and Contract)Location: Irving TX (Only local candidates to TX Allowed)Job Type: W2 and only USC/GC/EAD Required (No Sponsorship or visa transfer allowed)Job Description:Grow your career as a Big Data Engineer with an innovative global bank in Irving, TX. Contract role with strong possibility of...

  • Senior Data Architect

    3 weeks ago

    Irving, Texas, United States Anblicks Full time

    Job Title: Senior Big Data EngineerAbout the Role:We are seeking a highly skilled Senior Big Data Engineer to join our team at Anblicks. As a Senior Big Data Engineer, you will be responsible for designing and developing big data solutions using next-generation data techniques. You will work closely with product owners, customers, and technologists to...

  • Lead Big Data Engineer

    2 days ago

    Irving, United States InfoVision Inc. Full time

    Job title: Lead Big Data Engineer (GCP Cloud)Location: Irving, TXDuration: Long-term Skills Needed:10+ years of experience in designing and building data pipelines in large-scale distributed systems.Lead the design, development, and maintenance of scalable batch and real-time data processing pipelines.Proficiency with Google Cloud Platform (GCP) and tools...

  • Big Data Developer

    2 weeks ago

    Irving, United States Motion Recruitment Full time

    Grow your career as a Big Data Engineer with an innovative global bank in Irving, TX. Contract role with strong possibility of extension. Will require working a hybrid schedule 2-3 days onsite per week.Required Skills & Experience5+ years of working experiences with Starburst/TRINO, Impala, Hive, SparkSQL.Good knowledge with Hadoop environment and...