Senior Big Data Engineer

2 months ago

Irving, United States Anblicks Full time

Description:

The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.


Responsibilities:

  • Responsible for design and development of big data solutions. Partner with domain experts, product managers, analyst, and data scientists to develop Big Data pipelines in Hadoop
  • Responsible for moving all legacy workloads to cloud platform
  • Work with data scientist to build Client pipelines using heterogeneous sources and provide engineering services for data science applications
  • Ensure automation through CI/CD across platforms both in cloud and on-premises
  • Define needs around maintainability, testability, performance, security, quality and usability for data platform
  • Drive implementation, consistent patterns, reusable components, and coding standards for data engineering processes
  • Convert SAS based pipelines into languages like PySpark, Scala to execute on Hadoop and non-Hadoop ecosystems
  • Tune Big data applications on Hadoop and non-Hadoop platforms for optimal performance
  • Evaluate new IT developments and evolving business requirements and recommend appropriate systems alternatives and/or enhancements to current systems by analyzing business processes, systems and industry standards.
  • Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinates and contributes to the objectives of the entire function.
  • Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/taken.
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.

Qualifications:

  • 8+ years of total IT experience
  • 5+ years of experience with Hadoop (Cloudera)/big data technologies
  • Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)
  • Experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Java or Scala or Python.
  • Experience with Spark programming (pyspark or scala or java)
  • Expert level building pipelines using Apache Spark Familiarity with core provider services from AWS, Azure or GCP, preferably having supported deployments on one or more of these platforms
  • Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required;
  • Exposure to containerization and related technologies (e.g. Docker, Kubernetes)
  • Exposure to aspects of DevOps (source control, continuous integration, deployments, etc.)
  • Proficient in programming in Java or Python with prior Apache Beam/Spark experience a plus.
  • System level understanding - Data structures, algorithms, distributed storage & compute
  • Can-do attitude on solving complex business problems, good interpersonal and teamwork skills
  • Possess team management experience and have led a team of data engineers and analysts.
  • Experience in Snowflake is a plus.

Education:

  • Bachelor’s degree/University degree or equivalent experience

  • Senior Big Data Engineer

    4 weeks ago

    Irving, Texas, United States InfoVision Inc. Full time

    Job Title: Senior Big Data EngineerLocation: HybridDuration: 12+ MonthsKey Responsibilities:Lead the development of big data solutions using Hadoop and SparkDesign and implement data pipelines for batch and real-time streaming dataCollaborate with cross-functional teams to ensure data engineering projects meet business requirementsDevelop and maintain...

  • Big Data Engineering Lead

    4 weeks ago

    Irving, Texas, United States Infovision Full time

    Job Title: Senior Big Data EngineerLocation: Irving, TX (Hybrid 2-3 days/week)Duration: 12+ MonthsJob Summary:We are seeking a highly experienced Senior Big Data Engineer to lead our data engineering projects. The ideal candidate will have a strong background in Hadoop and cloud computing, with expertise in developing batch and real-time streaming data...

  • Senior Big Data Engineer

    4 weeks ago

    Irving, Texas, United States BTPSUSA, LLC Full time

    **Job Title:** Lead / Sr. Big Data Engineer**Location:** Irving, TX (Hybrid 2-3 days/week)**Duration:** 12+ MonthsThis role will play a key part in our data engineering team, requiring a strong technical lead with hands-on development experience. Key responsibilities include:• Developing and maintaining big data pipelines using Hadoop and Spark•...

  • Senior Big Data Engineer

    4 weeks ago

    Irving, Texas, United States BTPSUSA, LLC Full time

    Job Title: Lead / Sr. Big Data EngineerLocation: Irving, TX (Hybrid 2-3 days/week)Duration: 12+ MonthsThis role will play a key part in our data engineering team, requiring a strong background in big data and cloud technologies. The ideal candidate will have experience with Hadoop, Spark, and cloud platforms, as well as strong technical leadership skills.Key...

  • Senior Big Data Engineer

    4 weeks ago

    Irving, United States Anblicks Full time

    Description: The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for design and development of big data...

  • Senior Big Data Engineer

    7 days ago

    Irving, United States Anblicks Full time

    Description: The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for design and development of big data...

  • Big Data Engineer

    4 weeks ago

    Irving, Texas, United States Motion Recruitment Full time

    Job Title: Big Data EngineerAre you a skilled Big Data Engineer looking for a new challenge? We have an exciting opportunity for you to join our team at a leading global bank in Irving, TX.About the Role:We are seeking a highly motivated and experienced Big Data Engineer to join our team. As a Big Data Engineer, you will be responsible for the delivery of...

  • Big Data Engineer

    1 month ago

    Irving, United States Newt Global Full time

    Greeting from Newt Global LLC!!!We are currently looking for Big Data Engineer to join our team for long term project with our Banking client. Interested with the below job description, please share your resume with expected rate/hr. Big Data Engineer Irving, TX (Hybrid – 2 Days a Week)Contract Type: W2 or SCorp (No Third Party Resume)Long TermExp: 8+ yrs...

  • Big Data Engineer

    4 weeks ago

    Irving, Texas, United States Resource Informatics Group Full time

    Job Summary:Resource Informatics Group is seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience with Big Data platforms and tools, including Linux, HDFS, Hive, Hortonworks, Spark, Scala, Python, Java, and Kafka. Key Responsibilities: Design and develop scalable data processing systems using Big Data...

  • Senior Big Data Engineer

    7 days ago

    Irving, TX, United States Anblicks Full time

    Description: The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for design and development of big data...

  • Senior Data Engineer

    4 weeks ago

    Irving, Texas, United States NTT DATA Group Corporation Full time

    Job Title: Senior Data EngineerJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team at NTT DATA Group Corporation. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing large-scale data processing systems using Python and PySpark.Key Responsibilities:Design and develop complex data...

  • Senior Data Engineer

    4 weeks ago

    Irving, Texas, United States OTSI Full time

    Job Title: Senior Data EngineerJob Summary:OTSI is seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing and developing methods to connect and collect data from different data sources, as well as designing and developing SQL, Hive queries, and APIs to extract data from the store.Key...

  • Senior Data Engineering Lead

    3 weeks ago

    Irving, Texas, United States Infovision Full time

    Job Title: Senior Data Engineering LeadLocation: Hybrid 2-3 days/weekDuration: 12+ MonthsKey Responsibilities: As a Senior Data Engineering Lead, you will be responsible for leading data engineering projects and teams. You will work closely with cross-functional teams to design, develop, and implement big data solutions using Hadoop and Spark. You will also...

  • Lead/Sr. Big Data Engineer

    2 weeks ago

    Irving, United States America Technology Professionals LLC Full time

    Job Title: Lead / Sr. Big Data Engineer Location: Irving, TX (Hybrid – 2-3 days/week) Duration: 12+ Months USC/GC/H1B only C2C/1099 only This role can play the hand-on dev role along with tech lead skills Required Skills: Big data expert with 9+ years experience in Hadoop Big data ecosystem Spark - Batch Streaming (Python, Scala) Experience in cloud...

  • Big Data Engineer

    3 weeks ago

    Irving, Texas, United States Genesis10 Full time

    About the Role:Genesis10 is seeking a skilled Big Data Engineer to join our client in the financial industry in Irving, TX. This is a 12+ month contract position that requires expertise in Software Engineering and data processing.Key Responsibilities:Consult on or participate in moderately complex initiatives and deliverables within Software Engineering and...

  • Big Data Engineer

    4 weeks ago

    Irving, Texas, United States Resource Informatics Group Full time

    Job Opportunity:Resource Informatics Group is seeking a skilled Big Data Developer to join our team. The ideal candidate will have a strong background in Big Data platforms and tools, including Linux, HDFS, Hive, Hortonworks, Spark, Scala, Python, Java, Kafka, and API's. Key Responsibilities:• Design and develop data control checks• Process and analyze...

  • Senior Data Engineer

    4 weeks ago

    Irving, Texas, United States Citigroup Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled and experienced Senior Data Engineer to join our team at Citigroup. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining data infrastructure, pipelines, and solutions.Key Responsibilities:Design and implement robust data architectures,...

  • Lead/Sr. Big Data Engineer

    2 weeks ago

    Irving, United States America Technology Professionals LLC Full time

    Job Title: Lead / Sr. Big Data EngineerApply now, read the job details by scrolling down Double check you have the necessary skills before sending an application.Location: Irving, TX (Hybrid – 2-3 days/week)Duration: 12+ MonthsUSC/GC/H1B onlyC2C/1099 onlyThis role can play the hand-on dev role along with tech lead skillsRequired Skills:Big data expert with...

  • Lead / Sr. Big Data Engineer

    7 days ago

    Irving, United States Infovision Full time

    Job Title: Lead / Sr. Big Data EngineerLocation: Irving, TX (Hybrid 2-3 days/week)Duration: 12+ MonthsRequired Skills: Big data expert with 9+ years experience in Hadoop Big data ecosystemSpark - Batch Streaming (Python, Scala)Experience in cloud environment specially Google Cloud PlatformExperience in developing both batch and real-time streaming data...

  • Big Data Engineer

    4 weeks ago

    Irving, Texas, United States Resource Informatics Group Full time

    Job Opportunity:Resource Informatics Group is seeking a skilled Big Data Developer to join our team.Key Responsibilities:As a Big Data Developer, you will be responsible for designing and developing data control checks, working with Big Data platforms and tools, and interacting with data. You will also be expected to have experience with API's, reading from...