Senior Big Data Engineer

2 hours ago

Irving, United States Anblicks Full time

Must be local to TX


Description:

The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.


Responsibilities:

  • Responsible for design and development of big data solutions. Partner with domain experts, product managers, analyst, and data scientists to develop Big Data pipelines in Hadoop
  • Responsible for moving all legacy workloads to cloud platform
  • Work with data scientist to build Client pipelines using heterogeneous sources and provide engineering services for data science applications
  • Ensure automation through CI/CD across platforms both in cloud and on-premises
  • Define needs around maintainability, testability, performance, security, quality and usability for data platform
  • Drive implementation, consistent patterns, reusable components, and coding standards for data engineering processes
  • Convert SAS based pipelines into languages like PySpark, Scala to execute on Hadoop and non-Hadoop ecosystems
  • Tune Big data applications on Hadoop and non-Hadoop platforms for optimal performance
  • Evaluate new IT developments and evolving business requirements and recommend appropriate systems alternatives and/or enhancements to current systems by analyzing business processes, systems and industry standards.
  • Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinates and contributes to the objectives of the entire function.
  • Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/taken.
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.


Qualifications:


  • 8+ years of total IT experience
  • 5+ years of experience with Hadoop (Cloudera)/big data technologies
  • Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)
  • Experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Java or Scala or Python.
  • Experience with Spark programming (pyspark or scala or java)
  • Expert level building pipelines using Apache Spark Familiarity with core provider services from AWS, Azure or GCP, preferably having supported deployments on one or more of these platforms
  • Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required;
  • Exposure to containerization and related technologies (e.g. Docker, Kubernetes)
  • Exposure to aspects of DevOps (source control, continuous integration, deployments, etc.)
  • Proficient in programming in Java or Python with prior Apache Beam/Spark experience a plus.
  • System level understanding - Data structures, algorithms, distributed storage & compute
  • Can-do attitude on solving complex business problems, good interpersonal and teamwork skills
  • Possess team management experience and have led a team of data engineers and analysts.
  • Experience in Snowflake is a plus.


Education:

  • Bachelor’s degree/University degree or equivalent experience

  • Senior Big Data Engineer

    4 days ago

    Irving, United States Anblicks Full time

    Must be local to TXDescription:The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for design and...

  • Big Data Engineer

    5 days ago

    Irving, United States Newt Global Full time

    Greeting from Newt Global LLC!!!We are currently looking for Big Data Engineer to join our team for long term project with our Banking client. Interested with the below job description, please share your resume with expected rate/hr. Big Data Engineer Irving, TX (Hybrid – 2 Days a Week)Contract Type: W2 or SCorp (No Third Party Resume)Long TermExp: 8+ yrs...

  • Big Data Engineering Manager

    7 days ago

    Irving, Texas, United States LTIMindtree Full time

    Job DescriptionAs a Big Data Engineering Manager at LTIMindtree, you will lead our data engineering efforts and develop innovative solutions using big data technologies. You will work closely with cross-functional teams to identify business needs and develop data-driven solutions. If you have a passion for innovation and a drive to succeed, we encourage you...

  • Lead Big Data Engineer

    2 hours ago

    Irving, United States InfoVision Inc. Full time

    Job title: Lead Big Data Engineer (GCP Cloud)Location: Irving, TXDuration: Long-term Skills Needed:10+ years of experience in designing and building data pipelines in large-scale distributed systems.Lead the design, development, and maintenance of scalable batch and real-time data processing pipelines.Proficiency with Google Cloud Platform (GCP) and tools...

  • Big Data Systems Developer

    4 days ago

    Irving, Texas, United States Anblicks Full time

    Anblicks Job DescriptionWe are looking for a highly skilled Data Engineer to join our team at Anblicks. This role requires expertise in Hadoop and big data technologies, as well as experience with cloud platforms such as AWS, Azure, or GCP.About the PositionThis role involves working closely with product owners, customers, and technologists to deliver data...

  • Big Data Solutions Developer

    6 days ago

    Irving, Texas, United States LTIMindtree Full time

    Data Engineer Job Summary:We are seeking a highly skilled Data Engineer to join our team at LTIMindtree. The successful candidate will have extensive experience in designing, building, and maintaining large-scale data processing systems using Hadoop, Spark, and related technologies.The ideal candidate will have excellent technical skills, strong...

  • Senior Azure Data Platform Specialist

    7 days ago

    Irving, Texas, United States Atika Technologies Full time

    Job SummaryAs a Senior Azure Data Platform Specialist at Atika Technologies, you will lead the design and implementation of big data pipelines and architectures using Azure services. This role requires 8+ years of experience in Azure Data Factory and Databricks with Big Data ecosystem tools. The estimated annual salary for this position in Irving, TX is...

  • Big Data Developer

    2 hours ago

    Irving, United States Resource Informatics Group Full time

    Note: It is a W2 Opportunity, Client is accepting candidates from (New Jersey and New York) other location candidates will be straight away rejected and application candidate must have prior banking domain work experience. Role: Big Data Developer Location: Iselin, NJ In office: 3 days a week onsite Contract: 6-24 months to perm Interview process: screening...

  • Senior Data Engineering Professional

    5 days ago

    Irving, Texas, United States Atika Technologies Full time

    Required Skills and QualificationsWe are seeking a highly skilled individual with 8+ years of experience in Azure Data Factory and Databricks. A strong background in Python or C# programming languages is required, along with expertise in Big Data ecosystem tools such as Hadoop and Hive. Additionally, knowledge of serverless architecture, ARM templates, and...

  • Senior Infrastructure Engineer

    6 days ago

    Irving, United States NTT DATA Full time

    Req ID: 301051 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Senior Infrastructure Engineer - Irving, Texas to join our team in Irving, Texas (US-TX), United States (US). NTT Data...

  • Senior Infrastructure Engineer

    5 days ago

    Irving, United States NTT DATA Full time

    Req ID: 301051 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Infrastructure Engineer - Irving, Texas to join our team in Irving, Texas (US-TX), United States (US).NTT Data...

  • Big Data Architect

    4 days ago

    Irving, Texas, United States Resource Informatics Group Full time

    About the RoleJoin our dynamic team as a Big Data Architect and play a key role in shaping our data analytics strategy. You will design and implement scalable data warehousing solutions, collaborating closely with stakeholders to ensure alignment with business objectives.Responsibilities* Design and implement data warehousing solutions using Databricks or...

  • Cloud Infrastructure Engineer

    7 days ago

    Irving, Texas, United States NTT DATA Full time

    About the Role: We are looking for a highly skilled Senior Infrastructure Engineer to join our team in Irving, Texas. In this role, you will apply in-depth IT knowledge and experience towards gathering and analyzing business/application requirements in support of financial services.This is a permanent, full-time position with benefits. The estimated salary...

  • Sr. Data Engineer

    1 day ago

    Irving, United States Anblicks Full time

    Job Title: Data EngineerLocation: Irving, TX The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for...

  • Sr. Data Engineer

    23 hours ago

    Irving, United States Anblicks Full time

    Job Title: Data EngineerLocation: Irving, TX The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for...

  • Data Engineering Solutions Architect

    4 days ago

    Irving, Texas, United States Anblicks Full time

    Job OverviewWe are seeking an experienced Data Engineer to join our team at Anblicks. As a key member of our data engineering team, you will be responsible for designing and developing big data solutions using next generation data techniques.About the RoleThis position requires someone with a strong background in Hadoop and big data technologies, as well as...

  • Cloud Data Specialist

    5 days ago

    Irving, Texas, United States Verizon Full time

    As a Senior Data Engineer at Verizon, you will have the opportunity to work on cutting-edge data projects that drive business outcomes. Our team is passionate about delivering innovative data solutions that empower our customers and employees.What You'll DoAs a key member of our data engineering team, you will:Design and implement large-scale data platforms...

  • Lead Data Engineer

    2 hours ago

    Irving, United States IVID TEK INC Remote Work Freelance Full time $65 - $70

    Position: Lead Data Engineer (GCP,Azure)Location: Dallas, TXJob Type: Contract C2CRate: 65-70/ HrRoles and Responsibilities:Key Skills: Vertex AI , GCP , Kubeflow , CI/CD , Python , BigDataJob Description:Identify new opportunities to improve business processes and improve consumer experiences, and prototype solutions to demonstrate value with a crawl, walk,...

  • Senior Data Architect

    5 days ago

    Irving, Texas, United States Verizon Full time

    We are seeking a highly skilled Senior Data Engineer to join our team at Verizon. As a key member of our data engineering team, you will play a critical role in designing and implementing large-scale data platforms that drive business outcomes.Job ResponsibilitiesThe successful candidate will have experience in:Collecting and loading data from multiple...

  • Data Engineer with Machine Learning background

    6 days ago

    Irving, United States Indotronix International Corporation Full time

    Title: Data Analyst - IT - IV (Data Engineer with Machine Learning background) Location: Irving TX- Preferred Working Model: Hybrid. Expected to be in the office a min of 2 days a week Primary Location: Irving Tx- also open to Alpharetta , GA and Ashburn, VA Duration: 07/14/2024 to 07/12/2025 Max Bill: $130.00 Positions: 1 Description: ...