Big Data Developer Lead- Hive HDFS Spark Scala

22 hours ago

Irving, Texas, United States LTIMindtree Full time
About LTIMindtree

LTIMindtree is a global technology consulting and digital solutions company that enables enterprises to reimagine business models, accelerate innovation, and maximize growth by harnessing digital technologies. As a digital transformation partner to more than 750 clients, LTIMindtree brings extensive domain and technology expertise to help drive superior competitive differentiation, customer experiences, and business outcomes in a converging world.

Our company is powered by nearly 90,000 talented and entrepreneurial professionals across more than 30 countries, combining the industry-acclaimed strengths of erstwhile Larsen and Toubro Infotech and Mindtree in solving the most complex business challenges and delivering transformation at scale.

Job Title: Big Data Developer LeadWork Location: Irving, TX

The Data Engineering Technology Lead is a senior-level position responsible for establishing and implementing new or revised data platform ecosystems and programs in coordination with the Technology team. The overall objective of this role is to lead data engineering teams to implement business requirements.

Key Responsibilities:
  1. Design, build, and maintain batch or real-time data pipelines in data platforms.
  2. Maintain and optimize the data infrastructure required for accurate extraction, transformation, and loading of data from a wide variety of data sources.
  3. Develop ETL extract, transform, and load processes to help extract and manipulate data from multiple sources.
  4. Automate data workflows such as data ingestion, aggregation, and ETL processing.
  5. Prepare raw data in Data Warehouses into a consumable dataset for both technical and non-technical stakeholders.
  6. Partner with data scientists and functional leaders in sales, marketing, and product to deploy machine learning models in production.
  7. Build, maintain, and deploy data products for analytics and data science teams on data platforms.
  8. Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures.
  9. Monitor data systems performance and implement optimization strategies.
  10. Leverage data controls to maintain data privacy, security, compliance, and quality for allocated areas of ownership.
  11. Partner with multiple management teams to ensure appropriate integration of functions to meet goals and identify and define necessary system enhancements to deploy new products and process improvements.
  12. Resolve high-impact problems and projects through in-depth evaluation of complex business processes, system processes, and industry standards.
  13. Provide expertise in areas and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint.
  14. Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation.
  15. Develop comprehensive knowledge of how areas of business such as architecture and infrastructure integrate to accomplish business goals.
  16. Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions.
  17. Serve as an advisor or coach to mid-level developers and analysts, allocating work as necessary.
  18. Assess risk when business decisions are made, demonstrating particular consideration for the company's reputation and safeguarding client assets by driving compliance with applicable laws, rules, and regulations, adhering to policy, applying sound ethical judgment regarding personal behavior, conduct, and business practices, and escalating, managing, and reporting control issues with transparency.
Qualifications:
  1. 6 years of total experience, 6 years relevant experience in Data engineering role.
  2. Advanced SQL skills and experience with relational databases and database design.
  3. Strong proficiency in object-oriented languages, Python, Pyspark is a must.
  4. Experience working with data ingestion tools such as Talend, Ab Initio.
  5. Experience working with data lakehouse architecture such as Iceberg, Starburst.
  6. Strong proficiency in scripting languages like Bash.
  7. Strong proficiency in data pipeline and workflow management tools.
  8. Strong project management and organizational skills.
  9. Excellent problem-solving, communication, and organizational skills.
  10. Proven ability to work independently and with a team.
  11. Experience in managing and implementing successful projects.
  12. Ability to adjust priorities quickly as circumstances dictate.
  13. Demonstrated leadership and project management skills.
  14. Consistently demonstrates clear and concise written and verbal communication.
Education:
  1. Bachelor's degree, University degree, or equivalent experience.
  2. Masters degree preferred.

LTIMindtree is an equal opportunity employer that is committed to diversity in the workplace. Our employment decisions are made without regard to race, color, creed, religion, sex (including pregnancy, childbirth, or related medical conditions), gender identity or expression, national origin, ancestry, age, family-care status, veteran status, marital status, civil union status, domestic partnership status, military service, handicap or disability, or any other characteristic protected by applicable federal, state, or local law, except where such considerations are bona fide occupational qualifications permitted by law.

  • Big Data Developer

    5 days ago

    Irving, Texas, United States LTIMINDTREE Full time

    Job Title: Big Data Developer**Job Summary:**We are seeking a highly skilled Big Data Developer to join our Quality Engineering team at LTIMindtree. As a Big Data Developer, you will be responsible for developing automation frameworks for testing and manual testing until it is fully automated.Key Responsibilities:Develop automation solutions to validate the...

  • Spark/Big Data Developer

    4 months ago

    Irving, Texas, United States QData Full time

    Required Skills & Experience 5+ Years of application development experience Strong experience in Apache Spark Development Proficient in server side development with Java and J2EE Experience with Hadoop Ecosystem (HDFS HIVE HBASE etc.) Knowledge of Kafka Streaming including topics producers consumers Well versed in Linux Environment Familiarity with In-memory...

  • Data Engineer

    3 days ago

    Irving, Texas, United States Newt Global Full time

    About Newt GlobalNewt Global is a leading provider of data engineering services, helping businesses unlock the full potential of their data.Job InformationPosition: Data EngineerLocation: Remote (Hybrid)Duration: Contract (W2/Self Corp)Job Description:Key Responsibilities:Design, develop, and maintain large-scale data processing systems using Hadoop and...

  • Senior Data Engineer Lead

    1 day ago

    Irving, Texas, United States LTIMindtree Full time

    About LTIMindtreeLTIMindtree is a global technology consulting and digital solutions company that enables enterprises across industries to reimagine business models, accelerate innovation, and maximize growth by harnessing digital technologies.As a digital transformation partner to more than 750 clients, LTIMindtree brings extensive domain and technology...

  • Big Data Engineer

    5 months ago

    Irving, Texas, United States Maveric Systems Limited Full time

    Big Data Testing on Hadoop Hive etc. using HiveQL MapReduce or other big data technologies for automated validation Implementing DevOps / CI-CD using Bitbucket Jenkins etc. Implementing Test Automation Frameworks for back-end validation including RDBMS file Kafka sources and Big Data targets Working in an Agile environment on Jira ALM Confluence etc. Big...

  • Lead Java Spark Engineer

    2 weeks ago

    Irving, Texas, United States Infosys Full time

    Essential QualificationsA Bachelor's degree or an equivalent foreign qualification from a recognized institution is required. Candidates with three years of progressive experience in the field may be considered in lieu of each year of education.Minimum of 8 years of experience in Information Technology.Proficiency in Java, Spark, Hadoop, Impala, and Cloudera...

  • Lead Java Spark Engineer

    2 weeks ago

    Irving, Texas, United States Infosys Full time

    Essential QualificationsA Bachelor's degree or an equivalent qualification from a recognized institution is required. Candidates with three years of progressive experience in the relevant field may be considered in lieu of each year of education.Minimum of 8 years of experience in Information Technology.Proficiency in Java, Spark, Hadoop, Impala, and...

  • Lead Java Spark Engineer

    2 weeks ago

    Irving, Texas, United States Infosys Full time

    Essential QualificationsA Bachelor's degree or an equivalent qualification from a recognized institution is required. Candidates with three years of relevant experience in the field may be considered in lieu of each year of education.Minimum of 8 years of experience in Information Technology.Proficiency in Java, Spark, Hadoop, Impala, and Cloudera is...

  • Lead Java Spark Engineer

    2 weeks ago

    Irving, Texas, United States Infosys Full time

    Essential QualificationsA Bachelor's degree or its international equivalent from a recognized institution is required. Alternatively, three years of relevant experience in the field may be considered in place of each year of education.Minimum of 8 years of experience in Information Technology.Proficiency in Java, Spark, Hadoop, Impala, and Cloudera.Desirable...

  • Lead Java Spark Engineer

    2 weeks ago

    Irving, Texas, United States Infosys Full time

    Essential QualificationsA Bachelor's degree or an equivalent foreign qualification from an accredited institution is required. Candidates with three years of progressive experience in the relevant field may be considered in lieu of each year of education.Minimum of 8 years of experience in Information Technology.Proficiency in Java, Spark, Hadoop, Impala,...

  • Lead Java Spark Engineer

    2 weeks ago

    Irving, Texas, United States Infosys Full time

    Essential QualificationsA Bachelor's degree or its international equivalent from a recognized institution is required. Alternatively, a combination of three years of progressive experience in the field may be considered in lieu of each year of education.Minimum of 8 years of experience in Information Technology.Proficiency in Java, Spark, Hadoop, Impala, and...

  • Lead Java Spark Engineer

    2 weeks ago

    Irving, Texas, United States Infosys Full time

    Essential QualificationsA Bachelor's degree or its equivalent from an accredited institution is required. Candidates with three years of progressive experience in the field may be considered in lieu of each year of education.Minimum of 8 years of experience in Information Technology.Proficiency in Java, Spark, Hadoop, Impala, and Cloudera is...

  • Lead Java Spark Engineer

    2 weeks ago

    Irving, Texas, United States Infosys Full time

    Essential QualificationsA Bachelor's degree or an equivalent international qualification from a recognized institution is mandatory. Consideration will also be given to candidates with three years of progressive experience in the relevant field in lieu of each year of formal education.A minimum of 8 years of experience in Information Technology is...

  • Big Data Specialist

    2 weeks ago

    Irving, Texas, United States Maveric Systems Limited Full time

    **Job Title: Data Engineer****Location:** Maveric Systems Limited**Job Type:** Full Time**Job Description:**We are seeking a highly skilled Data Engineer to join our team at Maveric Systems Limited. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data processing systems using Spark and Kafka.**Key...

  • Big Data Engineer

    4 months ago

    Irving, Texas, United States Maveric Systems Limited Full time

    Analyze and understand data sources & APIs Design and Develop methods to connect & collect data from different data sources Design and Develop methods to filter/cleanse the data Design and Develop SQL Hive queries APIs to extract data from the store Work closely with data Scientists to ensure the source data is aggregated and cleansed Work with product...

  • Hadoop Developer

    5 months ago

    Irving, Texas, United States Elajika Inc Full time

    Designing developing and implementing data service architecture that ingests real-time data streams parses JSON and loads them into multiple data persistence stores to provide real-time and offline analytics. Building fact tables to facilitate quicker and easier data access. Building indices at Elastic Search to support real-time dashboards at Kibana and...

  • Spark Software Engineer

    2 weeks ago

    Irving, Texas, United States Infosys Full time

    Infosys is seeking a Spark Developer. In this position, you will facilitate digital transformation for our clients within a global delivery framework, independently research technologies, propose suitable solutions, and uphold technology-specific best practices and standards. Your responsibilities will include collaborating with key stakeholders and applying...

  • Spark Software Engineer

    2 weeks ago

    Irving, Texas, United States Infosys Full time

    Infosys is looking for a Spark Developer. In this position, you will facilitate digital transformation for our clients through a global delivery model, independently research emerging technologies, propose suitable solutions, and uphold technology-specific best practices and standards. Your responsibilities will include collaborating with key stakeholders...

  • Spark Engineer

    2 weeks ago

    Irving, Texas, United States Maveric Systems Limited Full time

    **Job Title: Data Engineer****Location:** On-site/Hybrid**Job Type:** Full Time**Job Category:** Not open for C2C**Job Description:**We are seeking a highly skilled Data Engineer to join our team at Maveric Systems Limited. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data processing systems using Spark and...

  • Spark Software Engineer

    2 weeks ago

    Irving, Texas, United States Infosys Full time

    Infosys is looking for a Spark Developer. In this position, you will facilitate digital innovation for our clients within a global delivery framework, conduct independent research on technologies, propose suitable solutions, and uphold technology-specific best practices and standards. Your responsibilities will include collaborating with key stakeholders and...