Big Data Developer Lead- Hive HDFS Spark Scala

3 months ago

Irving, United States LTIMindtree Full time

About LTIMindtree


LTIMindtree is a global technology consulting and digital solutions company that enables enterprises across industries to reimagine business models, accelerate innovation, and maximize growth by harnessing digital technologies. As a digital transformation partner to more than 750 clients, LTIMindtree brings extensive domain and technology expertise to help drive superior competitive differentiation, customer experiences, and business outcomes in a converging world. Powered by nearly 90,000 talented and entrepreneurial professionals across more than 30 countries, LTIMindtree — a Larsen & Toubro Group company — combines the industry-acclaimed strengths of erstwhile Larsen and Toubro Infotech and Mindtree in solving the most complex business challenges and delivering transformation at scale. For more information, please visit www.ltimindtree.com.



Job Title: Big DATA Developer

Work Location- Irving TX

Job Description:

The Data Engineering Technology is a senior level position responsible for establishing and implementing new or revised data platform eco systems and programs in coordination with the Technology team The overall objective of this role is to lead data engineering team to implement the business requirements

Responsibilities

  • Design build and maintain batch or realtime data pipelines in data platform
  • Maintain and optimize the data infrastructure required for accurate extraction transformation and loading of data from a wide variety of data sources
  • Develop ETL extract transform load processes to help extract and manipulate data from multiple sources
  • Automate data workflows such as data ingestion aggregation and ETL processing
  • Prepare raw data in Data Warehouses into a consumable dataset for both technical and nontechnical stakeholders
  • Partner with data scientists and functional leaders in sales marketing and product to deploy machine learning models in production
  • Build maintain and deploy data products for analytics and data science teams on data platform
  • Ensure data accuracy integrity privacy security and compliance through quality control procedures
  • Monitor data systems performance and implement optimization strategies
  • Leverage data controls to maintain data privacy security compliance and quality for allocated areas of ownership
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems projects through indepth evaluation of complex business processes system processes and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding testing debugging and implementation
  • Develop comprehensive knowledge of how areas of business such as architecture and infrastructure integrate to accomplish business goals
  • Provide indepth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to midlevel developers and analysts allocating work as necessary
  • Appropriately assess risk when business decisions are made demonstrating particular consideration for the firms reputation and safeguarding client its clients and assets by driving compliance with applicable laws rules and regulations adhering to Policy applying sound ethical judgment regarding personal behavior conduct and business practices and escalating managing and reporting control issues with transparency

Qualifications

  • 6 years of total experience 6 years relevant experience in Data engineering role
  • Advanced SQL skills and experience with relational databases and database design
  • Strong proficiency in object oriented languages Python Pyspark is must
  • Experience working with data ingestion tools such as Talend Ab Initio
  • Experience working with data lakehouse architecture such as Iceberg Starburst
  • Strong proficiency in scripting languages like Bash
  • Strong proficiency in data pipeline and workflow management tools
  • Strong proficiency in scripting languages like Bash
  • Strong project management and organizational skills
  • Excellent problem solving communication and organizational skills
  • Proven ability to work independently and with a team
  • Experience in managing and implementing successful projects
  • Ability to adjust priorities quickly as circumstances dictate
  • Demonstrated leadership and project management skills
  • Consistently demonstrates clear and concise written and verbal communication

Education

  • Bachelors degree University degree or equivalent experience
  • Masters degree preferred


LTIMindtree is an equal opportunity employer that is committed to diversity in the workplace. Our employment decisions are made without regard to race, color, creed, religion, sex (including pregnancy, childbirth or related medical conditions), gender identity or expression, national origin, ancestry, age, family-care status, veteran status, marital status, civil union status, domestic partnership status, military service, handicap or disability or history of handicap or disability, genetic information, atypical hereditary cellular or blood trait, union affiliation, affectional or sexual orientation or preference, or any other characteristic protected by applicable federal, state, or local law, except where such considerations are bona fide occupational qualifications permitted by law.

  • Big Data Developer

    5 days ago

    Irving, United States Resource Informatics Group Full time

    Note: It is a W2 Opportunity, Client is accepting candidates from (New Jersey and New York) other location candidates will be straight away rejected and application candidate must have prior banking domain work experience. Role: Big Data Developer Location: Iselin, NJ In office: 3 days a week onsite Contract: 6-24 months to perm Interview process: screening...

  • Big Data Specialist

    2 weeks ago

    Irving, United States Tata Consultancy Services Full time

    Job DescriptionBig Data (PySpark) Tech Lead10 + Years Overall Experience in Data Management, Data Lake and Data Warehouse6+ Years Hadoop, Hive, Sqoop, SQL, Teradata6+ Years PySpark(Python and Spark), UnixGood to have Industry leading ETL experienceBanking Domain experience Key ResponsibilitiesAbility to design, build and unit test applications on Spark...

  • Senior Big Data Engineer

    2 months ago

    Irving, United States Anblicks Full time

    Description: The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for design and development of big data...

  • Senior Big Data Engineer

    3 weeks ago

    Irving, United States Anblicks Full time

    Description: The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for design and development of big data...

  • Lead Big Data Engineer

    3 weeks ago

    Irving, United States Infovision Full time

    Job title: Lead Big Data Engineer (GCP Cloud)Location: Irving, TXDuration: Long-termSkills Needed:10+ years of experience in designing and building data pipelines in large-scale distributed systems.Lead the design, development, and maintenance of scalable batch and real-time data processing pipelines.Proficiency with Google Cloud Platform (GCP) and tools...

  • Lead Big Data Engineer

    1 month ago

    Irving, United States InfoVision Inc. Full time

    Job title: Lead Big Data Engineer (GCP Cloud)Location: Irving, TXDuration: Long-term Skills Needed:10+ years of experience in designing and building data pipelines in large-scale distributed systems.Lead the design, development, and maintenance of scalable batch and real-time data processing pipelines.Proficiency with Google Cloud Platform (GCP) and tools...

  • Senior Big Data Engineer

    3 weeks ago

    Irving, TX, United States Anblicks Full time

    Description: The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for design and development of big data...

  • Data Engineer

    3 weeks ago

    Irving, United States Newt Global Full time

    Big Data EngineerIrving, TX - HybridW2 ContractDirect ClientQualifications:8+ years of experience in hadoop/big data technologies.3+ years of experience in spark.2+ years experience in Snowflake2+ year of experience working on Google or AWS cloud developing data solutions. Certifications preferred.Hands-on experience with Python/Pyspark/Scala and basic...

  • Data engineer

    6 days ago

    Irving, United States Tata Consultancy Services Full time

    Position: Data Engineer (PySpark)Duration: Full TimeLocation - Irving, TX (onsite)Pay -130-140k+benefits Job Description Big Data (PySpark) Tech Lead– • 10+ Years Overall Experience in Data Management, Data Lake and Data Warehouse• 8+ Years Hadoop, Hive, Sqoop, SQL, Teradata• 8+ Years PySpark(Python and Spark), Unix• Good to have Industry leading...

  • Data Engineering Lead

    4 weeks ago

    Irving, Texas, United States INSPYR Solutions Full time

    Job SummaryINSPYR Solutions is seeking a highly skilled Data Engineering Lead to join our research group team. The ideal candidate will have a strong background in data engineering, data warehousing, and cloud platforms, with experience in leading the design and development of robust data pipelines, streaming ingestion frameworks, and scalable data...

  • Senior Data Engineer

    1 month ago

    Irving, Texas, United States Citigroup Inc Full time

    Citigroup Inc. is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing and developing Big Data solutions using next-generation data techniques. You will work closely with business users and technologists to exchange information about solutions, including requirements and usage...

  • Lead/Sr. Big Data Engineer

    3 weeks ago

    Irving, United States America Technology Professionals LLC Full time

    Job Title: Lead / Sr. Big Data Engineer Location: Irving, TX (Hybrid – 2-3 days/week) Duration: 12+ Months USC/GC/H1B only C2C/1099 only This role can play the hand-on dev role along with tech lead skills Required Skills: Big data expert with 9+ years experience in Hadoop Big data ecosystem Spark - Batch Streaming (Python, Scala) Experience in cloud...

  • Lead/Sr. Big Data Engineer

    4 weeks ago

    Irving, United States America Technology Professionals LLC Full time

    Job Title: Lead / Sr. Big Data EngineerApply now, read the job details by scrolling down Double check you have the necessary skills before sending an application.Location: Irving, TX (Hybrid – 2-3 days/week)Duration: 12+ MonthsUSC/GC/H1B onlyC2C/1099 onlyThis role can play the hand-on dev role along with tech lead skillsRequired Skills:Big data expert with...

  • Lead / Sr. Big Data Engineer

    3 weeks ago

    Irving, United States Infovision Full time

    Job Title: Lead / Sr. Big Data EngineerLocation: Irving, TX (Hybrid 2-3 days/week)Duration: 12+ MonthsRequired Skills: Big data expert with 9+ years experience in Hadoop Big data ecosystemSpark - Batch Streaming (Python, Scala)Experience in cloud environment specially Google Cloud PlatformExperience in developing both batch and real-time streaming data...

  • Lead / Sr. Big Data Engineer

    1 month ago

    Irving, United States InfoVision Inc. Full time

    Job Title: Lead / Sr. Big Data EngineerLocation: Irving, TX (Hybrid 2-3 days/week)Duration: 12+ Months Required Skills: Big data expert with 9+ years experience in Hadoop Big data ecosystemSpark - Batch Streaming (Python, Scala)Experience in cloud environment specially Google Cloud PlatformExperience in developing both batch and real-time streaming data...

  • Lead Data Engineer

    2 weeks ago

    Irving, TX, United States INSPYR Solutions Full time

    Title: Lead Data Engineer If you are interested in applying for this job, please make sure you meet the following requirements as listed below. Location: Irving, TX 75039 (Hybrid) Duration: Direct Hire Compensation: $175K - $200K Work Requirements: US Citizen, GC Holders or Authorized to Work in the U.S. Skillset / Experience: We are seeking a highly...

  • SQL Developer

    2 months ago

    Irving, United States NTT DATA Full time

    Req ID: 298203 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a SQL Developer - Irving, Texas to join our team in Irving, Texas (US-TX), United States (US).NTT Data Americas is...

  • Senior Data Engineer

    4 weeks ago

    Irving, Texas, United States Diverse Lynx Full time

    Job Title: Senior Data Engineer - Apache Spark DeveloperRole Overview:At Diverse Lynx LLC, we are seeking a highly skilled Senior Data Engineer - Apache Spark Developer to join our team. The ideal candidate will have expertise in Apache Spark, PySpark, and big data engineering. As a Senior Data Engineer, you will be responsible for designing, developing, and...

  • Data Scientist with Machine Learning Expertise

    4 weeks ago

    Irving, Texas, United States Indotronix International Corporation Full time

    Job Summary:Indotronix International Corporation is seeking a highly skilled Data Scientist with Machine Learning expertise to drive profitable growth and business innovation. The ideal candidate will apply cutting-edge machine learning techniques and AI technology to lead data science projects that drive channel optimization, better customer experience,...

  • Senior Data Engineer

    1 month ago

    Irving, Texas, United States OTSI Full time

    Job Title: Senior Data EngineerJob Summary:OTSI is seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing and developing methods to connect and collect data from different data sources, as well as designing and developing SQL, Hive queries, and APIs to extract data from the store.Key...