Big Data Developer Lead- Hive HDFS Spark Scala

3 weeks ago

Irving, United States LTIMindtree Full time

About LTIMindtree


LTIMindtree is a global technology consulting and digital solutions company that enables enterprises across industries to reimagine business models, accelerate innovation, and maximize growth by harnessing digital technologies. As a digital transformation partner to more than 750 clients, LTIMindtree brings extensive domain and technology expertise to help drive superior competitive differentiation, customer experiences, and business outcomes in a converging world. Powered by nearly 90,000 talented and entrepreneurial professionals across more than 30 countries, LTIMindtree — a Larsen & Toubro Group company — combines the industry-acclaimed strengths of erstwhile Larsen and Toubro Infotech and Mindtree in solving the most complex business challenges and delivering transformation at scale. For more information, please visit www.ltimindtree.com.



Job Title: Big DATA Developer

Work Location- Irving TX

Job Description:

The Data Engineering Technology is a senior level position responsible for establishing and implementing new or revised data platform eco systems and programs in coordination with the Technology team The overall objective of this role is to lead data engineering team to implement the business requirements

Responsibilities

  • Design build and maintain batch or realtime data pipelines in data platform
  • Maintain and optimize the data infrastructure required for accurate extraction transformation and loading of data from a wide variety of data sources
  • Develop ETL extract transform load processes to help extract and manipulate data from multiple sources
  • Automate data workflows such as data ingestion aggregation and ETL processing
  • Prepare raw data in Data Warehouses into a consumable dataset for both technical and nontechnical stakeholders
  • Partner with data scientists and functional leaders in sales marketing and product to deploy machine learning models in production
  • Build maintain and deploy data products for analytics and data science teams on data platform
  • Ensure data accuracy integrity privacy security and compliance through quality control procedures
  • Monitor data systems performance and implement optimization strategies
  • Leverage data controls to maintain data privacy security compliance and quality for allocated areas of ownership
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems projects through indepth evaluation of complex business processes system processes and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding testing debugging and implementation
  • Develop comprehensive knowledge of how areas of business such as architecture and infrastructure integrate to accomplish business goals
  • Provide indepth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to midlevel developers and analysts allocating work as necessary
  • Appropriately assess risk when business decisions are made demonstrating particular consideration for the firms reputation and safeguarding client its clients and assets by driving compliance with applicable laws rules and regulations adhering to Policy applying sound ethical judgment regarding personal behavior conduct and business practices and escalating managing and reporting control issues with transparency

Qualifications

  • 6 years of total experience 6 years relevant experience in Data engineering role
  • Advanced SQL skills and experience with relational databases and database design
  • Strong proficiency in object oriented languages Python Pyspark is must
  • Experience working with data ingestion tools such as Talend Ab Initio
  • Experience working with data lakehouse architecture such as Iceberg Starburst
  • Strong proficiency in scripting languages like Bash
  • Strong proficiency in data pipeline and workflow management tools
  • Strong proficiency in scripting languages like Bash
  • Strong project management and organizational skills
  • Excellent problem solving communication and organizational skills
  • Proven ability to work independently and with a team
  • Experience in managing and implementing successful projects
  • Ability to adjust priorities quickly as circumstances dictate
  • Demonstrated leadership and project management skills
  • Consistently demonstrates clear and concise written and verbal communication

Education

  • Bachelors degree University degree or equivalent experience
  • Masters degree preferred


LTIMindtree is an equal opportunity employer that is committed to diversity in the workplace. Our employment decisions are made without regard to race, color, creed, religion, sex (including pregnancy, childbirth or related medical conditions), gender identity or expression, national origin, ancestry, age, family-care status, veteran status, marital status, civil union status, domestic partnership status, military service, handicap or disability or history of handicap or disability, genetic information, atypical hereditary cellular or blood trait, union affiliation, affectional or sexual orientation or preference, or any other characteristic protected by applicable federal, state, or local law, except where such considerations are bona fide occupational qualifications permitted by law.

  • Java Spark- Hive, sqoop, HDFS

    3 months ago

    Irving, United States LTIMindtree Full time

    About LTIMindtreeLTIMindtree is a global technology consulting and digital solutions company that enables enterprises across industries to reimagine business models, accelerate innovation, and maximize growth by harnessing digital technologies. As a digital transformation partner to more than 750 clients, LTIMindtree brings extensive domain and technology...

  • Java Spark- Hive, sqoop, HDFS

    3 months ago

    Irving, United States LTIMindtree Full time

    About LTIMindtreeLTIMindtree is a global technology consulting and digital solutions company that enables enterprises across industries to reimagine business models, accelerate innovation, and maximize growth by harnessing digital technologies. As a digital transformation partner to more than 750 clients, LTIMindtree brings extensive domain and technology...

  • Big Data Developer

    3 days ago

    Irving, Texas, United States LTIMINDTREE Full time

    Job Title: Big Data Developer**Job Summary:**We are seeking a highly skilled Big Data Developer to join our Quality Engineering team at LTIMindtree. As a Big Data Developer, you will be responsible for developing automation frameworks for testing and manual testing until it is fully automated.Key Responsibilities:Develop automation solutions to validate the...

  • Spark/Big Data Developer

    4 months ago

    Irving, United States QData Full time

    Required Skills & Experience 5+ Years of application development experience Strong experience in Apache Spark Development Proficient in server side development with Java and J2EE Experience with Hadoop Ecosystem (HDFS HIVE HBASE etc.) Knowledge of Kafka Streaming including topics producers consumers Well versed in Linux Environment Familiarity with...

  • Spark/Big Data Developer

    4 months ago

    Irving, Texas, United States QData Full time

    Required Skills & Experience 5+ Years of application development experience Strong experience in Apache Spark Development Proficient in server side development with Java and J2EE Experience with Hadoop Ecosystem (HDFS HIVE HBASE etc.) Knowledge of Kafka Streaming including topics producers consumers Well versed in Linux Environment Familiarity with In-memory...

  • Data Engineer

    1 week ago

    Irving, United States InfoVision Full time

    • Experience with big data processing technology (e.g., Hadoop, Apache Spark, Pyspark, Python, Scala, HDFS, Hive, Impala etc.,), data warehouse technical architecture, infrastructure components, and reporting/analytic tools and environments. • Experience with any Big Data Platforms: Cloudera / Horton Works / Snowflake / AWS EMR / RedShift/ AWS Glue. •...

  • Big Data Developer Lead- Hive HDFS Spark Scala

    3 weeks ago

    Irving, United States LTIMindtree Full time

    About LTIMindtreeLTIMindtree is a global technology consulting and digital solutions company that enables enterprises across industries to reimagine business models, accelerate innovation, and maximize growth by harnessing digital technologies. As a digital transformation partner to more than 750 clients, LTIMindtree brings extensive domain and technology...

  • Data Engineer Lead

    3 weeks ago

    Irving, United States Newt Global Full time

    Data Engineer Lead / ArchitectJacksonville, FL / Irving, TX (Hybrid – 3 Days a Week)Contract Type: W2 / Self CorporationLong TermQualifications: Over 10+ years of total IT experience 5+ years of experience with Hadoop (Cloudera)/big data technologies Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop...

  • Data Engineer

    21 hours ago

    Irving, Texas, United States Newt Global Full time

    About Newt GlobalNewt Global is a leading provider of data engineering services, helping businesses unlock the full potential of their data.Job InformationPosition: Data EngineerLocation: Remote (Hybrid)Duration: Contract (W2/Self Corp)Job Description:Key Responsibilities:Design, develop, and maintain large-scale data processing systems using Hadoop and...

  • Sr Big Data- Java Spark combination

    3 months ago

    Irving, United States LTIMindtree Full time

    About LTIMindtreeLTIMindtree is a global technology consulting and digital solutions company that enables enterprises across industries to reimagine business models, accelerate innovation, and maximize growth by harnessing digital technologies. As a digital transformation partner to more than 750 clients, LTIMindtree brings extensive domain and technology...

  • Bigdata Engineer

    2 weeks ago

    Irving, United States Resource Informatics Group Full time

    Bigdata Engineer Location: Iselin, NJ - In office: 3 days a week onsite Contract: 6-24 months to perm (may C2H) Required only W2 Big Data Developer Strong Experience with Big Data platforms/tools (Linux, HDFS, Hive, Hortonworks, Spark, Scala, Python, Java, Kafka, etc.) Experience with API's, reading from and/or designing new Ability to write SQL and interact...

  • Lead Java Spark Developer

    2 months ago

    Irving, United States Infosys Full time

    Infosys is seeking a Lead Java Spark Developer. In the role, you will interface with key stakeholders and apply your technical proficiency across different stages of the Software Development Life Cycle in Java and Spark. You will also be expected to build high-throughput, fault tolerant systems with requirements including; elicitation, application...

  • Lead Spark Developer with Java

    2 months ago

    Irving, United States Infosys Full time

    Infosys is seeking a Lead Java Spark Developer. In the role, you will interface with key stakeholders and apply your technical proficiency across different stages of the Software Development Life Cycle in Java and Spark. You will also be expected to build high-throughput, fault tolerant systems with requirements including; elicitation, application...

  • Spark developer

    2 months ago

    Irving, United States Infosys Full time

    Infosys is seeking a Spark Developer. In this role, you will enable digital transformation for our clients in a global delivery model, research on technologies independently, recommend appropriate solutions and support technology-specific best practices and standards. You will be responsible to collaborate with key partners and apply your technical...

  • Spark developer

    2 months ago

    Irving, United States Infosys Full time

    Infosys is seeking a Spark Developer. In this role, you will enable digital transformation for our clients in a global delivery model, research on technologies independently, recommend appropriate solutions and support technology-specific best practices and standards. You will be responsible to collaborate with key partners and apply your technical...

  • Big Data Engineer

    5 months ago

    Irving, United States Maveric Systems Limited Full time

    Big Data Testing on Hadoop Hive etc. using HiveQL MapReduce or other big data technologies for automated validation Implementing DevOps / CI-CD using Bitbucket Jenkins etc. Implementing Test Automation Frameworks for back-end validation including RDBMS file Kafka sources and Big Data targets Working in an Agile environment on Jira ALM Confluence...

  • Big Data Engineer

    5 months ago

    Irving, Texas, United States Maveric Systems Limited Full time

    Big Data Testing on Hadoop Hive etc. using HiveQL MapReduce or other big data technologies for automated validation Implementing DevOps / CI-CD using Bitbucket Jenkins etc. Implementing Test Automation Frameworks for back-end validation including RDBMS file Kafka sources and Big Data targets Working in an Agile environment on Jira ALM Confluence etc. Big...

  • Senior Java Spark Developer

    2 months ago

    Irving, United States Infosys Full time

    Required QualificationsBachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.At least 8 years of Information Technology experience.Experience in Java, Spark, Hadoop, Impala, ClouderaPreferred QualificationsAt least 7 to 8...

  • Senior Java Spark Developer

    2 months ago

    Irving, United States Infosys Full time

    Required QualificationsBachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.At least 8 years of Information Technology experience.Experience in Java, Spark, Hadoop, Impala, ClouderaPreferred QualificationsAt least 7 to 8...

  • Lead Java Spark Engineer

    2 weeks ago

    Irving, Texas, United States Infosys Full time

    Essential QualificationsA Bachelor's degree or an equivalent foreign qualification from a recognized institution is required. Candidates with three years of progressive experience in the field may be considered in lieu of each year of education.Minimum of 8 years of experience in Information Technology.Proficiency in Java, Spark, Hadoop, Impala, and Cloudera...