Big Data Hadoop Engineer

2 months ago

Charlotte, United States Aloden, Inc. Full time
Technology and Data - Software Engineer 4 - Contingent (Big Data Hadoop Engineer)
Location:Charlotte, NC - 28202 - Hybrid Roles (3 Days Onsite/2 DaysWFH)

Job Descriptions:
  • In this contingent resource assignment, you may: Consult on complex initiatives with broad impact and large-scale planning for Software Engineering. Review and analyze complex multi-faceted, larger scale or longer-term Software Engineering challenges that require in-depth evaluation of multiple factors including intangibles or unprecedented factors.
  • Contribute to the resolution of complex and multi-faceted situations requiring solid understanding of the function, policies, procedures, and compliance requirements that meet deliverables.
  • Strategically collaborate and consult with client personnel.
Required Qualifications:
  • 5+ years of Software Engineering experience, or equivalent demonstrated through one or a combination of the following: work or consulting experience, training, military experience, education.
  • Design and implement automated spark-based framework to facilitate data ingestion, transformation and consumption.
  • Implement security protocols such as Kerberos Authentication, Encryption of data at rest, data authorization mechanism such as role-based access control using Apache ranger.
  • Design and develop automated testing framework to perform data validation.
  • Enhance existing spark-based frameworks to overcome tool limitations, and/or to add more features based on consumer expectations.
  • Design and build high performing and scalable data pipeline platform using Hadoop, Apache Spark, MongoDB, Kafka and object storage architecture.
  • Work with Infrastructure Engineers and System Administrators as appropriate in designing the big-data infrastructure
  • Collaborate with application partners, Architects, Data Analysts and Modelers to build scalable and performant data solutions.
  • Effectively work in a hybrid environment where legacy ETL and Data Warehouse applications and new big-data applications co-exist
  • Work with Infrastructure Engineers and System Administrators as appropriate in designing the big-data infrastructure.
  • Support ongoing data management efforts for Development, QA and Production environments
  • Provide tool support, help consumers troubleshooting pipeline issues.
  • Utilizes a thorough understanding of available technology, tools, and existing designs.
  • Leverage knowledge of industry trends to build best in class technology to provide competitive advantage.
Required Qualification:
  • 5+ years of experience of software engineering experience
  • 5+ years of experience delivering complex enterprise wide information technology solutions
  • 5+ years of experience delivering ETL, data warehouse and data analytics capabilities on big-data architecture such as Hadoop
  • 5+ years of Apache Spark design and development experience using Scala, Java, Python or Data Frames with Resilient Distributed Datasets (RDDs), Parquet or ORC file formats
  • 6+ years of ETL (Extract, Transform, Load) Programming experience
  • 2+ years of Kafka or equivalent experience
  • 2+ years of NoSQL DB like Couchbase/MongoDB experience.
  • 5+ experience working with complex SQLs and performance tuning
Desired Qualification:
  • 3+ years of Agile experience
  • 2+ years of reporting experience, analytics experience or a combination of both
  • 2+ years of operational risk or credit risk or compliance domain experience
  • 2+ years of experience integrating with RESTful API
  • 2+ years of experience with CICD tools.

  • Senior Data Engineer

    3 weeks ago

    Charlotte, North Carolina, United States ApTask Full time

    About This Role:Candidates for the Senior Data Engineer - Big Data Hadoop position at ApTask will be responsible for designing, developing, and implementing large-scale data processing systems using Big Data Hadoop. The ideal candidate will have experience working with big data technologies, including Hadoop, PySpark, and Teradata, and will be able to...

  • Big Data Hadoop Engineer

    4 weeks ago

    Charlotte, United States Aloden, Inc. Full time

    Technology and Data - Software Engineer 4 - Contingent (Big Data Hadoop Engineer) Location:Charlotte, NC - 28202 - Hybrid Roles (3 Days Onsite/2 DaysWFH) Job Descriptions: In this contingent resource assignment, you may: Consult on complex initiatives with broad impact and large-scale planning for Software Engineering. Review and analyze complex...

  • Big Data Hadoop Engineer

    3 weeks ago

    Charlotte, United States Aloden, Inc. Full time

    Technology and Data - Software Engineer 4 - Contingent (Big Data Hadoop Engineer) Location:Charlotte, NC - 28202 - Hybrid Roles (3 Days Onsite/2 DaysWFH) Job Descriptions: In this contingent resource assignment, you may: Consult on complex initiatives with broad impact and large-scale planning for Software Engineering. Review and analyze complex...

  • Big Data Engineer

    2 months ago

    Charlotte, United States Diverse Lynx Full time

    Role-Big Data Engineer Location-Charlotte, NC Type- ContractRole Description: Data Engineer - Bigdata, Hadoop, Spark, Scala, PySpark, Strong SQL, Cloudera Data Platform (CDP), Primary Skills: Big Data: Spark, Scala, Pyspark, HDFS Microsoft Stack: MS-SQL with strong knowledge in RDBMS concepts Hadoop (Hortonworks) and Hiveo Cloudera Data platform (CDP)...

  • Big Data Engineer

    4 weeks ago

    Charlotte, United States Diverse Lynx Full time

    Big Data Engineer with mapreduce , spark , hive , sql skillset" Expert in SQL and Data warehousing concepts. Hands-on experience with public cloud data warehouse (GCP, Azure, AWS). GCP certification will be very good to have. MapR experience is must Strong Hands on experience with one or more programming languages ( Python or Java). Hands-on...

  • Big Data/Hadoop QA Testing

    5 months ago

    Charlotte, United States Indotronix International Corporation Full time

    Role: Big Data Testing Primary Skill: Hadoop/Teradata/Agile Testing (Must have) Secondary Skill: Tosca (Good to have) Job Description: Experienced QA with expertise in Agile Functional, Teradata, Hadoop and Automation testing. Ability to lead the QA team across various LOBs. Ability to interface with onshore and offshore QA team for all QA and Cross...

  • Big Data Engineer

    3 weeks ago

    Charlotte, North Carolina, United States Atlantis IT group Full time

    About the RoleAt Atlantis IT group, we are seeking an experienced Big Data Engineer to join our team in Charlottte. This is a full-time W2 position that requires US citizenship or a Green Card.We need someone with expertise in Hadoop, Python, and Spark, as well as Pyspark and custom-built Spark SQL. MapR experience is preferred, but Hortonworks and Cloudera...

  • Big Data Engineering Manager

    5 days ago

    Charlotte, North Carolina, United States Veracity Full time

    Big Data Engineering ManagerWe are looking for an experienced Big Data Engineering Manager to lead our team in designing and implementing big data solutions using cutting-edge technologies such as Apache Spark, Hadoop, and NoSQL databases. The ideal candidate has 5+ years of experience in software engineering, with expertise in big data technologies,...

  • Big Data Engineering Lead

    4 weeks ago

    Charlotte, North Carolina, United States Motion Recruitment Full time

    We are seeking a highly skilled Big Data Engineering Lead to join our team at Motion Recruitment in Charlotte, NC. This full-time consulting assignment will be a long-term opportunity for the right candidate.The ideal candidate should have a strong understanding of big data development and experience working with Hadoop, Spark, Python, Scala, Unix, and SQL.A...

  • Big Data Engineer 4

    2 weeks ago

    Charlotte, United States TEKsystems Full time

    Description The Big Data Engineer is responsible for owning and driving the technical innovation along with big data technologies. The individual is a subject matter expert technologist with strong Python experience and deep hands-on experience building data pipelines for the Hadoop platform as well as Google cloud. This person will be part of successful Big...

  • Big Data Engineer 4

    2 weeks ago

    Charlotte, United States TEKsystems Full time

    *Description* The Big Data Engineer is responsible for owning and driving the technical innovation along with big data technologies. The individual is a subject matter expert technologist with strong Python experience and deep hands-on experience building data pipelines for the Hadoop platform as well as Google cloud. This person will be part of successful...

  • Senior Big Data Engineer

    1 week ago

    Charlotte, North Carolina, United States The Hartford Full time

    About The RoleThe Hartford is seeking a highly experienced Senior Big Data Engineer to join its team. As a Senior Big Data Engineer, you will design and implement large-scale data products that meet the company's needs.This is an exciting opportunity for individuals who want to shape the future of data engineering and drive innovation. You will work closely...

  • Senior Big Data Engineer

    3 weeks ago

    Charlotte, North Carolina, United States Diverse Lynx Full time

    Job DescriptionWe are seeking an experienced Data Engineer to join our team at Diverse Lynx LLC. As a key member of our data engineering group, you will design, build, and maintain large-scale data processing systems using big data technologies such as Spark, Scala, and PySpark.The ideal candidate will have a strong background in big data, including...

  • Big Data Engineer with AWS Experience

    1 week ago

    Charlotte, North Carolina, United States Global Channel Management Full time

    About the RoleWe are seeking an experienced Data Engineer to join our team and lead our efforts in developing and maintaining our big data platform.Job Summary:This is an exciting opportunity to leverage your skills and experience in Big Data technologies to drive business growth and innovation.Key Responsibilities:Design, develop, and maintain high-quality...

  • Senior Big Data Developer

    1 week ago

    Charlotte, North Carolina, United States ApTask Full time

    About UsApTask is a leading global provider of workforce solutions and talent acquisition services. We offer a comprehensive suite of services, including staffing, recruitment, managed services, IT consulting, and project management. Our mission is to shape the future of work and empower organizations to thrive while fostering a diverse and inclusive work...

  • Senior Big Data Architect

    1 week ago

    Charlotte, North Carolina, United States American Homes 4 Rent Full time

    Job Title: Senior Big Data ArchitectWe are seeking an experienced Senior Big Data Architect to lead our big data initiatives and design scalable data architectures. The ideal candidate will have a strong background in big data technologies and experience with data warehousing, ETL processes, and data visualization tools.The estimated salary for this position...

  • Big Data Systems Manager

    1 week ago

    Charlotte, North Carolina, United States TechnoGen Full time

    Job Title: Big Data Systems ManagerLocation: Charlotte, NCWe are seeking a highly skilled Big Data Systems Manager to join our team at TechnoGen. The ideal candidate will have strong experience with big data technologies such as Hadoop, Spark, and Kafka, and a solid understanding of data modeling concepts and data management best...

  • Data Engineer for Hadoop and SQL Server

    5 days ago

    Charlotte, North Carolina, United States Syntricate Technologies Full time

    Job Description\We are looking for a highly skilled Data Engineer for Hadoop and SQL Server to join our team at Syntricate Technologies. As a key member of our data engineering team, you will play a crucial role in designing, developing, and implementing large-scale data processing systems using Hadoop and SQL Server technologies.\\ Responsibilities:- Design...

  • Hadoop and Data Warehousing Expert

    1 week ago

    Charlotte, North Carolina, United States ApTask Full time

    Why Join ApTaskAt ApTask, we offer unparalleled opportunities for professional growth and development. Our team is dedicated to shaping the future of work, and we believe that by joining us, you will have the chance to make a real impact.We are seeking an experienced Data Engineer to join our team, with a strong ability to develop and implement Big Data...

  • Data Science Engineer

    1 week ago

    Charlotte, North Carolina, United States Aloden, Inc. Full time

    Job Title: Data Science EngineerAloden, Inc. is seeking a highly skilled Data Science Engineer to join our team in Charlotte, NC. As a key member of our engineering team, you will be responsible for developing and implementing advanced data science solutions using Python, Big Query, Hadoop, and Linux.We are offering a competitive salary of $140,000 -...