Big Data Specialist

5 days ago


New York, New York, United States Diverse Lynx Full time

Diverse Lynx LLC is seeking a highly skilled Big Data Specialist to join our team. This is a full-time position with an estimated annual salary of $140,000.

The ideal candidate will have strong coding skills in Python, Java, or Scala for data processing tasks and experience with big data technologies such as Hadoop, Hive, or Redshift. They should also be proficient in integrating Single Sign-On (SSO) with Microsoft Azure AD using OAuth, OpenID Connect (OIDC), and SAML protocols. Additionally, they should have a solid understanding of relational and NoSQL databases such as PostgreSQL, MySQL, DynamoDB, or MongoDB.

Required Skills and Qualifications:

  • Proficiency in containerization technologies such as Docker and Kubernetes
  • Hands-on experience with front-end framework like React, Vue.js, Next.js, or Angular.js
  • Experience in infrastructure as code (IaC) using Terraform, particularly with AWS provider
  • Strong Coding skills in Python, Java, or Scala for data processing tasks
  • Proficient in integrating Single Sign-On (SSO) with Microsoft Azure AD using OAuth, OpenID Connect (OIDC), and SAML protocols
  • Solid Understanding of relational and NoSQL database, such as PostgreSQL, MySQL, DynamoDB, or MongoDB
  • Familiarity with data pipelines orchestration tools such as Airflow or Flask
  • Proficiency in data transformation and ETL process, including tools like Apache Spark or Glue
  • Strong experience with cloud platforms, especially AWS, including services like Lambda, S3, RDS, and Glue
  • Familiarity with big data technologies, including Hadoop, Hive, or Redshift
  • Experience with user authentication and authorization using AWS Cognito
  • Expertise in API development, security, and implementing best practices for scalable solutions
  • Experience with monitoring and logging tools, such as Prometheus, Grafana, or ELK Stack to ensure system reliability and application performance
  • Strong understanding of DevOps principles, CI/CD pipelines, and best practices
  • Experience with core AWS technologies such as EC2, ELB, Auto Scaling, S3, EFS, Lambda, API Gateway, Step Functions, CloudWatch, VPC, Route 53, ACM
  • Experience with BI tools like Tableau, AWS Quick Sight

  • Big Data Specialist

    5 days ago


    New York, New York, United States Yahoo Holdings Inc. Full time

    We are seeking a highly skilled Big Data Specialist to join our team at Yahoo Holdings Inc. As a key member of our team, you will play a critical role in designing and building efficient data pipelines to move and transform data at scale.The ideal candidate will have a strong background in computer science and experience in ETL design using Big Data stack...


  • New York, New York, United States Glow Networks Full time

    Job Title: Big Data Engineer SpecialistGlow Networks is looking for an experienced Big Data Engineer Specialist to join our team in New York City. This is a 6-month contract position with possible extensions, offering a salary of $120,000 per year.Job Overview:The ideal candidate will have extensive hands-on experience in big data engineering in cloud...


  • New York, New York, United States TechnoGen Full time

    About the RoleWe are looking for an experienced Big Data Engineer Specialist to join our team at TechnoGen. As a Big Data Engineer Specialist, you will be responsible for designing, developing, and optimizing data pipelines within the Snowflake data platform. Your expertise in AWS services, including Data Migration Service (DMS) and Kinesis, will be...


  • New York, New York, United States CapB InfoteK Full time

    Job OverviewWe are seeking a Big Data Engineering Specialist to join our team at CapB InfoteK. As a Big Data Engineering Specialist, you will be responsible for designing, developing and implementing highly efficient and scalable ETL processes.About the Role:Design develop and implement highly efficient highly scalable ETL processes.Strong knowledge of...

  • Big Data Engineer

    4 days ago


    New York, New York, United States Diverse Lynx Full time

    About the PositionWe are seeking a Big Data Engineer to join our team at Diverse Lynx LLC. As a key member of our data engineering team, you will be responsible for designing and implementing big data pipelines using Apache Kafka and other technologies.Key ResponsibilitiesDesign and implement big data pipelines using Apache Kafka and other...

  • Big Data Developer

    1 week ago


    New York, New York, United States Infojini Full time

    **Job Description:**We're seeking a talented Big Data Developer to join our team at Infojini. As a Big Data Developer, you'll be responsible for designing, developing, and implementing large-scale big data solutions using Hadoop, Spark, and other relevant technologies.About the Role:Design, develop, and implement large-scale big data solutionsCollaborate...


  • New York, New York, United States Yellowbrick Data Full time

    Job ResponsibilitiesThe Enterprise Sales Specialist will develop and execute a comprehensive sales strategy within their assigned territory, targeting high-potential prospects and setting clear goals to achieve sales targets. This role requires collaboration with the marketing team to design targeted marketing initiatives, prospect qualification, product...

  • Big Data Architect

    5 days ago


    New York, New York, United States ApTask Full time

    As a Big Data Architect at ApTask, you will play a critical role in shaping the company's data strategy and driving business outcomes through data-driven insights. This position requires a deep understanding of big data technologies, including Hadoop, NoSQL databases, and data processing frameworks. You will design and implement scalable data architectures,...

  • Big Data Developer

    4 weeks ago


    New York, New York, United States Eliassen Group Full time

    We are seeking a talented Big Data Developer to join our team at Eliassen Group. This is a 100% remote role that involves building next-generation data analytics platforms using advanced big data technologies.The ideal candidate will have 5+ years of experience in designing, architecting, developing, scaling, and improving data platforms and services.We are...

  • Big Data Engineer

    1 week ago


    New York, New York, United States Diverse Lynx Full time

    Job Overview">We are seeking a highly skilled Databricks and PySpark Developer to join our team at Diverse Lynx LLC. This is an exciting opportunity for a seasoned professional to leverage their expertise in cloud computing, big data processing, and machine learning to drive business growth and innovation.">Key Responsibilities:">">Design, develop, and...

  • Big Data Engineer

    2 weeks ago


    New York, New York, United States InterSources Full time

    Job Summary:We are seeking an experienced Big Data Engineer to lead our ETL efforts. The successful candidate will have a strong background in building and optimizing data processing systems using cloud-based technologies such as Azure. This is an excellent opportunity for someone who wants to work on complex projects and collaborate with a talented team of...


  • New York, New York, United States S&P Global Full time

    About the Role:S&P Global seeks a skilled Data Engineer to join its Data Lake team. The ideal candidate will have expertise in designing and building data pipelines with an emphasis on scale, performance, and reliability. This role is responsible for providing technical expertise in the areas of design and implementation of Data Lake solutions powered by...


  • New York, New York, United States CyberTec Full time

    We are CyberTec, a leading technology company seeking a highly skilled Big Data Architect with DevOps expertise to join our team. Job Title: Big Data Architect with DevOpsEstimated Salary: $120,000 - $180,000 per annumAs a key member of our engineering team, you will be responsible for designing and implementing scalable data architectures using industry...

  • Big Data Engineer

    5 days ago


    New York, New York, United States CapB InfoteK Full time

    At CapB InfoteK, we are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have 8+ years of experience working with ETL, Hadoop, and related technologies. With a strong background in data warehousing and ETL tools, the successful candidate will be responsible for designing, developing, and implementing efficient ETL...


  • New York, New York, United States TechnoGen Full time

    Job OverviewWe are seeking a Senior Big Data Developer to join our team at TechnoGen in Brooklyn, NY. This is a 9-month full-time contractor position with a competitive salary of $200,000 per year.The ideal candidate will have experience building applications and managing infrastructure using major cloud providers. A strong understanding of distributed...


  • New York, New York, United States Blood Bank of Delmarva Full time

    Job DescriptionNew York Blood Center Enterprises (NYBCe) is a leading blood bank that provides blood products and services to hospitals and healthcare organizations. We are seeking a skilled Data Engineering Lead to join our team.The successful candidate will lead the architectural design and optimization of robust, scalable data pipelines using SQL, Python,...


  • New York, New York, United States HexaQuEST Global, Inc. Full time

    HexaQuEST Global, Inc. is seeking a highly skilled Senior Big Data Engineer to join our team.We are offering an estimated salary of $120,000 per year for this role in the United States.About UsAs a leading provider of innovative solutions, HexaQuEST Global, Inc. prides itself on its commitment to excellence and customer satisfaction.Job DescriptionThis is a...


  • New York, New York, United States Texture Full time

    Unlock your potential and join Texture's dynamic team, shaping the future of energy data management.About the RoleThis is a unique opportunity to work with a pioneering company that's revolutionizing the energy sector. As a senior data engineer, you'll play a key role in designing, constructing, and optimizing our data pipelines to support various data...

  • Big Data Engineer

    6 days ago


    New York, New York, United States ShiftCode Analytics Full time

    Job Description:Hiring for a Big Data Engineer in New York. The ideal candidate will have 5+ years of experience in Hadoop and related technologies. Responsibilities include designing and implementing scalable data architectures, developing and maintaining large-scale data processing systems, and collaborating with cross-functional teams to ensure seamless...

  • Big Data Expert

    2 weeks ago


    New York, New York, United States Fidelity Information Services Full time

    About the CompanyFidelity Information Services (FIS) is a leading FinTech product multinational company that empowers its employees to work on challenging and relevant issues in financial services and technology.Our team is passionate, entrepreneurial, and fun-loving, with a focus on delivering high-quality technical solutions that align with business and IT...