Data Pipeline Design Specialist

18 hours ago


Jersey City, New Jersey, United States Tiger Analytics Full time

About Your Career

You will have the opportunity to work with a talented team of experts in data science, machine learning, and AI. You will be responsible for designing, building, and maintaining scalable data pipelines on AWS cloud infrastructure. The ideal candidate will have strong experience with AWS services, Databricks, and Apache Airflow.

Responsibilities:
  • Design and Deploy Data Pipelines: Design, develop, and deploy end-to-end data pipelines on AWS cloud infrastructure using services such as Amazon S3, AWS Glue, AWS Lambda, and Amazon Redshift.
  • Implement Data Processing Workflows: Implement data processing and transformation workflows using Databricks, Apache Spark, and SQL to support analytics and reporting requirements.
  • Build Orchestration Workflows: Build and maintain orchestration workflows using Apache Airflow to automate data pipeline execution, scheduling, and monitoring.
  • Collaborate with Stakeholders: Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver scalable data solutions.
  • Optimize Data Pipelines: Optimize data pipelines for performance, reliability, and cost-effectiveness, leveraging AWS best practices and cloud-native technologies.
Requirements:
  • Experience: 8+ years of experience building and deploying large-scale data processing pipelines in a production environment.
  • AWS Services: Strong proficiency in AWS services such as Amazon S3, AWS Glue, AWS Lambda, and Amazon Redshift.
  • Databricks and Apache Spark: Strong experience with Databricks and Apache Spark for data processing and analytics.
  • Apache Airflow: Hands-on experience with Apache Airflow for orchestrating and scheduling data pipelines.
  • Version Control Systems: Experience with version control systems (e.g., Git) and CI/CD pipelines.
Compensation:

The estimated salary for this role is $180,000 - $220,000 per year. This position offers an excellent opportunity for significant career development in a fast-growing and challenging entrepreneurial environment with a high degree of individual responsibility.

  • Jersey City, New Jersey, United States Emonics LLC Full time

    About Emonics LLC">We are a leading provider of data processing and cloud solutions, serving various industries with our cutting-edge technologies. Our team is dedicated to delivering high-quality data pipelines that meet business requirements.">Job Requirements">We are seeking an experienced Backend Developer to join our team. The successful candidate will...


  • Jersey City, New Jersey, United States Futran Tech Solutions Pvt. Ltd. Full time

    Job OpportunityWe are seeking an experienced Senior Specialist to join our team at Futran Tech Solutions Pvt. Ltd. As a key member of our data engineering team, you will be responsible for designing, implementing, and maintaining our data pipelines. ResponsibilitiesDevelop and maintain data pipelines using Python, Spark, Hive/HadoopCollaborate with business...


  • Jersey City, New Jersey, United States Verisk Analytics Full time

    About Verisk AnalyticsAt Verisk Analytics, we are a leading data analytics and technology partner to the global insurance industry. We empower communities and businesses to make better decisions on risk, faster.Job DescriptionWe are looking for an experienced Data Engineer to join our team. The ideal candidate will be responsible for expanding and optimizing...


  • Jersey City, New Jersey, United States Synechron Full time

    Job DescriptionWe are seeking a talented Data Processing Specialist to join our team. As a key member of our data engineering group, you will be responsible for designing and implementing scalable data processing applications using Python along with Hadoop and distributed database technologies. You will collaborate with data engineers, analysts, and other...


  • Jersey City, New Jersey, United States Futran Tech Solutions Pvt. Ltd. Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Futran Tech Solutions Pvt. Ltd. This is a full-time position that requires 10+ years of experience in data engineering and the ability to drive adoption of modern data tools.Key ResponsibilitiesDevelop, modify, and adopt tools and processes to support self-service data pipeline...


  • Jersey City, New Jersey, United States Synechron Full time

    About the RoleSynechron is a global consulting firm that combines creativity and innovative technology to deliver industry-leading digital solutions. Our progressive technologies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering.We are seeking an experienced professional to join our team as a Data...

  • Software Engineer

    3 weeks ago


    Jersey City, New Jersey, United States Futran Tech Solutions Pvt. Ltd. Full time

    Futran Tech Solutions Pvt. Ltd. is a leading provider of innovative technology solutions, and we are currently seeking an experienced Software Engineer - Data Specialist.We offer a competitive salary of $120,000 - $180,000 per annum based on experience and qualifications.About the RoleThis is a fantastic opportunity for a talented Python Developer to join...


  • Jersey City, New Jersey, United States Diverse Lynx Full time

    Job Title: Cloud-Based Data Solutions SpecialistWe are seeking a skilled Data Engineer to join our team at Diverse Lynx LLC.About the Role:We design, build, and maintain cloud-based data solutions using Azure Data Factory (ADF) for data integration and transformation.Our ideal candidate will develop and optimize Databricks notebooks and pipelines for data...


  • Jersey City, New Jersey, United States Sumitomo Mitsui Banking Corp Full time

    Job Description:This Senior Data Engineer position will focus on designing, developing, and deploying cloud data solutions for risk, regulatory, finance, and compliance projects.The ideal candidate will have extensive experience with Azure cloud services and strong knowledge of Python, Java, or similar high-level server-side languages.This role will require...

  • Senior Data Engineer

    3 weeks ago


    Jersey City, New Jersey, United States Synechron Full time

    Company OverviewSynechron is a global consulting firm that combines creativity and innovative technology to deliver industry-leading digital solutions. Our progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, servicing an array of noteworthy...


  • Jersey City, New Jersey, United States Inizio Partners Full time

    We are seeking an experienced Data Architect Lead to join our team at Inizio Partners. The ideal candidate will have a strong background in data engineering principles, architectures, pipeline design, and cloud technologies.You will be responsible for designing and implementing scalable data architectures on AWS, leading the team's workflow, and ensuring the...

  • Senior Specialist

    18 hours ago


    Jersey City, New Jersey, United States Futran Tech Solutions Pvt. Ltd. Full time

    About UsFutran Tech Solutions Pvt. Ltd. is a leading provider of innovative data engineering solutions. We are seeking an experienced Senior Specialist to join our team and lead the development of our data pipeline management platform. Job SummaryThe Senior Specialist will be responsible for designing, implementing, and maintaining our data pipelines,...

  • Cloud Data Engineer

    1 week ago


    Jersey City, New Jersey, United States Synergy Global Technologies Inc. Full time

    We are seeking a highly skilled Cloud Data Engineer to join our team at Synergy Global Technologies Inc. in Jersey City, New Jersey. This is a full-time position that requires 7-10 years of total IT experience and 2+ years of experience in Google Cloud Platform (GCP).The ideal candidate will have hands-on experience in designing and developing enterprise...


  • Jersey City, New Jersey, United States Infojini Full time

    Job Requirements:We are looking for a highly skilled Sr. Azure Data Engineer who can design, implement, and optimize data pipelines using Microsoft Azure services. The ideal candidate will have a strong background in data engineering, Azure Cloud services, and Python/Pyspark programming.About the Role:This is a challenging and rewarding role that requires...

  • Data Engineering Lead

    15 hours ago


    Jersey City, New Jersey, United States Futran Tech Solutions Pvt. Ltd. Full time

    Company OverviewFutran Tech Solutions Pvt. Ltd. is a cutting-edge technology company that offers innovative data engineering solutions. We are committed to delivering exceptional results and fostering a collaborative work environment. Job DescriptionThis Senior Specialist role involves designing and developing high-quality data pipelines that meet the needs...


  • Jersey City, New Jersey, United States RSC Solutions Full time

    Job Description: ETL Developer Expert">RSC Solutions is seeking an experienced ETL Developer Expert to join our team. The successful candidate will be responsible for designing, implementing and maintaining large-scale data pipelines using IBM DataStage.">About You:">We are looking for someone with 8+ years of experience in developing and supporting IBM...


  • Jersey City, New Jersey, United States Tiger Analytics Full time

    About the RoleWe seek an experienced Data Engineer to join our team at Tiger Analytics. As a key member, you will design, build, and maintain data pipelines, integration processes, and infrastructure using Dataiku. Collaboration with data scientists, analysts, and stakeholders ensures efficient data flow and supports informed decision-making across the...


  • Jersey City, New Jersey, United States Sumitomo Mitsui Financial Group, Inc. Full time

    About the RoleWe are seeking an experienced Senior Data Engineer to join our Cloud data solutions team in the Americas.This role will involve designing, developing, and deploying cloud data solutions for risk, regulatory, finance, and compliance projects.Your Key ResponsibilitiesDesign and develop cloud data solutions for risk, regulatory, finance, and...

  • Cloud Data Architect

    16 hours ago


    Jersey City, New Jersey, United States Tiger Analytics Full time

    Role OverviewTiger Analytics is a pioneering advanced analytics consulting firm that has gained the trust of multiple Fortune 500 companies. Our team of experts brings deep expertise in Data Science, Machine Learning, and AI, enabling clients to generate significant business value from data.We are seeking an exceptional AWS Engineer Lead who will be...


  • Jersey City, New Jersey, United States Compu-Vision Consulting Full time

    About Us:Compu-Vision Consulting is a leading consulting firm specializing in data-driven solutions.We are committed to delivering exceptional results through our team's expertise and passion for innovation.Job Overview:This Sr. Consultant position will play a crucial role in driving our data strategy and implementing data warehouse solutions on Databricks...