Senior Spark Developer

6 days ago

New York, NY, United States Futran Tech Solutions Pvt. Ltd. Full time
Client - CPPIB

Location - New York City (3 DAYS IN THE OFFICE ARE MANDATORY)

In-Person client interview is MANDATORY

Role: Senior Spark Developer (Python, AWS, SQL)

Project Description

We are seeking a highly skilled Spark Developer with strong experience in Python, AWS, and SQL to join our team. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data processing solutions, ensuring data quality, scalability, and performance. This role requires a solid background in distributed computing, cloud environments, and data engineering best practices.

Responsibilities

Design, develop, and maintain scalable data pipelines using Apache Spark (batch and/or streaming).

Build, optimize, and manage ETL/ELT workflows integrating multiple data sources.

Develop data solutions in Python for data transformations, automation, and orchestration.

Leverage AWS services (S3, EMR, Glue, Lambda, Redshift, Kinesis, etc.) to implement cloud-native data platforms.

Write efficient SQL queries for data extraction, transformation, and reporting.

Ensure data quality, lineage, and governance across pipelines.

Collaborate with data engineers, architects, and analysts to deliver end-to-end data solutions.

Troubleshoot performance bottlenecks and optimize Spark jobs for speed and cost-efficiency.

Skills Must have

8+ years of experience in data engineering or backend development.

Hands-on experience with Apache Spark (PySpark) in large-scale data environments.

Strong proficiency in Python programming.

Expertise in SQL (including advanced queries, performance tuning, and optimization).

Experience working with AWS services such as S3, Glue, EMR, Lambda, Redshift, or Kinesis.

Understanding of data warehousing concepts and ETL best practices.

Strong problem-solving skills and ability to work in an agile, collaborative environment.

Nice to have

Experience with Databricks or similar Spark-based platforms.

Knowledge of streaming frameworks (Kafka, Flink).

Familiarity with CI/CD pipelines, Docker, Kubernetes, Terraform.

Exposure to data modeling (star schema, snowflake, data vault).

Experience in financial services / capital markets .

  • Scala Spark Developer

    6 days ago

    New York, NY, United States HCL Global Systems Full time

    As a Spark Scala Developer, you will play a critical role in the design, development, deployment and optimization of data processing application. Key Responsibilities: • Develop and maintain data processing applications using Spark and Scala. • Collaborate with cross-functional teams to understand data requirements and design efficient solutions. •...

  • Scala Spark Developer

    6 days ago

    New York, NY, United States HCL Global Systems Full time

    As a Spark Scala Developer, you will play a critical role in the design, development, deployment and optimization of data processing application. Key Responsibilities: • Develop and maintain data processing applications using Spark and Scala. • Collaborate with cross-functional teams to understand data requirements and design efficient solutions. •...

  • Scala Spark Developer

    4 days ago

    New York, NY, United States HCL Global Systems Full time

    As a Spark Scala Developer, you will play a critical role in the design, development, deployment and optimization of data processing application. Key Responsibilities: • Develop and maintain data processing applications using Spark and Scala. • Collaborate with cross-functional teams to understand data requirements and design efficient solutions. •...

  • Scala/ Spark Developer

    1 week ago

    New York, NY, United States InterSources Full time

    Title: Scala/ Spark Developer Duration: Long term Location: Broadway, NY - 3 days onsite Mandatory Skills: Big Data, Scala, Spark, Core Java Experience: As a Spark Scala Developer, you will play a critical role in the design, development, deployment and optimization of data processing application. Key Responsibilities: Develop and maintain data processing...

  • Scala/ Spark Developer

    2 days ago

    New York, NY, United States InterSources Full time

    Title: Scala/ Spark Developer Duration: Long term Location: Broadway, NY - 3 days onsite Mandatory Skills: Big Data, Scala, Spark, Core Java Experience: As a Spark Scala Developer, you will play a critical role in the design, development, deployment and optimization of data processing application. Key Responsibilities: Develop and maintain data processing...

  • Scala/ Spark Developer

    4 days ago

    New York, NY, United States InterSources Full time

    Title: Scala/ Spark Developer Duration: Long term Location: Broadway, NY - 3 days onsite Mandatory Skills: Big Data, Scala, Spark, Core Java Experience: As a Spark Scala Developer, you will play a critical role in the design, development, deployment and optimization of data processing application. Key Responsibilities: Develop and maintain data processing...

  • Python Developer

    1 week ago

    New York, NY, United States Group Nine LLC Full time

    Role: Python Developer Location: NYC, NY In-Person Interview is required Candidate should have recent experience in Banking or Finance DomainOverview We are seeking a highly skilled Senior Python Developer with extensive experience in capital markets to join our dynamic team. The ideal candidate will be responsible for designing, developing, and...

  • Python Developer

    4 days ago

    New York, NY, United States Group Nine LLC Full time

    Role: Python Developer Location: NYC, NY In-Person Interview is required Candidate should have recent experience in Banking or Finance DomainOverview We are seeking a highly skilled Senior Python Developer with extensive experience in capital markets to join our dynamic team. The ideal candidate will be responsible for designing, developing, and...

  • Python Developer

    4 days ago

    New York, NY, United States Group Nine LLC Full time

    Role: Python Developer Location: NYC, NY In-Person Interview is required Candidate should have recent experience in Banking or Finance DomainOverview We are seeking a highly skilled Senior Python Developer with extensive experience in capital markets to join our dynamic team. The ideal candidate will be responsible for designing, developing, and...

  • Senior Big Data Engineer

    6 days ago

    New York, NY, United States Zortech Solutions Full time

    Role: Senior Big Data Engineer Location: New Jersey (hybrid) Duration: 6+ Months Job Description: Please submit someone who can go for in person interview Job Description: We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. As a Senior Data Engineer, you will play a crucial role in designing, implementing, and...