Python Pyspark Developer

3 weeks ago

Columbus, United States Diamondpick Full time
Duties and responsibilities
• Collaborate with the team to build out features for the data platform and consolidate data
assets
• Build, maintain and optimize data pipelines built using Spark
• Advise, consult, and coach other data professionals on standards and practices
• Work with the team to define company data assets
• Migrate CMS' data platform into Chase's environment
• Partner with business analysts and solutions architects to develop technical
architectures for strategic enterprise projects and initiatives
• Build libraries to standardize how we process data
• Loves to teach and learn, and knows that continuous learning is the cornerstone of every
successful engineer
• Has a solid understanding of AWS tools such as EMR or Glue, their pros and cons and
is able to intelligently convey such knowledge
• Implement automation on applicable processes

Mandatory Skills:
• 5+ years of experience in a data engineering position
• Proficiency is Python (or similar) and SQL
• Strong experience building data pipelines with Spark
• Strong verbal & written communication
• Strong analytical and problem solving skills
• Experience with relational datastores, NoSQL datastores and cloud object stores
• Experience building data processing infrastructure in AWS
• Bonus: Experience with infrastructure as code solutions, preferably Terraform
• Bonus: Cloud certification
• Bonus: Production experience with ACID compliant formats such as Hudi, Iceberg or
Delta Lake
• Bonus: Familiar with data observability solutions, data governance frameworks
Requirements
Bachelor's Degree in Computer Science/Programming or similar is preferred
Right to work
Must have legal right to work in the USA

Job responsibilities
Your experience in public cloud migrations of complex systems, anticipating problems, and finding ways to mitigate risk, will be key in leading numerous public cloud initiatives
Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems
Own end-to-end platform issues & help provide solutions to platform build and performance issues on the AWS Cloud & ensure the deliverables are bug free
Drive, support, and deliver on a strategy to build broad use of Amazon's utility computing web services (e.g., AWS EC2, AWS S3, AWS RDS, AWS CloudFront, AWS EFS, AWS DynamoDB, CloudWatch, EKS, ECS, MFTS, ALB, NLB)
Design resilient, secure, and high performing platforms in Public Cloud using JPMC best practices
Measure and optimize system performance, with an eye toward pushing our capabilities forward, getting ahead of customer needs, and innovating to continually improve
Provide primary operational support and engineering for the public cloud platform and debug and optimize systems and automate routine tasks
Collaborate with a cross-functional team to develop real-world solutions and positive user experiences at every interaction
Drive Game days, Resiliency tests and Chaos engineering exercises
Utilize programming languages like Java, Python, SQL, Node, Go, and Scala, Open Source RDBMS and NoSQL databases, Container Orchestration services including Docker and Kubernetes, and a variety of AWS tools and services
Required qualifications, capabilities, and skills
Formal training or certification on software engineering concepts and 10+ years applied experience
Hands-on practical experience delivering system design, application development, testing, and operational stability
Advanced in one or more programming language(s) - Java, Python, Go
A strong understanding of business technology drivers and their impact on architecture design, performance and monitoring, best practices
Design and building web environments on AWS, which includes working with services like EC2, ALB, NLB, Aurora Postgres, DynamoDB, EKS, ECS fargate, MFTS, SQS/SNS, S3 and Route53
Advanced in modern technologies such as: Java version 8+, Spring Boot, Restful Microservices, AWS or Cloud Foundry, Kubernetes.
Experience using DevOps tools in a cloud environment, such as Ansible, Artifactory, Docker, GitHub, Jenkins, Kubernetes, Maven, and Sonar Qube
Experience and knowledge of writing Infrastructure-as-Code (IaC) and Environment-as-Code (EaC), using tools like CloudFormation or Terraform
Experience with high volume, SLA critical applications, and building upon messaging and or event-driven architectures
Deep understanding of financial industry and their IT systems
Preferred qualifications, capabilities, and skills
Expert in one or more programming language(s) preferably Java
AWS Associate level certification in Developer, Solutions Architect or DevOps
Experience in building the AWS infrastructure like EKS, EC2, ECS, S3, DynamoDB, RDS, MFTS, Route53, ALB, NLB
Experience with high volume, mission critical applications, and building upon messaging and or event-driven architectures using Apache Kafka
Experience with logging, observability and monitoring tools including Splunk, Datadog, Dynatrace. CloudWatch or Grafana
Experience in automation and continuous delivery methods using Shell scripts, Gradle, Maven, Jenkins, Spinnaker
Experience with microservices architecture, high volume, SLA critical applications and their interdependencies with other applications, microservices and databases
Experience developing process, tooling, and methods to help improve operational maturity

  • Python Pyspark

    1 month ago

    columbus, United States Centraprise Full time

    Job Title : Python PysparkJob Location : Columbus, OH (Fully ONSITE from Day1)Job Type : Long term Contract-W2Job Description:Position summary A Data Engineer at CMS is a software engineer with proficiency in data. The data engineer will build and maintain the CMS data warehouse which is used for both reporting and analytics across the company. The...

  • Python Spark Developer

    4 months ago

    Columbus, United States Diverse Lynx Full time

    Job Role : Python Spark Developer Location : Columbus, OH Experience : 9+ Years Responsibilities: Develop and maintain data platforms using Python, Spark, and PySpark. Handle migration to PySpark on AWS. Design and implement data pipelines. Work with AWS and Big Data. Produce unit tests for Spark transformations and helper methods. Create...

  • Senior Data Engineer

    4 weeks ago

    Columbus, Ohio, United States Diverse Lynx Full time

    Job Summary: We are seeking a highly skilled Senior Data Engineer to join our team at Diverse Lynx LLC. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data platforms using Python, Spark, and PySpark. Key Responsibilities: Develop and maintain data platforms using Python, Spark,...

  • Senior Data Engineer

    4 weeks ago

    Columbus, Ohio, United States Diverse Lynx Full time

    Job Title :Senior Data Engineer - Python Spark AWSLocation :Columbus, OH - non local candidates acceptedJob Responsibilities:Develop and maintain scalable data platforms using Python, Spark, and PySpark. Handle migration to PySpark on AWS. Design and implement efficient data pipelines. Work with AWS and Big Data. Produce unit tests for Spark transformations...

  • Senior Data Engineer

    4 weeks ago

    Columbus, Ohio, United States Worldpay Full time

    Unlocking the Potential of Data EngineeringAt Worldpay, we're on a mission to revolutionize the way we work with data. As a Senior Data Engineer - Python Airflow, you'll play a pivotal role in shaping our Cloud Data Platform team's vision and strategy.Key Responsibilities:Develop and implement data engineering initiatives using Python, AWS, Airflow, and...

  • Python Developer

    6 days ago

    Columbus, United States Radiantze Inc. Full time

    Title : Python Back End Developer ( Need 1 Lead and 3 Sr Developers ) Location : RemoteDuration : Long TermResponsibilities : Code software primarily in Python (Django), PHPDesign and build RESTful services and API integrations with 3rd party payment providersBuild payment integrations with 3rd party Payment vendorsBuild producers/consumers to propagate data...

  • Python Developer

    5 days ago

    Columbus, United States Radiantze Inc. Full time

    Title : Python Back End Developer ( Need 1 Lead and 3 Sr Developers ) Location : RemoteDuration : Long TermResponsibilities : Code software primarily in Python (Django), PHPDesign and build RESTful services and API integrations with 3rd party payment providersBuild payment integrations with 3rd party Payment vendorsBuild producers/consumers to propagate data...

  • Senior Python Developer

    1 month ago

    Columbus, Ohio, United States Insight Global Full time

    Job OpportunityA company in the Nashville area is seeking a skilled Python developer to join their team. The ideal candidate will have extensive experience with Python, mathematical skills, and the ability to convert legacy code to Python.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic...

  • Data Engineer

    4 weeks ago

    Columbus, Ohio, United States Aloden, Inc. Full time

    Job Title: Data Engineer - Cloud Computing ExpertAt Aloden, Inc., we are seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines using PySpark and Java.Responsibilities:* Design and develop data pipelines using PySpark and Java* Migrate data from legacy...

  • Python Consultant: Expert Developer for Omni Inclusive

    4 weeks ago

    Columbus, Georgia, United States Omni Inclusive Full time

    Job Title: Python ConsultantJob Description: At Omni Inclusive, we are seeking a skilled Python Consultant to join our team. As a Python Consultant, you will be responsible for developing back-end components to improve responsiveness and overall performance of our applications. Your expertise in Python will enable you to integrate user-facing elements into...

  • Senior Full Stack Python Developer

    1 month ago

    Columbus, Ohio, United States Saxon Global Full time

    Job Requirements:Education:• Bachelor\u0027s Degree in a related field or equivalent work experienceExperience:• Minimum 3 years of software development experience in delivering software products in an agile software development methodologyTechnical Skills:• Python required• Database performance, SQL, and schema management required• Clinical data...

  • Senior Software Developer with Proficiency in Python and JavaScript

    5 days ago

    Columbus, Ohio, United States Vestis Full time

    About the Role: We are seeking an experienced Software Developer to join our team. As a key member of our development team, you will be responsible for designing, developing, and maintaining complex software applications.Key Responsibilities:Design and develop software applications using Python and JavaScript.Collaborate with cross-functional teams to...

  • Python Software Engineer

    4 weeks ago

    Columbus, Ohio, United States Omni Inclusive Full time

    Job Title: Python Software EngineerAbout the Role:We are seeking a highly skilled Python Software Engineer to join our team at Omni Inclusive. As a Python Software Engineer, you will be responsible for designing, developing, and maintaining scalable and efficient software applications using Python.Key Responsibilities:Design and develop software applications...

  • Radiantze Inc. | Python Developer | columbus, oh

    5 days ago

    columbus, United States Radiantze Inc. Full time

    Title : Python Back End Developer ( Need 1 Lead and 3 Sr Developers ) Location : RemoteDuration : Long TermResponsibilities : Code software primarily in Python (Django), PHPDesign and build RESTful services and API integrations with 3rd party payment providersBuild payment integrations with 3rd party Payment vendorsBuild producers/consumers to propagate data...

  • Senior Software Engineer

    2 months ago

    Columbus, United States Jobot Full time

    Job DescriptionJob DescriptionGroundbreaking SaaS Data Analytics platform for Healthcare Providers / Meaningful equity!This Jobot Job is hosted by: Craig RosecransAre you a fit? Easy Apply now by clicking the "Apply Now" buttonand sending us your resume.Salary: $80,000 - $155,000 per yearA bit about us:We are seeking a dynamic and innovative Senior Software...

  • Sr. Full Stack Python Engineer

    2 months ago

    Columbus, United States Saxon Global Full time

    Job Description: Bachelors Degree in related field or equivalent work experience Minimum 3 years of software development experience in delivering software products in an agile software development methodology Python required Database performance, SQL, and schema management is required Clinical data experience is preferred ETL/Airflow experience preferred...

  • DataStage developer

    6 days ago

    Columbus, United States SysMind Tech Full time

    Title: DataStage developer Location : Cincinnati, OH, Columbus, OH Role Description: Datastage-Python Senior Developer Competencies: IBM InfoSphere DataStage Experience (Years): 8-10 Essential Skills: Datastage-Python Senior Developer with 8+ years of overall IT experience? At least 3 years of experience in ETL development within a Data Warehouse.? An...

  • Pyramid Consulting, Inc | Junior/ Mide-level Data Analyst | columbus, oh

    4 days ago

    columbus, United States Pyramid Consulting, Inc Full time

    Immediate need for a talented Junior/ Mide-level Data Analyst. This is a 06+months contract opportunity with long-term potential and is located in Columbus OH (Remote). Please review the job description below and contact me ASAP if you are interested. Job ID:24-51261 Pay Range: $35 - $45/hour. Employee benefits include, but are not limited to, health...

  • Pyramid Consulting, Inc | Junior/ Mide-level Data Analyst | columbus, oh

    6 days ago

    columbus, United States Pyramid Consulting, Inc Full time

    Immediate need for a talented Junior/ Mide-level Data Analyst. This is a 06+months contract opportunity with long-term potential and is located in Columbus OH (Remote). Please review the job description below and contact me ASAP if you are interested. Job ID:24-51261 Pay Range: $35 - $45/hour. Employee benefits include, but are not limited to, health...

  • Junior/ Mide-level Data Analyst

    6 days ago

    Columbus, United States Pyramid Consulting, Inc Full time

    Immediate need for a talented Junior/ Mide-level Data Analyst. This is a 06+months contract opportunity with long-term potential and is located in Columbus OH (Remote). Please review the job description below and contact me ASAP if you are interested. Job ID:24-51261 Pay Range: $35 - $45/hour. Employee benefits include, but are not limited to, health...