AWS Cloud engineer with knowledge of Spark

2 weeks ago


Plano, United States Compunnel Inc. Full time

Role: AWS Cloud Engineer with Knowledge of Spark/Python

Location: Plano, TX / Wilmington, DE

Type: Contract / Full time


Key Responsibilities:

Cloud Architecture & Infrastructure:

Design, implement, and manage scalable and reliable cloud infrastructure on AWS.

Utilize AWS services such as EC2, S3, Lambda, EMR, RDS, and Redshift to build and maintain data processing pipelines.

Implement security best practices and ensure compliance with industry standards.

Big Data Processing:

Develop and optimize big data processing workflows using Apache Spark.

Write efficient Spark jobs in Python to process large datasets.

Implement ETL (Extract, Transform, Load) processes to move and transform data across different systems.

Automation & Scripting:

Automate deployment, scaling, and management of applications on AWS using Python and AWS CloudFormation or Terraform.

Create and manage CI/CD pipelines to ensure smooth deployment of code into production.

Performance Optimization:

Monitor and optimize the performance of cloud infrastructure and data processing jobs.

Identify and address bottlenecks in the system to improve processing times and resource utilization.

Collaboration & Documentation:

Work closely with data engineers, data scientists, and other stakeholders to gather requirements and deliver cloud-based solutions.

Document architecture, processes, and configurations for future reference and maintenance.


Required Skills:

  • AWS Cloud Services: Proficiency in AWS services such as EC2, S3, Lambda, EMR, RDS, Redshift, and CloudFormation.
  • Programming: Strong programming skills in Python, with experience in writing and optimizing code for data processing.
  • Big Data Technologies: Hands-on experience with Apache Spark for big data processing.
  • Scripting & Automation: Experience in automation using Python, Shell scripting, and infrastructure as code tools like Terraform or CloudFormation.
  • Security: Knowledge of AWS security best practices, including IAM roles, security groups, and encryption.
  • Problem-Solving: Strong analytical skills to troubleshoot and resolve technical issues in a cloud environment.

Preferred Qualifications:

  • Experience: 5+ years of experience in cloud engineering, big data processing, and Python development.
  • Certifications: AWS Certified Solutions Architect, AWS Certified Big Data - Specialty, or similar certifications.
  • DevOps Tools: Familiarity with DevOps tools like Jenkins, Docker, and Kubernetes.
  • Agile Methodology: Experience working in Agile/Scrum environments.

Education:

  • Bachelor’s degree in Computer Science, Engineering, or a related field.



  • Plano, United States Compunnel Inc. Full time

    Role: AWS Cloud Engineer with Knowledge of Spark/PythonLocation: Plano, TX / Wilmington, DE Type: Contract / Full time Key Responsibilities:Cloud Architecture & Infrastructure:Design, implement, and manage scalable and reliable cloud infrastructure on AWS.Utilize AWS services such as EC2, S3, Lambda, EMR, RDS, and Redshift to build and maintain data...


  • Plano, United States Saxon Global Full time

    Hello Kenneth I Hope you are doing well. I have an immediate Job opportunity with one of my clients. Following are the position details. Job Title :- AWS cloud engineer Job Type :- Contract Location :- Remote till Covid Job Summary :- The Cloud Security Engineer/Architect is an expert in the AWS cloud environment, and cloud computing and...

  • AWS Devops Engineer

    3 weeks ago


    Plano, United States Diverse Lynx Full time

    Position: WS Devops Engineer Location: Plano/Dallas, TX Duration: Contract Required Skill: • AWS certified DevOps engineer with 10+ years of IT Experience in managing Cloud & on prem environments, skilled in AWS, cluster design, Jenkins, Git & CI/CD. • Working experience on Job Scheduling tools such as Control M, Orchestrator[Job Monitoring] etc •...

  • AWS Cloud Engineer

    2 days ago


    Plano, Texas, United States BCForward Full time

    Job OpportunityWe are seeking a highly motivated AWS Cloud Engineer to join our team at BCForward.About the RoleDesign and implement scalable and secure cloud-based solutions using AWS services.Collaborate with cross-functional teams to develop and deploy cloud-based applications.Ensure compliance with industry standards and best practices for cloud security...


  • Plano, United States Diverse Lynx Full time

    Role name: Developer Role Description: "• AWS certified DevOps engineer with 10+ years of IT Experience in managing Cloud & on prem environments, skilled in AWS, cluster design, Jenkins, Git & CI/CD.• Working experience on Job Scheduling tools such as Control M, Orchestrator[Job Monitoring] etc• Should be good at data analysis using SQL• Working...


  • Plano, United States Ccube Full time

    Job DescriptionJob DescriptionJob Title: Python PySpark Engineer with AWSLocation : Plano, TX/ Wilmington, DE  (Hybrid 3 Days a Week)W2/Full Time Highly PreferredExperience – 6+ years MinimumDuties and responsibilities:● Collaborate with the team to build out features for the data platform and consolidate data assets● Build, maintain and...


  • Plano, United States TalentOla Full time

    Job Title: AWS Cloud Engineer with terraformLocation: Plano, TXType: FulltimeJob responsibilities: -Your experience in public cloud migrations of complex systems, anticipating problems, and finding ways to mitigate risk, will be key in leading numerous public cloud initiativesExecutes creative software solutions, design, development, and technical...


  • Plano, United States Diverse Lynx Full time

    Role: AWS Cloud Engineer with Terraform Type: Contract Work Location: Plano, TX (onsite) Contract Duration: Long Term Experience Level : 10+ Years Job Description Job responsibilities Your experience in public cloud migrations of complex systems, anticipating problems, and finding ways to mitigate risk, will be key in leading numerous public cloud...


  • Plano, United States Diverse Lynx Full time

    • AWS certified DevOps engineer with 10+ years of IT Experience in managing Cloud & on prem environments, skilled in AWS, cluster design, Jenkins, Git & CI/CD. • Working experience on Job Scheduling tools such as Control M, Orchestrator[Job Monitoring] etc • Should be good at data analysis using SQL • Working experience on Incident/Problem/Change...


  • Plano, United States AIT Global inc. Full time

    Job DescriptionJob DescriptionJob Title: AWS Cloud Engineer with Terraform Location: Plano, TXJob responsibilities:Your experience in public cloud migrations of complex systems, anticipating problems, and finding ways to mitigate risk, will be key in leading numerous public cloud initiativesExecutes creative software solutions, design, development, and...


  • Plano, Texas, United States AIT Global inc. Full time

    Job OverviewPosition: Cloud Solutions Engineer specializing in AWS and TerraformKey Responsibilities:Lead public cloud migration projects, utilizing your expertise in complex systems to foresee challenges and implement risk mitigation strategies.Develop innovative software solutions, focusing on design, development, and technical troubleshooting, while...

  • Cloud Engineer

    2 weeks ago


    Plano, United States Next Ventures Full time

    A large financial institution are currently looking for an AWS Cloud Engineer to join their team on a 1-2 year project, with the view to start ASAP. The hourly rate for this position will be $30-$50 per hour and will be onsite one day a week in Plano Texas. The only visas being being considered for this position are US Citizens, Green Card Holders, H4EAD and...

  • Cloud Engineer

    2 weeks ago


    Plano, United States Next Ventures Full time

    A large financial institution are currently looking for an AWS Cloud Engineer to join their team on a 1-2 year project, with the view to start ASAP. The hourly rate for this position will be $30-$50 per hour and will be onsite one day a week in Plano Texas. The only visas being being considered for this position are US Citizens, Green Card Holders, H4EAD and...

  • Data Engineer

    3 days ago


    Plano, Texas, United States AIT Global inc. Full time

    Job OverviewAIT Global inc. is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines using Apache Spark and Java.Key ResponsibilitiesData Pipeline Development: Design and develop efficient data pipelines using Apache Spark and Java to process...


  • plano, United States JPMorgan Chase & Co Full time

    Job DescriptionJOB DESCRIPTIONThis is your chance to change the path of your career and guide multiple teams to success at one of the world's leading financial institutions.As a Manager of Software Engineering at JPMorgan Chase within the Corporate Sector, you lead multiple teams and manage day-to-day implementation activities by identifying and escalating...


  • Plano, United States JPMorgan Chase & Co Full time

    Job DescriptionJOB DESCRIPTIONThis is your chance to change the path of your career and guide multiple teams to success at one of the world's leading financial institutions.As a Manager of Software Engineering at JPMorgan Chase within the Corporate Sector, you lead multiple teams and manage day-to-day implementation activities by identifying and...


  • Plano, United States JPMorgan Chase & Co Full time

    Job DescriptionJOB DESCRIPTIONThis is your chance to change the path of your career and guide multiple teams to success at one of the world's leading financial institutions.As a Manager of Software Engineering at JPMorgan Chase within the Corporate Sector, you lead multiple teams and manage day-to-day implementation activities by identifying and escalating...

  • AWS Cloud Engineer

    2 months ago


    Plano, United States TrekRecruit Full time

    Job DescriptionJob DescriptionMust haves:Cloud Infrastructure work will be there and will be participation in migration from on prem to enterprise GitAWSLambda, Farget, ECS, EKS< ServerlessFamiliarity with different AWS components and servicesSome understanding with AZURE Cloud and services since it will be used for GitGIT HUB


  • Plano, United States AIT Global inc. Full time

    Job DescriptionJob DescriptionJob Title: Java Spark with AWS and Snowflake Location: Plano TX / Wilmington DEJob Description:Design, develop, and maintain scalable data pipelines using Apache Spark and Java.Implement data processing workflows and ETL processes to ingest, transform, and store large volumes of data.Collaborate with data scientists, analysts,...

  • AWS Engineer

    2 days ago


    Plano, United States Zenith services Full time

    Job responsibilities Your experience in public cloud migrations of complex systems anticipating problems and finding ways to mitigate risk will be key in leading numerous public cloud initiatives Executes creative software solutions design development and technical troubleshooting with ability to think beyond routine or conventional approaches to build...