Data Engineering Lead-AWS Admin

1 month ago


Malvern, United States TEPHRA Full time
Description:

Technology exp required:

Relational database, SQL, transaction log, replication, applying deltas

Amazon Web Services (AWS) - load data into S3 (ssh, sftp, AWS CLI)

Deep expertise with Python scripting - preferably including Anaconda on AWS (processing CSV / pipe-delimited data files)

Experience in building reliable multi-step distributed data pipelines

Provide technical leadership for data engineering enabling us to advance architecture

The Architect would help guide the technical architecture for projects such as data replication from OnPrem to AWS, Data management, DBaaS, data modeling for the Data Lake on Cloud and data security.

Shaping an architecture to meet the needs of cloud data architecture to service identified use cases will be the primary technical objective of the role

Advanced knowledge of NoSQL and Graph databases, Hadoop, and ETL technologies will be needed to design overall data platform design

Experience designing systems to ingest, process, and get Big Data consumption ready for Data Science is required

Jupyter notebook on SageMaker for simple Python analytics

Duties and Responsibilities

1. Provides the architectural leadership in shaping strategic technology programs, which focus on both Business of IT (e.g., Unified Communication, SOI) and LOB-specific strategic technology programs.

2. Continuously pursues advanced level technical acumen:

Attends conferences and engages in associated activities (e.g., conducting presentations, leading workshops, etc.).

Consumes and contributes content from and to Open Source communities.

3. Provides architecture thought leadership and expertise including cost optimization

4. Defines reference and implementation architectures.

5. Produces technology roadmaps in support of IT's vision and strategy.

6. Develops proof-of-concept prototypes and initial implementation models.

7. Monitors implementation activity to ensure architecture and design principles are upheld.

8. Ensures implementation solutions support architecture objectives (availability, scalability, performance, security, etc.), as appropriate.

9. Rolls up sleeves and does deep dives on Financial Services data sets to understand intricacies and come up with optimal data models

10. Is adept with handling multiple data types from multiple data sources and strategize on the optimal way to analyze/store such data sets

Qualifications

Minimum 10 years relevant work experience.

Highly experienced with Big Data, Glue, EMR, and Data Quality tools/procedures for Cloud native data stores

Strong database and cloud expertise, along with experience with data replication technology required

Strong knowledge of micro-services and DDAAS required

Expert level AWS and Cloud native data experience is required

Minimum five years big data and advanced analytics experience required

- Minimum 2 years of cloud-native database design, migration, and implementation experience required

Experience with Enterprise Information Management (EIM) is required

Experience with CDC tools (Attunity preferred) to move data from OnPrem sources

  • Malvern, United States Tephra Full time

    Description: Technology exp required:Relational database, SQL, transaction log, replication, applying deltasAmazon Web Services (AWS) - load data into S3 (ssh, sftp, AWS CLI)Deep expertise with Python scripting - preferably including Anaconda on AWS (processing CSV / pipe-delimited data files)Experience in building reliable multi-step distributed data...


  • Malvern, United States Spear Staffing Full time

    Job DescriptionJob DescriptionRole: AWS Data Engineer/AWS DeveloperClient: VanguardContract Length: 12+ monthsIV Process: 1-2Years of experience: Snr 7+ Rate: 55hr maxLocation: Malvern, PA Hybrid 3 days a weekTop Skills: AWS Cloud Glue, Lambda, Step Functions & ETL & Python these are the important skills, forget the JD okay? Team Size: 4 membersProject:...

  • AWS Engineer

    3 weeks ago


    Malvern, United States Capgemini Full time

    Title: AWS Data EngineerLocation: Dallas, TX/ Malvern, PAJob Description:Required SkillsAWS/DB Technology: AWS RDS Aurora PostgreSQL, AWS Redshift, Sagemaker , AWS Glue ETL, Athena, Lake formation , EMR, ECS/EKS, S3, IAM, ELB, VPC, Route53, Cloudwatch, Lambda Data pipeline, Kinesis, Lambda, DynamoDB, EMR, cloud formation, TerraformProgramming/Software...

  • AWS Cloud Engineer

    2 weeks ago


    Malvern, United States Saxon Global Full time

    This is a 12 month contract with Vanguard. 3 days onsite in Malvern, PA and 2 days remote. All visa - no h1b. Must have strong EKS exp. Qualifications The engineer should have good knowledge and experience building and managing AWS EKS The engineer should be able to build cloud formation for AWS services The engineer should have good knowledge and...


  • Malvern, United States Vanguard Full time

    Does leveraging modern technical stacks to transform and reimagine a critical data schema for multiple subdivisions sound fascinating? Would you like to have a role as a senior technical SME and provide critical input as we tackle this exciting and ambiguous task? Join us as we evolve the possibilities for our business by building a critical firm admin data...


  • Malvern, United States THE VANGUARD GROUP, INC. MALVERN, PA Full time

    Write ETL (Extract / Transform / Load) processes, design database systems and, develop tools for real-time and offline analytic processing; troubleshoot software and processes for data consistency and integrity; integrate complex and large-scale data from a variety of sources for business partners to generate insight and make decisions; translate business...

  • Data Engineer

    4 weeks ago


    Malvern, United States Cogent Infotech Corp Full time

    Desired Skills and Experience: *- 6+ years of experience in the Data Engineering Space - Experience writing ETL processes, designing database systems and developing tools for real-time and offline analytic processing. - Experience with AWS Technologies like: Glue ETL, S3 RDS etc. - Strong experience with PySpark, preferably for building Data Pipelines -...

  • Python AWS Engineer

    3 hours ago


    Malvern, United States BCforward Full time

    Python AWS Engineer BCforward is currently seeking a highly motivated Python AWS Engineer for an opportunity in PA.Position Title: Python AWS EngineerLocation: PAExpected Duration: 8 Months Anticipated Start Date: - ASAP Please note this is the target date and is subject to change. BCforward will send official notice ahead of a confirmed start date. Job...

  • AWS Cloud Engineer

    2 weeks ago


    Malvern, United States Diverse Lynx Full time

    Job Title - AWS Cloud Engineer/ Cloud Engineer/ AWS Engineer Location - Malvern, PA (Day1 Onsite) Type -Contract Job Description: Looking for resource who has 9-11 years of hands on experience in Configuring GRD files for Software installation in AWS. It is essential to have excellent communication and coordination skills to successfully integrate various...

  • AWS Developer

    1 month ago


    Malvern, Arkansas, United States Randstad USA Full time

    job summary: Responsibilities Experience working on a Cloud modernization project. Knowledge and experience of cloud batch concepts and technologiesProven experience using AWS services (Attunity, Kinesis, Glue, S3, SQS, SNS, Lambda, Aurora RDS, DynamoDB, Cloud Formation) .Have experience with software engineering fundamentals including object-oriented...

  • AWS Python Developer

    4 weeks ago


    Malvern, Arkansas, United States Randstad USA Full time

    job summary: Responsibilities Reconciles data between AWS and Mainframes, makes performance improvements for data processing, identifies current data issues, vulnerabilities and implements effective solutions during transformation.Collaborates with the testers to develop and execute manual test cases and exploratory tests following the Test Case standards....

  • aws developer

    4 weeks ago


    Malvern, Pennsylvania, United States Randstad Full time

    aws developer. malvern , pennsylvania posted may 17, 2024 job detailssummary $60 - $65 per hour contract bachelor degree category computer and mathematical occupations reference job detailsjob summary:Responsibilities Experience working on a Cloud modernization project. Knowledge and experience of cloud batch concepts and technologies Proven experience...

  • Data Engineer

    1 month ago


    Malvern, United States JobRialto Full time

    Description: Requirement Details: Strong experience with AWS services on big data platforms. Strong Python experience in building Dash application in AWS Spark Shell scripting and Linux knowledge Nice to have: Investment Knowledge preferred Looking for candidates who have worked in Investment/Finance space Machine Learning Skills Qualifications: Educational...

  • Data Engineer

    1 month ago


    Malvern, United States JobRialto Full time

    Description: Requirement Details: Strong experience with AWS services on big data platforms. Strong Python experience in building Dash application in AWS Spark Shell scripting and Linux knowledge Nice to have: Investment Knowledge preferred Looking for candidates who have worked in Investment/Finance space Machine Learning Skills Qualifications: Educational...


  • Malvern, United States Diverse Lynx Full time

    Role: Unica Consultant with AWS Location: Malvern, PA Hire type: Contract /Full time Essential Skills: Technical knowledge at senior level needed: Unica Campaign, Interact support activities such as Migration to cloud and upgrades SQL/Oracle Unix Know-How AWS Knowledge is plus Role Description: Experience in Unica Migration from on-Prem to Cloud (AWS), unica...

  • big data engineer

    7 days ago


    Malvern, United States Randstad Full time

    big data engineer. malvern , pennsylvania posted 1 day ago job details summary $56.33 - $61.33 per hour contract bachelor degree category computer and mathematical occupations reference1057503 job details job summary: Integrate AWS Services such as Lambda, DynamoDB, EC2, RDS, S3, Athena, Data pipeline, API gateway, Glue, EMR etc. in...


  • Malvern, United States Diverse Lynx Full time

    Role: Unica Consultant with AWS Location: Malvern, PA Hire type: Contract /Full time Essential Skills: Technical knowledge at senior level needed: Unica Campaign, Interact support activities such as Migration to cloud and upgrades SQL/Oracle Unix Know-How AWS Knowledge is plus Role Description: Experience in Unica Migration from on-Prem to Cloud (AWS),...


  • Malvern, United States Diverse Lynx Full time

    Role: Unica Consultant with AWS Location: Malvern, PA Hire type: Contract /Full time Essential Skills: Technical knowledge at senior level needed: Unica Campaign, Interact support activities such as Migration to cloud and upgrades SQL/Oracle Unix Know-How AWS Knowledge is plus Role Description: Experience in Unica Migration from on-Prem to Cloud (AWS),...

  • AWS DevOps Engineer

    4 weeks ago


    Malvern, United States Syntricate Technologies Full time

    Responsibilities: Highly skilled AWS Sr. DevOps Engineer with strong knowledge of Okta Security product configuration and capabilities. Strong experience in CI/CD (i.e., plan, develop, release) Global Authentication Services (i.e., OKTA SSO, MFA, Federations, 2-Factor Authentication etc) Implement and deploy Business and Vendor applications using Okta OIN,...


  • Malvern, United States CyberTec Full time

    Client: Vanguard Role: Sr. AWS Python Developer (8+ years of experience) Payrate: $59/hr on C2C - NO FLEX Duration: 6+ Months Location: Malvern, PA (hybrid- 3 days a week, onsite on day 1)Responsibilities: The ModelOps & Automated Controls team makes deploying machine learning models easy. With tools like Aws SageMaker, Lambda, Code Pipeline and many other...