Big Data Engineer

3 weeks ago


Malvern, United States Vision It US Full time
Job DescriptionJob Description

Responsibilities:

  • Leverages data pipeline designs and supports the development of data pipelines to support model development.
  • Proficient with software tools that develop data pipelines in a distributed computing environment (PySpark, GlueETL).
  • Supports integration of model pipelines in a production environment.
  • Develops understanding of SDLC for model production.
  • Reviews pipeline designs, makes data model design changes as needed.
  • Documents and reviews design changes with data science teams.
  • Supports data discovery and automated ingestion for model development.
  • Performs detailed analysis of raw data sources for data quality, applies business context, and model development needs.
  • Engages with internal stakeholders to understand and probe business processes in order to develop hypotheses.
  • Brings structure to requests and translates requirements into an analytic approach.
  • Participates in and influences ongoing business planning and departmental prioritization activities.
  • Runs model monitoring scripts, follows process for alerts to management as needed.
  • Addresses issues found in data pipelines from model monitoring alerts.

Qualifications:

Pyspark, Python, Advanced SQL, AWS, Sagemaker pipelines, ETL, Data Pipelines, understanding of AI/ML, Model Development and Deployment Life Cycle


Required Skills : AWS Cloud Formation,Python,SQL
Additional Skills : AWS Engineer,Software Developer,Big Data

  • Malvern, United States Tephra Full time

    Description: Technology exp required:Relational database, SQL, transaction log, replication, applying deltasAmazon Web Services (AWS) - load data into S3 (ssh, sftp, AWS CLI)Deep expertise with Python scripting - preferably including Anaconda on AWS (processing CSV / pipe-delimited data files)Experience in building reliable multi-step distributed data...


  • Malvern, United States (THE VANGUARD GROUP INC. MALVERN,PA) Full time

    Manage and optimize automation of key ETL (Extract / Transform / Load) and quality assurance processes for large volumes of data; consult with department or function heads on implementation, transition, and operating plans. Requires Master’s in Computer Science, Computer Engineering, Computer Science and Engineering, or closely related IT field and two...

  • Data Scientist

    1 week ago


    Malvern, United States Goli Tech Full time

    Job DescriptionJob DescriptionResponsibilitiesDevelops queries and performs extensive programming to access, transform, and prepare data for modeling using Python.Performs deep dive diagnostic, predictive, and prescriptive analytics to support data-driven business decision making.Identifies and diagnoses data inconsistencies and errors, documents data...


  • Malvern, United States The Vanguard Group Full time

    Data Engineer, Specialist (The Vanguard Group, Inc. / Malvern, PA) -- Write ETL (Extract / Transform / Load) processes, design database systems, utilize data transformation, normalization or consolidation, and develop tools for real-time and offline analytic processing; apply best practices of data quality and lineage; troubleshoot software and processes for...

  • Data Analyst

    1 week ago


    Malvern, United States Vision It US Full time

    Job DescriptionJob DescriptionResponsibilities:Analyzes data, assesses trends and develops actionable insights and recommendations in support of business objectives. Trains and develops junior analysts. Work closely with business partners to develop data-driven insights and recommendations to improve the client experience. Develop and maintain dashboards and...


  • Malvern, United States Georgia IT Inc Full time

    Job Title: Data Engineer Location: Malvern, PA Duration: Contract Rate: DOE Duties and responsibilities: Provides advanced data solutions by using software to process, store, and serve data to others. Tests data quality and optimizes data availability. Ensures that data pipelines are scalable, repeatable, and secure. Utilizes a deep dive analytical...


  • Malvern, PA, United States (THE VANGUARD GROUP INC. MALVERN,PA) Full time

    Manage and optimize automation of key ETL (Extract / Transform / Load) and quality assurance processes for large volumes of data; consult with department or function heads on implementation, transition, and operating plans. Requires Master’s in Computer Science, Computer Engineering, Computer Science and Engineering, or closely related IT field and two...

  • Data Analyst

    5 days ago


    Malvern, United States RIT Solutions, Inc. Full time

    Data Analyst - hybrid in malvern, pa Need only someone for 4 years of exp Responsibilities: Analyzes data, assesses trends and develops actionable insights and recommendations in support of business objectives. Trains and develops junior analysts. Work closely with business partners to develop data-driven insights and recommendations to improve the client...

  • AWS BIgdata Engineer

    4 weeks ago


    Malvern, United States Mergen IT Full time

    Design develop and deploy modern data warehouse solutions in AWS cloud platform.• Gather and document business requirements and translate it into technical design documents.• Create technical documentation e.g. data flow diagrams end user guide and technical operations guides.• Create technical documentation e.g. data flow diagrams end user guide...

  • AWS BIgdata Engineer

    4 weeks ago


    Malvern, Arkansas, United States Mergen IT Full time

    Design develop and deploy modern data warehouse solutions in AWS cloud platform. Gather and document business requirements and translate it into technical design documents. Create technical documentation e.g. data flow diagrams end user guide and technical operations guides. Create technical documentation e.g. data flow diagrams end user guide and technical...


  • Malvern, United States New Life Staffing LLC Full time

    Job DescriptionJob DescriptionDescription:Salary: 90-110KTravel Required: Typically, 2-3 times a month doing a client visit no more than 3 hoursAutoCad experience is a mustDesign experienceField role client facingPE required 10+ years of experienceSenior Mechanical Engineer with 10 years+ of consulting engineering experience in designing HVAC, plumbing and...

  • Sensor Engineer

    5 days ago


    Malvern, United States ifm Full time

    An excellent career opportunity is currently available for a Sensor Engineer in the Development Engineering team at ifm prover USA. The candidate will be heavily involved with developing and building prototype designs and testing cutting-edge prototype sensors. This position will work in collaboration with our development team (Mechanical Engineering,...

  • Sensor Engineer

    1 week ago


    Malvern, Arkansas, United States ifm Full time

    An excellent career opportunity is currently available for a Sensor Engineer in the Development Engineering team at ifm prover USA. The candidate will be heavily involved with developing and building prototype designs and testing cutting-edge prototype sensors. This position will work in collaboration with our development team (Mechanical Engineering,...

  • Solutions Engineer

    5 days ago


    Malvern, United States Circonus Full time

    As a Circonus Solutions Engineer, you’ll drive customer satisfaction and revenue by advising our customers and prospects on the value of Circonus' technology. Your responsibilities will revolve around supporting the pre and post sales process. This includes development and presentation of product demos, leading technically-oriented conversations with...


  • Malvern, United States Vanguard Full time

    Performs the development and programming of machine learning integrated software algorithms to structure, analyze, and leverage data in a production environment. Leverages detailed understanding of machine learning models and their deployment architecture. Core Responsibilities Develops complex data pipelines and implements data engineering design principles...

  • Solutions Engineer

    3 weeks ago


    Malvern, United States CareerBuilder Full time

    As a Circonus Solutions Engineer, you'll drive customer satisfaction and revenue by advising our customers and prospects on the value of Circonus' technology. Your responsibilities will revolve around supporting the pre and post sales process. This includes development and presentation of product demos, leading technically-oriented conversations with...


  • Malvern, United States Diverse Lynx Full time

    Job Title: Fullstack Java Developer Location: Malvern, PA (Day1 Onsite) Duration: (Contract) Specializations that will make an impact: Expertise in frontend development technologies (JavaScript/ AngularJS, HTML, CSS) Expertise in REST-full services development (Java, Spring) Expertise in DB development (SQL, Postgres) Background using modern frontend...


  • Malvern, United States Diverse Lynx Full time

    Job Title: Fullstack Java Developer Location: Malvern, PA (Day1 Onsite) Duration: (Contract) Specializations that will make an impact: Expertise in frontend development technologies (JavaScript/ AngularJS, HTML, CSS) Expertise in REST-full services development (Java, Spring) Expertise in DB development (SQL, Postgres) Background using modern frontend...

  • ServiceNow Engineer

    3 days ago


    Malvern, United States Saxon Global Full time

    This is a 12 month contract with Vanguard. 3 days onsite in ofice in Malvern PA and 2 days onsite. Try for a EST candidate. All visa. Job Summary Enhancement Activities - technical configuration and workflow enhancement support and deploy additional application functionality Guidance on solutions and alternatives Assist with knowledge transfer to...


  • Malvern, United States Diverse Lynx Full time

    Position - Tealeaf and Full Stack Developer with AWS Location - Malvern, PA Job Type - C2CJob ResponsibilitiesIn this role you will: Be a key technical contributor on this team. Good knowledge in AWS configurations. Partner with immediate team, architects, IT counterparts, leadership, and business partners Partner effectively with vendor in implementing...