Current jobs related to Senior Data Engineer - New York - TENTH MOUNTAIN LLC

  • Senior Azure Data Engineer

    5 days ago

    New York, New York, United States Hudson Data Full time

    Job Title: Senior Azure Data EngineerLocation: NYC, NYDuration: Long Term W2 RoleJob Description:The ideal candidate should have hands-on experience with various Azure services, including Azure Data Factory, Databricks, Data Lake, and Power BI. They should be able to design, build, and maintain ETL pipelines, manage data lakes, and create insightful reports...

  • Senior Data Engineer

    1 week ago

    New York, New York, United States The Data Sherpas Full time

    Job SummaryWe are seeking a highly skilled Senior Data Engineer to lead our team in designing, developing, and deploying high-performance data analytics solutions on Google Cloud Platform (GCP). As a key member of our team, you will be responsible for architecting transformation and modernization of enterprise data solutions on GCP, integrating native GCP...

  • Senior Data Engineer

    4 weeks ago

    New York, New York, United States Net2Source Inc. Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Net2Source Inc. As a Senior Data Engineer, you will be responsible for designing and managing data ecosystems, particularly within the finance sector.Key Responsibilities:Building and maintaining data pipelines to ensure data quality and integrityDriving...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States Capital One Full time

    Job Title: Senior Data EngineerCapital One is seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing data engineering solutions that drive business value.Key Responsibilities:Collaborate with cross-functional teams to design and develop data engineering solutionsDevelop...

  • Senior Data Engineer

    1 month ago

    New York, New York, United States Harnham Full time

    Senior Data EngineerAbout the RoleWe are seeking a highly skilled Senior Data Engineer to join our team at Harnham. As a Senior Data Engineer, you will be responsible for designing and developing comprehensive technical designs for both streaming and batch processing systems.Key ResponsibilitiesDevelop technical designs for streaming and batch processing...

  • Senior Data Engineer

    3 weeks ago

    New York, New York, United States Capital One Full time

    Job Title: Senior Data EngineerCapital One is seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing data engineering solutions that drive business value.Key Responsibilities:Collaborate with cross-functional teams to design and develop data engineering solutions that...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States Capital One Full time

    Job Title: Senior Data EngineerCapital One is seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing data engineering solutions that drive business value.Key Responsibilities:Collaborate with cross-functional teams to design and develop data engineering solutionsDevelop...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States Capital One Full time

    Job Title: Senior Data EngineerCapital One is seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing data pipelines and architectures to support our business needs.About the Role:Collaborate with cross-functional teams to design and develop data pipelines and...

  • Senior Data Engineer

    1 month ago

    New York, New York, United States A Place For Mom Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at A Place for Mom. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining our data warehouse on Databricks within AWS, ensuring optimal performance and reliability.Key Responsibilities:Design and develop ETL processes...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States Silver Search, Inc Full time

    Senior Data EngineerSilver Search, Inc is seeking a highly skilled Senior Data Engineer to join our data and analytics team. As a key member of our team, you will be responsible for designing and building data processing frameworks, working closely with data science and visualization teams, and ensuring the integrity and scalability of our data...

  • Senior Engineer

    2 weeks ago

    New York, New York, United States Acquired Data Solutions Full time

    Job SummaryWe are seeking a highly skilled Senior Level Engineer to join our Transportation Security Laboratory (TSL) Team in Egg Harbor, NJ as Key Personnel to support Development Test & Evaluation (DT&E) of existing and emerging technologies for airport security equipment.Key ResponsibilitiesPerform in-depth engineering analysis of security related systems...

  • Senior Data Engineer

    4 weeks ago

    New York, New York, United States Coinbase Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Coinbase. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining robust data pipelines, trusted data sources, and analytics/data products that inject automation into data science processes.Key Responsibilities:Design...

  • Senior Data Engineer

    1 month ago

    New York, New York, United States Coinbase Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Coinbase. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining robust data pipelines, trusted data sources, and analytics/data products that inject automation into data science processes.Key Responsibilities:Design...

  • Senior Data Engineer

    1 month ago

    New York, New York, United States Tech Tammina Full time

    Senior PySpark/Python EngineerWe are seeking a highly skilled Senior PySpark/Python Engineer to join our team at Tech Tammina. The ideal candidate will have a strong background in data engineering, with expertise in PySpark, Python, and related technologies.Key Responsibilities:Design and implement data pipelines using PySpark and PythonWork with data lakes,...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States MidStream PA Full time

    Join Our Team as a Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to lead the development of our core data processing platform. As a key member of our engineering team, you will be responsible for designing and implementing scalable, reliable, and high-performance data systems.Your expertise in data modeling, warehousing, and...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States A Place For Mom Full time

    Job Title: Senior Data EngineerA Place for Mom is seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining our data warehouse on Databricks within AWS, ensuring optimal performance and reliability.Key Responsibilities:Design and develop ETL processes to...

  • Senior Data Engineer

    1 month ago

    New York, New York, United States TENTH MOUNTAIN LLC Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at TENTH MOUNTAIN LLC. As a Senior Data Engineer, you will play a crucial role in migrating our existing data platform to an on-premises Hadoop environment.Key Responsibilities:Data Pipeline Development: Design and develop scalable, efficient ETL pipelines...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States Capital One Full time

    Job Title: Senior Data EngineerCapital One is seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing data engineering solutions that drive business value.Key Responsibilities:Collaborate with cross-functional teams to design and develop data engineering solutionsDevelop...

  • Senior Data Engineer

    5 days ago

    New York, New York, United States TechnoGen Full time

    Job Title: Senior Data EngineerJob Summary:We are seeking a Senior Data Engineer to join our Data Infrastructure team at TechnoGen. As a Senior Data Engineer, you will be responsible for building and maintaining tools and infrastructure to write, test, and schedule batch data pipelines. Your work will enable other developers, data scientists, and analysts to...

  • Senior Engineer

    4 weeks ago

    New York, New York, United States Acquired Data Solutions Full time

    Job OverviewWe are seeking a highly skilled Senior Level Engineer to join our Transportation Security Laboratory (TSL) Team in Egg Harbor, NJ as Key Personnel to support Development Test & Evaluation (DT&E) of existing and emerging technologies for airport security equipment.Key ResponsibilitiesPerform in-depth engineering analysis of security related...

Senior Data Engineer

2 months ago

New York, United States TENTH MOUNTAIN LLC Full time
Job DescriptionJob Description

Join KGES as a Senior Data Engineer

At KGES, we're dedicated to empowering veterans by connecting them with top-tier employers who value their unique military background. As a Senior Data Engineer, you'll have the opportunity to leverage your skills in data engineering, machine learning, and cloud platforms while making a meaningful impact. This is your chance to unlock your potential and embark on a fulfilling civilian career. Submit your information today and get ready to take the next step towards a brighter future with KGES

Position: Senior Data Engineer

Location: Remote
Pay Rate: $120,000 annually

Role Overview:

As a Senior Data Engineer at KGES, you'll play a crucial role in migrating our existing data platform to an on-premises Hadoop environment. You'll be responsible for implementing standards, governance, and automation processes. This role requires strong coordination and communication skills as you'll work closely with key stakeholders, data scientists, and other team members.

Key Responsibilities:

  • Design and Develop Data Pipelines:
    • Architect and implement scalable, efficient ETL (Extract, Transform, Load) pipelines using PySpark.
    • Optimize data processing workflows to handle large-scale datasets.
  • Machine Learning Model Development:
    • Develop and train machine learning models using appropriate algorithms and frameworks.
    • Collaborate with data scientists to translate models into production-ready code.
  • MLOps Implementation:
    • Establish and maintain automated CI/CD pipelines for machine learning models.
    • Implement version control for data, models, and code using tools like DVC, MLflow, or similar.
    • Monitor and automate the retraining of models as new data becomes available.
  • Data Quality and Governance:
    • Implement data validation, quality checks, and data governance best practices.
    • Ensure data lineage and documentation for reproducibility and compliance.
  • Performance Tuning:
    • Optimize PySpark jobs for performance, including tuning Spark configurations, optimizing shuffles, and managing memory.
    • Profile and debug PySpark applications to identify and resolve performance bottlenecks.
  • Integration with Cloud Platforms:
    • Deploy and manage data pipelines and machine learning models on cloud platforms such as AWS.
    • Utilize cloud-native services for data storage, processing, and orchestration.
  • Collaboration and Communication:
    • Work closely with data scientists, software engineers, and DevOps teams to integrate machine learning models into the broader software infrastructure.
    • Collaborate with business stakeholders to understand requirements and ensure the successful deployment of machine learning solutions.
    • Effectively communicate technical concepts and project status to non-technical stakeholders.

Qualifications:

  • 8+ years of experience in Engineering.
  • Strong experience with PySpark and Hadoop (on-prem).
  • Experience with MLOps tools like DVC, MLflow, or similar.
  • Proficiency with Jupyter, DataRobot, or similar tools.
  • Experience with AWS SageMaker.

Why KGES?

At KGES, we believe in your potential and are committed to helping you transition smoothly into a rewarding civilian career. Join our team and become part of a company that values your skills, experience, and dedication.

Ready to Apply?

Submit your information today and take the next step towards a brighter future with KGES