Current jobs related to Lead Data Engineer + AI - Boston, MA - DCM INFOTECH LIMITED

  • Lead Software Engineer, Data Engineering

    7 days ago

    Boston, MA, United States Relyance AI Full time

    Lead Software Engineer, Data Engineering At Relyance AI, as a Lead Software Engineer, you'll take charge of enhancing our API services, constructing reliable data pipelines, and maintaining our robust microservices architecture. This role offers the exciting opportunity to mentor junior engineers and create impactful data dashboards. You'll have complete...

  • Lead Software Engineer, Data Engineering

    5 days ago

    Boston, MA, United States Relyance AI Full time

    Lead Software Engineer, Data Engineering At Relyance AI, as a Lead Software Engineer, you'll take charge of enhancing our API services, constructing reliable data pipelines, and maintaining our robust microservices architecture. This role offers the exciting opportunity to mentor junior engineers and create impactful data dashboards. You'll have complete...

  • Data Engineer + AI

    7 days ago

    Boston, MA, United States DCM INFOTECH LIMITED Full time

    Position - Data Engineer + AI Location - Boston or, Remote Experience - 5 years to 10 years Note: Please submit only genuine candidate with LinkedIn. Local will be preferred. About the role We're looking for a Senior Data Engineer to build and scale our lakehouse and AI data pipelines on Databricks. You'll design robust ETL/ELT, enable feature engineering...

  • Lead Data Engineer

    5 days ago

    Boston, MA, United States C the Signs Full time

    We are seeking a Lead Data Engineer to architect, build, and scale our next-generation healthcare data platform. In this role, you will lead the effort to design robust pipelines, modernize data architecture, and ensure high-quality ingestion and transformation of clinical and operational data. You'll collaborate closely with product, analytics, clinical...

  • Lead Data Engineer

    6 days ago

    Boston, MA, United States C the Signs Full time

    We are seeking a Lead Data Engineer to architect, build, and scale our next-generation healthcare data platform. In this role, you will lead the effort to design robust pipelines, modernize data architecture, and ensure high-quality ingestion and transformation of clinical and operational data. You'll collaborate closely with product, analytics, clinical...

  • Lead Data Engineer

    7 days ago

    Boston, MA, United States C the Signs Full time

    We are seeking a Lead Data Engineer to architect, build, and scale our next-generation healthcare data platform. In this role, you will lead the effort to design robust pipelines, modernize data architecture, and ensure high-quality ingestion and transformation of clinical and operational data. You'll collaborate closely with product, analytics, clinical...

  • Senior Software Engineer

    5 days ago

    Boston, MA, United States Saviance Full time

    Senior Software Engineer - AI & Data Engineering Location: Boston, MA (Onsite every Thursday) About BigRio BigRio is a Boston-based technology consulting firm specializing in AI/ML, data engineering, custom software development, and digital transformation, with a strong focus on the healthcare and life sciences domains. We partner with leading...

  • Senior Software Engineer

    3 days ago

    Boston, MA, United States Saviance Full time

    Senior Software Engineer - AI & Data Engineering Location: Boston, MA (Onsite every Thursday) About BigRio BigRio is a Boston-based technology consulting firm specializing in AI/ML, data engineering, custom software development, and digital transformation, with a strong focus on the healthcare and life sciences domains. We partner with leading...

  • Senior Software Engineer

    6 days ago

    Boston, MA, United States Saviance Full time

    Senior Software Engineer - AI & Data Engineering Location: Boston, MA (Onsite every Thursday) About BigRio BigRio is a Boston-based technology consulting firm specializing in AI/ML, data engineering, custom software development, and digital transformation, with a strong focus on the healthcare and life sciences domains. We partner with leading...

  • Senior Software Engineer

    16 hours ago

    Boston, MA, United States Saviance Full time

    Senior Software Engineer - AI & Data Engineering Location: Boston, MA (Onsite every Thursday) About BigRio BigRio is a Boston-based technology consulting firm specializing in AI/ML, data engineering, custom software development, and digital transformation, with a strong focus on the healthcare and life sciences domains. We partner with leading...

Lead Data Engineer + AI

2 weeks ago

Boston, MA, United States DCM INFOTECH LIMITED Full time
Position - Lead Data Engineer
Location - Boston or, Remote
Experience - 10 years to 15years
Note: Please submit only genuine candidate with LinkedIn.

Need minimum 3 years of experience as Lead.
About the role
We're looking for a Senior Data Engineer to build and scale our lakehouse and AI data pipelines on Databricks. You'll design robust ETL/ELT, enable feature engineering for ML/LLM use cases, and drive best practices for reliability, performance, and cost.
What you'll do
  • Design, build, and maintain batch/streaming pipelines in Python + PySpark on Databricks (Delta Lake, Autoloader, Structured Streaming).
  • Implement data models (Bronze/Silver/Gold), optimize with partitioning, Z-ORDER, and indexing, and manage reliability (DLT/Jobs, monitoring, alerting).
  • Enable ML/AI: feature engineering, MLflow experiment tracking, model registries, and model/feature serving; support RAG pipelines (embeddings, vector stores).
  • Establish data quality checks (e.g., Great Expectations), lineage, and governance (Unity Catalog, RBAC).
  • Collaborate with Data Science/ML and Product to productionize models and AI workflows; champion CI/CD and IaC.
  • Troubleshoot performance and cost issues; mentor engineers and set coding standards.
Must-have qualifications
  • 10+ years in data engineering with a track record of production pipelines.
  • Expert in Python and PySpark (UDFs, Window functions, Spark SQL, Catalyst basics).
  • Deep hands-on Databricks: Delta Lake, Jobs/Workflows, Structured Streaming, SQL Warehouses; practical tuning and cost optimization.
  • Strong SQL and data modeling (dimensional, medallion, CDC).
  • ML/AI enablement experience: MLflow, feature stores, model deployment/monitoring; familiarity with LLM workflows (embeddings, vectorization, prompt/response logging).
  • Cloud proficiency on AWS/Azure/GCP (object storage, IAM, networking).
  • CI/CD (GitHub/GitLab/Azure DevOps), testing (pytest), and observability (logs/metrics).
Nice to have
  • Databricks Delta Live Tables, Unity Catalog automation, Model Serving.
  • Orchestration (Airflow/Databricks Workflows), messaging (Kafka/Kinesis/Event Hubs).
  • Data quality & lineage tools (Great Expectations, OpenLineage).
  • Vector DBs (FAISS, pgvector, Pinecone), RAG frameworks (LangChain/LlamaIndex).
  • IaC (Terraform), security/compliance (PII handling, data masking).
  • Experience interfacing with BI tools (Power BI, Tableau, Databricks SQL).