Data Engineer + AI

1 week ago

Boston, MA, United States DCM INFOTECH LIMITED Full time
Position - Data Engineer + AI
Location - Boston or, Remote
Experience - 5 years to 10 years
Note: Please submit only genuine candidate with LinkedIn.
Local will be preferred.
About the role
We're looking for a Senior Data Engineer to build and scale our lakehouse and AI data pipelines on Databricks. You'll design robust ETL/ELT, enable feature engineering for ML/LLM use cases, and drive best practices for reliability, performance, and cost.
What you'll do
  • Design, build, and maintain batch/streaming pipelines in Python + PySpark on Databricks (Delta Lake, Autoloader, Structured Streaming).
  • Implement data models (Bronze/Silver/Gold), optimize with partitioning, Z-ORDER, and indexing, and manage reliability (DLT/Jobs, monitoring, alerting).
  • Enable ML/AI: feature engineering, MLflow experiment tracking, model registries, and model/feature serving; support RAG pipelines (embeddings, vector stores).
  • Establish data quality checks (e.g., Great Expectations), lineage, and governance (Unity Catalog, RBAC).
  • Collaborate with Data Science/ML and Product to productionize models and AI workflows; champion CI/CD and IaC.
  • Troubleshoot performance and cost issues; mentor engineers and set coding standards.
Must-have qualifications
  • 6-10+ years in data engineering with a track record of production pipelines.
  • Expert in Python and PySpark (UDFs, Window functions, Spark SQL, Catalyst basics).
  • Deep hands-on Databricks: Delta Lake, Jobs/Workflows, Structured Streaming, SQL Warehouses; practical tuning and cost optimization.
  • Strong SQL and data modeling (dimensional, medallion, CDC).
  • ML/AI enablement experience: MLflow, feature stores, model deployment/monitoring; familiarity with LLM workflows (embeddings, vectorization, prompt/response logging).
  • Cloud proficiency on AWS/Azure/GCP (object storage, IAM, networking).
  • CI/CD (GitHub/GitLab/Azure DevOps), testing (pytest), and observability (logs/metrics).
Nice to have
  • Databricks Delta Live Tables, Unity Catalog automation, Model Serving.
  • Orchestration (Airflow/Databricks Workflows), messaging (Kafka/Kinesis/Event Hubs).
  • Data quality & lineage tools (Great Expectations, OpenLineage).
  • Vector DBs (FAISS, pgvector, Pinecone), RAG frameworks (LangChain/LlamaIndex).
  • IaC (Terraform), security/compliance (PII handling, data masking).
  • Experience interfacing with BI tools (Power BI, Tableau, Databricks SQL).

  • Lead Software Engineer, Data Engineering

    1 week ago

    Boston, MA, United States Relyance AI Full time

    Lead Software Engineer, Data Engineering At Relyance AI, as a Lead Software Engineer, you'll take charge of enhancing our API services, constructing reliable data pipelines, and maintaining our robust microservices architecture. This role offers the exciting opportunity to mentor junior engineers and create impactful data dashboards. You'll have complete...

  • Lead Software Engineer, Data Engineering

    6 days ago

    Boston, MA, United States Relyance AI Full time

    Lead Software Engineer, Data Engineering At Relyance AI, as a Lead Software Engineer, you'll take charge of enhancing our API services, constructing reliable data pipelines, and maintaining our robust microservices architecture. This role offers the exciting opportunity to mentor junior engineers and create impactful data dashboards. You'll have complete...

  • Lead Data Engineer + AI

    1 week ago

    Boston, MA, United States DCM INFOTECH LIMITED Full time

    Position - Lead Data Engineer Location - Boston or, Remote Experience - 10 years to 15years Note: Please submit only genuine candidate with LinkedIn.Need minimum 3 years of experience as Lead. About the role We're looking for a Senior Data Engineer to build and scale our lakehouse and AI data pipelines on Databricks. You'll design robust ETL/ELT, enable...

  • Senior Software Engineer

    6 days ago

    Boston, MA, United States Saviance Full time

    Senior Software Engineer - AI & Data Engineering Location: Boston, MA (Onsite every Thursday) About BigRio BigRio is a Boston-based technology consulting firm specializing in AI/ML, data engineering, custom software development, and digital transformation, with a strong focus on the healthcare and life sciences domains. We partner with leading...

  • Senior Software Engineer

    4 days ago

    Boston, MA, United States Saviance Full time

    Senior Software Engineer - AI & Data Engineering Location: Boston, MA (Onsite every Thursday) About BigRio BigRio is a Boston-based technology consulting firm specializing in AI/ML, data engineering, custom software development, and digital transformation, with a strong focus on the healthcare and life sciences domains. We partner with leading...

  • Senior Software Engineer

    7 days ago

    Boston, MA, United States Saviance Full time

    Senior Software Engineer - AI & Data Engineering Location: Boston, MA (Onsite every Thursday) About BigRio BigRio is a Boston-based technology consulting firm specializing in AI/ML, data engineering, custom software development, and digital transformation, with a strong focus on the healthcare and life sciences domains. We partner with leading...

  • Senior Software Engineer

    2 days ago

    Boston, MA, United States Saviance Full time

    Senior Software Engineer - AI & Data Engineering Location: Boston, MA (Onsite every Thursday) About BigRio BigRio is a Boston-based technology consulting firm specializing in AI/ML, data engineering, custom software development, and digital transformation, with a strong focus on the healthcare and life sciences domains. We partner with leading...

  • Agentic AI

    2 weeks ago

    Boston, MA, United States Kyndryl Full time

    Who We Are At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward - always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities. The Role As an...

  • Agentic AI

    4 days ago

    Boston, MA, United States Kyndryl Full time

    Who We Are At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward - always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities. The Role As an...

  • Agentic AI

    1 week ago

    Boston, MA, United States Kyndryl Full time

    Who We Are At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward - always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities. The Role As an...