Data Engineer

Job Title:Data Engineer (10+ Years)

Location:
Texas
(Onsite)

Employment:
Full Time/ W2
(NO C2C)

Job Summary:
We are seeking a
Senior Data Engineer with 10+ years of hands-on experience
in designing, building, and optimizing
scalable, high-performance data platforms and pipelines
. The ideal candidate will have deep expertise in
data ingestion, ETL/ELT, data warehousing, big-data processing, cloud-native architectures, real-time streaming, and analytics enablement
, and will partner closely with
Analytics, Data Science, Product, and Engineering teams
across the
full data lifecycle
.

Key Responsibilities:

• Design, develop, and maintain
  end-to-end data pipelines
  for
  structured, semi-structured, and unstructured data
  using
  batch and real-time processing frameworks
  .
• Build and optimize
  ETL/ELT pipelines
  using
  Python, SQL, PySpark, Spark SQL
  , and orchestration tools such as
  Apache Airflow, Azure Data Factory, AWS Glue, or Prefect
  .
• Develop
  scalable big-data solutions
  using
  Apache Spark, Hadoop ecosystem
  , and distributed processing techniques for high-volume data workloads.
• Design and manage
  cloud-based data platforms
  on
  AWS, Azure, or GCP
  , including
  Data Lakes, Lakehouse architectures, and Cloud Data Warehouses
  .
• Implement and optimize
  data warehousing solutions
  using
  Snowflake, Amazon Redshift, Google BigQuery, Azure Synapse
  , and dimensional modeling techniques.
• Develop
  data transformation layers
  to support
  analytics, reporting, and machine learning workloads
  , ensuring high data quality and performance.
• Build and maintain
  real-time and streaming data pipelines
  using
  Apache Kafka, Kafka Streams, Spark Streaming, or Azure Event Hubs
  .
• Design and implement
  data models
  using
  star schema, snowflake schema, and normalized models
  , aligned with business and analytics requirements.
• Ensure
  data quality, validation, reconciliation, and lineage
  using
  data quality frameworks, metadata management, and governance tools
  .
• Implement
  data security, access controls, encryption, and compliance
  standards across platforms, supporting
  PII, GDPR, SOC, and regulatory requirements
  .
• Optimize
  SQL performance, partitioning strategies, indexing, and query tuning
  across relational and analytical databases.
• Containerize and deploy data workloads using
  Docker
  and orchestrate pipelines using
  Kubernetes
  where applicable.
• Build and maintain
  CI/CD pipelines for data engineering workflows
  using
  Jenkins, GitHub Actions, Azure DevOps
  , and infrastructure-as-code tools.
• Monitor and troubleshoot data pipelines using
  logging, alerting, and observability tools
  to ensure reliability and SLA adherence.
• Collaborate with
  Data Architects, Data Scientists, BI Developers, and Product teams
  to deliver analytics-ready datasets.
• Support
  UAT, production releases, incident management, and root cause analysis
  for data platforms.
• Lead
  architecture decisions
  , conduct
  code reviews
  , mentor junior engineers, and enforce
  data engineering best practices
  .
• Drive
  data modernization, cloud migration, and performance optimization initiatives
  across enterprise data ecosystems.

Required Skills:

• Programming & Querying:
  Python, SQL, PySpark, Spark SQL
• Big Data:
  Apache Spark, Hadoop, Kafka
• Data Integration:
  Airflow, Azure Data Factory, AWS Glue
• Data Warehousing:
  Snowflake, Redshift, BigQuery, Azure Synapse
• Databases:
  PostgreSQL, Oracle, SQL Server, MongoDB
• Cloud Platforms:
  AWS / Azure / GCP
• Streaming:
  Kafka, Spark Streaming, Event Hubs
• DevOps & Automation:
  Docker, Kubernetes, Jenkins, Git
• Data Modeling:
  Star Schema, Snowflake Schema, Dimensional Modeling
• Methodologies:
  Agile, SDLC, DataOps