Data Architect

3 days ago

Santa Clara, CA, United States Futran Tech Solutions Pvt. Ltd. Full time
Skills

Years of experience

Rating out of 5

big data and cloud data warehousing 12+years

Pyspark, Python, SQL 8+ years

Datalake and Metadata 3+ years

Spark, AWS, Databricks 5+years

databases using JDBC, ODBC, SFTP 5+ years

CICD, Unit testing, Integration testing 5+ years

Databricks 5+ years

Airflow

Experience in handling Planning, forecasting, logistics,

fulfillment related Ops data from SAP, Anaplan, Agile PLM, etc

Job Title: Data Architect

Location: Santa Clara, CA (Onsite)

JC: 58529

Salary: $145k

Job Description:

Key Responsibilities
  • Implement data warehousing and data lake architectures using major cloud platforms like AWS, Azure, Databricks using their services and best practices
  • Enable data virtualization solutions like delta sharing across clouds and platforms with deep understanding of security and networking fundamentals
  • design scalable data pipelines ingesting and transforming structured and unstructured data from multiple sources(file storage, S3, HANA, other databases, SaaS application).
  • Build robust, scalable, and reusable data pipelines that are modular ensuring that data sources, ingestion components, validation functions, transformation functions, and destination are well understood for implementation.
  • Deal with Schema management and evolution. File formats for object storage (Parquet, Avro). Stages of the data pipeline (e.g., Databrick's Bronze/Silver/Gold zones).
  • Understand the data challenges and business requirements and create solutions
  • Create ERDs, complex data model designs understanding the intricacies and relationships of data appropriate for staging stores / data lakes, data warehouses, and data marts.
  • Create and present the systems, data and pipeline designs and documentation to the respective stakeholders and peers for review and feedback.
  • Write and execute automated tests (unit, integration, end-to-end) to ensure code quality and reliability.
  • Build robust CI/CD design and pipelines using Pulumi, Git, including effective branching strategies, merging, and resolving conflicts.
  • Create migration paths to unify plethora of data systems to fully managed Databricks
  • Support all the nonfunctional requirements of data, building dashboards for observability, debuggability, alerting and performance monitoring
  • Understand data governance, quality control, policies around data duplication, data definitions, company-wide processes around security and privacy, access control, lineage
  • Coordinate with IAM and other teams to implement Oauth, SSO, data access control and policy enforcement solutions within data lakes and cloud environments enabling secure user access and cross application integrations.
  • Identify and solve complex technical problems effectively communicating and collaborating with stakeholders and explaining technical concepts.
  • Lead discussions with stakeholders and IT to identify and implement the right data strategy given data sources, data locations, and use cases.
  • Build/develop code, frameworks, and data enabling solutions that enable the Ops teams to make critical business decisions.
  • strong technical skills, leadership abilities, and communication skills, enabling the team to design, build, and maintain robust data platform while helping other team members and collaborating effectively across teams.
  • Must-Have Skills/Experience Required:
  • Master's or bachelor's degree in computer science or information system, or equivalent experience.
  • 12+ Years in big data and cloud data warehousing technologies
  • 8+ years of relevant experience including programming knowledge (i.e Pyspark, Python, SQL).
  • 5+ years of relevant experience in big data technologies and cloud platforms (i.e Spark, AWS, Databricks).
  • 3+ years of relevant experience in data lake technologies (i.e Iceberg, Delta, Huidi) and Metadata catalogs (e.g., AWS Glue, Hive, Unity)
  • 5+ years of experience in development best practices like CICD, Unit testing, Integration testing
  • 5+ years of experience grabbing data from source systems like REST APIs, other databases using JDBC, ODBC, SFTP servers etc.
  • Experience handling Planning, forecasting, logistics, fulfillment related Ops data from SAP, Anaplan, Agile PLM, etc.
Ways to stand out from the crowd:
  • Analyze and debug data pipelines, ETL processes, and data warehouses built using technologies like Spark, Glue, Airflow, Redshift, Athena etc.
  • Self-starter, positive mindset with integrity and accountability, highly motivated, driven, and high reaching.
  • Solid ability to drive continuous improvement of systems and processes.
  • A consistent record to work in a fast-paced environment where good interpersonal skills are crucial.
  • Experience in developing required infrastructure for optimal extraction, transformation, and loading of data from various sources using AWS, Azure, SQL or other technologies.

  • Data Architect

    3 days ago

    Santa Clara, CA, United States Veracity Full time

    Job Title: Data Architect - Databricks Location: Santa Clara, CA (Fully Onsite) - Need Local Duration: 6+ Months Over 14+ years About the Role We're seeking a visionary Data Architect with deep expertise in Databricks to lead the design, implementation, and optimization of our enterprise data architecture. You'll be instrumental in shaping scalable data...

  • Data Architect

    2 days ago

    Santa Clara, CA, United States Veracity Full time

    Job Title: Data Architect - Databricks Location: Santa Clara, CA (Fully Onsite) - Need Local Duration: 6+ Months Over 14+ years About the Role We're seeking a visionary Data Architect with deep expertise in Databricks to lead the design, implementation, and optimization of our enterprise data architecture. You'll be instrumental in shaping scalable data...

  • Data Architect

    1 week ago

    Santa Clara, CA, United States Veracity Full time

    Job Title: Data Architect - Databricks Location: Santa Clara, CA (Fully Onsite) - Need Local Duration: 6+ Months Over 14+ years About the Role We're seeking a visionary Data Architect with deep expertise in Databricks to lead the design, implementation, and optimization of our enterprise data architecture. You'll be instrumental in shaping scalable data...

  • Data Architect

    1 week ago

    Santa Clara, CA, United States Veracity Full time

    Job Title: Data Architect - Databricks Location: Santa Clara, CA (Fully Onsite) - Need Local Duration: 6+ Months Over 14+ years About the Role We're seeking a visionary Data Architect with deep expertise in Databricks to lead the design, implementation, and optimization of our enterprise data architecture. You'll be instrumental in shaping scalable data...

  • Data Architect

    5 days ago

    Santa Clara, CA, United States Veracity Full time

    Job Title: Data Architect - Databricks Location: Santa Clara, CA (Fully Onsite) - Need Local Duration: 6+ Months Over 14+ years About the Role We're seeking a visionary Data Architect with deep expertise in Databricks to lead the design, implementation, and optimization of our enterprise data architecture. You'll be instrumental in shaping scalable data...

  • Data Architect

    2 weeks ago

    Santa Clara, CA, United States Merican Full time

    Job Title: Data Architect - Databricks Location: Santa Clara, CA (Fully Onsite) - Only Local Duration: 6+ Months MUST have skills: Data Bricks, AWS and SnowFlakes Over all 14+ years About the Role We're seeking a visionary Data Architect with deep expertise in Databricks to lead the design, implementation, and optimization of our enterprise data...

  • Data Architect

    3 days ago

    Santa Clara, CA, United States Merican Full time

    Job Title: Data Architect - Databricks Location: Santa Clara, CA (Fully Onsite) - Only Local Duration: 6+ Months MUST have skills: Data Bricks, AWS and SnowFlakes Over all 14+ years About the Role We're seeking a visionary Data Architect with deep expertise in Databricks to lead the design, implementation, and optimization of our enterprise data...

  • Data Architect

    5 days ago

    Santa Clara, CA, United States Merican Full time

    Job Title: Data Architect - Databricks Location: Santa Clara, CA (Fully Onsite) - Only Local Duration: 6+ Months MUST have skills: Data Bricks, AWS and SnowFlakes Over all 14+ years About the Role We're seeking a visionary Data Architect with deep expertise in Databricks to lead the design, implementation, and optimization of our enterprise data...

  • Data Architect

    15 hours ago

    Santa Clara, CA, United States Merican Full time

    Job Title: Data Architect - Databricks Location: Santa Clara, CA (Fully Onsite) - Only Local Duration: 6+ Months MUST have skills: Data Bricks, AWS and SnowFlakes Over all 14+ years About the Role We're seeking a visionary Data Architect with deep expertise in Databricks to lead the design, implementation, and optimization of our enterprise data...

  • Data Architect

    1 week ago

    Santa Clara, CA, United States ApTask Full time

    Rate Range: $80-$85/Hr Job Description: We're seeking a visionary Data Architect with deep expertise in Databricks to lead the design, implementation, and optimization of our enterprise data architecture. You'll be instrumental in shaping scalable data solutions that empower analytics, AI, and business intelligence across the organization. If you thrive in...