Senior Data Engineer

Join to apply for the Senior Data Engineer role at Athena About Athena At Athena, we empower possibility through transformative delegation. True leaders reflect on what they want and map the path to get there. We clear the way by pairing exceptional Executive Assistants with our driven clients and ensuring both receive ongoing support throughout the journey. The result is 10x more leverage, more time, and a greater impact on our clients' work and lives. We are on a mission to build the best delegation platform in the world. We believe that the Human+AI combination will provide a superior experience to delegate complex tasks by combining the strengths of highly trained assistants leveraging highly trained AI. Role Overview The Senior Data Engineer will be a key contributor to Athena's data infrastructure and machine learning initiatives. This role requires deep expertise in building and managing robust data pipelines, data lakes, and real-time streaming services that can handle the scale and complexity of Athena's growing data needs. The Senior Engineer will be responsible for maintaining and optimizing Kafka clusters, managing data schemas, and ensuring efficient data ingestion and processing to support analytics and machine learning applications. Responsibilities Data Infrastructure Development: Design, build, and maintain scalable and reliable data infrastructure to support the companys data needs. ML Infrastructure: Collaborate with the machine learning team to create and manage infrastructure that supports ML model training and deployment. Pipeline Management: Develop and manage end-to-end data pipelines, from data ingestion to processing and storage. Data Lake Architecture: Design and implement data lake solutions to store and organize with data scientists, product managers, and engineers to ensure data infrastructure aligns with company objectives and supports data-driven decision-making. Qualifications Experience: 7+ years of experience in data engineering or a related field, with a proven track record in managing large-scale data infrastructure. Technical Expertise: Strong knowledge of data infrastructure technologies, including Kafka, data lakes, data warehousing, ETL processes, and cloud services (AWS, GCP, Azure). Programming Skills: Proficient in programming languages such as Python, Java, or Scala and scripting languages for data processing. Data Processing: Experience with data processing frameworks like Apache Spark, Flink, or Beam. Database Management: Familiarity with database technologies, both SQL and NoSQL, and their application in large-scale environments. ML Infrastructure: Understanding of machine learning infrastructure requirements and best practices. System Design: Strong system design and architecture skills, with an ability to build for scalability, reliability, and performance. Problem-Solving: Excellent problem-solving abilities with a focus on delivering scalable and efficient data solutions. Communication: Clear communicator with the ability to explain complex technical concepts to non-technical stakeholders. Education: Bachelors or masters degree in Computer Science, Engineering, or a related technical field, or equivalent experience. Seniority level Mid-Senior level Employment type Full-time Job function Information Technology Industries Business Consulting and Services Referrals increase your chances of interviewing at Athena by 2x Get notified about new Senior Data Engineer jobs in Palo Alto, CA. San Bruno, CA $162,000.00-$230,000.00 3 weeks ago We're unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr