Sr. Data Engineer

Multiple Locations (US)

Operations Thrive

Remote

True is a global platform of companies that optimizes value creation. As one of the fastest-growing organizations in the talent advisory space, True places executive talent, assesses business leaders, creates diverse and inclusive networks, and uses innovative technology to advance executive talent priorities. Our clients include innovative growth brands backed by leading venture capital and private equity firms as well as established public and private organizations. With offices in North America, EMEA, & APAC, our team of industry and functional experts leverage their deep networks to connect companies with leading talent to transform businesses and industries. True was recognized as a 2022 best workplace by Inc. Magazine for the 3rd year and is North America's sixth largest firm in the industry according to the recently published Hunt Scanlon annual ranking. True was founded on the belief that doing good is the pathway to doing well. Our growth and success are a by-product of our valuestreating people right, listening to new ideas and keeping culture at the heart of our business.

The Role:

We are seeking a highly skilled and experienced Senior Data Engineer to join our growing AI/ML Platform team. In this pivotal role, you will be instrumental in designing, building, and optimizing the data pipelines and infrastructure that support our complex AI/ML projects. You will work closely with data scientists, machine learning engineers, software developers and Product managers to ensure reliable, scalable, and high-performance data solutions that drive our AI products forward.

This role reports directly to the CTO, CIO and Head of Data Privacy. This is a mid-level individual contributor role. The Sr. Data Engineer plays a crucial role in creating and managing the infrastructure and tools needed for efficient large-scale data processing and analysis, converting raw data into usable information.

Design and Develop Data Pipelines: Lead the design, development, and implementation of highly scalable, fault-tolerant, and optimized data pipelines (ETL/ELT) for batch and real-time data ingestion, processing, and transformation, specifically tailored for AI/ML workloads.

Build and Maintain Data Infrastructure: Architect, build, and maintain robust data platforms and data lakes/warehouses to support the storage, retrieval, and analysis of large datasets for AI/ML models.

Optimize Data for AI/ML: Collaborate with data scientists and machine learning engineers to understand their data needs, preprocess and feature engineer data for model training and inference, and optimize data structures for efficient model performance.

Ensure Data Quality and Governance: Implement strategies for data quality, data validation, data lineage, and data governance to ensure the accuracy, reliability, and security of data used in AI applications.

Performance Tuning and Scalability: Identify and resolve data performance bottlenecks, optimize data queries, and design solutions that can scale to handle ever-increasing data volumes and complex AI requirements.

Automate and Monitor: Develop and implement automation for data pipeline deployment, monitoring, and alerting to ensure data availability and pipeline health.

This role will create, develop and maintain scalable data and build new API integrations to continue supporting True's increase in data volume.

This role will work closely with internal stakeholders, including the data and product teams, as well as software developers, to implement software solutions and new methodologies for data correction, reconciliation, and quality assurance.

The Sr. Data Engineer will optimize data flow, ensure data quality, and consistently implement effective data management solutions across diverse data sources.

This role will transform raw data into impactful insights by analyzing data and developing innovative solutions that enable the firm to implement data driven strategies.

This role plays a crucial role in the data driven decision making process.

This role will be responsible for leading data engineering projects.

The Sr. Data Engineer proactively understands business needs and is able to select the best technologies, while optimizing data storage and retrieval.

Implement data replication, disaster recovery, and backup strategies to ensure data availability and business continuity.

Independently perform performance tuning and optimization of data systems.

Ensure data infrastructure is scalable, reliable, and aligned with business needs.

Experience of at least 5 years of work experience related to Data Engineering.

Strong knowledge and experience in programming languages.

Strong verbal and written communication skills.

Ability to work independently, across multiple projects.

Ability to articulate data strategies and technical concepts to non-technical stakeholders.

Strategic thinking and problem solving abilities.

Expert proficiency in at least one major programming language for data engineering (Python preferred).

Deep expertise in SQL and strong experience with relational and NoSQL databases.

Extensive experience with cloud-based data platforms (AWS, Azure, or GCP). Specific experience with services like S3/ADLS/GCS, EMR/Databricks/HDInsight, Redshift/Synapse/BigQuery, Glue/Data Factory/Dataflow is highly desirable.

Proven experience with big data technologies such as Spark, Hadoop, Kafka, Flink.

Solid understanding of data modeling, data warehousing, and ETL/ELT principles.

Proactive mindset with a passion for building innovative, secure, and user-friendly data solutions.

Experience with machine learning pipelines, data visualization tools, and advanced analytics.

Strong knowledge of data privacy laws and industry best practices.

Proven track record of managing numerous projects and competing demands with tight deadlines.

We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.