Principal Software Engineer

3 weeks ago

San Francisco, California, United States project44 Full time
About the Role

As a Principal Software Engineer - Data Engineering at project44, you will have opportunities to work on the latest technologies to streamline Machine Learning & AI Operations, build scalable data infrastructure, and democratize data access.

Key Responsibilities:

  • Work with software architecture and design as part of your job, leveraging and instituting best practices from distributed systems, databases, data platforms, infrastructure, and platform software, manageability, and observability.
  • Provide guidance on new technologies and continuous improvement in best practices, researching, implementing, and developing software development tools.
  • Build systems in a multi-cloud environment, utilizing AWS and GCP, with experience in other cloud environments such as Azure.
  • Build complex metrics solutions with data visualization support for actionable business insights.
  • Leverage expertise in latest Gen AI tools & methodologies like RAG, Vector DB, embeddings to architect and build automated data access & interpretation solutions.
  • Design and develop ETL/ELT using Python/Java with Snowflake, Postgres, and other data stores, developing and automating projects through their entire lifecycle.
  • Knowledge in Data Warehouse/Data Mart design and implementation, developing projects through their entire lifecycle.
  • Build distributed, reusable, and efficient backend ETLs, implementing security and data protection.
  • Understand repeatable automated processes for building applications, testing them, documenting them, and deploying them at scale.
  • Work collaboratively with insights and data science teams to understand end-user requirements, providing technical solutions, and implementing new features and data pipelines.
  • Establish quality processes to deliver stable and reliable solutions.
  • Efficient in writing complex SQL, stored procedures in Snowflake, Postgres, BigQuery.
  • Prepare documentation (Data Mapping, Technical Specifications, Production Support, data dictionaries, test cases, etc.) for all projects.
  • Coach junior team members and help your team to continuously improve by contributing to tooling, documentation, and development practices.
Requirements

We are seeking a highly experienced and skilled Principal Software Engineer - Data Engineering to join our team. The ideal candidate will have:

  • 8+ years of experience in leading Data Engineering efforts.
  • 3+ years of experience in Snowflake, Oracle, and knowledge in NoSQL databases like MongoDB.
  • 3+ years of experience in Python/Java.
  • 3+ years of experience in ETL Developer roles with deep knowledge of data processing tools like Airflow, Argo workflow.
  • 4+ years of experience with data engineering and operations, including administering production-level, always-on, high-throughput, complex OLTP RDBMS.
  • Experience in delivering software solutions in areas of distributed systems.
  • Experience with working with Neural networks and Gen AI methodologies.
  • Strong experience in building data warehouse solutions and Data Modeling.
  • Strong ETL performance-tuning skills and the ability to analyze and optimize production volumes and batch schedules.
  • Experience with ETL, GCP, Unix/Linux, Helm Charts, as well as Git or other version control systems.
  • Experience with PII redaction for traditional ETL pipelines, as well as in Gen AI solutions.
  • Expertise in operational data stores and real-time data integration.
  • Expert-level skill in modeling, managing, scaling, and performance-tuning high-volume transactional databases.
  • Bachelor's Degree in Computer Science or equivalent experience.
Technical Skills

The ideal candidate will have:

  • Strong programming/scripting knowledge in building and maintaining ETL using Java, SQL, Python, Bash, Go.
  • In-depth hands-on knowledge of public clouds - GCP (preferred)/AWS, PostgreSQL (version 9.6+), ElasticSearch, MongoDB, MySQL/MariaDB, Snowflake, BigQuery.
  • Participate in an on-call rotation to mitigate any data pipeline failures.
  • Strong experience with Kafka or equivalent event/streaming-based systems.
  • Experience with Docker, Kubernetes.
  • Experience with RAG, Vector DB, embeddings, etc.
  • Develop and deploy CICD pipelines for Data Engineering.
  • Experience and knowledge of optimizing database performance and capacity utilization to provide high availability and redundancy.
  • Proficiency with high-volume OLTP databases and large data warehouse environments.
  • Ability to work in a fast-paced, rapidly changing environment.
  • Understanding of Agile and its implementation for Data Warehouse Development.
Professional Skills/Competency

The ideal candidate will have:

  • Focused on development/improvement of frameworks to support repeatable and scalable solutions.
  • Demonstrates excellent communication and interpersonal skills; able to communicate clearly and concisely.
  • Takes initiative to recommend/develop innovative approaches to getting things done.
  • Is a team player and encourages collaboration.
About project44

At project44, we're designing the future of how the world moves and is connected through trade and global supply chains.

We're focused on creating a company where all team members can bring their authentic selves to work every day.

We're building a company that every one of us at project44 is proud to work for, and our journey of becoming a more diverse, equitable, and inclusive organization, where all have a sense of belonging, is shaped through the actions of our leadership, global teams, and individual team members.

We are resolute in our belief that each team member has an equal responsibility to mold and uphold our culture.

project44 is an equal opportunity employer seeking to enrich our work environment by creating opportunities for individuals of all backgrounds and experiences to thrive.

If you share our values and our passion for helping the way the world moves, we'd love to review your application.

For any accommodation needed during the hiring process, please email [insert email].

Even if you don't meet 100% of the above job description, you should still seriously consider applying.

Studies show that you can still be considered for a role if you meet just 50% of the role's requirements.

  • Principal Software Engineer

    3 weeks ago

    San Francisco, California, United States Salesforce Inc Full time

    At Salesforce, we're seeking a highly skilled Principal Member of Technical Staff to join our team in San Francisco, CA.The ideal candidate will have a Master's degree in Computer Science, Engineering, or a related quantitative discipline, and two years of experience in software development, data engineering, or a related field.Key responsibilities...

  • Principal Software Engineer

    4 weeks ago

    San Francisco, California, United States Nextdoor Full time

    About the RoleNextdoor is seeking a highly skilled Principal Software Engineer to join our Data Platform team. As a key member of our team, you will be responsible for designing, building, and maintaining a highly scalable data platform that will power the development of Nextdoor for the foreseeable future.The ideal candidate will have a deep understanding...

  • Principal Software Engineer

    4 weeks ago

    San Francisco, California, United States Bright Machines Full time

    About the RoleBright Machines is seeking a highly skilled Principal Software Engineer to join our team in San Francisco. As a key member of our software engineering team, you will be responsible for designing, developing, and deploying software components that help realize our vision of Software Defined Manufacturing.You will work closely with our team of...

  • Principal Software Engineer

    4 weeks ago

    San Francisco, California, United States Unreal Gigs Full time

    Job SummaryUnreal Gigs is seeking a seasoned Principal Software Engineer to lead the development of innovative software solutions. As a technical expert, you will provide guidance on software architecture, design, and development, and mentor junior engineers to help them grow their skills.Key ResponsibilitiesProvide technical leadership and guidance on...

  • Senior Principal Software Engineer

    4 weeks ago

    San Francisco, California, United States TENDO Full time

    About the RoleWe are seeking a highly skilled Senior Principal Software Engineer to lead our team of engineers in the development of next-generation healthcare software. This individual will be responsible for architecting, building, and maintaining large-scale consumer-facing applications using cloud-native technologies.The ideal candidate will have a...

  • Software Principal Engineer

    3 weeks ago

    San Diego, California, United States Northrop Grumman Full time

    Software Principal EngineerNorthrop Grumman is seeking a highly skilled Software Principal Engineer to join our team of talented engineers. As a Software Principal Engineer, you will be responsible for developing and maintaining complex software systems using Java and other technologies. You will work closely with cross-functional teams to design, develop,...

  • Principal Software Engineer

    4 weeks ago

    San Francisco, California, United States Fivetran Full time

    About the RoleFivetran is a leading provider of cloud-based data integration solutions. We're seeking a highly skilled Principal Software Engineer to lead the technical architecture, design, and development of our Data Lake product offering.The successful candidate will have a deep understanding of high-performance relational data management systems, as well...

  • Software Principal Engineer

    4 weeks ago

    San Diego, California, United States Northrop Grumman Full time

    Software Principal EngineerNorthrop Grumman is seeking a highly skilled Software Principal Engineer to join our team of experienced professionals. As a key member of our software development team, you will be responsible for designing, developing, and delivering complex software systems for our customers.Key Responsibilities:Develop Java applications for...

  • Software Principal Engineer

    4 weeks ago

    San Diego, California, United States Northrop Grumman Full time

    Software Principal EngineerNorthrop Grumman is seeking a highly skilled Software Principal Engineer to join our team of experienced professionals. As a key member of our engineering team, you will be responsible for developing and maintaining complex software systems for flight and mission critical applications. Your expertise in Java and real-time operating...

  • Principal Software Engineer

    3 weeks ago

    San Francisco, California, United States Tendo Systems Full time

    About TendoTendo Systems is a fast-growing, mission-driven company building a culture that enables teams and individuals to thrive. Our team-driven culture and rapid growth have earned us recognition as a leader in the industry. We live by our values and are always on the hunt for motivated people with diverse experiences and backgrounds to help us improve...

  • Principal Software Engineer

    4 weeks ago

    San Diego, California, United States Qualcomm Full time

    Job Title: Principal Software EngineerQualcomm is a company of inventors that has unlocked the potential of 5G, ushering in an era of rapid acceleration in connectivity and new possibilities that will transform industries, create jobs, and enrich lives. We are seeking a Principal Software Engineer to join our team and contribute to the development of...

  • Principal Software Engineer, Core Systems

    3 weeks ago

    San Francisco, California, United States Reddit Full time

    Job Title: Principal Software Engineer, Core SystemsAbout the Role:We are seeking a highly skilled Principal Software Engineer to join our Core Platform team at Reddit. As a key member of our engineering organization, you will be responsible for designing and evolving our Core Systems to support the continued scale and growth of Reddit.Responsibilities:Own...

  • Software Principal Engineer

    3 weeks ago

    San Diego, California, United States Northrop Grumman Full time

    Job Summary:We are seeking a highly skilled Principal Software Engineer to join our team at Northrop Grumman Aerospace Systems. As a Principal Software Engineer, you will be responsible for developing software working with real-time operating systems for use in flight and mission-critical systems.Key Responsibilities:Develop Java applications and work with a...

  • Principal Software Engineer

    4 weeks ago

    San Diego, California, United States Northrop Grumman Full time

    Requisition ID: RWe are seeking a highly skilled Principal Software Engineer to join our team in San Diego, CA or Oklahoma City, OK.This is a unique opportunity to work on cutting-edge software development projects, leveraging the latest technologies and innovations in the field.As a Principal Software Engineer, you will be responsible for designing,...

  • Principal Software Engineer

    4 weeks ago

    San Diego, California, United States Northrop Grumman Full time

    Job Title: Principal Software EngineerNorthrop Grumman is seeking a highly skilled Principal Software Engineer to join our Systems Infrastructure Engineering (SIE) team based in San Diego, California.Job Summary:We are looking for a talented software engineer with expertise in standing up and managing strategic UAV ground segment infrastructure hardware,...

  • Principal Software Engineer

    4 weeks ago

    San Diego, California, United States Northrop Grumman Full time

    Job Title: Principal Software EngineerNorthrop Grumman is seeking a highly skilled Principal Software Engineer to join our Systems Infrastructure Engineering (SIE) team based in San Diego, California.Job Summary:We are looking for a talented software engineer to lead a team of engineers and provide daily tasking that aligns with current priorities. The...

  • Principal Software Architect

    4 weeks ago

    San Francisco, California, United States Checkr Full time

    About CheckrCheckr is a leading provider of background check services, empowering businesses to make informed hiring decisions. We're seeking a highly skilled Principal Software Engineer to join our team and drive the design of our core systems.Job SummaryWe're looking for a seasoned software engineer with a strong background in architecture and leadership...

  • Software Principal Engineer

    3 weeks ago

    San Diego, California, United States Northrop Grumman Full time

    Software Principal Engineer OpportunityWe are seeking a highly skilled Software Principal Engineer to join our team of experienced professionals. This role will involve developing software solutions for real-time operating systems in a cloud-based environment.Key Responsibilities:Design and implement software solutions for embedded technology...

  • Principal Software Engineer

    4 weeks ago

    San Diego, California, United States Northrop Grumman Full time

    Job Title: Principal Software EngineerWe are seeking a highly skilled Principal Software Engineer to join our Systems Infrastructure Engineering (SIE) team at Northrop Grumman's Autonomous Design Center of Excellence in San Diego, California.Job Summary:The successful candidate will be responsible for standing up and managing a suite of strategic UAV ground...

  • Software Principal Engineer

    3 weeks ago

    San Diego, California, United States Northrop Grumman Full time

    Software Development OpportunityWe are seeking a highly skilled Principal Software Engineer to join our team at Northrop Grumman. This position will be located in San Diego, CA or Oklahoma City, OK.As a Principal Software Engineer, you will be responsible for developing Java applications in an embedded technology environment. You will work with real-time...