Senior Data Engineer

1 day ago

Secaucus, NJ, United States Omni Inclusive Full time
Mandatory - GCP BigQuery, Dataflow, Python, Cloud Composer, Data Governance & Validation

Job Description:

Key Responsibilities
- Design and implement end-to-end data pipelines using Cloud Dataflow (Python/Apache Beam) for batch and streaming data.
- Develop, optimize, and maintain BigQuery stored procedures (SPs), SQL scripts, and user-defined functions (UDFs) for complex transformations and business logic implementation.
- Build and manage data orchestration workflows using Cloud Composer (Airflow) with appropriate operators and dependencies.
- Establish secure and efficient connections to source systems for data ingestion and integration.
- Manage data ingestion workflows from on-premise and cloud sources into Google Cloud Storage (GCS) and BigQuery.
- Execute history data migration from legacy data warehouses (preferably Snowflake, Teradata, Netezza, Oracle, SQL Server) to BigQuery, ensuring accuracy and performance optimization.
- Collaborate with data analysts and architects to define scalable and reusable data models, views, and semantic layers in BigQuery.
- Troubleshoot data pipeline failures, perform root cause analysis, and implement preventive measures.
- Optimize cost and performance of GCP workloads using best practices for BigQuery and Dataflow.

Technical & Soft Skills Required
- Strong hands-on experience in Google Cloud Platform (GCP) with expertise in BigQuery, Dataflow, Cloud Composer, and GCS.
- Proficient in Python programming for data engineering (Dataflow pipelines, validation scripts, automation).
- Expertise in BigQuery SQL, Stored Procedures, Views, and User Defined Functions (UDFs).
- Experience with data migration from on-premise or cloud data warehouses (preferably Snowflake, Teradata, Netezza, Oracle, SQL Server) to BigQuery.
- Strong understanding of ETL/ELT frameworks, data modeling, and schema design (Star/Snowflake).
- Familiarity with data governance, metadata, and lineage frameworks on GCP.
- Knowledge of data validation and testing techniques, including reconciliation, rule-based checks, and automation.
- Hands-on experience in workflow orchestration using Cloud Composer (Airflow) with custom operators.
- Strong SQL tuning and BigQuery performance optimization skills.
- Good communication and collaboration skills to work with cross-functional teams in Agile environments.
- Proven ability to troubleshoot, optimize, and enhance complex data workflows at scale.

  • Data Engineer

    7 days ago

    Secaucus, NJ, United States Tekfortune Inc Full time

    Tekfortune is a fast-growing consulting firm specialized in permanent, contract & project-based staffing services for world's leading organizations in a broad range of industries. In this quickly changing economic landscape, virtual recruiting and remote work are critical for the future of work. To support the active project demands and skills gaps, our...

  • Data Engineer

    1 day ago

    Secaucus, NJ, United States Omni Inclusive Full time

    Job Summary: Strong Snowflake Data warehousing knowledge Basic-Medium level Python Development ETL-ELT Technologies (Expertise in any of the ELT Tool, preferably in Matillion) Basic understanding of AWS Cloud platform Basic understanding of BI ToolWe are looking for a skilled Data Engineer to join our team. The ideal candidate will have experience in...

  • Senior Database Engineer

    1 day ago

    Secaucus, NJ, United States Yusen Logistics (Americas), Inc. Full time

    Senior Database EngineerJob Locations US-NJ-Secaucus | US-FL-Jacksonville | US-TN-CordovaJob ID 2025-6026# of Openings 1Category Information TechnologyType Regular Full-TimeOverviewSalary Range: 122,528-$205,915 The Senior Database Engineer is responsible for all activities related to database applications and the corresponding support systems of the...

  • Senior Database Engineer

    1 day ago

    Secaucus, NJ, United States Yusen Logistics Full time

    Overview Salary Range: 122,528-$205,915 The Senior Database Engineer is responsible for all activities related to database applications and the corresponding support systems of the Company. The senior DBE is expected to implement and configure appropriate monitoring solutions, so potential database issues can be identified and corrected before they...

  • Senior Software Engineer: Data Center

    3 days ago

    Secaucus, NJ, United States AMD Full time

    A leading semiconductor manufacturer is looking for a Software Development Engineer in Secaucus, NJ. This mid-senior level role focuses on driving software solutions for innovative datacenter technology. Ideal candidates should have a strong foundation in software design principles and experience with programming languages like Python and C++. Collaborative...

  • Senior Software Engineer: Data Center

    3 days ago

    Secaucus, NJ, United States AMD Full time

    A leading semiconductor manufacturer is looking for a Software Development Engineer in Secaucus, NJ. This mid-senior level role focuses on driving software solutions for innovative datacenter technology. Ideal candidates should have a strong foundation in software design principles and experience with programming languages like Python and C++. Collaborative...

  • Senior Software Engineer: Data Center

    3 days ago

    Secaucus, NJ, United States AMD Full time

    A leading semiconductor manufacturer is looking for a Software Development Engineer in Secaucus, NJ. This mid-senior level role focuses on driving software solutions for innovative datacenter technology. Ideal candidates should have a strong foundation in software design principles and experience with programming languages like Python and C++. Collaborative...

  • Senior Software Engineer: Data Center

    3 days ago

    Secaucus, NJ, United States AMD Full time

    A leading semiconductor manufacturer is looking for a Software Development Engineer in Secaucus, NJ. This mid-senior level role focuses on driving software solutions for innovative datacenter technology. Ideal candidates should have a strong foundation in software design principles and experience with programming languages like Python and C++. Collaborative...

  • Senior Software Engineer: Data Center

    3 days ago

    Secaucus, NJ, United States AMD Full time

    A leading semiconductor manufacturer is looking for a Software Development Engineer in Secaucus, NJ. This mid-senior level role focuses on driving software solutions for innovative datacenter technology. Ideal candidates should have a strong foundation in software design principles and experience with programming languages like Python and C++. Collaborative...

  • Senior Software Engineer: Data Center

    3 days ago

    Secaucus, NJ, United States AMD Full time

    A leading semiconductor manufacturer is looking for a Software Development Engineer in Secaucus, NJ. This mid-senior level role focuses on driving software solutions for innovative datacenter technology. Ideal candidates should have a strong foundation in software design principles and experience with programming languages like Python and C++. Collaborative...