Senior Big Data Engineer

4 weeks ago

Iselin, United States CareerBuilder Full time

SoHo Dragon is looking to hire a Senior Big Data Engineer.

We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. As a Senior Data Engineer, you will play a crucial role in designing, implementing, and maintaining data pipelines and infrastructure for our big data projects. Your expertise in Java, Python, Spark cluster management, data science, big data, REST API development, and knowledge of Databricks and Delta Lake will be essential in driving the success of our data initiatives.

Responsibilities:

- Design, develop, and implement scalable data pipelines and ETL processes using Java, Python, and Spark.

- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and design efficient solutions.

- Manage and optimize Spark clusters to ensure high performance and reliability.

- Perform data exploration, data cleaning, and data transformation tasks to prepare data for analysis and modeling.

- Develop and maintain data models and schemas to support data integration and analysis.

- Implement data quality and validation checks to ensure accuracy and consistency of data.

- Utilize REST API development skills to create and integrate data services and endpoints for seamless data access and consumption.

- Monitor and troubleshoot data pipeline performance, identifying and resolving bottlenecks and issues.

- Stay updated with the latest technologies and trends in big data, data engineering, data science, and REST API development, and provide recommendations for process improvements.

- Mentor and guide junior team members, providing technical leadership and sharing best practices.

Qualifications:

- Master's degree in Computer Science, Data Science, or a related field.

- Minimum of 3 years of professional experience in data engineering, working with Java, Python, Spark, and big data technologies.

- Strong programming skills in Java and Python, with expertise in building scalable and maintainable code.

- Proven experience in Spark cluster management, optimization, and performance tuning.

- Solid understanding of data science concepts and experience working with data scientists and analysts.

- Proficiency in SQL and experience with relational databases (e.g., Snowflake, Delta Tables).

- Experience in designing and developing REST APIs using frameworks such as Flask or Spring.

- Familiarity with cloud-based data platforms (e.g.Azure)

- Experience with data warehousing concepts and tools (e.g., Snowflake, BigQuery) is a plus.

- Strong problem-solving and analytical skills, with the ability to tackle complex data engineering challenges.

- Excellent communication and collaboration skills, with the ability to work effectively in a team-oriented environment.
#J-18808-Ljbffr

  • Senior Data Engineer

    2 days ago

    Iselin, United States Webologix Ltd INC Full time

    Job Title: Sr. Data EngineerLocations: Iselin, NJ / NYC, NYType of hire: FulltimeExperience: 10+ yearsJob Description:Azure, ADF, Data bricks, Python, Snow SQL, and Snowflake.Primary Responsibilities:Strong understanding or Snowflake on Azure Architecture, design, implementation and operationalization of large-scale data and analytics solutions on Snowflake...

  • Senior Data Engineer

    1 day ago

    Iselin, United States Webologix Ltd INC Full time

    Job Title: Sr. Data EngineerLocations: Iselin, NJ / NYC, NYType of hire: FulltimeExperience: 10+ yearsJob Description:Azure, ADF, Data bricks, Python, Snow SQL, and Snowflake.Primary Responsibilities:Strong understanding or Snowflake on Azure Architecture, design, implementation and operationalization of large-scale data and analytics solutions on Snowflake...

  • Data Engineer

    5 days ago

    Iselin, United States WallStreet Investment Bank Full time

    DutiesWork Schedule: Hybrid (1-2 days in office)1. No more than 8 years of experience 2. No more than 2 pages of a resume - the role requires a concise communicator 3. Require vendors to give a reason for nominating a person/resumeWhat is this team – Data Team in COO in investment banking division. Full service of data management / support/ governernance /...

  • Data Engineer

    6 days ago

    Iselin, United States WallStreet Investment Bank Full time

    DutiesWork Schedule: Hybrid (1-2 days in office)1. No more than 8 years of experience 2. No more than 2 pages of a resume - the role requires a concise communicator 3. Require vendors to give a reason for nominating a person/resumeWhat is this team – Data Team in COO in investment banking division. Full service of data management / support/ governernance /...

  • Data Cloud Engineer

    7 days ago

    Iselin, United States Global Channel Management Full time

    Data Cloud Engineer needs 3+ years of relevant experience in major projects on Data and Cloud Solutions., Cloud Infrastructure, automated operation, ServiceNow integration. Data Cloud Engineer requires: Broomfield, CO, Iselin, NJ, Charlotte, NJ AWS, Database ;any Database - Oracle/SQL/Mongo DB/hadoop Experience with Automation development tools Terraform is...

  • Senior Environmental Engineer

    4 days ago

    Iselin, United States Apex Companies, LLC Full time

    AECOM New York, NY 10176 posted 05/10/2024 More... front runner Senior Environmental Engineer/Practice Leader GZA GeoEnvironmental New York, NY 10001 posted 05/11/2024 More... front runner Senior Environmental Engineer TRC Companies, Inc. New Providence, NJ 07974 posted 05/13/2024 More... front runner Senior Environmental Engineer/Scientist NewFields Red...

  • Senior SailPoint Engineer

    24 hours ago

    Iselin, United States Euclid Innovations Full time

    Role: SailPoint Consultant with Workday integration experience.Iselin, NJ - HybridContract: 12+ Months Job Description:The IAM Engineer is an engineering role responsible for performing system implementation and integration activities. The engineer role is critical to delivery of services and managing business expectations day-to-day. The role also includes...

  • Solutions Engineer Advisor Senior

    1 day ago

    Iselin, United States Elevance Health Full time

    Elevance Health Solutions Engineer Advisor Senior ISELIN , New Jersey Apply Now Location: This position will work a hybrid model (remote and office). The ideal candidate will live within 50 miles of one of our Elevance Health PulsePoint locations. Preferred Location: Grand Prairie, Texas The Solutions Engineer Advisor Senior is responsible for product...

  • AI/ML Engineering

    2 days ago

    Iselin, United States Triunity Software Full time

    Job DescriptionJob DescriptionExperience: 10 + yrs.Job Type: Full timeLocation : Iselin, NJResponsibilities:Design, develop, and deploy machine learning models and algorithms to solve business problems.Collaborate with cross-functional teams to gather requirements, understand business objectives, and deliver AI/ML solutions.Explore and experiment with new...

  • Senior Fircosoft Engineer

    2 weeks ago

    Iselin, United States Euclid Innovations Full time

    Anti-Money Laundering (AML) and Financial Crime platform in Compliance Systems team includes the AML/BSA systems, Transaction Monitoring system, OFAC/Sanctions systems and other tools Principal Duties and Responsibilities: Assist in the data analysis and documentation processes, development, testing, implementation, validation, and maintenance of...

  • Senior Fircosoft Engineer

    2 weeks ago

    Iselin, United States Euclid Innovations Full time

    Anti-Money Laundering (AML) and Financial Crime platform in Compliance Systems team includes the AML/BSA systems, Transaction Monitoring system, OFAC/Sanctions systems and other toolsPrincipal Duties and Responsibilities: Assist in the data analysis and documentation processes, development, testing, implementation, validation, and maintenance of BSA/AML/OFAC...

  • Senior Fircosoft Engineer

    2 weeks ago

    Iselin, United States Euclid Innovations Full time

    Anti-Money Laundering (AML) and Financial Crime platform in Compliance Systems team includes the AML/BSA systems, Transaction Monitoring system, OFAC/Sanctions systems and other toolsPrincipal Duties and Responsibilities: Assist in the data analysis and documentation processes, development, testing, implementation, validation, and maintenance of BSA/AML/OFAC...

  • Product Data Analyst

    3 weeks ago

    Iselin, United States Soro Technologies Full time

    Responsibilities Assist in developing and implementing data governance processes to maintain the accuracy and integrity of product taxonomy attributes and data. Support the collection and validation of product data from various suppliers and sources. Perform data profiling and analysis to assess product data quality. Contribute to recommendations for...

  • Product Data Analyst

    3 weeks ago

    Iselin, New Jersey, United States Soro Technologies Full time

    Responsibilities Assist in developing and implementing data governance processes to maintain the accuracy and integrity of product taxonomy attributes and data. Support the collection and validation of product data from various suppliers and sources. Perform data profiling and analysis to assess product data quality. Contribute to recommendations for digital...

  • Bigdata Lead

    24 hours ago

    Iselin, United States Tanisha Systems, Inc. Full time

    Position : Big Data Lead / Data Engineer Location : Philadelphia, PA (Remote) EST Prefer Duration : Contract JOB DESCRIPTION : Primary Skill : 1 Azure data bricks- Build Data Pipelines using ADB-Leveraging Python and Spark Build Data Transformation in ADB Convert Scala Pgms to Py Spark Pgms Understand the SCALA program in Data Bricks 2 ADF Secondary...

  • Senior SailPoint Engineer

    15 hours ago

    Iselin, United States Euclid Innovations Full time

    SailPoint Engineer with exp in Workday IntegrationJob Description:The SailPoint IAM Engineer is an engineering role responsible for performing system implementation and integration activities. The engineer role is critical to delivery of services and managing business expectations day-to-day. The role also includes planning and implementation of security...

  • Senior SailPoint Engineer

    3 weeks ago

    Iselin, United States Euclid Innovations Full time

    SailPoint Engineer with exp in Workday IntegrationJob Description:The SailPoint IAM Engineer is an engineering role responsible for performing system implementation and integration activities. The engineer role is critical to delivery of services and managing business expectations day-to-day. The role also includes planning and implementation of security...

  • Senior SailPoint Engineer

    4 weeks ago

    Iselin, United States Euclid Innovations Full time

    SailPoint Engineer with exp in Workday IntegrationJob Description:The SailPoint IAM Engineer is an engineering role responsible for performing system implementation and integration activities. The engineer role is critical to delivery of services and managing business expectations day-to-day. The role also includes planning and implementation of security...

  • Accounting Advisory

    10 hours ago

    Iselin, United States CFGI Full time

    Senior Managers at CFGI will work closely with ‘C-level’ management of large public, mid-cap, and start-up companies. CFGI Senior Managers lead project teams and interact with clients and their Big 4 auditors on all aspects of engagements, including report writing, developing and managing client relationships, and business development. Take ownership of...

  • Senior Drinking Water Engineer

    3 weeks ago

    Iselin, United States Hazen and Sawyer Full time

    Job DescriptionJob DescriptionHazen and Sawyer is a stable, growing, employee-owned company where engineers and scientists are valued. We are committed to making the world a better place by providingtechnically superior solutions to our clients for all things water. Hazen and Sawyer is seeking a water engineer with 10 - 15+ years of experience to join our...