Big Data Engineer

7 days ago


New York, United States E-Solutions Full time

Role : Big Data Engineer

Location : NYC office Fully Onsite (Local only)


Inperson Interview is must


Technical Skills:

• Big Data Ecosystem: HDFS, Hive, Oozie, Sqoop, Impala, Presto and Spark.

• Cloud Ecosystem: AWS (EMR, S3, Athena), Snowflake

• Languages: Python, SQL and Shell Scripting.

• Databases: MySQL, Teradata, and SQL SERVER.

• Orchestration/Tools: Oozie, Airflow, CTRL-M, Maven, Jenkins, IntelliJ, GIT


JD

• Expertise in all components of Big Data Ecosystem- Spark, Hive, Sqoop, Oozie, Impala and Hue

• Extensive Experience with AWS services ensuring efficient data management and analysis.

• Successfully led the migration of legacy systems to Datawarehouse environments and Cloud, utilizing Informatica, Snowflake and AWS technologies, resulting in improved scalability, performance, and cost-effectiveness.

• Extensively worked with lambda, S3, cloudwatch,SQS,SNS,Step Functions and IAM roles in AWS services.

• Extensive knowledge on data serialization techniques like Avro, Sequence Files, Parquet, JSON and ORC.

• Experienced in handling databases: MySQL, SQL Server, Teradata.

• Expertise working across all phases of SDLC requirements gathering, system design, development, enhancement, maintenance, testing, deployment, production support, and documentation.

• Working experience in creating complex data ingestion pipelines, data transformations, data management and data governance in a centralized enterprise data hub.

• Knowledge of Kubernetes ecosystem.

• Did Data analysis and interpretation Using Databricks features to perform data transformations, run queries, and build visualizations collaborating Working with diverse teams to deliver insights in Analytics platform. – good to have.

• Strong problem-solving skills

• Strong team player with good communication, analytical, presentation and inter-personal skills.



  • New York, United States InterSources Full time

    Job Title: Big Data Engineer Type: Contract Location: New York, NY ( Day 1 onsite)Job Description: A Big Data Engineering lead handles the ETL jobs, Optimization, Automation of data processing for complex scenarios. Responsibilities: Data ingestion from Source DQ checks Handling ETL Automation Building features / variable Required Skills: Should be subject...


  • New York, United States CyberTec Full time

    Role 3: Big Data Architect with Devops experience and local to NY Start Date: ASAP Rate - $90/hr. Maximum Start Date: ASAP Local to NY its onsite Client: Microsoft We'd like to see candidates at the Tech 3 level Description: Job Summary and Responsibilities: Hands on keyboard engineer with experience in the DevOps space and, preferably, experience in the...

  • Data Engineer

    4 weeks ago


    New York, United States hackajob Full time

    Exciting Opportunity Alert! Join our Team as a Data Engineer!hackajob has partnered with an innovative company seeking a skilled Data Engineer to leverage data for impactful solutions. If you have expertise in big data tools like Spark and Kafka, and experience with cloud services like AWS, we want to hear from YOU!Role: Data EngineerLocation: Hybrid (New...

  • Big data Architect

    1 week ago


    New York, United States SysMind Tech Full time

    Hi, Hope you are doing well. Here is the position. Kindly revert me back if you are interested. Role: Big Data Architect Location: Remote Duration: 12+ Months Skills: 1. 8-10 Years of strong Data architect\modeling experience 2. Design and architect the tables\data marts for staging, facts and dimensions 3. Develop mapping specs for Fact and Dimension...


  • New York, New York, United States RiVi Consulting Group Full time

    Job SummaryAs a seasoned Spark Developer at RiVi Consulting Group, you will play a vital role in designing and implementing cutting-edge big data solutions. Your expertise in Spark and HDFS will enable us to deliver scalable and efficient data processing capabilities to drive business growth.Key Responsibilities• Collaborate with stakeholders to understand...

  • Data Engineer

    4 weeks ago


    new york city, United States hackajob Full time

    Exciting Opportunity Alert! Join our Team as a Data Engineer!hackajob has partnered with an innovative company seeking a skilled Data Engineer to leverage data for impactful solutions. If you have expertise in big data tools like Spark and Kafka, and experience with cloud services like AWS, we want to hear from YOU!Role: Data EngineerLocation: Hybrid (New...

  • Data Engineer

    4 weeks ago


    new york city, United States hackajob Full time

    Exciting Opportunity Alert! Join our Team as a Data Engineer!hackajob has partnered with an innovative company seeking a skilled Data Engineer to leverage data for impactful solutions. If you have expertise in big data tools like Spark and Kafka, and experience with cloud services like AWS, we want to hear from YOU!Role: Data EngineerLocation: Hybrid (New...

  • Data Engineer

    4 weeks ago


    New York, United States Brooksource Full time

    Data EngineerLocation: New York, NY (Hybrid)Contract-to-HireWe’re looking for an experienced Data Engineer to join a Fortune 50 Broadcast Media & Entertainment leader. This role involves designing, building, and optimizing data pipelines, integrating multiple datasets, and creating applications that drive strategic business decisions. You’ll collaborate...


  • New York, United States The Data Sherpas Full time

    J ob Summary : As a GCP Machine Learning Engineer, you will drive technology innovation by delivering world-class machine learning solutions. You will integrate GCP-native and third-party technologies to build scalable data infrastructures, focusing on modernizing data pipelines and machine learning platforms. You will collaborate with implementation teams...

  • Data Engineer

    1 month ago


    New York, New York, United States Glow Networks Full time

    Job DescriptionAt Glow Networks, we are seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems in a cloud environment.Key Responsibilities:Design and implement big data architectures using cloud-based technologies such as AWS, Spark, and...


  • New York, United States Alltake Full time

    Job Details:Check out the role overview below If you are confident you have got the right skills and experience, apply today. Job Description:We are looking for a skilled Data Engineer with 6-8 years of experience in data management and processing to join our growing team. The ideal candidate will have experience in working with big data ecosystems and...


  • New York, United States Big Cloud Full time

    Are you interested in revolutionary technology?Are you passionate about large-scale NLP and search?Would you like to help shape a world-class ML platform with proven results?I'm recruiting for one of the most advanced NLP start-ups in New York City. Their open-source content discovery platform uses NLP to extract research and connect data points across the...


  • New York, United States Big Cloud Full time

    Are you interested in revolutionary technology?Are you passionate about large-scale NLP and search?Would you like to help shape a world-class ML platform with proven results?I'm recruiting for one of the most advanced NLP start-ups in New York City. Their open-source content discovery platform uses NLP to extract research and connect data points across the...


  • New Brunswick, United States eProSoft Full time

    Generative AI Architect (Big Data & Java Expertise)Hybrid roleNew Brunswick, NJWe are seeking a skilled and innovative Generative AI Architect with a strong background in Big Data and Java to lead our AI-driven initiatives. The ideal candidate will design and implement cutting-edge generative AI models and systems that leverage large datasets, while ensuring...


  • New York, United States Ichor Strategies Full time

    Job SummaryIchor Strategies seeks a Principal Data Engineer to lead the data architecture of our nascent SaaS platform which connects companies to the communities they operate in so that they can understand different perspectives, build authentic relationships, and foster mutual success. The ideal candidate will have extensive experience in driving...


  • New York, United States Ichor Strategies Full time

    Job SummaryIchor Strategies seeks a Principal Data Engineer to lead the data architecture of our nascent SaaS platform which connects companies to the communities they operate in so that they can understand different perspectives, build authentic relationships, and foster mutual success. The ideal candidate will have extensive experience in driving...

  • Data Engineer

    3 weeks ago


    New York, NY, United States hackajob Full time

    Exciting Opportunity Alert! Join our Team as a Data Engineer!hackajob has partnered with an innovative company seeking a skilled Data Engineer to leverage data for impactful solutions. If you have expertise in big data tools like Spark and Kafka, and experience with cloud services like AWS, we want to hear from YOU!Role: Data EngineerLocation: Hybrid (New...


  • new york city, United States E-Solutions Full time

    Role : Big Data EngineerLocation : NYC office Fully Onsite (Local only)Inperson Interview is mustTechnical Skills:• Big Data Ecosystem: HDFS, Hive, Oozie, Sqoop, Impala, Presto and Spark.• Cloud Ecosystem: AWS (EMR, S3, Athena), Snowflake• Languages: Python, SQL and Shell Scripting.• Databases: MySQL, Teradata, and SQL SERVER.• Orchestration/Tools:...


  • new york city, United States E-Solutions Full time

    Role : Big Data EngineerLocation : NYC office Fully Onsite (Local only)Inperson Interview is mustTechnical Skills:• Big Data Ecosystem: HDFS, Hive, Oozie, Sqoop, Impala, Presto and Spark.• Cloud Ecosystem: AWS (EMR, S3, Athena), Snowflake• Languages: Python, SQL and Shell Scripting.• Databases: MySQL, Teradata, and SQL SERVER.• Orchestration/Tools:...

  • Sr. Data Engineer

    1 month ago


    New York, United States iO Associates - US Full time

    Are you a seasoned data engineer looking to make a significant impact in a dynamic environment? We're seeking a highly skilled professional to join our team and drive our data initiatives forward.Responsibilities:Design, develop, and implement advanced data pipelines using industry-leading cloud platforms.Architect and manage scalable data solutions for...