Big Data Engineer
7 days ago
Role : Big Data Engineer
Location : NYC office Fully Onsite (Local only)
Inperson Interview is must
Technical Skills:
• Big Data Ecosystem: HDFS, Hive, Oozie, Sqoop, Impala, Presto and Spark.
• Cloud Ecosystem: AWS (EMR, S3, Athena), Snowflake
• Languages: Python, SQL and Shell Scripting.
• Databases: MySQL, Teradata, and SQL SERVER.
• Orchestration/Tools: Oozie, Airflow, CTRL-M, Maven, Jenkins, IntelliJ, GIT
JD
• Expertise in all components of Big Data Ecosystem- Spark, Hive, Sqoop, Oozie, Impala and Hue
• Extensive Experience with AWS services ensuring efficient data management and analysis.
• Successfully led the migration of legacy systems to Datawarehouse environments and Cloud, utilizing Informatica, Snowflake and AWS technologies, resulting in improved scalability, performance, and cost-effectiveness.
• Extensively worked with lambda, S3, cloudwatch,SQS,SNS,Step Functions and IAM roles in AWS services.
• Extensive knowledge on data serialization techniques like Avro, Sequence Files, Parquet, JSON and ORC.
• Experienced in handling databases: MySQL, SQL Server, Teradata.
• Expertise working across all phases of SDLC requirements gathering, system design, development, enhancement, maintenance, testing, deployment, production support, and documentation.
• Working experience in creating complex data ingestion pipelines, data transformations, data management and data governance in a centralized enterprise data hub.
• Knowledge of Kubernetes ecosystem.
• Did Data analysis and interpretation Using Databricks features to perform data transformations, run queries, and build visualizations collaborating Working with diverse teams to deliver insights in Analytics platform. – good to have.
• Strong problem-solving skills
• Strong team player with good communication, analytical, presentation and inter-personal skills.
-
Big Data Engineer Lead
2 weeks ago
New York, United States InterSources Full timeJob Title: Big Data Engineer Type: Contract Location: New York, NY ( Day 1 onsite)Job Description: A Big Data Engineering lead handles the ETL jobs, Optimization, Automation of data processing for complex scenarios. Responsibilities: Data ingestion from Source DQ checks Handling ETL Automation Building features / variable Required Skills: Should be subject...
-
Big Data Architect with Devops
6 months ago
New York, United States CyberTec Full timeRole 3: Big Data Architect with Devops experience and local to NY Start Date: ASAP Rate - $90/hr. Maximum Start Date: ASAP Local to NY its onsite Client: Microsoft We'd like to see candidates at the Tech 3 level Description: Job Summary and Responsibilities: Hands on keyboard engineer with experience in the DevOps space and, preferably, experience in the...
-
Data Engineer
4 weeks ago
New York, United States hackajob Full timeExciting Opportunity Alert! Join our Team as a Data Engineer!hackajob has partnered with an innovative company seeking a skilled Data Engineer to leverage data for impactful solutions. If you have expertise in big data tools like Spark and Kafka, and experience with cloud services like AWS, we want to hear from YOU!Role: Data EngineerLocation: Hybrid (New...
-
Big data Architect
1 week ago
New York, United States SysMind Tech Full timeHi, Hope you are doing well. Here is the position. Kindly revert me back if you are interested. Role: Big Data Architect Location: Remote Duration: 12+ Months Skills: 1. 8-10 Years of strong Data architect\modeling experience 2. Design and architect the tables\data marts for staging, facts and dimensions 3. Develop mapping specs for Fact and Dimension...
-
Big Data Solutions Developer
4 weeks ago
New York, New York, United States RiVi Consulting Group Full timeJob SummaryAs a seasoned Spark Developer at RiVi Consulting Group, you will play a vital role in designing and implementing cutting-edge big data solutions. Your expertise in Spark and HDFS will enable us to deliver scalable and efficient data processing capabilities to drive business growth.Key Responsibilities• Collaborate with stakeholders to understand...
-
Data Engineer
4 weeks ago
new york city, United States hackajob Full timeExciting Opportunity Alert! Join our Team as a Data Engineer!hackajob has partnered with an innovative company seeking a skilled Data Engineer to leverage data for impactful solutions. If you have expertise in big data tools like Spark and Kafka, and experience with cloud services like AWS, we want to hear from YOU!Role: Data EngineerLocation: Hybrid (New...
-
Data Engineer
4 weeks ago
new york city, United States hackajob Full timeExciting Opportunity Alert! Join our Team as a Data Engineer!hackajob has partnered with an innovative company seeking a skilled Data Engineer to leverage data for impactful solutions. If you have expertise in big data tools like Spark and Kafka, and experience with cloud services like AWS, we want to hear from YOU!Role: Data EngineerLocation: Hybrid (New...
-
Data Engineer
4 weeks ago
New York, United States Brooksource Full timeData EngineerLocation: New York, NY (Hybrid)Contract-to-HireWe’re looking for an experienced Data Engineer to join a Fortune 50 Broadcast Media & Entertainment leader. This role involves designing, building, and optimizing data pipelines, integrating multiple datasets, and creating applications that drive strategic business decisions. You’ll collaborate...
-
GCP Machine Learning Engineer
4 weeks ago
New York, United States The Data Sherpas Full timeJ ob Summary : As a GCP Machine Learning Engineer, you will drive technology innovation by delivering world-class machine learning solutions. You will integrate GCP-native and third-party technologies to build scalable data infrastructures, focusing on modernizing data pipelines and machine learning platforms. You will collaborate with implementation teams...
-
Data Engineer
1 month ago
New York, New York, United States Glow Networks Full timeJob DescriptionAt Glow Networks, we are seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems in a cloud environment.Key Responsibilities:Design and implement big data architectures using cloud-based technologies such as AWS, Spark, and...
-
Data Engineer Full time
4 weeks ago
New York, United States Alltake Full timeJob Details:Check out the role overview below If you are confident you have got the right skills and experience, apply today. Job Description:We are looking for a skilled Data Engineer with 6-8 years of experience in data management and processing to join our growing team. The ideal candidate will have experience in working with big data ecosystems and...
-
Senior Software Engineer
3 weeks ago
New York, United States Big Cloud Full timeAre you interested in revolutionary technology?Are you passionate about large-scale NLP and search?Would you like to help shape a world-class ML platform with proven results?I'm recruiting for one of the most advanced NLP start-ups in New York City. Their open-source content discovery platform uses NLP to extract research and connect data points across the...
-
Senior Software Engineer
6 months ago
New York, United States Big Cloud Full timeAre you interested in revolutionary technology?Are you passionate about large-scale NLP and search?Would you like to help shape a world-class ML platform with proven results?I'm recruiting for one of the most advanced NLP start-ups in New York City. Their open-source content discovery platform uses NLP to extract research and connect data points across the...
-
Generative AI Architect
2 months ago
New Brunswick, United States eProSoft Full timeGenerative AI Architect (Big Data & Java Expertise)Hybrid roleNew Brunswick, NJWe are seeking a skilled and innovative Generative AI Architect with a strong background in Big Data and Java to lead our AI-driven initiatives. The ideal candidate will design and implement cutting-edge generative AI models and systems that leverage large datasets, while ensuring...
-
Principal Data Engineer
4 weeks ago
New York, United States Ichor Strategies Full timeJob SummaryIchor Strategies seeks a Principal Data Engineer to lead the data architecture of our nascent SaaS platform which connects companies to the communities they operate in so that they can understand different perspectives, build authentic relationships, and foster mutual success. The ideal candidate will have extensive experience in driving...
-
Principal Data Engineer
2 months ago
New York, United States Ichor Strategies Full timeJob SummaryIchor Strategies seeks a Principal Data Engineer to lead the data architecture of our nascent SaaS platform which connects companies to the communities they operate in so that they can understand different perspectives, build authentic relationships, and foster mutual success. The ideal candidate will have extensive experience in driving...
-
Data Engineer
3 weeks ago
New York, NY, United States hackajob Full timeExciting Opportunity Alert! Join our Team as a Data Engineer!hackajob has partnered with an innovative company seeking a skilled Data Engineer to leverage data for impactful solutions. If you have expertise in big data tools like Spark and Kafka, and experience with cloud services like AWS, we want to hear from YOU!Role: Data EngineerLocation: Hybrid (New...
-
E-Solutions | Big Data Engineer
7 days ago
new york city, United States E-Solutions Full timeRole : Big Data EngineerLocation : NYC office Fully Onsite (Local only)Inperson Interview is mustTechnical Skills:• Big Data Ecosystem: HDFS, Hive, Oozie, Sqoop, Impala, Presto and Spark.• Cloud Ecosystem: AWS (EMR, S3, Athena), Snowflake• Languages: Python, SQL and Shell Scripting.• Databases: MySQL, Teradata, and SQL SERVER.• Orchestration/Tools:...
-
E-Solutions | Big Data Engineer
7 days ago
new york city, United States E-Solutions Full timeRole : Big Data EngineerLocation : NYC office Fully Onsite (Local only)Inperson Interview is mustTechnical Skills:• Big Data Ecosystem: HDFS, Hive, Oozie, Sqoop, Impala, Presto and Spark.• Cloud Ecosystem: AWS (EMR, S3, Athena), Snowflake• Languages: Python, SQL and Shell Scripting.• Databases: MySQL, Teradata, and SQL SERVER.• Orchestration/Tools:...
-
Sr. Data Engineer
1 month ago
New York, United States iO Associates - US Full timeAre you a seasoned data engineer looking to make a significant impact in a dynamic environment? We're seeking a highly skilled professional to join our team and drive our data initiatives forward.Responsibilities:Design, develop, and implement advanced data pipelines using industry-leading cloud platforms.Architect and manage scalable data solutions for...