Lead Databricks Engineer
3 days ago
Role : Lead Databricks Engineer
Location : Iselin NJ / New York, NY : Hyrbid fully onsite)
Must have : Expert in Databricks , Pyspark , python
JD:
Strong knowledge on data bricks architecture and tools
Have experience of task and wf jobs creations in data bricks.
Deep understanding of distributed computing and how to use spark for data processing.
SQL and pyspark : strong command over querying databases and proficiency in pyspark.
Cloud platform: Preferred Azure for data bricks deployment.
Qualifications
• Bachelor's degree in Computer Science, Engineering, or related field, or equivalent work experience
• 5+ years of experience in data engineering with Databricks and Spark
• Proficient in SQL and Python and Pyspark
• Experience with Azure Databricks Medallion Architecture with DLT, Iceberg
• Financial/Corporate Banking context would be a plus
• Experience with data integration and ETL tools, such as Azure Data Factory
• Experience with Azure cloud platform and services
• Experience with data warehouse and data lake concepts and architectures
• Good to have experience with big data technologies, such as Kafka, Hadoop, Hive, etc
• Strong analytical and problem-solving skills
• Excellent communication and teamwork skills
Responsibilities
• Design, develop, and maintain data pipelines using Databricks and Spark, and other cloud technologies as needed
• Optimize data pipelines for performance, scalability, and reliability
• Ensure data quality and integrity throughout the data lifecycle
• Collaborate with data scientists, analysts, and other stakeholders to understand and meet their data needs
• Troubleshoot and resolve data-related issues, and provide root cause analysis and recommendations
• Document data pipeline specifications, requirements, and enhancements, and communicate them effectively to the team and management
• Create new data validation methods and data analysis tools, and share best practices and learnings with the data engineering community
• Implement ETL processes and data warehouse solutions, and ensure compliance with data governance and security policies
-
Lead Databricks Engineer
2 months ago
Iselin, United States Persistent Systems Full timeAbout PersistentWe are a trusted Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what’s next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many...
-
Data Engineer
2 weeks ago
Iselin, United States Synechron Full timeWe areAt Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron’s progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud & DevOps,...
-
Data Engineer
1 week ago
Iselin, United States Synechron Full timeWe areAt Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron’s progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud & DevOps,...
-
Lead Software Engineer
4 months ago
Iselin, United States Veracity Full timeJob Title: Lead Software Engineer / Pega Engineer Duration: 12+ Months Location: Charlotte, NC - Hybrid Role (3 Days Onsite/2 Days WFH) Required Qualifications: • 5+ years of Software Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, or education. • Certified...
-
Lead Cloud Engineer
7 days ago
iselin, United States HAN Staffing Full timeLead Cloud Engineer Location: Piscataway, NJ • Hand's on experience managing 4 member teams. • Deep understanding of AWS Cloud platforms and services, • Strong experience in Cloud Infrastructure management, CI/CD pipeline, Security processes, Containerization and Orchestration , infrastructure optimization, Issues troubleshooting, • Proficiency in...
-
Senior Data Architect
4 weeks ago
Iselin, New Jersey, United States CloudBC Labs Full timeJob Title: Senior Data ArchitectJob Summary:CloudBC Labs is seeking a highly skilled Senior Data Architect to design and implement large-scale data solutions on Snowflake Cloud Data Warehouse. The ideal candidate will have a strong background in data warehousing, data modeling, and data engineering, with experience in Snowflake utilities such as Snow SQL,...
-
Senior Software Engineer/Technical Lead
3 weeks ago
Iselin, United States New York Technology Partners Full timeDescription: Research and analyze various software's/systems and database tools to identify best technologies for developing software applications in conjunction w/hardware. Participate in multiple phases of project life cycle development including requirement gathering, software design/architecture, development, testing, deployment and support of software...
-
Electrical Engineer
4 weeks ago
Iselin, United States Futures Consulting Full timeJob DescriptionJob DescriptionElectrical Project EngineerJob ResponsibilitiesEngineer TasksDesign of electrical systems for a variety of projects.Create plans, schematics, equipment selection, load calculations, systems narratives and specifications, system design and layout, field investigation and construction administration.Perform design work on Revit...
-
Principal Geotechnical Engineer
4 weeks ago
Iselin, New Jersey, United States Mott MacDonald Full timeMott MacDonald is a leading engineering and advisory consultancy with a strong national presence in transportation and infrastructure, water/wastewater, and energy sectors.We are seeking a highly skilled technical principal engineer to join our Geotechnics Practice and support our current and future infrastructure projects throughout North America.The...
-
Senior OpenShift Engineer
3 days ago
Iselin, United States Collabera Full timeAbout Collabera:Collabera is a leading global technology services and solutions provider committed to delivering high-quality, innovative solutions to our clients. Our diverse, global talent helps clients transform every aspect of their business and achieve exceptional results. We achieve success through collaboration and the use of our digital platforms....
-
ServiceNow Architect App Engine
4 weeks ago
Iselin, New Jersey, United States Lorven Technologies Full timeJob Summary: We are seeking a highly skilled ServiceNow Architect App Engine to lead our team in delivering exceptional IT service management solutions. The ideal candidate will have a minimum of 12 years of experience in App Engine development, with expertise in ITSM modules, integration, and migrations projects. Key Responsibilities:Design and implement...
-
Principal Geotechnical Engineer
4 weeks ago
Iselin, New Jersey, United States Mott MacDonald Group Full timeMott MacDonald is a multi-disciplined engineering and advisory consultancy, with a significant national presence in the transportation and infrastructure, water/wastewater, energy realm.We are seeking a technical principal engineer to support our current and future infrastructure projects throughout Mott MacDonald's East Unit and beyond.The successful...
-
Senior Cloud Software Engineer
4 weeks ago
Iselin, New Jersey, United States JobRialto Full timeJob Summary:We are seeking a highly skilled Senior Cloud Software Engineer to join our team at JobRialto. As a key member of our organization, you will be responsible for designing, developing, and deploying cloud-based software applications using Java and React. Your expertise in cloud infrastructure, middleware, and software engineering will be crucial in...
-
Senior Software Engineering Lead
3 weeks ago
Iselin, New Jersey, United States Diverse Lynx Full timeRole OverviewAt Diverse Lynx LLC, we are seeking a seasoned Technical Lead to spearhead the development of our cutting-edge software solutions. As a key member of our team, you will oversee the design, development, and implementation of complex web applications, leveraging your expertise in Spring Boot, Angular, and microservices.Key ResponsibilitiesLead the...
-
ServiceNow with App Engine
1 month ago
Iselin, United States Lorven Technologies Inc. Full timeOur client is looking ServiceNow with App Engine for long term project in Iselin, NJ Below is the detail requirement.Job Title: ServiceNow with App EngineLocation: Iselin, NJ (Hybrid Mode onsite for 3 days)Duration: 12+monthsJob Description: Bachelor’s degree or equivalent working experience with minimum of 10+ years of IT experience.Required Skills:Lead...
-
Site Reliability Engineer
4 weeks ago
Iselin, New Jersey, United States Diverse Lynx Full timeJob SummaryAs a Site Reliability Engineer at Diverse Lynx LLC, you will be responsible for ensuring the constant health of our supporting systems and maintaining the top performance of our customer-facing services. This role requires expertise in TCP/IP related technologies, CLI enterprise support of Unix variants, and strong understanding of monitoring...
-
Iselin, United States Wells Fargo Full timeAct as an advisor to leadership to develop or influence applications, network, information security, database, operating systems, or web technologies for highly complex business and technical needs across multiple groups. Lead the strategy and resolu Engineer, Principal, Technical Engineer, Leadership, Technology, Banking, Benefits
-
ServiceNow Architect App Engine
4 weeks ago
Iselin, New Jersey, United States Lorven Technologies Full timeRole: ServiceNow Architect App EngineLocation: Hybrid ModeContract PositionJob Description:Key Responsibilities:Minimum 12 years of expertise in ITSM and integration projects.Experience with ServiceNow issue analysis and fixing for ITSM modules.Proven knowledge of ServiceNow platform administration, configuration, customization, upgrade, and patching.Lead...
-
Sr. Java Tech Lead
2 months ago
Iselin, United States Open Systems Technologies Full timeA financial firm is looking for a Sr. Java Tech Lead to join their team in Iselin, NJ. Pay: $170-200kResponsibilities: Deliver solutions for critical service that is built on client server architecture using middleware components and database Lead a team of talented developers in the design, development, and implementation of Java-based applications...
-
ServiceNow Architect with App Engine
2 weeks ago
Iselin, United States Lorven Technologies Inc. Full timeOur client is looking ServiceNow Architect with App Engine for long term project in Iselin, NJ Below is the detail requirement.Job Title: ServiceNow Architect with App Engine Location: Iselin, NJ (Hybrid Mode onsite for 3 days)Duration: 12 months Job Description: Bachelor’s degree or equivalent working experience with minimum of 12+years of IT...