Data Engineer

1 month ago

Edison, United States Diverse Lynx Full time
Role: Data Engineer
Location: Edison NJ (Onsite)
Duration: Full-time

8-10 Years

Skill: Data Engineer
• Design, develop, monitor, and operate data pipelines and maintenance of existing data pipelines.
• Collaborating with product owner and analysts to provide data-driven solutions and insights.
• Implementing data transformations and aggregations for large datasets.
• Ensuring data security and compliance with data governance policies.
• Developing in languages like Pyspark and Python.
• Preprocess of structured and unstructured data.
• nalyze, transform, and create SQL queries.
• Developing and optimizing Spark-based data processing applications on the AWS Databricks platform.
• Troubleshooting and debugging data-related issues on the AWS Databricks platform.
• Integrating Databricks with other data storage and processing systems.
What we are looking for:
• Expertise in Apache Spark along with Spark streaming & Spark SQL.
• Highly proficient with Apache Spark, Scala/ Python and PySpark.
• Good understanding & hands-on experience Gitlab with CI/CD.
• Hands-on experience in AWS Services like S3, Glue is required.
• Working knowledge of AWS Databricks, Cluster Management.
• Design, build, and test data pipelines and workflows using AWS Databricks.
• Having the knowledge about Optimizing AWS Databricks Streaming jobs.
• Have a deep understanding of AWS Cloud platforms and data engineering technologies.
• Strong SQL skills to perform data analysis and understanding of source data.
• Familiarity with NoSQL databases and JSON formats.
• Experience in building ETL / data warehouse transformation processes.
• Strong knowledge of Data Management principles.
• Good to have experience in analytical tool.
• Good to have experience in Alteryx/Tableau.

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

  • Azure Data Engineer @ Edison

    3 weeks ago

    Edison, United States eSolutionsFirst, LLC Full time

    Azure Data Engineer Edison ,NJ (or) McLean, VA 12 Months Contract (Possible extension) Required Skills:Azure Data EngineerAzure Data Factory to the more kind of like a custom development side of things deployed on Azure Function app Azure Databricks, workflows using data stores.Data Eng needs to have some exp. with power BI - building models. not an expert...

  • Data Architect

    6 days ago

    Edison, United States birasoft Full time

    The Possibilities are Endless When You ChallengeTheNorm Birlasoft combines the power of domain, enterprise, and digital technologies to reimagine business processes for customers and their ecosystem. Its consultative and design thinking approach makes societies more productive by helping customers run businesses. As part of the multibillion-dollar...

  • Data Architect

    5 days ago

    Edison, United States birasoft Full time

    The Possibilities are Endless When You ChallengeTheNorm Birlasoft combines the power of domain, enterprise, and digital technologies to reimagine business processes for customers and their ecosystem. Its consultative and design thinking approach makes societies more productive by helping customers run businesses. As part of the multibillion-dollar...

  • Data Architect

    3 days ago

    Edison, United States birasoft Full time

    The Possibilities are Endless When You ChallengeTheNorm Birlasoft combines the power of domain, enterprise, and digital technologies to reimagine business processes for customers and their ecosystem. Its consultative and design thinking approach makes societies more productive by helping customers run businesses. As part of the multibillion-dollar...

  • Data Architect

    2 days ago

    Edison, United States birasoft Full time

    The Possibilities are Endless When You ChallengeTheNorm Birlasoft combines the power of domain, enterprise, and digital technologies to reimagine business processes for customers and their ecosystem. Its consultative and design thinking approach makes societies more productive by helping customers run businesses. As part of the multibillion-dollar...

  • Senior Azure Data Architect/Engineer with PySpark with 12+ years working experience

    3 weeks ago

    Edison, United States SANS Full time

    Please DO NOT send your resume if you have LESS THAN 12 years of professional experinece. We are seeking senior candidates with 13+20 years of database development experience. Must be expert with DataBricks and PySpark. Job Title - Azure Data Solutions ArchitectLocation - Edison, NJPosition: Fulltime (W2); Company will sponsor H1B Visas for full-timeJob...

  • AWS Data Architect

    3 weeks ago

    Edison, United States Brilliant Infotech Inc. Full time

    Role: Aws Data ArchitectLocation: NY, NJ locationsDuration: Long term Job Description: Proven experience as a Data Architect, with a focus on designing and implementing data solutions in AWS cloud environments.Extensive knowledge of AWS cloud services, data lake architecture, and big data technologies. Strong understanding of data modeling principles, data...

  • Senior Engineer

    2 weeks ago

    Edison, United States Brillio Full time

    Brillio, LLC Senior Engineer Edison , New Jersey Apply Now Senior Engineer needed in Edison, NJ and various unanticipated locations throughout the U.S. Responsible for creating operational insights for telecom network related data from Fuze and integrate with other data source to identify revenue growth/cost optimization/operational efficiency opportunities....

  • Azure Data Solutions Architect

    3 weeks ago

    Edison, United States SANS Full time

    Job Title - Azure Data Solutions ArchitectLocation - Edison, NJ Job Summary:We are seeking an experienced Azure Data Solutions Architect with P&C Industry experience.If you are data freak and looking for a professionally challenging and financially rewarding career this position might be of interest to you. Azure Data Solutions Architect will spearhead new...

  • Senior Engineer

    4 weeks ago

    Edison, United States Brillio Full time

    Senior Engineer needed in Edison, NJ and various unanticipated locations throughout the U.S. Responsible for creating operational insights for telecom network related data from Fuze and integrate with other data source to identify revenue growth/cost optimization/operational efficiency opportunities. Interpret data and define new KPIs for data analysis from...

  • Senior Engineer

    3 days ago

    Edison, United States Brillio Full time

    Senior Engineer needed in Edison, NJ and various unanticipated locations throughout the U.S. Responsible for creating operational insights for telecom network related data from Fuze and integrate with other data source to identify revenue growth/cost optimization/operational efficiency opportunities. Interpret data and define new KPIs for data analysis from...

  • Director - Data science / AI & ML Practice Lead

    1 month ago

    Edison, United States Birlasoft Full time

    Birlasoft combines the power of domain, enterprise, and digital technologies to reimagine business processes for customers and their ecosystem. It is a consultative and design-thinking approach that makes societies more productive by helping customers run businesses. As part of the multibillion-dollar diversified CK Birla Group, Birlasoft with its 12,500+...

  • Director - Data science / AI & ML Practice Lead

    2 months ago

    Edison, United States Birlasoft Full time

    Birlasoft combines the power of domain, enterprise, and digital technologies to reimagine business processes for customers and their ecosystem. It is a consultative and design-thinking approach that makes societies more productive by helping customers run businesses. As part of the multibillion-dollar diversified CK Birla Group, Birlasoft with its 12,500+...

  • Sr. Electrical Engineer

    1 month ago

    Edison, United States BP Human Resources Full time

    Job Description Job Description Sr. Electrical Engineer (PE) The ideal candidate has and the position requires experience designing the electrical systems of Data Center Facilities. The position requires experience with the design of the appropriately sized Generators, Switchgear, UPS Systems, transformers, PDU’s and other Mission Critical equipment...

  • Entry Level software engineer/Electrical Engineer

    2 weeks ago

    Edison, United States Tata Consultancy Services Full time

    If you are interested please send resume to vasanth.x@tcs.com.. Job Title Entry Level Software Engineer / EIS Engineer /Data Scientist Project Location Edison NJ Job Type Full Time Eligibility Criteria GPA should be on or above 3.0 on the their degree. Associate degree cannot be considered. Candidates should have completed bachelors degree from May-16...

  • Entry Level software engineer/Electrical Engineer

    3 days ago

    Edison, United States Tata Consultancy Services Full time

    If you are interested please send resume to vasanth.x@tcs.com.. Job Title Entry Level Software Engineer / EIS Engineer /Data Scientist Project Location Edison NJ Job Type Full Time Eligibility Criteria GPA should be on or above 3.0 on the their degree. Associate degree cannot be considered. Candidates should have completed bachelors degree from May-16...

  • Sr. Mechanical Engineer

    2 weeks ago

    Edison, United States BP Human Resources Full time

    Job DescriptionJob DescriptionSENIOR MECHANICAL ENGINEER (P.E.)Edison, New Jersey company has an ideal opportunity for a senior mechanical engineer with data center/mission project experience. The ideal candidate will have a minimum of seven (7) years of data center mission critical mechanical engineering design experience that includes:1. Cooling...

  • Sr. Mechanical Engineer

    3 days ago

    Edison, United States BP Human Resources Full time

    Job DescriptionJob DescriptionSENIOR MECHANICAL ENGINEER (P.E.)Edison, New Jersey company has an ideal opportunity for a senior mechanical engineer with data center/mission project experience. The ideal candidate will have a minimum of seven (7) years of data center mission critical mechanical engineering design experience that includes:1. Cooling...

  • Azure Data Solutions Architect

    3 weeks ago

    Edison, United States SANS Full time

    Job Title - Azure Data Solutions ArchitectLocation - Edison, NJ Position: Fulltime (W2); Single Person C2C may be considered Job Summary:We are seeking an experienced Azure Data Solutions Architect with P&C Industry experience.If you are data freak and looking for a professionally challenging and financially rewarding career this position might be of...

  • Principal Engineer, Mechanical Engineering

    2 weeks ago

    Edison, United States Eos Energy Enterprises Inc. Full time

    At Eos, we believe in great chemistry. The combination of core elements into an amazing product and the mixture of skills into a unified team. We seek the best talent to just us in our mission to bring affordable, sustainable, clean energy to all. Come change the world with us. The Principal Engineer - Mechanical Engineering is accountable for driving...