Remote Lead Data Engineer

2 weeks ago


San Jose, United States Insight Global Full time

Position: Lead Data Engineer (Databricks/AWS)

Duration: 6 Month Contract – w/extensions or conversion

Location: Remote ; East Coast



Must Haves:

  • 2-3 years exp. as a data engineer
  • 3 years of RECENT experience with Data Bricks
  • Must be their most relevant experience
  • Experience with Cloud ( Azure, GCP, or AWS)
  • Experience with SQL – they will be writing queries
  • Lead experience – Okay with not just being heads down
  • Python/pyspark experience


Plusses:

  • AWS Certifications
  • Data Governance experience
  • Bach Degree


Day to Day:

Insight Global is seeking a Lead Data Engineer to sit remote (EAST COAST) for a large retail client headquartered in the greater Pittsburgh area Our client recently kicked off a major project to build their own home-grown eCommerce platform and applications that will be used internally and then deployed out into their 600+ retail locations. In this role, the Data Engineer will be a member of the RISC team and will be a design, develop, and maintain scalable and efficient data pipelines for extracting, transforming, and loading (ETL) data from various sources into our data warehouse leveraging AWS, Databricks, Python and SQL technologies.


Additional responsibilities will include:


  • Collaborate with cross-functional teams to understand data requirements and implement solutions that meet business needs.
  • Optimize and troubleshoot existing data pipelines for performance and reliability.
  • Implement data quality and validation processes to ensure accuracy and consistency of data.
  • Stay updated on industry trends and best practices in data engineering to continuously improve our data infrastructure.
  • Work closely with data scientists, analysts, and other stakeholders to understand data use cases and provide the necessary infrastructure and support.
  • Assemble large, complex sets of data that meet non-functional and functional business requirements.
  • Identify, design and implement internal process improvements including re-design infrastructure for greater scalability, optimize data delivery, and automating manual processes.


Compensation: $70.00-80.00/HR


  • Remote Data Engineer

    2 weeks ago


    San Francisco, California, United States Archipelo Full time

    CompanyArchipelo is building a code security platform that gives organizations the ability to verify the authenticity and provenance of code within their software development lifecycle. They are solving a painful problem that affects every software developer on the planet: ensuring software security, authenticity, integrity, and compliance - by providing the...

  • Lead Data Engineer

    2 weeks ago


    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building scalable and reliable data infrastructure that enables advanced analytics, machine learning,...


  • San Jose, United States TikTok Full time

    ResponsibilitiesTikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul and Tokyo. Why Join UsCreation is the core of TikTok's purpose. Our platform is built to help imaginations thrive....


  • San Francisco, United States People Data Labs Full time

    Job DescriptionJob DescriptionAbout UsAt People Data Labs, we're committed to democratizing access to high-quality B2B data and leading the emerging DaaS economy. We empower developers, engineers, and data scientists to create innovative, compliant data products at scale with our clean, easy-to-use datasets of resume, company, location, and education...

  • Data Plane Engineer

    3 days ago


    San Jose, United States Cisco Full time

    Who We Are: The Cisco Distributed System Engineering (DSE) group is at the forefront of developing products that power the largest networks in the world. The networking industry is going through a massive transformation to build the next generation infrastructure to meet the needs of AI/ML workloads and continuously increasing internet users and application....


  • San Jose, United States tekHouse Full time

    100% Remote Contract 1099 or C2C 6 -9 month plusMUST be USCThe primary role of the Data Architect is to support and expand the Data & Analytics platforms that process, store, organize the data critical for the data and analytics team. This role will have expertise in data movement and database technologies support data analytics and Machine Learning (ML) /...


  • San Francisco, United States Unreal Gigs Full time

    Company Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building scalable and efficient data warehousing solutions that enable advanced analytics, reporting, and business intelligence. Join us...


  • San Francisco, United States Unreal Gigs Full time

    Company Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building scalable and efficient data warehousing solutions that enable advanced analytics, reporting, and business intelligence. Join us...


  • San Francisco, United States Unreal Gigs Full time

    Job Description Job Description Company Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data systems that enable efficient data management, processing, and analysis. Join...


  • San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building scalable and efficient data warehousing solutions that enable advanced analytics, reporting,...


  • San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data systems that enable efficient data management, processing, and analysis....


  • San Francisco, United States Unreal Gigs Full time

    Company Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data systems that enable efficient data management, processing, and analysis. Join us and lead our efforts in...

  • Data Engineering

    19 hours ago


    San Francisco, CA, United States Circle Full time

    Circle is a financial technology company at the epicenter of the emerging internet of money, where value can finally travel like other digital data — globally, nearly instantly and less expensively than legacy settlement systems. This ground-breaking new internet layer opens up previously unimaginable possibilities for payments, commerce and markets that...


  • San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is a trailblazer in leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge cloud data solutions that harness the scalability, flexibility, and power of...


  • San Francisco, United States Unreal Gigs Full time

    Company Overview: Welcome to the forefront of data-driven innovation! Our company is a trailblazer in leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge cloud data solutions that harness the scalability, flexibility, and power of cloud platforms. Join us and lead...


  • San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is a trailblazer in leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge cloud data solutions that harness the scalability, flexibility, and power of...


  • San Francisco, United States StreetLight Data Full time

    StreetLight pioneered the use of Big Data analytics to shed light on how people, goods, and services move, empowering smarter, data-driven transportation decisions. The company applies proprietary machine-learning algorithms and data processing resources to measure travel patterns of vehicles, bicycles and pedestrians that enable complex transportation...

  • Engineering Team Lead

    1 month ago


    San Francisco, CA, United States SnapEDA Full time

    Full Time] Engineering Lead, AI at SnapEDA (United States) | BEAMSTART Jobs Engineering Lead, AI Full Time Remote Work Stock Options SnapEDA, the leading platform for electrical engineers, is revolutionizing electronics design with artificial intelligence. We help 1.5 million engineers design products like drones, electric cars, and virtual...


  • San Jose, United States Motion Recruitment Full time

    This client is building a platform that enables businesses to adopt a comprehensive strategy for addressing code, infrastructure, and application risk. It incorporates AI technology to automate prioritizing, combining, and contextualizing data from various detection systems. They are looking for a Python software engineer with experience in Big Data and...


  • San Jose, United States Metric Geo Full time

    I am working with a leading producer of Transportation Designs for GDOT, looking for a senior roadway engineer to rapidly grow into a project management position. This role is 100% remote while offering one of the best salary and bonus packages available on the GA Market. This consultancy is completely employee owned, with options for ownership and...