Data Engineer

1 day ago


Portland, United States VDart Full time
Job Title: Data Engineer (Azure Databricks)

Job Type : Fulltime

Location: Pittsburgh, PA (Onsite/Hybrid or ready to travel) & Portland OR (Onsite/Hybrid or ready to travel)

Required Skills/Qualifications:
  • 7-15 years of relevant experience
  • Bachelor's and/or master's degree in computer science or equivalent experience.
  • Strong communication, analytical and problem-solving skills with a high attention to detail.
Desired Experience:
  • At least two years of experience building and leading highly complex, technical engineering teams.
  • Strong hands-on experience in Databricks
  • Implement scalable and sustainable data engineering solutions using tools such as Databricks, Azure, Apache Spark, and Python. The data pipelines must be created, maintained, and optimized as workloads move from development to production for specific use cases.
  • Experience managing distributed teams preferred.
  • Comfortable working with ambiguity and multiple stakeholders.
  • Comfortable working cross functionality with product management and directly with customers; ability to deeply understand product and customer personas.
  • Expertise on Azure Cloud platform
  • Good SQL knowledge
  • Knowledge on orchestrating workloads on cloud
  • Ability to set and lead the technical vision while balancing business drivers
  • Strong experience with PySpark, Python programming
  • Proficiency with APIs, containerization and orchestration is a plus
  • Experience handling large and complex sets of data from various sources and databases
  • Solid grasp of database engineering and design principles.
  • Experience with Unity Catalog.
  • Familiarity with CI/CD methods desired
  • Good to have Teradata Experience (not Mandatory)
Responsibilities:
  • Manage end to end delivery by Investigating problem areas, working cross-functionally with product manager & other stakeholders
  • Follow the Agile development methodology; think strategically and execute methodically
  • Develop and manage capacity and growth projection forecasts of the environment within budgets
  • Create and maintain optimal data pipeline architecture, Assemble large, complex data sets that meet functional / non-functional business requirements
  • Drive optimization, testing and tooling to improve quality of solutions
  • Manage teams that build and operate high volume distributed systems in a SaaS environment
  • Great at devising efficient processes that increase velocity and quality
  • Train counterparts in the data pipelining and preparation techniques, which make it easier for them to integrate and consume the data they need for their own use cases.
  • Promote the available data and analytics capabilities and expertise to business unit leaders and educate them in leveraging these capabilities in achieving their business goals.
  • Accountable for operational effectiveness in performance, uptime, release management of the enterprise data platform.


  • Portland, United States Futran Tech Solutions Pvt. Ltd. Full time

    Job Title: Snowflake Data Engineer Work Location Portland,Oregon Job Description: Snowflake, AWS , SQL 6+ years of experience as a data engineer • Strong experience in executing Snowflake development projects Implementing a fully operational production grade large scale data solution on Snowflake Data Warehouse. Must have hands-on AWS cloud experience and...

  • Sr Data Engineer

    2 months ago


    Portland, United States Acumenz Consulting Full time

    Role : Sr Data Engineer Location : Onsite at Portland ORNote : Also candidate should be ready for F2F in Portland OR for final round if asked. Skills to target: Hands-onAWS, Airflow, DatabricksPysparkSQLKey Responsibilities:Collect, organize, and analyse large sets of data from various sources.Develop and implement data analyses, data collection systems,...


  • Portland, United States Providence Service Full time

    Description This role is currently remote, but may have the possibility of becoming a hybrid role. The Senior Data Engineer designs and builds modern data-centric software applications to support clinical and operational processes across all parts of the healthcare system. These applications leverage cloud computing, big data, data science, and modern...


  • Portland, Oregon, United States PacifiCorp Full time

    About UsPacifiCorp is a leading utility company committed to providing exceptional customer service, environmental sustainability, and diversity, equity, and inclusion.Job SummaryWe are seeking a skilled Data Engineer to join our team in Portland, OR or Salt Lake City, UT. The successful candidate will design, develop, and maintain robust data pipelines to...

  • Data Engineer

    3 weeks ago


    Portland, OR, United States Aquent Full time

    As a Data Engineer, you will work closely with business and technical teams to adapt and transform business requirements into effective data management solutions. You will leverage cutting-edge technologies such as Databricks, AWS EMR, Spark, Snowflake, and DBSQL to design, optimize, and implement data models and solutions that drive business value. Key...

  • Data Engineer

    3 weeks ago


    Portland, Oregon, United States PacifiCorp Full time

    About the Role:The T&D Data Engineer will be responsible for developing data pipelines and data governance standards to support technical and operational programs in electric operations. This position will work closely with data scientists, analysts, and other stakeholders to ensure data integrity, availability, and accessibility.Key Responsibilities:Design,...


  • Portland, United States GEI Consultants Full time

    Description Your role at GEI. The Jr. Web Application Data Engineer position is responsible for the development and support of web applications that are highly data centric. This position goes beyond the typical web programmer and will design and support objects within Microsoft SQL Server. The person hired for this position will work with a team of Web...


  • portland, United States Right! Systems, Inc. Full time

    SENIOR DATA CENTER ENGINEER – MICROSOFT/CLOUD (Portland,OR)CANDIDATES MUST BE LOCAL TO THE SERVICE AREA – INTERNATIONAL CANDIDATES NEED NOT APPLYSince 1993, Right! Systems, Inc. (RSI) has been providing exceptional solutions and uncommon expertise to help our customers achieve lasting benefits. As a multi-vendor systems integrator, we maintain high-level...


  • Portland, United States Right! Systems, Inc. Full time

    SENIOR DATA CENTER ENGINEER – MICROSOFT/CLOUD (Portland,OR)CANDIDATES MUST BE LOCAL TO THE SERVICE AREA – INTERNATIONAL CANDIDATES NEED NOT APPLYSince 1993, Right! Systems, Inc. (RSI) has been providing exceptional solutions and uncommon expertise to help our customers achieve lasting benefits. As a multi-vendor systems integrator, we maintain high-level...


  • portland, United States Right! Systems, Inc. Full time

    SENIOR DATA CENTER ENGINEER – MICROSOFT/CLOUD (Portland,OR)CANDIDATES MUST BE LOCAL TO THE SERVICE AREA – INTERNATIONAL CANDIDATES NEED NOT APPLYSince 1993, Right! Systems, Inc. (RSI) has been providing exceptional solutions and uncommon expertise to help our customers achieve lasting benefits. As a multi-vendor systems integrator, we maintain high-level...


  • Portland, United States CompuNet, Inc Full time

    Why join CompuNet? CompuNet values its people more than any other asset-and realizes the contributions made by each employee are a reflection of their education, experience, certifications, expertise and passions. We strive to take care of each other, do the right thing and help our customers succeed. We work to build lasting relationships and are proud that...


  • Portland, United States CompuNet, Inc Full time

    Why join CompuNet? CompuNet values its people more than any other asset-and realizes the contributions made by each employee are a reflection of their education, experience, certifications, expertise and passions. We strive to take care of each other, do the right thing and help our customers succeed. We work to build lasting relationships and are proud that...

  • Data Architect

    1 week ago


    Portland, United States Matlen Silver Full time

    Data ArchitectCompensation: $82 - $90/HourDomain: Supply ChainTeam: Warehousing Domain Data InterfaceContract To HireHybrid: 2-3 Days Onsite Portland, OregonAbout the Role:As a Data Architect, you will lead the design and development of warehousing domain data interfaces, playing a key role in shaping data architecture and delivering impactful data...


  • Portland, United States US012 Marsh & McLennan Agency LLC Full time

    Description: Our not-so-secret sauce. Award-winning, inclusive, Top Workplace culture doesn’t happen overnight. It’s a result of hard work by extraordinary people. More than 11,000 of the industry’s brightest talent drive our efforts to deliver purposeful work and meaningful impact every day. Learn more about what makes us different and how...

  • Data Scientist

    3 weeks ago


    portland, United States HCLTech Full time

    1. Experience in cleaning, preprocessing, and analyzing large datasets( more than 3 years)2. Ability to develop and implement data models and algorithms (preferred more than 3 years)3. Knowledge of data mining and pattern recognition with 3 years experience preferred 4. Establish and maintain the data pipeline and infrastructure. Collaborate with...

  • Data Scientist

    3 weeks ago


    portland, United States HCLTech Full time

    1. Experience in cleaning, preprocessing, and analyzing large datasets( more than 3 years)2. Ability to develop and implement data models and algorithms (preferred more than 3 years)3. Knowledge of data mining and pattern recognition with 3 years experience preferred 4. Establish and maintain the data pipeline and infrastructure. Collaborate with...

  • Data Scientist

    3 weeks ago


    Portland, United States HCLTech Full time

    1. Experience in cleaning, preprocessing, and analyzing large datasets( more than 3 years)2. Ability to develop and implement data models and algorithms (preferred more than 3 years)3. Knowledge of data mining and pattern recognition with 3 years experience preferred 4. Establish and maintain the data pipeline and infrastructure. Collaborate with...

  • Data Scientist

    3 weeks ago


    Portland, United States HCLTech Full time

    1. Experience in cleaning, preprocessing, and analyzing large datasets( more than 3 years)2. Ability to develop and implement data models and algorithms (preferred more than 3 years)3. Knowledge of data mining and pattern recognition with 3 years experience preferred 4. Establish and maintain the data pipeline and infrastructure. Collaborate with...

  • Data Engineer

    4 weeks ago


    Portland, OR, United States Aquent Full time

    Establishes database management systems, standards, guidelines and quality assurance for database deliverables, such as conceptual design, logical database, capacity planning, external data interface specification, data loading plan, data maintenance plan and security policy. Documents and communicates database design. Evaluates and installs database...


  • Portland, Oregon, United States PacifiCorp Full time

    Wildfire Mitigation ExpertisePacifiCorp is seeking a highly skilled data scientist to provide technical leadership on complex engineering projects related to wildfire mitigation. The successful candidate will facilitate internal training for engineers and technicians, act as the company subject matter expert, and develop presentations for senior...