Senior Data Engineer

1 month ago


Sioux City, Iowa, United States Horizontal Talent Full time

Overview:

  • Architect and refine our Databricks-centric data platform, emphasizing scalable integrations and advanced entity resolution strategies.
  • Lead the charge towards a data mesh infrastructure, promoting domain-centric design and decentralized data management.
  • Enhance our DataOps capabilities, emphasizing data observability, discovery, and lineage to maintain data integrity and accuracy.
  • Pioneer the adoption of Databricks data lakehouse architectures, focused on simplifying data management and enhancing data processing capabilities.
  • Serve as a technical liaison among analytics engineers, BI developers, and analysts within the Databricks environment, integrating solutions like
  • Develop and optimize SQL and Python/Scala scripts within Databricks for complex data transformation and integration tasks.
  • Work closely with cross-functional teams to align data strategies with operational needs and objectives, incorporating advanced data resolution methodologies.
  • Stay at the forefront of industry trends and technological advancements to ensure our Databricks platform remains innovative and effective.


Must Have:

  • Bachelor's
  • Strong leadership skills and the ability to advocate for best practices in data management and processing.
  • 5+ years of experience in a data engineering role, with history of implementing data platform modernization
  • 2+ years of hands on databricks experience.
  • In-depth understanding of Databricks, along with experience in other cloud data warehouses like Snowflake, Redshift, Big Query.
  • Building data integration and orchestration within the databricks environment, with hands-on experience in technologies such as Delta Live Tables, CDC, dbt, airflow,
  • Advanced skills in SQL, Python, and Scala, with an emphasis on their application within Databricks for complex data tasks.
  • Knowledge of best practices in version control and CI/CD with Git and GitHub Actions; Agile project management with Jira and Confluence.
  • Proficiency with the Azure data suite (Azure SQL, Data Factory, Synapse Analytics, Power BI).
  • Solid understanding of data observability, discovery, and lineage, and their application in maintaining high data quality standards.
  • Additional experience with AI and ML capabilities in the context of Databricks is a plus.

Horizontal is proud to be an Equal Opportunity and Affirmative Action Employer. We seek to provide employment opportunities to talented, qualified candidates regardless of race, color, sex/gender including gender identity and/or expression, national origin, religion, sexual orientation, disability, marital status, citizen status, veteran status, or any other protected classification under federal, state or local law.

In addition, Horizontal will provide reasonable accommodations for qualified individuals with disabilities. If you need to request a reasonable accommodation in order to complete the application or interview process, please contact us. All applicants applying must be legally authorized to work in the country of employment.



  • Sioux City, Iowa, United States Expleo Full time

    Overview:Trissential is a trusted partner for end-to-end quality services and management consulting. As a part of our parent company Expleo, we have a footprint in 30+ countries. Our Mission is to have a positive impact on every person who interacts with Trissential. Clients love us because we are service focused. Employees love us because they are valued...


  • Sioux City, Iowa, United States CDM Smith Full time

    CDM Smith is seeking a Senior Automation and Controls Engineer with broad experience in the design and implementation of SCADA systems, controls, and instrumentation in the process industries. The successful candidate will have excellent communicatio Controls Engineer, Controls, Automation, Engineer, Automation Engineer, Senior, Technology, Sales


  • Sioux City, Iowa, United States CivicMinds, Inc Full time

    Position: Senior Engineer/PM Location: Woodbury, New York Full-time direct hire Overview: Hiring an Electrical Project Manager/Senior Engineer. This is an exciting opportunity to join our collaborative team of experienced engineers, architects, scientists and other professionals. Offers flexible hours and a unique environment, particularly well-suited for...


  • Sioux City, Iowa, United States CivicMinds, Inc Full time

    Position: Senior Engineer/PMLocation: Woodbury, New YorkFull-time direct hire Overview:Hiring an Electrical Project Manager/Senior Engineer. This is an exciting opportunity to join our collaborative team of experienced engineers, architects, scientists and other professionals.Offers flexible hours and a unique environment, particularly well-suited for those...


  • Sioux City, Iowa, United States Advantage Point Group Inc Full time

    Advantage Point Group (APG) is seeking to add an experienced Senior Electrical Design Engineer to our team in St. Paul, MN (near the 3M Center). NO RECRUITING OR STAFFING AGENCIES PLEASEType of Projects: Facilities Design at industrial plants such as lighting systems, power distribution systems, classified areas, clean rooms, construction, data centers,...


  • Charles City, Iowa, United States Weston Solutions Full time

    At Weston Solutions, Inc. you will do meaningful work and make valuable contributions. Employee ownership at Weston is a path to professional growth and access to diverse opportunities in a highly connected community that works together across key service areas to make a difference in the environment. Weston has continued to evolve and adapt to our changing...


  • Charles City, Iowa, United States Franklin Energy Full time

    As a combined organization, Franklin Energy and AM Conservation Group is undertaking what is perhaps the central challenge of our times to help drive the transition to net-zero carbon economies while ensuring that no working families, businesses, or local communities are left behind. To do that, we are the utility industrys top provider of turn-key...


  • Charles City, Iowa, United States NeoGenomics Full time

    Are you motivated to participate in a dynamic, multi-tasking environment? Do you want to join a company that invests in its employees? Are you seeking a position where you can use your skills while continuing to be challenged and learn? Then we encourage you to dive deeper into this opportunity.We believe in career development and empowering our employees....


  • Sioux City, Iowa, United States MasTec Inc Full time

    Overview MasTec Utility Services delivers critical infrastructure construction and engineering services for power delivery, gas, and water customers, specializing in overhead and underground electric distribution for power delivery systems, gas distribution construction for gas systems, and turnkey solutions for a variety of water, sewer, and civil...


  • Iowa City, Iowa, United States A. O. Smith Full time

    Press Tab to Move to Skip to Content LinkSelect how often (in days) to receive an alert:Select how often (in days) to receive an alert:We are one of the world's leading manufacturers of water heating and water treatment technologies in the world. We are a $3.5 billion company with 140+ years history and we employ more than 12,000 individuals globally who...


  • Mason City, Iowa, United States ASSA ABLOY Full time

    Career Opportunities: Associate Product Engineer Requisition ID Posted - AADG, Inc - ASSA ABLOY Door Group - USA: Iowa (Mason City) - Engineering & Science - Posting Country (1) - Yes - Associate - Travel Required: 0%-10% ? Job Description Print Preview Do you possess an Engineering degree? Do you want to work for a company where you truly have a voice...


  • Charles City, Iowa, United States AtScale Full time

    AtScale enables smarter decision-making by unlocking data-driven insights. The company's semantic layer platform simplifies, accelerates, and extends business intelligence and data science capabilities for enterprise customers across all industries. AtScale empowers customers to democratize data, implement self-service BI, and build a more agile analytics...

  • Senior Geologist

    1 week ago


    Iowa City, Iowa, United States Civil & Environmental Consultants, Inc. Full time

    Overview: Civil & Environmental Consultants, Inc. (CEC) continues to grow and expand. We have an excellent opportunity for a Senior Geologist to support the growth and establishment of an Environmental Practice in Johnson City office. You will be responsible for managing current clients, developing new business, as well as managing projects, mentoring junior...


  • Iowa City, Iowa, United States Cynet Systems Full time

    Job DescriptionWe are currently seeking an Engineering Technician for a client located in Johnson City, TN.About the Company: Join a reputable company in Johnson City that is in need of an Engineering Technician to support their projects.Job Title: Engineering TechnicianJob Location: Johnson City, TNJob Type: ContractResponsibilities:Participates in various...


  • Iowa City, Iowa, United States The University Of Iowa Full time

    The Design Engineer, IIHR will contribute to the research mission of IIHR-Hydroscience and Engineering and the Iowa Flood Center by collaborating with faculty and research staff to manage project tasks and deadlines to meet stakeholder expectations and project deliverables. Communicate project updates to internal and external stakeholders and perform other...


  • Iowa City, Iowa, United States University of Iowa Full time

    The Senior Behavior Health Consultant is focused on workplace health and well-being services, and reports through UI Wellness/University Human Resources and will be a member of the liveWELL team focused on well-being of UI faculty and staff ().This position will have primary responsibility for well-being programs focused on providing a high-quality Health...


  • Iowa City, Iowa, United States The University Of Iowa Full time

    The Quality Improvement Program at the University of Iowa Health Care Medical Center has openings for full-time Quality Improvement Engineers. The Quality Improvement Engineer partners with key stakeholders in the organization to analyze current processes and make changes where improvements will optimize organizational performance, ultimately improving the...


  • Charles City, Iowa, United States Databricks Full time

    This role can be remote. While candidates in the Bay Area are strongly preferred , candidates in other locations in the U.S. will be considered. The Machine Learning (ML) Practice team is a specialized customer-facing ML team at Databricks facing an increasing demand for Large Language Model (LLM)-based solutions. We deliver professional services...


  • Sioux City, Iowa, United States NYU Langone Health Full time

    NYU Langone Hospital-Long Island is a 591-bed university-affiliated medical center, which offers sophisticated diagnostic and therapeutic care in virtually every specialty and subspecialty of medicine and surgery. We are a major regional healthcare resource with a deep commitment to medical education and research, offering a full complement of inpatient and...


  • Sioux City, Iowa, United States Allina Health Full time

    Number of Job Openings Available:1Date Posted:March 06, 2024Department: Allina Health Group OB GYN EastShift:Day/Evening/Night (United States of America)Shift Length:Hours Per Week:40Union Contract:Non-UnionWeekend Rotation: OtherJob Summary:Join our busy and collaborative group at providing comprehensive and distinct complement of services designed to meet...