Data Engineer, Data Warehouse

3 weeks ago

Overland Park, United States CareerBuilder Full time

Company Overview

Shamrock Trading Corporation is the parent company for a family of brands in transportation services, finance and technology. Headquartered in Overland Park, KS, Shamrock is frequently recognized among the Best Places to Work in Kansas City and Chicago and was most recently recognized as one of Americas top 100 Most Loved Workplaces by Newsweek." We also have offices in Atlanta, Chicago, Dallas, Ft. Lauderdale, Houston, Laredo, Midland, Nashville and Phoenix.

With an average annual revenue growth of 25% over several decades, Shamrocks success is attributed to three key factors: hiring the best people, cultivating long-term relationships with our customers and continually evolving in the marketplace.

Responsibilities

Shamrock Trading Corporation is looking for a Data Engineer who wants to utilize their expertise in data warehousing, data pipeline creation/support and analytical skills by joining our Data Services team.This role is responsible for gathering and profiling data from many internal and external sources, building pipelines and designing a reliable cloud-focused data platform for analytics and business intelligence teams.This role requires significant understanding of data mining and analytical techniques. An ideal candidate will have strong technical capabilities, business acumen, and the ability to work effectively with cross-functional teams. Responsibilities include but are not limited to:

Work with Architects to understand current data models, to build pipelines for data ingestion and transformation
Design and implement pipelines and/or ETL/ELT jobs in order to centralize data from internal and external sources, focusing on reliability and performance of jobs and ensuring the quality and consistency of data is maintained at all times
Follow framework and code patterns established by Architects and Data Engineers for reliable job execution
Gather, understand and fulfill technical requirements from other units and provide necessary updates throughout the project lifecycle
Qualifications

Bachelors degree in computer science, data science or related technical field, or equivalent practical experience
Experience extracting and manipulating data from a variety of sources, primarily cloud-based
Experience building and maintaining data pipelines and ETL/ELT jobs: currently utilizing Matillion, Databricks, and Airflow
Expertise in optimizing performance when loading into and extracting from relational and other data structures, including SQL tuning, indexing, etc.
5+ years of development experience with the following, and Subject Matter Expert (SME) with at least one:
Strong SQL writing and tuning skills
Designing and deploying jobs within Matillion or similar GUI-based ETL/ELT tool
Expertise in designing, developing, deploying and/or supporting data pipelines using Databricks (pySpark, pySQL, DLT)
Relational Databases (MS SQL, PostgreSQL, Redshift)
Demonstrated analytical and problem-solving skills
Strong communication and listening skills, ability to work directly with customer teams (Business units and internal IT)
Preferred Qualifications

Understanding the frameworks and principles of traditional data warehousing
Experience with data processing and analytics within AWS eco-system
Experience data processing using Parquet and Avro files
Working with event based / streaming technologies to ingest and process data
Development experience delivering analytics processes using Python
Experience with front-end BI and data visualization tools (such as Power BI, Grafana, and QuickSight)
Benefits Package

At Shamrock we hire bright, ambitious people and give them the tools they need to be successful. By investing in training and development, we hope to become a long-term career for employees, where there are always opportunities for advancement. Shamrock also offers a premier set of benefits for employees and their families:

Medical:

Fully paid healthcare, dental and vision premiums for employees and eligible dependents
Work-Life Balance:

Competitive PTO and paid leave policies
Financial:

Generous company 401(k) contributions and employee stock ownership after one year
Wellness:

Onsite gym and discounted membership to select fitness centers. Jogging trails available at Overland Park offices
#LI-NB1 #LI-Hybrid
#J-18808-Ljbffr

  • Data Engineer

    1 week ago

    Overland Park, United States BAUER AG Full time

    Bauer For subsidiary BAUER Magyarország SpeciálisMélyépítő Kft. and its major special foundation project in Paks, Hungary, weare looking for an experienced Data Engineer - Construction - in rotation (gn). Your tasks:Processing, overseeing and monitoring computer data from our Drilling Rigs on our major construction sitePreparation of drilling rig...

  • Data Analytics Lead

    2 weeks ago

    Overland Park, KS, United States The Scoular Company Full time

    We are seeking a highly skilled Data Analytics Lead to serve as a technical lead within our analytics engineering team. This pivotal role is tailored for an individual with a profound technical background in data engineering and analytics, who thrives on leading projects and guiding teams towards achieving technical solutions. As a cornerstone of our team,...

  • Data Engineer

    4 weeks ago

    Elmwood Park, United States RedStream Technology Full time

    Data Engineer Elmwood Park, NJ (Hybrid, 3 days on-site per week) RedStream Technology is recruiting for a Data Engineer to work with our client, a leading mdeical services provider. The Data Engineer needs to be proficient with Matillion, Snowflake, and dbt in a hybrid Azure cloud environment. This role involves designing, implementing, and managing secure...

  • Data Engineer

    4 weeks ago

    Elmwood Park, United States RedStream Technology Full time

    Data Engineer Elmwood Park, NJ (Hybrid, 3 days on-site per week) RedStream Technology is recruiting for a Data Engineer to work with our client, a leading mdeical services provider. The Data Engineer needs to be proficient with Matillion, Snowflake, and dbt in a hybrid Azure cloud environment. This role involves designing, implementing, and managing secure...

  • Data Engineer

    4 days ago

    Menlo Park, United States Vertisystem Full time

    Onsite and only W2 Candidate required.MUST BE FULLY KNOWLEDGABLE IN PYTHON, SQL, AND ETL • Ability to work as part of a team, as well as work independently or with minimal direction. • Excellent written, presentation, and verbal communication skills. • Collaborate with data architects, modelers and IT team members on project goals. Summary: The main...

  • Data Engineer

    5 days ago

    Menlo Park, United States Vertisystem Full time

    Onsite and only W2 Candidate required.MUST BE FULLY KNOWLEDGABLE IN PYTHON, SQL, AND ETL • Ability to work as part of a team, as well as work independently or with minimal direction. • Excellent written, presentation, and verbal communication skills. • Collaborate with data architects, modelers and IT team members on project goals. Summary: The main...

  • Civil Engineering Internship

    5 days ago

    Overland Park, United States Olsson Full time

    Olsson provides multidisciplinary design services for some of the largest and most forward-thinking and desirable companies in the world to work for. The large hyperscale data center campuses we design throughout the U.S. will give you the opportunit Civil Engineer, Civil, Design, Engineer, Intern, Data, Business Services

  • Civil Engineering Internship

    3 days ago

    Overland Park, United States Olsson Full time

    Job Description Olsson provides multidisciplinary design services for some of the largest and most forward-thinking and desirable companies in the world to work for. The large hyperscale data center campuses we design throughout the U.S. will give you the opportunity to work on some of the largest and most complex engineering-driven projects being built...

  • Cyber Security Engineer

    5 days ago

    Overland Park, United States MDS (Micro-Data Systems) Full time

    Security Operations Automation Engineer Overland Park, KS - Hybrid 60% on-site Job DescriptionAs an Security Operations Automation Engineer, you are a critical part of our customers’ adoption of Cloud security integrations. You may be engaged with multiple clients as the subject matter expert, establishing relationships with them to ensure best practices...

  • Cyber Security Engineer

    7 days ago

    Overland Park, United States MDS (Micro-Data Systems) Full time

    Security Operations Automation Engineer Overland Park, KS - Hybrid 60% on-site Job DescriptionAs an Security Operations Automation Engineer, you are a critical part of our customers’ adoption of Cloud security integrations. You may be engaged with multiple clients as the subject matter expert, establishing relationships with them to ensure best practices...

  • Cyber Security Engineer

    7 days ago

    Overland Park, United States MDS (Micro-Data Systems) Full time

    Security Operations Automation Engineer Overland Park, KS - Hybrid 60% on-site Job DescriptionAs an Security Operations Automation Engineer, you are a critical part of our customers’ adoption of Cloud security integrations. You may be engaged with multiple clients as the subject matter expert, establishing relationships with them to ensure best practices...

  • Data Engineer

    4 days ago

    Menlo Park, United States Spectraforce Technologies Full time

    Data Engineer Menlo Park, CA 12 months Job Description: Perform analysis of data extracted from testbeds / test campaigns. Writing/scripting queries to collect data across the testbeds and various test equipment (e.g, traffic generators, cellular), systems (e.g, linux systems) and other types (e.g, telecom, cellular, wifi) Data extracting...

  • Data Engineer

    1 day ago

    Menlo Park, United States SPECTRAFORCE Full time

    Data EngineerMenlo Park, CA12 monthsJob Description:Perform analysis of data extracted from testbeds / test campaignsWriting/scripting queries to collect data across the testbeds and various test equipment (e.g, traffic generators, cellular), systems (e.g, linux systems) and other types (e.g, telecom, cellular, wifi)Data extracting results, observations,...

  • Data Engineer

    5 days ago

    Menlo Park, United States SPECTRAFORCE Full time

    Data EngineerMenlo Park, CA12 monthsJob Description:Perform analysis of data extracted from testbeds / test campaignsWriting/scripting queries to collect data across the testbeds and various test equipment (e.g, traffic generators, cellular), systems (e.g, linux systems) and other types (e.g, telecom, cellular, wifi)Data extracting results, observations,...

  • Data Engineer

    4 days ago

    Menlo Park, United States SPECTRAFORCE Full time

    Data EngineerMenlo Park, CA12 monthsJob Description:Perform analysis of data extracted from testbeds / test campaignsWriting/scripting queries to collect data across the testbeds and various test equipment (e.g, traffic generators, cellular), systems (e.g, linux systems) and other types (e.g, telecom, cellular, wifi)Data extracting results, observations,...

  • Data Engineer

    7 days ago

    Menlo Park, United States Spectraforce Technologies Inc Full time

    Data Engineer Menlo Park, CA 12 months Job Description: ● Perform analysis of data extracted from testbeds / test campaigns. ● Writing/scripting queries to collect data across the testbeds and various test equipment (e.g, traffic generators, cellular), systems (e.g, linux systems) and other types (e.g, telecom, cellular, wifi)...

  • Senior Data Engineer

    2 weeks ago

    Menlo Park, United States Vertisystem Full time

    ONLY W2 role No C2C Local Candidate requiredThe main function of the Data Engineer is to develop, evaluate, test and maintain architectures and data solutions within our organization. The typical Data Engineer executes plans, policies, and practices that control, protect, deliver, and enhance the value of the organization’s data assets.Job...

  • Senior Data Engineer

    2 weeks ago

    Menlo Park, United States Vertisystem Full time

    ONLY W2 role No C2C Local Candidate requiredThe main function of the Data Engineer is to develop, evaluate, test and maintain architectures and data solutions within our organization. The typical Data Engineer executes plans, policies, and practices that control, protect, deliver, and enhance the value of the organization’s data assets.Job...

  • Senior Data Engineer

    1 week ago

    Menlo Park, United States Vertisystem Full time

    ONLY W2 role No C2C Local Candidate requiredThe main function of the Data Engineer is to develop, evaluate, test and maintain architectures and data solutions within our organization. The typical Data Engineer executes plans, policies, and practices that control, protect, deliver, and enhance the value of the organization's data assets.Job Responsibilities:...

  • Data Engineer 3

    1 week ago

    Ridgefield Park, United States Aditi Consulting Full time

    Summary: Client's Bigdata team is looking to hire a Full Stack Engineer who will build and own the engineering roadmap of their cutting edge inhouse marketing SaaS solution. This solution harnesses the power of Company's big data to empower all marketing teams reach their customers throughout the country. Responsibilities: Design, develop, test, deploy,...