Data Engineer

4 weeks ago

Chicago, United States Tiger Analytics Full time
Job DescriptionJob Description

Tiger Analytics is a fast-growing advanced analytics consulting firm. Our consultants bring deep expertise in Data Science, Machine Learning and AI. We are the trusted analytics partner for multiple Fortune 500 companies, enabling them to generate business value from data. Our business value and leadership has been recognized by various market research firms, including Forrester and Gartner. We are looking for top-notch talent as we continue to build the best global analytics consulting team in the world.

The Data Engineer will be responsible for architecting, designing, and implementing advanced analytics capabilities. The right candidate will have broad skills in database design, be comfortable dealing with large and complex data sets, have experience building self-service dashboards, be comfortable using visualization tools, and be able to apply your skills to generate insights that help solve business challenges.We are looking for someone who can bring their vision to the table and implement positive change in taking the company's data analytics to the next level.

Requirements

Key Responsibilities:

Data Integration:

Implement and maintain data synchronization between on-premises Oracle databases and Snowflake using Kafka and CDC tools.

Support Data Modeling:

Assist in developing and optimizing the data model for Snowflake, ensuring it supports our analytics and reporting requirements.

Data Pipeline Development:

Design, build, and manage data pipelines for the ETL process, using Airflow for orchestration and Python for scripting, to transform raw data into a format suitable for our new Snowflake data model.

Reporting Support:

Collaborate with data architect to ensure the data within Snowflake is structured in a way that supports efficient and insightful reporting.

Technical Documentation:

Create and maintain comprehensive documentation of data pipelines, ETL processes, and data models to ensure best practices are followed and knowledge is shared within the team.

Tools and Skillsets:

Data engineering: proven track record of developing and maintaining data pipelines and data integration projects

Databases: Strong experience with Oracle, Snowflake, and Databricks.

Data Integration Tools: Proficiency in using Kafka and CDC tools for data ingestion and synchronization.

Orchestration Tools: Expertise in Airflow for managing data pipeline workflows.

Programming: Advanced proficiency in Python and SQL for data processing tasks.

Data Modeling: Understanding of data modeling principles and experience with data warehousing solutions.

Cloud Platforms: Knowledge of cloud infrastructure and services, preferably Azure, as it relates to Snowflake and Databricks integration.

Collaboration Tools: Experience with version control systems (like Git) and collaboration platforms.

CI/CD Implementation: Utilize CI/CD tools to automate the deployment of data pipelines and infrastructure changes, ensuring high-quality data processing with minimal manual intervention.

Communication: Excellent communication and teamwork skills, with a detail-oriented mindset. Strong analytical skills, with the ability to work independently and solve complex problems.

Requirements

  • 8+ years of overall industry experience specifically in data engineering
  • 5+ years of experience building and deploying large-scale data processing pipelines in a production environment.
  • Strong experience in Python, SQL, and PySpark
  • Creating and optimizing complex data processing and data transformation pipelines using python
  • Experience with “Snowflake Cloud Datawarehouse” and DBT tool
  • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
  • Understanding of Datawarehouse (DWH) systems, and migration from DWH to data lakes/Snowflake
  • Understanding of ELT and ETL patterns and when to use each. Understanding of data models and transforming data into the models
  • Strong analytic skills related to working with unstructured datasets
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management

Benefits

This position offers an excellent opportunity for significant career development in a fast-growing and challenging entrepreneurial environment with a high degree of individual responsibility.

  • Software Engineer

    1 month ago

    Chicago, United States Spline Data Full time

    Spline Data is a focused on building modern quantitative market data feeds and visualizations for fixed income trading desks, portfolio managers, issuers, and beyond. We regard the data we create as if we were trading it ourselves, meaning detail, demonstrability, and robustness are key in every facet of our business. Changing a market isn't always about...

  • Lead Software Engineer

    2 weeks ago

    Chicago, Illinois, United States Spline Data Full time

    Spline Data specializes in developing contemporary quantitative market data feeds and visualizations tailored for fixed income trading desks, portfolio managers, issuers, and other stakeholders. We treat the data we generate with the same care as if we were trading it ourselves, emphasizing detail, demonstrability, and robustness in every aspect of our...

  • Systems Engineer

    1 month ago

    Chicago, United States Data Theorem Full time

    Job DescriptionJob DescriptionData Theorem is an exciting company focused on creating a more secure world for data. Rooted in a strong engineer first culture, every employee has an impact on product and direction. We are searching for exceptional talent pursuing an opportunity to grow and take ownership of the projects that resonate most with them.Our...

  • Data Engineer

    3 weeks ago

    Chicago, United States McDonald's Full time

    McDonald's evolving Accelerating the Arches growth strategy puts our customers and people first and demonstrates our competitive advantages to strengthen our brand. We are recognized on lists like Fortune’s Most Admired Companies and Fast Company’s Most Innovative Companies.Doubling Down on the 4Ds (Delivery, Digital, Drive Thru, and Development)Our...

  • Data Engineer

    3 weeks ago

    Chicago, United States McDonald's Full time

    McDonald's evolving Accelerating the Arches growth strategy puts our customers and people first and demonstrates our competitive advantages to strengthen our brand. We are recognized on lists like Fortune’s Most Admired Companies and Fast Company’s Most Innovative Companies.Doubling Down on the 4Ds (Delivery, Digital, Drive Thru, and Development)Our...

  • Data Engineer

    2 months ago

    Chicago, Illinois, United States United Airlines Full time

    There's never been a more exciting time to join United Airlines. We're on a path towards becoming the best airline in the history of aviation. Our shared purpose – Connecting People, Uniting the World – is about more than getting people from one place to another. It also means that as a global company that operates in hundreds of locations around the...

  • Data Engineer

    16 hours ago

    Chicago, United States Selby Jennings Full time

    We are seeking a Data Engineer to join the commodities team at a leading trading firm, where you'll be responsible for building and maintaining data pipelines for the ingestion, management, and analysis of datasets used by traders. Key Responsibilities:Develop and maintain reliable ETL pipelines using Python.Collaborate within a small, experienced...

  • Senior Software Engineer

    4 weeks ago

    Chicago, United States Spline Data Full time

    Spline Data is a focused on building modern quantitative market data feeds and visualizations for fixed income trading desks, portfolio managers, issuers, and beyond. We regard the data we create as if we were trading it ourselves, meaning detail, demonstrability, and robustness are key in every facet of our business. Changing a market isn't always about...

  • Data Engineer

    2 months ago

    Chicago, United States Prodigy Resources Full time

    We are seeking a Senior Data Engineer specializing in data UX/visualization to join our dynamic data infrastructure and analytics team. In this pivotal role, you will lead the design, development, and maintenance of robust data pipelines. Your expertise will ensure that data is not only efficiently processed but also presented in intuitive visual formats...

  • Data Engineer

    1 month ago

    Chicago, United States Prodigy Resources Full time

    We are seeking a Senior Data Engineer specializing in data UX/visualization to join our dynamic data infrastructure and analytics team. In this pivotal role, you will lead the design, development, and maintenance of robust data pipelines. Your expertise will ensure that data is not only efficiently processed but also presented in intuitive visual formats...

  • Regional VP of Construction, Data Centers

    4 weeks ago

    Chicago, United States Prime Data Centers Full time

    Job DescriptionJob DescriptionPrime Data Centers develops, acquires, and operates data centers for some of the world's largest enterprises. A private firm owned by a group controlling $6 billion in assets, with a 15-year tenure in technology and real estate development, Prime provides customers with ownership options and dynamic leasing models, defining...

  • Regional VP of Construction, Data Centers

    1 week ago

    Chicago, United States Prime Data Centers Full time $180,000 - $200,000

    Job DescriptionJob DescriptionPrime Data Centers develops, acquires, and operates data centers for some of the world's largest enterprises. A private firm owned by a group controlling $6 billion in assets, with a 15-year tenure in technology and real estate development, Prime provides customers with ownership options and dynamic leasing models, defining...

  • Data Engineer

    2 months ago

    Chicago, United States SpiderRock Holdings LLC Full time

    Job DescriptionJob DescriptionWe are searching for a Data Engineer with a passion for data, technology, and finance to improve our data product. This role will join our team in our Data Development team in our Chicago office. The qualified candidate will be responsible for developing various ETL pipelines, data warehouse architecture, streamlining data...

  • Data Engineer

    2 months ago

    Chicago, United States MaetaData Full time

    Job Description for Data EngineerUse programming language and tools (SQL, Python) to automate data intake and cleansing processesCreate and maintain data visualization dashboards and reports for internal and external useIdentify ways to improve data reliability, efficiency and qualityDevelop, construct, test and maintain cloud architecturesInterface with...

  • Data Engineer

    2 months ago

    Chicago, United States MaetaData Full time

    Job Description for Data EngineerUse programming language and tools (SQL, Python) to automate data intake and cleansing processesCreate and maintain data visualization dashboards and reports for internal and external useIdentify ways to improve data reliability, efficiency and qualityDevelop, construct, test and maintain cloud architecturesInterface with...

  • Data Engineer

    2 weeks ago

    Chicago, United States McDonald's Full time

    McDonald’s is proud to be one of the most recognized brands in the world, with restaurants in over 100 countries that serve 70 million customers daily. We continue to operate from a position of strength. Our updated growth strategy is focused on staying ahead of what our customers want and realizing further growth potential. Our relentless ambition is why...

  • Data Engineer

    2 weeks ago

    Chicago, United States McDonald's Full time

    McDonald’s is proud to be one of the most recognized brands in the world, with restaurants in over 100 countries that serve 70 million customers daily. We continue to operate from a position of strength. Our updated growth strategy is focused on staying ahead of what our customers want and realizing further growth potential. Our relentless ambition is why...

  • Data Engineer

    3 weeks ago

    Chicago, United States Tardis Tech Full time

    About the Role:We’re looking for a Cloud-Focused Data Engineer to design and manage data pipelines within AWS and Hadoop environments. This role offers an opportunity to work closely with cloud technologies and take ownership of building robust, scalable data solutions.Key Responsibilities:Design and implement data pipelines using AWS and Hadoop...

  • Senior Data Engineer

    3 days ago

    Chicago, United States Creospan Inc. Full time

    Creospan is a growing tech collective of makers, shakers, and problem solvers, offering solutions today that will propel businesses into a better tomorrow. “Tomorrow’s ideas, built today!” In addition to being able to work alongside equally brilliant and motivated developers, our consultants appreciate the opportunity to learn and apply new skills and...

  • Senior Data Engineer

    3 days ago

    Chicago, United States Creospan Inc. Full time

    Creospan is a growing tech collective of makers, shakers, and problem solvers, offering solutions today that will propel businesses into a better tomorrow. “Tomorrow’s ideas, built today!” In addition to being able to work alongside equally brilliant and motivated developers, our consultants appreciate the opportunity to learn and apply new skills and...