Big Data Engineer
6 days ago
We are seeking a highly skilled Big Data Engineer to join our team at Collabera. As a Big Data Engineer, you will be responsible for designing, building, and maintaining large-scale data processing systems.
Key Responsibilities:- Design and implement data pipelines using Kafka, Pyspark, and real-time data processing techniques
- Develop and maintain large-scale data processing systems using Elastic Search and Kibana
- Work with cross-functional teams to design and implement data ingestion solutions
- Collaborate with data scientists to develop and deploy machine learning models
- Ensure data quality, security, and compliance with company policies
- Strong communication and collaboration skills
- Experience with Kafka, Pyspark, and real-time data processing
- Proficiency in SQL and Spark
- Experience with data ingestion from message queues, file shares, REST APIs, and relational databases
- Experience with data formats like JSON, CSV, and XML
- Experience with Spark Structured Streaming and Hadoop/Big Data and Distributed Systems
- Experience with Elastic Search and Kibana
- Experience with designing system from scratch and designing abstract layers
- Experience with Linux operating system
- Experience with Agile development process and Software Development Life Cycle
- Experience with Source Code and Version Control systems like SVN, Git, and Bit Bucket
- Competitive salary and benefits package
- Opportunities for professional growth and development
- Collaborative and dynamic work environment
-
Big Data Engineer
2 weeks ago
Chicago, Illinois, United States Collabera Full timeJob DescriptionJob Title: Big Data EngineerCompany: CollaberaLocation: Remote/HybridJob Type: Full-timeJob Category: EngineeringAbout the Role:We are seeking a highly skilled Big Data Engineer to join our team at Collabera. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Key...
-
Big Data Engineer
2 weeks ago
Chicago, Illinois, United States Collabera Full timeAbout the RoleWe are seeking a highly skilled Big Data Engineer to join our team at Collabera. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data processing systems.Key ResponsibilitiesData Ingestion and Processing: Design and implement data ingestion pipelines to collect and...
-
Big Data Architect
3 weeks ago
Chicago, Illinois, United States Tiger Analytics Full timeJob OverviewTiger Analytics, a leading advanced analytics consulting firm, is seeking a highly skilled Big Data Engineer to join our team. Our company provides expert analytics services to Fortune 500 companies, helping them unlock business value from data.The ideal candidate will have a strong background in traditional and modern Big Data technologies,...
-
Big Data Engineer
3 weeks ago
Chicago, Illinois, United States RSN GINFO SOLUTIONS LLC Full timeJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team at RSN GINFO SOLUTIONS LLC. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems using PySpark, Spark, and SQL.Key Responsibilities:Design and implement data pipelines using PySpark and SparkDevelop...
-
Big Data Developer
2 weeks ago
Chicago, Illinois, United States Collabera Full timeAbout the RoleWe are seeking a highly skilled Big Data Developer to join our team at Collabera. As a Big Data Developer, you will play a key role in designing and developing large-scale data processing systems.Key ResponsibilitiesDesign and develop data processing systems using Python and PysparkWork on individual modules and come up with design for a...
-
Lead Data Engineer
6 hours ago
Chicago, Illinois, United States Epsilon Full timeJob Title: Lead Data EngineerWe are seeking a highly skilled Lead Data Engineer to join our team at Epsilon. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining scalable, cloud-based software solutions.Key Responsibilities:Technical Expertise:Design and develop scalable, cloud-based software solutions...
-
Software Development Lead
2 weeks ago
Chicago, Illinois, United States Epsilon Data Interactive, Inc. Full timeAbout the RoleWe are seeking a highly skilled Senior Software Engineer to join our team at Epsilon Data Interactive, Inc. as an AI/ML Expert. In this role, you will collaborate with decision scientists and cross-functional teams to develop innovative AI and machine learning applications using Spark and Scala.Key ResponsibilitiesCollaborate with decision...
-
Data Pipeline Engineer
2 weeks ago
Chicago, Illinois, United States Jobot Full timeAbout the RoleWe are seeking a highly skilled Data Pipeline Engineer to join our team at Jobot. As a key member of our data infrastructure team, you will be responsible for designing, building, and deploying ETL and ELT flows for consuming data from disparate sources.Key ResponsibilitiesCreate and deploy ETL and ELT flows for consuming data from various...
-
Data Engineer
1 week ago
Chicago, Illinois, United States Capital One Full timeAbout the RoleCapital One is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will play a critical role in designing, developing, and implementing technical solutions that drive business value.Key ResponsibilitiesCollaborate with cross-functional teams to design and develop scalable data architecturesDevelop and maintain...
-
Data Engineer
2 weeks ago
Chicago, Illinois, United States McDonald's Full timeAbout the RoleWe are seeking a highly skilled Data Engineer to join our Global Technology – Data & Analytics team at McDonald's. As a Data Engineer, you will play a critical role in building scalable and efficient data solutions to support our company's data products and analytics initiatives.Key ResponsibilitiesDesign and Develop Data Pipelines: Develop...
-
Azure Data Engineer With Medallion
2 weeks ago
Chicago, Illinois, United States Magicforce Full timeAbout This RoleWe are seeking a highly skilled Azure Data Engineer with expertise in Medallion Architecture to join our team at Magicforce.Key ResponsibilitiesData Platform Architecture: Design and implement large-scale data and analytics platforms on Azure, leveraging Medallion Architecture to drive business growth.Data Integration and Governance: Develop...
-
Data Engineer
3 weeks ago
Chicago, Illinois, United States United Airlines Full timeJob SummaryWe are seeking a highly skilled Data Engineer to join our team at United Airlines. As a Data Engineer, you will be responsible for designing and building scalable and reliable data infrastructure and pipelines to support data for measurement and reporting across our digital platforms.Key ResponsibilitiesDesign and build data infrastructure and...
-
AWS Data Engineer
6 hours ago
Chicago, Illinois, United States Epsilon Full timeJob Title: Sr. AWS Data EngineerWe are seeking a highly skilled Sr. AWS Data Engineer to join our team at Epsilon. As a key member of our interdisciplinary team, you will be responsible for designing, developing, and implementing cloud-based data solutions that drive business growth and customer engagement.Job Responsibilities:80% of time spent on actual...
-
Senior Data Reliability Engineer
3 weeks ago
Chicago, Illinois, United States CME Group Full timeJob OverviewA critical role in CME Group's Cloud data transformation, the Data SRE will be aligned to data product pods ensuring the firm's data infrastructure is reliable, scalable, and efficient as the GCP data footprint expands rapidly.Key ResponsibilitiesOptimize data pipelines to ensure efficient data processing and reduce latency.Ensure data integrity...
-
Lead Data Engineer
3 weeks ago
Chicago, Illinois, United States Foursquare Full timeAbout FoursquareFoursquare is a leading independent location technology and data cloud platform, dedicated to building meaningful bridges between digital spaces and physical places. Our proprietary technology unlocks the most accurate, trustworthy location data in the world, empowering businesses to answer key questions, uncover hidden insights, improve...
-
Senior Data Engineer
3 weeks ago
Chicago, Illinois, United States United Airlines Full timeJob SummaryWe are seeking a highly skilled Data Engineer to join our team at United Airlines. As a Data Engineer, you will play a critical role in designing and building scalable and reliable data infrastructure and pipelines to support data for measurement and reporting across our digital platforms.Key ResponsibilitiesDesign and build scalable and reliable...
-
Senior Software Engineer
2 weeks ago
Chicago, Illinois, United States Epsilon Data Interactive, Inc. Full timeAbout the RoleWe are seeking a highly skilled Senior Software Engineer to join our team at Epsilon Data Interactive, Inc. as an AI/ML Expert. In this role, you will collaborate with decision scientists and cross-functional teams to develop innovative AI and machine learning applications using Spark and Scala.Key Responsibilities:Collaborate with decision...
-
Data Scientist
4 days ago
Chicago, Illinois, United States Lorven technologies Full timeJob DescriptionPosition: Data EngineerLocation: Chicago, ILDuration: ContractSkills: ETL concepts, GCP and its services, Big Query, Python, AirflowJob Requirements12+ Years of Overall IT ExperienceDesign and development of data ingestion and transformation processesProficiency in Python, BigQuery, DataProc, Cloud Storage and other GCP ServicesData Migration...
-
Machine Learning Engineer
2 weeks ago
Chicago, Illinois, United States minden Full timeAbout UsMinden is a technology venture that redefines the engagement between brands and consumers in Southeast Asia. We are a strategic partnership between Temasek, DFI Retail Group, and coalition partners.Our CultureWe value diversity, inclusivity, and collaboration. Our team is built on respect, authenticity, and transparency. We prioritize agility,...
-
Data Engineer
2 days ago
Chicago, Illinois, United States WHIZTEK CORP Full timeJob Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Whiztek Corp. As a Data Engineer, you will play a critical role in designing, building, and maintaining large-scale data pipelines and architectures.Key Responsibilities:Design and implement data pipelines using AWS services such as Redshift, S3, and AthenaDevelop and...