Big Data Engineer

4 weeks ago

United, United States HonorVet Technologies Full time
Project Details: Our client is looking for a Big Data Engineer to come in to work with a team of engineers on our cloud data platform that streams data from a variety of health care software and hardware systems in real-time to create transformational recommendations for our customers. Our solutions help to drive improved financial performance, compliance, and better patient outcomes.

Must-Have Skills:
  • Minimum of 8+ years of Software Engineering experience
  • 4+ Experience developing ETL processing flows usingSpark
  • 4+ years of experience writing in Scala code in Spark
  • Extensive experience with ETL of data
  • Experience with Hive data warehouse
  • Soft Skills:
    • Good communication is required
    • Independent, able to take initiative
    • Able to ask for clarification, problem solve, etc

Nice-To-Have Skills:
  • Experience with Kafka, Zookeeper, or Yarn
  • Experience with snowflake data warehouse
  • Experience with DataBricks
  • Demonstrated experience working in a cloud platform such as AWS, Microsoft Azure
  • 1+ years' experience with Big Data Client toolkits, such as Mahout, SparkML, or H2O
  • Demonstrable understanding of Service Oriented Architecture

Education & Certifications: Bachelor's Degree required

Daily Responsibilities:
  • Develop custom real-time streaming data pipelines working within the MapReduce ecosystem using Spark streaming
  • Ensure proper data governance policies are followed by implementing or validating data lineage, quality checks, classification, etc.
  • Act in a technical leadership capacity: Mentor junior engineers and new team members, and apply technical expertise to challenging programming and design problems
  • Resolve defects/bugs during QA testing, pre-production, production, and post-release patches
  • Have a quality mindset, squash bugs with a passion, and work hard to prevent them in the first place through unit testing, test-driven development, version control, continuous integration and deployment.
  • Ability to lead change, be bold, and have the ability to innovate and challenge status quo
  • Be passionate about solving customer problems and develop solutions that result in a passionate customer/community following
  • Conduct design and code reviews
  • Analyze and improve efficiency, scalability, and stability of various system resources
  • Contribute to the design and architecture of the project
  • Operate within Agile Development environment and apply the methodologies

  • Data Engineer

    2 weeks ago

    United, United States eTeam Full time

    Databricks, Spark Snowflake Cloud Data Platform Data Vault 2.0 model Enterprise Data Integrations WS Cloud architecture Event and messaging patterns, streaming data, AWS, Kafk Role & Responsibilities: Data Engineers will be responsible for design, build and maintain data pipelines ensuring data quality, efficient processing, and timely delivery...

  • Hadoop Big Data Architect

    2 months ago

    United States Ai Cuboid It Solutions Full time

    We currently have a contract to hire position for a Jr. Associate Hadoop Big Data Architect in Cleveland OH.W2 Candidates Only. We cannot sponsor.Will do some remote and onsite work.Business need for this role The team is implementing a new project with go live in 9/2022. Need member to help manage go live with the vendor anticipating new development in...

  • Hadoop Big Data Architect

    2 months ago

    United States Ai Cuboid It Solutions Full time

    We currently have a contract to hire position for a Jr. Associate Hadoop Big Data Architect in Cleveland OH.W2 Candidates Only. We cannot sponsor.Will do some remote and onsite work.Business need for this role The team is implementing a new project with go live in 9/2022. Need member to help manage go live with the vendor anticipating new development in 2022...

  • Senior Data Reliability Engineer

    2 weeks ago

    Chicago, IL, United States CME Group Full time

    Description This role is hybrid requires to be 2 days on site in our Chicago office. This role does not allow to work outside of Illinois state. Position Overview: Data System Reliability Engineer (dSRE) CME Group: Where Futures Are Made CME Group is the world's leading and most diverse derivatives marketplace. But who we are goes deeper than...

  • Data Engineer

    2 weeks ago

    Charlotte, NC, United States Synechron Inc. Full time

    We are At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron's progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud &...

  • Data Engineer

    2 weeks ago

    New York, NY, United States Synechron Inc. Full time

    We are At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron's progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud &...

  • Staff Data Engineer

    1 week ago

    Boston, MA, United States State Street Corporation Full time

    Who we are looking for The State Street Security Architecture, Analytics & Fusion Engineering (SA 2 FE) team is looking for a Staff Data Engineer, Integrations Lead . The Fusion Analytics and Data Engineering team delivers models, insights, and tooling to help Cybersecurity teams make faster, more informed decisions as we work to secure State Street's...

  • Data Engineer Intern

    3 weeks ago

    Chicago, IL, United States CME Group Full time

    Description CME Group is currently looking for a Data Science summer intern. This candidate will assist the Data Science team on day-to-day activities in support of Data Engineering such as building python and bigquery based ETL applications, build bigquery SQLs to enhance business users experience with GCP datasets. Principal Accountabilities: Create...

  • Principal Software Engineer

    3 days ago

    Norristown, PA, United States Comcast Corporation Full time

    FreeWheel, a Comcast company, provides comprehensive ad platforms for publishers, advertisers, and media buyers. Powered by premium video content, robust data, and advanced technology, we’re making it easier for buyers and sellers to transact across all screens, data types, and sales channels. As a global company, we have offices in nine countries and can...

  • Lead Data and Integration Engineer

    2 weeks ago

    United States Magellan Health Full time

    Interested in a fully remote opportunity as a Lead Data and Integration Engineer while working an innovative healthcare company where you can have an impact on people's lives? Apply today and come join us at Magellan Health as we empower people to lead healthier, more vibrant lives.we're looking for people with the following skill set for this role:In-depth...

  • Data Science Software Engineer

    2 weeks ago

    Austin, TX, United States State Street Corporation Full time

    State Street's Bionics team has the mission to explore, enable and exploit artificial intelligence, machine learning, natural language processing, computer vision and cognitive computing at scale for multiple use cases across Global Delivery, Global Advisors, Global Markets, and Global Exchange business lines. The team brings together a good mix of...

  • Data Engineer

    1 month ago

    United States Rakirs Systems LLC Full time

    Job Title Data EngineerLocation Alpharetta GADuration 12 months (extendable)Experience 2 - 8+yrspay rate $47.26/hourJob DescriptionThe Senior/ Junior Data Admin Analyst will work in the Brokerage platform that develops and supports applications that is responsible for compliance and reporting. The title/role is synonomous with that of Senior &...

  • Data Engineer

    1 month ago

    United States Rakirs Systems LLC Full time

    Job Title Data EngineerLocation Alpharetta GADuration 12 months (extendable)Experience - 8+yrspay rate $47.26/hourJob DescriptionThe Senior/ Junior Data Admin Analyst will work in the Brokerage platform that develops and supports applications that is responsible for compliance and reporting. The title/role is synonomous with that of Senior & Junior Data...

  • DATA ENGINEER III

    3 weeks ago

    , RI, United States FM Global Full time

    FM Global is a leading property insurer of the world's largest businesses, providing more than one-third of FORTUNE 1000-size companies with engineering-based risk management and property insurance solutions. FM Global helps clients maintain continuity in their business operations by drawing upon state-of-the-art loss-prevention engineering and research;...

  • Lead Data Software Engineer

    4 days ago

    Norristown, PA, United States Comcast Corporation Full time

    FreeWheel, a Comcast company, provides comprehensive ad platforms for publishers, advertisers, and media buyers. Powered by premium video content, robust data, and advanced technology, we’re making it easier for buyers and sellers to transact across all screens, data types, and sales channels. As a global company, we have offices in nine countries and can...

  • Sr. Manager Software Engineering

    4 days ago

    York, PA, United States Comcast Corporation Full time

    FreeWheel, a Comcast company, provides comprehensive ad platforms for publishers, advertisers, and media buyers. Powered by premium video content, robust data, and advanced technology, we’re making it easier for buyers and sellers to transact across all screens, data types, and sales channels. As a global company, we have offices in nine countries and can...

  • Data Center Operations Engineer

    3 days ago

    Ashburn, VA, United States Comcast Corporation Full time

    Make your mark at Comcast -- a Fortune 30 global media and technology company. From the connectivity and platforms we provide, to the content and experiences we create, we reach hundreds of millions of customers, viewers, and guests worldwide. Become part of our award-winning technology team that turns big ideas into cutting-edge products, platforms, and...

  • Data Engineer III

    2 weeks ago

    Johnston, RI, United States FM Global Full time

    FM Global is a leading property insurer of the world's largest businesses, providing more than one-third of FORTUNE 1000-size companies with engineering-based risk management and property insurance solutions. FM Global helps clients maintain continuity in their business operations by drawing upon state-of-the-art loss-prevention engineering and research;...

  • Software Engineer- Data

    4 weeks ago

    United States of America Venture Search Full time

    Software Engineer- Data Python New York 150k-175k USD base + bonus Our client, a leading quantitative investment company focused on computer-driven trading in global financial markets, is seeking a Software Engineer to join their Core Technology team.As a key player, the Software Engineer will contribute to building and scaling the company's central...

  • Software Engineer- Data

    4 weeks ago

    United States of America Venture Search Full time

    Software Engineer- Data  Python  New York  150k-175k USD base + bonus  Our client, a leading quantitative investment company focused on computer-driven trading in global financial markets, is seeking a Software Engineer to join their Core Technology team. As a key player, the Software Engineer will contribute to building and scaling the...