Lead Bigdata Spark Engineer

1 month ago

Pasco, United States ApTask Full time
About Client:
The client provides information technology (IT) services, including business outsourcing, infrastructure technology, and application services. The application service offered by the company includes application development, maintenance, and support. The markets served by the company are financial services and insurance, healthcare, manufacturing, government, transportation, communications, and consumer and retail industries.

Salary Range: $110K-$120K/Annum+ Benefits

Job Description:
  • The data engineering role is a team member that will help enhance and maintain the CSS Business Analytics and Instant Ink Business Intelligence system.
  • You will drive work you're doing to completion with hands-on development responsibilities, and partner with the Data Engineering leaders to implement data engineering pipelines to build solution to help provide trusted and reliable data to customers.
  • Years of experience needed - 8+ years
Technical Skills:
  • Design and implement distributed data processing pipelines using Spark, Python, SQL and other tools and languages prevalent in the Big Data/Lakehouse ecosystem.
  • Analyzes design and determines coding, programming, and integration activities required based on general objectives.
  • Play the technical lead role representing deliverables from vendor team resources at onsite and offshore locations.
  • Lead the technical co-ordination and Business Knowledge transition activities to offshore team.
  • Reviews and evaluates designs and project activities for compliance with architecture, security and quality guidelines and standards
  • Writes and executes complete testing plans, protocols, and documentation for assigned portion of data system or component; identifies defects and creates solutions for issues with code and integration into data system architecture.
  • Collaborates and communicates with project team regarding project progress and issue resolution.
  • Works with the data engineering team for all phases of larger and more-complex development projects and engages with external users on business and technical requirements.
  • Collaborates with peers, engineers, data scientists and project team.
  • Typically interacts with high-level Individual Contributors, Managers and Program Teams on a daily/weekly basis.
Certifications Needed:
  • Bachelor's or Master's degree in Computer Science, Information Systems, Engineering or equivalent.
  • 6+ years of relevant experience with detailed knowledge of data warehouse technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools.
  • 3+ years of experience with Cloud based DW such as Redshift, Snowflake etc.
  • 3+ years' experience in Big Data Distributed ecosystems (Hadoop, SPARK, Unity Catalog & Delta Lake)
  • 3+ years experience in Workflow orchestration tools such as Airflow etc.
  • 3+ years' experience in Big Data Distributed systems such as Databricks, AWS EMR, AWS Glue etc.
  • Leverage monitoring tools/frameworks, like Splunk, Grafana, CloudWatch etc.
  • Experience with container management frameworks such as Docker, Kubernetes, ECR etc.
  • 3+ year's working with multiple Big Data file formats (Parquet, Delta Lake)
  • Experience working on CI/CD processes such as Jenkins, Codeway etc. and source control tools such as GitHub, etc.
  • Strong experience in coding languages like Python, Scala & Java
Knowledge and Skills:
  • Fluent in relational based systems and writing complex SQL.
  • Fluent in complex, distributed and massively parallel systems.
  • Strong analytical and problem-solving skills with ability to represent complex algorithms in software.
  • Strong understanding of database technologies and management systems.
  • Strong understanding of data structures and algorithms
  • Database architecture testing methodology, including execution of test plans, debugging, and testing scripts and tools.
Nice to Have:
  • Experience with visualization tools such as PowerBI, Tableau etc.
  • Experience with transformation tools such as dbt.
  • Have experience in building realtime streaming data pipelines
  • Experience in pub/sub streaming technologies like Kafka, Kinesis, Spark Streaming etc


About ApTask:
Join ApTask, a global leader in workforce solutions and talent acquisition services, as we shape the future of work. We offer a comprehensive suite of offerings, including staffing and recruitment services, managed services, IT consulting, and project management, providing unparalleled opportunities for professional growth and development. As a member of our dynamic team, you'll have the chance to connect businesses with top-tier professionals, optimize workforce performance, and drive success for our clients across diverse industries. If you are passionate about excellence, collaboration, and innovation, and aspire to make a meaningful impact in the world of work, come join us at ApTask and be a part of our mission to empower organizations to thrive.

Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government issued ID during each interview.

Candidate Data Collection Disclaimer:
At ApTask, we prioritize safeguarding your privacy. As part of our recruitment process, certain Personally Identifiable Information (PII) may be requested by our clients for verification and application purposes. Rest assured, we strictly adhere to confidentiality standards and comply with all relevant data protection laws. Please note that we only collect the necessary information as specified by each client and do not request sensitive details during the initial stages of recruitment.

If you have any concerns or queries about your personal information, please feel free to contact our compliance team at businessexcellence@aptask.com

  • Senior Machine Learning Engineer

    7 days ago

    Pasco, United States Sayari Labs Full time

    About Sayari: Sayari is the counterparty and supply chain risk intelligence provider trusted by government agencies, multinational corporations, and financial institutions. Its intuitive network analysis platform surfaces hidden risk through integrated corporate ownership, supply chain, trade transaction and risk intelligence data from over 250...

  • Union Lead Engineer

    1 week ago

    Pasco, Washington, United States CBRE Full time

    Union Lead Engineer Job ID Posted 07-Jun-2024 Service line Advisory Segment Role type Full-time Areas of Interest Engineering/Maintenance, Property Management Location(s) Washington, D.C. - District of Columbia - United States of America Are you ready to start an exciting career with CBRE? About the Role: As a Union Lead Building...

  • Lead Incident Response Engineer

    1 week ago

    Pasco, United States Blue Cross Blue Shield companies Full time

    Job Description Summary The Lead Threat Detection and Response Engineer is highly technical role that will lead the Incident Response process within BCBSA Cyber Defense team. This role acts as the DFIR subject matter expert that plans and oversees initiatives to bring the best techniques in detection and response to BCBSA. The Lead Threat Detection and...

  • Senior Network Engineer

    1 week ago

    Pasco, Washington, United States Fox News Network Full time

    OVERVIEW OF THE COMPANY Fox Corporation Under the FOX banner, we produce and distribute content through some of the world's leading and most valued brands, including: FOX News Media, FOX Sports, FOX Entertainment, FOX Television Stations and Tubi Media Group. We empower a diverse range of creators to imagine and develop culturally significant content, while...

  • Senior Machine Learning Engineer

    3 weeks ago

    Pasco, United States Remitly Full time

    Job Description: About the Role At Remitly, our Central Machine Learning Team plays a pivotal role in empowering various business verticals through cutting-edge machine learning solutions. As a Senior Machine Learning Engineer, you will be at the forefront of this endeavor, collaborating closely with diverse teams such as Customer Success and Treasury....

  • Azure Data Engineer

    7 days ago

    Pasco, United States Ciber Full time

    HTC Global Services wants you. Come build new things with us and advance your career. At HTC Global you'll collaborate with experts. You'll join successful teams contributing to our clients' success. You'll work side by side with our clients and have long-term opportunities to advance your career with the latest emerging technologies. At HTC Global Services...

  • Level 2 Data Scientist with Security Clearance

    1 month ago

    Pasco, United States Synovix Full time

    Level II Data ScientistFalls Church, Virginia Required Education: Bachelor's degree in computer science or related technical field plus 4 years' experience, Master's Degree in Computer Science w/ 2 years' experience. Required Skills/Experience: Experience with data science techniques, classical machine learning, and deep learning for anomaly detection....

  • Cloud Solutions Engineer

    1 month ago

    Pasco, United States Ardent Services Full time

    Why do you need to choose between doing important work and having a fulfilling life? At Ardent, we have both. Ardent employees are committed to solving our customers' most difficult problems-and we are committed to the well-being, personal goals, and professional development of our employee. We are "All In." We put forth our strongest effort possible to get...

  • Cloud Solutions Engineer

    2 weeks ago

    Pasco, United States Ardent Services Full time

    Why do you need to choose between doing important work and having a fulfilling life? At Ardent, we have both. Ardent employees are committed to solving our customers' most difficult problems-and we are committed to the well-being, personal goals, and professional development of our employee. We are "All In." We put forth our strongest effort possible to get...

  • Cloud Solutions Engineer

    4 weeks ago

    Pasco, United States Ardent Services Full time

    Why do you need to choose between doing important work and having a fulfilling life? At Ardent, we have both. Ardent employees are committed to solving our customers' most difficult problems-and we are committed to the well-being, personal goals, and professional development of our employee. We are "All In." We put forth our strongest effort possible to get...

  • Java Developer

    3 weeks ago

    Pasco, United States Saxon Global Full time

    Title: Java Developer Location: Remote Role Overview: Apex Systems is looking for a Senior Java Developer to join the leading financial regulatory authority, who was voted the #1 Mid-Size Technology Company to work for in the country. The Full Stack JAVA developer will be responsible for hands-on software engineering to create AND enhance several...

  • Java Developer

    3 weeks ago

    Pasco, United States Saxon Global Full time

    Title: Java Developer Location: Remote Role Overview: Apex Systems is looking for a Senior Java Developer to join the leading financial regulatory authority, who was voted the #1 Mid-Size Technology Company to work for in the country. The Full Stack JAVA developer will be responsible for hands-on software engineering to create AND enhance several...

  • Infrastructure Engineer

    2 weeks ago

    Pasco, United States Palantir Technologies Full time

    A World-Changing Company Palantir builds the world's leading software for data-driven decisions and operations. By bringing the right data to the people who need it, our platforms empower our partners to develop lifesaving drugs, forecast supply chain disruptions, locate missing children, and more. The Role Palantir has been selected as the prime...

  • Mechanical Engineer II

    1 week ago

    Pasco, Washington, United States RealStreet Full time

    Since 2001, RealStreet has connected management executives for the construction industry to contractors working on commercial and government projects. As a leading firm in the country which specializes entirely in senior level construction employment, we have cultivated long-term relationships with many of the industry's leading companies enabling us access...

  • Structural Engineer III

    1 month ago

    Pasco, United States RealStreet Full time

    Since 2001, RealStreet has connected management executives for the construction industry to contractors working on commercial and government projects. As a leading firm in the country which specializes entirely in senior level construction employment, we have cultivated long-term relationships with many of the industry's leading companies enabling us access...

  • Lead RactJS

    1 week ago

    Pasco, Washington, United States Diverse Lynx Full time

    Job Title Lead RactJS - Front End Developer Relevant Experience (in Yrs) 13-15 Years Exp Technical/Functional Skills React JS Development Responsive Web Development Java Development Experience Required Developing, implementing, and sustaining test automation processes, practices, and controls in support of application and system requirement,...

  • Technical Lead

    4 days ago

    Pasco, United States Prestige Development Group Full time

    We are a fast-growing digital consultancy in banking, payments, and fintech, leveraging a global network of top talent driven by innovation. In the past decade, we've delivered over 150 products for major clients like Visa, American Express, and Capital One. Our founders previously built Monsoon, acquired by Capital One in 2015. Responsibilities: Lead the...

  • Biomedical Engineer

    3 weeks ago

    Pasco, United States Falcon IT & Staffing Solutions Full time

    15th May, 2024Role: Biomedical Engineer / Clinical Engineer. Location: Seattle, WA. Job Description: We are seeking an experienced and highly qualified Biomedical Engineer to join our healthcare organization/Seattle VA Department. As a Biomedical Engineer, you will play a pivotal role in managing our medical equipment program and ensuring the safety,...

  • Sr.Manager, Data Platform

    1 week ago

    Pasco, United States Spire Full time

    Spire Global is a space-to-cloud analytics company that owns and operates the largest multi-purpose constellation of satellites. Its proprietary data and algorithms provide the most advanced maritime, aviation, and weather tracking in the world. In addition to its constellation, Spires data infrastructure includes a global ground station network and 24/7...

  • Senior Software/Systems Resiliency Engineer

    7 days ago

    Pasco, United States Software Engineering Institute Full time

    About the role Do you have experience delivering software engineering innovations to customers in the Department of Defense (DoD) and intelligence communities? Are you a problem solver for system and software resiliency challenges in the presence of rapidly increasing threats? The Software Solutions Division (SSD) is looking for a customer-focused technical...