Spark Tech Lead with Big Data

1 week ago

McLean, United States Diverse Lynx Full time
Job Description:

Role: Spark Tech Lead with Big Data Experience

Location: McLean, VA (Day-1 Onsite| 3 Days Onsite & 2 Days Remote)

Hiring Mode: Long Term Contract | Full Time

NOTE: We are looking for strong Spark Lead with Big Data experience, Less Java experience will also work. But candidate must be senior level.

Mandatory Skills:
  • 10+ years of experience in solution, design and development of applications using Java 8+/J2EE, Spring, Spring-Boot, MicroServices, RESTful Services and with experience in Big Data and with experience working in heavy data background needed.
  • Develop, program, and maintain applications using the Apache Spark open-source framework
  • Work with different aspects of the Spark ecosystem, including Spark SQL, DataFrames, Datasets, and streaming
  • Spark Developer must have strong programming skills in Java, Scala, or Python
  • Familiar with big data processing tools and techniques
  • Proven experience as a Spark Developer or a related role
  • Strong programming skills in Java, Scala, or Python
  • Experience with the Hadoop ecosystem
  • Good understanding of distributed systems
  • Experience with streaming data platforms
  • Must have strong experience in Big Data and with experience working in heavy data background needed
  • Must be strong in Cloud AWS event-based architecture, Kubernetes, ELK (Elasticsearch, Logstash & Kibana)
  • Must have excellent experience in designing and Implementing cloud-based solutions in various AWS Services (: S3, Lambda, Step Function, AMQ, SNS, SQS, CloudWatch Events, etc.)
  • Must be well experienced in design and development of Microservice using Spring-Boot and REST API and with GraphQL
  • Must have solid knowledge and experience in NoSQL (MongoDB)
  • Good knowledge and experience in any Queue based implementations
  • Strong knowledge/experience in ORM Framework - JPA / Hibernate
  • Good knowledge in technical concepts Security, Transaction, Monitoring, Performance
  • Should we well versed with TDD/ATDD
  • Strong experience in DevOps tool chain (Jenkins, Artifactory, Ansible/Chef/Puppet/Spinnaker, Maven/Gradle, Atlassian Tool suite)
  • Very Good knowledge and experience in Non-Functional (Technical) Requirements like Security, Transaction, Performance, etc.
  • Excellent analytical and problem-solving skills.
Nice to have:
  • Experience in Experience with OAuth implementation using Ping Identity
  • Experience in API Management (Apigee) or Service Mesh (Istio)
  • Good knowledge and experience in Queue/Topic (Active-MQ) based implementations.
  • Good knowledge and experience in Scheduler and Batch Jobs
  • Experience with scripting languages using Unix
  • Preferably certified in AWS.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

  • Java Tech Lead

    1 week ago

    McLean, United States Diverse Lynx Full time

    Title: Java Tech Lead (Spark and Python expertise) Location: McLean, VA Type : Contract Job Description: Job description: Mandatory: 10+ years of experience in solution, design and development of applications using Java 8+/J2EE, Spring, Spring-Boot, Micro Services, RESTful Services and with experience in Big Data and with experience working...

  • Big Data Engineer

    1 week ago

    McLean, United States CareerBuilder Full time

    Description: Our client is looking for a Big Data Engineer to join them on the modernization and organization of their data as they go through wide-scale upgrades of their current legacy systems. We are seeking an entry level-mid level engineer located in/around the Mclean, VA area. Requirements: 1+ years of hands on experience in IICS or Informatica Power...

  • Senior Data Engineer

    7 days ago

    McLean, United States The Depository Trust & Clearing Corporation Full time

    Job Tittle: Data Engineer (Spark Exp.) Duration: Full-time Location: Mclean, VA/TX/FL/NJ (Hybrid 3 days onsite, Rest days remote) Must Have: Spark Your Primary Responsibilities: Work with large, complex data sets and high throughput data pipelines that meet business requirements. Build the infrastructure required for optimal extraction, transformation, and...

  • Senior Data Engineer

    Found in: Appcast Linkedin GBL C2 - 2 weeks ago

    McLean, United States The Depository Trust & Clearing Corporation (DTCC) Full time

    Job Tittle: Data Engineer (Spark Exp.)Duration: Full-timeLocation: Mclean, VA/TX/FL/NJ (Hybrid 3 days onsite, Rest days remote)Must Have: SparkYour Primary Responsibilities:Work with large, complex data sets and high throughput data pipelines that meet business requirements. Build the infrastructure required for optimal extraction, transformation, and...

  • Senior Data Engineer

    Found in: Appcast US C2 - 2 weeks ago

    McLean, United States The Depository Trust & Clearing Corporation (DTCC) Full time

    Job Tittle: Data Engineer (Spark Exp.)Duration: Full-timeLocation: Mclean, VA/TX/FL/NJ (Hybrid 3 days onsite, Rest days remote)Must Have: SparkYour Primary Responsibilities:Work with large, complex data sets and high throughput data pipelines that meet business requirements. Build the infrastructure required for optimal extraction, transformation, and...

  • Data Architect

    Found in: beBee S US - 2 weeks ago

    Mclean, United States Arkhya Tech Inc. Full time

    Role Data ArchitectLocation McLean VARequired Skill -This team is on an Enterprise team supporting use cases across multiple lines of business; wide diversity of type of data architecting needs. Main tech stack and skills – Sr. Data Architect who has a lot of modeling experience in warehousing and OLAP and data consumption space especially from...

  • Data Engineer.(W2)

    2 weeks ago

    McLean, United States Dice Full time

    Dice is the leading career destination for tech experts at every stage of their careers. Our client, System Edge (USA) L.L.C., is seeking the following. Apply via Dice today! Position: Data Engineer.(W2) Client : Capital One. Location:: McLean, VA (3 Days Onsite 2 Days Remote) Duration : Long Term. Visa: /EAD. Skills: Data Engineer - Python, Spark/PySpark,...

  • Hybrid Data Engineer with Security Clearance

    Found in: Dice One Red US C2 - 1 week ago

    McLean, United States ALTA IT Services Full time

    ALTA IT Services has a permanent direct hire opportunity for a Data Engineer to support a federal end-customer. Data Engineer McLean, VA, Crystal City, VA, Mark Center or Pentagon (Hybrid, 2-3 Days On-Site, rotating location)Permanent/Direct Hire Clearance – Active Secret Clearance or Interim Secret Required Our client is seeking an experienced Data...

  • Cloud Data Architect

    1 day ago

    McLean, United States Lorven technologies Full time

    Job DescriptionJob DescriptionTitle: Cloud data Architect Location: MClean, VA - hybrid Duration: Contract Mandatory Skills:Skill 1 Data Engineer 10+ years Skill 2 Data Architecture 3+ years Skill 3 - Azure Data Bricks - 5+ years Skill 4 - Azure Data Lake - 3+ yearsQualificationsMinimum of 10 years of experience in a Data Engineering role, supporting...

  • Databricks Data Engineer with Security Clearance

    Found in: Dice One Red US C2 - 1 week ago

    McLean, United States Square Peg Technologies Full time

    Position Requires a Top Secret (TS/SCI) Clearance with a Polygraph. The Mission: Square Peg is seeking a Data Engineer to be responsible for designing, implementing, and maintaining data pipelines for our customers. The Data Engineer will develop automated pipelines for a variety of data types and datasets. Using Databricks, which was named to both the Magic...

  • Sr Data Engineer

    1 week ago

    McLean, United States Saxon Global Full time

    Sr Data Engineer | VA, TX, CA, NY, IL, GA | Freddie Mac is building a culture based on modern software delivery practices to support the secondary mortgage market through rapid technology and best practice innovation. DATA ENGINEER •Must have 4-5 years of experience. •Candidates must be local or willing to relocate at their own expense. •Option...

  • Data Protection Engineer

    3 days ago

    McLean, United States US Tech Solutions Full time

    *Job Title: Data Protection Engineer *Location: McLean, VA (22102) *Duration: 12 months contract, Full-Time * Employment Type: W-2 Job Description: · Experience in database and folder level security (encryption and data masking). · This position will be supporting the Applied Cryptography team within the Information Security Engineering department...

  • Databricks Data Scientist with Security Clearance

    Found in: Dice One Red US C2 - 1 week ago

    McLean, United States Square Peg Technologies Full time

    Position Requires a Top Secret (TS/SCI) Clearance with a Polygraph. Location: Northern Virginia The Mission: Square Peg is seeking a Data Scientist who can turn structure and unstructured data sets into useful information to solve challenges. You'll work closely with the customer and stakeholders to optimize business processes through data engineering and...

  • Data Architect with Cloud strong experience

    8 hours ago

    McLean, United States Smart IT Frame Full time

    Experience Level: 12 to 14 yearsThe Cloud Data Architect will be a key contributor to designing, evolving, and optimizing our companys cloud-based data architecture. This role requires a strong background in data engineering, hands-on experience building cloud data solutions, and a talent for communicating complex designs through clear diagrams and...

  • Cloud Data Architect(PySpark)

    5 days ago

    McLean, United States Smart IT Frame LLC Full time

    Hello Professionals,We are looking for a Cloud Data Architect at McLean, VA for a contract opportunity. Please find the below job description and apply for the job if you are interested.Title: Cloud Data Architect Location: McLean, VA (Hybrid)Mode: Long term ContractMandatory: PySpark, Spark, Azure databricksJob Description: The Cloud Data Architect will be...

  • Cloud Data Architect(PySpark)

    Found in: Appcast US C2 - 6 days ago

    McLean, United States Smart IT Frame LLC Full time

    Hello Professionals,We are looking for a Cloud Data Architect at McLean, VA for a contract opportunity. Please find the below job description and apply for the job if you are interested.Title: Cloud Data Architect Location: McLean, VA (Hybrid)Mode: Long term ContractMandatory: PySpark, Spark, Azure databricksJob Description: The Cloud Data Architect will be...

  • Cloud Data Architect(PySpark)

    Found in: Appcast Linkedin GBL C2 - 5 days ago

    McLean, United States Smart IT Frame LLC Full time

    Hello Professionals,We are looking for a Cloud Data Architect at McLean, VA for a contract opportunity. Please find the below job description and apply for the job if you are interested.Title: Cloud Data Architect Location: McLean, VA (Hybrid)Mode: Long term ContractMandatory: PySpark, Spark, Azure databricksJob Description: The Cloud Data Architect will be...

  • Data Architect with Cloud strong experience

    Found in: Appcast Linkedin GBL C2 - 5 days ago

    McLean, United States Smart IT Frame LLC Full time

    Experience Level: 12 to 14 yearsThe Cloud Data Architect will be a key contributor to designing, evolving, and optimizing our company’s cloud-based data architecture. This role requires a strong background in data engineering, hands-on experience building cloud data solutions, and a talent for communicating complex designs through clear diagrams and...

  • Data Architect with Cloud strong experience

    Found in: Appcast US C2 - 6 days ago

    McLean, United States Smart IT Frame LLC Full time

    Experience Level: 12 to 14 yearsThe Cloud Data Architect will be a key contributor to designing, evolving, and optimizing our company’s cloud-based data architecture. This role requires a strong background in data engineering, hands-on experience building cloud data solutions, and a talent for communicating complex designs through clear diagrams and...

  • Data Architect with Cloud strong experience

    4 days ago

    McLean, United States Smart IT Frame Full time

    Experience Level: 12 to 14 years The Cloud Data Architect will be a key contributor to designing, evolving, and optimizing our companys cloud-based data architecture. This role requires a strong background in data engineering, hands-on experience building cloud data solutions, and a talent for communicating complex designs through clear diagrams and...