ETL / Spark Developer

4 weeks ago

West McLean, United States Petadata Full time
Job DescriptionJob Description

POSITION: ETL / Spark Developer

LOCATION: McLean, VA. (Hybrid)

Experience: 10+ years

Work type: Fulltime (W2 / C2C)

PETADATA is currently looking to hire for the ETL / Spark Developer role for one of their clients.

Roles & Responsibilities:

  • Should have experience with both streaming and batch workflows will be essential in ensuring the efficient flow and processing of data to support our clients.

  • Collaborate with cross-functional teams to understand data requirements and design robust data architecture solutions.

  • Design, develop, and implement scalable data processing solutions using Apache Spark

  • Ability to organize and to keep the projects well-arranged and structured.

  • Ensure data quality, integrity, and consistency throughout the ETL pipeline.

  • Integrate data from different systems and sources to provide a unified view for analytical purposes.

  • Collaborate with data analysts to implement solutions that meet their data integration needs.

  • Design and implement streaming workflows using PySpark Streaming or other relevant technologies.

  • Develop batch processing workflows for large-scale data processing and analysis.

  • Analyze the business requirement to determine the volume of data extracted from different sources, data models, to ensure the quality of the data involved.

  • Should be able to figure out the best storage medium required for the data warehouse needed.

  • Identify the data storage needs to determine the amount of data to deal with the company's requirements.

  • Must ensure the data quality that everything is in place at the transformation stage to eliminate errors and fix unstructured and unorganized data extracted.

  • Responsible for ensuring that the data is loaded into the warehouse system and meets the business needs and standards.

  • Should be responsible for data flow validation, creating and building a secured database warehouse that meets a given company's needs and standards.

  • Must be responsible for determining the storage needs of a business and the volume of data involved.

Required skills:

  • Should have 10+ years of experience in implementing ETL processes to extract, transform, and load data from various sources to ensure data quality, integrity, and consistency throughout the ETL pipeline.

  • Must be expertise in Python, PySpark, ETL processes, CI/CD (Jenkins or GitHub).

  • Experienced in Python and PySpark to develop efficient data processing and analysis scripts.

  • Optimize code for performance and scalability, keeping up-to-date with the latest industry best practices.

  • Must load data and be proficient in valuable technical skills such as SQL, JAVA, XML, and DOM, among others.

  • Extensive knowledge and experience with Spark and its technologies.

  • Hands-on Experience with Apache Spark framework, including the Spark SQL module for querying databases.

  • Good knowledge on data analysis, design, and programming skills such as JavaScript, SQL and XML, and DOM.

  • Familiar with various coding languages used in web development, including HTML, CSS, JavaScript, Python, Java, Scala, or R proficiency.

  • Should have experience in writing clean code that's free of bugs and reproducible by other developers.

  • Experience in managing SQL databases and organizing big data.

  • Hands-on experience with ETL such as MS SQL, SSIS (Server Integration Services), Python / Perl, Oracle, SQL Server/ MySQL.

  • Solid understanding of Data warehousing schemes, Dimensional modeling, and implementing data storage solutions to support efficient data retrieval and analysis.

  • Must be an expert in debugging ETL processes, optimizing data flows, and ensuring that the data pipeline is robust and error-free.

  • Good knowledge of Verbal and communication skills.

Educational Qualification:

Bachelor's/ Master's degree in Computer Science, Engineering, or a related field.

We offer a professional work environment and are given every opportunity to grow in the Information technology world.

Note:

Candidates required to attend Phone/Video Call / In person interviews and after Selection of candidate (He/She) should go through all background checks on Education and Experience.

Please email your resume to: joelp@petadata.co

After carefully reviewing your experience and skills one of our HR team members will contact you on the next steps.

  • ETL / Spark Developer

    2 weeks ago

    West McLean, United States Petadata Full time

    Job DescriptionJob DescriptionPOSITION: ETL / Spark DeveloperLOCATION: McLean, VA. (Hybrid)Experience: 10+ yearsWork type: Fulltime (W2 / C2C)PETADATA is currently looking to hire for the ETL / Spark Developer role for one of their clients.Roles & Responsibilities: Should have experience with both streaming and batch workflows will be essential in ensuring...

  • ETL / Spark Developer

    1 week ago

    West McLean, United States Petadata Full time

    Job DescriptionJob DescriptionPOSITION: ETL / Spark DeveloperLOCATION: McLean, VA. (Hybrid)Experience: 10+ yearsWork type: Fulltime (W2 / C2C)PETADATA is currently looking to hire for the ETL / Spark Developer role for one of their clients.Roles & Responsibilities: Should have experience with both streaming and batch workflows will be essential in ensuring...

  • ETL / Spark Developer

    1 week ago

    West McLean, United States Petadata Full time

    Job DescriptionJob DescriptionPOSITION: ETL / Spark DeveloperLOCATION: McLean, VA. (Hybrid)Experience: 10+ yearsWork type: Fulltime (W2 / C2C)PETADATA is currently looking to hire for the ETL / Spark Developer role for one of their clients.Roles & Responsibilities: Should have experience with both streaming and batch workflows will be essential in ensuring...

  • ETL / Spark Developer

    4 weeks ago

    Mclean, United States PETADATA Full time

    POSITION: ETL / Spark Developer LOCATION: McLean, VA. (Hybrid) Experience: 10+ years Work type: Fulltime (W2 / C2C) PETADATA is currently looking to hire for the ETL / Spark Developer role for one of their clients. Roles & Responsibilities: Should have experience with both streaming and batch workflows will be essential in ensuring the efficient flow and...

  • Etl Developer

    4 days ago

    McLean, United States FourteenPosts Full time

    Job Summary: We are seeking an experienced ETL Developer to join our team. As an ETL Developer, you will be responsible for designing, developing, and implementing data integration solutions. You will work closely with cross-functional teams to analyze business requirements and ensure data accuracy and integrity. **Responsibilities**: - Design, develop,...

  • ETL Developer

    2 weeks ago

    West McLean, United States Forsys Inc Full time

    Job DescriptionJob DescriptionPosition 1:Role:- ETL Developer with Attunity | AR- 264967Rate: $65/hour Employment ContractLocation-Mclean VA Day 1 onsite in a hybrid setup. (Local Only)Job DescriptionMandatory skills:- ETL, Attunity, AWS, MongoDB, and Atlassian set of tools.8+ years of Strong ETL & Data warehousing conceptsStrong Attunity experienceDB2 and...

  • ETL Developer

    7 days ago

    West McLean, United States Forsys Inc Full time

    Job DescriptionJob DescriptionPosition 1:Role:- ETL Developer with Attunity | AR- 264967Rate: $65/hour Employment ContractLocation-Mclean VA Day 1 onsite in a hybrid setup. (Local Only)Job DescriptionMandatory skills:- ETL, Attunity, AWS, MongoDB, and Atlassian set of tools.8+ years of Strong ETL & Data warehousing conceptsStrong Attunity experienceDB2 and...

  • ETL Developer

    1 week ago

    West McLean, United States Forsys Inc Full time

    Job DescriptionJob DescriptionPosition 1:Role:- ETL Developer with Attunity | AR- 264967Rate: $65/hour Employment ContractLocation-Mclean VA Day 1 onsite in a hybrid setup. (Local Only)Job DescriptionMandatory skills:- ETL, Attunity, AWS, MongoDB, and Atlassian set of tools.8+ years of Strong ETL & Data warehousing conceptsStrong Attunity experienceDB2 and...

  • ETL Developer

    3 weeks ago

    McLean, United States Rishabh RPO Full time

    Job DescriptionJob DescriptionRole:- ETL Developer with Attunity Location-Mclean VA Job DescriptionMandatory skills:- ETL, Attunity, AWS, MongoDB and Atlassian set of tools.8+ years of Strong ETL & Data warehousing conceptsStrong Attunity experienceDB2 and MongoDB database working experience.AWS service utilization experience with services like EC2, EMR, S3,...

  • Etl Developer

    3 weeks ago

    McLean, United States Syms Strategic Group, LLC (SSG) Full time

    **Syms Strategic Group (SSG) **is seeking a talented **Extract, Transform, and Load (ETL) Developer** **Department**:Veterans Affairs **Location**:On-site, Washington, DC **Type**:Full Time **Min. Experience**:Experienced **Security Clearance Level**:Public Trust (MBI) **Salary Range**:$75,995 - $89,000 **Military Veterans are highly encouraged to...

  • Sr. ETL Administrator

    4 weeks ago

    McLean, United States Metric5 Full time

    Job DescriptionJob DescriptionDescriptionMetric 5 is currently seeking an Application Administrator to work within a team that provides Data Warehouse and Business Intelligence services to our government customer using Agile scrum processes. You will work with varying data sources with different schemas and data elements to ultimately produce an effective...

  • Sr. ETL Administrator

    2 weeks ago

    McLean, United States Metric5 Full time

    Job DescriptionJob DescriptionDescriptionMetric 5 is currently seeking an Application Administrator to work within a team that provides Data Warehouse and Business Intelligence services to our government customer using Agile scrum processes. You will work with varying data sources with different schemas and data elements to ultimately produce an effective...

  • Sr. ETL Administrator

    1 week ago

    McLean, United States Metric5 Full time

    Job DescriptionJob DescriptionDescriptionMetric 5 is currently seeking an Application Administrator to work within a team that provides Data Warehouse and Business Intelligence services to our government customer using Agile scrum processes. You will work with varying data sources with different schemas and data elements to ultimately produce an effective...

  • Sr. ETL Administrator

    7 days ago

    McLean, United States Metric5 Full time

    Job DescriptionJob DescriptionDescriptionMetric 5 is currently seeking an Application Administrator to work within a team that provides Data Warehouse and Business Intelligence services to our government customer using Agile scrum processes. You will work with varying data sources with different schemas and data elements to ultimately produce an effective...

  • ETL Developer with Pyspark

    3 weeks ago

    McLean, United States Excelon Solutions Full time

    Job DescriptionJob DescriptionJob Description Your expertise in Python, PySpark, ETL processes and experience with both streaming and batch workflows will be essential in ensuring the efficient flow and processing of data to support our clients.Collaborate with cross-functional teams to understand data requirements and design robust data architecture...

  • ETL Senior Developer

    3 weeks ago

    McLean, United States Diverse Lynx Full time

    ETL , Informatica, DBTIICS & Attunity (Qlick) - ETL Senior Developer Primary : IICS , Attunity (Qlick) , Mongo DB, AWS services - All should be hands-on Additional : ,Snowflake ,Control-M ,CI /CD Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All...

  • Senior Spark

    1 month ago

    McLean, United States Mavens Guild Full time

    What we would like to see: In a senior developer role, you will design and build data flow and data integration processes to enhance loss prevention technologies for a leading financing firm. Drawing from your vast hands-on experience in Python (PySpark), Spark, REST, Java, and Scala, you will develop, test, and deploy end-to-end solutions using full-stack...

  • Senior Spark

    2 weeks ago

    McLean, United States Mavens Guild Full time

    What we would like to see: In a senior developer role, you will design and build data flow and data integration processes to enhance loss prevention technologies for a leading financing firm. Drawing from your vast hands-on experience in Python (PySpark), Spark, REST, Java, and Scala, you will develop, test, and deploy end-to-end solutions using full-stack...

  • Senior Spark

    3 hours ago

    McLean, United States Mavens Guild Full time

    What we would like to see: In a senior developer role, you will design and build data flow and data integration processes to enhance loss prevention technologies for a leading financing firm. Drawing from your vast hands-on experience in Python (PySpark), Spark, REST, Java, and Scala, you will develop, test, and deploy end-to-end solutions using full-stack...

  • Developer-ETL/Informatica Senior

    2 weeks ago

    McLean, United States Saxon Global Full time

    Developer-ETL/Informatica Senior Experience with Informatica PowerCenter (v 9.x and higher). Experience with Informatica Cloud / IICS. To ensure success as an ETL Developer, you should have extensive knowledge of coding languages and data warehouse architecture, and be able to clearly communicate your ideas to the team. ETL Developer Responsibilities: •...