Bigdata / Pyspark Developer
2 weeks ago
Experience in building SparkStreaming process. Proficient in understanding distributed computing principles. Experience in managing Hadoop cluster with all services.Proficiency with Hadoop MapReduce HDFS Pig Hive and Impala. Experience with Nosql Databases and Messaging systems like Kafka. Designing building installing configuring and supporting Hadoop Perform analysis of vast data stores. Good understanding of cloud technology. Must have strong technical experience in Design Mapping specifications HLD LLD. Must have the ability to relate to both business and technical members of the team and possess excellent communication skills. Leverage internal tools and SDKs, utilize AWS services such as S3, Athena, and Glue, and integrate with our internal Archival Service Platform for efficient data purging. Lead the integration efforts with the internal Archival Service Platform for seamless data purging and lifecycle management. Collaborate with the data engineering team to continuously improve data integration pipelines, ensuring adaptability to evolving business needs.
Performed database health checks and tuned the databases using Teradata Manager.
Develop and maintain data platforms using Python
Work with AWS and Big Data, design and implement data pipelines, and ensure data quality and integrity
Collaborate with crossfunctional teams to understand data requirements and design solutions that meet business needs
Implement and manage agents for monitoring, logging, and automation within AWS environments
Handling migration from PySpark to AWS
Experience in building SparkStreaming process. Proficient in understanding distributed computing principles. Experience in managing Hadoop cluster with all services.Proficiency with Hadoop MapReduce HDFS Pig Hive and Impala. Experience with Nosql Databases and Messaging systems like Kafka. Designing building installing configuring and supporting Hadoop Perform analysis of vast data stores. Good understanding of cloud technology. Must have strong technical experience in Design Mapping specifications HLD LLD. Must have the ability to relate to both business and technical members of the team and possess excellent communication skills. Leverage internal tools and SDKs, utilize AWS services such as S3, Athena, and Glue, and integrate with our internal Archival Service Platform for efficient data purging. Lead the integration efforts with the internal Archival Service Platform for seamless data purging and lifecycle management. Collaborate with the data engineering team to continuously improve data integration pipelines, ensuring adaptability to evolving business needs.
Performed database health checks and tuned the databases using Teradata Manager.
Develop and maintain data platforms using Python
Work with AWS and Big Data, design and implement data pipelines, and ensure data quality and integrity
Collaborate with crossfunctional teams to understand data requirements and design solutions that meet business needs
Implement and manage agents for monitoring, logging, and automation within AWS environments
Handling migration from PySpark to AWS
#J-18808-Ljbffr
-
Pyspark
2 weeks ago
Plano, United States E-Solutions INC Full timeJob DescriptionJob DescriptionJob Title: Pyspark Location: Plano, TXFulltime/Contract(W2)Hands-on Pyspark SME with multiple project experience with Data planforms comprising of Hadoop, Teradata Data Warehouse, Ab Initio, Informatica, Java Spark (DPL), SSIS, AWS Lake Formation (S3), SnowflakeAbility to design, build and unit test applications on Spark...
-
Data Engineer
4 weeks ago
Plano, United States Promantus Inc Full timeData Engineer Location Plano Texas (Remote until COVID) Duration Long-term Job Description Develops Python/PySpark HQL queries. Develops new data models as necessary with Lead Backend Developer and Architect. Performs data visualization and analysis. Produces data samples for UI/UX Designer. Communicates insights to Architect Lead Backend Developer ...
-
Data Engineer
4 weeks ago
Plano, Texas, United States Promantus Inc Full timeData Engineer Location Plano Texas (Remote until COVID) Duration Long-term Job Description Develops Python/PySpark HQL queries. Develops new data models as necessary with Lead Backend Developer and Architect. Performs data visualization and analysis. Produces data samples for UI/UX Designer. Communicates insights to Architect Lead Backend Developer Lead...
-
Mid-Senior Data Engineer
6 days ago
Plano, United States Ascentt Full timeJob Summary: We are seeking an experienced Mid-Senior Data Engineer (PySpark Engineer) to join our team of data professionals. In this role, you will collaborate closely with Data Scientists to prepare and transform large-scale datasets using PySpark, a popular open-source Python library for Apache Spark. You will play a crucial role in enabling effective...
-
Mid-Senior Data Engineer
6 days ago
Plano, United States Ascentt Full timeJob Summary: We are seeking an experienced Mid-Senior Data Engineer (PySpark Engineer) to join our team of data professionals. In this role, you will collaborate closely with Data Scientists to prepare and transform large-scale datasets using PySpark, a popular open-source Python library for Apache Spark. You will play a crucial role in enabling effective...
-
Mid-Senior Data Engineer
6 days ago
Plano, United States Ascentt Full timeJob Summary: We are seeking an experienced Mid-Senior Data Engineer (PySpark Engineer) to join our team of data professionals. In this role, you will collaborate closely with Data Scientists to prepare and transform large-scale datasets using PySpark, a popular open-source Python library for Apache Spark. You will play a crucial role in enabling effective...
-
Python Developer
1 week ago
Plano, United States Avacend Inc. Full timeResponsibilities2-4 years of experience developing Data engineering, and ad-hoc transformation of unstructured raw dataUse of orchestration toolsDesign, build, and maintain workflows/pipelines to process a continuous stream of data with experience in end-to-end design and build process of Near-Real-Time and Batch Data Pipelines.Expected to work closely with...
-
AWS and Python and Java
1 week ago
Plano, United States Diverse Lynx Full timeRole: AWS and Python and Java (Microservices) Developer Location - Plano TX, Northern VA (3 Days Office 2 Days remote) Experience: 10+ Year Duration: Long Term Mandatory Skills: AWS and Python and Java (Microservices) Job Description: • Strong working Python, AWS experience, web Application, UI experience (JSP, Angular), Java, Spring, Springboot, Databases...
-
AWS and Python and Java
3 days ago
Plano, United States Diverse Lynx Full timeRole: AWS and Python and Java (Microservices) Developer Location - Plano TX, Northern VA (3 Days Office 2 Days remote) Experience: 10+ Year Duration: Long Term Mandatory Skills: AWS and Python and Java (Microservices) Job Description: • Strong working Python, AWS experience, web Application, UI experience (JSP, Angular), Java, Spring, Springboot, Databases...
-
Software Engineer/Developer
1 week ago
Plano, United States Teleworld Solutions Full timeOverviewTeleWorld Solutions is seeking a experienced Software engineer with focus on Data Engineering, ETL processes, preferably with exposure to both batch and streaming data. The candidate should have familiarity with use of Databases and DataLake infrastructure and associated tools for ingestion, transformation and efficient querying across distributed...
-
Plano, United States Avacend Full timeJob DescriptionJob DescriptionOur client is looking for a Python Developer with Data Engineering @ Plano, TX. I am reaching you for a new job opportunity. Please let me know if you are interested, please share your updated resume and the following information.Responsibilities2-4 years of experience developing Data engineering, and ad-hoc transformation of...
-
ETL/ Data Warehousing Developer
4 days ago
Plano, United States AA SOFTWARE & NETWORKING PRIVATE LIMITED Full timeJob Title: ETL/Data Warehousing DeveloperJob Location: Plano, TXJob Type: W2 OnlyJob responsibilities:Executes software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems.Creates secure and high-quality production code and maintains...
-
ETL/ Data Warehousing Developer
4 days ago
Plano, United States AA SOFTWARE & NETWORKING PRIVATE LIMITED Full timeJob Title: ETL/Data Warehousing DeveloperJob Location: Plano, TXJob Type: W2 OnlyJob responsibilities:Executes software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems.Creates secure and high-quality production code and maintains...
-
ETL/ Data Warehousing Developer
2 days ago
Plano, United States AA SOFTWARE & NETWORKING PRIVATE LIMITED Full timeJob Title: ETL/Data Warehousing DeveloperJob Location: Plano, TXJob Type: W2 OnlyJob responsibilities:Executes software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems.Creates secure and high-quality production code and maintains...
-
React.JS Developer
4 weeks ago
Plano, United States Pinnacle Group, Inc. Full timeRole: React DeveloperLocation: Plano, TX- Hybrid 2/3 days in officeDuration: 6/12 Month Contract (Possible Extension or Hire)W2 ONLY NO C2CUSC/GC onlyMust have:Must Have: (1) React JS (2) AWS red shift and data bricks (3) PySparkReact and AWS CloudJob Description:Expert in React JS and design technique as well as experience working across large environments...
-
React.JS Developer
4 weeks ago
Plano, United States Pinnacle Group, Inc. Full timeRole: React DeveloperLocation: Plano, TX- Hybrid 2/3 days in officeDuration: 6/12 Month Contract (Possible Extension or Hire)W2 ONLY NO C2CUSC/GC onlyMust have:Must Have: (1) React JS (2) AWS red shift and data bricks (3) PySparkReact and AWS CloudJob Description:Expert in React JS and design technique as well as experience working across large environments...
-
Business Analyst
3 days ago
Plano, United States Tata Consultancy Services Full timeJob Title Data Engineering Tech LeadRelevant Experience (in Yrs) 10 -14 yearsMust Have Technical/Functional Skills1. Strong communication2. Has closely worked with Business users in understanding their requirements and processes3. Working knowledge of python, Pyspark , ETLs , Data loading Strategies - Ability to code and Troubleshoot4. Solid understanding of...
-
Sr. Developer- on site
4 weeks ago
Plano, United States Cognizant Technology Solutions Full timeTechnical LeadQualification:Bachelors in science , engineering or equivalentResponsibility:Project Planning and Setup: Understand the project scope, identify activities/ tasks, task level estimates, schedule, dependencies, risks and provide inputs to Module Lead for review. Provide inputs to testing strategy, configuration, deployment, hardware/software...
-
Python Software Engineer
4 weeks ago
Plano, United States JPMorgan Chase & Co. Full timeJob responsibilities Executes software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problemsCreates secure and high-quality production code and maintains algorithms that run synchronously with appropriate systemsProduces architecture...
-
Python Software Engineer
4 weeks ago
Plano, United States JPMorgan Chase & Co. Full timeJob responsibilities Executes software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problemsCreates secure and high-quality production code and maintains algorithms that run synchronously with appropriate systemsProduces architecture...