Big Data Engineer
1 month ago
Malvern, PA - Hybrid
Job Description
Responsibilities
- Client's Strategy team leverages data, execution technology, and research to advance clinet's trading strategies, empowering traders to achieve the lowest trading costs, unmatched efficiency, and increased competitiveness.
- As an Engineer on the full stack team, you will be expected to help design, implement, and maintain a modern, robust, and scalable platform which will enable the TAS team to meet the increasing demands from the various trading desks.
- Proficiency in Python programming Strong expertise in SQL, Presto, HIVE, and Spark
- Experience in big data technologies such as Spark and developing distributed computing applications using PySpark
- Experience with libraries for data manipulation and analysis, such as Pandas, Polars and NumPy
- Understanding of data pipelines, ETL processes, and data warehousing concepts
- Strong experience in building and orchestrating data pipelines
- Experience in building APIs
- Experience in designing and building web environments on AWS
- Write, maintain, and execute automated unit tests using Python
- Follow Test-Driven Development (TDD) practices in all stages of software development
- Proficiency in data visualization tools and ability to create visual representations of data, particularly using Tableau
- Extensive experience with key AWS services/components including EMR, Lambda, Glue ETL, Step Functions, S3, ECS, Kinesis, IAM, RDS PostgreSQL, Dynamodb, Timeseries database, CloudWatch Events/Event Bridge, Athena, SNS, SQS, and VPC
- Proficiency in developing serverless architectures using AWS services
- Experience with both relational and NoSQL databases
- Skills in designing and implementing data models, including normalization, denormalization, and schema design
- Knowledge of data warehousing solutions like Amazon Redshift
- Strong analytical skills with the ability to troubleshoot data issues
- Good understanding of source control, unit testing, test-driven development, and CI/CD
- Ability to write clean, maintainable code and comprehend code written by others
- Strong communication skills
- Knowledge of trading and investment data
- Experience with OneTick / KDB
-
big data engineer
1 week ago
Malvern, United States Randstad Full timebig data engineer. malvern , pennsylvania posted 7 days ago job details summary $66.40 - $71.40 per hour contract bachelor degree category computer and mathematical occupations reference1074011 job details job summary: Proficiency in Python programming expertise in SQL, Presto, HIVE, and Spark Experience in big data technologies such as Spark...
-
big data engineer
7 days ago
Malvern, United States Randstad Full timebig data engineer. malvern , pennsylvania posted december 4, 2024 job details summary $66.40 - $71.40 per hour contract bachelor degree category computer and mathematical occupations reference1074011 job details job summary: Proficiency in Python programming expertise in SQL, Presto, HIVE, and Spark Experience in big data technologies...
-
Big Data
4 days ago
Malvern, United States Veridic Solutions Full timeRole : Big Data Engineer with AWS Location : Malvern, PAConsidering only those candidates who are on H1B and ready to transfer their visa to Veridic Solutions.
-
Big Data Solutions Architect
2 weeks ago
Malvern, Pennsylvania, United States Goli Tech Full timeAbout Goli TechGoli Tech is a forward-thinking organization at the forefront of innovation in the tech industry.Job OverviewWe are seeking an exceptional Big Data Solutions Architect to join our team. This role offers the opportunity to design, implement, and maintain a modern, robust, and scalable platform for our Trading Analytics & Strategy team.Key...
-
Veridic Solutions | Big Data
3 days ago
malvern, United States Veridic Solutions Full timeRole : Big Data Engineer with AWS Location : Malvern, PAConsidering only those candidates who are on H1B and ready to transfer their visa to Veridic Solutions.
-
Data Engineer
5 months ago
Malvern, United States CyberTec Full timeClient: Vanguard New Role: Data Engineer Pay Rate: $68.00/hr. on C2C) Location: Malvern, PA(Hybrid 3 days a week, must be onsite on day 1) Owns and manages the overall data strategy and architecture for Charitable (enterprise-wide) Understand the data and data needs across both SaaS applications and their on-prem legacy apps Go deep on evolving data &...
-
Data Engineer
3 weeks ago
Malvern, United States Saxon Global Full timeThis is a 12 month contract with Vanguard. 2 days onsite in Malvern PA and 3 days remote. All visa. Need a write up from the candidate how they fit in the role. Take candidates from Data Quad. They have good Data Engineers. MUST have Education must have month and year they graduated. Job Description As a Data Engineer, you'll build sophisticated solutions...
-
Investment Data Engineer
5 days ago
Malvern, Pennsylvania, United States Vanguard Full timeAbout the RoleWe are looking for an Investment Data Engineer with a strong background in software development and data management. Your primary responsibility will be to design, develop, and implement data-driven solutions that support Vanguard's investments domain. This is an exciting opportunity to work on complex projects, collaborate with...
-
Cloud Data Engineer Lead
3 weeks ago
Malvern, Pennsylvania, United States Randstad Digital Full timeRandstad Digital is looking for a Cloud Data Engineer Lead who can drive the development of scalable and efficient data pipelines using AWS technologies. As a lead, you'll collaborate with cross-functional teams to design, build, and maintain data architectures that meet business needs.You'll develop and implement data pipelines that integrate with various...
-
Tableau Data Engineer
3 weeks ago
Malvern, Pennsylvania, United States Syntricate Technologies Full timeJob Summary Syntricate Technologies is seeking an experienced Tableau Data Engineer to join our team. As a key member of our technical staff, you will be responsible for administering our Tableau SaaS infrastructure, developing dashboards, and providing expert-level support to our clients.About the Role We are looking for a skilled professional with...
-
Data Engineer- Malvern, PA
2 weeks ago
Malvern, United States Georgia IT Inc Full timeJob Title: Data Engineer Location: CA, PA Duration: Contract Rate: DOE(USC. GC, GCEAD and L2EADS are only preferred)Candidate Should be a local to Malvern or Charlotte, Relocation wont work on thisDuties and responsibilities: Provides advanced data solutions by using software to process, store, and serve data to others. Tests data quality and optimizes data...
-
Data Storage Solutions Engineer
2 weeks ago
Malvern, Iowa, United States Diverse Lynx Full timeData Storage Solutions EngineerAbout the RoleWe are seeking a skilled Data Storage Solutions Engineer to design and implement scalable and efficient data storage solutions using Snowflake. The ideal candidate will have experience with AWS Cloud, S3, EC2, Lambda, and other related technologies.Key Responsibilities* Design, implement, and optimize data storage...
-
AWS Data Engineer
1 month ago
malvern, United States Pronix Inc Full timeONLY W2 NO C2CPosition: Aws Data EngineerLocation: Malvern, PADuration: Long TermJOB DETAILS- 10+ years of experience (required)- Python/PySpark- Strong SQL- Serverless AWS: Lambda, Step Functions, EMR, Glue, RDS, etc.
-
AWS Data Engineer
1 month ago
Malvern, United States Pronix Inc Full timeONLY W2 NO C2CPosition: Aws Data EngineerLocation: Malvern, PADuration: Long TermJOB DETAILS- 10+ years of experience (required)- Python/PySpark- Strong SQL- Serverless AWS: Lambda, Step Functions, EMR, Glue, RDS, etc.
-
Senior Data Engineer
4 weeks ago
Malvern, United States Hirextra -World's First Staffing Aggregator Full timeData Engineer- 10+ years.Briefly describe the duties and responsibilitiesResponsibilitiesAnalyze complex business problems, track down required information, document requirements, and develop technical solutions.Strong analytical and problem-solving skills and the tenacity to overcome technical challenges.Troubleshoot and optimize new and existing ETL...
-
Cloud Data Engineer
3 weeks ago
Malvern, Pennsylvania, United States Goli Tech Full timeAt Goli Tech, we are seeking a highly skilled Cloud Data Engineer to join our team. The ideal candidate will have experience in designing, building, and maintaining large-scale data processing systems on cloud platforms.About the RoleThis is a fantastic opportunity for a talented individual to work with cutting-edge technology and contribute to the...
-
Sr Data Engineer
3 weeks ago
Malvern, United States Randstad Digital Full timeExclusive rolesLooking to interview now and have starts on 1/6/20259 month project Onsite Tues-Thurs Role Summary:The Data Engineer will be responsible for building and maintaining the data pipelines that process data from CSV files through various transformation stages, following the Architect's data model designs. This role requires hands-on experience...
-
Sr Data Engineer
1 month ago
Malvern, United States Randstad Digital Full timeExclusive rolesLooking to interview now and have starts on 1/6/20259 month project Onsite Tues-Thurs Role Summary:The Data Engineer will be responsible for building and maintaining the data pipelines that process data from CSV files through various transformation stages, following the Architect’s data model designs. This role requires hands-on experience...
-
Operating Engineer-Technician
1 week ago
Malvern, United States Pioneer Data Systems Full timeOur client, a world-leading Pharmaceutical Company in Malvern, PA is currently looking for an Operating Engineer-Technician to join their expanding team.Apply (by clicking the relevant button) after checking through all the related job information below.Position Details:Job Title: Operating Engineer-Technician (Boiler, Cooler, etc.) / Pharma...
-
Data Analyst
2 months ago
Malvern, United States VDart Full timeRole: Data analyst Location: Malvern PA 19355 Hybrid 3 days a week in Malvern, PA Duration: 6 months About the Role: (SQL, excel, ETL, SAP Business Object Web Intelligence) They are implementing a new system (Adacom). During the initial mapping of data fields that are required, they found a bunch of missing data and clean up that is needed anyone who has...