Data Engineer

4 weeks ago

Austin, United States OneSource Regulatory Full time

Company Introduction

OneSource Regulatory Technology hosts a number of innovative solutions to enhance job performance in the Pharmaceutical space. OSR Technology is looking for an experienced and dedicated data engineer to join our product solutions team

Job Description

OneSource Regulatory is trying to identify a full-time contractor with at least 4+ years of experience to assist us with ongoing R&D projects.

We are looking for a data engineer to pull data from various sources and do all the necessary steps to clean, normalize, possibly annotate, and finally load the data into databases. The candidate should be able to develop and implement a strategy for testing the data integrity of the collected data. This role requires extreme attention to detail to ensure data quality is top priority.

Responsibilities

  • Well versed in parsing and synthesizing of XML and/or JSON documents.
  • Curating of data that can involve some intermediate to advanced web scraping. (data may need to be fetched via SFTP, FTP, Wget, Curl, REST APIs, GraphQL queries from spots on the Internet)
  • Proficiency with Linux command line and various simple tools, such as grep, wc, sed, awk, find, ls, cat, piped commands and possibly some very light Bash shell scripting, setting up crontab schedules and programs
  • Must have basic knowledge of SQL with the following databases: PostGres, MySQL, Google BigQuery
  • Must have basic knowledge of No-SQL database knowledge such as MongoDB or similar
  • Familiarity with basic Cloud technology such as storage buckets, cloud serverless functions
  • Must have experience extracting text and images from PDF files
  • Knowledge of Puppeteer or other automatable web client technologies
  • Understanding JavaScript, HTML/CSS and HTTP methods (for understanding page structure for web scraping)
Skills
  • Solid experience with Python and Python Libraries such as Pandas, requests, etc
  • Skill set should match up with required responsibilities listed above
  • Strong English skills (e.g. grammatical analysis and rhetorical structure)
  • Team Player
  • Great communication skills
Bonus Skills
  • Experience within the Pharmaceutical Space
  • Ability to expose data via C# NETCore and/or GraphQL
  • Google Cloud Platform (Cloud Buckets, Google Cloud Functions (.NET, Python, Node.JS))
  • Ability to parallelize data manipulation and scraping via Python multi-threading, etc.
  • Python BeautifulSoup
  • Scrapy
  • Docker (setting up Kubernetes style processing if warranted for data scraping/data ingestion/normalization)
  • Multithreading concepts

  • Data Center Engineering Operations Technician

    11 hours ago

    Austin, United States Amazon Data Services, Inc. Full time

    AWS Infrastructure Services (AIS) owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we’re the people who keep the cloud running. We support all AWS data centers and all of the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they...

  • Sr. Physical Security Engineer, Data Center Design Engineering

    2 weeks ago

    Austin, United States Amazon Data Services, Inc. Full time

    AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we’re the people who keep the cloud running. We support all AWS data centers and all of the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely...

  • Data Engineer

    2 weeks ago

    Austin, United States Tech M USAAvance Consulting Full time

    Data Engineer (Day 1 onsite) Austin, TX Must to have skills Python Pyspark SQL Data Engineering Big Data Job Description We're seeking a Data Engineer to take the lead in implementing and scaling data collection, storage, processing, and filtering for fine-tuning large language models (LLMs) within Conversational Engineering. These data...

  • Data Engineer

    3 weeks ago

    Austin, United States Avance Consulting Full time

    Job DescriptionJob DescriptionData Engineer (Day 1 onsite)Austin, TXMust to have skillsPythonPysparkSQLData EngineeringBig DataJob DescriptionWe're seeking a Data Engineer to take the lead in implementing and scaling datacollection, storage, processing, and filtering for fine-tuning large language models (LLMs) withinConversational Engineering. These...

  • Data Engineer

    3 weeks ago

    Austin, United States Tech M USAAvance Consulting Full time

    Data Engineer (Day 1 onsite) Austin, TX Must to have skills Python Pyspark SQL Data Engineering Big Data Job Description We're seeking a Data Engineer to take the lead in implementing and scaling data collection, storage, processing, and filtering for fine-tuning large language models (LLMs) within Conversational Engineering. These data pipelines are...

  • Data Engineer

    6 days ago

    Austin, United States Tech Mahindra Full time

    Data Engineer (Day 1 onsite) Auston, TXFulltimeMust to have skillsPythonPysparkSQLData EngineeringBig DataJob Description We're seeking a Data Engineer to take the lead in implementing and scaling data collection, storage, processing, and filtering for fine-tuning large language models (LLMs) within Conversational Engineering. These data pipelines are...

  • Data Engineer

    1 week ago

    Austin, United States Tech Mahindra Full time

    Data Engineer (Day 1 onsite) Auston, TX Fulltime Must to have skills Python Pyspark SQL Data Engineering Big Data Job Description We're seeking a Data Engineer to take the lead in implementing and scaling data collection, storage, processing, and filtering for fine-tuning large language models (LLMs) within Conversational Engineering. These data...

  • Data Engineer

    2 weeks ago

    Austin, United States Tech Mahindra Full time

    Data Engineer (Day 1 onsite) Auston, TXFulltimeMust to have skillsPythonPysparkSQLData EngineeringBig DataJob Description We're seeking a Data Engineer to take the lead in implementing and scaling data collection, storage, processing, and filtering for fine-tuning large language models (LLMs) within Conversational Engineering. These data pipelines are...

  • Data Engineer

    5 days ago

    Austin, Texas, United States IBM Full time

    Data EngineerIntroductionAt IBM, work is more than a job – it's a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you've never thought possible. Are you ready to lead in this new era of technology and solve some of the...

  • Data Engineer

    5 days ago

    Austin, United States IBM Full time

    Data Engineer IntroductionAt IBM, work is more than a job – it’s a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you’ve never thought possible. Are you ready to lead in this new era of technology and solve some...

  • Data Engineer

    5 days ago

    Austin, United States augmentjobs Full time

    Job DescriptionJob DescriptionPosition Overview: We are seeking a talented and experienced Data Engineer to join our dynamic tech team. As a Data Engineer, you will be responsible for designing, constructing, and maintaining our data architecture and infrastructure. You will work closely with data scientists, analysts, and other stakeholders to understand...

  • Data Engineer

    2 weeks ago

    Austin, United States XeoMatrix Full time

    Data Engineer We are currently seeking an experienced data engineer with 7 - 10 years with hands-on data engineering experience. This candidate must possess technical, business analysis, and communication skills. This position offers an opportunity to work directly with clients to design strategic data solutions that help them visualize complex data in a...

  • Data Engineer

    2 weeks ago

    Austin, United States XeoMatrix Full time

    Data Engineer We are currently seeking an experienced data engineer with 7 - 10 years with hands-on data engineering experience. This candidate must possess technical, business analysis, and communication skills. This position offers an opportunity to work directly with clients to design strategic data solutions that help them visualize complex data in a...

  • Data Engineer

    4 weeks ago

    Austin, United States Loxo Full time

    As an early hire to our engineering team, you will be responsible for managing Loxo's data integration function. You will be primarily responsible for migrating new clients' legacy data from their previous recruitment platform to Loxo and will work closely with the Customer Success team to deliver a positive onboarding experience for all new Loxo users. You...

  • Data Engineer

    2 weeks ago

    Austin, Texas, United States Loxo Full time

    As an early hire to our engineering team, you will be responsible for managing Loxo's data integration function. You will be primarily responsible for migrating new clients' legacy data from their previous recruitment platform to Loxo and will work closely with the Customer Success team to deliver a positive onboarding experience for all new Loxo users. You...

  • Data Engineer

    2 weeks ago

    Austin, United States Loxo Full time

    As an early hire to our engineering team, you will be responsible for managing Loxo's data integration function. You will be primarily responsible for migrating new clients' legacy data from their previous recruitment platform to Loxo and will work closely with the Customer Success team to deliver a positive onboarding experience for all new Loxo users. You...

  • Data Engineer

    3 weeks ago

    Austin, United States Loxo Full time

    As an early hire to our engineering team, you will be responsible for managing Loxo's data integration function. You will be primarily responsible for migrating new clients' legacy data from their previous recruitment platform to Loxo and will work closely with the Customer Success team to deliver a positive onboarding experience for all new Loxo users. You...

  • Data Engineer

    1 month ago

    Austin, Texas, United States Apple Full time

    SummaryPosted: Apr 24, 2024Weekly Hours: 40Role Number: At Apple, we work every day to create products that enrich people's lives Our Advertising Platforms group makes it possible for people around the world to easily access informative and imaginative content on their devices while helping publishers and developers promote and monetize their work. Today,...

  • Data Engineer

    2 weeks ago

    Austin, Texas, United States Apple Full time

    SummaryPosted: Apr 24, 2024Weekly Hours: 40Role Number: At Apple, we work every day to create products that enrich people's lives Our Advertising Platforms group makes it possible for people around the world to easily access informative and imaginative content on their devices while helping publishers and developers promote and monetize their work. Today,...

  • Data Engineer

    4 weeks ago

    Austin, Texas, United States Apple Full time

    SummaryPosted: Apr 24, 2024Weekly Hours: 40Role Number: At Apple, we work every day to create products that enrich people's lives Our Advertising Platforms group makes it possible for people around the world to easily access informative and imaginative content on their devices while helping publishers and developers promote and monetize their work. Today,...