Data Engineer

4 weeks ago

Waltham, United States Tata Consultancy Services Full time

Job Title: Staff Data Engineer

Must Have Technical/Functional Skills:

Someone with hands on experience with cloud infrastructure, PySpark/Spark(strong), CICD, basic software engineering, data engineering, etc. This role will be responsible for handling multiple products and multiple stake holders, so we are looking a candidate with lots of hands on experience and who enjoys learning and sharing the latest trends in the tech space related to DE.

Tech Stack – Snowflake, Databricks, Python, pyspark, Databricks, SQL and Azure.


Relevant Experience

(in Yrs) 12+ (E3)


Roles & Responsibilities :

A Staff Data Engineer will design and develop highly scalable and extensible data pipelines which enable collection, storage, distribution, modeling, and analysis of large data sets from many channels. This position requires an innovative software engineer who is passionate about data & data quality. The ideal candidate will have strong data warehousing and API integration experience and the ability to develop scalable data pipelines that make data management and analytics/reporting faster, more insightful, and more efficient.

Lead the Data engineering Team to Develop, test, document and support scalable data pipelines.

Build out new data integrations including APIs to support continuing increases in data volume and complexity.

Establish and follow data governance processes and guidelines to ensure data availability, usability, consistency, integrity, and security.

Build and implement scalable solutions that align to our data governance standards and architectural road maps for data integrations, data storage, reporting, and analytic solutions.

Collaborate with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility, and fostering data-driven decision making across the organization.

Design and develop data integrations and a data quality framework. Write unit/integration/functional tests and document work.

Design, implement, and automate deployment of our distributed system for collecting and processing streaming events from multiple sources

Perform data analysis needed to troubleshoot data-related issues and aid in the resolution of data issues.

Guide and mentor junior engineers on coding best practices and optimization.

Qualifications:

Education: 4-year college degree or equivalent combination of education and experience. Prefer an academic background in Computer Science, Mathematics, Statistics, or related technical field.

8 years of relevant work experience in analytics, data engineering, business intelligence or related field.

Skilled in object-oriented programming (Python in particular).

Strong Experience in Python, PySpark and SQL.

Strong Experience in Databricks & Snowflake.

Experience developing integrations across multiple systems and APIs.

Experience with or knowledge of Agile software development methodologies.

Experience with cloud-based databases, specifically Azure technologies (e.g., Azure data lake, ADF, Azure DevOps and Azure Functions).

Experience using SQL queries as well as writing and perfecting SQL queries in a business environment with large-scale, complex datasets.

Experience with data warehouse technologies. Experience creating ETL and/or ELT jobs.

Excellent problem solving and troubleshooting skills.

Process oriented with great documentation skills.

Experience designing data schemas and operating SQL/NoSQL database systems is a plus.

  • Data Engineer

    2 weeks ago

    Waltham, United States Tata Consultancy Services Full time

    Job Title: Staff Data EngineerMust Have Technical/Functional Skills: Someone with hands on experience with cloud infrastructure, PySpark/Spark(strong), CICD, basic software engineering, data engineering, etc. This role will be responsible for handling multiple products and multiple stake holders, so we are looking a candidate with lots of hands on experience...

  • Principal Data Engineer

    3 weeks ago

    Waltham, United States Madrigalpharma Full time

    Reporting to Senior Director of Data and Analytics as Principal Data Engineer, you will be responsible for leading the design, development, and implementation of our data infrastructure. Leveraging your deep technical expertise and leadership skills, you will play a critical role in shaping our data platform strategy, architecture, and capabilities to unlock...

  • Principal Data Engineer

    4 weeks ago

    Waltham, United States Madrigal Pharmaceuticals Corporation Full time

    Job DescriptionJob DescriptionReporting to Senior Director of Data and Analytics as Principal Data Engineer, you will be responsible for leading the design, development, and implementation of our data infrastructure. Leveraging your deep technical expertise and leadership skills, you will play a critical role in shaping our data platform strategy,...

  • Manager, Clinical Data Management

    4 weeks ago

    Waltham, United States Statistics & Data Corporation (SDC) Full time

    Job DescriptionJob DescriptionManager, Clinical Data ManagementSDC is a team of diversified professionals who deliver exceptional Biometric Services, Consulting, and Technology Solutions to pharmaceutical, biologic, and medical device/diagnostic companies. Since 2005 our purpose has been to partner with sponsors to provide high quality and experienced team...

  • Manager, Clinical Data Management

    2 months ago

    Waltham, United States Statistics & Data Corporation (SDC) Full time

    Job DescriptionJob DescriptionManager, Clinical Data ManagementSDC is a team of diversified professionals who deliver exceptional Biometric Services, Consulting, and Technology Solutions to pharmaceutical, biologic, and medical device/diagnostic companies. Since 2005 our purpose has been to partner with sponsors to provide high quality and experienced team...

  • Staff Data Engineer

    1 week ago

    Waltham, Massachusetts, United States National Grid Full time

    About National GridAt National Grid, we are a leading energy company that delivers essential services to our customers. Our mission is to provide safe, reliable, and efficient energy solutions that meet the evolving needs of our communities.We are seeking a highly skilled Staff Data Engineer to join our Product Engineering department. This role will be...

  • Staff Data Engineer

    4 weeks ago

    Waltham, United States Tata Consultancy Services Full time

    Job Title Staff Data Engineer - LeadLocation - Waltham. MARelevant Experience (in Yrs) 12+ Must Have Technical/Functional Skills Staff Data Engineering role - someone with hands on experience with cloud infrastructure, PySpark/Spark(strong), Snowflake, Matillion, CICD, basic software engineering, data engineering, etc. This role will be responsible for...

  • Staff Data Engineer

    1 week ago

    waltham, United States Tata Consultancy Services Full time

    Job Title Staff Data Engineer - LeadLocation - Waltham. MARelevant Experience (in Yrs) 12+ Must Have Technical/Functional Skills Staff Data Engineering role - someone with hands on experience with cloud infrastructure, PySpark/Spark(strong), Snowflake, Matillion, CICD, basic software engineering, data engineering, etc. This role will be responsible for...

  • Data Engineer, Customer

    2 weeks ago

    Waltham, United States DraftKings Full time

    We’re defining what it means to build and deliver the most extraordinary sports and entertainment experiences. Our global team is trailblazing new markets, developing cutting-edge products, and shaping the future of responsible gaming. Here, “impossible” isn’t part of our vocabulary. You’ll face some of the toughest but most rewarding challenges of...

  • Staff Data Engineer

    2 days ago

    Waltham, Massachusetts, United States Tata Consultancy Services Full time

    Job Title: Staff Data Engineer - LeadLocation: Waltham, MARelevant Experience: 12+ yearsJob SummaryWe are seeking a highly experienced Staff Data Engineer - Lead to join our Data Engineering team at Tata Consultancy Services. As a key member of our team, you will be responsible for designing and developing highly scalable and extensible data pipelines that...

  • Data Science Engineer

    6 days ago

    Waltham, United States Gardner Resources Consulting, LLC Full time

    Our client in Wellesley, MA is looking for a Data Science Engineer.SQL, Python, Spark, ElasticSearch, Data VisualizationHubSpotTickets Creation

  • Data Science Engineer

    7 days ago

    Waltham, United States Gardner Resources Consulting, LLC Full time

    Our client in Wellesley, MA is looking for a Data Science Engineer.SQL, Python, Spark, ElasticSearch, Data VisualizationHubSpotTickets Creation

  • Data Science Engineer

    13 hours ago

    Waltham, United States Digital Prospectors Full time

    Position: Data Science EngineerLocation: Waltham, MA (Onsite)Length: 3-6 months*W2 Candidates OnlyJob Description:Are you a data-driven problem solver with a passion for creating impactful solutions? Our client is seeking a talented Data Science Engineer to join our team their team. In this role, you'll have the opportunity to work on cutting-edge data...

  • Data Science Engineer

    7 days ago

    Waltham, United States Digital Prospectors Full time

    Position: Data Science EngineerLocation: Waltham, MA (Onsite)Length: 3-6 months*W2 Candidates OnlyJob Description:Are you a data-driven problem solver with a passion for creating impactful solutions? Our client is seeking a talented Data Science Engineer to join our team their team. In this role, you'll have the opportunity to work on cutting-edge data...

  • Data Engineer

    3 days ago

    Waltham, Massachusetts, United States Tata Consultancy Services Full time

    About the RoleWe are seeking a highly skilled Staff Data Engineer to join our team at Tata Consultancy Services. As a key member of our data engineering team, you will be responsible for designing and developing highly scalable and extensible data pipelines that enable collection, storage, distribution, modeling, and analysis of large data sets from multiple...

  • Senior Data Engineer

    2 months ago

    Waltham, United States Scipher Medicine Full time

    Senior Data EngineerScipher Medicine is a precision immunology company that uses AI and network science to match patients with the most effective therapies. Its flagship product, PrismRA, is a blood-based test predicting whether a patient with RA will likely respond to anti-TNF therapy. Scipher Medicine is also developing the Spectra platform, which can be...

  • Senior Data Engineer

    2 months ago

    Waltham, United States Scipher Medicine Full time

    Senior Data EngineerScipher Medicine is a precision immunology company that uses AI and network science to match patients with the most effective therapies. Its flagship product, PrismRA, is a blood-based test predicting whether a patient with RA will likely respond to anti-TNF therapy. Scipher Medicine is also developing the Spectra platform, which can be...

  • Data Engineer

    1 month ago

    Waltham, United States K&K Global Talents Full time

    Job DescriptionJob DescriptionRole Description: A Senior Data Engineer will design and develop highly scalable and extensible data pipelines which enable collection, storage, distribution, modeling, and analysis of large data sets from many channels. This position requires an innovative software engineer who is passionate about data & data quality. The ideal...

  • Machine Learning Engineer Data Scientist

    21 hours ago

    Waltham, Massachusetts, United States NoWorkerLeftBehind LLC Full time

    About NoWorkerLeftBehind LLCNoWorkerLeftBehind LLC is a leading company in the digital experience industry, dedicated to empowering individuals and organizations to create exceptional digital experiences. Our mission is to drive innovation and excellence in the field of artificial intelligence and analytics.The OpportunityWe are seeking an experienced...

  • Data Analyst

    1 month ago

    Waltham, United States Scipher Medicine Full time

    About Us: Scipher Medicine is a precision immunology company that uses AI and network science to match patients with the most effective therapies. Its flagship product, PrismRA, is a blood-based test that predicts whether a patient with RA is likely to respond to anti-TNF therapy. Scipher Medicine is also developing the Spectra platform, which can be used to...