Data Engineering Consultant

2 weeks ago

Boston, United States Ask Staffing Full time
Job Title: Data Engineering Consultant

Duration: 5 months

Work Location: Open to 100% Remote

Job Description:

• As a Data Engineering Consultant specializing in Data Warehouse Migration, you will be at the forefront of our efforts to transition our data ecosystem to a more efficient and modern platform.

• Your expertise in data engineering, AWS cloud technologies, and migration strategies will be crucial in ensuring a seamless migration process. Collaborating closely with cross-functional teams, you'll design, implement, and optimize ETL pipelines, enabling the secure and accurate transfer of data to our new AWS-based data warehouse environment.

• BachelorÂÂÂs or masterÂÂÂs degree in computer science, Data Engineering, or a related field.

• Strong proficiency in Python and shell scripting, with a focus on software engineering best practices and a deep understanding of development lifecycle.

• Experience with stream processing technologies, preferably Kafka.

• Familiarity with Avro data serialization format and its usage in data engineering workflows.

• Expertise in using Databricks platform on AWS for data processing and analytics.

• Solid understanding of data warehousing concepts and experience with Delta Lake and Parquet files.

• Proficiency in SQL and experience with relational databases.

• Strong testing skills, with experience in implementing and executing unit tests, integration tests, and end-to-end tests using Python packages such as pytest.

• Familiarity with the Python ecosystem, including PyPI packages and their integration into data engineering workflows.

• Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment.

• Strong communication skills and ability to effectively communicate complex technical concepts to non-technical stakeholders.

What are the 3-5 Â nice to haveÂÂ attributes that would separate a successful candidate from their competition when interviewing?

1.Working experience with BI tools such as Power BI and Tableau

2.Experience with ETLs and job orchestration tools such as AirFlow

3.Working experience with cloud services such as AWS

What are the top three must-haves you will be looking for in the resumes/interviews for this role?

1.Experience working with Databricks

2.Proficiency working with very large data sets

3.Proficiency with SQL

What business unit is this supporting?

This is for Currenex

Is it supporting a specific project or initiative? If so, which one and in what way?

This resource will be helping with Migration of Currenex BI warehouse to Databricks

What type of software skills is an absolute must have?

Python, pyspark and SQL

Who will be responsible for interviewing?

The hiring manager will perform the first round

My team members will conduct the second round - technical interview

Members of CurreneX dev team conduct the third rounds - practical programming skills test

What will the interview process entail (phone screen, face to face, meet with team, how many rounds, etc). What will the interview look like? Is there a test in the interview the candidate should be prepared for? All interviews are phone/virtual interviews. We ask candidates to keep their cameras open at all times. The skill test will be done over a whiteboard software over internet where candidate will be able to write code and explain the logic

Will this position be remote, hybrid, or 5 days onsite?

Open to remote or hybrid. Preference is to have the candidate where Currenex has offices (Boston, New York, San Francisco) but fully remote is also an option. There is no need for candidate to be onsite.

If the position is fully remote, do you have a preference on where the contractor is located? (State/Time zone)?

United States - can be located in any time zone

What are the working hours? What timezone?

Flexible (9-5) local time

Will this contractor need their own laptop or will client provide one for them?

Contractor will be provided with a client issued laptop

Will there be any training provided or would the need to jump right in first day?

Currenex Data Engineering will provide enough training to get the new candidate on track with existing framework and SDLC

Who will be training the new hire?

Currenex Data Engineering team

What type of background would you like this person to have?

Software development and Data Engineering

How many years experience?

5 years

What is the day to day of the person going to look like? Be specific.

Will be coding pyspark jobs to run on Databricks to prepare silver or gold delta tables for BI team

When will you be interviewing for this position? Please provide specific days/times if possible.

All interviews will be performed during PT working hours (pacific timezone)

  • Data Engineer

    4 weeks ago

    Boston, United States Northeastern University Full time

    The College of Social Sciences and Humanities is a leader in the Experiential Liberal Arts. The College is strongly committed to fostering excellence through diversity and enthusiastically welcomes applications from members of groups underrepresented in higher education administration. The College’s schools, departments, and programs—with...

  • Data Engineer

    4 weeks ago

    Boston, Massachusetts, United States Northeastern University Full time

    The College of Social Sciences and Humanities is a leader in the Experiential Liberal Arts. The College is strongly committed to fostering excellence through diversity and enthusiastically welcomes applications from members of groups underrepresented in higher education administration.The College's schools, departments, and programs—with disciplines...

  • Windows Platform Engineer

    7 days ago

    Boston, United States Data Affect Full time

    About Data Affect: We are a boutique data/service management firm specializing in the delivery of data governance, enterprise data strategy, solutions architecture, data warehousing, data integrations, data security & privacy management, IT service management, business analysis (and) agile project management services to diverse clients across multiple...

  • Windows Platform Engineer

    7 days ago

    Boston, United States Data Affect Full time

    About Data Affect: We are a boutique data/service management firm specializing in the delivery of data governance, enterprise data strategy, solutions architecture, data warehousing, data integrations, data security & privacy management, IT service management, business analysis (and) agile project management services to diverse clients across multiple...

  • Windows Platform Engineer

    7 days ago

    Boston, United States Data Affect Full time

    About Data Affect: We are a boutique data/service management firm specializing in the delivery of data governance, enterprise data strategy, solutions architecture, data warehousing, data integrations, data security & privacy management, IT service management, business analysis (and) agile project management services to diverse clients across multiple...

  • Data Engineer

    4 weeks ago

    Boston, Massachusetts, United States Jellyfish Full time

    As a member of Jellyfish Research, you will work closely with data scientists to understand and facilitate data needs for high-impact research, analysis and reporting. You will optimize, expand or redesign our current data platforms, including storage and pipeline architecture, to ingest, combine, and aggregate data from multiple sources. You'll also build...

  • Data Engineer

    2 weeks ago

    Boston, United States Jellyfish Full time

    As a member of Jellyfish Research, you will work closely with data scientists to understand and facilitate data needs for high-impact research, analysis and reporting. You will optimize, expand or redesign our current data platforms, including storage and pipeline architecture, to ingest, combine, and aggregate data from multiple sources. You’ll also build...

  • Snowflake Data Consultant

    8 hours ago

    Boston, United States WiseHire Recruitment Full time

    WiseHire are seeking a Senior Data Engineer, specializing in Snowflake, skilled at addressing the most intricate technical challenges and driving substantial business growth. This position involves direct collaboration with clients to comprehend needs, develop strategic assessments and implement and optimize the Snowflake platform for all their data needs....

  • Snowflake Data Consultant

    2 hours ago

    Boston, United States WiseHire Recruitment Full time

    WiseHire are seeking a Senior Data Engineer, specializing in Snowflake, skilled at addressing the most intricate technical challenges and driving substantial business growth. This position involves direct collaboration with clients to comprehend needs, develop strategic assessments and implement and optimize the Snowflake platform for all their data needs....

  • Fire Protection Engineer

    1 week ago

    Boston, United States Burohappold Engineering Full time

    Fire Protection Engineering at Buro Happold has developed significantly and wenow have fire engineers located across the US, with additional capabilities in the UK and the Middle East. A combination of international experience and local knowledge enables our engineers to respond quickly to project demands and to fully understand the complexities of working...

  • VP Data Engineering

    4 weeks ago

    Boston, United States Socium Recruitment Full time

    VP Data Engineering Onsite – Boston – Unable to support Visa’s & C2C Hey there! Are you all about Big Data and building awesome teams? This Start-up is shaking things up in their world with some seriously cool data solutions. They aren’t afraid to do things differently, and they’re on a mission to make a real impact with our innovative approach.As...

  • Data Engineer

    2 weeks ago

    Boston, Massachusetts, United States firsthand Full time

    firsthand is changing the way individuals living with Serious Mental Illness (SMI) get care. We are focusing on delivering real outcomes for a cohort that has historically been underserved, stigmatized, and deprioritized. By building a service focused on whole-person care, firsthand aims to find, establish a trusting relationship with, and support quality...

  • Data Engineer

    2 weeks ago

    Boston, United States OneSource Regulatory Full time

    Company Introduction OneSource Regulatory Technology hosts a number of innovative solutions to enhance job performance in the Pharmaceutical space. OSR Technology is looking for an experienced and dedicated data engineer to join our product solutions team! Job Description OneSource Regulatory is trying to identify a full-time contractor with at least 4+...

  • Head of Data Engineering

    2 days ago

    Boston, United States WiseHire Recruitment Full time

    WiseHire are seeking a Head of Data Engineering to lead the Analytics, Data Science, and Data Engineering teams within a growing organization. You'll shape the data strategy, architecture, data platform, and collaborate with various stakeholders to address their data needs, all while aligning with the wider business goals. This hands-on role involves...

  • Head of Data Engineering

    3 days ago

    Boston, United States WiseHire Recruitment Full time

    WiseHire are seeking a Head of Data Engineering to lead the Analytics, Data Science, and Data Engineering teams within a growing organization. You'll shape the data strategy, architecture, data platform, and collaborate with various stakeholders to address their data needs, all while aligning with the wider business goals. This hands-on role involves...

  • Data Engineer

    4 weeks ago

    Boston, Massachusetts, United States Selby Jennings Full time

    We have a current opportunity for a Data Engineer - Markit EDM on a permanent basis. The position will be based in Boston. For further information about this position please apply.

  • Senior Data Engineer

    3 days ago

    Boston, United States Storm2 Full time

    ⚡ Senior Data Engineer🎯 Series B FinTech Hyper Growth🌎 Boston, MA Hybrid💸 Salary: $140-180k base + equityCompany:Join a dynamic and rapidly growing fintech company backed by $130M in funding. The company is revolutionizing the financial industry with cutting-edge technology and innovative solutions. They have been recognized by Forbes Fintech 50...

  • Senior Data Engineer

    3 days ago

    Boston, United States Storm2 Full time

    ⚡ Senior Data Engineer🎯 Series B FinTech Hyper Growth🌎 Boston, MA Hybrid💸 Salary: $140-180k base + equityCompany:Join a dynamic and rapidly growing fintech company backed by $130M in funding. The company is revolutionizing the financial industry with cutting-edge technology and innovative solutions. They have been recognized by Forbes Fintech 50...

  • Data Engineer

    4 weeks ago

    Boston, United States OneSource Regulatory Full time

    Job DescriptionJob DescriptionSalary: Company IntroductionOneSource Regulatory Technology hosts a number of innovative solutions to enhance job performance in the Pharmaceutical space.  OSR Technology is looking for an experienced and dedicated data engineer to join our product solutions team!  Job DescriptionOneSource Regulatory is trying to identify a...

  • Software Engineer

    4 days ago

    Boston, United States Vendelux LLC. Full time

    Vendelux is a Series A SaaS company and we provide the system of record for event marketing. Our software platform provides proprietary insights that helps high-growth companies find the highest ROI events, conferences and trade shows to attend and sponsor. We have built an AI-powered platform that customers describe as an event marketer’s dream. Vendelux...