Data engineer
1 month ago
Role: DBT (Core/ Cloud) Developer
Location: Remote
Please find the revised JD Senior DBT Developer. Please evaluate the resume based on below highlighted skills.
- Client looking for Senior/ Lead DBT Developer.
- Good AWS experience.
- Should have some DBT Admin experience.
Senior DBT developer:
Technical Skills:
DBT Proficiency:
- model development:
- Experience in creating complex DBT models including incremental models, snapshots and documentation
- Ability to write and maintain DBT macros for reusable code
- Testing and documentation:
- Proficiency in implementing DBT tests for data validation and quality checks
- Familiarity with generating and maintaining documentation using DBT's built in features
- Version control:
- Experience in managing DBT projects using git ,including implementing CI/CD process from the scratch
- AWS Expertise:
- Data STORAGE solutions:
- In depth understanding of AWS S3 for data storage, including best practices for organization and security
- Experience with AWS redshift for data warehousing and performance optimization
- Data Integration:
- Familiarity with Aws glue for ETL processes and orchestration -Nice to have
- Experience with AWS lambda for serverless data processing tasks
- Workflow Orchestration:
- Proficiency in using Apache Airflow on AWS to design ,schedule and monitor complex data flows
- Ability to integrate Airflow with AWS services and DBT models such as triggering a DBT model or EMR or reading from s3 writing to redshift
- Data Lakes and Data warehousing:
- Understanding the architecture of data lakes vs data warehouses and when to use each
- Experience with amazon Athena for querying data directly in s3 using SQL
- Monitoring and Logging:
- Familiarity with AWS cloud watch for monitoring the pipelines and setting up alerts for workflow failures
- Cloud Security:
- Knowledge of AWS security best practices ,including IAM roles, encryption, DBT profiles access configurations
Programming Skills:
- Python:
- Proficiency in Pandas and NumPy for data analysis and manipulation
- Ability to write scripts for automating ETL processes and scheduling jobs using airflow
- Experience in creating custom DBT macros using jinja and Python allowing for reusable components within dbt models
- Knowledge on how to implement conditional logic in DBT through python
- SQL:
- Advanced SQL skills, including complex joins ,window functions, CTE’s and subqueries
Experience in optimizing SQL queries for performance and optimization
-
Sr. Azure Data Engineer
4 months ago
New York, United States Hudson Data Full timeJob Title: Sr. Azure Data Engineer Location: NYC, NY Duration: Long Term W2 Role Job Description: The candidate should have hands-on experience with various Azure services including but not limited to Azure Data Factory, Databricks, Data Lake, and Power BI. They should be able to design, build, and maintain ETL pipelines, manage data lakes, and...
-
Data Engineer
1 month ago
new york city, United States Brooksource Full timeData EngineerLocation: New York, NY (Hybrid)Contract-to-HireWe’re looking for an experienced Data Engineer to join a Fortune 50 Broadcast Media & Entertainment leader. This role involves designing, building, and optimizing data pipelines, integrating multiple datasets, and creating applications that drive strategic business decisions. You’ll collaborate...
-
Data Engineer
1 month ago
new york city, United States Brooksource Full timeData EngineerLocation: New York, NY (Hybrid)Contract-to-HireWe’re looking for an experienced Data Engineer to join a Fortune 50 Broadcast Media & Entertainment leader. This role involves designing, building, and optimizing data pipelines, integrating multiple datasets, and creating applications that drive strategic business decisions. You’ll collaborate...
-
Data Engineer
3 weeks ago
new york city, United States Walsh Employment Full timeOur client is a leading quantitative investment company with a focus on computer-driven trading for global financial markets. The company harnesses the power of data and advanced technologies to drive innovative strategies and superior investment returns through a platform that is driven by cutting-edge, cloud-based data solutions. We are now looking for a...
-
Data Engineer
3 weeks ago
new york city, United States Walsh Employment Full timeOur client is a leading quantitative investment company with a focus on computer-driven trading for global financial markets. The company harnesses the power of data and advanced technologies to drive innovative strategies and superior investment returns through a platform that is driven by cutting-edge, cloud-based data solutions. We are now looking for a...
-
Data Engineer
1 month ago
new york city, United States hackajob Full timeExciting Opportunity Alert! Join our Team as a Data Engineer!hackajob has partnered with an innovative company seeking a skilled Data Engineer to leverage data for impactful solutions. If you have expertise in big data tools like Spark and Kafka, and experience with cloud services like AWS, we want to hear from YOU!Role: Data EngineerLocation: Hybrid (New...
-
Data Engineer
1 month ago
new york city, United States hackajob Full timeExciting Opportunity Alert! Join our Team as a Data Engineer!hackajob has partnered with an innovative company seeking a skilled Data Engineer to leverage data for impactful solutions. If you have expertise in big data tools like Spark and Kafka, and experience with cloud services like AWS, we want to hear from YOU!Role: Data EngineerLocation: Hybrid (New...
-
Data Engineer
3 weeks ago
new york city, United States Atlas Search Full timeResponsibilitiesCloud Data Engineer is responsible for building and maintaining data pipelines within an AWS cloud environment. Work heavily with MongoDB and Snowflake as well as Cypress, Selenium and Kafka. Requirements3+ years of experience in the Data Engineering space working heavily with Public cloud solutions.Experience with MongoDB or...
-
Senior Systems Engineer
1 week ago
New York, United States VAST Data Full timeVAST Data is looking for a Senior Systems Engineer to join our growing team! This is a great opportunity to be part of one of the fastest-growing infrastructure companies in history, an organization that is in the center of the hurricane being created by the revolution in artificial intelligence. "VAST's data management vision is the future of the...
-
Data Engineer
1 month ago
New York, United States Brooksource Full timeData EngineerLocation: New York, NY (Hybrid)Contract-to-HireWe’re looking for an experienced Data Engineer to join a Fortune 50 Broadcast Media & Entertainment leader. This role involves designing, building, and optimizing data pipelines, integrating multiple datasets, and creating applications that drive strategic business decisions. You’ll collaborate...
-
Data Engineer
6 months ago
New York, United States Silver Search, Inc Full timeData Engineer New York, NY Senior Data Engineer--Redshift/AWS/Python/Scripting---**Will be hybrid onsite in January 2022 onwards** Contract-to-hire Our client is a cutting edge E-Learning company based in NYC and is looking to hire a Senior Data Engineer to join their data and analytics center of excellence. He/She will be building...
-
Cloud Data Engineer
3 weeks ago
new york city, United States Atlas Search Full time3 days onsiteWe are seeking an experienced Cloud Data Engineer with expertise in Kafka, Snowflake, and MongoDB to design and manage robust, scalable data pipelines within a cloud environment. This role involves building CI/CD pipelines to automate and streamline data workflows, ensuring data integrity and security across test and production...
-
Cloud Data Engineer
3 weeks ago
new york city, United States Atlas Search Full time3 days onsiteWe are seeking an experienced Cloud Data Engineer with expertise in Kafka, Snowflake, and MongoDB to design and manage robust, scalable data pipelines within a cloud environment. This role involves building CI/CD pipelines to automate and streamline data workflows, ensuring data integrity and security across test and production...
-
Data Engineer
4 weeks ago
New York, United States Walsh Employment Full timeOur client is a leading quantitative investment company with a focus on computer-driven trading for global financial markets. The company harnesses the power of data and advanced technologies to drive innovative strategies and superior investment returns through a platform that is driven by cutting-edge, cloud-based data solutions. We are now looking for a...
-
Principal Data Engineer
2 months ago
new york city, United States Ichor Strategies Full timeJob SummaryIchor Strategies seeks a Principal Data Engineer to lead the data architecture of our nascent SaaS platform which connects companies to the communities they operate in so that they can understand different perspectives, build authentic relationships, and foster mutual success. The ideal candidate will have extensive experience in driving...
-
Data Engineer
3 weeks ago
New York, United States SimplifyVMS Full timeAs a Data Engineer, you will be responsible for building and optimizing data pipelines, architectures, and datasets for effective data analysis and machine learning applications. Responsibilities: Developing, constructing, testing, and maintaining architectures for data ingestion, processing, and extraction. Designing and implementing data pipelines to...
-
Senior Data Engineer
1 month ago
new york city, United States Coda Search│Staffing Full timeA tech startup company in the healthcare space is looking to hire a Senior Data Engineer.Duties & Essential Job FunctionsDevelop, release, and maintain high quality data pipelines using Python, FiveTran, DBT, and SnowflakeOwn and guide the development of our data infrastructureDevelop custom integrations using DagsterConfigure reverse ETL integrations using...
-
Senior Data Engineer
2 months ago
new york city, United States Coda Search│Staffing Full timeA tech startup company in the healthcare space is looking to hire a Senior Data Engineer.Duties & Essential Job FunctionsDevelop, release, and maintain high quality data pipelines using Python, FiveTran, DBT, and SnowflakeOwn and guide the development of our data infrastructureDevelop custom integrations using DagsterConfigure reverse ETL integrations using...
-
Data Engineer
5 days ago
New York, United States Glow Networks Full timeRole: Data Engineer Location: New York City, NY (Hybrid-3 days a week onsite) Duration: 6 Months with possible extensions Contract: C2C/ W2 Job Requirements: Deep hands-on knowledge in big data engineering in a cloud environment, including: Excellent knowledge of SQL in a large scale data warehouse or data lakehouse environment such as Spark, Databricks,...
-
Data Engineer
1 week ago
New York, United States TechnoGen Full timeData Engineer Remote Snowflake Expertise: Design, develop, and optimize data pipelines within the Snowflake data platform. Implement best practices for Snowflake data modeling, warehouse architecture, security and performance optimization. Proficient in using Snowflake Snowpipe streaming for real-time data streaming and ingestion and different batch load...