Lead Data Engineer

6 months ago

Irving, United States IVID TEK INC Remote Work Freelance Full time $65 - $70
Position: Lead Data Engineer (GCP,Azure)
Location: Dallas, TX
Job Type: Contract C2C
Rate: 65-70/ Hr
Roles and Responsibilities:Key Skills: Vertex AI , GCP , Kubeflow , CI/CD , Python , BigData
Job Description:
  • Identify new opportunities to improve business processes and improve consumer experiences, and prototype solutions to demonstrate value with a crawl, walk, run mindset.
  • Work with data scientists and analysts to create and deploy new product features on the ecommerce website, in-store portals and the Client's mobile app
  • Implement end-to-end solutions across the full breadth of ML model development lifecycle. The specific role includes working hand in hand with the scientists from the point of data exploration for model development to the point of building features, mls and deploying them in production. You will have an opportunity to work on both batch and real time models. The role also involves operational support.
  • Establish scalable, efficient, automated processes for data analyses, model development, validation and implementation
  • Write efficient and scalable software to ship products in an iterative, continual-release environment
  • Contribute to and promote good software engineering practices across the team and build cloud native software for ML pipelines
  • Contribute to and re-use community best practices
  • Embody the values and passions that characterize client., with empathy to engage with colleagues from multiple backgrounds

About You
  • University or advanced degree in engineering, computer science, mathematics, or a related field
  • 10+ years' experience developing and deploying machine learning systems into production, and independent contributor.
  • Comfortable with Python ecosystem, vscode, jupyternotebooks.
  • Experience working with big data tools: Hadoop, Spark, Kafka, etc.
  • Experience with at least one cloud provider solution (AWS, GCP) and understanding of serverless code development (GCP preferred)
  • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc. (Python preferred)
  • CI/CD expert. And can work on GitHub actions, harness, Jenkins
  • Can work with Google Big Query, or similar warehouse.
  • Work on Kubeflow pipelines independently and propose standards.
  • Knowledge of Feature Engineering, Feature Store, and audit capabilities.
  • Expertise in standard software engineering methodology, e.g. unit testing, test automation, continuous integration, code reviews, design documentation
  • Working experience with native ML orchestration systems such as Kubeflow, Step Functions, MLflow, Airflow, TFX...
  • Relevant working experience with Docker and Kubernetes is a big plus
  • Review and integrate all application requirements, including functional, security, integration, performance, quality and operations requirements.
  • Experience on Data Analytics Solutions Cloud Architect experience of at least 3-4 years in development using Cloud Technologies and awareness of different processes around Big Data platforms GCP Experience of 1-2 years on BigQuery, GCS, Dataproc, Composer, Scheduler, Pub/Sub, Dataflow, Cloud Function Proficient in Python scripting language 1-2 years of development experience and 1 year of experience Professional Certification Google Cloud Platform
  • Hands on experience in Azure Cloud components such as Azure Data Factory, Azure Logic apps service, Data Bricks, AKS, Azure devops or any Cloud technologies
  • Provide technical leadership to the project team to perform design to deployment related activities, provide guidance, perform reviews, prevent and resolve technical issues.
  • Hands on experience with GCP services using GCP Console or Deployment Manager
  • Excellent communication written and oral and interpersonal skills

  • Data Engineering Lead

    4 weeks ago

    Irving, Texas, United States INSPYR Solutions Full time

    Job SummaryINSPYR Solutions is seeking a highly skilled Data Engineering Lead to join our research group team. The ideal candidate will have a strong background in data engineering, data warehousing, and cloud platforms, with experience in leading the design and development of robust data pipelines, streaming ingestion frameworks, and scalable data...

  • Lead Data Engineer

    4 weeks ago

    Irving, Texas, United States Lennar Full time

    About the RoleWe are seeking a highly skilled Lead Data Engineer to join our Enterprise Data and Analytics team at Lennar. As a key member of our team, you will be responsible for providing technical leadership and expertise in designing and implementing our next-generation data and analytics platforms and products.Key ResponsibilitiesDesign, build, and...

  • Data Engineer

    4 weeks ago

    Irving, Texas, United States NTT DATA Group Corporation Full time

    Job Title: Data Engineer - SQL DeveloperAbout the Role:We are seeking a highly skilled Data Engineer - SQL Developer to join our team at NTT DATA Group Corporation. As a Data Engineer - SQL Developer, you will be responsible for designing, developing, and maintaining large-scale databases and data systems.Key Responsibilities:Design and develop databases and...

  • Senior Data Engineering Lead

    4 weeks ago

    Irving, Texas, United States Infovision Full time

    Job Title: Senior Data Engineering LeadLocation: Hybrid 2-3 days/weekDuration: 12+ MonthsKey Responsibilities: As a Senior Data Engineering Lead, you will be responsible for leading data engineering projects and teams. You will work closely with cross-functional teams to design, develop, and implement big data solutions using Hadoop and Spark. You will also...

  • Senior Data Engineering Lead

    4 weeks ago

    Irving, Texas, United States Resource Informatics Group Inc Full time

    Job DescriptionWe are seeking a highly experienced Data Engineering Lead to join our team at Resource Informatics Group Inc. The ideal candidate will have a strong background in ETL and a proven track record of leading teams and delivering results.The successful candidate will be responsible for leading a team of data engineers, designing and implementing...

  • Lead/Sr. Big Data Engineer

    3 weeks ago

    Irving, United States America Technology Professionals LLC Full time

    Job Title: Lead / Sr. Big Data Engineer Location: Irving, TX (Hybrid – 2-3 days/week) Duration: 12+ Months USC/GC/H1B only C2C/1099 only This role can play the hand-on dev role along with tech lead skills Required Skills: Big data expert with 9+ years experience in Hadoop Big data ecosystem Spark - Batch Streaming (Python, Scala) Experience in cloud...

  • Lead/Sr. Big Data Engineer

    4 weeks ago

    Irving, United States America Technology Professionals LLC Full time

    Job Title: Lead / Sr. Big Data EngineerApply now, read the job details by scrolling down Double check you have the necessary skills before sending an application.Location: Irving, TX (Hybrid – 2-3 days/week)Duration: 12+ MonthsUSC/GC/H1B onlyC2C/1099 onlyThis role can play the hand-on dev role along with tech lead skillsRequired Skills:Big data expert with...

  • Lead / Sr. Big Data Engineer

    2 weeks ago

    Irving, United States Infovision Full time

    Job Title: Lead / Sr. Big Data EngineerLocation: Irving, TX (Hybrid 2-3 days/week)Duration: 12+ MonthsRequired Skills: Big data expert with 9+ years experience in Hadoop Big data ecosystemSpark - Batch Streaming (Python, Scala)Experience in cloud environment specially Google Cloud PlatformExperience in developing both batch and real-time streaming data...

  • Lead / Sr. Big Data Engineer

    1 month ago

    Irving, United States InfoVision Inc. Full time

    Job Title: Lead / Sr. Big Data EngineerLocation: Irving, TX (Hybrid 2-3 days/week)Duration: 12+ Months Required Skills: Big data expert with 9+ years experience in Hadoop Big data ecosystemSpark - Batch Streaming (Python, Scala)Experience in cloud environment specially Google Cloud PlatformExperience in developing both batch and real-time streaming data...

  • Lead Data Engineer

    1 week ago

    Irving, TX, United States INSPYR Solutions Full time

    Title: Lead Data Engineer If you are interested in applying for this job, please make sure you meet the following requirements as listed below. Location: Irving, TX 75039 (Hybrid) Duration: Direct Hire Compensation: $175K - $200K Work Requirements: US Citizen, GC Holders or Authorized to Work in the U.S. Skillset / Experience: We are seeking a highly...

  • Lead Big Data Engineer

    2 weeks ago

    Irving, United States Infovision Full time

    Job title: Lead Big Data Engineer (GCP Cloud)Location: Irving, TXDuration: Long-termSkills Needed:10+ years of experience in designing and building data pipelines in large-scale distributed systems.Lead the design, development, and maintenance of scalable batch and real-time data processing pipelines.Proficiency with Google Cloud Platform (GCP) and tools...

  • Lead Big Data Engineer

    1 month ago

    Irving, United States InfoVision Inc. Full time

    Job title: Lead Big Data Engineer (GCP Cloud)Location: Irving, TXDuration: Long-term Skills Needed:10+ years of experience in designing and building data pipelines in large-scale distributed systems.Lead the design, development, and maintenance of scalable batch and real-time data processing pipelines.Proficiency with Google Cloud Platform (GCP) and tools...

  • Data Engineer

    2 weeks ago

    Irving, United States Synechron Full time

    Our challengeWe are seeking a highly skilled Data Engineer to join our dynamic team having experience in Cloud technologies. (Snowflake, GCP, Cloudera etc.).The RoleResponsibilities:Dev Lead / Engineer for Data – Cloud technologies. (Snowflake, GCP, Cloudera etc)Able to Lead & solution AWS, Snowflake data engineering team & designs.Design, develop, and...

  • Senior Linux Engineer

    2 months ago

    Irving, United States NTT DATA Full time

    Req ID: 299982 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Linux Engineer - Irving, Texas to join our team in Irving, Texas (US-TX), United States (US).NTT Data Americas is...

  • Senior Data Engineer

    4 weeks ago

    Irving, Texas, United States Randstad North America, Inc. Full time

    Job SummaryWe are seeking a highly skilled Data Engineering Expert to join our team at Randstad North America, Inc. As a key member of our technology team, you will be responsible for designing and implementing advanced data engineering solutions using Python and Pyspark.Key ResponsibilitiesDesign and develop scalable data engineering solutions using Python...

  • Senior Data Engineer

    1 month ago

    Irving, Texas, United States IVID TEK INC Full time

    Job Title: Lead Data EngineerAt IVID TEK INC, we are seeking a highly skilled and experienced Lead Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and deploying large-scale data processing systems and architectures.Key Responsibilities:Design and implement scalable data...

  • Big Data Engineer

    1 month ago

    Irving, Texas, United States Motion Recruitment Full time

    Job Title: Big Data EngineerAre you a skilled Big Data Engineer looking for a new challenge? We have an exciting opportunity for you to join our team at a leading global bank in Irving, TX.About the Role:We are seeking a highly motivated and experienced Big Data Engineer to join our team. As a Big Data Engineer, you will be responsible for the delivery of...

  • Senior Data Engineer

    4 weeks ago

    Irving, Texas, United States Randstad North America, Inc. Full time

    Job SummaryWe are seeking a highly skilled Data Engineer to join our team at Randstad North America, Inc. As a key member of our technology team, you will be responsible for designing, developing, and maintaining large-scale data systems.Key ResponsibilitiesDesign and implement data pipelines using advanced SQL skills and experience with relational databases...

  • Senior Data Engineer

    2 weeks ago

    Irving, United States Synechron Full time

    Senior Lead Data EngineerLocation: Irving TX (hybrid/3 days a week from day one)Overview:We are seeking a highly skilled Data Engineer to join our dynamic team having experience in Cloud technologies. (Snowflake, GCP, Cloudera etc).The RoleResponsibilities:Dev Lead/Engg for Data – Cloud technologies. (Snowflake, GCP, Cloudera etc)Able to Lead & solution...

  • Data Architecture Senior Lead Analyst

    4 weeks ago

    Irving, Texas, United States Citigroup Inc Full time

    About the RoleCitigroup Inc. is seeking a highly skilled Data Architecture Senior Lead Analyst to join our team. As a key member of our Data Engineering team, you will be responsible for designing, developing, and implementing complex cloud-based data architectures.Key ResponsibilitiesLead the design and development of cloud-based data architecturesProvide...