Big Data Architect

2 weeks ago

New York, United States Diverse Lynx Full time
Role: Big Data Architect

Location: Pittsburgh, PA (Onsite)

Duration : Long Term

Job Description

SKill: Big Data Architect, Cloudera Big Data Tool stack (Hive ,Impala, OoZie), Pyspark, Teradata, Shell Script, Unix programing, Hadoop
  • 8+ years of data engineering/architecture experience in Waterfall and Agile Methodology, with a proven track record of delivering complex data solutions, preferably in the financial sector.
  • Proficiency in Python for advanced data analysis, scripting, automation, and integration with Big Data platforms and ecosystems.
  • Strong SQL experience (Oracle, Hive, Impala) in designing and implementing complex data models, DDLs, and DMLs, with a focus on performance optimization and scalability.
  • Proficiency in shell scripting for automation, task orchestration, and system integration in Unix/Linux environments.
  • Expertise in Cloudera Big Data Technologies, including Cloudera Distribution of Hadoop (CDP), Apache Hadoop, Apache Spark, Apache Hive, Apache Impala, and Apache HBase, with hands-on experience in deploying, configuring, and optimizing these technologies for large-scale data processing and analytics.
  • In-depth knowledge of data governance policies, metadata management, and regulatory compliance requirements, with a focus on protecting sensitive data and ensuring data privacy and security.
  • Proven experience migrating data from relational databases to Hadoop platforms, with a deep understanding of data migration strategies, tools, and best practices.
  • Ability to integrate complex big data infrastructure with diverse systems, applications, and data sources, using industry-standard APIs, connectors, and integration patterns.
  • Familiarity with data encryption tools and techniques, including Protegrity, for securing data at rest and in transit, and ensuring compliance with industry regulations and data protection standards.
  • Experience working with development tools like Jenkins for continuous integration, deployment, and pipeline automation, streamlining the software development lifecycle and ensuring rapid delivery of high-quality solutions.
  • Proficiency in Apache Oozie for workflow management, job scheduling, and orchestration of complex data processing workflows in Hadoop ecosystems, ensuring reliable and efficient execution of data pipelines and batch processing jobs.
  • Hands-on experience with ELK (Elasticsearch, Logstash, Kibana) stack, including configuration, customization, and management of Kibana dashboards for real-time monitoring and visualization of system metrics and application logs.
  • Effective communication, presentation, and organizational skills, with the ability to articulate complex technical concepts to diverse audiences and influence stakeholders at all levels of the organization.
  • Proficiency in Microsoft Visio, Excel, PowerPoint, and Word, with the ability to create compelling visualizations, documentation, and presentations to support decision-making and knowledge sharing.
  • Demonstrated leadership, analytical, and problem-solving skills, with a track record of driving innovation, fostering collaboration, and delivering results in dynamic, fast-paced environments.
  • Excellent written and oral communication skills, with the ability to convey complex technical concepts clearly and persuasively to technical and non-technical audiences.
  • Proven ability to work independently and as part of a high-performing team, with a collaborative and customer-centric approach to problem-solving and decision-making.
  • Commitment to continuous learning and professional development, with a passion for staying abreast of emerging technologies, trends, and best practices in data management, analytics, and related fields.
  • Strong critical thinking and decision-making abilities, with the capacity to prioritize competing demands, discern critical from minor issues, and innovate creative solutions to complex challenges.


Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

  • Big Data Architect

    1 month ago

    New York, New York, United States Mackenzie Health Care Full time

    Company DescriptionAbout Mackenzie Search:Great talent is both rare and valued the ability to recognize great talent is even more rareOne premise remains constant, even with the technological advances over the past half-century:It takes Tier-1 Leadership & Talent to build Strong, Successful Organizations That's where Mackenzie comes in.Mackenzie was founded...

  • Big Data Architect

    6 days ago

    New York, United States Diverse Lynx Full time

    Role: Big Data Architect Location: Pittsburgh, PA (Onsite) Duration : Long Term Job Description SKill: Big Data Architect, Cloudera Big Data Tool stack (Hive ,Impala, OoZie), Pyspark, Teradata, Shell Script, Unix programing, Hadoop 8+ years of data engineering/architecture experience in Waterfall and Agile Methodology, with a proven track record of...

  • Big Data Architect with Devops

    6 days ago

    New York, United States CyberTec Full time

    Role 3: Big Data Architect with Devops experience and local to NY Start Date: ASAP Rate - $90/hr. Maximum Start Date: ASAP Local to NY its onsite Client: Microsoft We'd like to see candidates at the Tech 3 level Description: Job Summary and Responsibilities: Hands on keyboard engineer with experience in the DevOps space and, preferably, experience in the...

  • Data Architect

    2 days ago

    New York, United States Coforge Full time

    We at Coforge are seeking Data Architect with mentioned skill-set and who can work onsite from Orlando, FL or Princeton, NJ or Jersey City, NJ locations.Role: Data ArchitectLocation: Orlando, FL, Princeton, NJ and Jersey City, NJExperience: 09+ yearsKey Skills: Data architecture, SQL, AWS, data modelingMode of Hire: Full TimeSkills Required :- Bachelor's...

  • Data Solutions Architect

    2 weeks ago

    New York, United States Brains Workgroup, Inc. Full time

    One of our clients, major health care company, is looking for an enthusiastic Senior Data Solutions Architect** Must be authorized to work in US for ANY EMPLOYER.No H1 Visa support for this position.The Location is Brooklyn, New YorkPermanent position with great benefits.Hybrid commute - 2/3 days Home/Office.Salary between 100K and 130K DOE. Senior Data...

  • Data Architect

    2 weeks ago

    New York, United States The Astor Group Full time

    Our Firm is responding to the growing demand for alternative data by modernizing their process of building data pipelines. In doing so, we are looking for a senior data architect/engineer:The Senior Azure Data Architect/Engineer s main responsibilities will be:Designing and implementing high-volume data ingestion and transformation pipelines.Experience in...

  • Senior Big Data Specialist

    6 days ago

    New York, United States Openmesh Network Full time

    About Openmesh and Xnode Today a handful of corporations control 90% of the world's data and IT infrastructure and we are expected to trust them. This is not just bad for business; it's bad for everyone. Information asymmetry has led to social inequality, scandals, polarization, and corruption. Sometimes, it has even led to war. Centralized data control acts...

  • Big Data Engineer

    2 weeks ago

    New York, United States Motion Recruitment Full time

    Outstanding long-term contract opportunity! A well-known Financial Services Company is looking for a Big Data Engineer in New York, NY. (Hybrid). Work with the brightest minds at one of the largest financial institutions in the world. This is long-term contract opportunity that includes a competitive benefit package! Our client has been around for over 150...

  • Big Data Engineer

    2 weeks ago

    New York, United States Motion Recruitment Partners, LLC Full time

    Outstanding long-term contract opportunity! A well-known Financial Services Company is looking for a Big Data Engineer in New York, NY. (Hybrid). Work with the brightest minds at one of the largest financial institutions in the world. This is long-term contract opportunity that includes a competitive benefit package! Our client has been around for over 150...

  • Data Architect

    2 weeks ago

    New York, United States MAPSYS INC. Full time

    Our New York based client is seeking a motivated and ambitious mission-driven professional interested in a challenging role that will help them develop their career in data strategy, data engineering, reporting, and analytics. The Data Architect will be responsible for the Data & Research Program, working closely with all team members to identify, develop,...

  • Senior Big Data Engineer

    2 months ago

    New York, United States Atechstar Full time

    Job description 0-2 years of demonstrable experience designing technological solutions to complex data problems developing & testing modular reusable efficient and scalable code to implement those solutions. Ideally this would include work on the following technologies Expert-level proficiency in at-least one of Java C++ or Python (preferred)/Scala...

  • Senior Big Data Engineer

    2 months ago

    New York, New York, United States Atechstar Full time

    Job description 0-2 years of demonstrable experience designing technological solutions to complex data problems developing & testing modular reusable efficient and scalable code to implement those solutions. Ideally this would include work on the following technologies Expert-level proficiency in at-least one of Java C++ or Python (preferred)/Scala knowledge...

  • Data Architect

    3 days ago

    New York, United States PSRTEK Full time

    Job DescriptionJob DescriptionJob Title: Data Architect RemoteJob DescriptionSkills: Must have Should have expertise in data architecture, data modelling, and data integration. Should be well versed with at least two of the Data Warehouse Platforms - AWS Redshift, Snowflake or equivalent. Candidate should be well versed with Data integration (from diverse...

  • Systems Engineer

    4 days ago

    New York, United States Hudson River Trading Full time

    Hudson River Trading (HRT) is looking for Systems Engineers to join our growing Research & Development team. This team builds and maintains an exceptionally large and growing distributed compute cluster, many-PB-scale storage systems, operating systems, automation software, and development tools. We are seeking a highly skilled engineer with expertise in...

  • Solutions Data Architect

    1 month ago

    New York, United States Pierce Technology Corp Full time

    Job DescriptionJob DescriptionSolutions/data architect who understands data, can create a data model and design a new solution for an enterprise data warehouse. Legacy system is limited in documentation, no real data model or map, and no user interface. Internal clients are looking for easier, better, and more efficient ways to access data. Looking for...

  • Solutions Data Architect

    1 month ago

    New York, United States Pierce Technology Corp Full time

    Job DescriptionJob DescriptionSolutions/data architect who understands data, can create a data model and design a new solution for an enterprise data warehouse. Legacy system is limited in documentation, no real data model or map, and no user interface. Internal clients are looking for easier, better, and more efficient ways to access data. Looking for...

  • Senior Data Architect

    1 month ago

    New York, United States TopView Sightseeing Full time

    Job DescriptionJob DescriptionNote: This is a full-time, on-site position based at our NYC office.We are looking for an experienced and hands-on Senior Data Architect to work in our New York City office full-time and in person. The Senior Data Architect will have a key role in our platform architecture by modeling, designing and implementing a modern data...

  • Senior Data Architect

    2 months ago

    New York, United States TopView Group Full time

    Note: This is a full-time, on-site position based at our NYC office.We are looking for an experienced and hands-on Senior Data Architect to work in our New York City office full-time and in person. The Senior Data Architect will have a key role in our platform architecture by modeling, designing and implementing a modern data stack that enables the creation...

  • Senior Data Architect

    2 months ago

    New York, United States TopView Sightseeing Full time

    Job DescriptionJob DescriptionNote: This is a full-time, on-site position based at our NYC office.We are looking for an experienced and hands-on Senior Data Architect to work in our New York City office full-time and in person. The Senior Data Architect will have a key role in our platform architecture by modeling, designing and implementing a modern data...

  • Systems Engineer

    5 days ago

    New York, United States Hudson River Trading Full time

    Hudson River Trading (HRT) is looking for Systems Engineers to join our growing Research & Development team. This team builds and maintains an exceptionally large and growing distributed compute cluster, many-PB-scale storage systems, operating systems, automation software, and development tools. We are seeking a highly skilled engineer with expertise in...