Data Lake Architect

1 month ago

San Francisco, United States FreightVerify Full time

FreightVerify is a supply chain technology company that provides real-time transportation visibility and business intelligence for global enterprise clients. Utilizing the latest IoT, big data, and AI technologies, FreightVerify provides a neutral platform that simplifies complex global supply chains through a common design language to leverage new and emerging tracking technologies.

AI (our comprehensive term for data science, machine learning, and deep learning) is the key technology that gives our platform its unique edge over our competition. This role is your opportunity to help build and deploy new AI products from the ground up.

The Data Lake Senior Architect will be part of the company AI team responsible for the system architecture, design, and development of a large, scaled-out, real-time, high-performing Hadoop infrastructure.

Responsibilities

Architect, design, and build a big data infrastructure platform, primarily based on Hadoop technologies on the Azure cloud platform, ensuring that the data lake meets specific minimum availability and security thresholds. Actively collaborate with the Advanced Product and AI teams to define the big data platform to achieve company product and business objectives. Actively collaborate with other technology teams and architects to define and develop cross-company/function technology stack synergies and interactions. Actively collaborate with the cross-functional teams to establish, document, implement and reinforce disciplined software development processes and best-practices. Be proficient in SQL and ETL processes, ETL and DB performance tuning, table partitioning, shell scripting, driving prototypes and proof of concepts. Set up and implement infrastructure for custom Data visualization tools. Research and experiment with emerging technologies and tools related to big data. Share your findings with the Advanced Products and AI teams. Maintain and support the platform to agreed service standards on a day-to-day basis.

Qualifications

Bachelor's Degree from accredited university or college in Business, Information Systems, Information Technology (IT) or Computer Science Minimum of 8 years of experience as a technologist, designing and developing data modeling. A minimum of 3 years specializing in big data architecture or data analytics (including technologies such as Hadoop, NoSQL, Map-Reduce, and other Industry BigData Frameworks). Understanding of Apache Hadoop and Hadoop ecosystem. Experience with one or more relevant tools such as Sqoop, Flume, Kafka, Oozie, Zookeeper, HCatalog. Experience in database and big data technologies, including MPP and NoSQL databases, data warehouse design, ETL, BI reporting, and dashboard development. Familiarity with one or more Hadoop technologies (Hive, Impala, Spark SQL, Presto). Experience developing software code in one or more programming languages (Python, R, etc.) Knowledge of best practices related to Data lakes, Data lake governance, Data Security (e.g., HITRUST), and Data Integration & Interoperability. Experience with agile development and DevOps.

We are an equal opportunity employer, hiring and developing individuals from diverse backgrounds and experiences to add to our collaborative culture. We do not discriminate in our recruiting, hiring, and promoting processes. Candidates and employees are treated with respect regardless of race, religion, sex, age, sexual orientation, gender identity and expression, national origin, veteran status, or disability. #J-18808-Ljbffr

  • Data Architect I

    2 weeks ago

    San Francisco, United States eTeam Inc. Full time

    Job Title: Data Architect Duration: 3 Months Location: CAPay Rate - $65/hr. - $70/hr. on W2 Job Description: Minimum 8 years of recent experience in database development, relational databases, data lakes, data analytics solutions, and data warehousing, including cloud-native databases and modern cloud-based data warehouse systems. Strong hands-on experience...

  • Data Architect I

    2 weeks ago

    San Francisco, United States eTeam Inc. Full time

    Job Title: Data Architect Duration: 3 Months Location: CAPay Rate - $65/hr. - $70/hr. on W2 Job Description: Minimum 8 years of recent experience in database development, relational databases, data lakes, data analytics solutions, and data warehousing, including cloud-native databases and modern cloud-based data warehouse systems. Strong hands-on experience...

  • Data Architect I

    1 week ago

    San Francisco, California, United States eTeam Inc. Full time

    Job Title: Data ArchitectDuration: 3 MonthsLocation:CAPay Rate - $65/hr. - $70/hr. on W2 Job Description:Minimum 8 years of recent experience in database development, relational databases, data lakes, data analytics solutions, and data warehousing, including cloud-native databases and modern cloud-based data warehouse systems.Strong hands-on experience in...

  • Data Architect

    1 week ago

    San Francisco, California, United States Unreal Gigs Full time

    Job Description Job Description Company Overview:Welcome to the forefront of data architecture At our company, we're dedicated to leveraging the power of data to drive insights and innovation. Our mission is to develop scalable and efficient data architecture solutions that enable organizations to unlock the full potential of their data. Join us and be part...

  • Data Architect

    1 month ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data architecture! At our company, we're dedicated to leveraging the power of data to drive insights and innovation. Our mission is to develop scalable and efficient data architecture solutions that enable organizations to unlock the full potential of their data. Join us and be...

  • Data Architect

    1 week ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data architecture! At our company, we're dedicated to leveraging the power of data to drive insights and innovation. Our mission is to develop scalable and efficient data architecture solutions that enable organizations to unlock the full potential of their data. Join us and be...

  • Data Architect

    4 weeks ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data architecture! At our company, we're dedicated to leveraging the power of data to drive insights and innovation. Our mission is to develop scalable and efficient data architecture solutions that enable organizations to unlock the full potential of their data. Join us and be...

  • Associate Data Architect

    4 weeks ago

    San Francisco, United States BN Associates Inc Full time

    Job DescriptionJob DescriptionJob Role: Associate Data ArchitectLocation: Remote, CA but HM would like candidates to live near Oakland or San Francisco, CA officesDuration: 12+ Months ContractRate $80/hr on W2 Client: Delta DentalVisa USC, GC, GC EAD, TN, H4Job DescriptionPre-Screen QuestionsDetail Candidate's Data Lake architecture knowledgeDetail...

  • Lead Data Architect

    2 weeks ago

    San Francisco, United States CareerBuilder Full time

    Company Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data architecture solutions that enable efficient data management, storage, and analytics. Join us and lead our...

  • Lead Data Architect

    1 month ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data architecture solutions that enable efficient data management, storage, and...

  • Lead Data Architect

    1 week ago

    San Francisco, California, United States Unreal Staffing, Inc Full time

    Company Overview:Welcome to the forefront of data-driven innovation Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data architecture solutions that enable efficient data management, storage, and analytics. Join us and lead our...

  • Lead Data Architect

    1 week ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data architecture solutions that enable efficient data management, storage, and...

  • Lead Data Architect

    4 weeks ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data architecture solutions that enable efficient data management, storage, and...

  • Senior Data Architect

    1 month ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data architecture solutions that enable efficient data management, storage, and...

  • Senior Data Architect

    4 weeks ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data architecture solutions that enable efficient data management, storage, and...

  • Senior Data Architect

    1 week ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data architecture solutions that enable efficient data management, storage, and...

  • Sr. Data Architect

    2 weeks ago

    San Francisco, United States Presence Full time

    Sr. Data Architect (Contractor) Presence is a San-Francisco based consultancy specializing in digital product development and innovation for global brands. We've been delivering cutting-edge projects for ten years- and we're growing! We're currently seeking a Sr Data Architect (Contractor) to join our engineering team. This role is based in San Francisco....

  • Data Architect

    1 week ago

    San Diego, California, United States Tekgence Full time

    Desired Skills and Experience:Check below to see if you have what is needed for this opportunity, and if so, make an application asap.12+ years of Software experience with 5-7 years on Data architecture and Data Strategy, 5+yrs on Databases, ETL , Data modelling, Reporting and 3 + years on cloud services.Mandatory Skills:Experience with enterprise data...

  • GCP Data Solution Architect

    2 weeks ago

    San Francisco, United States TEKsystems Full time

    Description: As an organization, Global Services provides a continuum of services ranging from Engagement Management to full functional Outsourcing including Offshore Centers in Canada and India. Our model provides the ability to engage customers beyond staffing when asked for more ownership, capabilities, or methodology while enhancing client/consultant...

  • Data Architect

    1 week ago

    San Francisco, California, United States Penn Foster Inc Full time

    Company DescriptionLogic20/20 invests in being a "Best Company to Work For," where talented people come together to deliver exceptional solutions. We bring clarity, insights, and optimization to enterprise organizations spanning technology, telecommunications, utilities, healthcare, and more.We thrive as One Team , built on values:We Foster a Culture of We...