Senior Systems Administrator

The Scientific Computing and Data group at the Icahn School of Medicine at Mount Sinai partners with scientists to accelerate scientific discovery. To achieve these aims, we support a cutting-edge high-performance computing and data ecosystem along with MD/PhD-level support for researchers. The group is composed of a high-performance computing team, the research clinical data warehouse team and a research data services team.

The Senior Systems Administrator/Engineer, as a member of the Scientific Computing and Data group, is responsible for a computational and data science ecosystem for researchers at Mount Sinai. The Administrator is the principal technology expert for Windows and Linux systems, and help support high-performance computing (HPC) environment in the Scientific Computing group. The incumbent utilizes a thorough understanding of available technology, tools and best practices to design, manage, maintain, upgrade and monitor Scientific Computing’s systems. The incumbent will develop and implement solutions responsive to researcher needs, in conjunction with other technology professionals and consistent with IT policies and Compliance. The systems will support a wide array of applications, including VMware, REDCap, Jira, Confluence, Postgres, MySQL, SQL server, Tivoli Storage Manager (TSM), and other custom Sinai-developed software. In total, there are >100 servers including physical servers and VMs along with an archival storage system containing over 20 petabytes of data. The TSM system is integrated with the 25,000-core, 30 petabyte HPC system. This position reports to the Director for Computational & Data Ecosystem in Scientific Computing. Specific responsibilities are listed below.

Responsibilities

• Design, develop, implement all system administration tasks, including hardware and software configuration and maintenance, configuration management, system monitoring, upgrade, usage monitoring and reporting, system performance, security, networking and metrics, etc. The infrastructure includes both Windows and Linux systems with file servers in multiple physical locations, and a HPC system with 25,000-cores and 30 petabyte of storage.
• Design and develop scripts for system administration and monitoring for Ansible configuration management, Grafana/Nagios/Zabbix system monitoring, Splunk and other tools.
• Research, deploy and manage security infrastructure, including implementation of policies and procedures from IT Security and Compliance.
• Plan, implement, troubleshoot and maintain software including databases (SQL, MySQL, PostgreSQL, and other databases), REDCap, Jira, Confluence, TSM, VMware and other software.
• Troubleshoot system and application issues across multiple environments and operating platforms.
• Research, suggest and implement new uses of information technologies, policies and procedures for continued improvement.
• Develop processes and policies for a 20-petabyte TSM tape archival storage system with thousands of users. Perform system administration support for TSM, including management of the 300 terabyte TSM disk cache, 12 LTO9 tape drives and 12 LTO5 tape drives. Assist with end researcher support to place and retrieve files. Develops and implements backup policies.
• Assist in the management and maintenance of HPC cluster and data center work, including troubleshooting for resolving system problems, coordinating with users and vendors, monitoring, audit and logging etc.
• Answer and resolve user tickets.
• Develop and create effective system documentation for all.
• Provide off-hours support for critical and other production issues.
• Performs other duties as assigned or requested.

Qualifications

• Bachelors degree in a technical discipline; Masters degree preferred
• Experience working in a research environment preferred
• 10 years of experience installing, configuring, managing, provisioning, automating tasks and monitoring hardware and software. Experience with data and security best practices.
• At least 6 years of experience in designing, administering and troubleshooting Linux and Windows systems, storage systems, network and VMs.
• The ability to communicate effectively and manage multiple conflicting priorities and projects simultaneously.
• Excellent analytical ability, strong judgment and management skills, and the ability to work effectively and independently with clients, vendors, IT management and staff.
• Experience with JIRA, Confluence administration, databases (MS SQL, MySQL, MySQL Galera, Oracle, PostgreSQL, etc.), container and VMWare preferred.
• Ability to lead the project to successful completion with little or no guidance
• Experience with supporting HPC environments including configuration management (such as xCAT, Puppet or Ansible), node installation and provision, networking, storage and job scheduler are preferred.

Strength Through Diversity

The Mount Sinai Health System believes that diversity, equity, and inclusion are key drivers for excellence. We share a common devotion to delivering exceptional patient care. When you join us, you become a part of Mount Sinai’s unrivaled record of achievement, education, and advancement as we revolutionize medicine together. We invite you to participate actively as a part of the Mount Sinai Health System team by:

• Using a lens of equity in all aspects of patient care delivery, education, and research to promote policies and practices to allow opportunities for all to thrive and reach their potential.
• Serving as a role model confronting racist, sexist, or other inappropriate actions by speaking up, challenging exclusionary organizational practices, and standing side-by-side in support of colleagues who experience discrimination.
• Inspiring and fostering an environment of anti-racist behaviors among and between departments and co-workers.

At Mount Sinai, our leaders strive to learn, empower others, and embrace change to further advance equity and improve the well-being of staff, patients, and the organization. We expect our leaders to embrace anti-racism, create a collaborative and respectful environment, and constructively disrupt the status quo to improve the system and enhance care for our patients. We work hard to create an inclusive, welcoming and nurturing work environment where all feel they are valued, belong and are able to advance professionally.

Explore more about this opportunity and how you can help us write a new chapter in our history

About the Mount Sinai Health System:

Mount Sinai Health System is one of the largest academic medical systems in the New York metro area, with more than 43,000 employees working across eight hospitals, more than 400 outpatient practices, more than 300 labs, a school of nursing, and a leading school of medicine and graduate education. Mount Sinai advances health for all people, everywhere, by taking on the most complex health care challenges of our time — discovering and applying new scientific learning and knowledge; developing safer, more effective treatments; educating the next generation of medical leaders and innovators; and supporting local communities by delivering high-quality care to all who need it. Through the integration of its hospitals, labs, and schools, Mount Sinai offers comprehensive health care solutions from birth through geriatrics, leveraging innovative approaches such as artificial intelligence and informatics while keeping patients’ medical and emotional needs at the center of all treatment. The Health System includes approximately 7,400 primary and specialty care physicians; 13 joint-venture outpatient surgery centers throughout the five boroughs of New York City, Westchester, Long Island, and Florida; and more than 30 affiliated community health centers. We are consistently ranked by U.S. News & World Report's Best Hospitals, receiving high "Honor Roll" status, and are highly ranked: No. 1 in Geriatrics and top 20 in Cardiology/Heart Surgery, Diabetes/Endocrinology, Gastroenterology/GI Surgery, Neurology/Neurosurgery, Orthopedics, Pulmonology/Lung Surgery, Rehabilitation, and Urology. New York Eye and Ear Infirmary of Mount Sinai is ranked No. 12 in Ophthalmology. U.S. News & World Report’s “Best Children’s Hospitals” ranks Mount Sinai Kravis Children's Hospital among the country’s best in several pediatric specialties. The Icahn School of Medicine at Mount Sinai is ranked No. 14 nationwide in National Institutes of Health funding and in the 99th percentile in research dollars per investigator according to the Association of American Medical Colleges. Newsweek’s “The World’s Best Smart Hospitals” ranks The Mount Sinai Hospital as No. 1 in New York and in the top five globally, and Mount Sinai Morningside in the top 20 globally.

The Mount Sinai Health System is an equal opportunity employer. We comply with applicable Federal civil rights laws and does not discriminate, exclude, or treat people differently on the basis of race, color, national origin, age, religion, disability, sex, sexual orientation, gender identity, or gender expression. We are passionately committed to addressing racism and its effects on our faculty, staff, students, trainees, patients, visitors, and the communities we serve. Our goal is for Mount Sinai to become an anti-racist health care and learning institution that intentionally addresses structural racism.”

EOE Minorities/Women/Disabled/Veterans